The Australian Securities and Investments Commission (ASIC) is not ready for machine learning to replace or supplement human judgement.
“It’s certainly clear to us that, at this stage, machine learning cannot replace human judgement or experience,” ASIC Commissioner John Price said, at the 4th RegTech Liaison Meeting last night. “But where we see real potential is that it can perform clearly-defined tasks with greater efficiency, and certainly with greater speed, than a human.”
Earlier this year, ASIC put out a tender for natural language processing (NLP) trials to help to improve their monitoring capabilities of their regulated entities.
Review and evaluation on financial advice trial
Financial advice is the first trial to be completed, and Price has said now ASIC is in the process of review and evaluation. This is particularly significant, as there have been major challenges in this area, as illustrated by the Royal Commission, especially in the area of fees-for-no-service in superannuation accounts.
The financial advice industry is also due for big changes as a result of the dual regulatory model, which will be posed by the ASIC and the FASEA, both of which will be raising and the regulating the new professional standards.
The trials were conducted in the areas of financial advice and statements of financial advice, which have been analysed in a manual fashion previously.
“One particular use of natural language processing…is that instead of requesting one or two files from an organisation, all the files from an organisation are requested,” said Price.
Other trials still on progress
Product Disclosure Statements (PDS’) is another area on trial, involving an automated review of managed fund PDS’ to flag those ASIC believes might have posed higher compliance risks.
PDS documents is another issue that has been raised a number of times throughout the Royal Commission into misconduct in the Banking, Superannuation and Financial Services Sectors.
“The current process that ASIC uses on the current PDS reviews is a manual one, and it is resource-intensive,” explained Price. “So, we wanted to trial natural language processes to see if it would allow for a more efficient review of a higher volume of PDS’.”
Price said that, while a formal evaluation of the review has not yet been completed, the trials did yield what he described as some interesting results for the regulator when it comes to the analysis of language complexity and when looking at what the regulator considers as ‘risky phrases’, from their experience.
The second trial still in progress concerns company announcements for those companies listed in the ASX.
“Our challenge in this area is that ASIC doesn’t have a means of filtering key ASX announcements,” explained Price. “So, while the ASX does share their company announcements with us, those documents are shared in PDF form. What our trial was looking to do was extract information from the PDF format into a more useful format.”
Two trials still in the ‘scoping’ stage
According to Price, while two trials were announced as under consideration, at this point, both remain in what he described as the ‘scoping’ stage.
One of these trials is the promotions trial, intended to look at potential compliance issues across various mediums. The second will look at fundraising documents and prospectuses.
With regards to the latter, Price said the corporate regulator only reviews about 70 per cent of lodged fundraising documents. NLP may help to automate this process so this figure can reach 100 per cent, improving ASIC’s ability to target those prospectuses requiring the most attention.
Education for the regulator
One of the key takeaways for the regulator is that NLP cannot transmute poor-quality data into high-quality data. Thus, they need to source the right kinds of data, and enough quantities of that data.
“A good example of some of these limitations is around statements of advice in the financial advice trials,” said Price.
Price added that NLP would have to contend with ‘noisy text recognition’ because of PDF documents. Tables in documents also present a challenge.
At present, however, ASIC is still at the stage of establishing the skills they need to use machine learning for their regulatory monitoring.