In recent years, businesses across the globe have been investing heavily in audit analytics to confirm and then enhance the effectiveness of their internal controls and risk management processes. This development is a positive one and has been encouraging to see.
However, we’ve also observed an alarming trend emerge from the big four accounting firms: expensive analytic exercises that deliver little to no value.
So, what is happening – and why?
The illusion of auditing competence
Many businesses engage audit firms to undergo periodic audits using data analytics as a part of their internal or external audit program. However, a common complaint arises when audits conclude with a clean bill of health because an overwhelming number of false positives mask genuine findings and issues that reside below the surface.
These outcomes not only erode the trust between internal auditors and the business but also lead to a perception that audit analytics are more of a box-ticking exercise than a genuine effort to improve internal controls. They also promote a false sense of confidence in the business since, with a lack of evidence to the contrary, they start believing that “we have it all under control”.
Real-world damage to major organisations
The experience of one of our ASX top 100 customers when engaging an Audit firm for audit analytics serves as a stark example of the challenges organisations are facing. The scope of this particular engagement included running analytics across purchase-to-pay, credit cards, and payroll processes. Despite the significant investment, the results were disheartening: there were false positives galore.
The audit firm reported potential issues in the previous 12 months, including tens of thousands of duplicate invoices and an extraordinary number of instances where employee bank account details matched vendor bank account details, raising concerns about a conflict of interest risk.
However, upon closer inspection, the small sample of results provided by the firm were proven to be false positives. No true duplicate invoices were found in the sample.
This can happen because Audit firms don’t understand the way invoices are processed, and the supposed conflict of interest risk is actually expense reimbursement details for employees. The lesson is that “vanilla” tests are not reliable. Expertise is required to design and refine the tests to ensure that only “high probability exceptions” are produced.
The cost of ineffectiveness
These engagements can cost companies hundreds of thousands of dollars to complete. IT teams spend resources gathering the data for analysis, and the business begrudgingly invests time to review the output samples provided – only to find each transaction has a valid explanation and seems legitimate.
There are also opportunity costs in that genuine exceptions are discarded due to the volume of transactions swept up by the poorly designed tests.
The biggest cost, though, is the loss of confidence by corporations and business professionals that testing controls with the use of data analytics is of any value.
How are businesses responding?
Frustrated by the apparent lack of value-add delivered by data analytics, businesses are increasingly turning away from using direct testing of data to gather evidence that policies and procedures are well designed and operating effectively.
Those who have poor experiences become more resistant to future projects initiated by internal audit and assume that key controls are achieving their objectives. Three-way matching, invoice scanning, and automation have become popular alternatives, with companies asserting that they have these processes under control.
Our experience is that while useful, three-way testing does not capture the full range of possible exceptions arising from poor control design or breakdowns. Our evidence is simple: we are finding hundreds of thousands to millions of dollars every year for each of our customers despite these controls being present.
Where to from here?
The example of our ASX100 customer is only one example of a business investing substantial sums without receiving adequate value in return. This outcome is disappointingly common. It is an easy solution for an external provider to apply “vanilla tests” and throw lengthy, meaningless reports back; however, the design of effective data interrogation testing always requires a solid understanding of both the system and the data it produces. Poorly designed testing usually produces both false positives and false negatives, with high costs attached to both.
As the landscape of audit analytics evolves, businesses must reconsider their reliance on traditional ‘point in time’ audit methods and explore alternatives such as data analytics. All the while remembering that the use of ‘out of the toolbox’ data interrogations very rarely deliver value, and the direct testing of data requires strong expertise in the design of such testing.
And lastly, experience has consistently shown that the real value of audit analytics is only achieved when the process is continuous – rather than one-off. That way, the business benefits from:
- Timely detection of problems (e.g. identifying an erroneous invoice before it gets paid).
- An automated process that minimises the consumption of business and IT resources
- Ensuring that anything reported does get actioned – rather than just getting left in an audit report
- A focus on true exceptions that are material to the business
- Continuous improvement through learning from any identified problems, determining the root causes and improving the control environment in response to these.
For more information about audit analytics and improving the value of your audit activities, get in touch with the Satori team.
