Continuous monitoring is a valuable tool and a key mechanism to drive change for CFOs, Heads of Shared Services, Internal Auditors, Risk Managers and CIOs. Anyone who is ready to reap the benefits of CCM needs to be aware that, even with high level support, it’s often challenging to implement.
There are key priorities, success factors, perceived barriers, procurement processes, and decision criteria to be determined before successful implementation.
Confusion about what CCM is
What do we mean by Continuous Controls Monitoring? It’s a relevant question. It’s often very difficult to use technology to test control operation. Often the controls are manual or they are embedded in systems. In either case, it’s difficult to meaningfully test controls using automated techniques.
However, what CCM can do is to test the data itself. CCM tests the data for any indications that the controls are not working. If the key controls are working then the processes should produce certain data outputs. Testing these output for potential errors can give the auditors insights as to whether key controls are achieving their objectives.
It’s useful to regard CCM as a form of End User Computing.
Resistance
Even with Executive or board support, resistance can be encountered at various levels. The IT function may regard CCM as an intrusion into their area of expertise. IT staff may use CCM to justify new, and expensive toolsets that they can use in other areas. Line managers and their staff will tend to resist new and more powerful audit techniques that may expose unknown (or known) problems.
Data
A regular problem is obtaining data. IT Departments have a duty to protect data and are often reluctant to make datasets available on a regular basis. As a last resort, the auditors will have to sometimes point to their Audit Charter to justify access. However, even if access is achieved, regularly accessing and downloading it can be a technical and political challenge. CCM is nothing without access to data.
First Steps
The advice here is to “Keep it simple. The IIA publishes a series of Global Technology Audit Guides (GTAG), one of which concerns Continuous Monitoring. The author of one of these guides privately confessed to an error. His first instinct was to advise auditors to start with a generalised risk assessment and identify CM opportunities on a top down basis. However, after painful experience, he had changed his mind. His revised view was that existing internal audit reports are the best source of potential value add Continuous Monitoring Routines. The auditors will have identified a range of issues over time. Management may have agreed to correct the issue. Developing a CMR to test the data for evidence that the corrections have been effective (and continue to be so) is a useful (and often high return) means of “putting your toe into the water”. This approach will engage the internal auditors who often value the opportunity to establish continuous monitoring over problematic areas identified in previous audits.
Potted solutions
When CCM is suggested, a response is often “We can buy an off the shelf solution that will do all of that”. If that were true, it would be wonderful. However, the reality is often quite different. Every organisation’s systems are different and, even if they use the same core Enterprise Software, there will be differences in configuration as well as interfaces with legacy systems and differing manual systems. Often, potted solutions will run but their outputs will tend to be very large and can’t be efficiently dealt with. As well, the most powerful CMRs often cross match data between systems. A tool is required that acts to efficiently “extract, transform and load” data from differing systems.
The CMR should be able to produce “high probability exceptions” (HPEs). That is, exceptions that are likely to be genuine indictors of control failure. Developing CMRs that reliably generate HPEs often requires insight and frequent iteration.
Managing Exceptions
If CCM is going to be effective, then the exceptions that it generates must be captured and analysed. However, simply making the exceptions available to the operational staff won’t allow the auditors to form an opinion about control effectiveness (i.e. design and continuity of operation). The exceptions must be captured in a secure environment that requires operational staff to act to resolve them (i.e. by closing an exception with a reason code).
It is in this context that the careful design of CMRs becomes critical. Floods of “false positives” will not continue to be reviewed by the operational staff. They’ll “throw up their hands” and walk away from the process.
Investigate the capabilities of Continuous Control Monitoring Satori CCM
