Audit

Compared to humans, new technologies like Artificial Intelligence (AI) are ideal for automating mundane tasks related to auditing. AI can also verify every transaction and codify human knowledge, enabling auditors to review, analyze and audit more effectively. As big firms adopt AI in their auditing processes, smaller firms are also realizing the value and becoming frontrunners for this positive change in the industry.

Garbelman Winslow CPAs (“Garbelman Winslow”) is a full-service public accounting firm servicing businesses and non-profit organizations for over seventy years throughout the Washington, D.C. metropolitan area, has adopted the MindBridge Ai Auditor™. As part of our Customer Spotlight series, I had the chance to catch up with Samantha Bowling, CPA, CGMA, Partner at Garbelman Winslow. Samantha has been with the firm for over 25 years and has been involved with auditing services since her career started. She has utilized technology to enhance the audit process from paper to paperless and is now implementing AI to assist in risk assessment, audit assurance, and effectiveness.

Let’s meet Samantha!

John Colthart: Tell us about Garbelman Winslow.

Samantha:  We are a small public accounting firm located in Prince Georges County, Maryland.  Our firm has been in existence for seventy years. We provide various services to our clients: Audit, Review, Compilation, Tax, Bookkeeping, Payroll, CFO, Information Technology, Financial Planning and Consulting Services. Our clients include non-profit organizations, construction contractors, manufacturers, professional services, wholesalers, and distributors.  We have utilized technology since our formation starting with a mainframe computer and punch cards to desktops and tablets.  We provide services to other public accounting firms who do not invest in technology so that they can meet their customers’ needs.  We are involved in the accounting profession so that we can make it better for everyone.  We are not afraid of change and embrace technology to enhance efficiency for our office and for our clients.

John Colthart: What do you see as your biggest opportunity?

Samantha: The biggest opportunity I see is to be a part of the development of AI technology to make sure it really does enhance audit efficiency, reduce risk and produce a better audit.  I believe our feedback during this implementation will help make AI relevant and demonstrate to other small firms that AI is available to them.

John Colthart: What do you see as the biggest threat or challenge?

Samantha: The threat is not embracing AI. I feel other small firms will not embrace the technology and they will no longer perform audits.  Auditing services will move to larger firms. In today’s rapidly changing technology environment more and more “CPA” functions are being automated. If CPA’s at small firms stop performing audits, then they will have to redefine their services and relevance.

John Colthart: How do you plan to address it?

Samantha: Implementing MindBridge Ai Auditor™, providing feedback and making sure it works the way we need it to be the first step. Also, exploring additional uses; I believe this is a valuable management tool for outsourced CFO services as well as an oversight tool for all businesses looking to identify risk within their financial transactions. Ultimately using this platform to demonstrate the importance of AI supported services and explaining how it works to all Maryland CPAs.

John Colthart: What made you choose MindBridge Ai Auditor™? What are the features that you plan to use?

Samantha: I was looking for an affordable AI platform.  I used my “Google” machine and this was the only application that I thought would work for my small firm.  We plan to use all features in our implementation phase to assess which may be valuable to each of our clients, not just our audit clients.

To see an announcement of our relationship with Garbelman Winslow, please click here.

With the audit season fast upon us, a person stops to think about ‘reasonable assurance’.

Audit standards such as AU-C 240, CAS 240 and ISA 240 stipulate that “…an auditor has a responsibility to plan and perform the audit to obtain reasonable assurance about whether the financial statements are free of material misstatement, whether caused by error or fraud.”

‘Reasonable assurance’ is a term used in industry standards and the audit profession to render an opinion as to whether financial statements are free of error, or in some cases, fraud.

An article published by The Institute of Chartered Accountants in England and Wales (ICAEW) defines ‘reasonable assurance‘ as “…the level of confidence that the financial statements are not materially misstated that an auditor, exercising professional skill and care, is expected to attain from an audit.”

To render a reasonable level of confidence, auditors rely on many factors including the following three key items:

1. Professional skepticism
   Professional skepticism requires a questioning mind and being alert to conditions which may indicate possible misstatement.
   The Public Company Accounting Oversight Board (PCAOB) standards define ‘professional skepticism’ as “…an attitude that includes a questioning mind and a critical assessment of audit evidence, and it is essential to the performance of effective audits under Board standards.”
2. Sample selection of financial transactions
   Given the volume of financial transactions, a sampling methodology is used to analyze the datasets. As it would be labor-prohibitive and not cost efficient to review all transactions manually, a sample selection of the dataset is identified.
   There are two general approaches to ‘audit sampling’: non-statistical and statistical. Both approaches require that the auditor use professional judgment in planning, performing, and evaluating a sample and in relating the evidential matter produced by the sample to other evidential matter when forming a conclusion about the related account balance or class of transactions.
3. Testing of the selected sample of financial transactions
   Traditional computer-assisted audit techniques (CAATs) that are used to perform tests on financial transactions are typically based upon rules. Rule-based testing requires knowledge of the rule to detect and, in the case of traditional software, technical skills to script the rules. Rules-based systems can’t catch unanticipated scenarios—and, in many cases, the rules themselves can be exploited—making this type of detection increasingly inefficient, not to mention labor-prohibitive to review all transactions manually as mentioned previously.

Each one of the above components is subject to human bias.

As evidenced through countless high-profile cases, and some not so high-profile cases, the fact that clean audit reports were given to many failed or failing companies supports the argument that the current method of asserting ‘reasonable assurance’ must be improved.

In fact, just last week, the U.S. Securities and Exchange Commission (SEC) unanimously approved the PCAOB’s new auditor’s reporting standard, supporting the communication of “critical audit matters” as a way for auditors to provide more information to investors and the public.

As defined in the PCAOB standard and related amendments, critical audit matters are any matter arising from the current period’s audit of the financial statements that was communicated or required to be communicated to the audit committee, and that:

• Relates to accounts or disclosures that are material to the financial statements, and
• Involved especially challenging, subjective, or complex auditor judgment.

Further compounding the problem over the last decade are the growth and adoption of:

• The Internet
• Hackers
• ERP systems with multiple touch points
• Big Data

The combination of these issues further hampers the ability to assert high-levels of confidence that rely solely on human, professional experience.

With the advancements of technologies such as artificial intelligence (AI) that are now available in the market, professionals, including auditors, must augment their capabilities with the power of AI to sift through and analyze large amounts of data.

At MindBridge™ Ai, we remove subjective, human bias from the audit process and perform a full coverage analysis using a hybrid of tests against the complete financial transaction dataset.

The automated analysis of the financial datasets is performed by the MindBridge Ai Auditor™, which combines a hybrid of algorithm-based business rules, statistical models, machine learning and artificial intelligence.

Through the analysis of 100% of the records in the datasets, a ‘risk score’ is generated for each transaction to highlight those that warrant further investigation.

The results are presented to the user in an intuitive, visual interface which augments the capabilities of audit and investigative professionals by allowing them to focus their analysis on the most relevant activities.

By performing an analysis and associating a ‘risk score’ on all financial transactions, coupled with MindBridge Ai’s intelligent statistical sampling capabilities, MindBridge Ai has elevated the definition of ‘reasonable assurance’ to help auditors ensure financial data is free of errors, and potentially intentional misstatement.

Here’s what customers are saying about the MindBridge Ai Auditor.

“The MindBridge Ai Auditor™ enables our team to pinpoint unusual transactions and enhances the thoroughness and precision of our analytics. Investigations that have historically involved months of combing through millions of transaction samples are now much more comprehensive—yet more narrowly focused—and can be concentrated in just hours.” (full text here)

“Rather than our trainees using random number sampling or a sample based on what they deem to be risky transactions, MindBridge ranks all of your transactions on a risk rating, and then depending on whatever the sample size is you pick it gives you, say, the top ten riskiest transactions.” (full text here)

“The machine learning capabilities of MindBridge Ai Auditor are unique in the audit field and go well beyond the capabilities of any data analytics technology we’ve seen for the full audit profession” (full text here)

“MindBridge’s tool demonstrated it can usefully detect anomalies in datasets. Its user interface is intuitive and presented the data visually, allowing the user to explore a time series of each variable, whilst comparing the result to the industry average.” (full text here)

If you’re interested to learn more about MindBridge™ and our Ai Auditor™ platform or would like to run your own A/B test, please visit www.mindbridge.ai.

What better way to start the first post in the “CPAs” series than with a world-class lady. Angelique Koopman, Partner Audit Innovation at Baker Tilly Berk, has been following MindBridge since our early days and recently visited our office. We could not resist asking her a lot of questions.

In this series, we will interview various CPAs in the profession as we travel across the globe, asking them for their opinions and personal stories. If you ever would like to share your own experience as a CPA, please do not hesitate to reach out directly.

Solon:  Where do you think analytics and artificial intelligence (AI) would be the most useful in the audit process today?

Angelique:  Planning and risk assessment are often based on professional judgment. And what I would like to see is the use of more data analytics in the planning phase of the audit so we can substantiate our considerations and our risk assessments with facts (quantitative data). This way we are collecting evidence to support our risk assessments early on.

Solon:  So how should we address that deficiency, and what impact do you think it has during the audit process? What surprised me recently is that there are firms that do it in the review phase, then they miss something, and they have to go back. If you do integrate next-generation AI systems in the planning phase, do you think it would help to decrease this need to re-do portions of the audit?

Angelique:  Yes, when you are doing your risk assessment or your planning, it’s better to have a set of preliminary analytics based on facts. That way, you can substantiate the choices you make which can help you decide to perform all the procedures or just a subset of procedures. Right now, we do risk assessments to limit the audit work. If you do not plan your audit well enough, at the end of an audit, you will find that you missed something somewhere and then you have to go back and do additional work, and I believe this is one of the best arguments for using analytics more in the planning phase.

Solon:  At the end of the day, is it just about analytics, or are there other industry best practices we can also be including?

Angelique:  It is the combination. It’s a combination of automating rigorous procedures and it’s about automating specific aspects related to the industry. I strongly believe in an industry driven approach. It is more of the rule-based part using the industry expertise. But I also believe that analytics must be used in combination with the auditor with all his knowledge and the holistic view of the client. Giving the auditor the technology to do better, to improve professional judgment, to improve decisions based on intuition or feeling and to substantiate those decisions more comprehensively. That is really important. To be honest, I don’t believe in just having a tool or platform only, because, in the end, an auditor needs to evaluate the outcome. But what you can do is help the auditor from performing tedious tasks that could otherwise be automated. Such tools provide the auditor with more insights and knowledge such as making cross-references and analyzing data from more than one dimension. What we normally do with audit analytics, is that we have a question, then we build the query and the query gives an answer that’s different, so I believe in the combination of both analytics and best practices in combination, that it will help auditors become more focused on delivering better results.

Solon:  There is a lot of confusion over the role AI plays in the audit. What would you tell a traditional audit partner who is evaluating if he/she should leverage machine learning and artificial intelligence (AI)? 

Angelique:  Mostly when people hear about AI, they believe that it is science fiction. And they might think it is in the preliminary stage and are not aware of the fact that nowadays lots of things in our society are run by algorithms. We use them everywhere. Unfortunately, we are not using it in the audit on a large scale yet. Maybe, we think it is scary. However, when we get into a plane, we fully trust the airplane, but in the audit world, we do not trust algorithms yet. Basically, it’s about combining statistical methods with machine learning. So I think they need to know what machine learning is and how it works. We have to explain it with simple examples, like how does a program operate and learn? As an example how to identify a cat from a picture? In fact, AI is about automating tasks that are beyond the capacity of human beings. Machine learning is a way of making complex analysis possible. Automating repetitive audit tasks makes even greater business sense when you combine it with machine learning and AI algorithms.

Solon:  People talk about the robot taking the job of an accountant… 

Angelique:  It’s very similar to the team that supports you when you are auditing a bank, and the bank has financial instruments, and you have team members who are specialized in that type of audit; they have the know-how of the financial instruments. Or when you are auditing a real estate company, you use the expertise of people who know that space, you cannot do it by yourself. It’s only expertise enshrined in a computer program. That’s the difference.

I was telling Robin and Eli, knowing that this technology is here, and at this point, it would be hard to go back to audit and do the role I used to do without leveraging this technology. I can clearly see the value it adds, I can do a manual sample, but recognizing that with AI I can sample the whole ledger with a higher degree of assurance. And given that it is my signature on that audit statements, I want to do the best job with the best tools available.

Solon:  So using artificial intelligence (AI) is going to become a matter of professional integrity?

Angelique:  I think, this is something in the audit community which is starting now. And I gave the same message at a recent conference in Belgium. People start to think about it, and they are starting to say, “Yes, why not? Change is coming!”

Originally posted on Solon Angel’s Linkedin post

Auditors love statistical sampling and so does the MindBridge Ai team. Why wouldn’t we—statistical sampling uses the laws of probability to measure sampling risk. We truly believe that statistical sampling largely outperforms judgment sampling.

Don’t get us wrong—the experience of a Partner is undisputed and their ability to focus the audit and identify areas of risk in financial statements is the key to a successful audit and, of course, the peace of mind once an audit opinion is issued. However, once the risk areas are identified and a population of thousands (if not millions) of transactions remains to be reviewed, experience alone might not be sufficient to select a test set.

Statistical sampling and the laws of probability make it feasible to pull a test set from a huge population so an audit team can use it to give reasonable assurance that the overall population is free from material misstatements.

Can audit teams give the same reasonable assurance and test fewer transactions?

There are many commonly used methods to enhance statistical sampling in the audit profession. For example, Monetary Unit Sampling is one (very, very popular) way that a test set can enrich the items most likely to be of interest to auditors. In Monetary Unit Sampling, every dollar is regarded as a distinct unit and given equal chance of being in a transaction picked for testing. Therefore, the higher the amount of a transaction is, the more likely it will be part of the test set. In an audit context, this is great as larger transactions, the ones that are more likely to cause a material misstatement, are more likely to be picked for testing.

How did MindBridge come up with an improved sampling methodology?

First, let’s clarify what improved means. For us improved means to enrich data in a meaningful way so it makes the whole testing process more efficient. Similar to Monetary Unit Sampling, we enrich a data set with additional information to make it more likely that transactions get sampled that are of audit interest.

The intelligent sampler

We run a number of tests to calculate a risk score for every single transaction in a data set. This risk score is the basis of our intelligent sampler which pulls a stratified sample set across the whole population. As with Monetary Unit Sampling, every transaction has a chance for being selected for field testing; however, the ones more likely to be of audit interest having a higher chance.

What does “of audit interest” mean?

“Of audit interest” means the likelihood that a transaction is misstated due to fraud or error. In a number of field experiments, the Ai Auditor™ has proven its ability to identify and push fraud- and error-prone transactions to the top of our risk ranking.

This ability of our Ai Auditor™ to identify transactions which are more likely to be fraudulent gives audit teams the ability to reach the desired confidence level with a smaller test set of transactions. MindBridge’s intelligent sampler is a focusing tool that helps the auditor identify audit areas that they should spend the most time on, but at the same time makes sure they also spend enough time in other areas beyond the high-risk transactions.

Comparing statistical random sampling with intelligent sampling

Our data science team created a general journal for educational purposes. This journal consists of 2966 transactions of which 24 transactions are non-compliant. The non-compliance rate in this file is 0.8%.

Assuming an acceptable error rate of 1%, it would be required to test a sample set of 230 transactions to achieve a 90% confidence utilizing statistical random sampling methodology.

Our intelligent sampler takes an equal number of samples from the Low and Medium/High-risk categories. This ensures that every transaction has a chance of being selected in the sample, but those with higher risk have a higher chance. Using this method the auditor has a 90% chance of discovering non-compliance in the file with just testing a sample set of 14 transactions.

Approach	Confidence level / error rate	Required Sample Size
Statistical Random Sampling	90%/1%	230
Intelligent Sampling	90%/1%	14

 

If you’re interested to learn more about MindBridge™ and our intelligent sampler, or would like to run your own A/B test,  please get in touch.

Throughout my investing and business life, I’ve been amazed by the number of accounting scandals and re-statements that have occurred. The issue of regulatory oversight is raised in the press, but even with increased enforcement and higher penalties, a new accounting issue is never far behind, and it’s continued to get worse. Just look at the impact on British Telecom’s stock because of the irregularities found in accounting in their Italian division (read about it here).

One week into joining MindBridge Ai, I can see how we are focused on the goal of solving one of the massive issues underlying these scandals, and I can see how we are building an expert system to help both internal and external auditors provide us with higher levels of assurance about the financial statements they are sworn to certify.

And the baseline issue we are dealing with is the amount of financial data they need to consume. There has always been too much of it. The Big Data problem facing auditors is that they are asked to “go fishing in a murky lake” and find accounting anomalies and irregularities for further review. To this point, accountants and regulators are agreed that this should be done using statistical account sampling. However, the quality of the sample can only be measured after it has been sampled and analyzed. And with the time pressure on providing audited results, if an irregularity or anomaly is found, the process has to start again. And this leads to costly re-statements which impact shareholder value and confidence in the audit firms.

To solve this, Mindbridge Ai is leveraging machine learning AI to provide auditors with a more highly justifiable sample before the audit begins. Our Auditor AI service can ingest and analyze the entire leger and pinpoint areas they should be investigating early on with a higher degree of justification against the standard, and currently accepted sampling techniques. This gives auditors a higher assurance rate and helps them improve upon the current approaches that only meet the standard, and not move past it and towards the answers we, the press and firms like British Telecom need.