GPT-4 rolls artificial intelligence tanks onto accounting’s lawnby
Whether it’s preparing tax returns or passing accountancy exams, ChatGPT has taken another step towards the accounting profession over the past week. But when it comes to the work accountants currently do, where are its limits and are there opportunities for firms or vendors looking to ride the AI wave?
There are many ways to demonstrate the virtues of a groundbreaking new product. You can show people what it does, tell everyone how great it is, or just smash it with a baseball bat. However, outside the somewhat niche world of tax software, few demonstrators of a cutting-edge product have chosen to exhibit their wares by doing someone’s taxes.
But that’s what artificial intelligence (AI) research lab OpenAI chose as their Steve Jobs “one more thing” moment when demonstrating GPT-4, the latest version of the AI system that powers ChatGPT.
About 19 minutes into an online demonstration to showcase the new tool, OpenAI president Greg Brockman showed an audience of more than one million people that GPT-4 could ingest the entire US tax code and then calculate the correct tax liability for a fictional couple’s return.
Despite telling the model via an initial prompt that it was “TaxGPT”, Brockman was careful to qualify that GPT is “not a qualified tax professional” and the functionality was there to “help you empower yourself”, “solve problems” and “get a handle” on things.
At least one commentator pointed out that the answer it gave was technically incorrect. But does that matter when news outlets across the globe screamed “GPT-4 can do your taxes!” into the faces of accounting firms’ clients? How many clients have already pointed out the demo to their accountants? For the profession, has the damage been done?
Opportunities and ramifications
Stuart Cobbe, chartered accountant and principal consultant at The Analytical Accountant, the technology has the potential to rewrite the way a lot of basic work is being done in the profession.
“For tasks like responding to emails, audit working papers, basic content creation or content processing, there are plenty of opportunities for efficiencies coming soon, but there are ramifications for this. It’s scary for knowledge industries, not just accountancy, as there’s a real risk that knowledge will be hoovered up by these models, and value that was previously generated by study, brainpower and professional organisation just goes straight to the large tech players.”
Cobbe recently ran the new GPT-4 model through a sample ACA Assurance paper, where it made a significant improvement in performance compared with its predecessor – up from 42% to a pass mark of 78%.
Generative email replies
Jason Staats, a firm owner and CPA based in Oregon, is at the forefront of thinking about how AI tools could be used in accounting firms, and believes the new tools can add value across a whole range of accounting functions, from fully automating bookkeeping to tax research tools.
One example Staats uses relates to generating email replies – enhancing current autocomplete suggestions with the artificial general intelligence (AGI) embedded in the new solutions – significantly upgrading practice management solutions in the process.
“When your email can see projects, client relationships, files and statuses, you roll up your voice and your entire firm’s communication history with that client, plus the current state of all their work into suggestions that are actually helpful,” he said.
For Staats, examples of auto-generated responses created by the potential new tools could include:
- Client: “Can you send me my 2020 federal return?”
- Chatbot: “Hey Mike, here’s your 2020 return attached. See you next Thursday.”
- Client: “Are the financials ready yet?”
- Chatbot: “Hey Steve, looks like we’re waiting on a copy of bank statement X. If you can get that over in the next couple of days, we’ll do our best to meet the engagement target date of the 20th.”
“This will make email a non-negotiable component of cloud practice management systems and place a much greater emphasis on the built-in email experience, which right now is abysmal,” added Staats.
Generative advisory prompts
Another way AI tools can add value in the accounting world is by riding shotgun alongside an adviser, providing them with prompts in real-time based on the ability to absorb and process all client data.
In another example provided by Staats, an accountant could ask AdvisoryGPT to generate a list of 30 insights about a set of financials and the adviser selects the four they believe are most correct. Once informed about the adviser’s choice, the tool then drafts a cover letter to send with the financials.
“There have been versions of this in the past that just connect to an accounting file and vomit insights but in my experience, they aren’t usable within the context of the lifecycle of an accounting engagement,” said Staats. “A great implementation of this requires some engineering chops. The reality is a lot of the companies in our space building reporting solutions are small teams working on small budgets. This is an opportunity for engineering to win the day in a saturated reporting ecosystem.”
While the OpenAI demonstration specifically targeted tax, it’s likely that very soon we’ll see an AI tool that does more than answer queries about personal tax returns.
“It’s not a matter of if we get a killer AI optimised for tax, it’s a question of when,” said Staats. “If you’ve ever used tax research tools you know they don’t answer your question, they just point you to potential resources to help you figure it out yourself.”
According to Staats, the opportunity for AI tools in the tax sphere is to go beyond this text-based search functionality and handle more complex questions.
“When you combine AGI with search, you can get an actual answer according to each search result,” he said. “For example, you pose a tax question with conflicting answers according to different source materials. The search results aren’t a list of sources with matching words highlighted. It’s a list of sources, with an answer according to each source. It’s a fundamentally better approach to research.
“The current technical challenge here is chopping up source material into multiple embeddings, then scoring those embeddings in a way that doesn’t drop important context – and getting it all to GPT within its context limits.”
Staats believes an implementation of such a product stands a decent chance of being the first cab off the AGI rank when it comes to being built (and hoovered up by an incumbent tax research vendor within six months).
While it’s difficult to predict how the latest batch of AI technology will react when it meets the cold, hard and complex reality of the accounting profession, the idea that the current iteration of large language models could sweep away the profession as we know it seems simplistic to Cobbe.
“You need more than just the model,” added Cobbe. “It’s a regulated space and no one – firms or software vendors – wants to introduce systematic risk. There will be opportunities for those willing to enhance their offering and pivot to a forward-looking, design-focused experience.”
One likely scenario that could emerge sees specialist providers taking the underlying model of emerging AGI tools and adding the correct fine-tuning to avoid regulatory issues (for example, drawing only from the correct financial reporting standards relating to the query at hand).