Save content
Have you found this content useful? Use the button above to save it to your profile.
Pixelated and distorted image of a tank
istock_dr_evil_AW

GPT-4 rolls artificial intelligence tanks onto accounting’s lawn

by

Whether it’s preparing tax returns or passing accountancy exams, ChatGPT has taken another step towards the accounting profession over the past week. But when it comes to the work accountants currently do, where are its limits and are there opportunities for firms or vendors looking to ride the AI wave?

22nd Mar 2023
Save content
Have you found this content useful? Use the button above to save it to your profile.

There are many ways to demonstrate the virtues of a groundbreaking new product. You can show people what it does, tell everyone how great it is, or just smash it with a baseball bat. However, outside the somewhat niche world of tax software, few demonstrators of a cutting-edge product have chosen to exhibit their wares by doing someone’s taxes. 

But that’s what artificial intelligence (AI) research lab OpenAI chose as their Steve Jobs “one more thing” moment when demonstrating GPT-4, the latest version of the AI system that powers ChatGPT.

About 19 minutes into an online demonstration to showcase the new tool, OpenAI president Greg Brockman showed an audience of more than one million people that GPT-4 could ingest the entire US tax code and then calculate the correct tax liability for a fictional couple’s return.

Despite telling the model via an initial prompt that it was “TaxGPT”, Brockman was careful to qualify that GPT is “not a qualified tax professional” and the functionality was there to “help you empower yourself”, “solve problems” and “get a handle” on things.

At least one commentator pointed out that the answer it gave was technically incorrect. But does that matter when news outlets across the globe screamed “GPT-4 can do your taxes!” into the faces of accounting firms’ clients? How many clients have already pointed out the demo to their accountants? For the profession, has the damage been done? 

Opportunities and ramifications

Stuart Cobbe, chartered accountant and principal consultant at The Analytical Accountant, the technology has the potential to rewrite the way a lot of basic work is being done in the profession.

“For tasks like responding to emails, audit working papers, basic content creation or content processing, there are plenty of opportunities for efficiencies coming soon, but there are ramifications for this. It’s scary for knowledge industries, not just accountancy, as there’s a real risk that knowledge will be hoovered up by these models, and value that was previously generated by study, brainpower and professional organisation just goes straight to the large tech players.”

Cobbe recently ran the new GPT-4 model through a sample ACA Assurance paper, where it made a significant improvement in performance compared with its predecessor – up from 42% to a pass mark of 78%.

Generative email replies

Jason Staats, a firm owner and CPA based in Oregon, is at the forefront of thinking about how AI tools could be used in accounting firms, and believes the new tools can add value across a whole range of accounting functions, from fully automating bookkeeping to tax research tools. 

One example Staats uses relates to generating email replies – enhancing current autocomplete suggestions with the artificial general intelligence (AGI) embedded in the new solutions –  significantly upgrading practice management solutions in the process.

“When your email can see projects, client relationships, files and statuses, you roll up your voice and your entire firm’s communication history with that client, plus the current state of all their work into suggestions that are actually helpful,” he said.

For Staats, examples of auto-generated responses created by the potential new tools could include: 

  • Client: “Can you send me my 2020 federal return?”
  • Chatbot: “Hey Mike, here’s your 2020 return attached. See you next Thursday.”
  • Client: “Are the financials ready yet?”
  • Chatbot: “Hey Steve, looks like we’re waiting on a copy of bank statement X. If you can get that over in the next couple of days, we’ll do our best to meet the engagement target date of the 20th.”

“This will make email a non-negotiable component of cloud practice management systems and place a much greater emphasis on the built-in email experience, which right now is abysmal,” added Staats.

Generative advisory prompts

Another way AI tools can add value in the accounting world is by riding shotgun alongside an adviser, providing them with prompts in real-time based on the ability to absorb and process all client data.

In another example provided by Staats, an accountant could ask AdvisoryGPT to generate a list of 30 insights about a set of financials and the adviser selects the four they believe are most correct. Once informed about the adviser’s choice, the tool then drafts a cover letter to send with the financials.

“There have been versions of this in the past that just connect to an accounting file and vomit insights but in my experience, they aren’t usable within the context of the lifecycle of an accounting engagement,” said Staats. “A great implementation of this requires some engineering chops. The reality is a lot of the companies in our space building reporting solutions are small teams working on small budgets. This is an opportunity for engineering to win the day in a saturated reporting ecosystem.”

Tax research

While the OpenAI demonstration specifically targeted tax, it’s likely that very soon we’ll see an AI tool that does more than answer queries about personal tax returns.  

“It’s not a matter of if we get a killer AI optimised for tax, it’s a question of when,” said Staats. “If you’ve ever used tax research tools you know they don’t answer your question, they just point you to potential resources to help you figure it out yourself.”

According to Staats, the opportunity for AI tools in the tax sphere is to go beyond this text-based search functionality and handle more complex questions.

“When you combine AGI with search, you can get an actual answer according to each search result,” he said. “For example, you pose a tax question with conflicting answers according to different source materials. The search results aren’t a list of sources with matching words highlighted. It’s a list of sources, with an answer according to each source. It’s a fundamentally better approach to research. 

“The current technical challenge here is chopping up source material into multiple embeddings, then scoring those embeddings in a way that doesn’t drop important context –  and getting it all to GPT within its context limits.”

Staats believes an implementation of such a product stands a decent chance of being the first cab off the AGI rank when it comes to being built (and hoovered up by an incumbent tax research vendor within six months).

Specialist providers

While it’s difficult to predict how the latest batch of AI technology will react when it meets the cold, hard and complex reality of the accounting profession, the idea that the current iteration of large language models could sweep away the profession as we know it seems simplistic to Cobbe.

“You need more than just the model,” added Cobbe. “It’s a regulated space and no one – firms or software vendors – wants to introduce systematic risk. There will be opportunities for those willing to enhance their offering and pivot to a forward-looking, design-focused experience.”

One likely scenario that could emerge sees specialist providers taking the underlying model of emerging AGI tools and adding the correct fine-tuning to avoid regulatory issues (for example, drawing only from the correct financial reporting standards relating to the query at hand).

Replies (16)

Please login or register to join the discussion.

avatar
By hje
22nd Mar 2023 12:00

This sounds electric.

Thanks (2)
Replying to Justin Bryant:
avatar
By Hugo Fair
22nd Mar 2023 22:44

The bit where he says "Market forces won't naturally produce AI products and services that help the poorest ... the opposite is more likely"?

Or the bit where he called on governments to work with industry to "limit the risks" of AI?

Thanks (1)
John Toon
By John Toon
22nd Mar 2023 15:49

Not sure I agree with Staats on email being re-invigorated. Much better to have chat within the products you use regularly. That's just not where email sits (see my many previous comments on chat bots and their uses)

Thanks (2)
John Toon
By John Toon
22nd Mar 2023 15:51

I'm also loving this https://www.chatpdf.com/

Great for analysing documents, particularly those long sets of IFRS accounts where the first 80 pages are waffle...

Thanks (1)
Replying to johnt27:
Tom Herbert
By Tom Herbert
22nd Mar 2023 16:13

John, with these chatbot use cases you are really spoiling us!

Thanks (1)
the sea otter
By memyself-eye
22nd Mar 2023 17:09

Care - that's a T34 (85) a Russian tank. We know how effective they are!

Thanks (4)
Replying to memyself-eye:
Tom Herbert
By Tom Herbert
22nd Mar 2023 20:17

Amazing! Come for the AI accounting, stay for the military hardware :-)

Thanks (2)
Replying to TomHerbert:
the sea otter
By memyself-eye
23rd Mar 2023 16:07

we cater for all tastes here!
I'm a 'Tiger' fan myself....

Thanks (1)
Lone Wolf
By Lone_Wolf
22nd Mar 2023 17:11

I have been using ChatGPT to see it's capabilities. I will take standard questions I get asked and pop them in to ChatGPT to see how the answer compares.

You can see where it is going to go, but it's still pretty limited.

The most recent example - it gave a very convincing answer as to when to recognise fixed asset expenditure and what period you get capital allowances for. Provided legislative and FRS references when asked to back it up.

Unfortunately, it was wrong, and the references it gave didn't include the words it quoted from them.

When you tell it that, it apologises and rewords the answer. And again, it gave references and quotes that didn't match.

I'm sure it will get there with some work. But I think the danger of using it now is, it can give a very convincing and apparently well referenced answer. It's only because I knew the answer that I knew it was wrong. If you were using this as your source, then you'd be up a creek.

Thanks (8)
Replying to Lone_Wolf:
By ireallyshouldknowthisbut
23rd Mar 2023 18:02

its never going to be consistently right other than by luck. Its essentially a word generator. ie its looking what it thinks (from statistical analysis) what the next word might be.

it doesn't look what the word IS. Ie its not 'looking up' something in a database as we would look up some text or reference, but just guessing what is likely.

What it does do, is what is taught in many private schools. Say it with confidence, and hope no-one checks. And if they do, just apologise very politely.

For creative stuff where there is no right or wrong, its quite useful. For technical disciplines, not so much.

Thanks (1)
Replying to ireallyshouldknowthisbut:
Tornado
By Tornado
26th Mar 2023 16:00

"its looking what it thinks (from statistical analysis) what the next word might be"

I it really thinking though?

Thanks (0)
Replying to Lone_Wolf:
avatar
By yorkshirepudding
29th Mar 2023 10:13

I volunteer for an open source content management system project. A number of us have tested this on technical coding challenges. Many answers are wrong but confidently stated. Very dangerous to use unless you know enough to tell when its wrong.

Thanks (0)
avatar
By Hugo Fair
22nd Mar 2023 22:37

"At least one commentator pointed out that the answer it gave was technically incorrect.
But does that matter when news outlets across the globe screamed “GPT-4 can do your taxes!” into the faces of accounting firms’ clients?"

Well it might do (matter that is) if taxpayers realised that they're still responsible for the accuracy of their returns ... so if a Bot gives a wrong answer then the fact that the taxpayer didn't understand any of it won't protect them from penalties and fines!

Thanks (4)
By Duggimon
23rd Mar 2023 09:30

AI tools run off massive datacentre servers with obscure processing of data and, the closer the system is to true AI, the less even the people programming it know about what specifically it does with the data.

This makes it quite clearly a breach of GDPR to give it any personal information about your clients. There is no way for you to assess what an AI tool is doing with your client's data. You can't let the tools view all your correspondence with clients or give it access to privileged tax information, so it's currently very limited in scope.

You would need a tool that runs entirely in house in order to let it access client data, or at least that runs in an environment entirely ring-fenced, and that doesn't exist.

Thanks (3)
By Duggimon
23rd Mar 2023 09:59

I would also add that I've spent a couple of hours playing around with ChatGPT and am pretty confident I could now spot anything longer than two or three sentences that is written by it as it's cadence and structure is completely unchanging.

I'm not some sort of savant, I'm pretty sure anyone who reads a lot could do the same, I'm certainly noticing now how many websites are completely AI written in response to commonly googled questions, there are tons of them out there.

If you start using ChatGPT to send replies to your clients and they know you've done that, which they will in its current iteration, what will that do to your relationship?

Thanks (4)