HMRC’s API strategy: What it means for agents

Teamwork strategy
Charlie Carne
Chartered Accountant
Charlie Carne & Co
Share this content

Charlie Carne reports on the Agent Talking Points webinar discussing HMRC's API strategy and delivery plans. Although initially sceptical, Carne was pleasantly surprised by the webinar's presenter, Lee Hawksworth, who discussed HMRC’s role in building APIs.

Hawksworth is the head of software development collaboration in HMRC's Digital Service Team. He started by explaining that an API (application programming interface) defines the standards by which two systems communicate and exchange data.

How HMRC uses APIs

The API is effectively a language tool enabling communication between the two systems. In this case, the two systems that we're interested in are those of HMRC and that of third parties whose tax software we agents rely on to upload tax returns or PAYE data under RTI, etc. and via which we, in turn, will be able to access all our clients' data held by HMRC.

Lee said that HMRC has been building such APIs for 17 years. Currently over 75% of the data filed with HMRC is obtained via these APIs and very little of what is filed comes from HMRC's own applications.

The future role of APIs

For this reason, HMRC is keen to encourage and stimulate third party software development. They (rightly, in my opinion) recognise that commercial software houses are much better at developing these tools than HMRC themselves and that HMRC's role should be focused on building as many APIs as possible to enable much better collaboration with taxpayers and their advisers.

His explanation was thorough and he seemed genuinely to recognise the need for HMRC to open up their systems as widely and as soon as possible.

Lee went on to explain how his team have built up close relationships with a number of developers (no names were mentioned, but presumably most of those products with which AccountingWEB members are familiar feature strongly amongst that list). He gave an example of how such collaboration works when designing tools for digitising the 64-8 process.

Two step verification: APIs in action

Our third-party software will call an API from HMRC and start the authentication process via the existing Government Gateway user ID and password. This, in turn, issues an access code for two-factor authentication (2FA - sometimes called two-step verification 2SV). At the moment, this relies on a text message being sent to the agent containing a code to be input as verification.

However, in future, other options will be available, including (but not limited to) an app that is being developed by HMRC. At this point, I tried to ask via the webinar chat screen why they don't simply use existing and very reputable 2FA options, such as Duo Mobile. But, unfortunately, they weren't willing to take questions.

Each time the software requires access to a new piece of information from HMRC, the user has the power to grant or deny that access and, once granted, it can be revoked at any time. I was reassured that we will always remain in control of what data our software can access.

How HMRC relies on the agent community

The key thing that they are working on at the moment is to greatly expand the number of items that can be delivered to our software, as each of those items requires its own, unique API, whether that is a client's date of birth or income from a previous employment, etc.

Lee went on to emphasise how much they rely on the agent community to let them know what services would be valuable and, to that end, encouraged us to let them know what API services they would like to be delivered and what sort of processes we would like to see accessed via an API.

There was recognition that it is a waste of time for agents to provide HMRC with data that they already hold and that HMRC should make it easy to feed that information back to us seamlessly via APIs.


This was one of the better HMRC webinars that I have attended and it was refreshing to be updated by someone who clearly knew their subject. However, it would have helped if the support team for this webinar were equally up to speed, as too many of the questions that they filtered through to Lee to answer had either already been dealt with during his presentation or were too specific to be of general interest.

I'd encourage HMRC to put forward more of their team leaders to update the agent community about its direction (and speed) of travel. This was certainly a good start from which I learned a lot about their strategy.


Please login or register to join the discussion.

07th Sep 2017 10:35

I think it would have been a good idea for HMRC to ensure that the API set to deliver a client's PAYE details to third party software was fully developed and in place before they decided to refuse to give these details to agents by any other method.

Thanks (6)
07th Sep 2017 11:46

I'm not a fan of 2SV as I spend a lot of time trotting the globe and don't use my UK sim due to horrendous charges. Instead I rely on free wifi or buy a cheap holiday sim with a different number. If 2SV comes in for agents it will restrict me to filing returns when I'm in the UK.

Thanks (1)
to Marlinman
07th Sep 2017 12:16

Marlinman wrote:

I'm not a fan of 2SV as I spend a lot of time trotting the globe and don't use my UK sim due to horrendous charges. Instead I rely on free wifi or buy a cheap holiday sim with a different number. If 2SV comes in for agents it will restrict me to filing returns when I'm in the UK.

It worries me that you would want to file returns using free wifi in a foreign country. Surely that's a very sure way to get your client personal details stolen?

As for 2SV and filing, just use commercial software and you won't have the hassle.

Thanks (0)
By chatman
to Tim Vane
07th Sep 2017 20:14

Tim Vane wrote:
It worries me that you would want to file returns using free wifi in a foreign country. Surely that's a very sure way to get your client personal details stolen?

Yeah, those Trinidadian (or whatever country you're in) hackers are always on the lookout for UK tax return data, because it commands such a high price on the international black market.

Thanks (0)
to chatman
09th Sep 2017 18:58

You're far more likely to get personal data stolen in the UK. Its not too difficult and if anyone really wants your personal data, they will get it

Thanks (0)
to Marlinman
07th Sep 2017 12:18

Marlinman wrote:

I'm not a fan of 2SV as I spend a lot of time trotting the globe and don't use my UK sim due to horrendous charges. Instead I rely on free wifi or buy a cheap holiday sim with a different number. If 2SV comes in for agents it will restrict me to filing returns when I'm in the UK.

I mentioned this issue on a post yesterday, sorry to bang on about it but it really isn't practical to assume that a UK mobile phone number is something that everyone will have all the time. In addition to travellers there is the problem of people (and offices) which don't have a reliable mobile signal. If access to a mobile is effectively going to become a government requirement then the government will have to force the mobile service providers to ensure that everyone has a signal, and make arrangements for people who are abroad or, like me, on a boat or other moving object.
Thanks (1)
to C.Y.Nical
07th Sep 2017 13:46

To resolve this problem, HMRC are developing an app that can be installed on your mobile phone (and perhaps on a PC, too) so that you won't need to receive a text message. So long as you can get an internet connection, you'll be fine. And, if you can't get an internet connection, you can't file anyway!

Thanks (1)
to charliecarne
08th Sep 2017 08:49

HMRC have announced that 2SV will be compulsory for BTAs from later this month. It is no good HMRC developing an app: it needs to be ready before 2SV becomes compulsory. This is a bit like HMRC stopping giving PAYE info over the phone before the API is working.

Thanks (0)
to C.Y.Nical
07th Sep 2017 13:47


Thanks (0)
07th Sep 2017 12:43

Through BTC our self-assessment returns are linked to HMRC with a nice button that say's "Get Values from HMRC". It doesn't actually provide a lot of info though. Class 2 NIC's is fine, it won't populate employment income but (oddly) will point out differences in employer names e.g. in my case if I enter TLA Business Services and they hold TLA Business Services Ltd I'll be warned. It doesn't provide state pension income although normally (not always) that's in the agent online services portal. It's a great idea but I suggest they start with the easy bits like employment income.

Thanks (0)
07th Sep 2017 13:32

I've been using the APIs in our software PTP since they were switched on in June. They returning C2NIC and MA from day 1. Employment/benefits data is returned if HMRC has run a PAYE reconciliation for the taxpayer: we have no way of knowing whether a PAYE reconciliation has been done so if no data is returned we don't know if genuinely there is nothing to return or whether it is because the PAYE reconciliation has been done. To date I haven't had any errors in data returned. Why does HMRC insist on running the PAYE reconciliation first? All employers must submit their final FPS by 5 April so surely HMRC should release the API data on 6 April.
The APIs aren't stable: every now and again they'll return meaningless error messages because of problems HMRC's end. HMRC fixes the problems within a few hours.
We need other APIs urgently: CIS is top of my wish list. Others that would be useful include Student Loan and state pension.

Thanks (0)
By Char
07th Sep 2017 15:37

"Lee went on to explain how his team have built up close relationships with a number of developers" - So certain software developers have been given a competitve edge over others? How likely those being invited are those who are now partnering up with the Big4? It very much feels no one is paying attention to the monopolies occuring in our industry (and those related). Only yesterday it was leaked Theresa May is asking for support by the FTSE100 for Brexit and with that comes the simple question of "in exchange for what exactly?". Why are we seeing certain favourable conditions for a few not all? Whilst we allow this to continue, we are ensuring our voices become lost to big business and the corrupt world it brings. The fact HMRC is openly assisting the barriers to entry and unfair competition by partnering up with certain companies shows the lack of fair business and support for the small and medium firms (who are the very backbone of this country) by government. We should be fighting this, not pushing it.

Thanks (1)
to Char
08th Sep 2017 15:26

A valid point. There are already pretty much monopolies of social platforms and search engine advertising and the big accounting software houses either buy out or drive out competition, so choice is limited. However, based on past experience, if the Government departments are left to design and implement systems they will be as much use as chocolate teapots.

Thanks (0)
09th Sep 2017 11:46

Presumably, one concept of the API is to link any Open Database Connectivity (ODBC) compliant database to the HMRC portal to do its stuff. Given that Microsoft Excel is ODBC compliant then why don't we get an API to link to the spreadsheets; job done? Sure the spreadsheet will need to be robust enough to ensure that there is a defined table populated correctly with required summary data ( the three line summary - of which one of them might be a calculation of the others; or the boxes on the VAT return ). Cannot be too difficult can it?

Thanks (0)
to D V Fields
09th Sep 2017 13:11

Software will have to be approved for MTD - HMRC won't give access to the APIs unless the software is approved and if previously approved software is found wanting HMRC can switch off the APIs. HMRC needs to be able to do this to make sure the software is right and bugs are fixed quickly (pity HMRC don't do the same with their software). It's quite easy for HMRC to approve 50 or apps for MTD. But there must be millions of different spreadsheets out there. I've developed numerous ones for clients so I must use about 100 different ones myself. How could HMRC possibly approve them all? If HMRC just gave me access to the APIs HMRC would have no way of knowing whether the structure of the spreadsheet was correct eg whether my VAT box 4 was total inputs instead of just the VAT.

Thanks (0)
to kevinringer
09th Sep 2017 15:33

The key though is, I believe, the interface table. Whether you use a software package written in whatever language, ultimately you want the key data populated somewhere and then to be able to connect. Maybe I am being too simplistic but the accounting software is going to take the data entered and ultimately report, for example, the three line summary into a database table. Now presumably they have spent thousand on this because they didn't have the foresight to build reporting codes into the accounting structure to facilitate this. I am not saying they should have seen MTDfB coming, if indeed it ever does, but reporting codes is schoolboy stuff. Now once you populated this table - hook up with the API to do what ever is needed to transfer the data. If you have populated the table wrongly, then yes it will ultimately be wrong. However that error can be made equally by the software programmer or the spreadsheet designer; I know who I will trust. Equally, post income to an expenditure code, it may appear right!

I see HMRC checking the three line summary contains three numeric fields to prescribed number of decimal places; maybe that the third number is the first less the second, unless we are talking taxable not accounting. I don't see them testing irrelevant "prompts and nudges" even if they could define them or had the ability. Both you and I could put validation on any excel data input cell for such nonsense. I suspect HMRC are effectively approving the final table not how the data got there.

So whether you have 50 apps or millions of spreadsheets, what matters, in my view, and happy to be shown the error of my ways, is that final interface table. Excel is capable of providing that table. If spreadsheets are to be an acceptable recording tool, as we are led to believe, but it requires some intermediary, what exactly is this intermediary going to do, other than generate the same table to provide the ultimate link to the API? Cut out the middleman.

If you can navigate your way through registering your software with HMRC then I would guess configuring the table in the right format would have been the least of your concerns.

Maybe one for the Excel community at some point.

My thoughts anyway.

Thanks (1)
to D V Fields
10th Sep 2017 07:42

My understanding is HMRC want MTD because they are convinced businesses are making accounting errors using their manual/spreadsheet systems: errors that HMRC don't think happen when using bookkeeping software. That is why HMRC want businesses to use software that meets MTD testing. Though you or I could put validation into a spreadsheet, HMRC would have no way of knowing whether it was there. So I can't see how HMRC could accept spreadsheets for MTD.
Of course, HMRC have made a fundamental error: they assume software is correct. We have a client using SageOne which is supposed to be MTD-proof. The software allowed the client to claim VAT on his VAT payment to HMRC! He also claimed VAT on his insurance, council tax etc. So the software is no more robust than spreadsheets anyway. This merely proves that MTD can not achieve what HMRC wants.

Thanks (0)
to kevinringer
10th Sep 2017 10:25

Amongst many things HMRC have to abide by their "Your Charter" which point 1.2 indicates they will not cause unnecessay costs to the taxpayer. Possibly - I say possibly because I am not sure HMRC know of its existence - why spreadsheets are acceptable as a recording tool.

Sadly for now let's leave their misguided reasons for MTDfB to one side.

I do not have inside information so I can only go on reputable sources. Earlier in the year MTD was criticised by one of the Select Committees (one of those groups anyway) for not having provided enough specification as to what was required. The best I have seen since is this will indeed be the three line summary for eligible entities and ( not substantiated but probable ) a mini SA103 form. If correct then at some point in the process a table is required to hold the relevant data. Microsoft Excel is capable of achieving this. If Microsoft Excel meets the requirement of being able to link with the API to transfer data to HMRC then what can be the objection? I am not a great fan of Microsoft Access; not overly convinced by cloud based product but accept they have a vital role to play, but these are not reasons for barring those solutions.

You mention HMRC want software that meets MTD testing. Again without inside knowledge I have not seen this confirmed and most definately have not seen the "testing" specification. I am sure with a detailed specification we could design our spreadsheets accordingly. Firstly the three line summary or SA103. Generate a table and populate from required data sources.
Other tests? The "prompts and nudges?" Let's speculate on what these might be:
"Are you sure that figure is correct?"
"You must include those cash sales you keep leaving off the return."
"Your rent figure is higher than last quarter - are you sure it is correct?"
"Your rent figure is lower than last quarter, we think you have falsified last quarter's return and will prosecute you."
"Your invoice numbers are not consecutive  - are you trying to hide income - we'll catch you?

I don't see a problem building those in.

How will HMRC verify this, you ask? Two possible options: Tick a box to say you comply, or provide a checklist with determined inputs and expected results and then a tick box to say it complies. I can do that. I don't really see HMRC running the checks themselves - even if they had the resources.

If Excel has the capabilities of interfacing with the API's then I am struggling to see a valid reason for it not being a permissible solution. Maybe we've (I) have been making the assumption that it is not; cannot think of any reason why there would be a rush to dispel such confusion!

On your example, some twenty years' ago the software package I used would warn you if you posted to control accounts and would also have default tax codes assigned to customer and supplier accounts; now granted stupidity has no limits, it seems a shame ( or maybe preceding comment explains) if SageOne hasn't found a solution to that problem. Never been a problem with the spreadsheet solutions. Alas.

Thanks (1)