Speed problem solved? | AccountingWEB

Speed problem solved?

Some time ago Adrian Pearson wrote in New Tools are for New Services that "transactions cannot be keyed into the online accounting services...as quickly as in desktop software....The problem is speed, many say."

Now I've just seen a video interview with Duane Jackson where he says (if I understood correctly) that his firm is about to solve this problem.  This would be fantastic.  Bookkeepers will be able to provide an enhanced old service.

Can't wait.  How soon?


DuaneJAckson's picture

Not long now

DuaneJAckson | | Permalink

It's only a couple of weeks away from being released. For anyone else interested, the video is at: http://blog.kashflow.com/2010/09/03/me-me-me/

Sounds like ....

JC | | Permalink

a desktop component with replication - disconnected app
Horses for courses - good approach for high volume, high speed requirement

garyturner's picture

Sounds like....1972

garyturner | | Permalink

Can we take a step back?

We're talking about degrees of pain relief here, either way it's still very painful to sit and manually key hundreds of transactions into an application just so that you can leverage the benefit of the application's purpose. 


challisc's picture

An opportunity?

challisc | | Permalink

@garyturner reminds me that a few years ago BASDA, the Business Application Software Developers Association, recognised the 1972 problem. Isn't it a complete waste of time for one business to print a customer invoice from a computer and for the customer to have to key it back into a computer?

There has been widespread use of technical solutions to get round this problem, but these have typically involved one customer and its suppliers ("one-to-many"). Each system would have its own "schema" defining file structure for the data, so a company supplying various major companies, such as the national supermarkets, would likely have to deal with several different systems. Furthermore, smaller businesses trading with other smaller businesses had no such system.

BASDA's "eBIS-XML" idea was for a "many-to-many" system that would be supported by every accounting package, from entry-level to major corporate systems such as SAP. At the heart was a standard "XML" schema which could also be used for purchase orders and other inter-company transactions.

Many major software houses took part, and a lot of work was put in to develop a working prototype. But I understand (subject to being corrected!) that one major author whose products dominated the small business sector said "it's not possible" and didn't take part. I had concerns about controls around completeness and duplicates, but no more than with paper-based processes - and easily fixed. Whether the author's motives were technical or commercial I don't know. The idea therefore didn't take off.

I can find no mention of eBIS-XML on the BASDA website or anything recent elsewhere.

A lot of the hard work has already been done. Is this now an oppportunity for the cloud vendors to take a lead, to rid us of the stupidity of re-keying computer-generated documents?



daveforbes's picture

ebis xml is alive

daveforbes | | Permalink


I believe it is mainly used by suppliers to public sector.

David Forbes

cverrier's picture


cverrier | | Permalink

The electronic solutions (often used by Supermarkets) was called EDI, and it worked very well because of the rigidly structured nature of the relationship.  Both sides had a long-term agreement for a commodity product, and the system ran on rails most of the time with transaction volumes that a manual system would never have coped with.  I did some work many years ago for a big Potato farm, and the EDI file formats were just huge, with endless total and subtotal sections all over the shop and gigantic stock coding structures.

Outside those realms - I even have people querying the fact that I send invoices via e-mail ("Is that legal?").  It's not the technology that's the problem!

On the original point - Gary's right that you really shouldn't be getting yourself into situations where large volume data-entry still happens.  On the other hand, if you're a rural practice with lots of agricultural clients who still turn up with the traditional shoe-box full of receipts - what do you do?  You can only exhort them to get automated so many times!

SaaS apps are not great for the traditional data-entry tasks - but it's something that early Windows applications also had to deal with when faced with staff who could play their numeric keypad like a piano and didn't want to stop and click stuff with a mouse.

I'll be interested to see what Kashflow come up with.


garyturner's picture

Solve it with engineering

garyturner | | Permalink

In computing evolution terms, manual data entry is only one level removed from punch cards. 

While you'll never completely eliminate the requirement for manual keying of some transactions, for very small businesses surely the bank statement is a strong contender as a primary data source as an after-the-fact form of EDI that every business has effectively free access to if they have a bank account?

For those of us born before Sage was incorporated as a company, it is actually counterintuitive to think of it as such because the foundations for every traditional accounting app around today were laid when paper still played a very dominant part in every business process, and therefore our instinctive expectations are informed by a software design workflow that involves paper, documents and manual entry.

But we've got people literally running their businesses off the back of bank feed data capture now, automatically sucking in their bank transactions daily and then backfilling the coded transactions (automatically in some cases for repeat transactions etc.) directly into their source ledgers, just from a single bank rec process. I think the record number of transactions loaded and coded by a user this way was something like 15,000 lines in a five hour sitting.

challisc's picture

Rely on bank statements?

challisc | | Permalink

Invoices may not get paid for weeks. In any case in many businesses, of all sizes, payment is driven by entry of the supplier's invoice.

mkcdavies's picture

Use OCR to avoid manual processing

mkcdavies | | Permalink

I agree with Gary.  Focusng on speed of data entry fails to deliver the step change that enables accounting practices to transform back office efficiency.  How much better if all invoices are converted into electronic format?  Then the accounting system can understand the content and start to do the work for you.

Only the big boys (government, Tesco, etc.) can mandate that all their suppliers send only e-invoices.  But everyone can take a paper invoice and convert into electronic format without manual keying.  We already have many accounting practices (in Holland, but we're working on it for the UK) who have automated >80% of their invoice processing by using scanning and OCR.  Then the system can then prepare the invoice for approval and payment automatically using the same logic (or "rules") that would previously have been applied manually.

One day we will see most systems exchanging XML-type records and paper based documents will become ancient history.  That may be a way off yet, but the ability to automate is already proven.

garyturner's picture

You'd be surprised

garyturner | | Permalink

It's horses for courses in terms of the nature of business and its size, but you'd be surprised how many small businesses rack up large numbers of direct debit, cash and credit card expenses in a month where the paperwork appears long after the funds have left the bank account. This throws accurate visibility of cashflow a real curveball, so actually driving the business off the bank statement is probably more sensible for smaller businesses too.

daveforbes's picture

mandation of e-invoicing

daveforbes | | Permalink

Denmark mandated electronic invoicing for all interactions with the govt. way back in 2005. I really think UK should follow. It is a shameful waste of money (and paper).

.. but back on topic a common xml or csv format agreed by cloud vendors to get bulk data in and out would be a big step in the right direction.

cverrier's picture

Danish e-invoicing

cverrier | | Permalink

I was curious, so went looking for how the Danish did it.


It's only for invoices to the public sector, but yes, it's a great showcase for the idea.


challisc's picture

Expenses etc

challisc | | Permalink

@garyturner is completely right about volumes of direct debit, cash and credit card expenses in even a small business. Of course this only gets worse the bigger the business. But in my book the bank statement is for carrying our bank recs, picking up on forgotten transactions, not the principal source or prime entry:

(1) Direct debits s/b set up in the system for two extra reasons (a) Correctly processing utility bills that are in a combined invoice/statement format is a nightmare for many people, especially for VAT, and direct debits makes it all much worse (b) Consistent GL/job coding, especially to handle VAT-bearing DDs

(2) Petty cash and owner's/employees' expenses is always a time-consuming nightmare for bookkepers and staff alike. Some form of e-expenses system is the answer, inevitably cloud SAAS based

For practising firms that do the bookkeeping retrospectively for the "brown paper bag" clients, there's the opportunity to bring them into the brave new world using such tools for more regular data entry, cash flow forecasting and regular management accounts - before they are poached by someone who does. 



DuaneJAckson's picture

Certainly no Henry Ford

DuaneJAckson | | Permalink

 I guess you can either be an idealist, tell the potential customer they're doing it the "wrong" way and try to bring the world around to your way of thinking.

Or you can accept that that's what the customer wants and develop something that works how they want it to

Option 1: Redefine their needs on your terms
Option 2: Fulfill their needs on their terms

We're going to option 2, even if it does mean just a faster horse instead of a car.

Paul Scholes's picture

Am watching with interest

Paul Scholes | | Permalink

On the subject of bank v invoice accounting; with financial accounts having to represent events arising, ie the supply of good or services then invoice recording represents the closest approximation to what went on in the month, quarter or year.  It would be a nightmare producing monthly management accounts based around bank entries. 

The lack of documentation, e or paper, to back up online purchases is, in most instances down to laziness, ie the purchaser not pestering the supplier for an invoice as soon as the goods or services are paid for.  I've seen clients throw away hundreds of £s of VAT claims becasue they think the order can be used as an invoice.

Ultimately, I agree it is plain daft for a human to key in data one end, produce an e or paper invoice to send out only to have the same data re-keyed the other end by another human.  I'm not a tecky but if a global or even national electronic standard is still a way off how about a bar or other optical code to record all the standard fields that the purchaser will need eg name, date, invoice number, product description, £s etc?  This would leave only the internal stock and nominal codes to be allocated by the receiver?

With regard to speed of entry, the old days of numeric keypad entry are sorely missed, and so I have issues with both desk & online systems. Many high volume areas, eg expense claims (mentioned above) and credit card transactions are sometimes better entered on spreadsheets and then summarised for entry as an invoice, ideally though it's better to show individual sums and so I'd favour direct csv input from these sheets into the system. 

There is more of an argument for csv (or other direct) input when transferring a client between systems, eg deskbound to online.  The ideal is to move across at a year end, however if we are already several months into the year and want to replicate those months on the new system, unless there is the ability to input transactions via the csv file produced from the old system, it means re-keying data.

This is currently the biggest barrier to swapping my clients over to the online provider I've chosen and so it looks like change over will have to take place at VAT quarter ends, not ideal.  I'd be interested to know firstly whether online systems in general avoid csv input of transactions (or am I just unlucky in mine) and secondly what the technical difficulties are.

Finally, I'd have no problem with a faster horse (I'm vegan and so I'd get someone else to work it!)

garyturner's picture

Importing is a doddle

garyturner | | Permalink

Any decent system online or not should already handle importing either CSV for initial or ongoing data conversion and upload, or automatically ongoing with an API.

garyturner's picture


garyturner | | Permalink

Just noticed my other paragraph about bank data got lost between typing it and hitting Submit in the previous reply....now to remember what I wrote the first time...

Backfilling ledger transactions from the bank statement doesn't mean the transactions are filed with the banking transaction date, the statement data is merely a method of capturing and loading large amounts of provisional (not committed) data into the system without manual keying, the accounting process of reconciling each transaction will determine which date is chosen to record an invoice payment, receipt or other accounting transaction against.

As a side benefit to avoiding the need for manual entry, I'd argue that so tightly connecting actual cash transaction data in any accounting system provides small businesses with better monthly management accounts; management accounts based only upon invoiced transactions = possibility, cash in the bank = reality. Both are important to track. 


Castroggi's picture

Data Transfer

Castroggi | | Permalink

Gary, the difficulty in migrating data from one system to another is not confined to online applications. There are two issues in particular: i) the database in the legacy system may be very different to that in the new system, not just data in different table columns, but even in different tables, so whilst it might be worthwhile to prepare a data map (which says which piece of data from table/ colmn 'a' goes into table/ column 'z' in the new system) for one-or-two popular legacy applications, it would be very expensive and time consuming to do that for every, or even many, legacy applications and their various versions; ii) what data do you need to migrate - is it just financial transaction, or sales order lines, stock movements .... how many permutations can you think of? This is why many will provide import routines for the static data only (customer and supplier records for example) as the permutations are fairly limited, but not for the dynamic data (transactions), hence the cut off at a year or vat quarter end becomes a useful way of organising the migration - records up to the cut off date in the legacy system, thereafter in the new. Hope that's helpful. Regards, Paul

Paul Scholes's picture


Paul Scholes | | Permalink

Hi Gary - thanks for the extra input, as I've learned from reviewing online systems it's worthwhile challenging the "givens" of hundreds of years of bookkeeping (hopefully the same will apply to 66 years of PAYE - but that's another thread).

Sorry if I get the wrong end of the stick but are you saying we should see online banking uploading (dreadful terminology I know) as the first step to recording transactions?  Whilst deskbound, & now online, systems include this as a facility, in my experience people tend to treat it as an add-on rather than prime entry method, but ironically that is probably because they have been indoctrinated with "paper first".

I have to say that with one client (for whom we are covering bookkeeping during mat. leave) the sheer quantity of paper, emails & PDFs evidencing online rail & hotel bookings has lead us to indeed start with the bank statements and work backwards.  Having experienced this over months the bookkeeper had, in fact, moved away from invoice accounting across the board, ie had been running the books on a cashbook basis, leaving a pile of receipts & invoices in paper and on memory stick, to tie up at a later date (ie never), which is where VAT errors creep in.

So I suppose that my first practical problem is that it makes people lazy, ie they are not forced to match payment with paperwork.  This in fact has lead to a number of problems such as bills being paid twice and, with a series of identical payments, not knowing whether the April payment was for the March bill (which was missing) or the April one.

From a management accounts point of view I still envisage a problem with period end creditors & accruals, ie without a log of the expense/supplier invoices received, and with even a small number of supplier accounts, it is more difficult, under a cash based system to identify unpaid amounts and their nominal allocation.

I have re-read the above and suspect I am not seeing the wood for the trees but I too have suffered indoctrination.

garyturner's picture

Reduces manual data entry/errors/delays

garyturner | | Permalink

At a macro level, the bank statement method (we call it bank feed data capture) eliminates much of the time spent by practices collating paperwork from manual records and small business clients and keying their manual data, and keying errors etc.

So, the starting point (daily if required) is the live cash position, and where it is not obvious how to code, match or date a bank statement line, the fact the application is online and one one set of live data means collaborating with the client interactively is an easy way to handle exceptions. Either the client reconciles the majority of his or her own statement lines and then leaves and annotates the complicated ones for the accountant to come in and handle later in the day, or the accountant owns the entire bank rec process and pulls the client online in when queries an exceptions arise.

The client gets live visibility of cash, debtors and their business performance, the accountant is free to actually offer advisory services back to the client, rather than manually compiling historical, basic compliance and basic management accounts 3-4 weeks after month or quarter end.

Most online systems can import bank statements downloaded from online banking apps which is a good start and half the way there, but it's still a manual download and upload process that requires human intervention. Therefore unless you know that the statement download/upload process has been carried out, you cannot be certain that you're looking at the live position. So, with Xero we worked directly with the banks to link the Xero cashbook with their banking feeds online, we've been running this successfully with hundreds of HSBC customers over the last year, and we're rolling out direct live feeds from around 80 UK banks, credit cards and financial institutions by the end of this year. 

I'm reminded of this interesting thread on AccountingWeb about this very subject from 2006...


As an aside while reading that article, it's remarkable how much the landscape has changed in four years, no real mention at all of most of today's online solutions.





Paul Scholes's picture

Many thanks Gary

Paul Scholes | | Permalink

Given me something to think about. 

You're right the pace of change is scary but some things stay constant, eg John Stokdyk's good looks!

Adrian Pearson's picture

Microsoft Tag?

Adrian Pearson | | Permalink

I agree with Gary Turner's theme that the bank statements can be used to automate the data entry for many small businesses (how long before the banks steal this market?) and witness the success of BankLink in Australia.

As for sales and purchase invoices, being the missing link to complete a "proper" set of books in terms of a sales and purchase ledger, maybe if everyone agreed to use smart tags? http://topaccountants.com/2010/03/04/can-we-play-tag-with-invoices/


Paul Scholes's picture


Paul Scholes | | Permalink

Adrian impressive - I may be a bit out of date on this stuff but the possibilities are endless.  Send a set of accounts to a client with a tag which takes them to my Youtube video on "How to understand a set of accounts" (don't look still on my to-do list).

Having said all that however it's intersting to see that all of the discussion was over tags on printed material requiring a physical reader/scanner (or smart phone) whereas 99% of everything leaving this office is on PDF upon which will be a hyperlink to my Youtube video.  As you rightly say, receiving printed bills with a tag containing all the supplier & financial info (per my bar code thought above) would be a great start.

I can't think it will take too long for most suppliers to be sending e-bills so maybe a tag on a PDFs to be read by mouse curser and captured by accounting software?

daveforbes's picture

e invoicing

daveforbes | | Permalink

Sending an email with an attached pdf with embedded information in it would be workable. However it would be preferable to instead to attach an xml invoice rather than a pdf.

There is an example at www.forbes-computer.co.uk/0188.xml

It looks like a normal invoice to a human (I am particularly proud of the date paid stamp). Right click and choose View / Source and you can see what the data the computer sees.

The technology is simple. It just needs a bit of momentum. Perhaps there should be an AccountingWeb e-invoicing discussion group for those keen on pushing this forward.

garyturner's picture

Vendors should collaborate

garyturner | | Permalink

Makes sense for vendors & developers to bang heads together and agree a simple schema and just get on with it.


dahowlett's picture

Done years ago

dahowlett | | Permalink

@dave: that was done many years ago by BASDA: e-Biz-XML. Exchequer and CODA made it work, no-one else wanted to come to the party. @gary makes sense in that regard but try getting vendors to agree on anything is pretty darned difficult unless it affects all of them.  

daveforbes's picture


daveforbes | | Permalink

@dahowlett, re: ebis-xml - I was aware. It is still used, but I think predominantly in the public sector - as per my post earlier in this thread. It does seem a natural place to start. My particular xml invoice example above was from a BASDA "introduction to XML" lecture I gave with Dennis Keeling in 1999. It was intended to outline the concept.

I am not sure if you need to be a BASDA member to get the specs and examples. The person to contact is [email protected]

challisc's picture

Clarification re expenses

challisc | | Permalink

@DuaneJAckson You spoke about "idealist" and make two very good points - understanding what customers want, and providing them something that works how they want it to. Let me clarify:

(1) Bank account statements (not credit cards)

For micro businesses that only want to do book-keeping for VAT returns under "cash accounting", and then use these records as the basis of annual accounts, then importing bank account statements for cheques, DDs etc is indeed a good way. There have many solutions over the years that are focused to do just that. Indeed I have such a micro-business where I pull in csv files from internet banking, and then add the VAT analysis and extra details. A SaaS system that would let me do that, as a two stage analyse-post process, would indeed be attractive.

However if that business was much bigger, and certainly as soon as it's grown above the cash accounting threshold, then I want to import invoices and only use bank statements for bank recs. That was the scenario I had in mind with my previous posting, i.e. for "small" business upwards, rather than "micro" businesses. To avoid having to change systems (and the SaaS provider losing a customer) bank statement import for data entry should only be an option, rather than the principle basis. An option to use it for bank recs is desirable, and indeed what people will want at some stage as their business grows.

(2) Credit card statements

Using electronic credit card statements is a different proposition. These show the transactions dated when they happened, except for a slight delay in foreign transactions, and so do not suffer the timing problems of bank account statements. They also tend to contain far more transactions of lower value than typical bank accounts. A quick and easy way of data entry is vital.

Again a way of pulling in credit card statements into a pending area to allow analysis before posting would be ideal, especially if the credit card holder (or their PA) can access their account to provide that analysis. Add a simple approval system from a manager, and a review by accounts to check VAT analysis etc before posting, and you've got a neat little bookkeeping system. There are issues like getting hold of dockets to support the VAT claims, but these are all manageable with a little help from a good system.

So yes there is definitely a basis for using statements as the prime means of data entry, in some circumstances. But in the case of bank account statements, this needs to an option  which preferably changes so the import can be used for bank recs when required.



daveforbes's picture

Who are you calling micro !

daveforbes | | Permalink

It is £1.6M before you have to move away from cash accounting.

92% of limited companies have a turnover below £1M

99% of limited companies have a turnover below £10M

I have no stats on unincorporated businesses but surmise on average they are smaller than incorporated businesses.

I propose the terms regular (92%), large (next 7%) and supersize (top 1%).

challisc's picture

Definitions of micro, small etc

challisc | | Permalink

Thanks @daveforbes. Indeed there is a massive market for accounting solutions for "micro" businesses, which the EEC has defined since January 2005 as under EUR 2 million. Hence the upper threshold for cash accounting of £1.6m, presumably.

I'd be interested to see what proportion of these limited companies are in  fact individuals effectively forced to form companies to get work in  the IT and other "contractor" industries. That of course is an accounting market in its own right. That leaves X% of micro trading businesses that need limited liability or for tax or some other reason choose to be limited. To these can be added sole traders and smaller partnerships. All in all a big accounting market, hence so many practitioners around the country.

The EEC's business size definitions are used extensively, such as for grants and R&D tax credits. Whilst the definitions also embrace balance sheet value, and can take a multi-year view when businesses are growing or shrinking, the measures based on turnover and headcount ("Annual Work Unit") are:

<2m EUR , <10 AWU : "Micro"

<10m EUR, <50 AWU: "Small"

<50m EUR:, <250 AWU: "Medium"

In computing terms, these thresholds also broadly correlate to stepping stones in the sophistication of the IT systems a business requires, with one exception. I would define the EEC's "medium" as "lower mid-market", and add an "upper mid-market" being up to about 500m EUR. "Corporate" is then above 500m EUR. Having said that, individual IT companies like Microsoft and HP will define the break from "SMBs" (which they tend to call "small businesses") to "corporates" at different points, but always in the hundreds of millions.

The EEC defintions are at


I feel these definitions suit the IT business, as well as being consistent with the definitions used for government puposes such as tax and grants. So no new definitions are needed.




challisc's picture


challisc | | Permalink

Actually another definition could be useful, being the sub-micro businesses with annual revenues less than around £250k, including:

  • The masses of one-man companies in the IT and other sectors working through agencies
  • Single-shop traders
  • Smaller eBay traders
  • Legions of other businesses falling below the £70k VAT registration threshold, who may only produce annual accounts for income or corporation tax
  • Legions of other businesses between £70k and say £250k

Checking the VAT thresholds at http://www.hmrc.gov.uk/vat/forms-rates/rates/rates-thresholds.htm#2 , the upper limit for the Flat Rate Scheme is £225k. That means a business above this level must always account for VAT on an actual basis, either using Cash Accounting  or accruals accounting. So £225k seems an appropriate threshold to sub-divide the EEC's "micro" category.

"Sub-micro" sounds too tiny, as does "nano". "Lifestyle" is too condescending for those working their socks off. What term would you suggest for this category? Is £225k a sensible threshold, that's useful in IT terms?


daveforbes's picture

nano businesses

daveforbes | | Permalink

nano derives from the Greek word for dwarf.

Approximately 80% of uk limited companies have a turnover below £250K.

The 80th percentile for uk adult male height is marginally over 6'




garyturner's picture

Simple UBL

garyturner | | Permalink

 Rod Drury, Xero's CEO (and my boss) has just blogged about driving towards a simpler framework basis for vendors to firstly agree upon together, then adopt in the form of a simplified implementation of the OASIS Universal Business Language standard - Simple UBL.


daveforbes's picture


daveforbes | | Permalink


There are several XML based standards for business data exchange. From my brief research last week in terms of current usage (which is quite high with large organisations) I learned that BASDA's ebis-xml ( www.basda.org/ebis-xml-35485.htm )  is the de facto standard followed some way behind by cxml ( http://www.cxml.org ). The rest is non-xml based EDI. UBL did not feature. I know almost nothing about it - so maybe it is the standard people shoud be using, but currenlty I don't believe they are, not in the UK at least.

guyletts's picture

Setting the record straight

guyletts | | Permalink

Just to clarify - Sage supported the BASDA standard for eBis-xml from the outset.  I'm not sure where the impression came from that Sage opposed it, but it has been in the Sage 50 product for many years and it's still there.  It's called 'Transaction email' as a feature, but the data format is as per the BASDA standard.  I think it's also in TAS, but not sure.

That means several hundred thousand businesses already have the ability to trade with one another without paper.  (We had to clear that with HM Customs & Excise, as was).  I'm not sure why there wasn't better uptake.  Perhaps, for once, the standard was ahead of its time?

Sorry to be late to the thread but it seems only right to set the record straight.

(Disclosure - it was my decision to develop and include the feature, but I'm no longer at Sage)

Add comment
Log in or register to post comments