Share this content

The business intelligence debate: Who has the right skills? By John Stokdyk

20th Jun 2007
Share this content

Last week, consultant editor David Carter published a tirade about business intelligence (BI) tools that produced summary totals, but wouldn't let him drill down to the underlying transactions that made up the figures.

His argument, triggered by a series of frustrating product demonstrations, touched on technology tools such as online analytical processing (OLAP), but took on a political hue as he called on accountants to rise up and take control of business intelligence systems from technologists who had no appreciation for what the numbers meant.

The article was deliberately provocative and spilled out into the wider blogosphere. The result was a stimulating, high-quality debate about purpose and processes of management reporting, and the respective roles of finance and technology people.

The nuts and bolts of BI
As Carter explained it, business intelligence takes the multitude of transactions generated by an organisation, and summarises them into monthly totals that can be delivered on screen to help managers get an immediate view of business performance.

To illustrate how such information is used, Carter likened typical BI reports to a profit and loss account. But accountants who have any doubts about P&Ls are usually able to go into the trial balance and see the account totals, or run a transaction report on each balance to see the individual general ledger transactions that make up the totals.

"If it’s an on-screen report, often I can drill down on a total to display the transactions," he wrote. "But the worlds of accountancy and BI seem to have different views here. In BI they don’t seem to feel that drill down on totals is important."

Management accountant Garry Twiss voiced a related complaint that many "designed IT systems" lacked flexibility. "These systems tend to dictate what you can and can't do, and what you can't do is always in the next version. My advice is go right to the heart of the information - in the database tables themselves.

"Use the IT department with what they excel at, which is systems structures, and understand how the tables are populated. Then use tools such as Microsoft Access and SQL to extract, analyse and summarise the data, present it in say Microsoft Excel, and if you use a pivot table you have a built-in drill down facility.

"Equip accountants with strong IT analysis skills, particularly database technologies and languages, and you will have one seriously skilled employee."

But not everyone agreed with the prosecution's case, and the debate moved on to a discussion about different approaches to management reporting systems.

Paul Crinson said BI had more to offer than Carter's piece suggested. "As with all reporting tools you only get out what you put in. All of the data [David Carter] wanted to see is available and is easily calculated if it is not. [Microsoft SQL Server's] Analysis Services and Integration Services combined can be used to filter out specific data and be made to calculate columns. And you can include as much or as little as you want in your data warehouse."

Robert Brook, himself a developer of management reporting tools, acknowledged that drill down to transactions was rare, though "open" OLAP systems did exist which provide access to the underlying database tables, he explained.

For smaller businesses, having a dedicated OLAP database was not essential. You could construct a management dashboard by querying the data contained in the transactional systems using the structured query language, SQL.

"When a figure does not look right, you can build another SQL with a report to give the transaction data, or investigate using the normal reporting tools," Brook suggested. Anyone wishing to take this approach would need to have suitable SQL skills to construct the queries, he added.

Who's going to fill the finance-IT skills gap?
Carter's comments did ring true for several contributors, including Isolist developer Jim Johnson, who noted an important skills gap identified in the article.

"I’ve long recognised this gap, because I invariably took on the role of filling it in all of the companies I worked for," Johnson said in his Accounting Mechanics blog. He then posed the question, who was best placed to fill the skills gap? An IT person willing to learn financial concepts and take on responsibility for the meaning of data, or an accountant not afraid to grapple with SQL, data normalisation and storage technologies?

In his experience, "IT departments typically manage the storage and connectivity to large volumes of data without being accountable for the values.

"Conversely, accountants working with the same data are usually less concerned with the database but are accountable for the values and meanings contained. This seemingly small difference in the perspective of two sets of professionals looking at the same set of data makes a world of difference in practice."

Too many accountants shy away from technology and fail to realise that only they can provide the meaning," Johnson wrote. "The result is IT systems that fail to deliver what is truly required."

Ultimately, he endorsed the Carter BI manifesto: "At the end of the day, the accountant is accountable for the numbers, and if that means straddling the gap into IT skills to ensure they can be delivered and explained, then that needs to be done. No one else is going to do it."

Stephane-Robert Langer, a techie-turned management accountant, joined in on his Just another Knowledge Worker blog. While agreeing with the Carter-Johnson thesis, he warned accountants seeking to fill the BI skills gap to prepare themselves for several "positioning" issues:

  • Old-school finance people might see you too much like an IT resource
  • IT will most definitely see you as competition
  • The added value is obvious once you’re part of an organisation and start to deliver results, but it’s difficult to "sell" that particular skill set without sounding too much like “an IT guy”
  • You run the risk of ending up maintaining systems instead of using them to answer the very questions you built them to find answers to in the first place.

Having dispensed such useful advice, Langer had to return to his day-to-day workload: "Back to my EOM ETL jobs, SQL-based cost allocation processes and OLAP cubes…"

Microsoft answers back - define the user's needs
Having discussed the potential for open OLAP systems, Robert Brook suggested that David Carter had encountered a "closed" OLAP system, or that the salespeople didn’t understand the full scope of their product. "That’s not unusual in my experience," he wrote.

The demonstration that set Carter off was presented by Microsoft, which he credits with having created a marvellous environment for creating and distributing management information. But the company's technical team gave him figures he couldn't check. There were further problems as the sales reports included additional costs such as delivery charges which undermined their integrity and threw doubt on any margin calculations.

Paul White, head of Microsoft Dynamics in the UK, tackled the issues head on. While accepting in principle the argument that accountants should own the data, White warned against oversimplifying the BI challenge.

"I don't think it's responsible to make such general assertions without defining the market you're talking about. Reporting needs and sales margins are unique to individual organisations," White said.

"While the principles are consistent, the needs of enterprise, mid-size and small businesses are radically different."

For a smaller organisation, it would be feasible to design the margin calculation into the application, so that it was available as a standard sales report. But technical and commercial considerations made it difficult to support drill-down in bigger installations, White explained.

"Some Dynamics customers hold terabytes of data. If you attempted to bolt real-time drill-down capabilities and tools onto an installation of that size, it would be blown away." A 1.9 terrabyte transactional database would nearly triple to 5.9 terrabytes if it became a drill-down OLAP system, he added.

Ultimate responsibility for business intelligence lay not with the software vendors, he argued, but with the managers who commissioned and paid for the systems. "Technically, it might be possible to create the kind of corporate BI system you're looking for, but it's not commercially feasible in terms of the return on investment," he said.

Don't play politics with BI
Neil Conacher, an accountant whose firm successfully uses the BI tools within the APS practice management suite, urged participants to stop playing party politics with BI.

"OLAP is only one element of a reporting strategy, but a very powerful one. [It] should not be criticised for what it doesn’t do, but admired for what it can," he said.

"Microsoft should be congratulated for making this technology available to medium to small organisations at a price which represents extremely good value for money. However, it should be pointed out that the skill set required to deliver an effective OLAP solution is very different from that associated with a traditional transaction reporting system."

This skills needed to be identified along with the software product to ensure an effective solution - much as you would look for the correct skills needed to deliver effective tax or accounting advice, he added.

For any successful reporting solution to be delivered, whether it be OLAP or transactional, teamwork was essential between the users and developers. "What is vital is that both disciplines acknowledge each other’s potential contribution. IT developers are clearly not accountants and vice-versa. The best solution will come from the combination of the two skill sets," Conacher wrote

"It’s not about accountants knowing more than IT developers or the other way round, teamwork brings the best results in today’s business world."

He warned that the original article ran the risk of influencing views in a way that could discourage firms from enjoying the benefits of current technology. "BI is about using a set of tools and solutions to satisfy ongoing reporting requirements which will enable a firm to run efficiently and make not only the IT developers and accountants happy, but ultimately better serve the customers. Surely that is in everyone's interest."

Boost your BI knowledge and skills with AccountingWEB resources


Replies (9)

Please login or register to join the discussion.

By listerramjet
21st Jun 2007 22:43

David, don;t be so sure
it is sometimes appropriate to use a dimensional database structure for the technologies referred to (although not always), but that would not prelude drill down.

Thanks (0)
By MarkRyan
04th Jul 2007 18:35

Let's not get hung-up on BI
I find the definition of BI changes with each vendor.
BI = whatever tools they have to sell at the time
(just like the definition of "lunch" changes with each restaurant you visit)
There's nothing wrong in this... as long as we all understand

There's another approach...
Business decision-makers should make business decisions, not mess about with listings or OLAP cubes

They shouldn't waste time, spotting an "interesting number" and spending the rest of the day drilling in, out and all about, then calling up the scan of that petrol receipt which was erroneously entered into June instead of July

Create the reports (based upon the information you need in order to make the decisions)
Test them to destruction - once (with an occasional "audit")
Run them as often as you need to make a decision

Don't waste your time looking at a 4-page list, trying to find the exceptions
Just report the exceptions, which are based upon your rules

Better still
Get the system to report when it needs to
If you're not getting any reports about motor expenses, everything's operating within your rules - think about something more urgent/important

So BI's good, but it's "lunch" you still need breakfast and dinner

(Have I just agreed with Dennis?)

Mark Ryan
[email protected]

Thanks (0)
By AnonymousUser
25th Jun 2007 11:28

Business Intelligence Survey
Hi, you may remember from the VoIP discussion i posted a free guideline download on that very subject, well this time I can offer a free download on a recent survey we, at the the National Computing Centre, have just released. You can download it here.

This Rapid Survey conducted by NCC looks in more detail at the expectations and experiences of organisations, that are using or planning to use Business Intelligence (BI) applications / technologies.

Hope you find it of interest.

Andrew Thompson
National Computing Centre

Thanks (0)
By David Carter
21st Jun 2007 17:22

disagree with Denis and Alastair
Denis and Alastair are wrong, I think. The data structures required to support BI are completely different from the data structures of existing transaction processing systems. The two cannot be reconciled and trying to run BI straight off the transaction processing system is always going to be a bodge job, I think. (Jim Johnson's last sentence says it all).

The correct approach is to generate a secondary database that is optimised for reporting, and run BI off that..

Please read my interview with Stephen Bow of Topaz who is charge of designing Topaz's own BI system with SQL Server and Analysis Services. Stephen convinced me that a secondary database is the correct approach.

Denis, I'd also make the point that Performance Point, who supply an alternative BI system for Dynamics NAV and whom you've praised on your blog, take the secondary reporting database approach and create a separate data warehouse. Their MD, Nigel Geary, claimed to me that the biggest issue in BI these days was reconciliations - it seems that the practice of providing uncheckable figures is now starting to worry some BI customers.

[sorry, posted this last night and some of it dropped off]

[more apologies: sorry Dennis, that should have been PrecisionPoint, not PerformancePoint]

Thanks (0)
By dahowlett
25th Jun 2007 17:00

Ummmm - 2
With respect Andy - this is a vendor sponsored survey so it's tainted and it doesn't answer the fundamental question except to conclude (as might be expected): Throw more technology at the problem.

Interesting note in the forward though:
"Our experience leads us to understand that it isn’t easy to derive ongoing, tangible benefits from any BI system. It’s a careful balancing act to provide sufficient BI capability, at a controllable total cost of ownership, without losing agility, insight or competitive edge."

So if it doesn't work out then don't blame us, we did tell you. That's a dreadful indictment from a company that punts this stuff.

Thanks (0)
By dahowlett
21st Jun 2007 17:24

In which case David I suggest you check out SeeWhy and LucidEra, both of which are running production systems.

And BTW - in this particular article, BI is very poorly defined. Reporting per se doesn't provide 'intelligence.' That's so 1990s.

Update: David - I've never praised Performance Point on my blog

Thanks (0)
By dahowlett
21st Jun 2007 13:04

Not true
You CAN do real time analysis - even with large systems and no oit doesn't blow systems away. It's a different approach.

So Paul White's assertion may be true for Microsoft customers but it's a sweeping generalisation. I'll write about this as a separate blog post.


Thanks (0)
By listerramjet
21st Jun 2007 13:21

business intelligence
is such a wooly phrase. don't you think?

Vendors tend to use it as a shorthand for big ticket sales, and I suspect see their customers as mug punters. More to the point they tag a product as BI and then assert that BI is their product!

I have to agree with Dennis - of course you can do real time analysis and drill down.

More to the point I would welcome a more detailed analysis of how such things are and could be made relevant to the SME world.

Thanks (0)
By jumpalongjim
21st Jun 2007 14:01

ROI is the Real Challenge
Technology, and finding the skills to exploit it, can be difficult. However the greater challenge, in my opinion, is delivering a return on the BI investment. This challenge has three parts:
1) design reports that are actionable, that can be used to influence positive business changes;
2) educate managers to read and use the reports and overlook their shortcomings.
3) identifying and measuring the value return from the BI investment
BI systems from the bigger players can be very expensive, so it's important to buy wisely to maximse the chance that the ROI will be positive.

On the topic of whether reports can be delivered in real time, in practice it very much depends on the complexity of the reporting involved and the design of the underlying transactional systems. I consider these factors more important than simply the size of the databases involved. Complex reports from databases not designed to support real time reporting often can't be made to work in real time.

Thanks (0)