Excel handling 101 million records

Don't let anyone tell you that Excel cannot handle serious management reporting. 

Here's a video from Microsoft's Donald Farmer showing Excel 2010 handling 101 million records via its Gemini add-in.    http://www.youtube.com/watch?v=Zp0AADaW3QY

Excel 2010's new PowerPivot will allow you to summarise those 101 million records in seconds in a pivot table.  Amazing!   Thirty years of OLAP technology and pre-calculated cubes becomes obsolete overnight.

 

 

 

Comments

64 bit

Anonymous | | Permalink

With a 64bit CPU the addressable memory is now 2^64 or 16.8 million terabytes

John Stokdyk's picture

More resources on PowerPivot

John Stokdyk | | Permalink

I've been keeping an eye on Project Gemini (now renamed PowerPivot) since August last year. With the SQL Server 2008 launch, Microsoft has now opened a very useful online resource centre, which among other things features more videos from Donald Farmer. You can also follow the PowerPivot team on Twitter, but they've gone rather quiet since the launch last month.

The nature of this sort of technology is that you need to have all the requisite Microsoft facilities (eg SQL Server 2008, .Net frameworks and usually numerous other chunks of code including, no doubt, Silveright) and then it takes time to get familiar enough with the tools to get them to do what you want.

I certainly hope David and other members of this group can point us towards what is possible. Meanwhile, one of our members, Wojciech Gardziński from AfinNet, was less than complimentary, probably because he's developed arival project. According to AfinNet, "Project Gemini is the next step to make Excel a fat client." The OLAP machinery hidden behind the Excel PowerPivot worksheet is slower than normal usage of Microsoft ActiveX Data Objects (ADO) and the company has produced a webcast showing the difference between refreshing a pivot table based on a database file with more than 3m records. Where the AFIN.NET ADO tools took 32secs, Excel 2010 & PowerPivot took 110secs.

Looking forward to more reports from the reporting front lines!

 

PowerPivot is standalone (I think)

David Carter | | Permalink

 

John, as I understand it PowerPivot embodies its own super-fast "in-memory analytics" engine and you don't need to buy anything else for it work.

It's a free add-in with Excel 2010, so you just enable it, import your file of 100 million records and off you go!

David

 

AfinNet's picture

What about the architecture?

AfinNet | | Permalink

I've found your post right now. Thanks for a comment.

Why nobody discuss here the architecture of Excel in its environment?

I'm trying to do so, but IT experts totally don't understand experts needs. Particularily - financial experts needs.

The topic for discussion:

http://translate.google.pl/translate?hl=pl&sl=pl&tl=en&u=http%3A%2F%2Faf...

Greetings

Wojciech Gardziński, AFIN

Add comment
Log in or register to post comments
Group: Excel reporting with David Carter
Management reporting with David Carter