Overcoming issues with upgrades

Picking up a topic that had got lost in the wrong thread last week …

Having been bitten painfully in the rear by issues with upgrades to both cloud (SaaS) and on-premise software, I asked:

 “How about giving SaaS users an option to pre-test a new release [using a copy of the user’s own data and configuration], as a form of beta-testing, even if many users wouldn’t bother? Impractical or essential?”

The question arose because SaaS users do not usually have a choice of when or if to be upgraded. So there isn’t the option to wait and see if other users have issues before upgrading. Nor is it possible in advance to establish what changes are needed to user procedures (which the vendors cannot see), unless users have sight of the new software before release.

Two SaaS vendors replied with these useful points:

  • SaaS vendors have total control over the operating environment and visibility of how the software is being used
  • Bugs happen, all of the time
  • As on-going SaaS subscription revenues are at risk, vendors take greater care with new releases, especially replacing or removing functionality
  • SaaS goes on short release cycles, with smaller changes to the code, so these are easier to test
  • If any bugs do slip through, SaaS vendors can make fixes live more quickly
  • Saas vendors follow widely varying processes when putting new functionality or bug fixes live
  • Some allow a small number of users access to the functionality before it is let loose in the full production environment (but this is not usually an option for all users)
  • If you want to minimise risk then you must make sure that you understand the quality of service provided by the software vendor

Their full responses are in the “None of us have SaaS businesses, sorry.” thread.

Any other comments on upgrade issues from a user’s or vendor’s perspective?

Comments

Upgrades

rworboys | | Permalink

As far as the SaaS or cloud is concerned, I consider myself in the "Positive but cautious" camp and see cloud based applications taking over in many cases in the not too distant future.  This thread has flagged issues that concern me- those of forced upgrades where they have not been properly tested and the unprompted, forced, changes in reports available. Where the application is a financial one, I would have thought that many users would require the ability to produce bespoke reports.

However, it seems that where these isses occur, it is a case of "one size fits all". Am I correct in my assessment?

jimwmackenzie's picture

Sandbox environment

jimwmackenzie | | Permalink

One option that some SaaS providers (although admittedly not that I'm aware of in the finance/accounting sphere) use is to provide subscribers with a sandbox environment, where the user can test changes to their work processes within the application, without affecting the live data.

Zendesk, who provide helpdesk software, offer this sandbox to their Plus customers, and it's pretty useful for trying out changes before pushing them live (from a user point of view).  They are experimenting with testing new features with the sandbox too; they are making changes to the forum side of the software, and those customers with sandbox access can play around with it before the changes go live to everyone.

This type of arrangement could conceivably become more widespread, although there are cost implications for the software provider; it may be that they take the Zendesk route and offer it as a benefit for some form of premium membership (e.g. normal price £15 per month, no sandbox, premium price £18 per month, sandbox).  Whilst this does have it's advantages, access to a sandbox doesn't necessarily mean that any objections raised will be acted on; the software provider may be faced with a choice of making 90% of their customers very happy, and 10% unhappy.  Truly agile providers will find a way of mitigating the change for the 10%, but sometimes, that just isn't possible.

carnmores's picture

and..

carnmores | | Permalink
  • SaaS vendors have total control over the operating environment and visibility of how the software is being used
  • WHAT ARE THEY INFERING
  • Bugs happen, all of the time
  • NUF SAID
  • As on-going SaaS subscription revenues are at risk, vendors take greater care with new releases, especially replacing or removing functionality
  • TRY RUNNING THAT PAST SOME DESKTOP VENDORS
  • SaaS goes on short release cycles, with smaller changes to the code, so these are easier to test
  • PROBABLY TRUE
  • If any bugs do slip through, SaaS vendors can make fixes live more quickly
  • LIKE MICROSOFT AND ITS SECURITY UPDATES
  • Saas vendors follow widely varying processes when putting new functionality or bug fixes live
  • SURPRISE SURPRISE
  • Some allow a small number of users access to the functionality before it is let loose in the full production environment (but this is not usually an option for all users)
  • HOW NICE OF THEM TO ALLOW
  • If you want to minimise risk then you must make sure that you understand the quality of service provided by the software vendor
  • MEANINGLESS SPIN
daveforbes's picture

Multiple versions

daveforbes | | Permalink

version27.myonlinesoftware.co.uk

version26.myonlinesoftware.co.uk

If there were sufficient demand for that, I am sure the vendors would oblige.

David

 

 

SimonH's picture

Surely it's about customer service?

SimonH | | Permalink

If the vendor, be it of web-based or local software, has the right attitude to its customers, upgrades will be carefully thought out and based on user feedback and requirements.  Changes have to occur at some point otherwise programmers would be sitting around doing nothing. The software industry as a whole doesn't do itself any favours sometimes but if a vendor has got the right attitude they can find a way to accommodate what different users want. 

EH!...

JC | | Permalink

@carnmores

SaaS vendors have total control over the operating environment and visibility of how the software is being used

No inference at all - advantages:

  • ability to determine issues/bottlenecks & tune the system. Whether this takes the form of hardware or software depends upon the results
  • unlikely to get the QuickBooks operating system saga where each sucessive OS release from M$ endangers existing software and causes users immense issues

As on-going SaaS subscription revenues are at risk, vendors take greater care with new releases, especially replacing or removing functionality

  • once you sell a service rather than boxes one lives or dies on the service provided. Having sold a box you have all your money up front and no great incentive to reach out to the customer, whereas if your are being paid each month by users who have no period lock-in and can walk away at any time you tend to try and provide a better service

If any bugs do slip through, SaaS vendors can make fixes live more quickly

  • on-line software generally only has a single verion running and the application resides in one place. Therefore it is simpler to update a single area than multiple spread out over the globe
  • desktop software requires a cd or automatic download to each individual pc and futhermore the developer may not be certain of the OS being run on the target machines

Saas vendors follow widely varying processes when putting new functionality or bug fixes live

  • of course they do in the same way that every accountant has their own procedure for preparing accounts etc - what is the surprise with this?

Some allow a small number of users access to the functionality before it is let loose in the full production environment (but this is not usually an option for all users)

  • allow in this context is a generic term (it could have been provide access etc) - what is the problem
  • there are different stages of testing ranging from defined user groups through to everyone; whichever route one chooses the users have to be 'allowed' access to the test area otherwise thay cannot do any testing

If you want to minimise risk then you must make sure that you understand the quality of service provided by the software vendor

  • of course you need to understand what is being provided by any supplier and that is why Terms & Conditions, Service Level Agreements (call them what you want) exist
  • would you really take out travel insurance without reading about what was covered?

We seem to be unduly sensitive and looking for problems where none really exist
 

carnmores's picture

unduly sensitive MOI!

carnmores | | Permalink

come off it JC its not me its you lot as shown by your answer

my god some of you lot are very pious

there are good companies and bad companies in every sector

i cant see that SaaS is likely to be very different

some of you seem to be holding yourselves out as the saviours of the accounting world

i dont think so

ddruker's picture

Beta and change management are important at Intacct

ddruker | | Permalink

At Intacct, customer and partner formal beta testing is part of our normal process for major new features.

We recruit customers and partners into our beta programs, and we have the ability to replicate their production environments over into a separate beta test environment so they can try out the new features on their own setup and with their own data.  By environment I mean an entirely different set of servers, so there is complete isolation between beta and production.  We also offer beta testing on demo company data, but we find most of our customers and partners prefer to try out major new functionality against their own setup and data.

At the end of the beta,  depending on what we were testing we can often offer the ability to migrate what was being worked on in the beta environment back over to production, so work is not lost.

Our experience is that customers absolutely do "bother" - they know that by participating in this testing they get to work directly with product management and to provide feedback into how the features come out.  Plus they get experience ahead of time with the impact of new features.

There is another premise in the original question - that SaaS users do not have a choice about whether to be upgraded.  This isn't correct - we almost always give customers the choice of wheher they want to use something new or not - the new features become configuration options - eg do you want to do it the old way or the new way.  This isn't a beta question, it's a change management question.  We also offer sandboxes, where a client can copy their production enviroment and try out new production features - exactly to address the point of your question.

So I think what you are asking is really a SaaS maturity question - mature vendors like Intacct and Salesforce have quite robust processes for processes like beta and change management.  The SaaS model clearly does not obviate the need for this stuff.

 

Clearly any response is a waste of time .....

JC | | Permalink

@carnmores

Based upon your previous posting about on-line I really should have known better that to have attempted to provide a reasoned response

Clearly it is a waste of time preparing any such reply because you have absoulutely no intention of taking any notice of it and instead resort to trading random insults

Nobody is disputing that fact of good/bad companies or that on-line maybe unsuitable in certain circumstances but realistically the content in your post was 'sound bites' and did not even attempt a reasoned argument

Yes it is well worth trying to convince the profession about the benefits of on-line but ultimately one has to agreed to disagree and move on.

Quite frankly businesses themselves are the real target market. At the end of the day you are going to be driven by your customers and if they are comfortable using a product then you either 'use it or lose them'

challisc's picture

Impractical or essential?

challisc | | Permalink

Getting back to the question ...

Thanks @ddruker . I’ve been asking for these types of facilities in the UK ecommerce market for 5-10 years. It’s great to see a vendor that listens to its customers, understands their concerns, and provides a solution.

There is clearly a cost involved. That’s why I asked “Impractical or essential?”

But given the risk to a provider’s business, even at the budget end of the SaaS market, is this type of approach imperative?

Bordering on impractical

Anonymous | | Permalink

@ddruker

If I understand you posting correctly about upgrading you make the point that is optional - in which case how do handle the following

  • presumably each customer has a separate db so that you can maintain it's isolation even if the database structure changes in an upgrade - as opposed to a single db for all identified by CompanyID?
  • with say 5,000-30,000 customers the overheads of backing up each db is considerable (sql server 2000-2010 etc); are there enough hours in the day and what about restoring at a particluar point in time?
  • with changes in development software/platform (ie. VS2003-2010) eventually you will eventually end up with such a fragmented system that it will be almost unmaintainable and span a number of versions of the devt tool; so your overheads will be immense
  • even if everything is meta driven (config, screens, data etc) you will still end up in trouble over time
  • one can see how this would work satisfactorily in the early stages of a system but 20-30 releases later & 10 years down the line you run the risk of having a real mess
  • what happens if a customer never migrates from day 1 even after 10 years?
  • although a lot of redundant code (overhead) could be mitigated by web servcies etc. there will still remain an element of code (bloat) that is never going to be used x years later
  • Ultimately you could heve 10-20 or even 100 versions of each screen. report, class or even function to make up the system

The desktop contrast with this is simplicity itself - don't buy or install the next version

Perhaps I have mis-understood the situation but it does seem an immense undertaking - for marginal gain

challisc's picture

Of course bugs happen, all of the time

challisc | | Permalink

Whilst we wait for @ddruker's response to the latest questions, (which I'm sure will be enlightening), @mkcdavies made the point "Of course bugs happen, all of the time"

Indeed! I have a SaaS cloud bug right now, and I write hot-foot from talking to the vendor.

It’s not accounting. It could just as easily have been on-premise. It’s “business-embarrassing” rather than “business-critical”. It’s using a function I hadn’t used before, rather than an upgrade. The system is “on trial”, albeit live. But none of this matters.

The point is that having found and reported the bug last week, we’re now day 5 (talking 24x7). Last I heard they hadn’t found the cause, let alone provided a solution.

It’s a reputable supplier, and the staff seem genuinely keen to help. But we’ve been through all the usual hoops - a different support person at each stage, “send me a screen shot” etc etc. – and now it’s been rightly escalated to a “Higher Level” (spooky!).

But what if it had been a critical bug in an upgrade? Maybe the support would have been much quicker.  I suspect not. I’m just a small fish in their ocean of customers. I’ve now put a work-around in place, but it’s not a fix.

What if it was a business-critical application (as accounting often is)? In terms of pre-testing releases, what do other users think?

(If you’d like to see the bug in all its glory, click here and then click a “Read more” link near the bottom)

SimonH's picture

Hmmm...

SimonH | | Permalink

Some very good points made by Anonymous in "Bordering on the Impractical".  Hmm..Maybe Sage's decision to wait isn't so foolish after all, even if the style of delivery was somewhat gauche. They'll probably just buy someone out anyway. 

One of the potential advantages of Microsoft's Silverlight is that it could be run "out-of-browser" i.e. as a local application as well as a cloud app. This sort of technology may be a solution to the upgrade issue by giving the user the option to stay on a local, older version or the latest cloud version without changing programs.

challisc's picture

Multi-tenant – single vs multiple databases

challisc | | Permalink

Many SaaS cloud systems work with a large number of their “tenant” users being on the same server (or set of servers) – “multi-tenants”. Often all the tenants’ data is in the same shared database. “Anonymous” asked whether separate databases are needed to facilitate the upgrade processes mentioned by @ddruker at Intacct.

Shared databases also pose another issue – that of data confidentiality. I’ve used a shared database SaaS trading system for some years. Each morning at 7am a set of activity reports are produced for the previous day.  So far so good, but I dread the day I find other tenants’ data in our reports, and wonder who’s seen ours!

Likewise in logging on we’ve only ever seen our data. Again I dread the day that’s not the case. The vendor is using techniques that put pretty strong dividing walls in their database! But as “bugs always happen”, it strikes me daily that separate databases would be inherently safer.

Separate databases can potentially have other advantages. For example, it’s presumably easier to recover one tenant’s database (in the event of data corruption or user cock-up) without it affecting other tenants.

I came across a whitepaper. Written in 2006, the introduction covers an old chestnut that is still relevant today:

Trust, or the lack thereof, is the number one factor blocking the adoption of software as a service (SaaS). A case could be made that data is the most important asset of any business—data about products, customers, employees, suppliers, and more. And data, of course, is at the heart of SaaS. SaaS applications provide customers with centralized, network-based access to data with less overhead than is possible when using a locally-installed application. But in order to take advantage of the benefits of SaaS, an organization must surrender a level of control over its own data, trusting the SaaS vendor to keep it safe and away from prying eyes.

To earn this trust, one of the highest priorities for a prospective SaaS architect is creating a SaaS data architecture that is both robust and secure enough to satisfy tenants or clients who are concerned about surrendering control of vital business data to a third party, while also being efficient and cost-effective to administer and maintain.

It then goes on to discuss three data architectures, one of which lies between fully separate and fully shared. Since then there’s been plenty of debate elsewhere on the internet about the issue and other technical options (with a range of different database products) that can capture the advantages of shared databases whilst minimising their disadvantages.

(To confirm, I’m in the “Positive but cautious” camp in respect of using SaaS cloud solutions)

Does anyone have any particular thoughts to share here about the choice of architecture to facilitate data security, upgrade processes and related matters (at a level that the user community can understand)?

 

DuaneJAckson's picture

Multi-tennant

DuaneJAckson | | Permalink

 I don't see anything in the response from Intaact that implies they have multiple version or a single-tennant database architecture. They do what we do: new features are off an onable, defaulting to "off". So if a user doesn't want a new feature they don't enable it. That doesn't imply they're running an older "version"

@challisc - We could all spend lots of time worrying about things that may never happen. Why do you expect there will ever be a scenario where you'll have others peoples data mixed in with yours? A multi-tennanted database architecture doesn't imply it's likely to happen.

I strongly suspect a lot of firms that use the single-tennant DB structure do so because they have to perhaps due to transitioning from a desktop app to a web-based one and re-using the same code -  and as such they then find technical/marketing reasons to justify their supposed "choice".

There's much, much more to single/multi tenant than the broad strokes outlined in your post. Much of it is technical/scalability issues as opposed to anything else.

 

Afraid I don't understand

Anonymous | | Permalink

@duane

Maybe I have misinterpreted the response from Intaact

'.. we almost always give customers the choice of wheher they want to use something new or not - the new features become configuration options - eg do you want to do it the old way or the new way ..'

Of course this works OK if one is dealing with a upgrade to a single component in an either/or scenario.

However, in the lifetime of an app (10 years +) one component could have many upgrades (100 or more) and the implication from the quote above is that a user could decide to remain on one version of one component througout the life of the app

Now bear in mind that this only applies to a single component and within an accounting app we could have 300-400 or more compnents (unless some of the screens are meta driven).

So just to have switches for the basic one off configuration could mean 300+ flags - now factor in 100+ versions of each component over the lifetime of the app and we end up with rather a lot of flags to keep track of

So I guess the question is:

Is the above interpretation correct and if not how does the provider maintain user choice 'not to upgrade' a specific component in say year 4 of the application lifespan?

in your own response

'.. new features are off an onable, defaulting to "off" ..'

For example if you change(update) a screen_ver00 then presumably this is a 'new feature' screen_ver01 and the user would have the option of remaining on screen_ver00 and not enabling the new one?

Now what happens if next year screen_ver01 is changed again to a screen_ver02 - can the user choose to remain on the screen_ver00 or is the only option available a choice between screen_ver01 (which they didn't want) and screen_ver02?

DuaneJAckson's picture

Pass

DuaneJAckson | | Permalink

@anonymous Probably not for me to clarify what I think another company is saying, so I'll leave them to do so!

Quite understand but

Anonymous | | Permalink

@duane

Quite understand - about I @ddruker and Intacct

However, could you explain your own approach

in your response

'.. new features are off an onable, defaulting to "off" ..'

For example if you change(update) a screen_ver00 then presumably this is a 'new feature' screen_ver01 and the user would have the option of remaining on screen_ver00 and not enabling the new one (screen_ver01)?

Now what happens if next year screen_ver01 is changed again to a screen_ver02 - can the user choose to remain on the screen_ver00 or is the only option available a choice between screen_ver01 (which they didn't want) and screen_ver02?

DuaneJAckson's picture

Not screens

DuaneJAckson | | Permalink

 @anonymous I can't think when we've ever replaced a whole screen. Screens are made up of elements.

so for instance, when we added the option to send an invoice using ViaPost, this needs a button element on the invoice page to send it to viapost. That button only appears if you've enabled that option within your settings.

challisc's picture

Risk management

challisc | | Permalink

@DuaneJAckson . Thanks for clarifying the likely IntAcct approach, and your own..

You’re of course right that there's much, much more to single/multi tenant than the broad strokes outlined in my post, and that technical/scalability (and cost) issues  are important.

Looking from a user perspective, I took the stance of opening the bonnet and admiring the engine. But like most users, cannot see what’s going on inside.

In terms of risk management I have identified a risk and evaluated the consequences (bad!). Moving on to assess the probability, this cannot immediately be done for many SaaS offerings. The probability may be low-ish but not necessarily low enough (or preferably zero given the consequences).

To answer your question “Why do you expect there will ever be a scenario where you'll have others peoples data mixed in with yours?”:

The whitepaper I mentioned confirms that one shared database approach is for “a Tenant ID column [that] associates every record with the appropriate tenant”. That’s clear that data would be mixed using this approach. I have seen this approach used in practice, so tend to assume this is the case for any shared-database SaaS offering unless proven otherwise. 

What alternative approach do you suggest?

still confused

Anonymous | | Permalink

@duane

Appologies obviously haven't made myself very clear which has mean't you have been unable to answer the question

'screen_verXX' was used in a generic sense to describe any aspx, ascx, control, component, web service(api), stored_proc etc; however one wishes to refer to them they are a change that has to be loaded at some point

Irrespective of venacular how do you handle multiple changes over many years so using a simple example of the 'button' mentioned in your previous posting

With this button you have the following:

  • year 1 - button.text = 'hello'
  • year 2 - button.text = 'hi'
  • year 3 - button.text = 'hello there'
  • year 4 - button.text = 'hi there'

2 questions

  • How does your system allow me to keep the 'year 1' option (button.text = 'hello') in 'year 4'
  • Can I subsequently choose to have the 'year 3' (button.text = 'hello there') version if I wish

This is a very basic example (not necessary real life) to outline the point being made

DuaneJAckson's picture

No alternatives

DuaneJAckson | | Permalink

 @challisc. I don't propose any alternatives. There's nothing wrong with the multi-tenanted approach IMHO. As long as the developers know what they're doing there's no risk of data leakage or seeing each others data. The web is a multi-tenanted environment, so it seems backwards to me to have single-tenanted database architectures. The risk is that a crap development team will do something silly which results in mixed data. If the development team is that bad then there are just as many things they can do to mess things up on a single-tenant system.

@anonymous In that example, the string on the button would be a variable and you can call it whatever you like.

Over 5 years of constant incremental development, we've not had problems with the off-and-onable approach we've taken to new features.

david_terrar's picture

On testing, total control, and other matters

david_terrar | | Permalink

@challisc - Thanks for separating out this discussion.  Haven't joined in for a few days, so I'm responding back in sequence to the comments above from the top.

@rworboys - "not been properly tested" - that's a very subjective statement -  If any software provider releases software that's not been properly tested they're crazy.  However, there are limits to what you can do, and software is never perfect.  On changing reports, that's always tricky.  Our platform did a dramatic upgrade of the reporting environment (a  change long overdue), but we left the old reporting environment in place for about 6 months to allow people to make the transition.  If there had been a big customer backlash, we would have left the old reports in place.  When the time came to remove them, everybody had moved across, loved the new environment and there wasn't any issue.  SaaS providers need to cater for that kind of transition and listen to the customers. 

@carnmores - although I agree with much of JC's response to your comments, let me answer some of the points you've raised. 

On "total control of the operating environment" - it's our operating system, our database, our hardware.  When I was at CODA we had to test the same code against 26 different combinations of operating systems, database and hardware that we "certified".  With PC software it is even worse, because almost every desktop PC can have something different installed.   When you control ONE operating environment it's much easier and quicker to test.

"Try running that past Desktop Vendors" - talk to SaaS vendors.  The Desktop guys get the bulk of their revenue up front, and then they're off to sell the next one (they make margin on maintenance revenue, but the emphasis is entirely different).  SaaS vendors start with a small subscription fee, maybe a pilot.  They start making money when you've rolled out the solution to more users and kept the software for 2 or 3 years or more.  It means the business model is different.  More spent on service and support and making the project a success, less spent on sales, sales people and software demos. 

The rest of your first comment from "Probably true" - I'm with JC, particularly his last sentence.

On your next comment - I don't see anyone is being pious or suggesting they are the saviour of accounting software.  It's in  the eye of the beholder.

Getting back to @challisc's original question - I can see that some vendors may provide the sandbox approach or the @ddruker's Intuit beta approach, but probably at a premium.  Most SaaS providers won't use that route - they will have a test group of friendly users that they move across first for final testing phase, and then the whole community moves to the new release subsequently.  

@challisc - on your bug, if it was business critical you ought to be getting a lot of attention  from the supplier.  The provider should be motivated to deal with it as promptly as possible because with today's social media tools you can spread the word on bad support very quickly.  One of the difficulties of being a software supplier is that you can guarantee times to respond and escalate, but you can't guarantee time to fix.  However, your due diligence will uncover the supplier's track record.

@challisc - on the multi-tenant thing, first I'm with @Duane, but in any case I think it is an architectural decision that is separate from data security.  You can have a shared database across all customers,  maybe for performance or ease of upgrades, but you HAVE to handle the security between customers properly.   I'm really not sure how useful it will be for the accountant audience to dive too much in to the pros and cons of multi tenant vs other alternatives (even though we developers care a great deal). 

David Terrar
www.d2c.org.uk and www.twinfield.co.uk 
 

dahowlett's picture

Back to reality

dahowlett | | Permalink

@michaeltrigg: unfortunately, our friend @challisc seems to be opening up a debate that boils down to an implied consulting engagement. Even more unfrotunate, most of his arguments are breathtakingly ill informed if beautifully articulated. If you read this piece on security, there are some pointers as to what you SHOULD be doing. And yes, if you want the meat and potatoes on this one, I'm more than happy to provide a consulting service that addresses those issues. Read this piece and you'll see IRIS is most definitely moving to the cloud. That's why Phill Robinson was hired as head of APS. If you know anything about Phill, you'll also know he came from Salesforce.com. When you know how the IT industry works, it doesn't take more than 20 mano-seconds thought to join the dots. This is not new. It's been in gestation for some time. Add in the recent FreeAgent investment and you know what's coming.

To the broader point, I'm just decompressing from 2 weeks in the US with vendors who are moving to or have products in the cloud. Multi-tenancy IS an industry issue that speaks to cost. That is a pre-determinant of YOUR final cost. Lots of factors around that. Analysts like myself don't care whether it is multi-tenant or not provided the vendor can get the economics correct. Reality suggests that multi-tenant is an important factor BUT, it depends what the vendor is offering/doing, what's running and a slew of technical factors way too boring for these pages. One data point. A vendor I just met has now achieved a 5x reduction in costs by adopting multi-tenancy. It will soon be 10x. One vendor I met offers single or multi-tentant configurations. You pay an order of magnitude more to have your data held on your blades in the data center. If that's the price you want to pay then fine. But then there's also other complications around SLAs in those situations.

As to security more broadly, there are many potential models and as I've said - not all vendors are born equal on this issue. Neither do all vendors approach the topic in the same way. But to the central point I have made on many occasions: there are no recorded instances of catastrophic data loss from cloud providers. There are instances where 30 minutes' worth of data was lost but that's about it. That's trivial for 99% of current environments. That is a powerful data point. Compare that to the on-premise failures that occur year in and year out. There is a history here that stretches right back to the earliest days of client/server and which is still evidenced in continuing cases. 17-18 years of failure of one kind or another. That's another broad data point. Tell me. What more do you need to know as the starting point for an intelligent discussion around this topic?

To the point about network speed. Yes, that's an issue for some vendors but I've seen massive systems running just fine on hotel hosted EDGE networks. It's not about the network, it's about what the vendor has (or has not) done to optimize across many network environments.

challisc's picture

Still “positive but cautious”

challisc | | Permalink

Thanks @michaeltrigg and @dahowlett for enlivening the debate.  “Beautifully crafted” has made my day!

On the key points I’ve raised:

  1. @ddruker has agreed that “Beta and change management are important”
  2. @david_terrar has agreed that “you HAVE to handle the security between customers properly”
  3. My concerns about database security have been expressed far more eloquently in the white paper I mentioned

@DuaneJAckson’s question that prompted my original contribution to this forum happened to be a couple of days before a conversation over lunch with a former client FD. He’s now involved with a SaaS player specialising in data rooms for highly confidential data (clinical trial data and corporate finance due diligence).

He was telling me about their approach to SaaS. They had clearly listened to the market, had understood their needs and concerns, and are providing a service that addresses them. Take-up of their service is swift. I would be comfortable in principle to use this service, subject to closer assessment – even for highly confidential data

I then asked myself the question: “Would I be as comfortable with the current crop of SaaS offerings in the accounting and ERP markets?”.  Up to now client FDs had expressed a strong preference for on-premise solutions, but are under increasing pressure to take to the cloud. With recent advances in SaaS offerings, should I encourage or stall them? Not sure.

Having worked successfully “at the coal face” with both on-premise and SaaS systems over many years, I had some critical concerns. This forum has been a useful way to air them, and I’ve been heartened by positive points made by vendors (such as in the summary at the top of this post). We’re all learning.

The vendors have also highlighted that not all SaaS players are equal. As @ddruker says, the industry needs to mature.

(As to Dennis’s “piece on security”, for now I’ll leave it to others to comment on the flaws in his principle arguments. Pity as his articles tend to be spot-on.)

IN CONCLUSION

Bottom line is I remain “positive but cautious” about SaaS. It is clearly the future. But I believe the quality of vendors needs careful assessment in each market, across a range of issues of which these are just a couple. Some vendors are up to the job, many aren’t (yet). In my view it is dangerous to assume that all SaaS players are equally “safe” and acceptable.

As @ddruker says, what I’m asking is “really a SaaS maturity question”. On the commercial side, could vendors turn these issues into a business opportunity, by charging higher fees for premium services?
 

dahowlett's picture

Flaws

dahowlett | | Permalink

@challisc - don't fight shy my friend...tell me where the flaws (in your opinion) lay. The fundamental basis of your stated argument is plain wrong. Like nearly all doubting Thomas's, you fall into the trap of extrapolating what MIGHT go wrong into something that becomes inevitable. That despite facts and data points to the contrary. It is Y2K syndrome. It is also a great example of irrational buying behavior I see on numerous occasions that almost always leads to sub-optimal results.

The other error you're making is in failing to recognize what's going on in the market place. I don't see any pressure as you call it, unless you're referring to success stories. There's plenty of talk but much of it is ill-informed and ill-advised. But what's happening generally is that CLIENTS are buying ahead of the accounting fraternity adoption curve. Embrace and make sensible choices rather than be faced with the unpleasant prospect of losing clients to those that 'get it.'

The only people who truly understand the security topic to the kind of depth I believe would satisfy you are the kind of 20+ year specialist security engineers I speak to. Very few of the vendors have access to those people. If you think you are worried now, those guys will make your hair curl...or curl more if it is already curly. BUT - the people are trained to think about outlier cases that might arise once in 100 million episodes. Why - because if you're running a banking system for example, that could be once a day in some of the largest data center operations I know about. This is NOT to that scale or anything remotely like it.

I could understand your concern if we were tlkaing about something that was invented yesterday but we're not. The broad guiding principles and some of the products have been around over a decade. You claim to have ben using cloud for some while. Surely you must realize this.

If you're begging for standards then certailny they need to evolve. But if you are relying on waiting for those to 'happen' I'm reckoning you're in for a long wait. In the meantime, some of us are quietly beavering away on the topic.

Let's be honest here. IE 6 has been a dog's breakfast with security holes exploited almost weekly and sometimes more frequently for years. Yet the vast majority of clients use IE6 and patching is hit and miss at best. IE 7 is not free and clear. A simple step such as switching to Mozilla would fix that. Nothing to do with the application but everything to do with poor software selection choice.

But don't expect to see definitive answers on this media site because what we're talking about is IP.

This industry is extraordinary in developing myths and legends...the fun, I am sure, will go on.

challisc's picture

Re: Flaws

challisc | | Permalink

It’s been eerily quiet this morning. Busy people I guess.

@dahowlett: Probably best for me to start a separate thread on security, when I’ve got more than a moment. Having carried out security reviews at a range of businesses, from mega-corporate to mom’n’pop, I certainly don’t think bank-grade security is needed in most SaaS situations. For accounting, it can certainly be below that of the data room service I mentioned. In the meantime anyone else like to comment on Dennis’s blog?

challisc's picture

Cowpatulation?

challisc | | Permalink

Reading back through @dahowlett’s posting to prepare my security piece (and thanks again Dennis for “beautifully crafted”) one phrase took me aback : “It is Y2K syndrome”

Not being a pro tennis player, I rushed to check the meaning. I confirmed it’s the term often used by people detached from reality.

For example, around 2000 I did some work with the BBC’s shared service centre (SSC), We were using this relatively new-fangled internet-thingy to cut costs in communication between business units and the SSC. I was told that one of the drivers for forming the SSC was that some of the divisional systems had not been millennium compliant. Too late to implement SAP, they had to bear the additional costs of an interim implementation of Tetra Chameleon (now morphed into Sage 1000 / Line 500).

Chameleon had itself been non-compliant as recently as the mid-90s. It had a very peculiar form of millennium bug. Dates were stored as 2-digit text in the range 70-99 (which made it difficult to link to Excel), and so could not cope at all with dates 2000 onwards. Users had to upgrade or change systems. I know of a company that ignored my advice and chose the latter. Choosing inappropriate systems, the impact on business processes and customer service was a major factor in them losing competitiveness and going out of business shortly after.

Readers will know of many other millennium bug problems. Indeed the short-cuts taken in making some systems compliant are still with us, and ready to bite perhaps sooner than we think. In any case, shouldn’t they be “Centennium bugs”?

Getting back to this post. I know:@dahowlett is well informed. Much of what he has to say is enlightening. But taking a reality check, please tell me Dennis what you meant by “It is Y2K syndrome” if it wasn’t it’s usual meaning.

challisc's picture

Back to reality with a bang!

challisc | | Permalink

From an email received last night from a SaaS service:

“On the afternoon of 19th May, our security systems detected a concerted and sophisticated attempt to hack into user accounts on our sites. As a precautionary measure we temporarily suspended all user accounts whilst we investigated. We are now able to reactivate all user accounts.”

My first reaction was that this was a phishing scam. But it does seem genuine. Did the hackers get through? Or did this particular system have access controls strong enough to withstand? Not all SaaS systems do, be they accounting or otherwise. Knowing that this is always a possibility, I therefore remain “Positive but cautious” and consider each SaaS offering on its merits for the particular application.

Let’s get real!

(P.S. Let’s keep this thread focused on the technical and business issues. Anything on a more personal level, please DM me through the AccountingWeb system. First click on my username below, then "send a private message".)
 

david_terrar's picture

Security, hacking and standards

david_terrar | | Permalink

@challisc - I'd be interested in knowing which and what type of SaaS service suspended user accounts while they sorted out the hack attack.  I'm hoping no online accounting provider would do anything like that, and they should be better protected.  Better providers will use login encryption and things like McAfee Secure to show they are constantly monitoring, and being tested to check that they are "hacker safe".

On that score, I wanted to mention that the 3 UK SaaS vendor groups (Intellect SaaS Group, Euroloud UK and BASDA Cloud SIG) have been talking about security standards together with ICAEW, but have decided to work with the Cloud Industry Forum, which was instigated by the Federation Against Software Theft, but now has many of the major industry players involved.  The intention is to have 2 levels of quality mark, or standard incorporating a code of practice.  Part of the intention is to make any accreditation practical and affordable for the smaller vendors and start ups, as well as the big boys.  Find out more, and join in the public consultation here:

http://www.cloudindustryforum.org/

David Terrar

www.d2c.org.uk and www.twinfield.co.uk   

daveforbes's picture

When upgrades go wrong....

daveforbes | | Permalink

Most software, cloud or traditional, works for most users most of the time. In this socially networked world, even one upset user has a loud voice.

When software does not work for a particular user it tends to be polarised towards either end of the spectrum from small parts not working to catastrophic failure, but never much in between.

The former tend to be faults that would affect all users if they used that particular feature, but are obscure enough for it not to affect many. The latter tend to be installation issues - either problems occurring during installation or conflicts with other software on the system.

If I am using traditional software with a long upgrade cycle and it is my year end on Monday I may well choose to postpone installing a CD I receive on Friday. I do not have this option with web based software - but is this important ? Web based software does not suffer from the "catastrophic installation failure" kind of problem.

That said a minor change might be major for a particular user - as in the example that spurred the original post. Having Duane's options may work in the short term - switching features on and off on user selection. However one day Cloud vendors will be encumbered with a legacy too - "that feature you made optional in 1995 - why does it not work in the latest release". An interesting point that was made that cloud vendors potentially at least know which parts of the software are being used. It would be interesting to hear how much use is made of this.

On the "minor" type of failure there will be a very different perception of quality amonst users. If one cloud software user spots a problem and it is fixed before the 999 others notice they will have a very different perception of the quality of the software from the 999 users of on premises software that have chosen not install the latest download.

On a minor point, if a cloud vendor does introduce a fault and then corrects it, in extreme circumstances, I may be lacking in evidence to pursue a claim. Is there anything in the evolving codes of practice that cover documenting changes and bugs in a manner that is accesible if there is a dispute ?

David Forbes (cloud agnostic)

Forbes Computer Systems

 

challisc's picture

Upgrade bug

challisc | | Permalink

Just chased up the SaaS vendor for a fix to the problem I mentioned in this thread above.  I last heard from them about it on Tuesday over a week ago, when the issue was passed to a "Higher Level".

The system is "content management",  not accounting. It doesn't relate directly to an upgrade. It isn't "business critical", or I would have been chasing harder. But the issue that started my recent involvement in this forum was a painful near-critical reporting problem with a different SaaS system, for ecommerce trading. This problem was never resolved, apparently because the user business was the only tenant/customer to have made use of the particular end-user function.

You may have seen some wag recently suggested that the proposed "Great Repeal Bill" should include the abolition of "Sod's Law". I fully support that. Things do have a habit of going wrong in any walk of life!

But in case I'm the only person suffering at Sod's hands in respect of SaaS upgrades (and I know there are plenty of examples of on-premise upgrade problems over the years), does anyone else have examples of pain with the upgrade to a SaaS system ?

(P.S. I am currently writing a comment on security for the other thread, so I suggest this thread is kept for the original issue of upgrades.)

 

Add comment
Log in or register to post comments
Group: Cloud Computing for Accountants discussion group
A place for accountants to share their thoughts about web-based systems