An article in CIO magazine revealed a fascinating paradox. 63% of IT executives claimed that they are required to present a cost justification for IT projects (a survey by Cutter) yet according to Aberdeen Group, just 5% of companies actually collect ROI data after the event to see whether the benefits actually appeared. I have to say that my own experience in large companies bears out this “ROI gap”. Most post implementation reviews occur when a project has gone hideously wrong and a scapegoat is required. There are exceptions - I have been impressed at the way that BP rigorously evaluate their projects, and Shell Australia used to have a world-class project office in which IT productivity was rigorously measured for several years (a sad footnote is that this group was eventually shut down to reduce costs). However, overall I think these apparently contradictory survey findings are right: a lot of IT projects have to produce a cost/benefit case, but hardly ever are these benefits tested.

It is not clear that the failure to do so is an IT problem, but rather a failure of business process. Surely the corporate finance department should be worried about this lack of accountability - it is hardly IT’s problem if the business doesn’t bother to check whether projects deliver value. It really should not be that hard. Project costs (hardware, software, consultants, personnel costs)are usually fairly apparent or can be estimated (unless it would seem, you work in government) while benefits are more slippery. This is mainly because they vary by project and so don’t fit into a neat template. However they will usually fall into the broad categories of improved productivity (e.g. staff savings), improved profitability e.g. reduced inventory, or (indirect and woollier) improved customer value e.g. the falling price of PCs over the years. It should be possible to nail down estimates of these by talking to the business people who will ultimately own the project. Once these benefits have been estimated then it is a simple matter to churn out an IRR and NPV calculation - these are taught in every basic finance class, and Excel conveniently provides formulae to make it easy. Of course there are some IT projects that don’t require a cost-benefit case: regulatory being one example (”do this or go to jail”) but the vast majority should be possible to justify.

By going through a rigorous analysis of this type, and then checking afterwards to see what really happened, IT departments will build credibility with the business, something that most CIOs could do with more of.

A recent article in Infoconomy wonders whether SQL Server 2005 will be “enterprise scale”. Wake up call - it already is. It is intriguing that analysts and journalists continue to perpetuate the myth that SQL Server is somehow a departmental solution, not really up to serious usage. This is nonsense; even in 1997, when working at Shell, I led a piece of research cost of ownership of database platforms and was not surprised to find that the DBA facilities of SQL Server were much easier to use than those of Oracle. What I was surprised to discover was just how large some SQL Server implementations were. Of course SQL Server was originally based on the Sybase code base, which to this day runs many of the systems at giant financial institutions, but somehow the myth of “departmental” had stuck in my mind. Recently one of our customers has been looking seriously at switching from Oracle to SQL Server and has tried running some of its largest applications on it. A trial of a multi terabyte Kalido data warehouse, for example, showed that SQL server was slightly slower on some things and faster on others, but broadly speaking there was no performance disadvantage to Oracle. SAP runs happily on SQL Server, so why does the myth persist?

I think it comes down to marketing dollars. Microsoft does not market SQL Server heavily to the enterprise, and spends less money with analyst firms than one might expect. By contrast Oracle and IBM are marketing machines who constantly stress how powerful their databases are. Hence a marketing perception, unchallenged, becomes perceived wisdom. Microsoft is missing a trick here, as Oracle behaves very aggressively to its customers and will not win many popularity polls amongst its large corporate customers. Some would be very tempted to switch to an alternative supplier, and while switching costs would be huge, part of the reason for the inertia is this perception of SQL Server as a departmental database. Given that IBM was outmarketed by Oracle when DB2 had a clear technical lead, it would be a shame to not see Microsoft put up more of a fight in this arena - competition is good for everyone except monopolistic vendors.