I enjoyed an article by Madan Sheina on the failure of BI projects. 87% of BI projects fail to meet expectations, according to a survey by the UK National Computing Centre. I wish I could say this was a surprise, but it is not. Any IT project involves people and processes as well as technology, yet many project focus almost entirely on the technology: tool choices, database performance etc. Yet in practice the issues which confound a BI project are rarely the technology itself. Projects fail to address actual customer needs, and frequently don’t acknowledge that there are several user constituencies out there with quite different requirements. Frequently a new technology is stuffed down the customer’s throat, and projects often neglect data quality to their peril.

From my experience, here are a few things that cause projects to go wrong.

1. Not addressing the true customer need. How much time does the project team spend with the people who are actually going to use the results of the BI project? Usually there are a subset of users who want flexible analytical tools, and others who just need a basic set of numbers once a month. A failure to realise this can alienate both main classes of user. Taking an iterative project to project development rather than a waterfall appraoch is vital to a BI project.

2. Data is only useful if it is trusted, making data quality a key issue. Most data is in a shocking state in large companies, and the problems often come to light only when data is brought together and summarised. The BI project cannot just gloss over this, as the customers will quickly avoiding using the new shiny system if they find they cannot trust the data within it. For this reason the project teams needs to encourage the setting up of data governance processes to ensure that data quality improves Such initiatives are often outside the project scope, are unfunded and require business buy-in, which is hard. The business people themselves often regard poor data quality as an IT problem when in fact it is an ownership and business process problem.

3. “Just one more new user interface” is not what the customer wants to hear. “Most are familiar with Excel and are not willing to change their business experience” was one quote from a customer in the article. Spot on! Why should a customer whose main job is, after all, not IT but something in the business, have to learn a different tool just to get access to data that he or she needs? Some tool vendors have done a good job of integrating with Excel, and yet are often in denial about this since they view their proprietary interface as a key competitive weapon against other vendors. Customers don’t care about this; they just want to get at the data they need to do their job on an easy and timely way. Hence a BI project should, if at all possible, look at ways of allowing users to getting data into their familiar Excel rather than foisting new interfaces on them. A few analyst types will be prepared to learn a new tool, but this is only a small subset of the audience for a BI project, likely 10% or less.

4. Data being out of date, and the underlying warehouse being unable to respond to business change, is a regular problem. Traditional data warehouse designs are poor at dealing with change caused by reorganisations, acquisitions etc, and delays in responding to business change cause user distrust. Unchecked, this causes users to hire a contractor to get some “quick and dirty” answers into a spreadsheet and bypass the new system, causing the proliferation of data sources to continue. Using packaged data warehouse that are good at dealing with business change is a good way of minimising this issue, yet even today most data warehouse are hand-built.

5. Training on a new application is frequently neglected in IT projects. Spend the time to sit down with busy users and explain to them how they are to access the data in the new system, and make sure that they fully understood how to use the system. It is worth going to some trouble to sit down with users and train them one to one if you have to, since it only takes a few grumbling voices to sow the seeds of discontent about a new system. Training the end users is never seen as a priority for a project budget, yet this can make the world of difference to the likelihood of a project succeeding.

6. Running smaller projects sounds crass but can really help. Project management theory shows that the size of a project is the single biggest predictor of success: basically, if projects fail, small ones do better, and yet you still see USD 100 million “big bang” BI projects. Split the thing into phases, roll out by department and country, do just about anything to bring the project down to a manageable size. If your BI project has 50 people or more on it then you are already in trouble.

7. Developing a proper business case for a project and then going back later and doing a post implementation review happens surprisingly rarely, yet can help shield the project from political ill winds.

You will notice that not one of the above issues involves a choice of technology, technical performance or the mention of the word “appliance”. Yes, it is certainly important to pick the right tool for the job, to choose a sufficiently powerful database and server and to ensure adequate systems performance (which these days appliances can help with in the case of very large data volumes). The problem is that BI projects tend to gloss over the “soft” issues above and concentrate on the “hard” technical issues that we people who work in IT feel comfortable with. Unfortunately there is no point in having a shiny new server and data warehouse if no one is using it.

The annual IDC business intelligence report shows a reassuring 11% rise in overall BI tool revenues in 2006. Regular readers of my blog know that a couple of my long-term viewpoints are that: (a) Microsoft is the vendor with the long-term best position due to its ownership of Excel, which is still the BI front-end that end-users actually want and (b) that specialist BI tools will never, contrary to many BI vendor projections, have a place on every desktop, due to the rather dull reason that most people do not need one.

Is there any evidence for these hypotheses? Well, Microsoft’s BI revenues grew 28%, while the major specialist BI vendors grew by 7% (Business Objects) and 9.8% (Cognos). As IDC analyst Dan Vesset notes, “IDC does not yet see a substantial impact on the market from the strategy and marketing messages of most BI vendors seeking to reach a broader use base”. Nor will it ever, in my opinion.

Also of note is the formidable performance of Qliktech, which grew by a little matter of 97% to USD 43.6 million revenue. Its offering to the mid-market offering based on in-memory search technology continues to get considerable customer traction. I have a soft spot for this company, having been asked to look at it for a venture capital firm as an investment when it was still a tiny company; I am very relieved that I recommended that they invest - otherwise I imagine that they would be hunting me down like a dog right now.

There is an interesting article in CIO Insight by Peter Fade, a professor of marketing at the top-rated Wharton Business School. in this he discusses the limitations of data mining, and it is an article that anyone contemplating investing in this technology should read carefully. I set up a small data mining practice when I was running a consulting division at Shell, and found it a thankless job. Although I had an articulate and smart data mining expert and we invested in what at the time was a high quality data mining tool, we found time and again that it was very hard to find real-world problems where the benefits of data mining could be shown. Either the data was such a mess that little sense could be made of it, or the insights shown by the data mining technology were, as Homer Simpson might say, of the “well, duh” variety.

Professor Faber argues that in most cases the best you can hope for is to develop simple probabilistic models of aggregate behaviour, and you simply cannot get down to the level of predicting individual behaviour using the level of data that we typically have, however alluring the sales demonstrations may be. Moreover, such models can mostly be built in Excel and don’t need large investments in sophisticated data mining tools.

While I am sure there are some very real examples where data mining can work well e.g. why some groups of people are better credit risks than others, the main point he makes is that the vision of 1-1 marketing via a data mining tool is a fantasy, and that the tools have been seriously oversold. Well, that is something that we in the software industry really do understand. We all want technology to provide magical insights into a messy and complex world that is hard to predict. Unfortunately the technology at present is generally as useful as a crystal ball when it comes to predicting individual behaviour. Yet there is still that urge to go into the tent and peer into the mists of the crystal ball in search of patterns.

Currently enterprise software companies have been trading at around three times revenues, with premiums for particularly good firms up to about five times revenues, less for firms that are not showing much market progress. There have been several examples of this type of deal recently. Enterprise software CEOs could be forgiven for a casting an envious eye at the internet software market. On the UK market AIM a company called Blinkx, which offers the ability to search video clips (using technology from Autonomy) recently raised money, with its first day trading giving a valuation of GBP 180 million. Any guesses to the revenues or profitability of this company? Revenues of GBP 60 million perhaps, maybe as low as GBP 40 million? Nope. Revenues are expected to be just over GBP 2 million in 2007. Profits? “Profitability is not expected until 2010″.

How about the teenage scribblers who presumably can explain this kind of valuation? According to an analyst at Dresdner Kleinwort: “It is hard to value because we don’t know what it is going to focus on. There’s no proven management history, and few historical numbers to play with”.

Does this kind of language ring any bells? Does anyone recall a time in the far distant past when companies could not be valued using “old fashioned” methods like price/sales or price/earnings, since the internet was a new business model? Maybe you were wiser than me, but I admit to to buying some shares in basket cases like Commerce One at the height of the bubble, believing the previous generation of teenage scribblers that the internet “changed everything” and old fuddy duddies who fretted about irrational exuberance just “didn’t get it”.

Those who cannot remember the past are doomed to repeat it. For the latest lesson we don;t have to think back to the South Sea Bubble or the Amsterdam Tulip fiasco. We just have to cast our minds back six years or so. I for one will not be investing in Blinkx.

Footnote. After writing this I found a thoughtful blog on the same subject.

I attended an interesting talk today by Greg Hackett, who founded financial benchmarking company Hackett Group before selling this to Answerthink and “going fishing for a few years”. He is now a business school professor, and has been researching into company performance and, in particular, company failure. Studying the 1,000 largest US public companies from 1960 to 2004 his research shows:

- company profitability is 40% lower in 2004 than in 1960, with a fairly steady decline starting in the mid 1960s
- the average net income after tax of a company in 2004 was just 4.3%
- half of companies were unprofitable for at least two out of five years
- 65% of those top 1,000 companies in 1965 have disappeared since, with just half being acquired but 15% actually going bankrupt.

He gave four reasons for company failure: missing external changes in the market, inflexibility, short term management and failing to use systems that would show warning signs of trouble. What I found most surprising was that the correlation between profitability and stock market performance was zero.

The research suggests that the world is becoming a more competitive place, with pricing pressure in particular reducing profitability despite greater efficiency (cost of goods sold is 67% of turnover, down from 75% in 1960, though SG&A is up from around 13% or turnover to around 18%). All those investments in technology have made companies slightly more efficient, but this has been more than offset by pricing pressure.

I guess this also tells you that holding a single blue chip stock and hanging onto it is a risky business over a very long time; with 15% of companies folding over that 45 year period, it pays to keep an eye on your portfolio.

A key implication is that companies need to get better at implementing management information systems that can react quickly to change and help give them insight into competitive risks, rather than just monitoring current performance. Personally I am unsure that computer systems are ever likely to provide sufficiently smart insight for companies to take consistently better strategic decisions e.g. divesting from businesses that are at risk; even if they did, would management be smart enough to listen and act on this information? It does imply that systems which are good at handling mergers and acquisitions should have a prosperous future. This is one thing, at least, that seems to have a growing future.

This week is the CDI/MDM Institute London conference. It is a useful bellwether of MDM progress in the UK, and based on the attendance today it looks like MDM interest is indeed picking up in the UK. There are just over 300 attendees and 22 vendors exhibiting. Compared to the one last year there are encouragingly more customer case studies (last year the speakers were mostly vendors presenting), for example from Panasonic, BT, Harrods, Allied Bakeries, M&S and the Co-Op.

It is noticeable that the CDI v MDM debate continues to favour the broader view that customer is just one (important) type of master data, with the MDM acronym now being used by most of the vendors. The Panasonic case study was a good example, starting out as a product master data initiative and now spreading to customer, and then on to market information. The speaker was able to share some real business benefits form the initiative (enabling new products to be launched two weeks quicker as well as data quality savings), measured in millions of pounds. IBM claimed to be integrating its various acquired technologies, which is an improvement from the conversation I had with them a year ago when it was claimed that there was nothing wrong with having a clutch of separate, incompatible repositories, one for customer, one for product etc. When I asked how many different repositories would be needed to cope with all the different types of master data in an enterprise I received the mystical answer “seven”, at which point I gave up as the conversation had seemed to move into the metaphysical realm. We shall see where the integration efforts lead.

Aaron Zornes gave a useful high level split of the MDM market into the groupings of:

- operational e.g. CDI hubs like Siperian, Oracle
- analytical e.g. Hyperion Razza, Data Foundations
- collaborative i.e. workflow (e.g. Kalido does a lot of this)

which seems to me a useful split. Certainly no one vendor does everything, so understanding where the strengths of the vendors are, even in this simplistic way, at least helps customers narrow down which vendors are most likely to match their particular problem.

IBM, Initiate, SAS and Kalido are the main sponsors of the event, and once again SAP chose not to attend (to be fair, SAP did speak at two of the US MDM conferences). Nimesh Mehta assures me that SAP MDM is making steady market progress, but with no numbers he is willing to share I cannot verify this. However the buzz at the conference suggest that most customers here are using products from specialist vendors. One repeated theme in talking to SAP MDM early adopters is its apparent inability to deal well with customer data, perhaps not surprising given the A2i heritage of the product. No doubt SAP has lots of resources to throw at this problem, but at present it is not obvious that it is getting much in the way of production deployments. Clearly SAP’s dominant market position should get it on to every MDM shortlist, but how many real broad deployments there are in production is much less clear.

There were a couple of entertaining exhibit conversations. One Dilbert-esque one was with a sales person. I asked the following question “what does your product do - is it a repository, or data quality tool, or something else?”, The sales person took a sudden physical step back like a scalded cat and said “oh, a technical question; I’ll need to find someone else to answer that.” Now maybe I’m old-fashioned but “what does your product do?” seems to me a question that even a software salesman should be able to hazard a guess at. What kind of questions do you think this salesperson is likely to be able to field? I’m guessing anything beyond “where is the bar?” or “where do I sign the order form” is going to prove challenging.

I was amused to see Ab Initio had a stand. Ab Initio is famous in the industry for its secretive nature e.g. customers have to sign a NDA in order to see a product demo. This is driven by its eccentric founder Sheryl Handler, and makes life hard for its sales and marketing staff. There was indeed no printed brochure or material of any kind, and they (very charming) sales person I spoke to was unable to confirm very much about the company other than it seemed pretty certain that there was a UK office. Ab Initio’s technology has the reputation of being the fastest performing data transformation tool around, and in the UK has most of the really heavy-duty data users (BT, Vodaphone, Tesco, Sainsbury etc) as customers. It must certainly make it interesting trying to sell the thing, but perhaps the aura of mystery paradoxically helps; after all, this is not a company that anyone could accuse of aggressive marketing.

The consolidation trend in the BI industry continued today with Business Objects (ticker symbol BOBJ) announcement of their intention to buy Cartesis, who are essentially a poor man’s Hyperion. One in four Fortune 500 companies use Cartesis for financial consolidation, budgeting and forecasting, and they had USD 125M in revenues, but reportedly had struggled with growth. The purchase price of USD 300M is less than two and a half times revenues, so is hardly what you would call a premium price (Hyperion went for 3.7 times revenues), though no doubt Apax, Partech and Advent (the VCs involved) will be grateful for an exit. This is not the first time Cartesis was bought (PWC bought Cartesis in 1999) but Business Objects is a more logical owner. Not only it is a software company, but the French history of Cartesis should make it an easy cultural fit for Business Objects. With Hyperion disappearing into the maw of Oracle then there were only so many opportunities out there in this space. Business Objects superior sales and marketing should be able to make more of Cartesis than had been done, and strategically this takes Business Objects up-market relative to its core reporting, which makes good sense.

There is a useful overview of the growing appliance market in Computer Business Review:

http://www.cbronline.com/article_cbr.asp?guid=9104551D-56C1-4EE7-BDF9-BD219E8685BF

The appliance market is nothing if not growing, with no fewer than ten appliance vendors now identified by analyst Madan Sheina (who by the way, is one of the smarter analysts out there). Of course apart from Teradata many of these are small or very new. Teradata accounts for about USD 1 billion in total revenue (the accounts will become much clearer once they separate from NCR) though this includes services and support, not just licences. The next largest vendor is Netezza, who does not publish their revenue (though I would estimate over USD 50M). Kognitio used to be around USD 5M in revenue, though they seemed perky when I last spoke to them so may be a little bigger now. DataAllegro will certainly be smaller than Netezza, as will be the other new players. It is too early to say how well HP’s Neoview appliance will do, though clearly HP has plenty of brand and channel clout, especially now that it has acquired Knightsbridge.

Still, so many entrants to a market certainly tell you that plenty of people feel that money can be made. So far Teradata and Netezza have had the field pretty much to themselves, but the entrance of HP and the various newer vendors will create greater competition, which ultimately can only be of benefit to customers.

I spent the last two days at the eWorld Purchasing and Supply conference on London, where I had a couple of speaking slots (you can’t escape your past, and a long time ago I worked in Shell’s technology planning area, which involved software.procurement). I was pleasantly surprised by the scale of the conference. These days most IT conferences are struggling to get decent attendance, as more and more people seek out information on-line. Yet this specialist conference managed to get over 300 attendees on both days, and from the conversations I had these were mostly “real” delegates i.e. people with actual projects and problems to solve, rather than the tyre-kickers who sales people dread and often seem to constitute much of the attendance of some IT conferences.

For another perspective on the conference see the following blog:

http://www.esourcingforum.com/?p=388

Credit where credit is due to the organisers, Revolution Events, who did an excellent job with administration and organisation (only let down a bit by the caterers on day 1 who were of the “oh, we didn’t think this many people would be coming” variety). The exhibits area had decent flow of traffic and the speaking slots stayed well on time, a particular bugbear of mine. Perhaps these more specialist conferences, concentrating on a particular vertical or in this case functional niche, are the way to go.

What we call things is important. The recent rise of data warehouse “appliances”, pioneered by Netezza (and arguably Teradata before that) is an interesting case in point. For years the relational database vendors spent their energy in making sure that transaction systems ran quickly and reliably. Business intelligence applications were not a major focus, and this led to a number of approaches to dealing with very large data warehouse applications. Certain types of index scheme would work very well for read-only BI queries, for example, and Red Brick was an early example of a database optimised as such. Later Teradata did a superb job of carving out a high end niche by using parallel processing hardware and specialist database software to take advantage of this properly. They did such a good job that after a while Teradata almost became synonymous with large data warehouses, of the types typically encountered in retail banks, supermarket chains, telcos etc. Oracle and othe others made some half-hearted attempts to fight back with features like star joins, but by then it was too late: the specialist data warehouse device, in the form of Teradata, had become established. Of course such projects were still large and complex. Most data warehouse project costs are associated with people, not hardware or software, and this does not change whether you are using SQL Server or Teradata as your database.

However, marketing can at times (not often, but sometimes) be a clever and subtle thing. When Netezza brought out essentially a device like Teradata, but quicker and cheaper, the label “appliance” was used, and a very clever one it is. In normal English usage an appliance is something that we just plug in, like a toaster or a coffee maker. Without making any such overt claims, the “appliance” label has a comforting implication that your data warehouse project will have that toaster installation-like quality previously lacking with pesky traditional databases. Given that a DW appliance is just some clever hardware and an optimised database, your project issues are in fact identical to those of any other DW project. Analysis, user requirements, data quality, sourcing, design and reporting all have to be done, although the appliance may certainly be able to handle large volumes of data at a much better price point than a traditional hardware/database combination. Since the hardware and software on a project may typically account for less than 20% of the project costs, this is an undeniably useful thing, but hardly takes us into toaster territory.

Yet the label matters. In a rather breathless blog yesterday:

http://www.itbusinessedge.com/blogs/mia/index.php/2006/09/05/flaming-web-20/

Mike Stevens, who I don’t know personally but appears to have a background in PR rather than hands-on data warehouse project implementation, claims that appliances spell “trouble for traditional data warehouse vendors” since an appliance may cost just USD 150k whereas “conventional solutions cost millions”. He falls into the language trap of the appliance. Your data warehouse still has to to deal with all those people-intensive things (data sourcing reporting, testing) whether you use a conventional SQL database and a regular server, or a specialist DW appliance. The issues are all identical, except with an appliance you have some additional cost since less familiar skills will need to be brought to bear (there are more Oracle skills out there than Netezza ones). The savings on hardware by using an appliance may be very significant and comfortably justified on a large data warehouse, but such a project is not going to cost USD 150k and a quick plug in the wall socket.

If this kind of misconception is so easily repeated by journalists (or at least bloggers) then I wonder how widespread this view is amongst IT managers, and how much this has helped data warehouse “appliances” catch on? Would Netezza have done quite so well if they had been labelled something less reassuring, like a “data warehouse turbo toolkit”? It was said that HP was so bad at marketing it would, if it sold sushi, describe it as “cold dead fish”. The “appliance” vendors shows that smart marketing can still be done within hi-tech.