It would seem self-evident that accountants enjoy dealing with numbers, but those who may not have worked in large corporations would be shocked by the number of meetings that occur arguing over whose version of data is correct. Just answering a simple question like “how profitable is customer x to us” is a nightmarish task in large companies, because business to business enterprises have data about customers scattered throughout a raft of different computer systems, and may not account for costs uniformly across the enterprise. This is not isolated to a few backward companies: a friend of mine who used to work for Goldman Sachs confirmed that if they wanted to know how much business they did with a major corporate client then it was a significant project to gather this information from around the corporation. This is especially a problem with global companies, whose operating subsidiaries and structure may be opaque. For example, Calvin Klein is owned by Unilever, but how many people know that? Consequently it is easy to see how a company supplying Calvin Klein may not necessarily record that information as actually being linked to Unilever. There are many, many examples like this, which would cause issues even without the problem of uniformity of cost allocation across an enterprise.

An article in Accounting and Finance mentions the notion of the “data driven CFO”, which alludes to the notion that CFOs these days really need to get to grips with this kind of issue. Not only are they under pressure to answer questions from their business colleagues about corporate performance (such as “how profitable is customer x to us”) but they also have to contend with increasing regulation such as Sarbanes Oxley and various industry specific rules such as Basel 2 in banking, or FDA rules within life sciences. People often gripe about such regulations, but there are some sound reasons for them. An amusing example of the need for the latter was a discussion my wife (who worked at Smith Kline Beecham at the time) shared with me. SKB had an anti-worming drug which they decided was not worth marketing to humans for technical reasons, but were pondering whether it could be usefully put into pet food, since dogs in particular have a lot of problems with worms. After much ferreting around, it turned out that this could not actually be done because in fact a scary percentage of people actually eat dog food, and so it would have to go through the same regulatory regime as if it was for humans (as an aside, I found this quite surreal, but when I mentioned it at lunch that day at Shell, two of the five people at the table admitted to having eaten dog food; if you think I am joking see the internet for recipes involving pet food).

The point is that the regulations are a fact of life and are increasingly specific about the auditability of reported data. Much of which may involve reconstructing past history at some point a few years back. Now if a company can’t tell how profitable a customer is without a significant effort, just how easy will it be for them to answer a question regarding profitability four years ago, since when the business has probably restructured a couple of times? I suspect that there are a lot of increasingly nervous CFOs out there, who my be sorely tested if they actually have to go back and reconstruct historical data. Yet even apart from regulatory compliance, for departments such as marketing really need to understand trends over time in order to be effective. How many of these are getting the trend data that they need from their finance groups? As pressures increase for performance improvement, more and more CFO departments are going to have to become more data driven in years to come, like it or not. Let’s hope their suporting systems can deal with such requests, or they will spend a long time barking up the wrong tree.

One of the perennial issues that dogs IT departments is the gap between customer expectations and the IT system that is actually delivered. There are many causes of this e.g. the long gap between “functional spec” and actual delivery, but one that is rarely discussed is he language of the business model. When a systems analyst specifies a system they will typically draw up a logical data model and a process model to describe the system to be built. The standard way of doing the former is witti entity relationship modelling, which is well established but has one major drawback in my experiences: business people don’t get it. Shell made some excellent progress in the 1990s at trying to get business people to agree on a common data model for the various parts of Shell’s business, a thankless task in itself. What was interesting was that they had to drop the idea of using “standard” entity relationship modelling to do it, as the business people at Shell just could not relate to it.

At that time two very experienced consultants at Shell, Bruce Ottmann and Matthew West, did some ground-breaking research into advanced data modelling that was offered to the public domain and became ISO standard 15926. One side effect of the research was a different notation to describe the data used by a business, which turned out to be a lot more intuitive than the traditional ER models implemented in tools like Erwin. This notation, and much else besides is described in an excellent whitepaper by Bruce Ottmann (who is now with Kalido).
We use this notational form at Kalido when delivering projects to clients as diverse as Unilever, HBOS and Intelsat, and have found it very effective in communicating between IT and business people. The notation itself is public domain and not specific to Kalido, and I’d encourage you to read the whitepaper and try it out for yourself.

An article in Intelligent Enterprise asks “why can’t vendors and customers just get along?” after explaining the many issues at which they are usually at loggerheads. Having been both a customer and a software vendor, I think Joshua Greenbaum points to one key point in his article: honesty. As a customer I found that software vendors frequently made patently absurd claims about their software “this tool will speed up application development by 1000%” being one memorable example. Release dates for software were another issue: vendors fail to grasp that businesses have to plan for change all the time, so a release date slipping by three months is rarely an issue provided that you are told about it in good time. However if you have lined up a load of resources to do an upgrade, it just not turning up on the appointed day does cause real cost and annoyance.

Another bug-bear is testing, a tricky subject since it is impossible to fully test all but the most trivial software (see the excellent book “the art of software testing“) . However vendors vary dramatically on the degree of effort that they put in. At Kalido we have extensive automated test routines which run on every build of the software, which at least means that quite a lot of bugs get picked up automatically, though bugs of course still get through. Yet according to someone who used to work at Oracle, there it was policy to do minimal testing of software, where the testing strategy was described as “compile and ship”. This certainly avoids the need for lots of expensive testers, but is hardly what customers expect.

However, customers can be unreasonable too. Their legal departments insist on using sometimes surreal contract templates that were often designed for buying bits of building equipment rather than software, resulting in needless delays in contract negotiation (but in-house corporate lawyers don’t care about this, indeed it helps keep them busy). They can also make absurd demands: we recently lost a contract after refusing to certify that we would fix ANY bug in the software within eight hours, something which patently cannot be guaranteed by anyone, however responsive. A small vendor who won the bid signed up to this demand, and so will presumably be in breach of contract pretty much every day. Quite what the customer thinks they have gained from this is unclear. It is not clear why some customers behave in such ways, perhaps they feel like exacting revenge for previous bad experiences with vendors, or maybe some corporate cultures value aggressive negotiating.

From my experience on both sides of the fence, the best relationships occur when both parties understand that buying enterprise software is a long-term thing, in which it is important that both sides feel they are getting value. Vendors are more inclined to go the extra mile to fix a customer problem if that customer has been doing lots of reference calls for them, and actively participates in beta test programs, for example. As with many things in life, there needs to be a spirit of mutual respect and co-operation between customers and vendors if both are to get the best out of their relationship.

An article I read in in DM Review today highlights Forrester Research saying that “25 percent and 40 percent of all enterprise users” would eventually use BI software. I’m not quite sure what they are smoking over at Forrester, but this seems to me like another of those lessons in the danger of extrapolating. You know the kind of thing: “if this product growth of x% continues, within ten years everyone on the planet will have an iPod/Skype connection/blog/whatever.” The flaw with such thinking is that there is a natural limit to the number of people that will ever want a particular thing. In the case of enterprise software that number is, I would suggest, much lower than commonly supposed. This is for the simple reason that most people are not paid to do ad hoc analysis of data in their jobs. Sure, some finance and marketing analysts spend their days doing this, but how many powerful BI tools does the average sales rep/bank clerk/shelf stacker really need? I’m thinking none at all, since their job is to sell things or do whatever it is that bank clerks do, not be musing on the performance of their company’s products or its customer segmentation.

In my experience of implementing large data warehouse systems at large corporations, there are remarkably few people who need anything more than a canned report, or just possibly a regular Excel pivot table. A salesman needs to work out his commission, a support manager needs to track the calls coming in that week, but these are for the most part regular events, needing a regular report. In a large data warehouse application that has 1,000 end users of the reports produced from it, the number of people setting up these reports and doing ad hoc analysis may well be just 10 i.e. around 1% of the total. Once you get past management and the people paid to answer management’s questions, there are just not that many people whose job it is to ponder interesting trends, or explore large data sets for a living. For this reason a lot of companies end up procuring a lot more “seats” of BI software than they really need. In one case I am intimately familiar with, even after five years of rolling out a leading BI product, the penetration rate was always much lower than I had expected, and never went as high as 5%, and much of this usage was not for genuine “BI” usage.

Of course this is not what the salesmen of BI vendors want to hear, but it is something that IT and procurement departments should be aware of.