http://andyonenterprisesoftware.com/2007/03/philosophy-and-data-warehouses/ Andy Hayler, founder of Kalido and The Information Difference, gives his views on the enterprise software market. Issues covered include data warehousing, master data management, business intelligence and data quality. Tue, 07 Oct 2008 20:01:09 +0000 http://wordpress.org/?v=2.0.3 http://andyonenterprisesoftware.com/2007/03/philosophy-and-data-warehouses/#comment-25873 Thu, 29 Mar 2007 20:02:50 +0000 http://andyonenterprisesoftware.com/2007/03/philosophy-and-data-warehouses/#comment-25873 Another reason a data warehouse may be necessary over the source system is related to your item (b). Transactional systems often don't contain all historical information, in archived files or otherwise. Oftentimes, a transactional system may simply overwrite a data field when new data arrives. This may be in accordance with the business rules and the "lost" data may be meaningless in the context of the transactional system. For example, a CRM system is concerned mostly with the current status of trouble tickets in the aggregate, and less so with what the status was three weeks ago. But a data warehouse is often designed to allow for historical trending. In that case, it's important to know there were 400 tickets on hold on January 1 and 350 tickets on hold February 1. Another reason a data warehouse may be necessary over the source system is related to your item (b). Transactional systems often don’t contain all historical information, in archived files or otherwise. Oftentimes, a transactional system may simply overwrite a data field when new data arrives. This may be in accordance with the business rules and the “lost” data may be meaningless in the context of the transactional system. For example, a CRM system is concerned mostly with the current status of trouble tickets in the aggregate, and less so with what the status was three weeks ago. But a data warehouse is often designed to allow for historical trending. In that case, it’s important to know there were 400 tickets on hold on January 1 and 350 tickets on hold February 1.

]]> http://andyonenterprisesoftware.com/2007/03/philosophy-and-data-warehouses/#comment-25477 Mon, 26 Mar 2007 14:31:02 +0000 http://andyonenterprisesoftware.com/2007/03/philosophy-and-data-warehouses/#comment-25477 Even if the data warehouse is, in part at least, 'virtual' (in the sense that it may access data from operational systems) there is still potentially a significant requirement to be able to build a data warehouse infrastructure to model the business from the viewpoint of the business user in order to be able to query across systems. (One may choose, of course to use a term other than 'data warehouse'). Even if the data warehouse is, in part at least, ‘virtual’ (in the sense that it may access data from operational systems) there is still potentially a significant requirement to be able to build a data warehouse infrastructure to model the business from the viewpoint of the business user in order to be able to query across systems. (One may choose, of course to use a term other than ‘data warehouse’).

]]> http://andyonenterprisesoftware.com/2007/03/philosophy-and-data-warehouses/#comment-25473 Mon, 26 Mar 2007 12:19:12 +0000 http://andyonenterprisesoftware.com/2007/03/philosophy-and-data-warehouses/#comment-25473 "data warehousing" is really about logistics. And just as in the world of "real" logistics it is about designing a process in which transport, storage (de-coupling points between different types of transport) and manipulation has to be arranged in such a way that customerneeds are satisfied at the desired level of quality, relevancy and - above all - timeliness. These demands need not only to be satisfied in the initial project but also in the ever changing world of the average multi national. That could mean that in a simple process no de-coupling at all is needed and - as is the case im most more complex situations - a much more complex combination of data warehouse(s) mind the plural and datamarts is more than necessary. I tend to believe that even if and when the master data problem is solved (and as Andy rightfully points out that will take some time yet) there are still good reasons to see this type of de-coupling (and integration points) for some time to come (never say never). And yes it very interesting to speculate on a "brave new world" for BI where is no need for this. “data warehousing” is really about logistics. And just as in the world of “real” logistics it is about designing a process in which transport, storage (de-coupling points between different types of transport) and manipulation has to be arranged in such a way that customerneeds are satisfied at the desired level of quality, relevancy and - above all - timeliness. These demands need not only to be satisfied in the initial project but also in the ever changing world of the average multi national. That could mean that in a simple process no de-coupling at all is needed and - as is the case im most more complex situations - a much more complex combination of data warehouse(s) mind the plural and datamarts is more than necessary. I tend to believe that even if and when the master data problem is solved (and as Andy rightfully points out that will take some time yet) there are still good reasons to see this type of de-coupling (and integration points) for some time to come (never say never). And yes it very interesting to speculate on a “brave new world” for BI where is no need for this.

]]>