Comments on: Data warehouse architectures http://andyonenterprisesoftware.com/2006/05/data-warehouse-architectures/ Andy Hayler, founder of Kalido and The Information Difference, gives his views on the enterprise software market. Issues covered include data warehousing, master data management, business intelligence and data quality. Thu, 16 Oct 2008 07:09:33 +0000 http://wordpress.org/?v=2.0.3 by: Andy Hayler http://andyonenterprisesoftware.com/2006/05/data-warehouse-architectures/#comment-98 Thu, 29 Jun 2006 14:49:00 +0000 http://andyonenterprisesoftware.com/2006/05/data-warehouse-architectures/#comment-98 I think it depends on the scale of data you are talking about. In a B2B situation data volumes may be relatively modest and you may be able to get away with storing full transactional data. However in B2C situations like Telco, retail banking etc the volumes can be truly huge and very challenging. In such situations in seems to me better to work on summaries and avoid the need to store massive volumes of transactions. Although hardware gets cheaper, the growth in data continues. As the old saying goes "Intel giveth, and Microsoft taketh away". I think it depends on the scale of data you are talking about. In a B2B situation data volumes may be relatively modest and you may be able to get away with storing full transactional data. However in B2C situations like Telco, retail banking etc the volumes can be truly huge and very challenging. In such situations in seems to me better to work on summaries and avoid the need to store massive volumes of transactions. Although hardware gets cheaper, the growth in data continues. As the old saying goes “Intel giveth, and Microsoft taketh away”.

]]> by: Stephen Barr http://andyonenterprisesoftware.com/2006/05/data-warehouse-architectures/#comment-88 Mon, 12 Jun 2006 17:56:00 +0000 http://andyonenterprisesoftware.com/2006/05/data-warehouse-architectures/#comment-88 While I take your point about growing data volumes and the current trend to store it "indefinitley", I can't help but draw parallels between what you are suggesting and the data marts of the 80's and 90's where non-transactional summary data was the order of the day because of hardware constraints. It didn't work then - so are you sure it can work now? An archiving strategy is always a major part of any warehouse design, and with cheap, alternative near-line storage available in abundance, I don't see the need to revert to summary data. With summary level data we're back again to the situation where someone has to decide how to summarise, and again leads to a situation where you're designing a warehouse around a set of "presently correct" questions. The beauty of transactional level data, is that it enables you to answer any question. While I take your point about growing data volumes and the current trend to store it “indefinitley”, I can’t help but draw parallels between what you are suggesting and the data marts of the 80’s and 90’s where non-transactional summary data was the order of the day because of hardware constraints.

It didn’t work then - so are you sure it can work now?

An archiving strategy is always a major part of any warehouse design, and with cheap, alternative near-line storage available in abundance, I don’t see the need to revert to summary data.

With summary level data we’re back again to the situation where someone has to decide how to summarise, and again leads to a situation where you’re designing a warehouse around a set of “presently correct” questions. The beauty of transactional level data, is that it enables you to answer any question.

]]>