The data warehouse is an established concept and discipline that is discussed in books, conferences, and seminars. Indeed, data warehouses are a standard feature of modern corporations. Corporations use data warehouses to make business decisions every day. In a word, the data warehouse represents conventional wisdom and is a standard part of the corporate infrastructure.
Into this world comes a new technology, big data. In some ways, big data competes or thinks that it competes with data warehousing. Indeed, there are some similarities between big data and a data warehouse. Big data and data warehouses both hold data, both hold lots of data, and both hold data we can use for decision making. So it is natural for vendors of big data to proclaim that with big data, you do not need a data warehouse. At least that is the impression that many big data vendors seem to give.
But is that impression correct? Is it true that with big data, you do not need a data warehouse? First off, what is a data warehouse? From the beginning, the accepted definition of a data warehouse is a set of data that is subject oriented, integrated, time variant, and non-volatile. We quote this definition of data warehouse.
How we define big data is not as clear. Indeed, there are different interpretations of what big data means. We will use the following definition of big data:
Big data:
- Big data encompasses enormous volumes of data.
- We store big data on affordable storage.
- One stores big data in an unstructured manner.
- We manage big data using the Roman census technique.
Read the 10-page whitepaper “Modern Data Architecture” by William Inmon to understand whether you need a data warehouse when you have big data.