What is Your Big Data Story?

by Nov 17, 2014


I've been speaking, teaching and ranting on big data and NoSQL technologies recently. I've noticed that when I chat with many data modelers, I've met with a lot of skepticism about big data.

You may be that "guy" if you've ever said: • It's just mainframe all over again.

• It's a fad to get out of data modeling tasks

• It's not something I need know about

• I don't have big data, so I don't need to know NoSQL

I think what's missing from that thinking is the fact that modern data architectures use technologies that are the best fit for solving a data "problem". I like to use the term "data story" instead of "problem." It's not that the newer technologies are replacing traditional relational database systems; they are complementing them.

We've Seen This Before

I'm experienced enough to remember when the big controversies were brewing over whether or not to build denormalized data structures to support reporting and data warehousing. What eventually came out of those conflicting points of view was a good understanding that we need to optimize structures for consuming data differently than we do for creating and maintaining data. I see no difference between that optimization of a data story than the use of NoSQL and big data technologies to solve the stories for those data uses. The industry tends to lump these all under the term "big data", but I don't believe all the data stories in this area are applicable just to high volume datasets.

If a modern data architecture makes use of a variety of technologies', then a modern data architect/modeler needs to understand what those solutions and provide modeling services for them.

So instead of asking are you team SQL or team Big Data, you should be asking "I have this data, which technologies should I be looking at to tell its story.

What is your big data story?