Why Do Data Modelers Experience So Much Conflict on Agile Projects?

by Feb 28, 2015


I spoke about agile methods, including Scrum, and data modeling in my Heart of Data Modeling webinar this month. One of the topics I touched upon was the concept of conflict and hostility between Data Modelers and Agile Developers.  We modelers have always experienced a bit of conflict with development teams due to the fact that our roles are measured and rewarded for conflicting outcomes.  Modelers are typically rewarded for ensuring data quality, for building models that are highly reusable and sharable, and for identifying design issues that conflict with those goals.

 Developers are typically rewarded for producing applications and reports as fast as possible.  They are rarely measured for data quality goals.  Sure management expects the data to be good, but they rarely include that outcome in a professional review of developers.

But in agile/Scrum projects there are some special considerations that tend to increase conflicts between modeler goals and developer goals.

Most people have been taught data models are documentation

One of the dysfunctions that I help correct on projects is the concept that data models are something that just describe someone else's work.  Sure, we have reverse engineering features in ER/Studio to do that, but that's not why we build data models.  Logical and Physical data models are created as a product of having preformed analysis and come to a decision of what something should be an or how it should work.  I can generate documentation of the data model using ER/Studio, but that documentation is not the model.

Strike the word documentation from your vocabulary when you talk about data models.  

Most people understand data models as ONLY mechanisms to generate DDL

Closely related to the former point, many people believe that the only reason to produce a data model is to build a database.  Sure, that's often my goal, but there are typically tens of thousands of other pieces of information in the model that never end up in the database.  Definitions to record what we meant or why we did something in the model.  Privacy and security requirements for communicating to the developers.  Data lineage information to show data provenance.  We have all kinds of data wealth in those models. 

You must provide access to all that data wealth in your models to get others to see that value.

Most data modelers are stuck to traditional development methods — overly stuck to them

It's not just developers who contribute to conflict on agile projects.  Too many data modelers have weak or non-existent skills in producing physical models and DDL. Too many data modelers have not practiced being iterative in producing physical artifacts.  Too many are stuck in a mindset that we do all the logical model, then do some physical models, then generate some DLL and are done.  Many projects do work that way.  Agile ones do not.  Modern data modelers need to be able to go from requirement to model to DDL in just days, not months.  I can be done. Really. 

Data modelers stuck in 1980s methods of software development won't be valuable to modern development projects.

Most people think software is the most important, most complex part of IT

The Agile Manifesto even reflects this belief in its principles. I think it is misguided.  All the parts are important and, while I may be biased, I believe that data is just important to get right.  Data last longer than code. Failing on data design can cost a company revenues, credibility and fines.  Remember, keeping your CIO out of jail is a good thing. 

Data modeling often involves not just building a database, but sourcing input data, determining cleansing and transformation needs, understanding external data, and…well, there's a lot more than just drawing boxes and lines.

Data models and the efforts to complete them are complex. If you want them to be simple, go out and make the world simple and come back.

Most people think data models are boxes and lines

They think this because data modelers have this odd habit of just showing people ERDs and calling that "the data model" . A diagram is just a tiny portion of a high quality data model.  If you give everyone access to ALL the data model, they will have a better understanding of how this works.  Especially if they can query the models without having access to a modeling tool.  Team Server Core, as an enterprise portal to data models and glossaries, is the perfect way to provide this access. 

An ERD is not a data model.  It's a diagram of a tiny portion of a data model.

Most people have never seen productive, iterative, responsive, flexible model-driven development

This, data modelers, is your stretch goal. Go out there and show them. You have the tools, now you just need the will and some practice. I promise if you work towards fixing the issues above, you will see less conflict on your agile projects.