The RDBMS is Dead

by Mar 14, 2018

Categories

Tags

Administration agent-based monitoring Agentless Monitoring alert responses alert thresholds alerting Alerts Amazon Aurora Amazon EC2 Amazon RDS Amazon RDS / Aurora Amazon RDS for SQL Server Amazon Redshift Amazon S3 Amazon Web Services (AWS) Analytics application monitoring Aqua Data Studio automation availability Azure Azure SQL Database azure sql managed instance Azure VM backup Backup and recovery backup and restore backup compression backup status Backup Strategy backups big data Blocking bug fixes business architecture business data objects business intelligence business process modeling business process models capacity planning change management cloud cloud database cloud database monitoring cloud infrastructure cloud migration cloud providers Cloud Readiness Cloud Services cloud storage cloud virtual machine cloud VM clusters code completion collaboration compliance compliance audit compliance audits compliance manager compliance reporting conference configuration connect to database cpu Cross Platform custom counters Custom Views customer survey customer testimonials Dark Theme dashboards data analysis Data Analytics data architect data architecture data breaches Data Collector data governance data lakes data lineage data management data model data modeler data modeling data models data privacy data protection data security data security measures data sources data visualization data warehouse database database administration database administrator database automation database backup database backups database capacity database changes database community database connection database design database developer database developers database development database diversity Database Engine Tuning Advisor database fragmentation database GUI database IDE database indexes database inventory management database locks database management database migration database monitoring database navigation database optimization database performance Database Permissions database platforms database profiling database queries database recovery database replication database restore database schema database security database support database synchronization database tools database transactions database tuning database-as-a-service databases DB Change Manager DB Optimizer DB PowerStudio DB2 DBA DBaaS DBArtisan dBase DBMS DDL Debugging defragmentation Demo diagnostic manager diagnostics dimensional modeling disaster recovery Download drills embedded database Encryption End-user Experience entity-relationship model ER/Studio ER/Studio Data Architect ER/Studio Enterprise Team Edition events execution plans free tools galera cluster GDPR Getting Started Git GitHub Google Cloud Hadoop Healthcare high availability HIPAA Hive hybrid clouds Hyper-V IDERA IDERA ACE Index Analyzer index optimization infrastructure as a service (IaaS) infrastructure monitoring installation Integrated Development Environment interbase Inventory Manager IT infrastructure Java JD Edwards JSON licensing load test load testing logical data model macOS macros managed cloud database managed cloud databases MariaDB memory memorystorage memoryusage metadata metric baselines metric thresholds Microsoft Azure Microsoft Azure SQL Database Microsoft PowerShell Microsoft SQL Server Microsoft Windows MongoDB monitoring Monitoring Tools Monyog multiple platforms MySQL news newsletter NoSQL Notifications odbc optimization Oracle PeopleSoft performance Performance Dashboards performance metrics performance monitoring performance schema performance tuning personally identifiable information physical data model Platform platform as a service (PaaS) PostgreSQL Precise Precise for Databases Precise for Oracle Precise for SQL Server Precise Management Database (PMDB) product updates Project Migration public clouds Query Analyzer query builder query monitor query optimization query performance Query Store query tool query tuning query-level waits Rapid SQL rdbms real time monitoring Real User Monitoring recovery regulations relational databases Releases Reporting Reports repository Restore reverse engineering Roadmap sample SAP Scalability Security Policy Security Practices server monitoring Server performance server-level waits Service Level Agreement SkySQL slow query SNMP snowflake source control SQL SQL Admin Toolset SQL CM SQL code SQL coding SQL Compliance Manager SQL Defrag Manager sql development SQL Diagnostic Manager SQL Diagnostic Manager for MySQL SQL Diagnostic Manager for SQL Server SQL Diagnostic Manager Pro SQL DM SQL Doctor SQL Enterprise Job Manager SQl IM SQL Inventory Manager SQL Management Suite SQL Monitoring SQL Performance SQL Quality SQL query SQL Query Tuner SQL Safe Backup SQL script SQL Secure SQL Security Suite SQL Server sql server alert SQL Server Migration SQL Server Performance SQL Server Recommendations SQL Server Security SQL statement history SQL tuning SQL Virtual Database sqlmemory sqlserver SQLyog Storage Storage Performance structured data Subversion Support tempdb tempdb data temporal data Tips and Tricks troubleshooting universal data models universal mapping unstructured data Uptime Infrastructure Monitor user experience user permissions Virtual Machine (VM) web services webinar What-if analysis WindowsPowerShell

 If you suspected the title of this blog is click-bait, then you would be correct.  But bear with me anyway on a little retrospection of the relational database market, where it has been, and where I think it is going to end.

What is an RDBMS?

Anyone reading this blog surely knows the definition already, but let us clarify what relational data management systems mean in contrast to all the data management systems.

Relational database refers to the organization of datasets into discrete tables that are related to each other by common data fields.  This evolved from efforts to improve performance from earlier flat data management systems that took a long time to search.  Modern RDMS vendors really took off after ACID took hold as the governing principle of RDBMS development. Andreas Reuter and Theo Härder coined the acronym ACID as shorthand for Atomicity, Consistency, Isolation, and Durability in 1983.  Oracle, Sybase, and Informix all trace their earliest success to that period as the original RDBMS market leaders.

The ACID standard was key because it made for reliable data transactions that could be the basis of business applications requiring both speed and data reliability and integrity.  Durable transactions was the key ingredient that enabled ERP applications built on these enterprise-level databases.

So, why would anyone entertain the thought of even considering whether the relational database management system is dead given that success?

Hype and Reality… and the Reality of Hype

I began thinking of this subject a couple of years ago when a colleague in a group meeting nonchalantly declared that none of the hip voices in application development were talking about databases anymore (at least not the traditional variety).  No, the new hotness was around NoSQL, vastly scaling architectures that didn’t impose the tiresome restrictions on data stores for developers.   There was lots of chatter and excitement about this technology that were growing as people experimented and rolled out a different types of applications made for this purpose.

It was hard to contradict this exuberance for the obvious growing popularity among technology thought leaders without seeming to be a curmudgeon.  And his expressed opinion came from the chatter among developers eager to be at the forefront of the next technology explosion. There was definitely something there, but how would it play out and when would it play out?

Gartner coined the concept of the technology hype cycle from the repeated patterns of introductions of any revolutionary technology.  And the central idea is that the hype on these new concepts always gets ahead of productive usage of the technology.  The buzz is there, usually spawning start-ups galore in a technology gold rush to be the first to monetize that promise and ride a wave of popularity to riches.

Of course, the cycle part of the hype cycle implies that though the excitement may persist, the efforts to monetize the new thing can fail to meet inflated expectations.  The trough of disillusionment comes from failure to make the revolution a reality as fast as hoped.  There is usually a shake-out of the start-ups as the market doesn’t grow fast enough to sustain the initial enthusiasm. 

If the technology in question isn’t overtaken by something better to replace it, and if investors can weather that trough, then we gradually learn productive ways to make the technology cost effective.  Only then does the technology hit the plateau of productivity which implies it will spread and have a lasting effect.

NoSQL to NewSQL to Hybrid to “Nontraditional”

Although the NoSQL term was first coined by Carlos Strozzi in 1998 for his open-source relational database, the NoSQL databases that gained such popularity after 2009 were different.  Where Strozzi NoSQL merely excluded the Structured Query Language interface, most databases called NoSQL in 2009 to present depart from the relational model completely.  (Strozzi suggested they would more properly be called NoREL for ‘No Relational’ but the NoSQL name was already in common usage.)

The excitement over these NoSQL databases came from how they overcame scale limitations of traditional RDBMS, enabling applications that could also scale and without the restrictions of a fixed schema.  It enabled frequent changes to the databases adaptive to changes in the calling applications.  The downside was transaction integrity wasn’t as reliable and applications had to be written with a particular data store in mind with no standard interface or language.

NewSQL was coined for the class of RDBMS seeking to get the scalable performance of NoSQL but maintaining the ACID transaction standards and SQL interface of traditional RDBMS.  Soon NewSQL and NoSQL were being lumped together in market discussion with columnar databases, in-memory databases, and termed as Hybrid databases or currently “Nontraditional Databases”.  That term covers a wide variety of specific-purpose data stores uniquely suited to solve various problems where traditional RDBMS didn’t work as well.

So, while the hype and buzz of these new options to traditional RDBMS is not to be denied, it is getting hard to identify a unifying and superior aspect to them like RDBMS had with the ACID standard.

Familiar Patterns

I am not sure of the original source, but I once read that one sign that you are getting old is that everyone starts to look familiar. And if you have been embedded in particular markets long enough then you will start to recognize other familiar patterns.

History doesn’t repeat itself; but it rhymes.

Back in the day when the “Big Three” in the RDBMS world referred to Oracle, Sybase, and Informix there was a time when OODBMS (Object-oriented Database Management Systems) was the new hotness and all the rage in discussion groups.  IBM held a legacy position because of mainframe DBMS and Microsoft was an upstart selling rebranded Sybase SQL Server to the entry market.  Both IDC and Gartner covered OODBMS as a new category and the chatter in some discussion groups was whether OODBMS would completely overtake traditional RDBMS with the advantages it had for object oriented application development.

This chart outlines the evolution of database systems through the last decade.  But it doesn’t cover the individual vendors’ success or failure over this time.

The yearly major analyst reports for DBMS tracked the rapid growth of Microsoft SQL Server as it reshaped SQL Server into its own vision of a Microsoft-centric IT back office.  Sybase and Informix marketing failures and missteps took them out of the leadership group with Oracle and the Big Three became Oracle, Microsoft, and IBM; leaving Sybase and Informix with large installed bases and particular vertical markets but little growth.

As these DB market shifts happened, due more to marketing successes or failures than technical superiority, the leaders adopted OODBMS features into the traditional RDBMS offerings so they could easily accommodate new demands for data types.  The pure OODBMS offerings fell back into a few niche offerings for special applications.

Since that time, IBM bought Informix to consolidate further and SAP bought Sybase, primarily in a defense of Oracle aggressively pursuing SAP ERP customers. SAP ASE is still a solid technical offering as a back end for SAP applications, but unlikely to recapture a leading independent RDBMS market position again.

Meanwhile, back at the ranch…

Let’s try to put some quantitative perspective to the ‘new hotness’ hype I mentioned earlier about all the popularity around new data store offerings like NoSQL.  It is tough to measure “hype”, but DB-Engines does as good a job as any I have found.   DB-Engines Ranking

This recent graph from the DB-Engines site shows the trend of scores of all sorts of data store offerings.  Rather than attempt to measure revenue results or analyst opinions, it relies on scoring from publicly available information, searches, job offers, mentions on websites, etc.  It doesn’t measure concrete factors such as installations or revenue and therefore positions the rank as an early indicator of future success.  (More detail on how the score is calculated and corrected for comparison is on the DB-Engines website.)

Current DB Market Landscape

Interpreting the chart and the trend of “popularity” is revealing.  Three traditional RDBMS engines are flat, unchanging, and boring.  Oracle, Microsoft SQL Server, and MySQL don’t show much change in popularity over the past few years.  But they all are clustered together far above all other entries.  I suspect if we did find a way to incorporate revenue or actual IT installations that this separation would be even more pronounced.

But the growth in the trends indeed comes from NoSQL offerings like MongoDB, Cassandra, etc.… 

Meanwhile, back at the ranch …Part Deux

So what do the traditional market analysts think about the future of RDBMS vendors?  I must admit that I had not followed them as closely as the year-to-year reports became very similar with Microsoft and Oracle solidifying positions as the clear gorillas in the database market.  They use that leadership position to probe adjacent markets in applications and infrastructure.

A few years ago, Gartner even stopped reporting on the “RDBMS market” in favor of a category called “Operational Database Management Systems”.  This new market category encompassed both traditional RDBMS offerings and all the non-traditional DBMS that had enjoyed the increase popularity in technical discussions.

The Gartner Magic Quadrant shows how Gartner ranks all these vendors relative to one another based on not only short term results but also vision and strategy.

The first thing that sticks out in this Magic Quadrant ranking is that the old RDBMS market leaders are also the Operational Database market leaders.  As would be expected, they rank above many of the newer offerings in ability to execute.   But what is striking is both are also far ahead of the new contenders in completeness of vision as well.  Now this only represents one analyst company’s consensus opinion, but they clearly don’t see any in the Challenger group being an immediate threat to the big RDBMS vendors, why?

Summary and Prediction

The answer to my question is only my own opinion, but my guess goes to echoes of what happened with OODBMS and RDBMS.  The big vendors gradually found ways to incorporate the best parts of the OODBMS while maintaining the stability of the RDBMS for businesses.  While niche players persist there for special purposes; there are few applications that cannot be supported by one of the big RDBMS offerings.

I predict Microsoft and Oracle will continue to mitigate the RDBMS limitations by adding features that were advantages of NoSQL and NewSQL contenders, just enough to make the other transactional advantages of traditional RDBMS to keep the majority of market revenue.  Acquisitions and consolidation is inevitable and these two leaders are the most likely to consolidate power that way as well.

 As the distinction between different DBMS offerings are blurred, the old leaders will keep that leadership until they make major mistakes.

Long Live the Operational Database

Pardon any skepticism about all the new developments in NoSQL (NoREL), NewSQL, etc.   It isn’t that I don’t believe they are important developments that enable capability beyond what we had.  But I doubt we will see new DBMS market leaders coming purely from technical superiority.  The RDBMS leaders are Operational Database market leaders.  And I don’t see evidence of that changing anytime soon.

The RDBMS is not so much dead as rebranded.

Assimilation has begun.