Metadata Integration

The operational culture of the enterprise has gradually shifted towards a more business-centric model, one where business users and front line information consumers desire better access to enterprise data assets, and need to have a comprehensive view of the organization. This means that data, as well as the metadata that describes it, must be shared between IT and business users. This is often easier said than done. The main reason for sharing data is to provide insights to enable better decision-making within an organization.

The importance of integrating metadata cannot be overemphasized. In many ways, it is the foundation of an enterprise’s data warehouse and information management.

The term “metadata” arises frequently when discussing Data Architecture.  Metadata defines the structure of data in files and databases. In short, it is “data about data.”  However, interpretation can be rather ambiguous, since it is used in different contexts.  The most common context is known as structural metadata, which describes the design and specifications of data structures such as tables, columns, constraints and indexes.  However, this is evolving to incorporate unstructured data as well.

Another interpretation is descriptive metadata (also known as meta-content), which describes the content of specific instances of data.  For example, who created it and why, when it was created, and where it was created.  An example of this is the cataloging information that is captured with photographs using modern digital cameras: date, time, exposure settings, GPS coordinates, etc.  This adds context to the unstructured data object.

Metadata is typically stored in a repository, which is usually a database structure itself.  A variety of modeling, data integration, business intelligence and service-oriented architecture tools utilize metadata repositories, which are sometimes called registries.

Metadata needs to be organized and managed if it is to be leveraged effectively. Ideally, an enterprise data architect needs a tool that supports both top-down and bottom-up metadata creation, cataloging and sharing for the majority of industry-leading database platforms.  Top-down allows data models to be designed, elaborated and created as physical database specifications, facilitating collaboration at every step. Metadata includes, but is not limited to, tables, columns, relationships, permissions, definitions, notes, security information and data lineage.  Logical and physical layers are supported.

Bottom-up metadata creation is enabled through the powerful reverse engineering capabilities for most database platforms, as well as the ability to transform metadata to/from many data integration, business intelligence, big data and other 3^rd party platforms.  The metadata can be easily extended to support virtually limitless characteristics through the use of the built-in attachments functionality. 

To further enhance the sharing of metadata, the right tool will add powerful collaboration capabilities to metadata management. One way to address this is to create and maintain a searchable, extensible metadata registry that allows for data enrichment and facilitates sharing. The idea is to establish a single reference point that includes all enterprise data sources, internal and external, and allow users to add business information to the metadata.

Essentially, those who use the data become responsible for providing useful business definitions of what the data is and does. All stakeholders, ranging from technical staff and business analysts through business subject matter experts and data stewards, are able to work as a team, with views of the metadata that are appropriate to specific roles. 

The data and metadata are centered on an enterprise glossary of business definitions and data elements. This glossary is integrated with the data management tools, allowing users across the enterprise to access a single searchable registry of business definitions and data sources with each term that is linked to the metadata.

This social collaboration paradigm allows users to have meaningful discussions regarding specific areas of interest, with a full audit trail.  Thus, it is not only the decisions recorded, but also the process that led to those decisions.  This capability allows all participants to reach a much higher level of knowledge and understanding.  This is critical for both data creation and data consumption.

Embarcadero Technologies offers a metadata collaboration platform that delivers on all fronts and enables users to make better use of the data available to them. ER/Studio Team Server addresses the problem of disparate data sources and definitions with the shared registry and glossary. When combined with Embarcadero’s market-leading data modeling tool, ER/Studio Data Architect, it becomes a data enrichment platform which delivers an enterprise-wide solution for managing data and metadata. Contact us to learn more about ER/Studio.