Philip Miller, Senior Customer Success Manager, Progress, explains how metadata-driven insights enhance collaboration and business consistency within organisations. He explores how data harmonisation is achieved through new AI technology in the form of Semantic AI platforms, bridging the gap between taxonomists/ontologists, data architects and developers or across wider business departments by providing a common vocabulary.
While most organisations have realised that they need to embrace technology to be utilising their data to its full potential, the metadata evolution has moved the goalposts to a new level. Not only data, but more specifically data about data, known as ‘active metadata’, now plays a primary role in enterprise data management and it’s a critical enabler for Digital Transformation. According to IDC figures, 80% of data within an organisation will be unstructured by 2025 so harnessing this will be a priority for tech leaders.
This evolved type of metadata powers some of the most innovative data architectures, from data hubs, data fabrics and digital twins to semantic knowledge graphs, and already enables powerful new business use cases across industries. Yet CIOs may not know where to go about transforming their data into knowledge for real business value. The potential paybacks include preventing revenue leakage, minimising risk and accelerating growth.
CIOs and data leaders must first achieve data harmonisation, which means offering a common language to all data users to ensure that all metadata can be universally understood and used. In understanding and taking advantage of this valuable metadata, CIOs will facilitate more effective teamwork and informed decision-making. This is now possible through using new AI technology in the form of a Semantic AI platform, which can provide a common metadata vocabulary for data users, from taxonomists to ontologists, data architects and developers within large enterprises; while at the same time bringing this metadata closer to the business experts driving innovation forward, faster.
The evolution of metadata management
The practice of processing all this metadata – metadata management – has evolved, just as the type of metadata itself has changed. To most data architects, metadata management means defining and leveraging information about the data itself to achieve better data quality, governance, unity, reliability and security. However, in 2021 Gartner’s Market Guide for Active Metadata Management pronounced traditional metadata practices as ‘insufficient’ to meet the needs of the enterprise, due to the transformational capabilities of active metadata.
What determines metadata to be ‘active’ or passive can change as it moves through the information supply chain. Essentially, passive metadata is information about data – system-applied dates, creator, source semantic metadata. It describes the meaning of data topics, product, geography, audience, concepts and relationships. Active and augmented metadata is intelligent and dynamic data, for instance facts, status, personal identifiable information (PII), data orchestration and ML analytics.
To date, approaches to take advantage of active metadata have included deploying Machine Learning (ML) and/or business-oriented AI algorithms to automate metadata management through an AI/ML-powered data platform which triggers automated actions or proactive recommendations for more informed decisions. However, this approach is no longer enough. Separate layers suggest custom integration between different vendor products which carries a consequential risk of versioning, compatibility and loss of capability.
Metadata should now be addressable as data from a single data platform without compromise. This means only optimal metadata management can eliminate the effect of data and knowledge silos, deliver data agility, enhance collaboration and accelerate insightful business decision-making. For example, at a life sciences company, standardised metadata fields for clinical trials (e.g. agent dose, agent administered time, disease recurrence type, etc.) can help ensure that data is consistent, shareable and compliant with regulations. This gives pharmaceutical team members greater transparency into data, facilitating better communication, deeper insights and faster results.
How Semantic AI can standardise data interpretation
To embrace the metadata evolution and ensure that their data is universally understood and used, organisations must create data harmonisation and offer a common language to all data users, which will facilitate effective teamwork and informed decision-making. This data harmonisation is achieved through new AI technology in the form of Semantic AI platforms.
A Semantic AI data platform unifies data with its metadata to establish a single data resource. It creates and manages active metadata, allowing data leaders to integrate, store, manage, govern, contextualise and surface data regardless of format, schema, or type. It also employs Semantic AI to synthesise, enrich, extract and harmonise all types of metadata.
Most importantly, source systems cross-reference each other to avoid data silos with no contextual information. A fully linked, harmonised and query-able system provides full transparency and optimal data quality to generate meaningful business decisions.
Business benefits of a metadata-centric approach
There are several key areas that can fundamentally be transformed by how organisations use their data:
- Making proactive decisions is the first, with a good example being in manufacturing organisations. Inproactively evaluating data and using it to forecast and mitigate maintenance problems before they arise, business leaders can yield dramatic costs and efficiency gains. Where product shutdowns of highly temperamental CPU fabrication machinery cause production delays and force a manufacturer to scrap batches of products due to purity standards. Using data, engineers could predict when parts would fail and when cleaning would be needed. By leveraging their data and embracing proactive maintenance, a company such as this can avoid ditching batches of products and reap massive cost savings.
- Businesses can also innovate new revenue streams, for instance where products that are essentially loss-leaders. Minimising costs for customer support on loss-leaders is paramount as every minute engaging with customers over simple issues loses the company money – enabling customer self-service is essential.Using a knowledge graph that taps their documentation, knowledge base and support data, an organisation can use Semantic AI to create a data-powered web application where customers explain their problems using natural language akin to Generative AI or even helping in its use. With a simple troubleshooting guide to remedy their issue, customers are empowered to quickly solve their own problem and reduce service costs.
- Scaling for data agility is a key driver for many fast-growing businesses.However, many critical business systems use mainframes, which are very expensive because of the hardware, software application and maintenance implications. For instance, during its open enrolment period, a healthcare organisation may have hundreds of requests per minute. To scale the mainframe for these peak periods would potentially cost hundreds of millions, but it may sit idle the rest of the year being expensive to maintain. An operational data hub can abstract the enterprise systems and deliver high scalability and easier access to simpler data – outside of peak times, it can be scaled down, offering both power and agility.
- It can support web content management, where in the case of an organisation needing to manage hundreds of websites in dozens of languages, a highly disciplined approach to organising metadata becomes a business imperative. In fact, tagging is one area which can be a continual pain point for anyone using web content management systems. Authors often add tags without giving consideration to whether they are compatible or meaningful or create a metadata mess by applying divergent tags to analogous content. By using a Semantic AI platform to apply some metadata detail, this delivers a content database that makes sense, enabling future-facing solutions such as smart content management and authoring solutions. Scaling this scenario beyond a single web instance to thousands makes semantic metadata capabilities hard to overlook.
Metadata-driven insights are the future for business change
Metadata-driven insights will soon be table stakes for CIOs to make effective transformations and directly enhance collaboration and business consistency. By putting metadata at the centre of their data strategy, CIOs and data leaders can truly achieve data agility by connecting active data, active metadata and active meaning; keeping data and all associated detail together at all times is essential.
By making thoughtful use of the data they already use and store, CIOs will continue to find new and novel applications for business data utility that will power their people, processes and technology.
Click below to share this article