Back to AI Research

AI Research

Ontology-supported AI Model and Dataset Management | AI Research

Key Takeaways

  • The paper "Ontology-supported AI Model and Dataset Management" addresses the challenge of effectively exchanging and managing AI models and datasets in indus...
  • Recently, there has been a great deal of research into improving AI methods and their application.
  • The main focus is on tracking progress, enabling transparent comparisons, and fostering a more profound understanding of AI.
  • In that process, different organizations generate and use plenty of assets that need to be tracked, traced and managed.
  • Moreover, it is important to discover assets relevant for the task at hand.
Paper AbstractExpand

Recently, there has been a great deal of research into improving AI methods and their application. The main focus is on tracking progress, enabling transparent comparisons, and fostering a more profound understanding of AI. In that process, different organizations generate and use plenty of assets that need to be tracked, traced and managed. Moreover, it is important to discover assets relevant for the task at hand. This paper presents research aiming to contribute to answering the question of what is required to exchange and manage AI models and related assets effectively without semantic gaps in an industrial context. We introduce a platform for AI model exchange, which facilitates the usage, exchange, and analysis of AI models and datasets. The platform incorporates an ontology that can foster a more profound common understanding of what is required in these tasks and help tackle the issues mentioned above. Finally, we elucidate the utility of the platform through the illustration of a use case in the context of real-time critical systems.

The paper "Ontology-supported AI Model and Dataset Management" addresses the challenge of effectively exchanging and managing AI models and datasets in industrial environments. The authors propose a platform called AIMDEP (AI Model and Dataset Exchange Platform) that uses a specialized ontology, AIMDEO (AI Model and Dataset Exchange Ontology), to provide structured, semantically rich metadata for AI assets, aiming to bridge the gap between model creators and end users.

The Problem of Semantic Gaps

In industrial settings, such as the automotive supply chain, AI models are often developed by one team and used by another. Current platforms like MLflow, H2O, and Ray allow for metadata, but these specifications lack interoperability. When organizations exchange models, they often rely on semi-structured natural language descriptions, which are difficult to parse and lack explicit, machine-readable semantics. This creates "semantic gaps" that hinder the discovery, trust, and collaborative use of AI assets.

How the Platform Works

The AIMDEP platform functions as a central registry for AI models and datasets. It employs a client-server architecture using the Django framework and a REST API for interaction. The process involves:

  • Registration: Users upload assets, and the platform attempts to identify the necessary data handlers or machine learning frameworks (e.g., Scikit-Learn, TensorFlow, PyTorch).

  • Semantic Annotation: Using the AIMDEO ontology, users define input/output features, configuration parameters, and quality metrics. This metadata is stored in a database and can be exported as micro-ontologies.

  • Operations: The platform provides a semantic search function to help users find relevant assets. It also includes built-in tools for interactive data visualization (via Plotly) and online model deployment (via MLEM), allowing users to test models without setting up local runtime environments.

The Role of AIMDEO

The AIMDEO ontology is an OWL-based framework designed to standardize the description of machine learning assets. It captures essential concepts such as provenance, task/sub-task definitions, model parameters, and evaluation metrics. By providing a common vocabulary, the ontology ensures that when a model is shared, the recipient understands its functional scope, the data it was trained on, and its performance characteristics.

Evaluation and Limitations

The authors evaluated the platform through a use case involving the prediction of memory access time for safety-critical embedded systems. In this scenario, a hardware developer registered a dataset with specific memory configuration metadata, which an AI expert then used to develop and annotate a model. A software developer was subsequently able to search for and evaluate the model's suitability for their specific timing analysis needs.
Franklin analysis: The paper notes that while the platform and ontology are effective for collaborative development, they are not currently integrated with other existing standards like EMMM. The authors suggest that future work could focus on creating compatibility with EMMM and further aligning AIMDEO with other ontologies like the ITO to improve broader interoperability.

Comments (0)

No comments yet

Be the first to share your thoughts!