From Document Silos to Process Intelligence: A Mult... | AI Research

From Document Silos to Process Intelligence: A Multi-Layer Knowledge Graph for CMC Process Development
Chemistry, Manufacturing, and Controls (CMC) process development involves a vast, complex flow of technical information from drug discovery to commercial manufacturing. This data is often trapped in fragmented, heterogeneous documents, leading to high management costs and difficulties in tracking information during regulatory filings. This paper introduces a modular, agentic-AI platform designed to transform these disconnected document silos into a structured, queryable, dual-layer knowledge graph. By combining a lexical layer for precise document retrieval with an intelligence layer for cross-project reasoning, the system aims to preserve institutional knowledge and streamline process development. The same large language models question is explored in Geospatial AI, Dataverse Metadata, and the..., which adds a research perspective.

A Dual-Layer Architecture

The platform organizes information into two distinct but connected layers. The "knowledge layer" acts as a lexical graph, ingesting digital, scanned, and handwritten documents into a hierarchical structure of documents, sections, and chunks. This layer is ideal for locating specific facts or keywords within a single document. The "intelligence layer" sits on top, using an ontology-driven domain graph to identify real-world concepts—such as chemical substances or unit operations—across different documents and projects. By anchoring these domain entities back to their original source chunks, the system ensures that every insight can be traced back to the evidence, a critical requirement for regulated pharmaceutical environments.

Agentic Reasoning and Retrieval

To navigate this structure, the platform employs specialized LLM agents that act as routers. Depending on the nature of a user's question, the system selects the most appropriate retrieval path. For simple, fact-based queries, the agent utilizes the lexical graph to perform precise lookups. For more complex, corpus-wide questions—such as investigating the source of an impurity across multiple years of reports—the agent shifts to the domain graph. This modular approach allows the system to handle both routine document searches and high-level, cross-project analysis. The ai search story also surfaces in Qwen Developers Open-Source Local-First Search Layer..., adding another angle.

Benchmarking Reliability

The authors developed a novel three-tier protocol to measure the system's "deployment-fidelity," or how well it performs on proprietary, non-public data. Using 505 questions curated from 38 development reports, the team evaluated the platform through multiple-choice accuracy (Tier 1), LLM-judge scoring of free-form answers (Tier 2), and subject-matter-expert review (Tier 3). While the system achieved a 95% accuracy rate in Tier 1, the stricter Tier 2 pass rate of 85% revealed that the system struggled more with comparative and corpus-wide questions. This failure taxonomy highlights the importance of rigorous, multi-tiered testing for AI systems in professional settings.

Future Implications

The study demonstrates that a hybrid, graph-based architecture can address a wider spectrum of questions than either lexical or domain-based systems could alone. By providing a framework for benchmarking and a method for connecting fragmented data, the authors suggest that this approach can help pharmaceutical companies reduce the hidden costs of knowledge management. The ability to maintain provenance—linking every conclusion to its source—is presented as a vital step toward adopting AI in GxP-regulated environments, where traceability and auditability are essential. The ai search story also surfaces in New AI Architecture Mimics the Human..., adding another angle. as detailed in the full paper on Arxiv