FastOMOP: A Foundational Architecture for Reliable...

FastOMOP: A Foundational Architecture for Reliable... | AI Research

Key Takeaways

FastOMOP: A Foundational Architecture for Reliable Agentic Real-World Evidence Generation on OMOP CDM data Generating real-world evidence (RWE) from massive...
Yet generating real-world evidence (RWE) from these repositories remains a manual process requiring clinical, epidemiological and technical expertise.
No infrastructure exists to ensure agentic RWE generation is flexible, safe and auditable across the lifecycle.
We introduce FastOMOP, an open-source multi-agent architecture that addresses this gap by separating three infrastructure layers, governance, observability and orchestration, from pluggable agent-teams.
Governance is enforced at the process boundary through deterministic validation independent of agent reasoning, ensuring no compromised or hallucinating agent can bypass safety controls.

Paper AbstractExpand

The Observational Medical Outcomes Partnership Common Data Model (OMOP CDM), maintained by the Observational Health Data Sciences and Informatics (OHDSI) collaboration, enabled the harmonisation of electronic health records data of nearly one billion patients in 83 countries. Yet generating real-world evidence (RWE) from these repositories remains a manual process requiring clinical, epidemiological and technical expertise. LLMs and multi-agent systems have shown promise for clinical tasks, but RWE automation exposes a fundamental challenge: agentic systems introduce emergent behaviours, coordination failures and safety risks that existing approaches fail to govern. No infrastructure exists to ensure agentic RWE generation is flexible, safe and auditable across the lifecycle. We introduce FastOMOP, an open-source multi-agent architecture that addresses this gap by separating three infrastructure layers, governance, observability and orchestration, from pluggable agent-teams. Governance is enforced at the process boundary through deterministic validation independent of agent reasoning, ensuring no compromised or hallucinating agent can bypass safety controls. Agent teams for phenotyping, study design and statistical analysis inherit these guarantees through controlled tool exposure. We validated FastOMOP using a natural-language-to-SQL agent team across three OMOP CDM datasets: synthetic data from Synthea, MIMIC-IV and a real-world NHS dataset from Lancashire Teaching Hospitals (IDRIL). FastOMOP achieved reliability scores of 0.84-0.94 with perfect adversarial and out-of-scope block rates, demonstrating process-boundary governance delivers safety guarantees independent of model choice. These results indicate that the reliability gap in RWE deployment is architectural rather than model capability, and establish FastOMOP as a governed architecture for progressive RWE automation.

FastOMOP: A Foundational Architecture for Reliable Agentic Real-World Evidence Generation on OMOP CDM data
Generating real-world evidence (RWE) from massive electronic health record databases is currently a slow, manual process that requires deep clinical and technical expertise. While AI and large language models (LLMs) offer a way to automate these tasks, they are often unreliable, prone to errors, and difficult to govern. This paper introduces FastOMOP, an open-source, multi-agent architecture designed to make RWE generation safer, more transparent, and more reliable by embedding governance directly into the system's infrastructure rather than relying on the AI model alone.

A Layered Approach to Safety

FastOMOP addresses the risks of autonomous AI by separating the system into three independent layers: orchestration, governance, and observability. The orchestration layer manages how agents work together to complete complex tasks, while the observability layer creates a permanent, detailed record of every step taken during the process. The most critical component is the governance layer, which acts as a "safety gate" at the boundary of the system. Because this layer uses deterministic, rule-based validation that operates independently of the AI’s reasoning, it ensures that even if an agent hallucinates or is compromised, it cannot execute unsafe or unauthorized database queries.

Pluggable Agent Teams

The architecture is designed to be flexible, allowing specialized "agent teams" to be plugged into the system to handle specific parts of the RWE lifecycle, such as defining patient groups (phenotyping), designing studies, or performing statistical analysis. These agents inherit the system’s safety and auditability features automatically. By using the Model Context Protocol, the system ensures that agents only have access to the specific tools and data they need to perform their assigned tasks, following the principle of least privilege to minimize potential damage from any single agent failure.

Proven Reliability in Clinical Settings

To test the architecture, the researchers built a proof-of-concept system capable of converting natural language requests into SQL queries for OMOP-standardized databases. They validated this system across three different datasets, including synthetic data, the MIMIC-IV database, and a real-world dataset from the Lancashire Teaching Hospitals NHS Foundation Trust. The results showed high reliability scores (ranging from 0.84 to 0.94). Most importantly, the system achieved perfect scores in blocking both adversarial attacks and out-of-scope queries, proving that the architectural governance successfully prevents the system from performing unauthorized or irrelevant actions.

Shifting the Focus to Architecture

The findings suggest that the current "reliability gap" in clinical AI is not necessarily a failure of the AI models themselves, but rather a lack of proper architectural infrastructure. By moving safety controls out of the agent’s internal logic and into the process boundary, FastOMOP provides a way to deploy AI in high-stakes clinical environments where transparency and regulatory compliance are essential. This approach allows for the progressive automation of the RWE lifecycle while maintaining strict control over how data is accessed and analyzed.