Back to AI Research

AI Research

Tracing the Heart: An Evidence-Linked Pipeline for... | AI Research

Key Takeaways

  • The Nimblemind Multi-Agent System (nMAS) provides a method for automating the transformation of fragmented electronic health record (EHR) data into structure...
  • Electronic health record (EHR) feature engineering is a major bottleneck in clinical research and AI, accounting for 39-45% of data scientists' workload.
  • This is especially pronounced in heart failure, which affects an estimated 6.7 million U.S.
  • adults and requires integrating fragmented EHR data with disease-specific, guideline-based clinical reasoning.
  • Existing rule-based and large language model (LLM)-based approaches offer only partial automation with limited maintainability and evidence traceability.
Paper AbstractExpand

Electronic health record (EHR) feature engineering is a major bottleneck in clinical research and AI, accounting for 39-45% of data scientists' workload. This is especially pronounced in heart failure, which affects an estimated 6.7 million U.S. adults and requires integrating fragmented EHR data with disease-specific, guideline-based clinical reasoning. Existing rule-based and large language model (LLM)-based approaches offer only partial automation with limited maintainability and evidence traceability. We developed the Nimblemind Multi-Agent System (nMAS), an evidence-linked, rubric-grounded pipeline for automated heart-failure feature engineering, and evaluated it on 500 dummy patient records from nine EHR source tables. nMAS generated 132 structured and 70 rubric-scored aggregated features, verified for structural integrity, rubric compliance, and provenance, and audited by a restricted LLM. Adding the aggregated features improved held-out AUROC from 0.895 to 0.963 for HFrEF and 0.870 to 0.910 for HFpEF phenotyping, and an independent LLM-based rubric assessment of evidence support and methodological soundness scored the features at 81.5% of maximum points. These results demonstrate the feasibility of automated, auditable feature engineering for complex cardiovascular EHR data, though evaluation was limited to a single-institution cohort and external validation is needed.

The Nimblemind Multi-Agent System (nMAS) provides a method for automating the transformation of fragmented electronic health record (EHR) data into structured, clinically meaningful features for heart-failure research. By linking engineered features directly to clinical guidelines and source evidence, the system aims to reduce the manual workload typically required for data preparation while ensuring the resulting datasets are auditable and reproducible.

Addressing the Feature Engineering Bottleneck

Data scientists often spend 39–45% of their time on data loading and cleaning. In cardiology, this process is particularly complex because relevant information—such as laboratory results, medication histories, and imaging findings—is often stored in separate, heterogeneous EHR tables. Existing methods, including rule-based systems and general-purpose large language models (LLMs), often lack the ability to integrate this data with specific clinical guidelines or provide a clear audit trail for how a patient-level variable was derived. nMAS attempts to solve this by using a multi-agent pipeline that enforces clinical reasoning through an explicit, versioned rubric.

How the Pipeline Works

The nMAS pipeline operates in two primary stages:

  • Data Standardization and Aggregation: The system cleans and merges nine distinct EHR source tables into a single patient-level record. It uses clinically meaningful event keys to deduplicate data and parses longitudinal information, such as ejection fraction ranges, into standardized formats.

  • Rubric-Grounded Engineering: The system applies a clinical scoring rubric to generate composite features, such as disease severity and comorbid-burden scores. These scores are deterministic and remain linked to their source data.

  • Auditing: A restricted LLM (Qwen 2.5-1.5B-Instruct) reviews the generated features to ensure structural integrity and compliance with the rubric. This auditor is limited to specific fields, ensuring that the underlying evidence traces remain protected and unaltered.

Performance and Evaluation

In a pilot study using 500 dummy patient records, the researchers evaluated the system’s ability to generate valid, coherent features. The inclusion of these aggregated features improved the held-out AUROC for heart-failure phenotyping:

  • HFrEF (Reduced Ejection Fraction): Improved from 0.895 to 0.963.

  • HFpEF (Preserved Ejection Fraction): Improved from 0.870 to 0.910.
    Additionally, an independent LLM-based assessment of the features—evaluating methodological soundness and evidence support—scored the output at 81.5% of the maximum possible points.

Considerations for Future Use

While the results suggest that automated, auditable feature engineering is feasible for complex cardiovascular data, the study has notable limitations. The evaluation was conducted on a single-institution cohort using dummy records, meaning the system has not yet been tested against external, real-world datasets. Furthermore, the researchers note that while the system automates the construction of features, it relies on a predefined rubric that requires initial clinical expertise to develop and maintain. Future work is required to validate the pipeline’s performance in broader, multi-site clinical environments.

Comments (0)

No comments yet

Be the first to share your thoughts!