Back to AI Research

AI Research

Radiology AI simulation finds gaps in cleared tool coverage

Key Takeaways

  • A radiology preprint simulates AI-assisted report generation and finds that tool coverage and selection both limit recovery of actionable findings.
  • A simulated agent recovered fewer than a third of actionable findings with a cleared-tool catalog and the clinical indication alone.
  • A radiology preprint asks what an AI agent could cover if it were given modular tools for detecting and measuring findings.
  • With a catalog derived from FDA-cleared devices and only the clinical indication to guide planning, the authors report that the agent recovered fewer than a third of clinically actionable findings in their simulation.
  • The preprint was submitted to arXiv on November 6, 2026.

A radiology preprint asks what an AI agent could cover if it were given modular tools for detecting and measuring findings. With a catalog derived from FDA-cleared devices and only the clinical indication to guide planning, the authors report that the agent recovered fewer than a third of clinically actionable findings in their simulation. The preprint was submitted to arXiv on November 6, 2026.

How the study separates tools from planning

The researchers used reports across multiple imaging modalities. They compared a catalog limited to finding classes supported by cleared devices for the relevant modality with an unlimited catalog that could return any requested finding class. They also varied whether the agent's plan was based on the clinical indication or on the target report.
Tool responses were simulated from the reference reports. That choice makes it possible to study whether the necessary tool was available and whether the agent called it, without adding detection or measurement errors from an actual device. The paper's workflow diagrams lay out the stages from selecting tools to drafting and reviewing the report.

What the simulation missed

The authors attribute most missed findings to classes that the cleared catalog did not cover for the study's modality. They attribute about a third to classes or tools the agent could have used but did not call. Adding the most frequently missing tools narrowed much of the gap to unlimited coverage in held-out reports, while a smaller orchestrator lost the advantage of the unlimited catalog.

What the result means for evaluation

Catalog coverage and orchestration address different failure modes. Teams evaluating a reporting agent can look separately at which findings their tools can return and which tools the agent calls for a given indication. The experiment does not establish real-world clinical performance: the simulated tools take their outputs from existing reports, and planning from the target report is an experimental comparison rather than a workflow available at deployment.

Comments