Microsoft’s MAI Health team has reported new results for an AI diagnostic orchestrator and a benchmark that reveals medical information step by step. The October 22 announcement describes measured performance on constructed diagnostic cases and a separate plan to study clinical reasoning in real hospital settings.
Evaluate the path to an answer
In Microsoft’s account, the Sequential Diagnosis Benchmark, or SDBench, includes 304 complex cases from New England Journal of Medicine proceedings and more typical emergency-department presentations from MIMIC-IV-ED. Participants begin with limited information and choose what to request next.
The benchmark measures diagnostic accuracy, evidence requested and estimated test costs. This makes the sequence of questions and tests part of the evaluation, rather than presenting every fact upfront and scoring only one final answer.
The comparisons have specific conditions
Microsoft reports MAI-DxO accuracy of 83 percent on complex NEJM cases, compared with 22 percent for physicians without external references and 36 percent with them. For the emergency-department cases, it reports 80 percent versus 47 percent.
Those comparisons depend on the case selection, information interface and reference conditions. They do not establish that the system improves patient outcomes in ordinary clinical practice. Estimated test costs in a benchmark are also different from measured changes in total care costs.
Microsoft describes MAI-DxO as coordinating a panel-like exchange in which models propose diagnoses, question assumptions and weigh the value of additional evidence. It is making evaluation components available for selected NEJM cases so other researchers can investigate the results.
Real-world evaluation is a separate step
The company says a partnership with Beth Israel Deaconess Medical Center will evaluate clinical reasoning across hospital settings, including a prospective study comparing system performance with physicians’ independent assessments.
That planned work is important evidence still to come. The announcement supports reporting the benchmark results and evaluation design, while leaving clinical effectiveness, workflow fit and performance across different patient populations to further study.