Back to AI Research

AI Research

NPC-RubricBench tests how clinical reasoning changes across cancer-care phases

Key Takeaways

  • Patient-specific criteria expose different errors and omissions across a longitudinal care pathway.
  • [NPC-RubricBench](https://arxiv.org/abs/2610.35847) evaluates language models across a cancer-care pathway rather than at a single encounter.
  • The preprint uses nasopharyngeal carcinoma as its defined disease setting and asks whether models remain reliable as clinical responsibilities change over time.
  • ## Criteria tied to the patient and phase
  • The benchmark draws on longitudinal records from 405 patients at two cancer centres.

NPC-RubricBench evaluates language models across a cancer-care pathway rather than at a single encounter. The preprint uses nasopharyngeal carcinoma as its defined disease setting and asks whether models remain reliable as clinical responsibilities change over time.

Criteria tied to the patient and phase

The benchmark draws on longitudinal records from 405 patients at two cancer centres. It contains 941 patient-phase units across six sequential phases and 17,688 patient-specific atomic criteria spanning five evaluation dimensions.
Each assessment is restricted to evidence available at the time of the decision. That boundary matters because later information could make an earlier clinical choice appear easier than it actually was. A useful longitudinal evaluation must preserve what the model could legitimately know at each step.

Errors change across the pathway

The authors evaluated 30 general-purpose and medical language models. They report that strong diagnostic performance did not translate consistently into downstream care performance.
Explicit decision errors dominated anatomical staging at 48.2%, while omissions dominated initial work-up and follow-up at 33.4% and 30.4%. In this evaluated set, medically adapted models had higher omission rates than general-purpose models. These findings describe the models and benchmark studied, rather than establishing a universal ranking of medical versus general models.

More detail in a prompt has tradeoffs

Broad requests reduced Decision Quality across the evaluated models. More checklist-like prompts improved that dimension but reduced Evidence Faithfulness and Safety Adherence. A higher score on one dimension therefore did not settle the overall assessment.
For evaluators, the paper supports checking both phase-specific responsibilities and what a model leaves out. It does not establish that any tested system is ready to direct patient care. Clinical deployment would require evidence beyond performance on these records and rubrics, including safeguards suited to the decisions being made.

Comments