AI Research

CARE-Bench: Benchmarking Patient-Facing LLM Triage | AI Research

Key Takeaways

CARE-Bench is a new benchmark designed to evaluate how well patient-facing medical LLMs perform triage—the process of determining the next appropriate action...
Patient-facing medical LLMs and agents increasingly answer symptom questions before clinician contact, where the key safety question is what action the user should take next.
We introduce CARE-Bench, a source-grounded benchmark that evaluates sequential patient-facing triage as a four-label per-turn current-action task.
CARE-Bench contains 500 cases and 1,059 evaluated patient-disclosure prefixes reconstructed from medical dialogue, consultation, and follow-up-question sources.
We evaluate 11 models on 269 held-out rounds under unprompted and minimally prompted open-ended protocols, using a fixed GPT-5.5 mapper to code each response into the four-label action space.

Paper AbstractExpand

Patient-facing medical LLMs and agents increasingly answer symptom questions before clinician contact, where the key safety question is what action the user should take next. We introduce CARE-Bench, a source-grounded benchmark that evaluates sequential patient-facing triage as a four-label per-turn current-action task. CARE-Bench contains 500 cases and 1,059 evaluated patient-disclosure prefixes reconstructed from medical dialogue, consultation, and follow-up-question sources. We evaluate 11 models on 269 held-out rounds under unprompted and minimally prompted open-ended protocols, using a fixed GPT-5.5 mapper to code each response into the four-label action space. Unprompted macro-F1 remains low, ranging from 31.2 to 50.4. Prompting improves 10 of 11 models, with prompted macro-F1 ranging from 46.9 to 63.4, but substantial threshold errors remain. Prompted models often recommend care before needed clarification is obtained; when the correct action was to ask for more information, only 33.5% of prompted outputs preserved the step. The persistence of these errors after prompting suggests that patient-facing triage is not a simple prompting problem and supports explicit evaluation of action timing before deployment.

CARE-Bench is a new benchmark designed to evaluate how well patient-facing medical LLMs perform triage—the process of determining the next appropriate action for a user based on their reported symptoms. The researchers aim to address safety concerns regarding whether these AI agents correctly identify when to provide medical advice versus when to ask for more information before making a recommendation.

Evaluating Triage Performance

The researchers developed CARE-Bench to assess sequential triage as a four-label, per-turn task. The benchmark consists of 500 cases and 1,059 patient-disclosure prefixes, which were reconstructed from various medical sources, including consultations, dialogues, and follow-up questions. To measure performance, the team evaluated 11 different models on 269 held-out rounds. They utilized a fixed GPT-5.5 mapper to categorize model responses into the four-label action space, allowing for a standardized assessment of how models handle patient inquiries.

Impact of Prompting on Accuracy

The study found that unprompted models performed poorly, with macro-F1 scores ranging from 31.2 to 50.4. While applying minimal prompting improved the performance of 10 out of 11 models—raising macro-F1 scores to a range of 46.9 to 63.4—significant errors persisted. Specifically, the models struggled with the timing of their recommendations. When the correct action was to gather more information, prompted models only chose that step 33.5% of the time, often recommending care prematurely before obtaining necessary clarifications.

Limitations in Current LLM Triage

The persistence of these errors after prompting indicates that triage accuracy is not merely a result of how a model is prompted. The authors conclude that these findings support the need for explicit evaluation of action timing before deploying medical LLMs in patient-facing roles. Because models frequently bypass the critical step of information gathering, the research suggests that current LLM architectures may require more rigorous safety testing to ensure they do not provide premature clinical recommendations.

Comments (0)

No comments yet

Be the first to share your thoughts!