The complexities of patient-centred conversational artificial intelligence

This research addresses a critical gap in how health chatbots are developed and tested. While large language models (LLMs) are increasingly used for symptom assessment, they are typically evaluated using "idealized" patients—simulated users who are articulate, cooperative, and easy to understand. This paper argues that such testing fails to reflect the reality of human communication, potentially leading to systems that perform poorly or exacerbate health disparities when faced with the diverse ways real patients express their symptoms and emotions.

Analyzing real-world communication

To understand how people actually interact with health AI, the researchers analyzed 2,053 real conversations between patients and chatbots. They discovered that communication patterns, emotional states, and conversational strategies vary significantly from person to person. Relying on simplified, artificial test cases ignores this diversity, which is a major oversight for tools intended to provide clinical triage and support.

A new approach to patient simulation

The authors developed a sophisticated patient simulator designed to capture the nuance of human interaction. Unlike previous models, this simulator separates four distinct components: clinical content, emotional state, conversational strategy, and communication style. To test its effectiveness, they conducted a Turing-inspired evaluation where 15 human graders attempted to distinguish between real and simulated conversations. The graders achieved only 55% accuracy, indicating that the simulator successfully produced interactions that were nearly indistinguishable from those of real patients.

The impact of communication style on triage

Using this simulator, the researchers created five distinct patient personae to test how four different LLMs handled urgency assessment across 1,164 clinician-graded cases. The results were revealing: the way a patient communicates their symptoms can significantly change the triage outcome provided by the AI. This suggests that the AI’s performance is not just dependent on its medical knowledge, but also on its ability to interpret different styles of human expression.

Why realism matters for health equity

The study concludes that for conversational AI to be truly patient-centred, it must be robust enough to handle the full spectrum of human communication. If developers continue to design and evaluate systems based on idealized interactions, they risk creating tools that underperform in real-world settings. Ultimately, failing to account for communication diversity may amplify existing health disparities, as the AI may struggle to accurately assess patients who do not fit the narrow, "articulate" mold used during the development phase.