LLMs Struggle to Correct Medical Misconceptions in Multi-Turn Conversations

Jul 15, 2026

A new preprint introduces ThReadMed-QA, a dataset of 2,437 multi-turn patient-physician threads, to evaluate large language models (LLMs) on their ability to detect and correct medical misconceptions over several conversational turns. The study finds that even advanced models like GPT-5 and Claude-Haiku drop from about 85% accuracy on initial questions to around 50% after two follow-ups, with error propagation identified as a major cause of this decline. The results highlight that LLMs' ability to maintain accurate corrections diminishes over the course of a conversation.

Why it matters: This work exposes a significant safety risk in using LLMs for medical advice, as their corrections of misconceptions degrade in multi-turn interactions, potentially leading to unsafe outcomes.

Full story at: arXiv Computation and Language ↗