Faithful by Design: Evaluating and Improving LLM-Generated Clinical Trial Summaries for Multi-Stakeholder Audiences

Jul 14, 2026

A new preprint introduces a benchmark framework to evaluate the faithfulness of large language model (LLM)-generated clinical trial summaries tailored for healthcare providers, patients, and payers. The study assessed GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash on 1,800 summaries, finding 'Unsupported Claims' as the most common error. Incorporating a knowledge-graph-augmented retrieval system led to statistically significant improvements in faithfulness scores, though the nature of improvements varied by model.

Why it matters: This work proposes a systematic approach to identifying and mitigating hallucination risks in clinical summarization, which is crucial for the safe and trustworthy use of LLMs in healthcare.

Full story at: arXiv Computation and Language ↗