Claude Fable 5 Refuses Up to 99.4% of Biomedical Questions, Outperforms Peers When It Answers

Jul 14, 2026

A new arXiv preprint evaluates Claude Fable 5 on eight biomedical benchmarks, finding it refuses to answer between 8.0% and 99.4% of questions depending on the benchmark—a pattern not seen in its predecessors or GPT-5. When refusals are excluded, Fable 5's accuracy meets or exceeds all other models on every benchmark. The study also identifies distinct refusal patterns related to basic-science content and rare disease domains.

Why it matters: This suggests that the main limitation for deploying advanced language models like Fable 5 in biomedical settings may be their willingness to answer, rather than their underlying capability.

Full story at: arXiv Computation and Language ↗