LLMs Can See the Smoke but not the Fire: Evaluating Abductive Reasoning with Elenchos

Jul 15, 2026

A new evaluation framework called Elenchos assesses large language models' (LLMs) ability to perform abductive reasoning by having them infer hidden mutations in formal systems. The study finds that both frontier and mid-tier LLMs can often detect that a system has changed but struggle to accurately identify the specific underlying mutations. Performance drops further when multiple interacting mutations are present, and increasing inference-time reasoning yields only modest gains.

Why it matters: This work exposes a key limitation in LLMs' reasoning abilities, highlighting a gap between detecting anomalies and understanding their causes, which is important for applications requiring reliable causal inference.

Full story at: arXiv AI/ML ↗