CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness
Researchers introduce CASE, a framework that enhances the faithfulness of chain-of-thought (CoT) reasoning in large language models by combining training-time causal alignment with inference-time structural enforcement. CASE employs counterfactual datasets and selective-loss fine-tuning to reinforce the dependence of answers on reasoning chains while suppressing shortcuts from instructions directly to answers. Experimental results across multiple models and benchmarks demonstrate a 37% average relative improvement in CoT faithfulness compared to strong baselines, without sacrificing accuracy.
Why it matters: Improving the faithfulness of CoT reasoning could make language model outputs more reliable and interpretable, addressing a key limitation in current LLM reasoning methods.
Full story at: arXiv Computation and Language ↗