Audit Reveals Distributional Reinforcement Learning Agents' Risk Claims Often Unreliable

Jul 14, 2026

A new preprint audits distributional reinforcement learning (RL) agents—specifically QR-DQN, C51, and IQN—and finds that 40-95% of their strongest risk trade-off claims are statistically refuted at 95% confidence. The study shows that the 'risk' learned by these agents is largely a training artifact, not a reflection of true environment stochasticity, and that following the agents' risk-based advice can sometimes lead to outcomes worse than random choice. The authors provide a statistical toolkit for auditing such claims and highlight pitfalls that can lead to misleading audit results.

Why it matters: This work raises significant concerns about the reliability of risk estimates from distributional RL agents, which are increasingly considered for safety-critical applications.

Full story at: arXiv Statistical ML ↗