When Does Reward Teach State? A Hidden-Automaton Instrument and the Group-Language Boundary

Jul 15, 2026

A new preprint introduces a white-box instrument based on hidden deterministic finite automata to separately measure reward attainment and latent-state learning in reinforcement learning (RL) agents. The study demonstrates that high reward does not necessarily indicate that an agent has learned the underlying task state, distinguishing between 'perception gaps' (where latent state is not recoverable from observations) and 'planning gaps' (where state is recoverable but not used). The authors show that optimizer strength, task structure, and observation informativeness each influence the relationship between reward and state learning.

Why it matters: This work challenges the common assumption that high reward in RL implies genuine task understanding, providing a method to diagnose when agents exploit shortcuts rather than learning true latent states.

Full story at: arXiv Machine Learning ↗