MafiaScope: Non-Invasive Belief Probing for LLM Agents in Social Deduction Games

Jul 14, 2026

MafiaScope is an open testbed that adapts the social deduction game Mafia to probe the beliefs and Theory of Mind of LLM agents. After each public utterance, agents privately answer structured probe questions, with responses scored against ground truth and visualized as belief trajectories. In a 32-game case study using DeepSeek, the system revealed poor calibration (expected calibration error 0.17) and a tendency for agents to over-predict being suspected. The platform, including engine, viewer, and a large corpus of games, is released under an open license.

Why it matters: This introduces a novel, systematic approach to measuring and visualizing the internal beliefs and reasoning processes of LLM agents during social interactions.

Full story at: arXiv Computation and Language ↗