EYT-Bench: A Human-Centered Benchmark for Multi-Turn Dialogue Evaluation

Jul 14, 2026

Researchers have introduced EYT-Bench, a benchmark designed to evaluate large language models (LLMs) as multi-turn conversational partners. EYT-Bench uses a decoupled, three-party design involving a persona-grounded user simulator, a target model, and an independent judge, with personas sampled from human-curated corpora. In a 17-model evaluation, the benchmark reveals that while models are statistically similar on subjective measures like empathy and persona consistency, they differ by up to 9x on objective intent tracking. Additionally, enabling reasoning in models improves objective tracking without affecting subjective scores.

Why it matters: EYT-Bench exposes significant gaps in objective intent tracking among conversational AI models that are not captured by single-turn benchmarks, highlighting the need for more comprehensive evaluation methods.

Full story at: arXiv Computation and Language ↗