Multi-Agent Routing as Set-Valued Prediction: A WildChat Benchmark and Cost-Aware Evaluation

Jul 14, 2026

Researchers introduce a new benchmark based on WildChat for evaluating multi-agent routing from natural-language prompts as a set-valued prediction problem. The study demonstrates that supervised routing models, including a fine-tuned encoder and a linear multilabel classifier, significantly outperform nearest-neighbor and zero-shot LLM baselines. The benchmark includes a reproducible evaluation protocol with set-level metrics and cost-aware constraints, enabling systematic study of accuracy-cost trade-offs in multi-agent systems.

Why it matters: This work enables reproducible and systematic evaluation of multi-agent routing strategies, which is important for developing efficient and cost-effective multi-agent systems.

Full story at: arXiv Multiagent Systems ↗