Submodular Selection of LLM Benchmark Prompts Preserves Model Rankings Without Evaluation

Jul 14, 2026

Researchers propose an evaluation-unsupervised method for selecting a small subset of prompts from multiple LLM benchmarks that preserves model scores and rankings. Using submodular functions on semantic embeddings, the facility location approach outperforms score-based and diversity-based baselines across 35 benchmarks and 18 models. The method also matches or outperforms state-of-the-art approaches on MMLU and MTEB leaderboards at lower computational cost.

Why it matters: This work enables efficient and cost-effective LLM evaluation by compressing large benchmark suites without requiring model inference during selection.

Full story at: arXiv AI/ML ↗