CORE-Bench: A Comprehensive Benchmark for Code Retrieval in the Era of Agentic Coding

Jul 14, 2026

Researchers present CORE-Bench, a new benchmark designed to evaluate code retrieval in agentic coding scenarios. The benchmark assesses models on code understanding, issue-to-edit localization, and broader context retrieval, using over 180,000 queries and 106,000 relevance labels. Results show that current embedding models perform poorly on these tasks, but simple supervised fine-tuning leads to notable improvements, highlighting significant room for advancement.

Why it matters: CORE-Bench fills a critical gap in evaluating code retrieval for AI coding agents, supporting the development of more capable systems for navigating and understanding code repositories.

Full story at: arXiv Information Retrieval ↗