AI21 Labs Proposes Query-Dependent Chunking for RAG Retrieval

Jul 11, 2026

AI21 Labs has introduced a multi-scale approach to Retrieval-Augmented Generation (RAG) retrieval by indexing the same corpus at multiple chunk sizes (such as 100, 200, and 500 tokens) and aggregating results using Reciprocal Rank Fusion. This method reportedly improves retrieval performance by 1–37% across benchmarks without requiring model retraining, with oracle experiments showing 20–40% gains.

Why it matters: This technique addresses a key limitation of RAG systems by making chunk size adaptive to query needs, potentially improving accuracy in information retrieval tasks.

Full story at: AI21 Labs ↗