DIVE: Embedding Compression via Self-Limiting Gradient Updates

Jul 16, 2026

A new method called DIVE is introduced for compressing language-model embeddings using a residual compression adapter that incorporates a self-limiting hinge loss and geometry distillation. In experiments on five BEIR benchmarks with LLM2Vec backbones, DIVE consistently outperforms six baseline methods, including PCA and autoencoders, at both 128- and 256-dimensional outputs.

Why it matters: DIVE enables more efficient storage and retrieval in large-scale information retrieval systems by compressing embeddings without sacrificing retrieval quality.

Full story at: arXiv Information Retrieval ↗