SoftNav: Injecting 3D Scene Tokens into VLMs for Embodied Navigation

Jul 17, 2026

SoftNav introduces a method for injecting entity-level 3D continuous representations as soft tokens into the hidden space of vision-language models (VLMs), enabling goal-directed navigation with minimal training. The approach achieves state-of-the-art results on the HM3D-OVON benchmark and demonstrates zero-shot transfer to other navigation benchmarks and real-world robot deployment, all without retraining or architectural changes.

Why it matters: This work bridges the gap between 3D scene understanding and VLMs, enabling efficient and transferable embodied navigation with minimal data and parameters.

Full story at: arXiv Robotics ↗