VVM-Tuning: Generalizing LMMs to Unseen Visual Modalities via Fabricated Modality Synthesis

Jul 14, 2026

Researchers introduce VVM-Tuning, a training framework that enables large multimodal models (LMMs) to generalize to unseen visual modalities by synthesizing diverse appearance images from RGB scenes. The approach disentangles invariant scene semantics from modality-specific appearances and leverages modality contexts for zero-shot adaptation. The team also presents VVM-Bench, a benchmark evaluating semantic perception and modality understanding across six real and synthetic modalities. Experiments show that models trained with VVM-Tuning achieve consistent improvements on both real and synthetic modalities without requiring in-modality training data.

Why it matters: This work proposes a scalable method for improving LMMs' ability to generalize to new visual modalities, addressing a key challenge in multimodal AI.

Full story at: arXiv Computer Vision ↗