Masked Visual Actions for Unified World Modeling
Researchers propose Masked Visual Actions, a pixel-space control interface that represents robotic actions as partially revealed trajectories in video. A single model, finetuned on 15 hours of real and simulated data, demonstrates strong visual fidelity and controllability across various scenes and robot embodiments. The approach enables forward dynamics prediction, model-based planning, and inverse modeling within a unified framework.
Why it matters: This method offers a new way to connect video models and robotic control without explicit action tokens, potentially simplifying and unifying world modeling for manipulation tasks.
Full story at: arXiv Robotics ↗