Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents
Researchers introduce Agentic Real2Sim, a framework that leverages vision-language agents to automate the conversion of real-world recordings of object-robot interactions into simulatable digital twins. The system is demonstrated on tasks involving rigid objects, deformable objects, and humanoid motions, domains typically requiring separate pipelines. Agentic Real2Sim achieves conversion success rates comparable to leading models but at a lower computational cost, supporting downstream robotics tasks such as policy learning and evaluation.
Why it matters: This framework could streamline and scale the creation of realistic physics simulations from real-world robot interactions, potentially accelerating robotics research and development.
Full story at: arXiv Robotics ↗