In language, words give large models a shared token space: a way to compress, exchange, and reuse knowledge across people, books, domains, and time. What is the equivalent for robotics?
Robotics is radically heterogeneous. Bodies, cameras, sensors, action spaces, tools, tasks, and environments all differ. A useful physical representation must preserve what transfers across those differences.
The value of a robot world model may lie less in generating realistic videos than in discovering a reusable language of physical interaction.
The limits of pixels and black-box latents
Pixel prediction offers dense supervision, but it spends capacity on texture, lighting, background motion, viewpoint, and camera artifacts. Robots often care about something else: geometry, contact, object state, and how motion changes what can happen next.
Unconstrained latent spaces solve the compression problem but introduce another one. They can collapse, resist interpretation, and make intervention or correction difficult. A compact representation is not enough if neither a person nor an agent can understand what it believes about the world.
A structured third path
Our research increasingly explores a third path: representations that are learned but structured, compact but interpretable, and abstract while remaining grounded in physical evidence.
μ0 predicts semantic 3D interaction traces—the motion of object parts, tools, hands, and contact regions. These traces provide a shared space for learning motion priors from heterogeneous video without requiring action labels during world-model pretraining.
TraceGen develops the same representation-space idea for cross-embodiment learning. Rather than reproduce every visible detail, it models geometric and temporal structure that can transfer between human and robot video.
MomaGraph uses state-aware scene graphs containing spatial relations, functional relations, object states, and part-level affordances. Its world representation is designed for planning and intervention rather than passive scene description.
TraceVLA brings structured spatial-temporal traces into vision-language-action policies, while earlier projects such as TACO and Premier-TACO studied compact temporal representations for few-shot robot learning.
What should a physical language provide?
A useful physical language should abstract away appearance that does not affect action, preserve geometry and contact, express state changes, support intervention, and remain reusable across bodies and tasks. It should also connect with foundation models rather than force every robot to relearn the world from scratch.
This changes how we should evaluate world models. Visual realism is not the final objective. The stronger test is whether the representation gives an agent a space in which it can reason, plan, intervene, and act.