Large Models for Robotics 101

What do VLM, VLA, world models, policies mean?

Methods and named models belong to categories. Companies often span categories

Large models for robotics are not one single architecture. The field is better understood as a stack of routes: VLM backbones with action heads, end-to-end VLA policies, robot foundation models, explicit world models, planner-policy systems, simulation engines, classic robot learning policies, and 3D spatial memory.

This matters because a full-stack robotics company may have a VLA model, a planner, a data engine, a control stack, and a spatial perception system. Repeating the same company product name in every category can make the taxonomy look more precise than the public evidence supports.

1. VLM + Action Head

VLM + Action Head: reuse a pretrained vision-language model, then attach an action decoder that outputs robot trajectories, end-effector poses, gripper commands, or joint actions.

Core idea: reuse a pretrained vision-language model, then attach an action decoder that outputs robot trajectories, end-effector poses, gripper commands, or joint actions.

image + instruction -> VLM backbone -> action head -> robot action

This route is attractive because the model inherits strong visual and language priors. The hard part is turning semantic understanding into stable physical action.

Representative global methods:

• Gemini Robotics: arXiv
• NVIDIA GR00T: arXiv, GitHub
• Physical Intelligence pi0 / openpi: paper PDF, GitHub
• Figure Helix: official page

China examples:

• LingBot-VLA
• AgiBot GO-1 style embodied stack
• Galaxea G0 / G0+ style embodied stack
• X Square WALL-A style embodied stack

2. End-to-End VLA

Core idea: train a Vision-Language-Action model that maps observations and instructions directly to actions.

observation + language instruction -> robot action

This is the cleanest formulation of a robot policy foundation model. It is also heavily bottlenecked by robot data quality and embodiment mismatch.

End-to-End VLA: train a Vision-Language-Action model that maps observations and instructions directly to actions.

Representative global methods:

• RT-2: arXiv, project
• OpenVLA: arXiv, GitHub
• pi0 / openpi: GitHub
• GR00T N1: arXiv, GitHub

China examples:

• LingBot-VLA: paper page
• X Square WALL-A
• Qianxun Spirit v1
• Robot Era ERA-42
• Galaxea G0 / G0+
• AgiBot GO-1

3. Robot Foundation Model

Core idea: train a general robot brain across tasks, embodiments, sensors, and environments.

This category is broader than a single VLA policy. A robot foundation model may include perception, planning, memory, action, adaptation, and cross-embodiment transfer.

Robot Foundation Model: train a general robot brain across tasks, embodiments, sensors, and environments.

Representative global methods:

• Physical Intelligence pi0: paper PDF, GitHub
• Octo: arXiv, GitHub
• RDT-1B: arXiv
• Covariant RFM-1: official article
• Skild AI: official site

China companies positioning around this route:

• X Square Robot
• AgiBot
• Galaxea AI
• LingBot
• Galbot
• Robot Era
• UBTECH
• XPENG IRON

4. Explicit World Model

Core idea: predict future states, videos, or latent physical dynamics before acting.

current state + candidate action -> predicted future state

This is the stricter meaning of world model. It should support future prediction, counterfactual planning, synthetic data generation, or safety checking. Many companies use the phrase “world model,” but not all of them publicly show this capability.

Explicit World Model: predict future states, videos, or latent physical dynamics before acting.

Representative global methods:

• NVIDIA Cosmos: arXiv, GitHub
• Google DeepMind Genie: arXiv, official page
• Covariant RFM-1: official article

China named examples:

• LingBot-World: paper page
• Galaxea Fast-WAM / world-model stack

5. Planner + Policy

Core idea: separate high-level reasoning from low-level execution.

planner: understand task, decompose steps, check progress policy: execute grasping, walking, placing, manipulation

This is often more practical than a single end-to-end model. The planner can reason slowly; the policy must act quickly and reliably.

Planner + Policy: separate high-level reasoning from low-level execution.

Representative global methods:

• SayCan: arXiv, project
• Gemini Robotics / Robotics-ER: arXiv
• Figure Helix S2/S1: official page
• OpenHelix: GitHub

China examples:

• X Square WALL-A / embodied CoT
• AgiBot GO-1
• Galaxea G0-style task-policy stack
• Galbot
• LingBot
• Qianxun Spirit v1

6. Classic Robot Learning Policies

Core idea: use smaller specialized policies for precise, high-frequency control, imitation learning, manipulation, and locomotion.

These models may not have broad semantic reasoning, but they can be excellent at stable physical execution.

Classic Robot Learning Policies: use smaller specialized policies for precise, high-frequency control, imitation learning, manipulation, and locomotion.

Representative global methods:

• ACT / ALOHA: arXiv, ACT GitHub, ALOHA GitHub
• Mobile ALOHA: arXiv, GitHub
• Diffusion Policy: arXiv, GitHub
• Octo: arXiv, GitHub

China examples:

• Unitree
• LimX Dynamics
• Fourier
• Robot Era
• Qianxun Spirit v1
• EngineAI
• UBTECH
• Galbot
• Galaxea manipulation/control stack

One more thing: Simulation / Synthetic Data Engine

Core idea: use simulation, digital twins, generated video, synthetic trajectories, and scenario generation to scale robot data.

real data -> simulator / generator -> synthetic data -> policy training -> real robot validation

This route matters because real robot data is expensive, slow, and risky to collect.

Representative global platforms:

• NVIDIA Isaac Lab: GitHub
• NVIDIA Cosmos: GitHub, arXiv
• NVIDIA GR00T / GR00T data stack: GitHub
• NVIDIA Omniverse: official page

China examples:

• LingBot-World
• Galaxea data engine / robot data platform
• Galbot scene data flywheel
• AgiBot data flywheel
• Unitree, Fourier, UBTECH, XPENG IRON as embodiment and deployment data sources