A Practical Map of the Models That Turn Perception Into Robot Action
A structured map of Physical AI models covering VLA policies, embodied reasoning, world models, manipulation, locomotion, tactile learning and simulation.
Introduction
A language model can describe how to load a dishwasher without producing a single motor command. A robot policy can output joint actions without understanding why the plate belongs in a rack. A world model can predict what a future scene may look like without controlling real hardware. Physical AI is the stack that connects these separate functions: sensing, representation, reasoning, planning, action generation, low-level control, safety and recovery.
The term is often used so broadly that simulators, video generators, perception networks and complete robot brains appear in one undifferentiated list. This map separates them by input, output and role. It covers widely used or strategically important systems available by July 11, 2026, including NVIDIA Isaac GR00T 1.7, Gemini Robotics, Gemini Robotics-ER 1.6, π0 and π0.5, OpenVLA, Octo, RT-2, Figure Helix 02 and 1X Redwood. “Open” is also divided into weights, code, data and license because those are not the same form of access.
Key findings
- A VLA model converts visual observations and language instructions into actions; an embodied reasoning model may plan and verify without directly controlling joints.
- World models generate or predict future states and data. They support planning and training but are not automatically deployable robot policies.
- OpenVLA, Octo, openpi and NVIDIA’s GR00T releases provide different combinations of code, weights and licenses; none is plug-and-play across every robot.
- Figure Helix and 1X Redwood are vertically integrated proprietary policies tied to each company’s hardware and data pipeline.
- The largest missing layer in public benchmarks is long-duration recovery: most evaluations measure task success, not intervention rate across real shifts.
Major Physical AI models and their roles
Architecture details are limited to what the organization publishes. “Closed” means deployable weights are not publicly released.
| Model | Organization / date | Category | Inputs → outputs | Hardware or task evidence | Access and main limitation |
|---|---|---|---|---|---|
| Isaac GR00T 1.7 | NVIDIA; July 2026 | Open reasoning VLA / humanoid policy | Images, language and robot state → action trajectories | Humanoid manipulation, finger control and cross-embodiment workflows through Isaac Lab and ROS | Weights and code available under NVIDIA terms; still requires robot-specific data, tuning and safety integration |
| Gemini Robotics 1.5 | Google DeepMind; current 2026 page | VLA / multimodal control | Image, text, audio or video context → robot actions | Dexterous manipulation and multi-step physical tasks on several robot forms | Closed service and selected access; hardware latency, cost and deployment details vary |
| Gemini Robotics-ER 1.6 | Google DeepMind; 2026 | Embodied reasoning and orchestration | Multi-view perception, language and tools → plans, spatial references and success decisions | Task planning, instrument reading, pointing and retry/progress logic | Available through Google tooling; it is a high-level brain, not a universal low-level controller |
| Gemini Robotics On-Device | Google DeepMind; 2026 | On-device VLA | Image, text and action context → actions | Local dexterous manipulation and adaptation | Private preview / trusted testers; model weights and broad benchmarks are not public |
| π0 | Physical Intelligence; October 2024 | Generalist VLA policy | Images, language and proprioception → continuous actions | Multi-robot manipulation including laundry, boxes and table tasks | Base weights and code through openpi; substantial fine-tuning and hardware integration required |
| π0.5 | Physical Intelligence; April 2025 | Open-world VLA policy | Visual observations, language and robot state → actions | Generalization to new homes and semantically varied chores | Research release details are more limited than openpi; broad independent reproduction remains sparse |
| OpenVLA 7B | Stanford, UC Berkeley and collaborators; June 2024 | Open-source VLA | Image and language → tokenized robot actions | BridgeData V2 and LIBERO manipulation evaluations | Code and weights open; action representation and embodiment transfer need adaptation |
| Octo | UC Berkeley and collaborators; 2024 | Generalist robot policy | Images, language and task observations → action chunks | Fine-tuning across multiple robot datasets and embodiments | Open code and checkpoints; performance depends heavily on dataset and embodiment match |
| RT-2 | Google DeepMind; July 2023 | VLA | Images and text → action tokens | Semantic manipulation and web-knowledge transfer on robot tasks | Closed model; useful historical foundation but not a downloadable general platform |
| Helix 02 | Figure AI; 2026 | Proprietary full-body VLA / control hierarchy | Robot cameras, language and state → upper-body and locomotion actions | Figure 03 dishwasher, room-tidying and industrial tasks | Closed and tied to Figure hardware; benchmarks and intervention logs are company-controlled |
| Redwood AI | 1X Technologies; 2025-2026 | Proprietary home-robot policy | Vision, language and robot state → NEO actions | Household skills and data collection on NEO | Closed; Expert Mode means human assistance remains part of product operation |
| Skild Brain | Skild AI; current 2026 | Cross-embodiment robotics foundation model | Multimodal observations and instructions → robot policies | Claims across manipulation, locomotion and different embodiments | Closed commercial access; public architecture and benchmark detail remain limited |
| Cosmos world foundation models | NVIDIA; 2025-2026 | World models and synthetic data | Video, text and physical scene context → predicted or generated video/world states | Data generation, scenario expansion and robot training support | Models and tools vary by release; output is not a motor policy without a downstream controller |
| Open X-Embodiment / RT-X | Google DeepMind and 20+ institutions; 2023 | Dataset mixture and cross-robot policy research | Multi-robot trajectories → training data for generalist policies | Shared data from many robot embodiments and RT-X experiments | Dataset heterogeneity, licenses and action normalization complicate reuse |
Taxonomy from data to motion
This text structure can be converted directly into an infographic.
| Layer | Question answered | Examples | Output passed downstream |
|---|---|---|---|
| Data and simulation | What experiences can the system learn from? | Open X-Embodiment, Isaac Lab, teleoperation datasets, Cosmos-generated scenarios | Trajectories, labels, synthetic scenes and demonstrations |
| Perception and tactile representation | What objects, contacts and body states exist now? | Vision encoders, depth models, Sparsh-style tactile representations | Features, poses, contact state and uncertainty |
| World model | What may happen next? | NVIDIA Cosmos and learned dynamics models | Predicted future states or generated rollouts |
| Embodied reasoning | What is the task, constraint and next subgoal? | Gemini Robotics-ER and high-level planners | Plan, spatial grounding, tool call or success judgment |
| VLA / action policy | Which action sequence should execute the instruction? | GR00T, Gemini Robotics, π0, OpenVLA, Octo, Helix, Redwood | End-effector targets, action chunks or joint-level commands |
| Low-level manipulation and locomotion | How should motors track the target while maintaining contact and balance? | Whole-body controllers, reinforcement-learning locomotion policies, impedance control | Torque, position or velocity commands |
| Safety, recovery and supervision | Should the robot continue, retry, stop or ask for help? | Collision limits, fall detection, remote assistance and success detectors | Stop, retry, reduced-speed mode or human handoff |
Vision-language-action models are policies, not complete robots
A VLA model receives visual observations and an instruction, then predicts actions. Those actions can be Cartesian gripper targets, joint commands or tokenized trajectories. The model still depends on calibrated cameras, robot kinematics, a real-time controller, collision handling and a safe execution layer. A checkpoint trained on one arm cannot be assumed to control a biped because action spaces, timing and body dynamics differ.
OpenVLA, Octo and openpi give researchers inspectable starting points. NVIDIA GR00T adds a broader humanoid development workflow around teleoperation, simulation, training, evaluation and deployment. Gemini Robotics and proprietary systems such as Helix integrate reasoning and action more tightly but expose less implementation detail.
Embodied reasoning sits above continuous control
Gemini Robotics-ER 1.6 is a useful example of a high-level model. It reasons about multi-view scenes, selects subgoals, interprets instruments and checks whether a step succeeded. It can orchestrate tools and decide to retry. That output can guide a policy, but it is not a replacement for a stable controller running at motor frequency.
This distinction matters when companies call a model a robot brain. A planner operating once per second and a torque controller operating hundreds or thousands of times per second are both necessary, yet they solve different problems. Public model maps should label the temporal and control level instead of placing every model in one “intelligence” box.
World models produce futures, not necessarily actions
World models learn how scenes may evolve. They can generate synthetic training clips, predict the effect of a candidate action or expose rare failure states in simulation. NVIDIA Cosmos is positioned as a family of world foundation models supporting physical AI data and simulation. A generated future can improve a policy, but the model does not become a deployable controller unless a planner or action policy consumes that prediction.
Video-generation quality is also not physical validity. Contact forces, friction, object mass and actuator limits can be visually plausible while mechanically wrong. Real-robot validation remains necessary, especially for dexterous manipulation and dynamic balance.
Manipulation, locomotion and touch need specialized layers
Generalist VLAs focus attention on semantic tasks, yet hand contact and biped balance are sensitive to milliseconds, force and local geometry. Many systems therefore pair a large model with specialized manipulation controllers, tactile estimators and reinforcement-learning locomotion policies. The large model selects an objective; lower layers execute it under physical constraints.
Tactile learning remains underrepresented in public foundation-model maps. Camera-based touch sensors can produce dense images, while force-torque sensors provide lower-dimensional contact signals. The model must synchronize those streams with vision and proprioception. Few public benchmarks compare tactile generalization across different hands.
Open source has four separate meanings
A project can release a paper without code, code without weights, weights without training data or all three under a license that restricts commercial use. OpenVLA and openpi provide meaningful code and checkpoint access. GR00T publishes code and weights under NVIDIA terms. Gemini Robotics, Helix, Redwood and Skild Brain remain controlled commercial systems.
Dataset access is equally important. Open X-Embodiment demonstrated the value of pooled robot trajectories, but heterogeneous sensors, coordinate frames and licenses make mixture training difficult. A model being downloadable does not remove the cost of collecting task-specific demonstrations on the target robot.
Benchmarks still miss operational reliability
LIBERO, BridgeData evaluations and laboratory task suites measure progress, but factories and homes need longer metrics: tasks per hour, intervention rate, recovery time, damage, safety stops and performance after lighting or object changes. A model can score well on short episodes and still fail repeatedly during a shift.
The next useful comparison should report policy success together with robot embodiment, control frequency, scene distribution, number of trials and human recovery. Without those fields, claims of generality remain difficult to compare across organizations.
Limitations and missing information
- The map prioritizes influential and currently accessible systems; it is not a list of every robotics model published worldwide.
- Organizations disclose architecture, training data and benchmarks at different levels of detail.
- Model names can cover several checkpoints or product versions, and access terms may change.
- Simulation tools such as Isaac Lab are included only as pipeline layers, not mislabeled as robot policies.
- No common benchmark compares all models on the same hardware, tasks, intervention rules and safety constraints.
Conclusion
Physical AI is a pipeline, not one giant model. Perception encodes the scene. A world model estimates possible futures. Embodied reasoning turns a goal into subgoals and checks progress. A VLA or specialized policy produces actions. Low-level controllers maintain contact, balance and timing. Safety and supervision decide when to stop or ask for help. Systems such as GR00T, Gemini Robotics, π0, OpenVLA, Helix and Redwood occupy different parts of that stack even when their marketing language overlaps.
For developers, model choice begins with the output required by the robot. A high-level planner is useful when task decomposition is the bottleneck. An open VLA is useful when the team can collect embodiment-specific data and tune the action space. A world model is useful for prediction and synthetic experience. None removes the need for controllers, calibration, safety and field evaluation. The most informative 2026 model map therefore labels role, access, embodiment and evidence rather than declaring a single “best robot brain.”
Frequently asked questions
What is a Physical AI model?
A Physical AI model processes information about the physical world and contributes to perception, reasoning, planning, prediction or control. The term can include VLAs, world models, tactile encoders and locomotion policies. It should not imply that one model performs every layer from camera input to safe motor torque.
What is a vision-language-action model?
A VLA takes visual observations and language instructions and predicts robot actions. Actions may be tokenized, continuous end-effector targets or joint trajectories. A VLA still needs robot-specific calibration, control, safety and often fine-tuning. It is closer to a task policy than to a complete autonomous robot product.
Is a robotics world model the same as a robot policy?
No. A world model predicts or generates possible future states. A policy selects actions. A planning system can use a world model to compare candidate actions, but generated video or predicted state alone does not move a robot. Some systems combine both functions, so architecture and outputs must be checked.
Which Physical AI models are open source?
OpenVLA, Octo, openpi and NVIDIA Isaac GR00T provide substantial public code or weights, though licenses and completeness differ. Open X-Embodiment provides datasets and research tooling. Gemini Robotics, Figure Helix, 1X Redwood and Skild Brain are controlled-access or proprietary systems. Open code does not guarantee open training data.
Can the same model control any robot?
Cross-embodiment models aim to transfer knowledge, but robots differ in cameras, joints, hands, timing, payload and safety limits. Most models require action-space adapters, demonstrations and fine-tuning for a target platform. General semantic knowledge transfers more easily than precise contact-rich control or dynamic locomotion.
Why are simulators not classified as models?
A simulator computes how a virtual environment changes under actions. It can generate data and evaluate policies, but it is not itself the learned policy. Learned world models can overlap with simulation, yet tools such as Isaac Lab remain environments and workflows. Keeping the categories separate prevents software infrastructure from being counted as robot intelligence.
Sources and methodology
Models were classified by published inputs, outputs and role rather than by company labels. Primary model pages, repositories, model cards and official technical blogs were reviewed. Access was separated into code, weights, data and commercial availability.
Verified July 11, 2026. Version numbers are stated where the organization publishes them. Missing architecture, license or benchmark detail is marked closed or not publicly disclosed instead of inferred.
- Develop Humanoid Robot Policies End-to-End with NVIDIA Isaac GR00T — NVIDIA · July 2026
- Isaac GR00T platform — NVIDIA · accessed July 11, 2026
- Gemini Robotics — Google DeepMind · accessed July 11, 2026
- Gemini Robotics-ER 1.6 — Google DeepMind · accessed July 11, 2026
- Gemini Robotics On-Device — Google DeepMind · accessed July 11, 2026
- π0: Our First Generalist Policy — Physical Intelligence · October 31, 2024
- Open Sourcing π0 — Physical Intelligence · February 4, 2025
- π0.5: A VLA with Open-World Generalization — Physical Intelligence · April 22, 2025
- OpenVLA repository — OpenVLA project · accessed July 11, 2026
- Octo project — Octo project · accessed July 11, 2026
- RT-2 — Google DeepMind · July 28, 2023
- Helix 02 — Figure AI · 2026
- Redwood AI — 1X Technologies · accessed July 11, 2026
- Skild Brain — Skild AI · accessed July 11, 2026
- Open X-Embodiment repository — Google DeepMind and collaborators · accessed July 11, 2026
Related TechniaHQ guides
Official image recommendations
- A layered diagram from robot sensors through reasoning, world modeling, action policy and motor control.
Physical AI model stack connecting perception, reasoning, planning, action and control — TechniaHQ original graphic - NVIDIA Isaac GR00T development pipeline from teleoperation and simulation to deployment.
NVIDIA Isaac GR00T workflow for humanoid data, policy training and robot deployment — NVIDIA - Gemini Robotics performing dexterous manipulation on supported robot hardware.
Robot controlled by Gemini Robotics manipulating household objects — Google DeepMind - Physical Intelligence π0 policy operating multiple robot embodiments.
Physical Intelligence pi-zero generalist robot policy on several robot platforms — Physical Intelligence - Open-access matrix for code, weights, data and commercial license.
Physical AI model openness matrix comparing code, weights, training data and licenses — TechniaHQ original graphic
Fact-check report
Verified: July 11, 2026
Confirmed
- GR00T 1.7, Gemini Robotics-ER 1.6 and current access states are tied to official pages.
- World models, simulators and action policies are separated by output role.
- Open access is split into code, weights, data and license.
Not confirmed or incomplete
- Training dataset composition is incomplete for several closed models.
- No common real-robot benchmark covers every model in the catalog.
- Independent reproduction of proprietary home and humanoid policies is unavailable.
Fast-changing information
- Model versions, licenses, preview access and public checkpoints can change within weeks.