Whole-Body Loco-Manipulation: Walking, Balance and Manipulation as One Control Problem
How humanoids coordinate feet, center of mass, arms and contact forces during real loco-manipulation, with systems, evidence, failures and safety limits.
Introduction
A humanoid that lifts a box changes its own balance before it takes a single step. The payload moves the combined center of mass, the arms create reaction forces and each foot contact limits which accelerations are physically possible. Treating walking and manipulation as separate modules can therefore produce a stable gait that fails as soon as the robot reaches, pushes or carries.
Whole-body loco-manipulation treats locomotion, balance and object interaction as one coupled control problem. This article explains floating-base dynamics, contact planning, center-of-mass regulation, inverse dynamics, model predictive control and reinforcement learning. It examines evidence from real humanoids and separates autonomous policies from teleoperation and motion imitation. Concrete tasks include carrying loads, opening doors, moving containers, reaching toward the floor, pushing carts and manipulating while recovering from disturbances.
Key findings
- The arms, payload, torso and feet share one momentum budget; manipulation can destabilize locomotion even when each subsystem works alone.
- Whole-body controllers enforce contact, torque and balance constraints, while learned policies can improve adaptation and task coverage.
- Teleoperated whole-body demonstrations prove hardware coordination and data collection, not autonomous task execution.
- Reported success depends strongly on terrain, payload, sequence length, external support and whether falls or resets are counted.
- Real deployment needs recovery, collision limits and safe failure states, not only a successful nominal trajectory.
Verified whole-body loco-manipulation systems
Systems are compared by demonstrated role, not by a single success score. Teleoperation, imitation and autonomous execution are labeled separately.
| System | Robot | Task evidence | Control approach | Real or simulation | Autonomy classification | Main limitation |
|---|---|---|---|---|---|---|
| HOVER | Humanoid platforms in NVIDIA research | Whole-body motion tracking and control | Hierarchical policy with learned control | Simulation and reported real transfer | Autonomous policy execution after training | Task-level manipulation evidence is narrower than motion tracking |
| WholeBodyVLA | Humanoid platform reported by authors | Language-conditioned whole-body tasks | VLA plus whole-body action representation | Reported real robot | Supervised autonomous in experiments | Recent system with limited independent replication |
| DemoHLM | Humanoid platform reported by authors | Demonstration-driven loco-manipulation | Human demonstration learning and policy adaptation | Reported real robot | Autonomous execution from demonstrations | Training and test distributions remain controlled |
| HOMIE | Humanoid robot | Real-time whole-body teleoperation | Human motion retargeting and stabilization | Real robot | Teleoperated | Does not establish autonomous task planning |
| Helix 02 | Figure humanoid | Long mobile manipulation sequence | Company VLA and whole-body control stack | Real robot company demo | Autonomy claimed by company; supervision details limited | No independent protocol or full failure log |
| HumanoidMimicGen | Humanoid platforms reported by authors | Generated whole-body manipulation demonstrations | Data generation and imitation learning | Simulation and reported transfer | Policy execution after synthetic demonstration generation | Transfer quality depends on reconstruction and contact fidelity |
Definition: why locomotion and manipulation are coupled
A legged robot has a floating base: its torso is not bolted to the world, so every arm or leg acceleration creates reaction forces that must be balanced through contacts. Loco-manipulation combines base movement and physical interaction with objects. Whole-body loco-manipulation additionally coordinates all available joints, contacts and momentum rather than treating the legs as transport for an independent arm.
A wheeled mobile manipulator also couples base and arm motion, but a humanoid faces changing foot contacts and a smaller support polygon. A stationary dual-arm task is manipulation, not loco-manipulation. Walking to a table and stopping before reaching is sequential locomotion and manipulation; carrying, pushing or reaching while stepping is the coupled case.
How whole-body control works
State estimation fuses joint encoders, inertial measurements, foot contact estimates and vision. A planner chooses footholds, object contacts and task goals. The controller regulates center of mass, torso orientation, swing feet, hands and contact forces under joint, torque and friction constraints. Inverse dynamics computes joint torques or accelerations consistent with the full rigid-body model.
Model predictive control optimizes several future steps and contacts, then replans as the robot moves. Reinforcement-learning policies can learn residual control, contact transitions or full-body behaviors from simulation. Hybrid systems often use a learned policy for motion generation and a model-based layer for stabilization, torque limits and collision constraints.
Center of mass, zero moment point and support
The center of mass summarizes mass distribution, but balance also depends on momentum and contact forces. The zero moment point is useful for planned walking on approximately flat support, while dynamic motions may require capture-point or centroidal-momentum reasoning. Reaching with both arms can shift the center of mass beyond the current support polygon unless the robot steps, leans or creates an additional hand contact.
Contact planning and force control
A task may involve feet, hands, forearms and the manipulated object as contacts. The planner must decide when each contact is created or released. Force control matters for doors, tools and carts because pose control alone can generate excessive force when geometry is uncertain. Contact estimates also fail under slip, soft surfaces or delayed sensing.
Key systems and what their demonstrations prove
HOVER studies robust whole-body control and motion tracking, providing a reusable layer for humanoid behaviors. WholeBodyVLA and DemoHLM move toward instruction- or demonstration-conditioned loco-manipulation, but public evidence remains tied to specific robots and task sets. HOMIE demonstrates high-quality whole-body teleoperation and is valuable for collecting coordinated data; the human still supplies task decisions and motion intent.
Figure’s Helix 02 demonstration showed a company-reported multi-minute mobile manipulation sequence on a humanoid. That evidence is more informative than a cut of isolated motions, but it is not an independent deployment study and does not disclose every intervention, reset or failure. HumanoidMimicGen explores scalable demonstration generation, with transfer quality bounded by the fidelity of retargeted motion and simulated contacts.
Evidence from real robots
A complete evaluation should report robot mass, terrain, payload, speed, task duration, number of trials and perturbations. Carrying a light box on a flat laboratory floor is different from moving an unknown load across a ramp. Opening a known door with a prepositioned handle is different from locating and manipulating doors of varied geometry.
Simulation enables thousands of falls and contact variations without damaging hardware, but transfer exposes actuator delay, joint friction, structural compliance and imperfect state estimation. Real tests are strongest when they include pushes, object shifts, missed contacts and recovery rather than replaying one successful trajectory. Human supervision should be reported even when the learned policy generates every nominal action.
Why walking and manipulation cannot be evaluated separately
A locomotion benchmark may reward speed while ignoring how the gait behaves with arms extended or a payload held asymmetrically. A manipulation benchmark may assume a fixed base and stable camera. Combining two high-scoring modules can fail because the arm planner requests motion that violates balance constraints or the gait moves the camera faster than the perception system can track.
The relevant unit is the complete task. For carrying, measure pickup, transition to walking, transport, placement and recovery. For a door, measure approach, handle contact, force application, stepping through and avoiding collision. Success should include the absence of unsafe contact, not only task completion.
Failure modes
Foot-slip or a false contact estimate can invalidate the planned support. Payload mass may be wrong, shifting momentum. An arm collision can rotate the torso and trigger a step. Perception delay causes the hand target to move relative to the body. Learned policies can exploit simulation artifacts, while model-based controllers can fail when contact geometry differs from the model.
Recovery may be blocked by the object itself: a robot holding a crate cannot freely swing both arms, and dropping the load may be unsafe. Controllers therefore need priorities and fallback states, such as placing the object down, widening stance, releasing contact or stopping before joint or force limits are exceeded.
Practical applications and maturity
Credible applications include moving standardized containers, tending workstations separated by short walks, pushing known carts and handling light materials in controlled facilities. These tasks can use mapped floors, restricted human access and predefined object interfaces while still benefiting from a humanoid’s reach and ability to use human-designed spaces.
Irregular homes, construction sites and emergency response remain experimental. They combine unknown terrain, moving people, deformable objects and high consequence failures. For those settings, whole-body capability must be paired with validated perception, safe contact forces, protective hardware and a recovery strategy that has been tested under realistic disturbances.
Limitations and missing information
- Published systems use different robots, payloads, terrains and task definitions, preventing an absolute ranking.
- Many videos do not disclose failed trials, resets, remote supervision or safety personnel.
- Whole-body teleoperation is often confused with autonomy even though a human supplies continuous intent.
- Simulation success may rely on actuator and contact models that do not match the physical robot.
- Long-duration wear, thermal limits and repeated fall recovery are rarely included in academic evaluations.
Conclusion
Whole-body loco-manipulation is the control problem that appears when a humanoid must move and physically interact at the same time. Foot placement, center of mass, arm motion, payload and contact forces cannot be optimized independently because each changes the robot’s momentum and feasible support.
Research systems now demonstrate increasingly complex sequences on real hardware, using model-based control, reinforcement learning, imitation and multimodal policies. The evidence is still task- and robot-specific, and teleoperation remains important for data collection and difficult demonstrations. A mature evaluation should cover the complete task, disturbances, recovery, force limits and the conditions under which human intervention occurs. Walking speed and grasp success alone do not describe whether a humanoid can safely carry, push or manipulate while remaining balanced.
Frequently asked questions
What is whole-body loco-manipulation?
Whole-body loco-manipulation is coordinated control of locomotion, balance and object interaction using the robot’s full body. The controller considers foot contacts, torso motion, arms, hands and payload together. It differs from walking to a fixed pose and then manipulating with a stationary base because the motions and forces overlap in time.
Why does moving an arm affect humanoid balance?
The torso of a humanoid is a floating base. Accelerating an arm produces reaction momentum, and holding a payload shifts the combined center of mass. The feet must generate compensating contact forces or the robot must step. Fast reaches, long lever arms and asymmetric loads therefore change balance even when the feet do not move.
Is whole-body teleoperation autonomous?
No. Whole-body teleoperation maps a human operator’s motion or commands onto the robot while stabilization software keeps the machine feasible. It proves that hardware and controllers can coordinate complex motion and can produce useful training data. Autonomous execution requires the robot to perceive, select and complete the task without continuous human motion input.
What is floating-base dynamics?
Floating-base dynamics models a robot whose main body is not fixed to the ground. The base position and orientation change according to joint motion, gravity and external contacts. Humanoids need this model because feet alternately create and release support, and arm forces can move the torso or change the required ground reaction forces.
Can reinforcement learning replace whole-body control?
Reinforcement learning can produce effective whole-body policies, especially after large-scale simulation training, but deployed systems still need state estimation, actuator limits, collision checks and emergency behavior. Many architectures combine learned motion generation with model-based stabilization or safety constraints. The balance depends on task complexity, hardware reliability and the evidence required for certification.
How should loco-manipulation be benchmarked?
A benchmark should measure the complete sequence, including approach, contact, transport or manipulation, release and recovery. It should report terrain, payload, duration, trials, failures, human intervention and safety violations. Separate walking and grasp scores are insufficient because the difficult part is maintaining control when the two behaviors interact.
Sources and methodology
The review separates autonomous policy execution, demonstration learning, simulation transfer and teleoperation. Systems were included when the public material involved coordinated legged movement and manipulation, not merely a walking robot with independently posed arms.
Numeric payloads and success rates are omitted when protocols are incompatible or not publicly reported. Company demonstrations are labeled as company-reported and are not treated as independent deployment evidence.
- HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots — NVIDIA Research · 2025 · accessed July 11, 2026
- WholeBodyVLA official repository — OpenDriveLab · Accessed July 11, 2026
- DemoHLM: Learning Humanoid Loco-Manipulation from Demonstrations — Research collaboration · October 2025 · accessed July 11, 2026
- HOMIE: Humanoid Whole-Body Teleoperation — Robotics: Science and Systems · RSS 2025 · accessed July 11, 2026
- Task and Motion Planning for Humanoid Loco-Manipulation — Research collaboration · August 2025 · accessed July 11, 2026
- Helix 02: Full-Body Autonomy — Figure AI · January 27, 2026 · accessed July 11, 2026
- HumanoidMimicGen — Research collaboration · May 2026 · accessed July 11, 2026
Related TechniaHQ guides
Official image recommendations
- TechniaHQ force-and-contact diagram for a humanoid carrying a box while stepping
Humanoid carrying a box with arrows for foot forces, payload, center of mass and momentum — TechniaHQ original based on whole-body control literature - Official HOVER whole-body control demonstration
Humanoid tracking a dynamic whole-body motion using the HOVER controller — NVIDIA Research - HOMIE whole-body teleoperation experiment
Humanoid mirroring an operator during coordinated arm and leg motion — RSS 2025 authors - Maturity matrix for walking, carrying, pushing, door opening and disturbance recovery
Matrix comparing task complexity, contact uncertainty and public real-robot evidence — TechniaHQ original - Whole-body control stack from perception and task goal to contacts, MPC, inverse dynamics and actuator safety
Block diagram of a humanoid loco-manipulation controller with feedback and fallback states — TechniaHQ original
Fact-check report
Verified: July 11, 2026
Confirmed
- Each included example coordinates locomotion or floating-base motion with manipulation.
- Teleoperation is explicitly separated from autonomous policy execution.
- Simulation-only and real-robot evidence are identified.
Not confirmed or incomplete
- Independent success rates and long-duration reliability are unavailable for several recent systems.
- Complete intervention and failure logs for company demonstrations are not public.
- Payload and speed values are not comparable across different robots and protocols.
Fast-changing information
- Recent 2026 whole-body systems may release code, datasets or additional trials after verification.
- Company autonomy descriptions may change as deployment programs mature.