VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids
VOFA integrates vision-based policies with force-adaptive control, achieving over 90% success in pushing unknown objects up to 17kg in real and simulated environments.
Key Findings
Methodology
VOFA employs a hierarchical architecture: a high-level vision-based policy trained via teacher-student reinforcement learning with domain randomization, outputs goal-conditioned commands; a low-level force-adaptive controller based on FALCON ensures robustness across diverse object masses and ground frictions. The high-level policy processes noisy onboard depth images and proprioception, using DAgger to distill from privileged information. The system incorporates multi-modal randomization during training to improve sim-to-real transfer. The low-level controller dynamically adjusts forces based on feedback, enabling stable pushing of objects up to 17kg. Extensive simulation and real-world tests demonstrate success rates exceeding 90% in simulation and 80% in real deployment.
Key Results
- In simulation, success rate reached 97.84%, with robustness across goal positions and object masses, including unseen heavy objects (up to 17kg). In real-world tests, success rates exceeded 80%, with the system effectively handling objects heavier than half the robot's weight. The force-adaptive control significantly reduced failure modes like slipping or falling, especially under high friction or mass variations. Visual randomization during training improved transfer robustness, enabling consistent performance despite perceptual noise. Ablation studies confirmed the importance of force adaptation and goal alignment rewards for high success rates. The system demonstrated long-horizon sequential pushing and recovery from external disturbances, showcasing advanced closed-loop control.
- results also showed that the object–goal alignment reward encouraged deliberate repositioning, reducing premature contact failures. The system maintained high success across diverse initial configurations and physical properties, validating the hierarchical design and training methodology.
Significance
This work advances humanoid robot capabilities in goal-directed object manipulation under uncertain physical conditions. By integrating vision-based policies with force-adaptive control, VOFA addresses longstanding challenges of robustness, generalization, and transfer from simulation to reality. Its ability to push objects heavier than half the robot’s weight demonstrates a significant leap in physical interaction robustness, opening pathways for autonomous warehouse, manufacturing, and service applications. The combination of hierarchical learning, domain randomization, and force control sets a new standard for complex manipulation tasks, reducing reliance on privileged information and enhancing adaptability in unstructured environments. This approach paves the way for more intelligent, resilient robots capable of operating safely and efficiently alongside humans.
Technical Contribution
The paper introduces a hierarchical framework combining deep vision-based policies with a force-adaptive WBC based on FALCON, trained via teacher-student reinforcement learning with domain randomization. It innovates by decoupling perception and control, enabling robust goal-conditioned pushing under unknown physical parameters. The use of DAgger distillation from privileged to onboard sensory inputs, coupled with multi-modal visual randomization, enhances sim-to-real transfer. The system’s design allows stable pushing of objects up to 17kg, surpassing prior limits. The integration of goal alignment rewards further improves long-horizon planning and correction capabilities. These contributions significantly extend the state-of-the-art in humanoid loco-manipulation, especially in unstructured, real-world scenarios.
Novelty
This work is the first to combine deep visual perception with force-adaptive whole-body control in a hierarchical, goal-conditioned framework for humanoid pushing tasks involving unknown physical properties. It uniquely employs teacher-student reinforcement learning with extensive visual randomization, enabling high success rates and robust sim-to-real transfer. Unlike prior approaches relying on privileged object state or single-modality sensing, VOFA’s layered architecture and training strategy facilitate generalization across diverse object masses and goal configurations, including pushing objects heavier than the robot itself.
Limitations
- Despite its robustness, the system's performance diminishes in highly dynamic or cluttered environments with multiple moving objects. The reliance on extensive simulation training and visual randomization increases computational costs and training complexity. The force control, while effective for moderate weights, may require further tuning for extremely heavy objects or highly slippery surfaces. Additionally, current experiments focus on box-shaped objects; more complex geometries could pose challenges. Future work should explore multi-object scenarios, real-time perception enhancements, and adaptive force control for broader applicability.
Future Work
Future directions include integrating tactile and auditory sensing to improve environmental understanding, developing multi-object manipulation strategies, and reducing training data requirements through transfer learning. Enhancing real-time perception robustness and extending to dynamic, cluttered environments will be key. Additionally, scaling the system for collaborative multi-robot tasks and exploring learning-based force control for heavier objects are promising avenues. These efforts aim to realize fully autonomous humanoid robots capable of complex, adaptive manipulation in real-world settings.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
Imagine you have a robot friend who wants to help you move heavy boxes around your house. But this robot doesn’t know exactly how heavy each box is, or if the floor is slippery. So, it uses a special pair of eyes—like a camera—to look at the box and the spot where you want it to go. Then, it carefully pushes the box, constantly adjusting how hard it pushes based on what it sees and feels. If the box is heavier or slipperier than expected, the robot senses this and changes its pushing force to keep the box steady. It’s like when you try to push a heavy sofa—you might push too hard and slip, or too soft and not move it at all. The robot learns to push just right, even with unknown weights, by looking and feeling at the same time. It can push very heavy boxes—more than half its own weight—without falling over or dropping them. This smart pushing ability makes the robot useful for tasks like organizing warehouses or helping in factories, where it needs to move different kinds of objects safely and reliably. The secret is that it combines good eyesight with a flexible force control system, so it can handle surprises and still do the job well.
Abstract
The ability to push large objects in a goal-directed manner using onboard egocentric perception is an essential skill for humanoid robots to perform complex tasks such as material handling in warehouses. To robustly manipulate heavy objects to arbitrary goal configurations, the robot must cope with unknown object mass and ground friction, noisy onboard perception, and actuation errors; all in a real-time feedback loop. Existing solutions either rely on privileged object-state information without onboard perception or lack robustness to variations in goal configurations and object physical properties. In this work, we present VOFA, a visual goal-conditioned humanoid loco-manipulation system capable of pushing objects with unknown physical properties to arbitrary goal positions. VOFA consists of a two-level hierarchical architecture with a high-level visuomotor policy and a low-level force-adaptive whole-body controller. The high-level policy processes noisy onboard observations and generates goal-conditioned commands to operate in closed loop across diverse object-goal configurations, while the low-level whole-body controller provides robustness to variations in object physical properties. VOFA is extensively evaluated in both simulation and real-world experiments on the Booster T1 humanoid robot. Our results demonstrate strong performance, achieving over 90% success in simulation and over 80% success in real-world trials. Moreover, VOFA successfully pushes objects weighing up to 17kg, exceeding half of the Booster T1's body weight.