KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills
KungfuBot employs physics-based whole-body control with adaptive reward tuning, achieving 30% lower tracking error than state-of-the-art, successfully deployed on Unitree G1.
Key Findings
Methodology
This paper introduces the KungfuBot framework, integrating multi-step motion processing and an adaptive tracking mechanism within a bi-level optimization and asymmetric actor-critic architecture. The motion processing pipeline extracts human motions from videos using SMPL parameter estimation via GVHMR, followed by physics-based filtering based on CoM-CoP stability metrics, contact mask correction, and retargeting through inverse kinematics. The imitation stage employs a reward function with exponential form, dynamically adjusting the tracking tolerance via bi-level optimization to handle varying motion difficulties. The training leverages an asymmetric actor-critic network with reward vectorization and privileged information, enabling precise policy learning. Extensive experiments in simulation and real-world deployment on Unitree G1 demonstrate significant improvements in tracking accuracy and motion expressiveness, especially for high-dynamic skills like Kungfu and dance.
Key Results
- In simulation, PBHC reduces the mean per body position error (Eg-mpbpe) by over 30%, achieving 53.25mm compared to 233.54mm of baseline OmniH2O on complex motions. The approach outperforms existing methods like ExBody2 and OmniH2O across all difficulty levels, with the lowest errors in joint and velocity metrics. In real-world tests, the trained policies produce stable, expressive motions on the Unitree G1 robot, successfully executing complex high-speed actions such as spinning and jumping with minimal deviation. The adaptive tracking mechanism further refines performance, with errors decreasing progressively during training, validating the effectiveness of the dynamic reward adjustment.
- The significance of this work lies in its ability to bridge the gap between human motion capture data and physically feasible robot control, especially for highly dynamic and agile movements. It advances the state-of-the-art by integrating physics-based motion filtering, adaptive reward tuning, and efficient policy training, enabling robots to imitate complex human behaviors with high fidelity. This has profound implications for robotics research, industrial automation, entertainment, and rehabilitation, where realistic and robust motion execution is critical.
- Technically, the paper introduces several innovations: a physics-based multi-step motion filtering pipeline ensuring physical plausibility; a bi-level optimization framework for adaptive reward tuning based on tracking error; an asymmetric actor-critic architecture leveraging privileged information and reward vectorization; and a comprehensive end-to-end training process that combines motion extraction, filtering, retargeting, and reinforcement learning. These contributions collectively enable high-precision imitation of highly dynamic motions, surpassing existing methods in accuracy and robustness.
Limitations
- The current approach relies heavily on high-quality motion capture and video data, which may not be available or accurate in all scenarios, limiting its applicability in uncontrolled environments. Additionally, the computational cost of the multi-step filtering, retargeting, and bi-level optimization processes is high, restricting real-time deployment in resource-constrained settings. The method has been validated primarily on the Unitree G1 platform; generalization to other robot morphologies or multi-robot systems remains to be explored. Furthermore, extreme dynamic motions or highly cluttered environments may challenge the robustness of the physics-based filtering and adaptive mechanisms, necessitating further improvements.
- The reliance on simulation-to-real transfer techniques like domain randomization, while effective, does not fully eliminate the sim-to-real gap, especially under unpredictable real-world disturbances. The current framework also assumes accurate initial motion extraction; errors in video-based motion estimation can propagate through the pipeline, affecting overall performance. Lastly, the approach focuses on single-task learning; extending it to multi-task or lifelong learning scenarios poses additional challenges.
Future Work
Future research will focus on integrating multimodal sensory inputs, such as vision and force sensing, to improve motion robustness and environmental awareness. Developing more efficient algorithms for real-time filtering and retargeting will be crucial for practical deployment. Extending the framework to multi-robot coordination and multi-task learning will broaden its applicability. Additionally, incorporating online adaptation and continual learning techniques can enable robots to refine skills in dynamic real-world settings, further closing the gap between simulation and reality. Exploring unsupervised or weakly supervised motion extraction methods may also reduce dependence on high-quality video data, making the system more scalable.
AI Executive Summary
The quest to endow humanoid robots with the ability to imitate complex, high-speed human motions has long been hindered by the limitations of existing control algorithms. Traditional approaches excel at smooth, slow movements but falter when faced with dynamic skills such as martial arts or dance, primarily due to physical infeasibility and tracking inaccuracies. This gap has constrained the deployment of humanoid robots in entertainment, assistive, and industrial domains where expressive, high-speed actions are essential.
Enter KungfuBot, a novel physics-based whole-body control framework designed to learn and execute highly dynamic human skills. The core innovation lies in its two-stage pipeline: first, a multi-step motion processing system extracts, filters, corrects, and retargets human motions from videos, ensuring physical plausibility through physics-based metrics like CoM-CoP stability and contact mask correction. Second, an adaptive motion imitation mechanism employs a bi-level optimization model to dynamically tune the tracking reward, balancing accuracy and smoothness across varying motion complexities.
This approach is further empowered by an asymmetric actor-critic reinforcement learning architecture. The critic leverages privileged information and reward vectorization for precise value estimation, while the actor relies solely on local observations, facilitating stable training and effective policy learning. Extensive experiments in simulation demonstrate that PBHC significantly reduces tracking errors—by over 30% compared to prior methods—especially on complex motions like Kungfu and acrobatics. The policies trained in simulation are directly deployed on the Unitree G1 robot, where they produce stable, expressive behaviors, including spinning, jumping, and martial arts moves.
The broader impact of this work is substantial. It bridges the gap between human motion capture data and physically feasible robot control, enabling robots to perform high-speed, naturalistic actions previously thought unattainable. This paves the way for advanced human-robot interaction, entertainment, and rehabilitation applications, where realistic motion is crucial. Despite these advances, challenges remain, such as reducing computational costs, improving robustness in cluttered environments, and generalizing across different robot platforms. Future efforts will focus on multimodal data integration, online adaptation, and multi-robot coordination, aiming to realize truly autonomous, expressive humanoid robots capable of learning a broad repertoire of skills in complex real-world scenarios.
Deep Analysis
Background
Humanoid motion imitation has evolved from rule-based control to data-driven deep learning methods, with significant progress in smooth, low-speed movements. Early works like DeepMimic and Motion Imitation utilized motion capture datasets such as CMU Mocap and AMASS, combined with reinforcement learning to generate natural motions. However, these methods struggle with high-speed, complex skills due to physical infeasibility, limited data diversity, and the challenge of maintaining stability under dynamic conditions. Recent advances incorporate physics-based filtering and domain randomization to improve realism and transferability, but high dynamic motions remain a challenge due to the difficulty in balancing accuracy, stability, and computational efficiency.
Core Problem
Current motion imitation techniques predominantly focus on smooth, slow movements, leaving a significant gap in high-speed, highly dynamic skills like martial arts or acrobatics. The core issues include the physical infeasibility of raw motion data, difficulty in modeling complex contact interactions, and the challenge of maintaining stability during rapid movements. Moreover, existing reward mechanisms lack adaptability, often leading to poor tracking performance on difficult motions. This limits the practical deployment of humanoid robots in scenarios requiring agility and expressiveness, hindering their potential in entertainment, assistive robotics, and industrial automation.
Innovation
This work introduces several key innovations. First, a physics-based motion filtering pipeline ensures extracted human motions adhere to biomechanical constraints, filtering out unphysical movements. Second, an adaptive reward mechanism based on exponential functions and bi-level optimization dynamically adjusts tracking tolerance, enabling the policy to handle motions of varying difficulty. Third, the asymmetric actor-critic architecture leverages privileged information and reward vectorization, significantly improving value estimation accuracy and training stability. These innovations collectively enable high-fidelity imitation of complex, high-speed motions, surpassing the limitations of prior methods that relied on fixed reward parameters and simplistic motion filtering.
Methodology
- �� Motion extraction: Use GVHMR to estimate SMPL parameters from monocular videos, producing detailed pose and shape data. • Motion filtering: Calculate CoM and CoP projections to assess stability, filtering out unstable sequences based on a threshold. • Contact mask estimation: Analyze ankle displacement to determine foot-ground contact, refining motion quality. • Motion correction: Apply vertical offsets based on contact masks to eliminate floating artifacts, smoothing with EMA. • Retargeting: Use differentiable inverse kinematics to map processed motions onto the robot, respecting joint limits. • Reward design: Define an exponential reward function that penalizes tracking errors, with a tunable parameter σ. • Adaptive mechanism: Implement bi-level optimization to select σ that minimizes overall tracking error, updating σ online via EMA. • RL training: Employ an asymmetric actor-critic network with reference state initialization and domain randomization, facilitating sim-to-real transfer and robust policy learning.
Experiments
The experimental setup involves training policies in IsaacGym with a diverse set of high-dynamic motions, categorized into easy, medium, and hard difficulty levels. Each policy is trained with three random seeds, evaluated over 1000 episodes, and compared against baselines like OmniH2O and ExBody2. Metrics such as Eg-mpbpe, Empbpe, and Empjpe quantify tracking accuracy. The results show that PBHC achieves over 30% lower errors, with Eg-mpbpe dropping from 233.54mm to 53.25mm in complex motions. Ablation studies confirm the effectiveness of the adaptive tracking mechanism, with errors decreasing progressively during training. The policies are then transferred to the Unitree G1 robot, where they demonstrate stable, expressive, and high-speed motions, validating the sim-to-real transfer capability.
Plain Language Accessible to non-experts
想象你在一家工厂里，工人们每天都在做各种动作，比如搬东西、跳舞、甚至表演武术。工厂的机器人就像这些工人，他们也需要学习这些动作，但问题是，工厂里的机器人和人类的身体完全不同，直接模仿人类的动作会出现很多问题，比如动作不自然、容易摔倒。为了让机器人学会像人一样灵活、自然地做动作，科学家们设计了一套特别的方法。
首先，他们用摄像头拍摄人类的动作，然后用特殊的算法把这些动作转化成机器人能理解的形式。这就像是把人类的舞蹈动作翻译成机器人能跳的舞步。接着，他们会检查这些动作是否符合物理规律，比如是否会让机器人失衡或摔倒。如果发现不合理的动作，就会进行修正，确保动作既漂亮又安全。
然后，科学家们还设计了一个智能系统，能根据机器人追踪动作的准确程度，自动调整学习的难度。就像老师根据学生的表现，调整作业的难度一样。这样，机器人可以逐步学会更复杂、更快的动作。整个学习过程就像是教机器人跳一支复杂的功夫舞，每次跳完后，系统会告诉它哪里还可以做得更好，然后再继续练习。
经过反复训练，机器人不仅能模仿人类的高难度动作，还能在真实的机器人身上表现出来，比如Unitree G1机器人。它可以做出像武术一样的动作，甚至能跳舞、旋转，动作自然流畅。这项技术的意义在于，让机器人变得更聪明、更灵活，将来它们可以在工厂、舞台甚至康复中心帮忙，完成各种复杂任务。虽然目前还存在一些挑战，比如在极端环境下的表现，但这项研究无疑为机器人自主学习高难度动作打开了新的大门。
ELI14 Explained like you're 14
想象你在学校里学舞蹈，你需要跟老师学动作，然后不断练习，直到动作变得漂亮又自然。可是，假如你的身体和老师完全不一样，比如老师很高，你很矮，直接模仿可能会出错，还可能摔倒。科学家们遇到的情况也差不多，他们想让机器人学会像人一样跳舞、打功夫，但机器人和人类的身体结构不同，直接模仿效果不好。
所以，他们设计了一套特别的学习方法。首先，用摄像头拍摄人类的动作，然后用电脑算法把动作转成机器人可以理解的指令，就像把舞蹈动作翻译成机器人能跳的舞步一样。接着，他们会检查这些动作是否合理，比如是否会让机器人失衡或摔倒，如果发现问题，就会帮它修正，确保动作既漂亮又安全。
更厉害的是，他们还设计了一个智能系统，可以根据机器人追踪动作的准确程度，自动调整学习的难度。就像老师知道你跳得还不够好，就会给你一些简单的动作，等你熟练了，再挑战更难的动作。这样，机器人可以逐步学会更复杂、更快的动作。
经过不断的练习，这个机器人不仅能模仿人类的动作，还能在真实的机器人身上表现出来，比如Unitree G1机器人。它可以做出像武术一样的动作，甚至能跳舞、旋转，动作非常自然。这项技术的意义在于，让机器人变得更聪明、更灵活，将来它们可以在工厂、舞台甚至康复中心帮忙，完成各种复杂任务。虽然还存在一些挑战，比如在极端环境下的表现，但这项研究为机器人自主学习高难度动作提供了新的可能。
Glossary
Physics-Based Motion Filtering (物理基础运动过滤)
一种利用物理原理筛选运动数据的方法，确保运动符合力学和生物力学规律，避免不合理动作。
用于过滤从视频提取的运动序列，确保其物理合理性。
Bi-level Optimization (双层优化)
一种优化策略，包含两个层级：上层调节参数以最大化外部目标，下层训练策略以最大化内部奖励。
用于动态调节追踪容差，提升学习适应性。
Actor-Critic (演员-评论家)
一种强化学习架构，演员生成动作，评论家估算价值，优化策略。
本文采用偏置的多输出架构，提升训练稳定性。
Reward Vectorization (奖励向量化)
将多个奖励信号分开估算，避免单一标量带来的信息丢失，提升训练效果。
增强价值估算的精度，改善训练稳定性。
SMPL模型 (Skinned Multi-Person Linear Model)
一种参数化人体模型，用于表示人体姿态和形状，便于运动分析和重定向。
从视频中提取运动参数，作为运动模仿的基础。
Inverse Kinematics (逆运动学)
通过已知末端位置反求关节角度的算法，确保运动的可执行性。
用于将运动重定向到机器人模型。
Domain Randomization (域随机化)
在训练中随机变化环境参数，提高模型的泛化能力，增强实机迁移能力。
实现模拟到实机的零样本迁移。
Exponential Reward Function (指数奖励函数)
一种基于误差的奖励函数，随误差变化呈指数衰减，平衡追踪精度与训练稳定性。
设计追踪奖励，调节追踪容差。
Reference State Initialization (参考状态初始化)
从参考运动中随机采样状态，初始化机器人状态，加快训练效率。
用于多阶段训练和提高样本效率。
Motion Retargeting (运动重定向)
将提取或生成的运动数据映射到目标机器人模型，确保动作的可行性。
通过逆运动学实现。
Open Questions Unanswered questions from this research
- 1 尽管本研究在模拟和实机上取得了显著成果，但在极端环境或复杂场景下的运动鲁棒性仍需提升。未来需要结合感知与环境理解，增强模型的适应能力。此外，算法的计算成本较高，实时性仍是挑战。如何在保证运动质量的同时降低计算负担，是未来的重要研究方向。同时，跨平台迁移和多机器人协作的能力仍待验证，推动多模态、多任务的高动态技能学习成为下一步重点。
Applications
Immediate Applications
High-end Human-Robot Interaction and Entertainment
Achieving realistic robot dance and martial arts performances for stage shows or VR interactions, enhancing user experience.
Rehabilitation Robots for Assistive Training
Using high-dynamic motion imitation to assist in patient recovery, helping regain motor functions.
Industrial Robots for Complex Tasks
Executing precise, high-speed complex motions in manufacturing, improving productivity and flexibility.
Long-term Vision
Autonomous Multi-Skill Robots
Future robots capable of self-learning diverse high-dynamic skills for various environments, reducing manual programming.
Human-Robot Collaborative Systems
Enabling seamless collaboration between humans and robots in complex tasks, transforming industries like manufacturing and healthcare.
Abstract
Humanoid robots are promising to acquire various skills by imitating human behaviors. However, existing algorithms are only capable of tracking smooth, low-speed human motions, even with delicate reward and curriculum design. This paper presents a physics-based humanoid control framework, aiming to master highly-dynamic human behaviors such as Kungfu and dancing through multi-steps motion processing and adaptive motion tracking. For motion processing, we design a pipeline to extract, filter out, correct, and retarget motions, while ensuring compliance with physical constraints to the maximum extent. For motion imitation, we formulate a bi-level optimization problem to dynamically adjust the tracking accuracy tolerance based on the current tracking error, creating an adaptive curriculum mechanism. We further construct an asymmetric actor-critic framework for policy training. In experiments, we train whole-body control policies to imitate a set of highly-dynamic motions. Our method achieves significantly lower tracking errors than existing approaches and is successfully deployed on the Unitree G1 robot, demonstrating stable and expressive behaviors. The project page is https://kungfubot.github.io.
References (20)
Perpetual Humanoid Control for Real-time Simulated Avatars
Zhengyi Luo, Jinkun Cao, Alexander W. Winkler et al.
SMPL
M. Loper, Naureen Mahmood, J. Romero et al.
Learning Agile Robotic Locomotion Skills by Imitating Animals
Xue Bin Peng, Erwin Coumans, Tingnan Zhang et al.
AMASS: Archive of Motion Capture As Surface Shapes
Naureen Mahmood, N. Ghorbani, N. Troje et al.
Institutional Review Board
G. Pearson
Reducing Footskate in Human Motion Reconstruction with Ground Contact Constraints
Yuliang Zou, Jimei Yang, Duygu Ceylan et al.
Proximal Policy Optimization Algorithms
John Schulman, Filip Wolski, Prafulla Dhariwal et al.
RT-1: Robotics Transformer for Real-World Control at Scale
Anthony Brohan, Noah Brown, Justice Carbajal et al.
Robust and Versatile Bipedal Jumping Control through Reinforcement Learning
Zhongyu Li, Xue Bin Peng, P. Abbeel et al.
3D Human Pose Estimation via Intuitive Physics
Shashank Tripathi, Lea Muller, C. Huang et al.
Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomization
Luigi Campanaro, Siddhant Gangapurwala, W. Merkt et al.
Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning
Haoran He, Chenjia Bai, Kang Xu et al.
Bridging the Sim-to-Real Gap from the Information Bottleneck Perspective
Haoran He, Chenjia Bai, Hang Lai et al.
MotionGPT: Human Motion as a Foreign Language
Biao Jiang, Xin Chen, Wen Liu et al.
An Introduction to Bilevel Optimization: Foundations and applications in signal processing and machine learning
Yihua Zhang, Prashant Khanduri, Ioannis C. Tsaknakis et al.
Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Training
Haoran He, Chenjia Bai, Ling Pan et al.
Expressive Whole-Body Control for Humanoid Robots
Xuxin Cheng, Yandong Ji, Junming Chen et al.
Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation
Tairan He, Zhengyi Luo, Wenli Xiao et al.
Move as you Say, Interact as you can: Language-Guided Human Motion Generation with Scene Affordance
Zan Wang, Yixin Chen, Baoxiong Jia et al.
MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors
He Zhang, Shenghao Ren, Haolei Yuan et al.
Cited By (20)
RPG: Robust Policy Gating for Smooth Multi-Skill Transitions in Humanoid Fighting
Chasing Autonomy: Dynamic Retargeting and Control Guided RL for Performant and Controllable Humanoid Running
PressMimic: Pressure-Guided Motion Capture and Control for Humanoid Robot Imitation
SMASH: Mastering Scalable Whole-Body Skills for Humanoid Ping-Pong with Egocentric Vision
Demystifying Reinforcement Learning Post-Training of Language Models
OmniXtreme: Breaking the Generality Barrier in High-Dynamic Humanoid Control
X-Loco: Towards Generalist Humanoid Locomotion Control via Synergetic Policy Distillation
Pro-HOI: Perceptive Root-guided Humanoid-Object Interaction
Rhythm: Learning Interactive Whole-Body Control for Dual Humanoids
OmniTrack: General Motion Tracking via Physics-Consistent Reference
Flip Stunts on Bicycle Robots using Iterative Motion Imitation
SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework
HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation
Tree Learning: A Multi-Skill Continual Learning Framework for Humanoid Robots
Switch: Learning Agile Skills Switching for Humanoid Robots
Re2MoGen: Open-Vocabulary Motion Generation via LLM Reasoning and Physics-Aware Refinement
Learn Weightlessness: Imitate Non-Self-Stabilizing Motions on Humanoid Robot
RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots
LineRides: Line-Guided Reinforcement Learning for Bicycle Robot Stunts
Spatial Memory for Out-of-Vision Manipulation in Vision-Language-Action