[00:00] Introduction to Hover: Neural Whole Body Controller for Humanoids

[00:15] Problem: Current controllers lack versatility across tasks

[00:50] Human motion imitation as a unified control approach

[01:23] Policy distillation: Learning from an oracle policy

[02:01] Command space: Kinematic, joint angle, and root tracking modes

[02:34] Motion retargeting: From human data to robot movements

[03:09] Performance comparison with specialist policies

[03:43] Real-world testing on Unitree H1 robot

[04:15] Comparison with MHC and Masked Mimic approaches

[04:49] Future work and current limitations

[05:18] Reward function design and components

[06:02] D-Agger advantages in policy learning

[06:33] Domain randomization for sim-to-real transfer

[07:06] Conclusions on Hover's contributions

Authors: Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, Yuke Zhu

Affiliations: NVIDIA, CMU, UC Berkeley, UT Austin, UC San Diego

Abstract: Humanoid whole-body control requires adapting to diverse tasks such as navigation, loco-manipulation, and tabletop manipulation, each demanding a different mode of control. For example, navigation relies on root velocity tracking, while tabletop manipulation prioritizes upper-body joint angle tracking. Existing approaches typically train individual policies tailored to a specific command space, limiting their transferability across modes. We present the key insight that full-body kinematic motion imitation can serve as a common abstraction for all these tasks and provide general-purpose motor skills for learning multiple modes of whole-body control. Building on this, we propose HOVER (Humanoid Versatile Controller), a multi-mode policy distillation framework that consolidates diverse control modes into a unified policy. HOVER enables seamless transitions between control modes while preserving the distinct advantages of each, offering a robust and scalable solution for humanoid control across a wide range of modes. By eliminating the need for policy retraining for each control mode, our approach improves efficiency and flexibility for future humanoid applications.

Link: https://hover-versatile-humanoid.github.io/