Learning Humanoid Locomotion with Perceptive Internal Model

Junfeng Long*,1, Junli Ren*,1,2, Moji Shi*,1,, Zirui Wang1,3, Tao Huang1,4, Ping Luo1,2, Jiangmiao Pang†,1,
1Shanghai AI Laboratory, 2The University of Hong Kong, 3Zhejiang University, 4Shanghai Jiao Tong University
Paper Code arXiv

Our locomotion policy can drive humanoid robots to walk across difficult terrains. It is powered by Perceptive Internal Model that uses the robot’s historical internal states to simulate implicit response and estimate the robot's velocity with the integrating of perceptive information so that the policy can estimate disturbances from environmental dynamics.

Abstract

In contrast to quadruped robots that can navigate diverse terrains using a blind policy, humanoid robots require accurate perception for stable locomotion due to their high degrees of freedom and inherently unstable morphology. However, incorporating perceptual signals often introduces additional disturbances to the system, potentially reducing its robustness, generalizability, and efficiency. This paper presents the Perceptive Internal Model (PIM), which relies on onboard, continuously updated elevation maps centered around the robot to perceive its surroundings. We train the policy using ground-truth obstacle heights surrounding the robot in simulation, optimizing it based on the Hybrid Internal Model (HIM), and perform inference with heights sampled from the constructed elevation map. Unlike previous methods that directly encode depth maps or raw point clouds, our approach allows the robot to perceive the terrain beneath its feet clearly and is less affected by camera movement or noise. Furthermore, since depth map rendering is not required in simulation, our method introduces minimal additional computational costs and can train the policy in 3 hours on an RTX 4090 GPU. We verify the effectiveness of our method across various humanoid robots, various indoor and outdoor terrains, stairs, and various sensor configurations. Our method can enable a humanoid robot to continuously climb stairs and has the potential to serve as a foundational algorithm for the development of future humanoid control methods.

Framework Overview

Within PIM, we integrate perceptive information into the state predictor to achieve more comprehensive and accurate state prediction. A LiDAR-based elevation map serves as the perception model, enabling more precise perception alignment between simulation and real-world environments.

The robot walks throught human-level stairs.

The robot steps on a 50cm platform and then steps over a 70cm gap.

The robot walks throught different outdoor terrains.

Fourier GR1 walks throught human-level soft stairs.

Fourier GR1 outdoor walk.

Acknowledgement

This work was supported by Shanghai AI Laboratory. We want to thank Rongzhang Gu, Jia Zeng, and Xuekun Jiang for their help in the creating of demo video. We also want to thank the colleagues from Fourier Intelligence for their help with the experiments of the GR1 robot.

BibTeX


        @misc{long2024learninghumanoidlocomotionperceptive,
          title={Learning Humanoid Locomotion with Perceptive Internal Model}, 
          author={Junfeng Long and Junli Ren and Moji Shi and Zirui Wang and Tao Huang and Ping Luo and Jiangmiao Pang},
          year={2024},
          eprint={2411.14386},
          archivePrefix={arXiv},
          primaryClass={cs.RO},
          url={https://arxiv.org/abs/2411.14386}, 
        }