OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning

Tairan He^*, Zhengyi Luo^*, Xialin He^*, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, Guanya Shi

Conference on Robot Learning (CoRL), 2024

paper website dataset code IEEE Spectrum

TL;DR: OmniH2O provides a universal whole-body humanoid control interface that enables diverse teleoperation and autonomy methods.

BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning

Yitang Li^*, Zhengyi Luo^*, Tonghe Zhang, Cunxi Dai, Anssi Kanervisto, Andrea Tirinzoni, Haoyang Weng, Kris Kitani, Mateusz Guzek, Ahmed Touati, Alessandro Lazaric, Matteo Pirotta^†, Guanya Shi^†

International Conference on Learning Representations (ICLR), 2026

paper website code

TL;DR: BFM-Zero enables zero-shot goal reaching, tracking, and reward optimization (any reward at test time) from one policy.

ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills

Tairan He^*, Jiawei Gao^*, Wenli Xiao^*, Yuanhang Zhang^*, Zi Wang, Jiashun Wang, Zhengyi Luo, Guanqi He, Nikhil Sobanbabu, Chaoyi Pan, Zeji Yi, Guannan Qu, Kris Kitani, Jessica Hodgins, Linxi "Jim" Fan, Yuke Zhu, Changliu Liu, Guanya Shi

Robotics: Science and Systems (RSS), 2025

paper website code

TL;DR: ASAP learns agile whole-body humanoid motions via learning a residual action model from the real world to align sim and real physics.

TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System

Yanjie Ze, Siheng Zhao, Weizhuo Wang, Angjoo Kanazawa^†, Rocky Duan^†, Pieter Abbeel^†, Guanya Shi^†, Jiajun Wu^†, C. Karen Liu^†

International Conference on Robotics and Automation (ICRA), 2026

paper website code

TL;DR: TWIST2 is a portable whole-body humanoid teleoperation system that enables scalable data collection.

OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction

Lujie Yang^*, Xiaoyu Huang^*, Zhen Wu^*, Angjoo Kanazawa^†, Pieter Abbeel^†, Carmelo Sferrazza^†, C. Karen Liu^†, Rocky Duan^†, Guanya Shi^†

International Conference on Robotics and Automation (ICRA), 2026

(Best Conference Paper Award)

(Best Paper Award on Robot Manipulation and Locomotion)

paper website dataset

TL;DR: High-quality interaction-preserving motion reference generation that enables agile whole-body skills with minimal RL tracking.

Perceptive Humanoid Parkour: Chaining Dynamic Human Skills via Motion Matching

Zhen Wu^*, Xiaoyu Huang^*, Lujie Yang^*, Yuanhang Zhang, Xi Chen, Pieter Abbeel^†, Rocky Duan^†, Angjoo Kanazawa^†, Carmelo Sferrazza^†, Guanya Shi^†, C. Karen Liu^†

Robotics: Science and Systems (RSS), 2026

paper website

TL;DR: PHP enables humanoid robots to autonomously perform long-horizon, vision-based parkour across challenging obstacle courses.

FALCON: Learning Force-Adaptive Humanoid Loco-Manipulation

Yuanhang Zhang, Yifu Yuan, Prajwal Gurunath, Tairan He, Shayegan Omidshafiei, Ali-akbar Agha-mohammadi, Marcell Vazquez-Chanlatte, Liam Pedersen, Guanya Shi

Learning for Dynamics and Control Conference (L4DC), 2026

(Oral Presentation)

paper website code

TL;DR: FALCON enables various heavy-duty humanoid loco-manipulation tasks via a new dual-agent force-adaptive RL framework.

VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation

Tairan He^*, Zi Wang^*, Haoru Xue^*, Qingwei Ben^*, Zhengyi Luo, Wenli Xiao, Ye Yuan, Xingye Da, Fernando Castañeda, Shankar Sastry, Changliu Liu, Guanya Shi, Linxi Fan^†, Yuke Zhu^†

IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2026

paper website

TL;DR: VIRAL investigates the scaling law of visual sim-to-real and finds a recipe to achieve zero-shot, robust, and continuous real-world deployment.

RPL: Learning Robust Humanoid Perceptive Locomotion on Challenging Terrains

Yuanhang Zhang, Younggyo Seo, Juyue Chen, Yifu Yuan, Koushil Sreenath, Pieter Abbeel^†, Carmelo Sferrazza^†, Karen Liu^†, Rocky Duan^†, Guanya Shi^†

arXiv preprint

paper website

TL;DR: A single policy trained by RPL enables multi-directional robust humanoid locomotion over various challenging terrains.

Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion

Tairan He^*, Chong Zhang^*, Wenli Xiao, Guanqi He, Changliu Liu, Guanya Shi

Robotics: Science and Systems (RSS), 2024

(Outstanding Student Paper Award Finalist)

paper website code IEEE Spectrum CMU News

TL;DR: ABS enables fully onboard, agile (>3m/s), and collision-free locomotion for quadrupedal robots in cluttered environments.

LadderMan: Learning Humanoid Perceptive Ladder Climbing

Siheng Zhao, Yuanhang Zhang, Ziqi Lu, Pieter Abbeel, Rocky Duan, Koushil Sreenath, Yue Wang, C. Karen Liu^†, Guanya Shi^†

arXiv preprint

paper website code

TL;DR: LadderMan enables humanoids to climb diverse ladders zero-shot and perform stable manipulation while balancing on the ladder.

SPIDER: Scalable Physics-Informed Dexterous Retargeting

Chaoyi Pan, Changhao Wang, Haozhi Qi, Zixi Liu, Homanga Bharadhwaj, Akash Sharma, Tingfan Wu^†, Guanya Shi^†, Jitendra Malik^†, Francois Hogan^†

arXiv preprint

paper website code

TL;DR: A dynamically feasible, cross-embodiment retargeting framework for both humanoid and dexterous hand. Human → physics → real at scale.

Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition

Harsh Gupta, Guanya Shi^†, Wenzhen Yuan^†

arXiv preprint

paper website

TL;DR: LUCID separates human-video intent from sim-trained control, enabling scalable real-world dexterous manipulation across tasks and embodiments w/o any real robot data.

HDMI: Learning Interactive Humanoid Whole-Body Control from Human Videos

Haoyang Weng, Yitang Li, Nikhil Sobanbabu, Zihan Wang, Zhengyi Luo, Tairan He, Deva Ramanan, Guanya Shi

arXiv preprint

paper website code

TL;DR: From human videos, HDMI learns robust humanoid loco-manipulation skills (e.g., opening a door continuously for 67 times).

Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds

Michael O'Connell^*, Guanya Shi^*, Xichen Shi, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, Soon-Jo Chung

Science Robotics

paper video Caltech news Reuters CNN code

TL;DR: Neural-Fly uses adaptive control to online fine-tune a meta-pretrained DNN representation, enabling rapid adaptation in strong winds.

AnyCar to Anywhere: Learning Universal Dynamics Model for Agile and Adaptive Mobility

Wenli Xiao^*, Haoru Xue^*, Tony Tao, Dvij Kalaria, John M. Dolan, Guanya Shi

International Conference on Robotics and Automation (ICRA), 2025

paper website code IEEE Spectrum

TL;DR: AnyCar is a transformer-based dynamics model that can adapt to various vehicles, environments, state estimators, and tasks.

Self-Improving Vision-Language-Action Models with Data Generation via Residual RL

Wenli Xiao^*, Haotian Lin^*, Andy Peng, Haoru Xue, Tairan He, Yuqi Xie, Fengyuan Hu, Jimmy Wu, Zhengyi Luo, Linxi "Jim" Fan, Guanya Shi, Yuke Zhu

International Conference on Learning Representations (ICLR), 2026

paper website

TL;DR: Probe, Learn, Distill (PLD): On-policy probing from a base VLA model + off-policy residual RL + distillation for VLA post-training.

Much Ado About Noising: Dispelling the Myths of Generative Robotic Control

Chaoyi Pan, Giri Anantharaman, Nai-Chieh Huang, Claire Jin, Daniel Pfrommer, Chenyang Yuan, Frank Permenter, Guannan Qu^†, Nicholas Boffi^†, Guanya Shi^†, Max Simchowitz^†

International Conference on Learning Representations (ICLR), 2026

paper website code

TL;DR: In most benchmarks, the success of generative policies is NOT from its distributional-learning formulation.

CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation

Max Fu^*, Justin Yu^*, Karim El-Refai^*, Ethan Kou^*, Haoru Xue^*, Huang Huang, Wenli Xiao, Guanzhi Wang, Fei-Fei Li, Guanya Shi, Jiajun Wu, Shankar Sastry, Yuke Zhu, Ken Goldberg^†, Linxi "Jim" Fan^†

International Conference on Machine Learning (ICML), 2026

paper website code

TL;DR: CaP-X benchmarks and improves embodied coding agents, enabling LMs to write robot-control code that generalizes zero-shot.

Full-Order Sampling-Based MPC for Torque-Level Locomotion Control via Diffusion-Style Annealing

Haoru Xue^*, Chaoyi Pan^*, Zeji Yi, Guannan Qu, Guanya Shi

International Conference on Robotics and Automation (ICRA), 2025

(Best Paper Award Finalist)

paper website code

TL;DR: DIAL-MPC is the first training-free method achieving real-time whole-body torque control using full-order dynamics.

TD-M(PC)²: Improving Temporal Difference MPC Through Policy Constraint

Haotian Lin, Pengcheng Wang, Jeff Schneider, Guanya Shi

Learning for Dynamics and Control Conference (L4DC), 2026

paper website code

TL;DR: We observe the value overestimation issue in planner-based MBRL and propose a policy constraint solution with SOTA performance.

Model-Based Diffusion for Trajectory Optimization

Chaoyi Pan^*, Zeji Yi^*, Guanya Shi^†, Guannan Qu^†

Neural Information Processing Systems (NeurIPS), 2024

paper website code

TL;DR: MBD is a diffusion-based traj optimization method that directly computes the score function using models without any external data.

Flying Calligrapher: Contact-Aware Motion and Force Planning and Control for Aerial Manipulation

Xiaofeng Guo^*, Guanqi He^*, Jiahe Xu, Mohammadreza Mousaei, Junyi Geng, Sebastian Scherer, Guanya Shi

IEEE Robotics and Automation Letters (RA-L), 2024

paper website IEEE Spectrum

TL;DR: Flying calligrapher enables precise hybrid motion and contact force control for an aerial manipulator in various drawing tasks.

Flying Hand: End-Effector-Centric Framework for Versatile Aerial Manipulation Teleoperation and Policy Learning

Guanqi He^*, Xiaofeng Guo^*, Luyi Tang, Yuanhang Zhang, Mohammadreza Mousaei, Jiahe Xu, Junyi Geng, Sebastian Scherer, Guanya Shi

Robotics: Science and Systems (RSS), 2025

paper website

TL;DR: A general-purpose aerial manipulation framework with an EE-centric interface that bridges whole-body control and policy learning.

UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policies

Harsh Gupta, Xiaofeng Guo, Huy Ha, Chuer Pan, Muqing Cao, Dongjae Lee, Sebastian Scherer, Shuran Song, Guanya Shi

International Conference on Robotics and Automation (ICRA), 2026

paper website

TL;DR: EADP steers UMI's embodiment-agnostic diffusion policy using the gradient of the low-level controller's tracking cost for cross-embodiment.

Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion

Tairan He^*, Chong Zhang^*, Wenli Xiao, Guanqi He, Changliu Liu, Guanya Shi

Robotics: Science and Systems (RSS), 2024

(Outstanding Student Paper Award Finalist)

paper website code IEEE Spectrum CMU News

TL;DR: ABS enables fully onboard, agile (>3m/s), and collision-free locomotion for quadrupedal robots in cluttered environments.

Agile Continuous Jumping in Discontinuous Terrains

Yuxiang Yang, Guanya Shi, Changyi Lin, Xiangyun Meng, Rosario Scalise, Mateo Guaman Castro, Wenhao Yu, Tingnan Zhang, Ding Zhao, Jie Tan, Byron Boots

International Conference on Robotics and Automation (ICRA), 2025

paper website code

TL;DR: Continuous, agile, and autonomous quadrupedal jumping via hierarchical model-free RL and model-based control.

Neural Lander: Stable Drone Landing Control Using Learned Dynamics

Guanya Shi^*, Xichen Shi^*, Michael O'Connell^*, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, Soon-Jo Chung

International Conference on Robotics and Automation (ICRA), 2019

paper video Caltech homepage news code

TL;DR: Spectrally normalized deep learning and nonlinear control enable provably stable agile drone landing.