• 2024 Oct: CMU RI Seminar Talk "Building Generalist Robots with Agility via Learning and Control: Humanoids and Beyond" (one hour) [ recording] [ 2025 April version at ETH Zürich] [ slides]
• 2024 Sep: Georgia Tech IRIM Seminar Talk "Unifying Semantic and Physical Intelligence for Generalist Humanoid Robots" (one hour) [ recording]

Humanoid robots offer two unparalleled advantages in general-purpose embodied intelligence. First, humanoids are built as generalist robots that can potentially do all the tasks humans can do in complex environments. Second, the embodiment alignment between humans and humanoids allows for the seamless integration of human cognitive skills with versatile humanoid capabilities, making humanoids the most promising physical embodiment for AI.

Humanoid control in a whole-body manner is challenging, due to its high degrees of freedom and contact-rich nature (see this survey). We aim to develop learning-based whole-body control methods for humanoid locomotion and loco-manipulation problems, enabling humanoids to interact with the physical world, adapt quickly to many tasks and environments, and integrate with high-level decision-making layers such as VLMs.

Offline learned policies for robotic control have shown great success in the robot learning community. For example, learning manipulation policies from demonstrations using imitation learning with generative models; learning locomotion policies in simulation using sim2real reinforcement learning. However, those policies are frozen in test time, and cannot efficiently adapt to new environments or tasks. We aim to develop methods that can effectively learn "adaptable" representation from offline data and efficiently adapt in real time.

Sim2real Reinforcement Learning (RL) has become the dominant method for many locomotion problems, especially for humanoids, legged robots, drones, and ground vehicles. However, the large sim2real gap, tedious reward tuning, and the lack of diversity hinder its applications in other problems such as loco-manipulation and dexterous manipulation. We aim to address these limitations via enhancing the simulation training environment using real-world data, which is often referred to as real2sim2real.

Model-based RL (MBRL) first learns a dynamics model (or a "world" model) and then generates a policy via planning/optimization, policy learning, or search. The most compelling part of MBRL is that the learned model is task-agnostic. We are interested in all aspects of MBRL, including model learning, theoretical framework, and planning using optimal control. In particular, recently, we have developed new sampling-based optimal control frameworks that enable efficient online decision-making.

Compared to learning-based methods, one problem of traditional control methods is that their performance cannot effectively scale up with the amount of data and parallel computing. Our focus is around a new concept called computational control, i.e., control methods that can effectively scale up with the amount of data and parallel computing. Examples include sampling-based optimal control and "generative action model" (policy learning using generative models). The goal is to systematically understand, analyze, and enhance computational control methods and apply them to robotics. One of recent interests is to understand how diffusion/flow-based methods work in decision-making problems.

To some extent, existing works on aerial manipulation primarily focus on the aerial perspective, rather than the general-purpose manipulation perspective. The goal is to study aerial manipulation from the embodied intelligence perspective, building general hardware platforms and control methods. Similar to humanoids, we aim to solve the whole-body control problem for aerial manipulation. We are also interested in designing the high-level policy via vision-language-action (VLA) models or imitation learning.

Most RL algorithms are general for all tasks. In contrast, drastically different control methods are developed for different systems/tasks, and their successes highly rely on structures inside these systems/tasks. We seek to encode these structures and algorithmic principles into black-box RL algorithms, to make RL algorithms more data-efficient, robust, interpretable, and safe. Examples include hierarchical RL and optimal control methods, learning safety filter for RL policies, and learning-based nonlinear control with stability guarantees.