LeCAR Lab — Learning and Control for Agile Robotics Lab
All posts

Test

LeCAR Lab

Placeholder post

A note about this post. This is a placeholder for future LeCAR Lab blog posts. The examples below illustrate the article format and do not report new research results.

How can a robot learn from experience while responding to a changing world? Learning and control offer complementary ways to approach this question: a policy can capture patterns in data, while feedback uses new observations to adjust the robot’s actions.

In this sample note, we walk through a small learning-and-control example. We introduce a policy, write down its training objective, and connect it to a feedback loop. The equations and figures serve as examples of how a future research story might unfold.

The setup

Consider a robot with state \(x_t\), observation \(o_t\), and action \(u_t\) at time \(t\). A policy \(\pi_\theta\), parameterized by \(\theta\), maps observations to actions. We can describe the interaction with a simple pair of equations:

\[ \begin{aligned} u_t &= \pi_\theta(o_t), \\ x_{t+1} &= f(x_t, u_t) + w_t. \end{aligned} \]
(1)

Here, \(f\) represents the dynamics and \(w_t\) represents an additive disturbance. After each action, the robot receives a new observation and repeats the process.

A simple feedback loop An observation goes to a policy, which sends an action to a robot. A new observation from the robot closes the loop. Input Policy Robot Action New observation
Figure 1. An illustrative feedback loop. The policy acts on an observation, and the next observation closes the loop.

Learning from demonstrations

Suppose we have a dataset \(\mathcal{D}\) of expert observation–action pairs. For a continuous-action policy, an example behavioral cloning objective is the mean squared error between predicted and demonstrated actions:

\[ \mathcal{L}(\theta) = \mathbb{E}_{(o,u^*)\sim\mathcal{D}} \left[\left\|\pi_\theta(o)-u^*\right\|_2^2\right]. \]
(2)

Equation (2) describes agreement with the demonstrations. A separate question is how the policy behaves when its own actions lead it to observations outside the training data. This distinction motivates work on distribution shift in imitation learning, such as DAgger [1].

Why might a small training error be insufficient?

The dataset describes observations collected by an expert. During deployment, the learned policy influences which observations it will encounter. An early mistake can bring it into an unfamiliar state, where subsequent predictions may be less reliable.

This is a useful place for a longer derivation, an implementation detail, or an additional experiment in a future post.

Connecting learning and control

A controller can also reason about future actions. For example, a finite-horizon objective balances tracking a desired state \(x_k^{\mathrm{ref}}\) against control effort:

\[ J = \sum_{k=0}^{H-1} \left(\left\|x_k-x_k^{\mathrm{ref}}\right\|_Q^2 + \left\|u_k\right\|_R^2\right). \]
(3)

The horizon is \(H\), and the positive semidefinite matrices \(Q\) and \(R\) set the relative penalties, with \(\|v\|_Q^2 = v^\top Qv\). This objective is illustrative; a complete controller would also specify dynamics, constraints, and an optimization procedure.

For a real example of research connecting learning and agile control, see the lab’s Neural-Fly work [2]. The existing lab demonstration below is included as sample media for this post.

Neural-Fly demonstration showing a quadrotor tracking a trajectory in windy conditions.
Figure 2. An existing Neural-Fly demonstration from the lab’s website [2], used here as a placeholder figure. It is separate from the illustrative setup in this post.
What would a full experiment include?

A future post could describe the task and dataset, compare relevant baselines, specify evaluation conditions, and report variation across runs. Figures would then link each observation to the evidence that supports it.

Closing thoughts

This sample starts with a feedback loop, introduces a learning objective, and adds a control objective. Future posts can use the same space to explain a new result, share a practical observation, or work through an open question in robotics.

For the lab’s current work, explore our publications and research topics.

References

  1. Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. AISTATS, 2011. ↩
  2. Michael O’Connell, Guanya Shi, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, and Soon-Jo Chung. Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds. Science Robotics, 2022. ↩