Learning and Control for Agile Robotics Lab at @CarnegieMellon @SCSatCMU @CMU_Robotics.

Pittsburgh
Joined August 2023
LeCAR Lab at CMU retweeted
Common belief: Robot policies need continuous action heads; discrete actions are 𝐥𝐞𝐬𝐬 𝐧𝐚𝐭𝐢𝐯𝐞 𝐚𝐧𝐝 𝐥𝐞𝐬𝐬 𝐩𝐫𝐞𝐜𝐢𝐬𝐞. ❗We find policies w/ the simplest discrete representations 𝐦𝐚𝐭𝐜𝐡 𝐨𝐫 𝐛𝐞𝐚𝐭 continuous ones—in sim & real, in single-step & diffusion/flow settings. Here's why 🧵
10
49
11
284
29,863
LeCAR Lab at CMU retweeted
Check out our 2nd paper in "Robot Learning Science"! Key question we aim to answer: Are robot actions unique among other AI modalities in that they must be predicted as continuous values, or can discrete action representations compete with, or even outperform, their continuous counterparts? (Perhaps) surprising fundings: 1. Even with the simplest discretion action representation (512 bins for each action dimension), discrete actions match or beat continuous actions, for both single-task policies or generalist policies, in both sim and real, in both single-step and diffusion/flow settings. 2. Continuous regression/flow policies tend to over-smooth action chunks. In RoboCasa, we found that the jerk (describe how fast acceleration changes) of action chunks predicted by continuous regression is only 7% of expert actions'. The number for discrete method is a lot higher. In other words, discrete actions keep expert's temporal variation better. 3. Precision is not the main issue of discrete methods! Even for one-step discrete regression, the end-effector prediction accuracy is similar to continuous methods', actually even better on fine motions. 4. Instead, the main issue of discrete method is factorization error, which can be fixed by iterative computing (guidance or diffusion/flow or AR). Meanwhile, we found that smarter tokenizers cannot replace iterative computing.
Common belief: Robot policies need continuous action heads; discrete actions are 𝐥𝐞𝐬𝐬 𝐧𝐚𝐭𝐢𝐯𝐞 𝐚𝐧𝐝 𝐥𝐞𝐬𝐬 𝐩𝐫𝐞𝐜𝐢𝐬𝐞. ❗We find policies w/ the simplest discrete representations 𝐦𝐚𝐭𝐜𝐡 𝐨𝐫 𝐛𝐞𝐚𝐭 continuous ones—in sim & real, in single-step & diffusion/flow settings. Here's why 🧵
2
11
1
86
5,921
LeCAR Lab at CMU retweeted
How far can coding agents take us toward general-purpose robots? Embodied First Exam lets us study this across 1K+ tasks—from spatial puzzles to agile whole-body control and real-time interaction. Top-1 model is 33% now. Excited to help build a shared testbed for agentic robotics!
Introducing Embodied First Exam: One line of code to run coding agents in 1,090 embodied tasks across 31 simulation suites and 44 robots. Can your agent go far beyond pick-and-place: Puzzles with spatial constraints; Olympic sports with multi-robot collab; Real-time eval, where the world won't pause; Dynamic, agile, whole-body control; AutoResearch with sim2sim Try your own agent: harbor run -d embodied-first-exam -a <your_own_harness> -m <your_own_model>
4
11
1
74
7,627
LeCAR Lab at CMU retweeted
Replying to @OpenAI
@OpenAI is so lucky to have @TairanHe99! @TairanHe99 is the first PhD student to graduate from my group, and it only took him two years and eight months to get this degree (Sep 2023 → Apr 2026). When @TairanHe99 joined my group, I thought general-purpose humanoids were an interesting research problem, rather than a solution. Humanoid whole-body control barely worked. Hardware broke everyday. Physical AGI was not even a legitimate dream. I always joke that “nothing really works in robotics,” but looking back, the community has made incredible progress—and Tairan has been a big part of that journey. I am incredibly proud of everything @TairanHe99 has achieved: @CMU_Robotics Presidential Fellowship, @nvidia Graduate Fellowship, many open-source contributions to the community, and WhynotTV podcast—one of the most influential deep tech podcasts, to name just a few. @TairanHe99’s thesis: Scalable Sim-to-Real Learning for General-Purpose Humanoid Skills. Committee members: me, @ChangliuL, @kkitani, Marco Hutter, and @pabbeel.
some news: I’ve joined OpenAI. After wrapping up my PhD in Robotics, I’m excited to keep working toward AGI in the physical world. exciting journey ahead :)
4
13
1
466
27,140
LeCAR Lab at CMU retweeted
some news: I’ve joined OpenAI. After wrapping up my PhD in Robotics, I’m excited to keep working toward AGI in the physical world. exciting journey ahead :)
240
57
25
2,479
242,667
LeCAR Lab at CMU retweeted
Congrats to @LeCARLab's 2026 #CMUGrad!
4
10
273
15,682
LeCAR Lab at CMU retweeted
My lab at CMU @LeCARLab is hiring a postdoc! (Vibe-made the poster via @ChatGPTapp)
🤖 Made with AI
1
27
4
215
31,295
LeCAR Lab at CMU retweeted
I’m so tired of writing rebuttals to this kind of “lack of novelty” review: “This paper trivially combines A, B, and C, so the algorithmic novelty is limited.” Technically, most (if not all) robotics papers are convex combinations of existing ideas. I still deeply appreciate A+B+C papers—especially when they deliver: - New capabilities: the “trivial combination” unlocks behaviors we simply couldn’t achieve before - Sensible & organic design: A+B+C is clearly the right composition—not some arbitrary A′+B+C′ - Nontrivial interactions: careful analysis of the dynamics, coupling, or failure modes between A, B, C - Rehabilitating old ideas: A was dismissed for years, but paired with modern B/C, it suddenly works—and teaches us why - System-level & "interface" insight: the contribution is not any single piece, but how the pieces talk to each other - Scaling laws or regimes: identifying when/why A+B+C works (and when it doesn’t) - Engineering clarity: making something actually work robustly in the real world is not “trivial” - New problem formulations: sometimes the real novelty is in the reformulation—only under this view does A+B+C make sense. Maybe worth keeping these in mind when reviewing the next A+B+C paper : )
29
123
25
988
118,436
LeCAR Lab at CMU retweeted
Generative models (diffusion/flow) are taking over robotics 🤖. But do we really need to model the full action distribution to control a robot? We suspected the success of Generative Control Policies (GCPs) might be "Much Ado About Noising." We rigorously tested the myths. 🧵👇
16
89
24
548
112,961
LeCAR Lab at CMU retweeted
We tried to rethink modern robot learning policy design in this paper. TL;DR: Generative policies perform well NOT because of their distributional-learning formulation or their ability to capture “multi-modality.” Our proposed Minimal Iterative Policy (MIP) achieves similar performance to flow with much lower inference & training cost. Some interesting takeaways: 1. “Multi-modal behavior” is a very nuanced concept in robotics. Robot learning cares about the conditional distribution P(a|o), where a is the action (chunk) and o is the observation. The objective of generative modeling in vision/language is fundamentally different from the goal in control. In vision/language, we want diverse samples from the data distribution. In control, any action that leads to better downstream performance is sufficient. Also, robotics has very little (labeled) action data, and they lie on a low-dimensional manifold. As a result, even if the marginal P(a) is multi-modal in many tasks (e.g., PushT), the conditional P(a|o) is often not multi-modal in sparse-data regions. Empirically, we observe that the common explanation—“flow/diffusion policies win because demonstrations are multi-modal”—does not hold for most studied behavior cloning benchmarks. 2. Architecture and action representation matter more than flow vs. regression. We find that the strong performance of flow/diffusion policies is largely due to their modern architectures and action representations. UNet / Transformer designs and action chunking play a vital role. In fact, L1/L2 regression policies can match flow/diffusion when paired with the right architectures and chunking, except in a few high-precision tasks. 3. What actually helps in flow/diffusion? Stochasticity injection and supervised iterative computation. To isolate these factors, we performed a “surgery” on flow to design Minimal Iterative Policy (MIP), a deterministic two-step policy that keeps only these mechanisms. Surprisingly, with far lower compute, MIP is comparable to—or even better than—flow in most studied behavior cloning benchmarks. Why? We hypothesize that these mechanisms provide good inductive biases that improve manifold adherence, especially for OOD states. Flow and MIP exhibit significantly better manifold adherence than regression. I don’t think these conclusions are complete—there are still many mysteries. But I strongly feel we need a new subfield: “the physics/science of robot learning,” aimed at understanding fundamental learning mechanisms and principles for data-driven robotics. As @ilyasut said for LLMs: “We’re moving from the age of scaling to the age of research.” For robot learning, I believe we must embrace the age of research to unlock better scaling to ignite the age of scaling.
Generative models (diffusion/flow) are taking over robotics 🤖. But do we really need to model the full action distribution to control a robot? We suspected the success of Generative Control Policies (GCPs) might be "Much Ado About Noising." We rigorously tested the myths. 🧵👇
2
30
179
14,575
LeCAR Lab at CMU retweeted
🕸️ Introducing SPIDER — Scalable Physics-Informed Dexterous Retargeting! A dynamically feasible, cross-embodiment retargeting framework for BOTH humanoids 🤖 and dexterous hands ✋. From human motion → sim → real robots, at scale. 🔗 Website: jc-bao.github.io/spider-proj… 🧵 1/n
12
58
16
232
66,131
LeCAR Lab at CMU retweeted
Introduce SPIDER - a unified scalable dynamics-level retargeting method for both dex hand and humanoid! What does "dynamics-level retargeting" mean? Human motions in, physically feasible high-quality robot motions out (BOTH state and action). These robot motions are so good that you can directly open-loop execute the action sequence in the real world! Also, dynamics-level retargeting makes the downstream RL policy learning super easy since it only needs to figure out a "small residual" on top of the already physically feasible refs. Even better, we can easily do physics-level data augmentation & robustification, such as adding external forces and changing frictions. SPIDER can be used as a data engine for real2sim2real and learning from human data. How it works? It is based on the diffusion-inspired sampling-based optimal control framework we developed in the past (MBD, DIAL-MPC) and virtual contact guidance. Led by @ChaoyiPan and collaboration with @Meta FAIR.
🕸️ Introducing SPIDER — Scalable Physics-Informed Dexterous Retargeting! A dynamically feasible, cross-embodiment retargeting framework for BOTH humanoids 🤖 and dexterous hands ✋. From human motion → sim → real robots, at scale. 🔗 Website: jc-bao.github.io/spider-proj… 🧵 1/n
1
9
69
8,236
LeCAR Lab at CMU retweeted
Meet BFM-Zero: A Promptable Humanoid Behavioral Foundation Model w/ Unsupervised RL👉 lecar-lab.github.io/BFM-Zero… 🧩ONE latent space for ALL tasks ⚡Zero-shot goal reaching, tracking, and reward optimization (any reward at test time), from ONE policy 🤖Natural recovery & transition
7
75
23
271
84,175
LeCAR Lab at CMU retweeted
Excited to release BFM-Zero, an unsupervised RL approach to learn humanoid Behavior Foundation Model. Existing humanoid general whole-body controllers rely on explicit motion tracking rewards, on-policy PG methods like PPO, and distillation to one policy. In contrast, BFM-Zero directly learns an effective shared latent representation that embeds motions, goals, and rewards into a common space, which enables a single policy zero-shot perform multiple tasks: (1) natural transition from any pose to any goal pose, (2) real-time motion following, (3) optimize any user-specified reward function at test time, etc. How it works? We don't give the model any specific reward in training. It builds upon recent advances in Forward-Backward (FB) models, where a latent-conditioned policy, a deep "forward dynamics model" and a deep "inverse dynamics model"are jointly learned. In such a way, the learned representation space understands humanoid dynamics and unifies different tasks. More details: lecar-lab.github.io/BFM-Zero…
Meet BFM-Zero: A Promptable Humanoid Behavioral Foundation Model w/ Unsupervised RL👉 lecar-lab.github.io/BFM-Zero… 🧩ONE latent space for ALL tasks ⚡Zero-shot goal reaching, tracking, and reward optimization (any reward at test time), from ONE policy 🤖Natural recovery & transition
2
12
1
97
18,750
LeCAR Lab at CMU retweeted
✈️🤖 What if an embodiment-agnostic visuomotor policy could adapt to diverse robot embodiments at inference with no fine-tuning? Introducing UMI-on-Air, a framework that brings embodiment-aware guidance to diffusion policies for precise, contact-rich aerial manipulation.
8
32
17
212
66,785
LeCAR Lab at CMU retweeted
100% human data -> long-horizon💡insertion on an aerial manipulator Key: Embodiment-aware diffusion policy (EADP) steers UMI's embodiment-agnostic DP using the gradient of the low-level controller's tracking error. My favorite: We quantify how "UMI-able" different robots are.
✈️🤖 What if an embodiment-agnostic visuomotor policy could adapt to diverse robot embodiments at inference with no fine-tuning? Introducing UMI-on-Air, a framework that brings embodiment-aware guidance to diffusion policies for precise, contact-rich aerial manipulation.
4
12
1,377
LeCAR Lab at CMU retweeted
Didn't get chance to attend #CoRL2025 in person, but students from @LeCARLab will present three papers: - Sampling-based system ID for legged sim2real lecar-lab.github.io/spi-acti… at Oral5 by @NikhilSoban353 - HoldMyBeer: learning humanoid end-effector stabilization control lecar-lab.github.io/SoFTA/ at Spotlight3 by @li_yitang - Humanoid Policy ~ Human Policy human-as-robot.github.io/ at Spotlight1 by @chaitanya_cha Check them out!
1
11
60
5,404
LeCAR Lab at CMU retweeted
We present HDMI, a simple and general framework for learning whole-body interaction skills directly from human videos — no manual reward engineering, no task-specific pipelines. 🤖 67 door traversals, 6 real-world tasks, 14 in simulation. 🔗 hdmi-humanoid.github.io/
24
139
35
754
137,017
LeCAR Lab at CMU retweeted
On my way ✈️ to ATL for @ieee_ras_icra! @LeCARLab will present 8 conference papers (including DIAL-MPC as the Best Paper Finalist) and one RA-L paper. Details: lecar-lab.github.io/ Hope to meet old & new friends and chat about building generalist 🤖 with agility 🚀
19
2
107
9,125
🦾How can humanoids unlock real strength for heavy-duty loco-manipulation? Meet FALCON🦅: Learning Force-Adaptive Humanoid Loco-Manipulation. 🌐: lecar-lab.github.io/falcon-h… See the details below👇:
4
51
13
181
58,177