PhD student @UTCompSci | Learn to understand ourselves and build intelligence.🤖🧠👁️
Austin, TX
Joined July 2023
- Tweets564
- Following3.1K
- Followers823
- Likes9.7K
Yu Lei retweeted
Introducing GR00T 2: our most advanced, open System One model for robotics. It runs entirely on your robot’s onboard compute, fully untethered, and you own every weight of it. GR00T 2 features:
1. Long-horizon tasks: dexterous manipulation and whole-body control chained over 20+ steps and 5+ minutes, such as performing chemistry lab experiments or assembling model airplanes end to end.
2. X-Adapt: adapting from easy data sources (human wearables and simulation) and deploying zero-shot to diverse real robots.
3. Unified Cognition: native integrations with frontier agents for advanced physical reasoning, robust error recovery, and safety monitoring.
It's an open-source buffet! We are releasing not only the model weights and X-Adapt recipes (coming December), but also our agentic framework, simulation engine, annotation engine, hardware design, and even training data (100K hours, that's ~11 yrs of experience), all available today!
Everything that moves will be autonomous. Anyone can become a roboticist.
Lock in, build with us!
Deep dive in thread:
Yu Lei retweeted
Fable 5.2 shows substantially improved skills in controlling physical robots, even on tasks it has never explicitly been trained to do.
On RoboDojo, a benchmark with a simulated two-arm robot, it outperformed other models and most AI systems built to control robots.
Yu Lei retweeted
Meet GR00T 2, our new generation of world-action models (WAMs): it imagines how the physical world will unfold and generates the actions that get it there. One model runs on parallel grippers, dexterous hands, and humanoids.
research.nvidia.com/labs/gea…
Yu Lei retweeted
In sim-and-real co-training, robots learn from a few real demos plus ample sim data. But which sim data should we generate?
MetaGenSim meta-learns what to simulate: 2.5× the real-world success of uniform sim data, matching expert hand-tuning.
meta-gensim.github.io
#CoRL2026
Yu Lei retweeted
The science of pretraining deserves more attention!
What makes a good pretrained model? How to evaluate the it? When should we stop training? And how do we optimize pretraining for downstream SFT or RL?
Some answers may be counterintuitive, e.g., the best-performing checkpoint may not be the best starting point for downstream learning.
If you're curious now, check out @s_zhanyi's work “Beyond Validation Loss” 👇
Can more pretraining improve a robot policy’s success rate, yet makes it worse for porttraining? 🤔
In our new work, we find that it can. Continued supervised pretraining of generative robot policies can improve reliability at the expense of coverage, and this tradeoff affects downstream posttraining.
Read on to see why this happens and how we can preserve coverage. 1/n 🧵
Yu Lei retweeted
Back in ~Dec 2025, I really wanted to work on mobile manipulation, but robots were either too expensive or not performant enough.
So we built our own!
Excited to finally share something I’ve had so much fun working on:
1. Pixel: a co-designed full stack system for learning mobile manipulation.
2. Pixel-UMI: environment aware whole body manipulation without robot data.
w/ @JasonJZLiu
(1/N)
Yu Lei retweeted
Can more pretraining improve a robot policy’s success rate, yet makes it worse for porttraining? 🤔
In our new work, we find that it can. Continued supervised pretraining of generative robot policies can improve reliability at the expense of coverage, and this tradeoff affects downstream posttraining.
Read on to see why this happens and how we can preserve coverage. 1/n 🧵
Yu Lei retweeted
i've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models.
Yu Lei retweeted
We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090.
Our model predicts in a unified 3D representation space, capable of generating:
• Novel view appearance
• Multi-view consistent RGB-D
• 3D geometry
• Spatially aligned tactile signals
Play with the world model now at jwang771.github.io/3dworldsi…!
Yu Lei retweeted
The Platonic Representation Hypothesis is now a Theorem ◼️
Train JEPAs separately, on any modalities: they provably learn the same representation up to one rotation.
The result? A 109M JEPA-Text model that aligns with vision models better than 72B LLMs.
w/ @ylecun 🧵
Yu Lei retweeted
Introducing DreamFAR, a world model trained from scratch using only robot data, without internet pretraining. It supports:
🎮 Real-time teleoperation
⏱️ Minutes of long-horizon rollout
🧩 Generalization across 1,000+ tasks
yimingdou.com/DreamFAR
🧵 (1/8)
Yu Lei retweeted
This is Genie 4, our most realistic and interactive world model yet. 🌐
Explore dynamic environments in real time to see how the scene responds, sounds, and how the people in it move, react, and speak.
Yu Lei retweeted
🎉 After one year of robot pretraining, we learned one thing the hard way: data is everything.
Today we're excited to release FOMI, a Force-feedback Manipulation Interface for teaching contact-rich manipulation!
Inspired by @SongShuran's UMI, FOMI keeps handheld, robot-free data collection and adds what operators have been missing: you can feel what the gripper touches ✋ No force sensors, no motors.
Two key results:
🔑 29% more successful demos than UMI in the same time, and 18% more than GELLO in teleoperation
🔑 π0.5 fine-tuned on about an hour of FOMI data per task reaches 78.4% average success on four contact-rich tasks
Let's check out our demos 👇
Yu Lei retweeted
What can a robot hand learn from seconds of human motion?
It turns out: dexterous skills that generalize!
Excited to share AdaDex: Learning Adaptable Dexterous Skills via Hybrid Dexterous Mimicking.
From one short hand–object clip per skill, AdaDex learns an adaptable dexterous skill, with skills spanning functional grasping, object reposing, in-hand reorientation, and throw-and-catch.
Project page: yuxuank.com/AdaDex/
🧵👇 (1/8)
Yu Lei retweeted
Check out our 2nd paper in "Robot Learning Science"!
Key question we aim to answer: Are robot actions unique among other AI modalities in that they must be predicted as continuous values, or can discrete action representations compete with, or even outperform, their continuous counterparts?
(Perhaps) surprising fundings:
1. Even with the simplest discretion action representation (512 bins for each action dimension), discrete actions match or beat continuous actions, for both single-task policies or generalist policies, in both sim and real, in both single-step and diffusion/flow settings.
2. Continuous regression/flow policies tend to over-smooth action chunks. In RoboCasa, we found that the jerk (describe how fast acceleration changes) of action chunks predicted by continuous regression is only 7% of expert actions'. The number for discrete method is a lot higher. In other words, discrete actions keep expert's temporal variation better.
3. Precision is not the main issue of discrete methods! Even for one-step discrete regression, the end-effector prediction accuracy is similar to continuous methods', actually even better on fine motions.
4. Instead, the main issue of discrete method is factorization error, which can be fixed by iterative computing (guidance or diffusion/flow or AR). Meanwhile, we found that smarter tokenizers cannot replace iterative computing.
Yu Lei retweeted
Last year I trained the advantage-conditioned VLA for π*0.6 during my internship @physical_int. Now, I have a much improved recipe.
Introducing ✨Advantage Guidance✨, a performant RL method with stable advantage-conditioned supervised training and contrastive guidance sampling
Yu Lei retweeted
Discovering dexterity:
At home with my infant, and at Origami @origami_robotic ...
More tomorrow 👀