Rob (@roblee_rl) header
Rob (@roblee_rl) profile picture

reinforcement learning & robotics @sydekickbot

Joined October 2018
Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)
Pinned Tweet
Watch our robot repeatedly assemble a collapsible crate, even while I perturb it 😅 It's a long-horizon task that requires many subtle non-prehensile actions. This model was trained from scratch locally on my 5090 with only ~2 hours of data. More soon! @sydekickbot
7
8
1
72
8,904
when we talk about intelligence, we’re talking about RL. all other learning paradigms are shortcuts and hacks.
1
1
40
2,004
Working with RL is always full of fun surprises. This policy learned to adjust its grasp using contact with the table. RGB policy, trained entirely in sim with sparse reward. More soon @sydekickbot
14
15
5
209
13,077
pendulum has fully swung back from where it was last year (“scale up data collection, behavior clone large models”). everyone is finally seeing the shining light of RL!
Imitation learning plateaus with more data. Additional benefit of collecting more data stops improving performance -- hence, Kay was convinced they needed reinforcement learning.
5
8
3
93
14,838
still lots of fun research to do in robotics
🤖 Made with AI
Dream-1 is single-handedly holding up the dreams of every robot learning researcher
1
6
78
3,051
"the bitter lesson comes for robotics" and it's literally a hand-designed architecture with its every component and hyperparameter painstakingly ablated against human-specified performance metrics that is then initialized by standing on the shoulders of humans as it consumes digitized civilizational knowledge of immense scale before it gets slow-cooked during an extended phase of further distilling human knowledge from pristine resources only to then be relentlessly beaten against the well-tuned reward functions of a million partly crowd-sourced and partly prompted-into-existence environments that humans decided were worth hillclimbing in the current quarterly iteration of the human-orchestrated cycle of intelligence creation
12
34
6
333
12,883
I love when robots move like robots should instead of human teleoperated or programmed behaviour. RL can show the real potential of a given embodiment, beyond human priors.
Replying to @octi_zhang
(2/5) No human demonstrator would teach these behaviors. Our robot tips a rod upright by pushing it against the board’s leg, sets a gear upright by pushing it against its shaft, and flips the nut instead of regrasping it. These are strategies suited to its own gripper, not to a human hand. Pure RL discovered them in sim, and they transferred zero-shot to the real robot.
2
9
1
95
5,643
Astra attempts a shortcut through the table:
5
1
67
65,464
This is super impressive, you can see it attempt to get the bonus strawberries a few times and then give up and go back to clearing the level (eg. 1:30).
so it seems like you can let Astra play games by looking at the screen and it'll do pretty well here it completed Celeste chapter 1 with only a few retries (this is non-realtime, game is paused when the model thinks)
3
227
Nice!
Replying to @hansenlillemark
In fact, we show that EqF can still learn the same optimal denoising field as Flow Matching! This is because in high dimensions, the noisy sample itself carries the noise level information: explicit conditioning is superfluous.
2
306
Both XM and IMLE have been applied to imitation learning and planning, with similar benefits over Diffusion Policy and Diffuser respectively. See page 1 of the IMLE policy paper (arxiv.org/pdf/2502.12371) by @krshnrana @roblee_rl David Pershouse and @nikoSuenderhauf. 11/n
1
3
66
13,381
Whew! That was probably the longest thread I've ever written on Twitter. I hope you all found that to be informative :) In summary, XM is just a special case of the earliest version of IMLE that we had back in 2018. We've made a number of improvements since then - if you are interested in learning more, I'll tweet about the important ones in the coming weeks. Stay tuned!
4
7
3
288
69,296
I trained the policy on the right in 2020 with about 1 hour of robot data for my first CoRL paper. I had to use a lot of tricks to get it to work. The policy on the left was also trained with about 1.5 hours of robot data in total. Robot learning has come a long way! @sydekickbot
1
5
1
45
3,660
perpetual cup stacking @sydekickbot
5
15
1,730
less than 1 hour of data. i love policy eval timelapses :)
4
5
61
6,028
Seeing a model work like this for the first time is such a good feeling
a good model trained on even a simple task with a tiny amount of data feels mesmerising, no matter how many times you see it.
2
5
71
7,600
a good model trained on even a simple task with a tiny amount of data feels mesmerising, no matter how many times you see it.
15
14
2
164
23,294