Rob (@roblee_rl) header
Rob (@roblee_rl) profile picture

reinforcement learning & robotics @sydekickbot

Joined October 2018
Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)Photo by Rob (@roblee_rl)
when we talk about intelligence, we’re talking about RL. all other learning paradigms are shortcuts and hacks.
1
1
40
2,008
pendulum has fully swung back from where it was last year (“scale up data collection, behavior clone large models”). everyone is finally seeing the shining light of RL!
Imitation learning plateaus with more data. Additional benefit of collecting more data stops improving performance -- hence, Kay was convinced they needed reinforcement learning.
5
8
3
93
14,838
"the bitter lesson comes for robotics" and it's literally a hand-designed architecture with its every component and hyperparameter painstakingly ablated against human-specified performance metrics that is then initialized by standing on the shoulders of humans as it consumes digitized civilizational knowledge of immense scale before it gets slow-cooked during an extended phase of further distilling human knowledge from pristine resources only to then be relentlessly beaten against the well-tuned reward functions of a million partly crowd-sourced and partly prompted-into-existence environments that humans decided were worth hillclimbing in the current quarterly iteration of the human-orchestrated cycle of intelligence creation
12
34
6
333
12,883
Both XM and IMLE have been applied to imitation learning and planning, with similar benefits over Diffusion Policy and Diffuser respectively. See page 1 of the IMLE policy paper (arxiv.org/pdf/2502.12371) by @krshnrana @roblee_rl David Pershouse and @nikoSuenderhauf. 11/n
1
3
66
13,381
Whew! That was probably the longest thread I've ever written on Twitter. I hope you all found that to be informative :) In summary, XM is just a special case of the earliest version of IMLE that we had back in 2018. We've made a number of improvements since then - if you are interested in learning more, I'll tweet about the important ones in the coming weeks. Stay tuned!
4
7
3
288
69,296
Seeing a model work like this for the first time is such a good feeling
a good model trained on even a simple task with a tiny amount of data feels mesmerising, no matter how many times you see it.
2
5
71
7,600
New imitation learning algo just dropped
Are Diffusion and Flow Matching the best generative modelling algorithms for behaviour cloning in robotics? ✅Multimodality ❌Fast, Single-Step Inference ❌Sample Efficient 💡 We introduce IMLE Policy, a novel behaviour cloning approach that can satisfy all the above. 🧵👇
1
8
63
5,872
This looks like a game-changer for robotics and imitation learning! Diffusion policy is hard to run in real-time; the paper proposes IMLE policy, which can run at >110 Hz (a 61x speedup), use 38% less data and achieve better success rates. Note: I wasn't involved in this work, but nevertheless it’s great to see RS-IMLE working so well in robotics :) All credit goes to @krshnrana, Robert Lee, David Pershouse, @nikoSuenderhauf for such amazing work! Check out their paper here: arxiv.org/abs/2502.12371
1
18
1
116
9,010
Introducing ✨Posed DROID✨, results of our efforts at automatic post-hoc calibration of a large-scale robotics manipulation dataset. We provide: 🤖 ~36k calibrated episodes with good quality extrinsic calibration 🦾 ~24k calibrated multi-view episodes with good-quality multi-view camera calibration ✅ Quality assessment metrics for all provided camera poses To achieve this, we utilize: 1️⃣ Auto Segment Anything (SAM) based filtering (Camera-to-Base Calibration) 2️⃣ Tuned CtRNet-X for bringing in additional cams (Camera-to-Base Calibration) 3️⃣ Pretrained DUST3R with depth-based pose optimization (Camera-to-Camera Calibration) Try it out at: droid-dataset.github.io/ Learn more at: 🌐 arXiv: arxiv.org/pdf/2403.12945 📄 Blog: medium.com/p/4ddfc45361d3 🧵 1/n
3
27
3
183
13,644
1/ Real2Sim and Sim2Real are well-known paradigms in robotics🤖. Is there another paradigm where sim and the real world are more tightly interlinked? We introduce Real-IS-Sim: a framework featuring an always-in-the-loop, correctable simulator for behavior cloning policies.
4
50
4
359
31,623
🚀 Embodied Gaussians is now opensource! 🔥 📂 GitHub: github.com/bdaiinstitute/emb… ✨ Key Features: 1️⃣ Gaussian Splatting (GSplat) + Physics (Warp) 💥 2️⃣ Parallelizable environments! ⚡ 3️⃣ Render virtual cameras in parallel 📸 🔜 Coming Soon (hopefully): 🌀 Deformables!
1
30
5
212
22,937
Stoked to announce that BricksRL, our paper on using PyTorch/TorchRL to train the LEGO® robots in the wild has been accepted at @NeurIPSConf as a spotlight paper! 🪢
2
21
6
117
19,197
The wait is finally over!!! 😁 We just dropped an in-depth tutorial on how to build your own robot! Teach it new skills by showing it a few moves with just a laptop. Then watch your homemade robot act autonomously 🤯 1/🧵👇
47
450
88
2,269
400,896
✨Our 3D vision meets robotics papers list in the era of large models now has 80+ papers and crossed 50 stars and growing🚀 github.com/zubair-irshad/Awe… Topics include policy learning, VLMs and VLAs, 3D Representations including NeRF, Gaussian Splatting, Simulations and Benchmarks and 3D Pretraining. Check it out now. Contributions/Feedback welcome.
3
56
8
263
25,016
10 years after DQN, what are deep RL’s impacts on robotics? Which robotic problems have seen the most thrilling real-world successes thanks to DRL? Where do we still need to push the boundaries, and how? Our latest survey explores these questions! Read on for more details. 👇
2
94
8
503
65,693
A great research project I would like to share: 𝗙𝗢𝗨𝗡𝗗𝗔𝗧𝗜𝗢𝗡 𝗣𝗢𝗦𝗘 𝗠𝗘𝗘𝗧𝗦 𝗗𝗜𝗙𝗙𝗨𝗦𝗜𝗢𝗡 𝗣𝗢𝗟𝗜𝗖𝗬 Long-horizon tasks with different objects are hard for robots to learn by imitation because they struggle with sample efficiency, generalization, and modularity. ✅ Images and fixed coordinates limit understanding. ✅ Their method uses “affordance-centric” coordinates for better learning. ✅ This helps robots combine small tasks for complex ones and adapt to new tasks. With this method, robots can effectively learn complex tasks with fewer demonstrations. Project: policy-decomposition.github.… Thanks to @krshnrana for bringing my attention to this project! 🫶 • • • • 🔔 Follow @Ilir_AI for updates on #robotics & #ai. —————————————— 🤖 Robotics Telegram: t.ly/gaTBP 🎙 Robotics Podcast: t.ly/MyoUO
2
11
70
4,144