CTO @FlexionRobotics | Previously IsaacLab / IsaacGym @NVIDIARobotics

Zurich, Switzerland
Joined January 2019
One of the most impactful lines of work in recent years, and we see similar scaling laws at @FlexionAI. It raises the question: Why spend millions collecting real world data when you can learn this in sim way cheaper? Kudos to @octi_zhang @mateoguaman @abhishekunique7 and co!
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before. The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices. Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL. No demonstrations, no per-task reward tuning. Just PPO at scale. Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
10
11
4
96
10,495
David Hoeller retweeted
"A parcel with snacks has been delivered for Flexion. Retrieve it using the stairs and come up using the elevator. Then unpack it and place the items into the empty drawer on the shelf in the snack area." One instruction. No human operator. Everything that follows is autonomous. Today we're introducing Reflect v1.0, our robotics intelligence platform for long-horizon work. From a single natural-language command, the robot understands the task, navigates a multi-floor building, calls elevators, handles doors, uses tools to unpack a box, and puts the items away. The biggest shift in v1.0 is that we use reinforcement learning across every layer, from low-level control to high-level reasoning. Long-horizon autonomy is unforgiving. The robot must recover on its own when things don't go to plan because in the real world, they never do. Combining reasoning, perception, physical execution and runtime robustness into a single mission-capable system is the foundation required to solve humanoid autonomy. Our team is just getting started. #HumanoidRobots #Flexion
25
102
59
439
144,084
Long horizon sim-to-real. Its impressive how fast companies like this are operating-- flexion is very new!
In our latest video, our agent tidies up our office fully autonomously, starting from a simple user prompt. No scripts. No pre-computed trajectories. Continues in the thread.
1
6
1
65
7,517
David Hoeller retweeted
We’ve raised $50 million in Series A funding from DST Global Partners, @nvidia (NVIDIA’s venture capital arm), @redalpine , @prosusventures , and @Moonfire_VC , following our $7.35M seed round from @frst_vc , @Moonfire_VC , and @redalpine just a few months earlier, to build the autonomy stack that makes humanoid robots adaptive, intelligent, and ready for real-world deployment at scale. In less than a year, our team has shown that long-horizon whole-body humanoid control can scale across hardware and tasks by leveraging the power of simulation and reinforcement learning. This funding will help us grow our team, scale our compute and robot fleets, and accelerate the commercialization of our autonomy stack with OEM partners globally. You can find more details in the links shared in the comments.
19
56
18
362
75,804
12kg is very impressive, congrats @2nisi
A 9.2 kg robotic arm that lifts more than its weight and works anywhere? That’s real muscle. Meet the latest innovation from Duatic AG. 🎙️ I sat down with Duatic Co-Founder Eris Dhionis Sako to talk about how they built it and where it’s heading. ⬇️ ✅ Lifts up to 12 kg despite its lightweight build ✅ Controlled with a PlayStation joystick ✅ Built for rough environments like rain and dust ✅ Works great on mobile robots, indoors or out Small arm, big power. Credit: Duatic AG And saying thank you to Eris Dhionis Sako for having me! Ep 60 - Building Deep Tech: "Once You Know What You Want to Do, Life Gets Easier." 🎧 On Spotify: open.spotify.com/episode/18D… 🎧 On Apple Podcast: podcasts.apple.com/us/podcas… 🎬 On YouTube: youtu.be/DxC7deB2K7A
3
1
7
1,326
Any plans to combine it with @BotaSystems force torque sensing? @Klajd_Lika
2
12
582
🔥 Exciting news 🤖 Our latest research by @HoellerDavid, @rdn_nikita, @2nisi in @SciRobotics unlocks new achievements: Unprecedented agility in quadrupedal robots, mastering locomotion, navigation, and perception through deep reinforcement learning! @NVIDIARobotics
4
7
1
93
5,395
I may have missed recent progress, but that's extremely impressive. Definitely the best quadruped robot motion I have ever seen, by far. @leggedrobotics youtube.com/watch?v=PjWvf90l…
🔥Exciting news 🤖 Our latest research by @HoellerDavid, @rdn_nikita, @2nisi in @SciRobotics unlocks new achievements:  Unprecedented agility in quadrupedal robots, mastering locomotion, navigation, and perception through deep reinforcement learning! @NVIDIARobotics
1
3
18
4,901
🔥Exciting news 🤖 Our latest research by @HoellerDavid, @rdn_nikita, @2nisi in @SciRobotics unlocks new achievements:  Unprecedented agility in quadrupedal robots, mastering locomotion, navigation, and perception through deep reinforcement learning! @NVIDIARobotics
20
112
42
574
172,059
David Hoeller retweeted
Roboticists from @leggedrobotics and @NVIDIAEmbedded are teaching four-legged robots climb and jump. After training in simulation, the robots can autonomously decide how to scramble over and under obstacles, which will help them do dangerous jobs so that humans don't have to.
121
645
562
2,762
2,318,606
Mapping the terrain for locomotion is hard for quadrupedal robots. The camera setup results in blind spots and the state estimator suffers from drift.We propose a solution in our RA-L @iros2022 paper youtu.be/3zsvqCrztLg @HoellerDavid @rdn_nikita @ChrisChoy208 @AnimaAnandkumar
1
6
34
Interested in pushing the limits of legged robots? Check out how ANYmal learns to jump, climb and run in our @iros202 paper: "Advanced Skills by Learning Locomotion and Local Navigation End-to-End". @rdn_nikita, @HoellerDavid, @MBjelonic youtube.com/watch?v=Xoe8a_2t…
David Hoeller retweeted
Whoa. Learned behavior from end-to-end training that combines navigation and locomotion (w/ motion capture for state estimation) by @leggedrobotics. Paper at @iros2022 next month, more here: sites.google.com/leggedrobot…
David Hoeller retweeted
Just pushed version 0.2.0 of Warp. Adds support for multidimensional arrays, slices, additional NVDB formats, and better support for differentiating through dynamic loops: github.com/NVIDIA/warp
2
29
3
141
If you missed the humanoid-quadruped transformer at @nvidia's #GTC, then you can watch the full video here: youtu.be/kEdr0ARq48A #Robotics #reinforcementlearning #nvidia #transformer #swissmile @ETH_en @nvidiaomniverse @NVIDIAGTC
22
776
124
1,691
David Hoeller retweeted
Our stuff was shown in @NVIDIAGTC
3
6
69
David Hoeller retweeted
I’m super excited to see ideas from complex systems such as swarm intelligence, self-organization, and emergent behavior gain traction again in AI research. We wrote a survey of recent developments that combine ideas from deep learning and complex systems: arxiv.org/abs/2111.14377
23
281
22
1,410
David Hoeller retweeted
Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning A video from their paper of thousands of robots learning to walk within minutes in a simulation on a single GPU. Near the end, they transfer the policy to the actual robot. arxiv.org/abs/2109.11978
10
138
24
623
Our team @CerberusSubt won the @DARPA #SubTChallenge!!! #Robotics is becoming real!
4
41
11
206
Our #NVIDIA Isaac Gym detailed technical report and benchmark results are now up on arXiv. Supercharge the training of your Reinforcement Learning (RL) networks by harnessing the power of the #GPU. Check out more details here: nvda.ws/3mD2HcO
6
14
5
61