Co-founder of Physical Intelligence (π.com, @physical_int)

San Francisco, CA
Joined November 2014
We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.
40
292
80
2,190
447,362
it's wild to me how far we've come without memory in our policies, but see pi.website/research/memory for how it unlocks long-horizon and partially observable tasks the secret sauce is doing it efficiently and over varying horizons, but otherwise, at scale, it kind of just works!
We’ve developed a memory system for our models that provides both short-term visual memory and long-term semantic memory. Our approach allows us to train robots to perform long and complex tasks, like cleaning up a kitchen or preparing a grilled cheese sandwich from scratch 👇
1
1
20
2,549
Amazing to see our models out in the world powering robots I've never even seen in person. In collaboration with @weaverobotics and @Ultraroboticsco. Here's our model folding sweaters
General-purpose AI models are behind some of the most exciting applications we now can't live without. We envision that an analogous “physical intelligence layer” built with models like π0.6 will similarly spur a new wave of applications for the physical world. We’ve recently begun working with a handful of companies that have deployed their robots to do real-world, useful things. pi.website/blog/partner/?v=1
1
2
1
35
8,577
and recovering from failure.
2
377
π-0.5, a model doing long-horizon tasks in real, unseen homes with unseen objects! @physical_int one of my favorites below at 10x sharing a few of my favorite results in thread, with many more detailed ablations in the paper: pi.website/blog/pi05
We got a robot to clean up homes that were never seen in its training data! Our new model, π-0.5, aims to tackle open-world generalization. We took our robot into homes that were not in the training data and asked it to clean kitchens and bedrooms. More below⤵️
3
9
2
163
12,070
for language following: multi-environment, cross-embodiment, and web data were all critical to ood performance
2
4
1,138
zooming in. as we scale environments, language following improves too, and particularly recognizes objects lags being able to manipulate them
1
4
842
Today we released Hi Robot, a high and low level approach that enables robot's to think through tasks and adjust to feedback. Turns out it not only enables interactivity, but improves performance significantly.
Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).
2
53
4,038
I'll be speaking at Founders You Should Know in SF on Feb 5th about Physical Intelligence (π.com) -- our progress and why you should join. Plus it's a great place to meet founders of some of the best SF startups! Apply to join at foundersysk.com @foundersysk @nifferkin
2
3
1
38
3,086
There are great tokenizers for text and images, but existing action tokenizers don’t work well for dexterous, high-frequency control. We’re excited to release (and open-source) FAST, an efficient tokenizer for robot actions. With FAST, we can train dexterous generalist policies via simple next token prediction, and get a 5x training speed-up over prior state of the art!
10
91
18
734
129,620
Really excited to announce what we at @physical_int have been working on... π_0! Our first generalist model and accompanying paper/blog, physicalintelligence.company….
2
5
103
6,430
So many amazing capabilities shown on our blog, but a highlight for me is how well it pays attention to language. This means it be guided by people or used with a high-level policy to improve further.
1
3
699
At Physical Intelligence (π) our mission is to bring general-purpose AI into the physical world. We're excited to show the first step towards this mission - our first generalist model π₀ 🧠 🤖 Paper, blog, uncut videos: physicalintelligence.company…
50
320
180
1,600
567,205
Excited to announce that with a few incredible folks, we're starting a company -- Physical Intelligence (Pi, π, @physical_int). We're focused on bringing the amazing recent breakthroughs of AI and foundation models into the physical world.
12
7
3
108
18,736
We're developing robotic foundation models capable of doing any task, anywhere. Too many exciting developments to list, but first and foremost is the incredible team of @chelseabfinn, @hausman_k, @lachygroom, @svlevine, @QuanVng, @SurajNair_1, and a few more to be announced.
1
8
5,260