Saketh Rambhatla (@rssaketh) profile picture

Helping robots learn @SkildAI. Prev. GenAI research @Meta. Postdoc @Meta. PhD UMD College Park

San Francisco, CA
Joined October 2010
Photo by Saketh Rambhatla (@rssaketh)Photo by Saketh Rambhatla (@rssaketh)Photo by Saketh Rambhatla (@rssaketh)Photo by Saketh Rambhatla (@rssaketh)Photo by Saketh Rambhatla (@rssaketh)Photo by Saketh Rambhatla (@rssaketh)Photo by Saketh Rambhatla (@rssaketh)Photo by Saketh Rambhatla (@rssaketh)Photo by Saketh Rambhatla (@rssaketh)Photo by Saketh Rambhatla (@rssaketh)
Saketh Rambhatla retweeted
In-context learning!! We’ve spent years in robotics assuming a new task means new data. Collect demos —> finetune —> eval —> repeat. S1 is much more elegant and intuitive: show it one video of an unseen and the model just works. No fine-tuning or gradient updates needed. It can do so many complex tasks, even ones that run for 10 minutes or longer. It’s especially impressive to see completely new behaviors emerge at test time, like flipping a pancake, even though that behavior never appeared in the training data. It’s amazing to see S1 pick up a new task from a single example - something I really thought was still years away.
Introducing S1, our new foundation model that learns from one example. It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning. Watch S1 operate in real-time via in-context learning:
3
7
81
4,770
Saketh Rambhatla retweeted
Language Models are Few-Shot Learners showed that GPT-3 could tackle almost any language task with zero, one, or just a few examples. We’ve heard “the ChatGPT moment for robotics” many times. IMO, the definition is similar: a robot that can take on arbitrary new tasks from zero, one, or a few demonstrations without further tuning, even when those demos come from humans, other embodiments, or are suboptimal. Today, I’m excited that we’re one big step closer : ) Enjoy S1 🚀 skild.ai/blogs/s1
Introducing S1, our new foundation model that learns from one example. It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning. Watch S1 operate in real-time via in-context learning:
2
25
1,937
Saketh Rambhatla retweeted
This is the first genuine ChatGPT moment in robotics. I head they are already in production?!
Introducing S1, our new foundation model that learns from one example. It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning. Watch S1 operate in real-time via in-context learning:
4
6
95
17,041
Saketh Rambhatla retweeted
Strong scaling laws for one-shot learning in long-horizon manipulation tasks. It's only a matter of time until these models are ready for widespread deployment with very little task-specific data.
Introducing S1, our new foundation model that learns from one example. It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning. Watch S1 operate in real-time via in-context learning:
3
30
1,401
Saketh Rambhatla retweeted
Video editing is now available on the Meta AI app! A team of ~10 people built this from scratch in under a year. It was a fun and amazing team effort, with some of the nicest people to work with.
Presenting video editing (up to 10s videos) in Meta AI! Generate high-fidelity edits fast⚡(under 30 seconds) with no usage limits, powered by our model trained in-house last year! Took a while to get this out but excited to share with the world 🧵below with all the capabilities
1
5
238
Saketh Rambhatla retweeted
Pop! Art, now in the Meta AI app
1
2
171
Saketh Rambhatla retweeted
Presenting video editing (up to 10s videos) in Meta AI! Generate high-fidelity edits fast⚡(under 30 seconds) with no usage limits, powered by our model trained in-house last year! Took a while to get this out but excited to share with the world 🧵below with all the capabilities
1
3
1
22
2,040
Saketh Rambhatla retweeted
Warehouses are a critical part of the modern economy, but are still bottlenecked by human labor. We've acquired Fetch Robotics to tackle previously hard-to-automate parts of warehouse logistics with our omni-bodied AI! (Our factorio players are delighted)
We have acquired Zebra Technologies’ robotics arm (formerly Fetch Robotics). This is what happens when orchestration meets intelligence -- a major step toward fully autonomous warehouses. More robots. More environments. One unified brain.
1
2
13
1,062
Saketh Rambhatla retweeted
We have acquired Zebra Technologies’ robotics arm (formerly Fetch Robotics). This is what happens when orchestration meets intelligence -- a major step toward fully autonomous warehouses. More robots. More environments. One unified brain.
9
46
30
316
87,681
Saketh Rambhatla retweeted
We hosted Prof. Alyosha Efros (UC Berkeley) at @SkildAI! He didn't believe that robots could actually cook eggs reliably. :) Tested back-to-back 5times without fail! One batch of scrambled eggs every ~2.5mins nonstop. The same model assembles a GPU on a server rack too.
41
153
27
1,360
183,225
Saketh Rambhatla retweeted
At @SkildAI, we’ve raised $1.4B, bringing our valuation to over $14B. We’re on a generational mission, and I’m grateful to be working alongside an exceptional team. Thanks to our investors for the long-term conviction towards omni-bodied intelligence 🚀 bloomberg.com/news/articles/…
42
52
9
616
160,535
Saketh Rambhatla retweeted
Announcing Series C We’ve raised $1.4B, valuing the company at over $14B With this capital, we will accelerate our mission to build omni-bodied intelligence 🚀 skild.ai/blogs/series-c
24
69
42
585
351,817
Saketh Rambhatla retweeted
See our robot open doors, water plants, assemble a box, and more by learning from watching humans. Using <1hr of robot data.
2
11
4
79
36,286
Saketh Rambhatla retweeted
Let robots clean in peace 🧺🤖✨
6
42
25
195
170,926
Saketh Rambhatla retweeted
Humans learn by watching. Robots should too.
34
135
71
892
1,078,537
Saketh Rambhatla retweeted
Inference time objectives are amazing :) We show that LLMs can be upgraded to multimodal beings by a simple trick :) No training needed! Works on image generation, editing, style transfer and more!
Super excited to share some recent work that shows that pure, text-only LLMs, can see and hear without any training! Our approach, called "MILS", uses LLMs with off-the-shelf multimodal models, to caption images/videos/audio, improve image generation, style transfer, and more!
6
41
2
194
56,331
Saketh Rambhatla retweeted
Super excited to share some recent work that shows that pure, text-only LLMs, can see and hear without any training! Our approach, called "MILS", uses LLMs with off-the-shelf multimodal models, to caption images/videos/audio, improve image generation, style transfer, and more!
7
37
8
238
69,108
Super cool to see transformers scaling so effectively for image/video autoencoders! Our model also offers a flexible way to implement variable token length
Excited to share my work at Meta! We explore scaling tokenizers w/ ViT (ViTok) & found scaling tokenizers with DiT generation pipeline doesn’t boost performance for the current paradigm of auto-encoders! We develop SOTA tokenizers for images/videos. Thread for findings
1
4
47
5,394
Saketh Rambhatla retweeted
How can we better animate images solely following text descriptions? We present Motion Focal Loss (MotiF) (arxiv.org/abs/2412.16153) to better align motions with text descriptions in text-image-to-video (TI2V) task and release TI2V-Bench, a comprehensive TI2V benchmark. (1/n)
6
11
1
54
5,885
Saketh Rambhatla retweeted
Flow matching can transform one distribution to another. So why do text-to-image models map noise to images instead of directly mapping text to images? Wouldn't it be cool to directly connect modalities together? CrossFlow accomplishes exactly that! cross-flow.github.io/
2
44
2
320
33,387