
MTS @AnthropicAI, ex @Meta MSL/GenAI/FAIR. Researching multimodal frontier models, currently focusing on Vision RL. Opinions my own.
Manhattan, NY
Joined December 2010
- Tweets101
- Following255
- Followers999
- Likes8.4K
Some personal news: I've joined @AnthropicAI to work on RL for vision!
After 7 years at @AIatMeta, I'm excited to help Claude see and reason about the visual world, especially for long horizon tasks.
It's been an incredible run, and I'm proud of what we built along the way.
For 4 years at FAIR I researched and scaled weakly-supervised (SWAG, MAWS) and self-supervised (SEER, MAE pre-pretraining) vision models, researched multimodal models (Omni*, MDETR), and built scalable research codebases (Classy Vision, VISSL, Omnivision). One of my biggest highlights was the Omnivore → OmniMAE → ImageBind line of work unifying modalities — working with an amazing team while also having so much fun!
Then for the last 3 years I went on an exhilarating ride with my team/friends (and new teammates who are just as close friends now!) to work on image and video generation. We built SOTA generation models including Emu Video, Movie Gen, and Native video editing, while also improving video captioners in the process. The pace was relentless, but I enjoyed every bit of it because I got to work with a team full of smart and incredibly nice people! It was great to see it all culminate in a successful Muse image/video launch!
A huge thank you to everyone I got to work with and learn from at Meta — I will deeply cherish the last 7 years!
Looking back, the arc from teaching models to see, to generating what they see, to now teaching them to act on what they see feels like a natural next step.
On to the next chapter!
Mannat Singh retweeted
Dominos: <30 mins OR free
This: <30s AND free!
🧵 on the new video editing feature in Meta AI
You can edit videos sequentially, with one instruction at a time, or with a complex multi-step instruction. Even the multi-step runs in <30s!
Here's some sequential video edits
Presenting video editing (up to 10s videos) in Meta AI! Generate high-fidelity edits fast⚡(under 30 seconds) with no usage limits, powered by our model trained in-house last year! Took a while to get this out but excited to share with the world 🧵below with all the capabilities
It was my pleasure to have worked with a cracked team of researchers to build this model last year, and amazing product folks this year to launch it through several iterations + experiments. Big shoutout to the safety, privacy, & legal teams too! 🙌
1
4
201
Flow matching can transform one distribution to another. So why do text-to-image models map noise to images instead of directly mapping text to images?
Wouldn't it be cool to directly connect modalities together? CrossFlow accomplishes exactly that! cross-flow.github.io/
In fact, we find that this simple design scales *even better* than conventional FM with both model size and training steps.
Lots of other details, like enabling CFG, the importance of a Variational Encoders in the paper (arxiv.org/abs/2412.15213).
1
10
1,450
Great work led by Qihao Liu during his internship with our team @xi_yin_, @Andrew__Brown__, and @YuilleAlan
And props to Qihao for a github release (github.com/qihao067/CrossFlo…) with a complete reproduction + releasing models trained on public datasets.
17
928


