Mannat Singh (@mannat_singh) profile picture

MTS @AnthropicAI, ex @Meta MSL/GenAI/FAIR. Researching multimodal frontier models, currently focusing on Vision RL. Opinions my own.

Manhattan, NY
Joined December 2010
Photo by Mannat Singh (@mannat_singh)Photo by Mannat Singh (@mannat_singh)Photo by Mannat Singh (@mannat_singh)Photo by Mannat Singh (@mannat_singh)Photo by Mannat Singh (@mannat_singh)Photo by Mannat Singh (@mannat_singh)Photo by Mannat Singh (@mannat_singh)Photo by Mannat Singh (@mannat_singh)Photo by Mannat Singh (@mannat_singh)Photo by Mannat Singh (@mannat_singh)
Some personal news: I've joined @AnthropicAI to work on RL for vision! After 7 years at @AIatMeta, I'm excited to help Claude see and reason about the visual world, especially for long horizon tasks. It's been an incredible run, and I'm proud of what we built along the way. For 4 years at FAIR I researched and scaled weakly-supervised (SWAG, MAWS) and self-supervised (SEER, MAE pre-pretraining) vision models, researched multimodal models (Omni*, MDETR), and built scalable research codebases (Classy Vision, VISSL, Omnivision). One of my biggest highlights was the Omnivore → OmniMAE → ImageBind line of work unifying modalities — working with an amazing team while also having so much fun! Then for the last 3 years I went on an exhilarating ride with my team/friends (and new teammates who are just as close friends now!) to work on image and video generation. We built SOTA generation models including Emu Video, Movie Gen, and Native video editing, while also improving video captioners in the process. The pace was relentless, but I enjoyed every bit of it because I got to work with a team full of smart and incredibly nice people! It was great to see it all culminate in a successful Muse image/video launch! A huge thank you to everyone I got to work with and learn from at Meta — I will deeply cherish the last 7 years! Looking back, the arc from teaching models to see, to generating what they see, to now teaching them to act on what they see feels like a natural next step. On to the next chapter!
24
13
482
31,603
Mannat Singh retweeted
Dominos: <30 mins OR free This: <30s AND free! 🧵 on the new video editing feature in Meta AI You can edit videos sequentially, with one instruction at a time, or with a complex multi-step instruction. Even the multi-step runs in <30s! Here's some sequential video edits
1
1
4
326
Presenting video editing (up to 10s videos) in Meta AI! Generate high-fidelity edits fast⚡(under 30 seconds) with no usage limits, powered by our model trained in-house last year! Took a while to get this out but excited to share with the world 🧵below with all the capabilities
1
3
1
22
2,040
It was my pleasure to have worked with a cracked team of researchers to build this model last year, and amazing product folks this year to launch it through several iterations + experiments. Big shoutout to the safety, privacy, & legal teams too! 🙌
1
4
201
Also learned how much work goes in after model training. Proud of my work this year on deployment, inference optimization (drove latency down to near realtime; majority of the current latency is outside the backend call!), training-free quality fixes, and cross-function collab 🤩
3
180
Flow matching can transform one distribution to another. So why do text-to-image models map noise to images instead of directly mapping text to images? Wouldn't it be cool to directly connect modalities together? CrossFlow accomplishes exactly that! cross-flow.github.io/
2
44
2
320
33,387
In fact, we find that this simple design scales *even better* than conventional FM with both model size and training steps. Lots of other details, like enabling CFG, the importance of a Variational Encoders in the paper (arxiv.org/abs/2412.15213).
1
10
1,450