Yang Li (@yangli625) profile picture
Joined May 2015
Following the philosophy of Meta CLIP (github.com/facebookresearch/…, arxiv.org/abs/2309.16671) that emphasizes learning from authentic supervision signals, researchers @AIatMeta introduce Pixio (arxiv.org/abs/2512.15715, github.com/facebookresearch/…), a vision foundation model pre-trained solely using #Pixel -level supervision. Pixio is well suited for dense computer vision tasks, including depth estimation (e.g., Depth Anything arxiv.org/abs/2406.09414), feed-forward 3D reconstruction (e.g., MapAnything arxiv.org/abs/2509.13414), semantic segmentation, and robot learning. Built upon the simplicity of Masked Autoencoders (MAE) (arxiv.org/abs/2111.06377), Pixio introduces four major enhancements: 1, deeper decoder, 2, larger masking granularity, 3, more class tokens, 4, web-scale Meta CLIP data with self-curation. This is a joint work done by Lihe Yang (first author of Depth Anything v1, v2), @ShangwenLi1 , @yangli625, Xinjie Lei, @dongwang218, @AbdoMohamedML, @HengshuangZhao, @Hu_Hsu Paper: arxiv.org/abs/2512.15715 GitHub: github.com/facebookresearch/… Huggingface Transformers: huggingface.co/docs/transfor… Huggingface Collections: huggingface.co/collections/f… #AI #Robotics #Robot #3DModel
In collaboration with @AIatMeta, we added support for Pixio in the Transformers library! It proposes 4 changes to Masked AutoEncoders (MAE), including scaling it to 2B images. It outperforms/matches DINOv3 trained at similar scales Find the models here: huggingface.co/collections/f…
5
14
1
30
4,536
In collaboration with @AIatMeta, we added support for Pixio in the Transformers library! It proposes 4 changes to Masked AutoEncoders (MAE), including scaling it to 2B images. It outperforms/matches DINOv3 trained at similar scales Find the models here: huggingface.co/collections/f…
3
38
5
299
37,367
It's a great pleasure to work with you and present the community an OSS representation natively trained for dense CV tasks. As LLM going so fast, we believe dense perception will also catch up!
In collaboration with @AIatMeta, we added support for Pixio in the Transformers library! It proposes 4 changes to Masked AutoEncoders (MAE), including scaling it to 2B images. It outperforms/matches DINOv3 trained at similar scales Find the models here: huggingface.co/collections/f…
2
10
3
23
3,904
Training LLMs needs synthetic data at scale. Existing frameworks don't scale or are hardcoded for one usecase. Introducing Matrix: • 2-15× higher throughput • 10,000s of concurrent workflows • Adapts to diverse generation tasks • Fully open-source arxiv.org/abs/2511.21686
5
6
568
Our Computer Vision group is excited to host Shang-Wen (Daniel) Li and @Hu_Hsu, Researchers from FAIR at Meta, for a session on "MetaCLIP 2: A Worldwide Scaling Recipe" on October 7th. Thanks to community leads @Arkhymadhe and @cataluna84 for organizing this event! 💫 Learn more: cohere.com/events/cohere-lab…
3
8
3
33
10,705
The cross-modal multilingual capability in Meta CLIP 2 is naturally from scaling. It is a reflection of the Bitter Lesson(@RichardSSutton) on building a scalable learning environment, and let optimization and scale do the heavy lifting. Inspired by Jensen Huang’s scaling law plot (at CES2025), the scaling isn’t just about increasing compute or data at each stage—it’s about consistently identifying bottlenecks, removing constraints, and rethinking a simpler setup to make it more scalable. This mindset extends to data curation itself: starting from ImageNet to worldwide data curation. @AIatMeta
🌿Introducing MetaCLIP 2 🌿 📝: arxiv.org/abs/2507.22062 code, model: github.com/facebookresearch/… After four years of advancements in English-centric CLIP development, MetaCLIP 2 is now taking the next step: scaling CLIP to worldwide data. The effort addresses long-standing challenges: (1) large-scale non-English data curation pipelines are largely undeveloped, and (2) the curse of multilinguality, where English performance often degrades in multilingual CLIP compared to English-only CLIP. With a complete recipe for worldwide CLIP—spanning data curation, modeling, and training—we show that English and non-English worlds can mutually benefit and elevate each other, achieving SoTA multilingual performance. Join the Meta booth at #ACL2025 to learn more. (1/3)
3
6
3
23
15,013
Scaling CLIP on English-only data is outdated now… 🌍We built CLIP data curation pipeline for 300+ languages 🇬🇧We train MetaCLIP 2 without compromising English-task performance (it actually improves! 🥳It’s time to drop the language filter! 📝arxiv.org/abs/2507.22062 [1/5] 🧵
3
90
3
305
40,286
🌿Introducing MetaCLIP 2 🌿 📝: arxiv.org/abs/2507.22062 code, model: github.com/facebookresearch/… After four years of advancements in English-centric CLIP development, MetaCLIP 2 is now taking the next step: scaling CLIP to worldwide data. The effort addresses long-standing challenges: (1) large-scale non-English data curation pipelines are largely undeveloped, and (2) the curse of multilinguality, where English performance often degrades in multilingual CLIP compared to English-only CLIP. With a complete recipe for worldwide CLIP—spanning data curation, modeling, and training—we show that English and non-English worlds can mutually benefit and elevate each other, achieving SoTA multilingual performance. Join the Meta booth at #ACL2025 to learn more. (1/3)
14
64
7
336
66,167
Thanks for the invited talk and happy to share our industrial insights on “scaling data alignment” from Meta CLIP (its wide adoption and what’s next) in the DataWorld workshop #ICML2025 . happy to chat offline about data research.
If you are attending #ICML2025, check out our DataWorld workshop on Sat July 19. We have updated the website with more info on speakers & accepted papers! dataworldicml2025.github.io/ Also happy to chat offline about all things ✨ data ✨
2
9
1,145
My team will be at #ICML2025 for invited talk on foundational data research and co-design with training (i.e., MetaCLIP) at DataWorld workshop on Saturday (dataworldicml2025.github.io/…), #SelfCite (selfcite.github.io) on Tuesday, and Meta's booth. @Hu_Hsu @YungSungChuang @cfyehtw
2
4
47
3,230
Web data, the “fossil fuel of AI”, is being exhausted. What’s next?🤔 We propose Recycling the Web to break the data wall of pretraining via grounded synthetic data. It is more effective than standard data filtering methods, even with multi-epoch repeats! arxiv.org/abs/2506.04689
14
65
6
225
36,390
🌿Introducing NaturalThoughts 🌿 arxiv.org/abs/2507.01921 🎯 Data curation for general reasoning capabilities is still relatively underexplored. - We systematically compare different metrics for selecting high-quality and diverse reasoning traces in terms of data efficiency in the distillation setting. - We find diversity in reasoning strategies matters more than topics diversity, and challenging questions are more sample efficient in distilling reasoning capabilities. - We find that the Less-Is-More approach is not sufficient for solving general reasoning tasks, but scaling up data quantity always brings consistent gains. 🧵1/3
1
71
6
400
61,990