LLM failure case:
convert the list of papers from mlsys.org/virtual/2026/calen… into a google sheet. Three columns: paper title, author, arxiv link.
80
Dong Wang retweeted
Following the philosophy of Meta CLIP (github.com/facebookresearch/…, arxiv.org/abs/2309.16671) that emphasizes learning from authentic supervision signals, researchers @AIatMeta introduce Pixio (arxiv.org/abs/2512.15715, github.com/facebookresearch/…), a vision foundation model pre-trained solely using #Pixel -level supervision.
Pixio is well suited for dense computer vision tasks, including depth estimation (e.g., Depth Anything arxiv.org/abs/2406.09414), feed-forward 3D reconstruction (e.g., MapAnything arxiv.org/abs/2509.13414), semantic segmentation, and robot learning.
Built upon the simplicity of Masked Autoencoders (MAE) (arxiv.org/abs/2111.06377), Pixio introduces four major enhancements:
1, deeper decoder,
2, larger masking granularity,
3, more class tokens,
4, web-scale Meta CLIP data with self-curation.
This is a joint work done by Lihe Yang (first author of Depth Anything v1, v2), @ShangwenLi1
, @yangli625, Xinjie Lei, @dongwang218, @AbdoMohamedML, @HengshuangZhao, @Hu_Hsu
Paper: arxiv.org/abs/2512.15715
GitHub: github.com/facebookresearch/…
Huggingface Transformers: huggingface.co/docs/transfor…
Huggingface Collections: huggingface.co/collections/f…
#AI #Robotics #Robot #3DModel
In collaboration with @AIatMeta, we added support for Pixio in the Transformers library!
It proposes 4 changes to Masked AutoEncoders (MAE), including scaling it to 2B images. It outperforms/matches DINOv3 trained at similar scales
Find the models here: huggingface.co/collections/f…
Dong Wang retweeted
It's a great pleasure to work with you and present the community an OSS representation natively trained for dense CV tasks. As LLM going so fast, we believe dense perception will also catch up!
In collaboration with @AIatMeta, we added support for Pixio in the Transformers library!
It proposes 4 changes to Masked AutoEncoders (MAE), including scaling it to 2B images. It outperforms/matches DINOv3 trained at similar scales
Find the models here: huggingface.co/collections/f…
Dong Wang retweeted
In collaboration with @AIatMeta, we added support for Pixio in the Transformers library!
It proposes 4 changes to Masked AutoEncoders (MAE), including scaling it to 2B images. It outperforms/matches DINOv3 trained at similar scales
Find the models here: huggingface.co/collections/f…
Training LLMs needs synthetic data at scale. Existing frameworks don't scale or are hardcoded for one usecase.
Introducing Matrix:
• 2-15× higher throughput
• 10,000s of concurrent workflows
• Adapts to diverse generation tasks
• Fully open-source
arxiv.org/abs/2511.21686
Dong Wang retweeted
🌿Introducing MetaCLIP 2 🌿
📝: arxiv.org/abs/2507.22062
code, model: github.com/facebookresearch/…
After four years of advancements in English-centric CLIP development, MetaCLIP 2 is now taking the next step: scaling CLIP to worldwide data. The effort addresses long-standing challenges: (1) large-scale non-English data curation pipelines are largely undeveloped, and (2) the curse of multilinguality, where English performance often degrades in multilingual CLIP compared to English-only CLIP. With a complete recipe for worldwide CLIP—spanning data curation, modeling, and training—we show that English and non-English worlds can mutually benefit and elevate each other, achieving SoTA multilingual performance. Join the Meta booth at #ACL2025 to learn more.
(1/3)
Dong Wang retweeted
AI Research Agents are becoming proficient at machine learning tasks, but how can we help them search the space of candidate solutions and codebases? Read our new paper looking at MLE-Bench: arxiv.org/pdf/2507.02554
#LLM #Agents #MLEBench
Dong Wang retweeted
🌿Introducing NaturalThoughts 🌿
arxiv.org/abs/2507.01921
🎯 Data curation for general reasoning capabilities is still relatively underexplored.
- We systematically compare different metrics for selecting high-quality and diverse reasoning traces in terms of data efficiency in the distillation setting.
- We find diversity in reasoning strategies matters more than topics diversity, and challenging questions are more sample efficient in distilling reasoning capabilities.
- We find that the Less-Is-More approach is not sufficient for solving general reasoning tasks, but scaling up data quantity always brings consistent gains.
🧵1/3
Any good alternatives after Mendeley stopped IOS app? blog.mendeley.com/2021/03/11…
Dong Wang retweeted
Dataset from monitoring faces on TV: After Trump, GOP leaders in Congress are most popular faces on TV. Collected with @Matroid by @InternetArchive, looking at: BBC News, CNN, Fox, MSNBC.
Data: archive.org/details/faceomat…
Blog: blog.archive.org/2018/05/18/…
Code: github.com/slifty/tvarchive-…
Dong Wang retweeted
We're proud to announce two new products:
- Matroid Stream Monitoring
- Matroid On-Prem
Video: youtu.be/jfSRq1-VI9c
Dong Wang retweeted
Videos, slides, and photos from ScaledML 2018.
Intro: youtube.com/watch?v=hxOHrnSG…
All: matroid.com/blog/post/slides…
At #TFDevSummit, morning session: tf.data prefetch for input pipelining, synchronous multiple gpu training at 90% efficiency, inference/train using tensorflow.js, tensorboard debugger and interactive supervision
Dong Wang retweeted
Some photos I took for a super successful #scaledmlconf this weekend! teapawt.net/matroid/status/9784391…
Photos from the Scaled Machine Learning Conference this past weekend. #ScaledMConf photos.app.goo.gl/PRGLn94YEL…
Yangqing: Facebook regularly shutdown one cluster to test service stability #scaledmlconf
1
Jennifer Chayes of MSR on machine learning for gen editing on off target prediction, T cell receptor TCR to disease antigen prediction #scaledmlconf
3


























