Congratulations to @sarahookr and @sudip_r0y on the $50M raise for @adaptionlabs! 🚀
So excited to see you building AI that actually adapts and evolves across languages and cultures.
Beginnings are very special. Today is an important day for @adaptionlabs.
Today a handful of one-size-fits-all-models are optimized for the average use case.
Averages erase the exceptional.
Everything intelligent adapts. So should AI.
This is really nice work! We need more benchmarks to measure multilingual model performance.
So happy our new multilingual benchmark MultiLoKo is finally out (after some sweat and tears!)
arxiv.org/abs/2504.10356
Multilingual eval for LLMs... could be better, and I hope MultiLoKo will help fill some gaps in it + help study design choices in benchmark design
@metaai
Sharan Narang retweeted
We're glad to start getting Llama 4 in all your hands. We're already hearing lots of great results people are getting with these models.
That said, we're also hearing some reports of mixed quality across different services. Since we dropped the models as soon as they were ready, we expect it'll take several days for all the public implementations to get dialed in. We'll keep working through our bug fixes and onboarding partners.
We've also heard claims that we trained on test sets -- that's simply not true and we would never do that. Our best understanding is that the variable quality people are seeing is due to needing to stabilize implementations.
We believe the Llama 4 models are a significant advancement and we're looking forward to working with the community to unlock their value.
Don’t miss this - I’ve worked with Mike (@ml_perception) very closely at Meta and his talks are super informative and fun.
Want to learn about Llama's pre-training? Mike Lewis will be giving a Keynote at NAACL 2025 in Albuquerque, NM on May 1.
2025.naacl.org/
@naaclmeeting
Sharan Narang retweeted
Extremely proud to have pioneered large scale distillation for Maverick and really delighted to be working alongside an extremely talented team.
We truly hope the OSS community enjoys the fruits of our labour.
Today is the start of a new era of natively multimodal AI innovation.
Today, we’re introducing the first Llama 4 models: Llama 4 Scout and Llama 4 Maverick — our most advanced models yet and the best in their class for multimodality.
Llama 4 Scout
• 17B-active-parameter model with 16 experts.
• Industry-leading context window of 10M tokens.
• Outperforms Gemma 3, Gemini 2.0 Flash-Lite and Mistral 3.1 across a broad range of widely accepted benchmarks.
Llama 4 Maverick
• 17B-active-parameter model with 128 experts.
• Best-in-class image grounding with the ability to align user prompts with relevant visual concepts and anchor model responses to regions in the image.
• Outperforms GPT-4o and Gemini 2.0 Flash across a broad range of widely accepted benchmarks.
• Achieves comparable results to DeepSeek v3 on reasoning and coding — at half the active parameters.
• Unparalleled performance-to-cost ratio with a chat version scoring ELO of 1417 on LMArena.
These models are our best yet thanks to distillation from Llama 4 Behemoth, our most powerful model yet. Llama 4 Behemoth is still in training and is currently seeing results that outperform GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on STEM-focused benchmarks. We’re excited to share more details about it even while it’s still in flight.
Read more about the first Llama 4 models, including training and benchmarks ➡️ go.fb.me/gmjohs
Download Llama 4 ➡️ go.fb.me/bwwhe9
Yay! Maverick is #1 open model and #2 overall
The future of AI is open source - Come join us! I’m hiring
BREAKING: Meta's Llama 4 Maverick just hit #2 overall - becoming the 4th org to break 1400+ on Arena!🔥
Highlights:
- #1 open model, surpassing DeepSeek
- Tied #1 in Hard Prompts, Coding, Math, Creative Writing
- Huge leap over Llama 3 405B: 1268 → 1417
- #5 under style control
Huge congrats to @AIatMeta — and another big win for open-source! 👏 More analysis below⬇️
Wooooohoooo - Lets go Llama 4!
Introducing our first set of Llama 4 models!
We’ve been hard at work doing a complete re-design of the Llama series. I’m so excited to share it with the world today and mark another major milestone for the Llama herd as we release the *first* open source models in the Llama 4 collection 🦙. Here are some highlights:
📌 The Llama series have been re-designed to use state of the art mixture-of-experts (MoE) architecture and natively trained with multimodality. We’re dropping Llama 4 Scout & Llama 4 Maverick, and previewing Llama 4 Behemoth.
📌 Llama 4 Scout is highest performing small model with 17B activated parameters with 16 experts. It’s crazy fast, natively multimodal, and very smart. It achieves an industry leading 10M+ token context window and can also run on a single GPU!
📌 Llama 4 Maverick is the best multimodal model in its class, beating GPT-4o and Gemini 2.0 Flash across a broad range of widely reported benchmarks, while achieving comparable results to the new DeepSeek v3 on reasoning and coding – at less than half the active parameters. It offers a best-in-class performance to cost ratio with an experimental chat version scoring ELO of 1417 on LMArena. It can also run on a single host!
📌 Previewing Llama 4 Behemoth, our most powerful model yet and among the world’s smartest LLMs. Llama 4 Behemoth outperforms GPT4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on several STEM benchmarks. Llama 4 Behemoth is still training, and we’re excited to share more details about it even while it’s still in flight.
A big thanks to all of our launch partners (full list in blog) for helping us bring Llama 4 to developers everywhere including @huggingface, @togethercompute, @SnowflakeDB, @ollama, @databricks and many others👏 This is just the start, we have more models coming and the team is really cooking – look out for Llama 4 Reasoning 😉
A few weeks ago, we celebrated Llama being downloaded over 1 billion times. Llama 4 demonstrates our long-term commitment to open source AI, the entire open source AI community, and our unwavering belief that open systems will produce the best small, mid-size and soon frontier models. Llama would be nothing without the global open source AI community & we are so ready to begin this next chapter with you. 🦙
Read more about the release here: llama.com/, and try it in our products today.
Very excited to share Llama 4 models with the world. The pre-training team has cooked over the past few months to launch Llama 4 Scout, Maverick, and Behemoth.
A 🧵about pretraining
Blog link: ai.meta.com/blog/llama-4-mul…
We’ve also improved our mid-training recipe to improve the capabilities of these models and expand the context length. Llama 4 Scout supports 10M token length during inference!
Llama 3 405B was trained on 16k GPUs and training efficiency was a huge focus. We achieved 43% peak MFU while training on 8k GPUs. @vedanujg highlights all the improvements that enabled us achieve these results
xitter.cf/vedanujg/status/181579…
Llama3 405B is out with impressive performance across the board and 128k ctx length! Along with improved models for 8B and 70B.
Blog : llama.meta.com/llama-downloa…
Paper : ai.meta.com/research/publica…
A short 🧵 on what went into scaling and training the largest compute Llama model yet
The updated license is really cool since it enables developers to use the models for synthetic data generation and distillation. I'm very excited to see what the community builds with these models.
As promised a while back, we've published a research paper with all the findings.
Onward to Llama 4🦙🦙🦙🦙🚀
ai.meta.com/research/publica…
xitter.cf/sharan0909/status/1781…
It's also great to see that Scale AI evaluated our model and showed that it is competitive all closed source models on coding, math, and instruction following.
Full thread: xitter.cf/alexandr_wang/status/1…
