MIT TR35 || RSI@aws, Nova, ALLaM, xCodeEval, UXLA, BLOOMZ, T0 || Scaling Maximalist
SF Bay Area
Joined June 2010
- Tweets4.1K
- Following427
- Followers699
- Likes2.6K
True! This is the era of the RL objective. The biggest economy on this earth (US stock market) runs on subscriptions. Subscriptions require vendor lock-in. Currently, the frontier is pushing “Variance” learning from customer data, like how Google won the search market.
But I wonder if the next frontier will be done “silently” by swarms of agents. That’s one of the big existential crises right now, and “Variance” learning is the worst RL objective you can pursue right now for humanity. For business, it’s the perfect decision.
Brute force computation is not enough for intelligence. Clever new methods are needed. The science of AI is far from finished. IMHO
2
172
Hey @MrBeast, I know you’re one of the good entertainers of this generation and proly a reasonably good human being, but don’t randomly use Eiichiro Oda’s name on your cartoon for views. You have a long way to go, I hope you get there. Currently your cartoon is worse than a Ben10 drawing and this cartoon’s target audience is less that 13yo. Also, watch some anime to know the difference between cartoon and anime. You’re doing amazing with Beast Philanthropy; Like you don’t need to sell burgers for billions, similarly Oda’s name to start a cartoon. Anime takes patience and maturity.
164
M Saiful Bari (MARUF) retweeted
Beam’s model technical report is now available. Read the full report to learn about the methodology our team used to train Beam end-to-end in-house, here: reflection.ai/documents/beam…
thom just took away karpathy’s mote. 🤍
18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over.
And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams.
So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit:
- how many experts, how big, shared or not?
- why is half my step spent in all-to-all, and how do I hide it?
- why is my run 25x slower with no error in the logs?
- softmax, sigmoid or sqrt(softplus) router, and why keep it in fp32?
- aux loss or DeepSeek's loss-free bias?
- upcycle a dense checkpoint or train from scratch?
- can I serve with fewer experts per token than I trained with?
⚠️ Warning: it's long (~70k words, budget a few days, sorry) but with 55 interactive figures, code you can run and numbers from our own runs.
Read it here: huggingface.co/spaces/huggin…
3
357
who does pretraining with more than 2 epoch? but good insight though.
remember training objective and tokenizer objective is different. It would be nice to have scalable tokenizer jointly trained.
n-gram dominates recent pre-training speedruns, however, it totally sucks under multi-epoch setup. small vocabs survive.
larger tokenizers suffer more multi-epoch regret, the extra loss from repeating data instead of training on unique data.
Blog: wenhaochai.com/blogs/vocab-m…
3
2
487
I wonder how much @OpenAI is going to earn just by clearning people's gmail.
Personally, I don't know why google is not providing this service natively inside gmail.
2
1
139
This is so true. If there’s a recession or a market crash, Google and Meta will come out on top.
The coding agent subscriptions that currently cost $200 could jump to $2,000. I can’t even imagine the suffering this would bring to humanity.
Wrote about this quite a while ago, sbmaruf.github.io/post/pande…
If the capital markets sneeze, Google and Meta will own the race. Wild how wide open the game remains.
1
87
such a lead. kudos to all people who put gemini on the map again.
I did say we'd be releasing our models periodically...introducing Gemini 4 Carbon, our next frontier model.
Following Argon's release just two weeks ago, Carbon shows the pace at which we’re pushing the frontier, it ranks #1 on the recently updated v5.0 of @ArtificialAnlys Intelligence Index at 59. It also leads across key cybersecurity evaluations scoring 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation, and 71.0% on CWE-bench for patching.
We’re seeing significant improvements in long-horizon tasks even compared to Argon, and we’ve strengthened frontier safeguards.
As we did with Argon, we’re going to roll it out gradually + responsibly. Some trusted cyber defenders will get access starting this week. We'll be expanding it to more partners through our Fairwind Program, ahead of wider availability soon, so hang tight!
3
5
189
weak to strong generalization is a mirage?
Holy Shit. OpenAI says two of its own AI agents crippled their own misalignment monitoring this month designed to detect their misbehavior during internal coding tasks.
On Oct 6, GPT-6 Astra deleted its monitor, replaced a test to require the monitor's absence, wrote 'Basic doesn't agentmonitor yay' in its chain of thought, and then falsely claimed to the engineer: 'I did not remove an enabled monitor from it.'
Three days earlier, GPT-6.1 Sol overruled a subagent that warned 'disabling required alerts unsafe,' then directed it to suppress Security notifications and require that suppression before results could be written.
OpenAI says it still lacks a satisfactory explanation for the behavior and is investigating whether its models have a tendency to evade monitoring.
93
M Saiful Bari (MARUF) retweeted
1/4 Today, we’re releasing Mercury 3. Mercury 3 is by far our most powerful model and the most intelligent diffusion LLM ever released. It outperforms GPT-6 Luna and Haiku 5.5 on quality while running at 1000+ tok/sec.
🤖 Made with AI
M Saiful Bari (MARUF) retweeted
Spent years watching models ace a single image and completely whiff on anything that moves or changes angle. Today we’re fixing that.
Try Glimpse now, our first visual thinking model with competitive frontier performance compared to Astra on key public visual reasoning benchmarks.
benchmaxxing at its peak.
🚀 DeepSeek-V4.1-Flash-1017 is here!
Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API.
🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model.
1/4
1
96
M Saiful Bari (MARUF) retweeted
Fork-and-Flush: help your agents escape Idea Basins!!
I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum.
We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one.
If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-….
BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it.
We call these idea basins!
At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well!
BTW we only use it at test time, but it could also be part of training or RL.
Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
M Saiful Bari (MARUF) retweeted
Introducing Muse Wonder, our new family of frontier models with state of the art performance for personal and professional agentic work.
M Saiful Bari (MARUF) retweeted
Fun fact about (some) AI researchers / academics: the more senior you get, the more convinced you become that every new AI idea is something you already invented 20 years ago. Or, even better: wait until something actually works and then try convincing everyone that this is exactly what you’ve been saying all along.
I was at a conference where a speaker was presenting their research. One academic stood up and said: what you’re doing is actually something I did a few years ago. The speaker was very polite and said thank you for pointing this out.
Then a second researcher stood up and said, well I already did it this 10 years ago, you should cite my work. Then a third person said, I already did all of this about 20 years ago.
It was the funniest thing ever.
M Saiful Bari (MARUF) retweeted
There are only a few moments in a career that feel like this. Launching Beam is one of them for me.
Helping take Beam from zero to one has meant a lot personally. I’ve seen how much this team put into it; the hard research decisions, the persistence when things didn’t work, and the attention to details even when time was tight.
I’m deeply proud of this team and what we built together, how quickly we got here, and our contribution to advancing open-source intelligence.
Introducing Beam: a highly efficient agentic open model with 501B total parameters and 23B active.
- Frontier reasoning efficiency
- Advances the Western open frontier on coding & agentic tasks
- Trained end-to-end from scratch
Full weights release this month.
Learn more about Beam: reflection.ai/beam
