Founder / Member of Non Technical Staff @covenant_ai ( @tplr_ai : @basilic_ai : @grail_ai )
Joined April 2014
- Tweets10.1K
- Following2.9K
- Followers4.6K
- Likes35.7K
A small step for mankind, a massive leap for decentralised training... for agency.
In the space of 9 months, @tplr_ai went from 1.2B -> 72B.
It's never been easy, and has broken everyone on the team multiple times. But I speak for all of us when I say it is the most rewarding thing we have ever done.
We have a fraction of the resources. We don't have the PhDs. But Bittensor shows you it doesn't matter. Innovation happens at the edge. We innovate through scarcity.
The ones who rewrite the rules are never the ones with the most. They're the ones who refuse to accept the limits they were handed.
Bittensor is prophecy. Subnets (@covenant_ai and others) are the tools through which that prophecy is manifested.
Next stop: TRILLIONS.
Sam Dare retweeted
Tinker's new StableReinforce objective is Reinforce + Score Centering. Here's how I reverse-engineered it 🧵
Off-policy RL is difficult; runs can suddenly collapse because of async RL staleness or numerical mismatch.
StableReinforce is a new method on Tinker that keeps your RL runs stable, even when learning drifts from sampling.
tinker-docs.thinkingmachines…
Sam Dare retweeted
Minimal harness + strong model = flexible code review agent. That’s the idea behind RevEngine, Laude’s open-source tool now topping @GitHub’s ReviewBench. @NickAJalbert shares how it works, which model the harness helped most, and what else the team learned in the process: laude.org/updates/lab-notes/…
Sam Dare retweeted
Your neo-lab wants to pre-train a frontier model, but no provider will sell you enough co-located chips?
We got you.
Our new work shows that SOAP & SPlus achieve strong worker scaling to 96+ DiLoCo workers (e.g., 96 providers) by whitening the update’s covariance. 🧵1/9
I believe this school of thought is premised on :
- Chinese models been able to distill the closed source frontier
- Chinese models keep been open.
Rumors are that China is placing restrictions on the size of frontier models Chinese companies can release (sub 1T).
I predict get a pretraining resurgence
Sam Dare retweeted
Well said. “We got lucky with pretraining. The people who built Wikipedia and Stack Exchange weren’t making training data; they were making things they wanted to exist. But no one is going to inscribe the work of a tax attorney into a reproducible environment for fun.”
As a field we’re essentially rapidly constructing the internet-sized corpus for post-training on the fly.
I've been thinking about where the data for the next generation of open models will come from.
The internet gave us a remarkable commons for pretraining. Posttraining is different: it consumes tasks, not documents. And that commons doesn't exist. Not yet. openathena.ai/blog/there-is-…
I see where you are coming from , and currently have the same nits.
Thinking of having them work together in a local worktree , then ship that to main
Sam Dare retweeted
Scout’s hardest eval: explain the Datadog bill
Meet Scout 1.0, our first AI model built for investigating production incidents.
Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.
We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.
Learn more and sign up for the preview: bit.ly/3TJknGO
Sam Dare retweeted
Congrats @bthecohen and @datadoghq on releasing #Scout 1.0! I've been waiting for a post-trained SRE model for a while and #Scout makes my day.
Thanks for trying #SREGym !
and thanks for writing a very informative tech report!
web-assets.dd-static.net/425…
To check that Scout does not just fit our harness, we also ran the public SREGym benchmark: 90 faults, with the supplied Stratus harness.
Scout 1.0 xhigh resolves 80 of 90 faults, competitive with the best frontier models at least 2× cheaper. Scout 1.0 medium beats GLM-5.3 at more than 4× lower cost.
Delta weight sync : March 2026
arxiv.org/abs/2602.03839
He’s the CEO of a billion dollar company why does his explanation need to be scientifically rigorous? Not like he’s submitting the tweet to NeurIPS
I think they operate at different ends of the spectrum , exploration v exploitation.
If you wanna rewrite an app, you don’t want your agents trying to figure out how to from first principles .
Here the orchestrator model seems to be the right choice .
However , for research / open ended tasks , I think evolving swarms might be spot on.
This paper seemed pretty interesting on that :
arxiv.org/abs/2610.04862
Sam Dare retweeted
OpenAI's Navier-Stokes effort used ~10,000 concurrent agents and ~130B output tokens. It made us rethink the scale ahead for RL infrastructure.
slime v0.4.0 is a first step toward greater scale, with four architectural changes:
• Focus: synchronous training for algorithmic exploration and correctness; fully async training for throughput at scale.
• Distributed rollout: spread orchestration across cluster CPUs, with independent generation pools and event loops.
• Persistent storage: introduce straw, our new storage middleware for rollout and training data, providing durable queues and shared tensor storage.
• Independent recovery: restart Megatron while healthy SGLang engines stay running, preserving accepted rollout work.
More on the architecture below 👇
github.com/THUDM/slime/relea…
1/5
Sam Dare retweeted
In Mixture of Expert (MoE) models the cost of moving tokens to their experts can dominate total runtime.
At Zyphra research, we use patterns in how tokens are routed to experts to make that communication faster by up to 2.63x on @AMD MI300X GPUs, with the model unchanged.
