Dan Saunders (@djsaunde) header
Dan Saunders (@djsaunde) profile picture

RL infra, robotics @LumaLabsAI prev @UnslothAI @axolotl_ai @awscloud @UMassAmherst small homestead owner 🐑🐓

Brimfield, MA
Joined December 2008
Photo by Dan Saunders (@djsaunde)Photo by Dan Saunders (@djsaunde)Photo by Dan Saunders (@djsaunde)Photo by Dan Saunders (@djsaunde)Photo by Dan Saunders (@djsaunde)Photo by Dan Saunders (@djsaunde)Photo by Dan Saunders (@djsaunde)Photo by Dan Saunders (@djsaunde)Photo by Dan Saunders (@djsaunde)Photo by Dan Saunders (@djsaunde)
Fable 5.2's launch impressed with strong robot manipulation. How does ZAP, our 27B model, do on the same tasks? We recreated the tasks in sim and ran both through the same harness (the model is the policy, calling robot tools directly). ZAP only trained on sim pick-and-place, so every object and goal is new to it. Task 1, open a bottle: ZAP is 2.5x faster and costs $0.43 vs. Fable's $6.50. It plans the grasp up front, and it pays off: a firm hold and clean turns. Fable drops the bottle, recovers nicely, and still finishes. Sim only for now since I'm at CoRL, but the real robot is waiting back home 👀 🧵 1/4
Astra made robot control look like a job for frontier LLMs. But small VLMs already know what to do. They just don't know how. Turns out, simulation is all you need. Meet ZAP. No real robot data, no frontier teacher. Matches Astra on real tasks, 3× faster, 17× cheaper. 🧵 (1/11)
3
10
47
2,854
Dan Saunders retweeted
Our infra has really improved quite a bit since the last time we released a model - this one took just under 3 days to pretrain! Also, it's quite cheap on Tinker (0.30/1.00/1.50 for in/out/train)! Hope you guys find this model great for customization purposes :)
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor. Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon. thinkingmachines.ai/news/ink…
41
36
14
821
75,164
Luma trained a 27B open model only in simulation. zero real robot data. on a real two-arm robot it ties GPT-6 Astra 🤖 🧠 base Qwen already plans right 95% of the time, but only 36% of its moves are right 🎮 RL on 40 simple sim tasks takes that to 85% ⚡ avg 146 s vs 397 s and $0.29 vs $4.87 per task, same harness why it matters: pretraining knows what, sim teaches how
9
7
1
45
3,276
Mechanistic interpretability was a huge missed opportunity. Many people wanted to join that effort, assuming it meant a mechanistic understanding of generalization. Instead, it was a bait and switch into an extremely niche, esoteric, and largely ill-conceived effort around understanding the roles of individual neurons. This has often been a challenge in alignment and interpretability: compelling initiatives that ought to be broadly defined, or at least mean what we would expect they mean, end up being restricted to a very narrow and esoteric set of topics. For example, if someone says they work on “interpretability” it does not mean what one would naively assume, e.g., interpretable methods, algorithms, representations, or model deconstruction, understanding the data, etc., but rather sparse autoencoders, influence functions, saliency maps. What a waste!
43
68
11
952
86,198
Dan Saunders retweeted
i guess we're doing a paper thread for this one...
4
17
5
180
14,147
Dan Saunders retweeted
Exact sequence parallelism for packed multimodal masks. It picks ring attention, a sparse all-to-all or Ulysses per layer, matches single-GPU numerics, and beats a FlashMask + Ulysses hybrid in 47 of 48 cases on up to 128 H100s. lumalabs.ai/news/packed-mult…
7
17
4
100
13,213
These 0.99999984 solutions are not as "ugly" as it may seem. The truth is, those are not the real runtimes of the algorithms published. To make the proof simpler, they use absurd safety bounds like 2^50 that are entirely arbitrary. For example, imagine proving that 1/x^2 > 0. This proof is much harder if you include the area around x=0, as it blows up to infinity there. Instead, if you only prove it for x>=99999, it avoids this blowup and simplifies the proof, but still gets the point across. The "real" bounds of those solutions are probably much tighter, maybe even something "beautiful" like O(n*lg lg n). But the goal of the AI papers was just to show that improvement is possible, not that the bounds are perfect (which is usually much harder to prove). Over the coming weeks and months, we'll likely see a lot of tightening.
Oh 3sum takes O(n^1.9992) time (!?). Oh int multiplication takes O(n (lg n)^0.99999999999999999999999999999999999999999999999999999984) time (!?!?!?!?!?!?!?!?!?). Odds on most asymptotic complexities ending up with finicky complicated inelegant constants just shot up.
13
66
8
1,905
133,568
Dan Saunders retweeted
opsd and its variants don’t work, no matter what they are biased and bias kills llms, please please please stop working on it, so many other great directions to explore that aren’t opsd
Meta's team proposes DCE + SRCL as an alternative to on-policy self-distillation (OPSD). Achieved significant gains: 30.76% avg acc -> 65.97% on Qwen3-8B arxiv.org/abs/2609.30652
24
34
11
439
89,226
Dan Saunders retweeted
Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks. Streaming the run: mimo.xiaomi.com/rl/
423
974
554
9,876
3,140,351
Dan Saunders retweeted
1/ ✨ World record today on bounded gaps between primes, one of number theory’s oldest open conjectures.  We show that infinitely many pairs of prime numbers are separated by 212 or less.
35
83
48
696
291,053
Dan Saunders retweeted
🧵Scaling Laws for T2I Diffusion We do a scaling laws ladder for t2i diffusion models all the way from 60M to 2B, spanning 3 orders of magnitude in training flops
13
45
9
286
44,154
Dan Saunders retweeted
when a RL researcher does SFT for you, it is a sign of great respect in their culture
15
22
4
616
55,534
Dan Saunders retweeted
Also, it’s much easier for an infra engineer to pick up ML research while it seems almost impossible for an ML researcher to actually build the infra, especially the one that scales ;)
my spicy theory is that chinese ai labs keep winning because of culture, not talent or resources. chinese labs still have a deeply hands-on engineering culture. deepseek and kimi are flat organizations where science and engineering are fused: the same people move between algorithms, data, and infra, doing whatever it takes to make the model work. but sf tech bros have decided that “researcher” is the high-status title while infra is merely support work. every new sf ai startup calls itself a “lab,” every ambitious engineer quietly upgrades their title to “research engineer,” and the infra work is left to whoever failed to escape it. but at frontier scale, infra determines experiment velocity, and experiment velocity determines research output. at frontier scale, infra **is** research.
6
12
290
35,310
Dan Saunders retweeted
33
649
40
11,201
283,837
Dan Saunders retweeted
AI ppl loooove to pretend they hold tons secret knowledge. Everyone has Seen Things, and have been trusted with the Esoteric Knowledge. It's also cringe if you don't partake because it implies you're not in the Inner Circle. In reality most people are kinda confused, working it all out ad hoc, relying on the same rumours, and so on - but the status games are hard to resist!
63
71
25
985
80,293
Dan Saunders retweeted
Step 1 - never Step 2 - kill yourself
4
2
35
1,763
Dan Saunders retweeted
At humans&, we train models from the long-term impacts of their interactions with people. This requires prioritizing long-horizon multi-agent RL. We've developed and are excited to share an open-source, hardware-native 4-bit RL recipe, significantly accelerating training
14
148
31
1,168
287,565
a very hungry caterpillar is ~32 pages. it would take you 20 minutes of focused reading to read it. the opportunity cost of that time is too high to warrant reading a very hungry caterpillar
31
19
525
13,799
Dan Saunders retweeted
Humanity has not prevailed ( • ᴖ • 。) AWTF Heuristic is now over and OpenAI has completely demolished human competitors. In the end, humans performed quite well, but the performance of OpenAI was outstanding. I'll post my detailed thoughts about this in a few days, since this deserves a longer commentary.
32
138
45
1,059
163,351
Excited to announce an arXiv note on an interesting mathematical symmetry I noticed... that connects the classic MLP to the Gated MLP in a way beyond 'it empirically works better'. Thread below!
10
14
194
20,700