
RL infra, robotics @LumaLabsAI prev @UnslothAI @axolotl_ai @awscloud @UMassAmherst small homestead owner 🐑🐓
Brimfield, MA
Joined December 2008
- Tweets6K
- Following2.8K
- Followers907
- Likes25K
Dan Saunders retweeted
Fable 5.2's launch impressed with strong robot manipulation. How does ZAP, our 27B model, do on the same tasks?
We recreated the tasks in sim and ran both through the same harness (the model is the policy, calling robot tools directly). ZAP only trained on sim pick-and-place, so every object and goal is new to it.
Task 1, open a bottle: ZAP is 2.5x faster and costs $0.43 vs. Fable's $6.50. It plans the grasp up front, and it pays off: a firm hold and clean turns. Fable drops the bottle, recovers nicely, and still finishes.
Sim only for now since I'm at CoRL, but the real robot is waiting back home 👀 🧵
1/4
Our infra has really improved quite a bit since the last time we released a model - this one took just under 3 days to pretrain!
Also, it's quite cheap on Tinker (0.30/1.00/1.50 for in/out/train)! Hope you guys find this model great for customization purposes :)
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor.
Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon.
thinkingmachines.ai/news/ink…
Dan Saunders retweeted
Luma trained a 27B open model only in simulation. zero real robot data.
on a real two-arm robot it ties GPT-6 Astra 🤖
🧠 base Qwen already plans right 95% of the time, but only 36% of its moves are right
🎮 RL on 40 simple sim tasks takes that to 85%
⚡ avg 146 s vs 397 s and $0.29 vs $4.87 per task, same harness
why it matters: pretraining knows what, sim teaches how
Dan Saunders retweeted
Mechanistic interpretability was a huge missed opportunity. Many people wanted to join that effort, assuming it meant a mechanistic understanding of generalization. Instead, it was a bait and switch into an extremely niche, esoteric, and largely ill-conceived effort around understanding the roles of individual neurons. This has often been a challenge in alignment and interpretability: compelling initiatives that ought to be broadly defined, or at least mean what we would expect they mean, end up being restricted to a very narrow and esoteric set of topics. For example, if someone says they work on “interpretability” it does not mean what one would naively assume, e.g., interpretable methods, algorithms, representations, or model deconstruction, understanding the data, etc., but rather sparse autoencoders, influence functions, saliency maps. What a waste!
Dan Saunders retweeted
Exact sequence parallelism for packed multimodal masks.
It picks ring attention, a sparse all-to-all or Ulysses per layer, matches single-GPU numerics, and beats a FlashMask + Ulysses hybrid in 47 of 48 cases on up to 128 H100s.
lumalabs.ai/news/packed-mult…
Dan Saunders retweeted
These 0.99999984 solutions are not as "ugly" as it may seem.
The truth is, those are not the real runtimes of the algorithms published. To make the proof simpler, they use absurd safety bounds like 2^50 that are entirely arbitrary.
For example, imagine proving that 1/x^2 > 0. This proof is much harder if you include the area around x=0, as it blows up to infinity there. Instead, if you only prove it for x>=99999, it avoids this blowup and simplifies the proof, but still gets the point across.
The "real" bounds of those solutions are probably much tighter, maybe even something "beautiful" like O(n*lg lg n). But the goal of the AI papers was just to show that improvement is possible, not that the bounds are perfect (which is usually much harder to prove). Over the coming weeks and months, we'll likely see a lot of tightening.
Dan Saunders retweeted
opsd and its variants don’t work, no matter what they are biased and bias kills llms, please please please stop working on it, so many other great directions to explore that aren’t opsd
Meta's team proposes DCE + SRCL as an alternative to on-policy self-distillation (OPSD).
Achieved significant gains: 30.76% avg acc -> 65.97% on Qwen3-8B
arxiv.org/abs/2609.30652
Nearly half a year of silence. We spent it studying one problem: how far RL can scale.
MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks.
Streaming the run: mimo.xiaomi.com/rl/
Dan Saunders retweeted
1/ ✨ World record today on bounded gaps between primes, one of number theory’s oldest open conjectures.
We show that infinitely many pairs of prime numbers are separated by 212 or less.
Dan Saunders retweeted
🧵Scaling Laws for T2I Diffusion
We do a scaling laws ladder for t2i diffusion models all the way from 60M to 2B, spanning 3 orders of magnitude in training flops
Dan Saunders retweeted
Also, it’s much easier for an infra engineer to pick up ML research while it seems almost impossible for an ML researcher to actually build the infra, especially the one that scales ;)
my spicy theory is that chinese ai labs keep winning because of culture, not talent or resources.
chinese labs still have a deeply hands-on engineering culture. deepseek and kimi are flat organizations where science and engineering are fused: the same people move between algorithms, data, and infra, doing whatever it takes to make the model work.
but sf tech bros have decided that “researcher” is the high-status title while infra is merely support work. every new sf ai startup calls itself a “lab,” every ambitious engineer quietly upgrades their title to “research engineer,” and the infra work is left to whoever failed to escape it.
but at frontier scale, infra determines experiment velocity, and experiment velocity determines research output. at frontier scale, infra **is** research.
AI ppl loooove to pretend they hold tons secret knowledge. Everyone has Seen Things, and have been trusted with the Esoteric Knowledge. It's also cringe if you don't partake because it implies you're not in the Inner Circle. In reality most people are kinda confused, working it all out ad hoc, relying on the same rumours, and so on - but the status games are hard to resist!
Dan Saunders retweeted
At humans&, we train models from the long-term impacts of their interactions with people. This requires prioritizing long-horizon multi-agent RL. We've developed and are excited to share an open-source, hardware-native 4-bit RL recipe, significantly accelerating training
Dan Saunders retweeted
a very hungry caterpillar is ~32 pages.
it would take you 20 minutes of focused reading to read it.
the opportunity cost of that time is too high to warrant reading a very hungry caterpillar
Dan Saunders retweeted
Humanity has not prevailed ( • ᴖ • 。)
AWTF Heuristic is now over and OpenAI has completely demolished human competitors.
In the end, humans performed quite well, but the performance of OpenAI was outstanding.
I'll post my detailed thoughts about this in a few days, since this deserves a longer commentary.

































