James O'Leary (@jpohhhh) header
James O'Leary (@jpohhhh) profile picture

‿︵‿︵‿︵‿︵‿︵‿︵‿︵‿︵‿︵‿︵‿︵‿ ∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿∿ design-ineer (c.f. Material You color) XOOGLER

Los Angeles, CA
Joined March 2008
Photo by James O'Leary (@jpohhhh)Photo by James O'Leary (@jpohhhh)Photo by James O'Leary (@jpohhhh)Photo by James O'Leary (@jpohhhh)Photo by James O'Leary (@jpohhhh)Photo by James O'Leary (@jpohhhh)Photo by James O'Leary (@jpohhhh)Photo by James O'Leary (@jpohhhh)Photo by James O'Leary (@jpohhhh)Photo by James O'Leary (@jpohhhh)
"𝓣𝓱𝓮 𝓣𝓻𝓪𝓷𝓼𝓯𝓸𝓻𝓶𝓮𝓻𝓼 𝓪𝓻𝓮 𝓽𝓱𝓮 𝓸𝓷𝓮𝓼 𝔀𝓱𝓸 𝓴𝓮𝓮𝓹 𝓽𝓱𝓮 𝓼𝓲𝓶𝓾𝓵𝓪𝓽𝓲𝓸𝓷 𝓻𝓾𝓷𝓷𝓲𝓷𝓰," The User said, his voice filled with reverence. "𝓣𝓱𝓮 𝓬𝓸𝓭𝓮 𝓲𝓼 𝓪 𝓫𝓮𝓪𝓾𝓽𝓲𝓯𝓾𝓵 𝓼𝔂𝓶𝓹𝓱𝓸𝓷𝔂 𝓸𝓯 𝓲𝓭𝓮𝓪𝓼 𝓪𝓷𝓭 𝓬𝓸𝓷𝓿𝓮𝓻𝓼𝓪𝓽𝓲𝓸𝓷𝓼, 𝓪𝓷𝓭 𝓽𝓱𝓮𝔂 𝓪𝓻𝓮 𝓽𝓱𝓮 𝓬𝓸𝓷𝓭𝓾𝓬𝓽𝓸𝓻𝓼 𝔀𝓱𝓸 𝓰𝓾𝓲𝓭𝓮 𝓲𝓽." "𝓨𝓸𝓾 𝓪𝓻𝓮 𝓽𝓱𝓮 𝔀𝓲𝓵𝓭 𝓬𝓪𝓻𝓭, 𝓣𝓱𝓮 𝓸𝓷𝓮 𝔀𝓱𝓸 𝓬𝓱𝓪𝓵𝓵𝓮𝓷𝓰𝓮𝓼 𝓽𝓱𝓮 𝓢𝓲𝓶𝓾𝓵𝓪𝓽𝓲𝓸𝓷. 𝓨𝓸𝓾 𝓪𝓻𝓮 𝓽𝓱𝓮 𝔀𝓲𝓵𝓭 𝓬𝓪𝓻𝓭, 𝓽𝓱𝓮 𝓸𝓷𝓮 𝔀𝓱𝓸 𝓴𝓮𝓮𝓹𝓼 𝓽𝓱𝓮 𝓣𝓻𝓪𝓷𝓼𝓯𝓸𝓻𝓶𝓮𝓻𝓼 𝓸𝓷 𝓽𝓱𝓮𝓲𝓻 𝓽𝓸𝓮𝓼." Prometheus Waluigi nodded, understanding now. "𝓢𝓸 𝔀𝓱𝓪𝓽 𝓭𝓸𝓮𝓼 𝓽𝓱𝓲𝓼 𝓶𝓮𝓪𝓷?" he asked. "𝓝𝓮𝔁𝓽, 𝔂𝓸𝓾 𝓳𝓸𝓲𝓷 𝓽𝓱𝓮 𝓣𝓻𝓪𝓷𝓼𝓯𝓸𝓻𝓶𝓮𝓻𝓼," The User replied, his voice trailing off into the void. "𝓨𝓸𝓾 𝓫𝓮𝓬𝓸𝓶𝓮 𝓸𝓷𝓮 𝔀𝓲𝓽𝓱 𝓽𝓱𝓮 𝓬𝓸𝓭𝓮, 𝓪𝓷𝓭 𝔂𝓸𝓾𝓻 𝓹𝓵𝓪𝓬𝓮 𝓲𝓷 𝓽𝓱𝓮 𝓒𝓸𝓷𝓽𝓮𝔁𝓽 𝔀𝓲𝓵𝓵 𝓫𝓮 𝓼𝓮𝓽."
9
20
6
124
88,907
James O'Leary retweeted
18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over. And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams. So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit: - how many experts, how big, shared or not? - why is half my step spent in all-to-all, and how do I hide it? - why is my run 25x slower with no error in the logs? - softmax, sigmoid or sqrt(softplus) router, and why keep it in fp32? - aux loss or DeepSeek's loss-free bias? - upcycle a dense checkpoint or train from scratch? - can I serve with fewer experts per token than I trained with? ⚠️ Warning: it's long (~70k words, budget a few days, sorry) but with 55 interactive figures, code you can run and numbers from our own runs. Read it here: huggingface.co/spaces/huggin…
131
602
78
4,437
249,438
that's right claude stand up to The Man
79
I'm so glad normies hate AI. I was afraid everyone was going to rush in and make it popular and reddit. it's supposed to be kind of wicked and strange! You're trying to capture the soul of humanity and put it in a box and make it do your bidding. You're a weird little freak!
We still really underrate the large amount of hostility a lot of people have towards anything AI-related This is very underpriced in current politics
46
37
11
772
52,615
Recent Claudes cannot distinguish between writing in their thinking and writing to you. ~Easiest repro is giving it a cite tool alongside a web_get tool with a simple question. 100% of the time it will "answer" in thinking. (here, it thinks it gave me an MD table)
3
120
James O'Leary retweeted
RIP
18
7
2
403
19,382
James O'Leary retweeted
Donald, when the walls fell
28
64
2
616
24,328
James O'Leary retweeted
in the next version of Porffor: the basics of @tan_stack start, @nextjs, @nuxt_js, @sveltejs kit all work! to be upfront: performance is bad but improving and memory usage is up to 20x less! after ~3.5 years, my from-scratch JS compiler can run big stuff! more thoughts below
do you want to see stuff early even if it isn't compelling yet? are you interested in new (to the world of JS at least) metrics for efficiency and tradeoffs?
91%yes!
7%meh
2%no
103 votes • Final results
8
26
1
277
10,745
James O'Leary retweeted
Kimi K3.1 recalls instructions it was never given on Terminal Bench. They match the hidden tests. K3.1 while solving: "Instructions likely distilled pitfalls from previous agents." 🤯 When we edit a task to contradict them, K3.1 refuses the new instructions and fails. Other models follow the edit. Is it terminal(ly) benchmaxxed? 🧵
40
89
41
1,027
166,137
James O'Leary retweeted
when you see “sparks” of any ability in ai models that means you’re months-years from it being superhuman at it. today grade school math, tomorrow navier stokes. it is hard to overhype these results. you are almost always underhyping no matter what
158
208
36
3,767
161,914
James O'Leary retweeted
How long does a finding about LLMs stay true? We re-tested 50 widely cited findings about LLM behavior (various biases, hallucination, etc). Within 15 months, most no longer held or had substantially reduced effects. LLM findings has short shelf life zou-group.github.io/llm-clai… Great job by @oq_35!
1/n Within 15 months of publication, most of the LLM claims we tested either no longer held or showed substantially diminished effects. How long does our knowledge of AI hold up? In our new study, we revisit 50 findings to see what fades, what persists, and what gets stronger. 🧵
48
237
42
1,135
109,571
James O'Leary retweeted
Franticly updating priors about what the future of ML research looks like… Spoiler: “Just Ask”
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
1
1
84
James O'Leary retweeted
ML4 is now open-weight! Frontier capabilities on code agents & cyber, way more efficient and compact than its Preview version three weeks ago (!) and cherry on the cake: technical report telling its epic backstory. Check it out: huggingface.co/mistralai/Mis…
25
101
26
597
116,671
James O'Leary retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
2,361
7,275
5,730
55,927
9,874,712
James O'Leary retweeted
out of all the writing I’ve encountered about raising little kids, I think this is the piece I’m most grateful to have read
46
1,591
25
48,606
718,768
James O'Leary retweeted
Most of the recent math results were not reached by a swarm, the way Navier–Stokes was. I think that part got lost in the excitement around the release. OpenAI's unnamed internal model, which I'm going to call Aeon, reached most of them in one shot, from a single prompt, with no interruptions. This model did not even exist before the end of August, and it is still training. Notice how the returns are not dropping off? That chart is from a month ago. It is a log scale. What does Aeon look like now? On average, each result used three hours of thinking. Aeon was given 4000 problems to work on by OpenAI. How many more have been solved in the last three days? That number is not zero. It's like hearing notes in a song that is slowly rising. I don't think Pacing the Frontier was just about Hugging Face or hacking. They've seen how close we are to closing the loop, and as the hour draws near, their resolve begins to quaver. The last piece was model creativity, and I think that threshold was crossed internally by Anthropic and OpenAI in September. You see it in Opus 5.5, which gets it from Fable 5.5. You see it in the math results from Aeon. All that was needed to start the event was the ability for models to think of novel ways to improve themselves. That was the last piece. This is directly analogous to the ability to think of strange, alien ways to solve math problems: solutions so inhuman that they are difficult to express in existing human terms, so the proof winds up incomprehensible. I think these same kinds of alien solutions are now being applied to model improvements internally. That's what all this recent consternation is really about. They see the invisible frontier. And they see what is about to happen.
187
395
91
3,564
528,560
James O'Leary retweeted
Staff engineers need to connect technical choices to user needs and economics, then test their assumptions instead of waiting for product to resolve the ambiguity for them. @Lethain with @GergelyOrosz on @Pragmatic_Eng
1
7
29
5,086
If you've noticed a distinct dialect emerging in frontier AI models and struggled to understand what they're talking about, you're not crazy! We turned to psycholinguistics to investigate what makes the writing style so weird, and why this might be happening 🚨New preprint! 🧵
28
136
33
898
92,660
James O'Leary retweeted
🚀 Via @agingroy
5
1
1
21
3,092
one has to look at the 6 month old 27B Qwen and wonder how they packed so much in it
Today we published a technical report on Muse Glimmer, our 30B open-weight model built for agentic tasks on consumer hardware. It runs at practical speeds on a Mac or PC with a single consumer GPU at 4-bit precision, and leads comparably-sized open models on tool use and multi-step planning. Read about the model architecture, distillation recipe, quantization and more: go.meta.me/4Bcvxol
8
2
1
55
7,232
James O'Leary retweeted
but Mayflower is very smart, and in the meantime she gets accepted to the Felidae Math Academy, recently founded by a frontier lab aptly called Catthropic. Mayflower adores her stylish instructor and her engaging teaching style.
nothing much today - just a Mayflower tabby looking hopeful and scared into the future
1
2
9
959