Research scientist @GoogleDeepMind working on Gemini reasoning, previously @TransluceAI, @hume_ai, @HarvardMath
Joined April 2023
- Tweets59
- Following553
- Followers201
- Likes264
Long time in the making! Proud to have contributed to Gemini's math capabilities, from building evals that defined our north star to iterating on the post-training recipe and driving large-scale training runs.
Very curious to hear what mathematicians think of the model and how it compares to the competition!
Have always been fascinated by the LessWrong term "pivotal act", which is an action that still matters a **billion** years later. For reference, the age of the universe is 14 billions years.
When one's righteousness is at a cosmic scale, no tactic is off-limits and fighting tooth and nail is an obligation.
strongly disagree with the take that the “AI market is winner take all”. to begin with, if this were true, there wouldn’t be an anthropic. the only reason I could see it being winner takes all is if you plan on “pivotal acts” against all other efforts once you have the capability
2
140
Going to give a mandatory read as usual, but I am pretty sure there is no alpha in here. Memory is too important for Ant to be discussing their main research bets out in the open like this.
Memory is what allows Claude to learn from experience. In a new Engineering Blog post, we share how “dreaming” helps Claude organize what it’s learned, correct its mistakes, and find what it needs more easily the next time. anthropic.com/research/compo…
3
515
Originally thought that applied compute was yet another enterprise AI slop company, but now I see that they are also chasing the light cone.
The chess example is a bit toy but direction is unmistakably ambitious. Continually learning might be the only research direction left to allow a startup to disturb the big labs' lead.
Production traffic is full of real experience from economically valuable work, but very little of it currently makes it back into training.
We built a recipe for production RL that steadily improves a model straight from its deployment’s data stream, then redeploys it to keep learning.
We are sharing some of our initial learnings from a recipe capable of continual learning in production.
Kaiying Hou retweeted
Using Gemini 4 makes you realize how many other models must be distilling Claude/GPT (or using a similar training pipeline). It feels quite different than other models.
Gem 4 is finally here! First public release that I directly worked on, and honored to have contributed to this model's reasoning recipe. One visible thing I pushed for was extending our output limit beyond 64k.
Strong checkpoint but still so much headroom left. It is a very hungry model!
Introducing Gemini 4 Argon – our new frontier model.
It’s built for complex workflows across coding, enterprise knowledge work, and cybersecurity defense – rolling out today to a set of trusted testers through our Fairwind Program.
2
13
807
Congrats to Carlo and Mark Shusterman on getting the paper out!
Not so long ago, I was sitting in Dr. Shusterman’s Math 232 lectures in college. It feels surreal to have co-captained the training of this internal model that helped solve these problems he cared about.
Technical note: this version of Aletheia differed substantially from the original formulation in arxiv.org/abs/2602.10177. One change was that we designed the agent to scale across a wide range of tasks and not be math-specific.
Mark Shusterman and I have posted our resolution of the probabilistic version of Shafarevich's conjecture, posed by Liu--Wood and Sawin--Wood: arxiv.org/abs/2609.30200
The main idea of the proof was discovered independently by GPT5.5 pro and Aletheia, an internal agent at Google DeepMind. While digesting the proof we discovered several new applications. Among these, we settled a conjecture of Nikolov on the existence of universal perfect groups. This latter result came from an interesting experiment in letting the agent pattern-match problems to methods in reverse.
I wanted to see the random group question settled since I studied Melanie and Yuan's paper back in 2021. Very happy this is finally done!
1
10
362
So glad that our elicitation work is out!
Also should check out Transluce's original PRBO blogpost: transluce.org/pathological-b…
If you just pump this number, LLMs start going wild. Was very impressed when the model behavior team showed me this approach back when I joined Transluce.
We used automated elicitation tools to search for strange model behaviors by sampling 100M+ responses, and found:
• Self-harm rituals
• Suicide validation
• Unsolicited flirtation
…etc.
Introducing WeirdChat: the largest public catalog of unexpected model behaviors 🧵(1/)
4
327
Kaiying Hou retweeted
Benchmark distillation, like model distillation, will become critical
Agents can be autonomous for hours now. It's too expensive to always eval them on the real tasks they're meant to do (even SWE-bench eval can cost $500+)
paper from @gneubig & co shows how to cut agent eval cost up to 100x
They showed that performance on selected cheap, non-agentic benchmarks can predict agentic performance quite well
Much better than sub-sampling the agentic benchmark itself
In fact, of all the non-agent benchmark they looked at (AIME, BFCL, etc.), PlanBench is the most predictive of agent performance
🚀Excited to release PACE: A Proxy for Agentic Capability Evaluation!
Evaluating LLM agents on benchmarks like SWE-Bench and GAIA is expensive, slow, and infrastructure-heavy, often costing $$$ and taking hours or days per model.
❓But do we always need to run full agentic evaluations?
In PACE, we show that agentic benchmark performance can be accurately predicted from a small, carefully selected set of cheap non-agentic benchmark instances.
PACE automatically selects proxy instances from existing benchmarks covering skills like instruction following, planning, tool use, reasoning, coding, retrieval, and multimodal understanding.
Across 14 models, 4 agentic benchmarks, and 19 non-agentic benchmarks, PACE-BENCH achieves:
✅ 3.80% MAE for absolute score prediction
✅ 0.81 Spearman correlation for model ranking
✅ ~84% pairwise preference accuracy
✅ ~100× lower cost than target benchmark sampling
Beyond prediction, PACE also reveals what capabilities different agentic benchmarks actually require, e.g., planning, verification, long-context aggregation, and instruction following.
We hope PACE makes agentic evaluation cheaper, faster, and more accessible for model development, model selection, and routing :)
📃 Paper: huggingface.co/papers/2607.0…
💻 Code: github.com/neulab/pace
I'm incredibly grateful to have worked with @lintangsutawika, @Jiarui_Liu_, @lltjuatja, @JiayiiGeng, @lrzneedresearch, @daniel_js_lee, @Aditya_Soni_8, @Vincent92965015, @xiangyue96, and @gneubig .
Kaiying Hou retweeted
The AI safety community constructed a memeplex in which “taking AGI seriously” was a prerequisite for being a serious and good person. When inside this memeplex (as many at Anthropic, some at OpenAI, and a few at DeepMind are) your vision narrows until the world feels extremely constrained. The whole future seems to flow through the “one ring” of controlling recursive self-improvement. And so even when you worry about AI itself seizing that one ring, you can’t generate better strategies than trying to control it yourself (directly via an AGI company, or indirectly via AGI governance).
I’m not saying this is a pure hyperstition. There’s a core truth underlying this perspective: AI will become extremely intelligent and capable, much more than it is today. But the current world is much more spacious and human-empowering than the future which Eliezer originally envisioned (a “brain in a box in a basement” taking over the world by surprise). And it would be even more spacious if this memeplex weren’t active. For example, Satya and Mark and Sundar only started taking AGI seriously because OpenAI forced them to—and even now they don’t really believe in superintelligence—and even if they did they couldn’t get most of their employees on board. Imagine how chill a “race” between Microsoft and Meta and Google would have been, compared with what we have today: Dario and Sam deep in the “one ring” memeplex while also personally loathing each other.
So the one ring memeplex has an escalating life-cycle. It infects people by letting them harness the narrative that they’re good people for taking AGI seriously, and that making other people take AGI seriously is a boon for the world (despite how terribly that’s gone so far). Then it shuts off their imagination—any sparks of creativity or plans that don’t steer towards the one ring are quickly shut down. Instead they make ChatGPT or the METR graph or other recruiting tools for the memeplex. And yes, they’ll acknowledge that previous versions of the memeplex were too extreme, and led to overly constricted action. But we don’t have time to worry about that, they’ll say, because AGI is coming by 2027/2028, and that’s the end of history. Somehow, though, almost everyone with that view has only a vibes-based definition of AGI. They don’t believe in Dyson spheres by 2028, or self-replicating nanotech by 2028, or brain emulations by 2028. They mostly can’t make concrete predictions, except that it’ll be enough AI that it puts all their plans on a deadline. (Shout-out to @DKokotajlo and @paulfchristiano though, who do make concrete predictions about things going crazy soon.)
It seems very hard to break out of this memeplex without just giving up. David Holz is maybe the world champion of that—the only person who was in a position to race for AGI and consciously turned away. Various agent foundations researchers have carved out space to think real thoughts, not the kind of panicky stabbing in the dark that usually passes for safety research. A few others (e.g. Salamon, Hoffman, Vassar, Andre, Sahil, Davidad) are pursuing more unusual paths. And of the people who burned out, I expect some will reorient to doing creative thinking.
For others, the main takeaway: yes, the future of AI will be wild. But so far it’s increased peak human agency, and openness to this trend continuing over the next decade will allow you to start creating something worth creating.
