ceo/co-founder of autoresearch, inc. | prev ml enjoyer @reductoai @dynamo_ai, math enjoyer @uoft
San Francisco, CA
Joined May 2025
- Tweets1.5K
- Following1.4K
- Followers564
- Likes10.5K
Karan Brar retweeted
There lies his crown in waters deep, till Waleed finally gets his PhD (and joins @autoresearchinc)
Karan Brar retweeted
lol what a gross post
Welcome Xi! He's joining us from @OpenAI after 18 years as a Professor of Physics at Harvard 🐐
As AI eats mathematics, the best theoretical physicists seem to be reorienting towards bridging theory and experiment. This is much harder to hill-climb, but incredibly impactful!
Karan Brar retweeted
I found all the people saying negative things about Sutton and quoting bitter lesson to be quite surprising. It's as if they completely forgot how basically everyone was talking (including Dwarkesh Patel with Sutton on his podcast) when pretraining only was the dominant paradigm (pre o1 etc) and everyone thought just keep scaling and you'll get PhD level math models with just pretraining and then reasoning models came and completely shattered that ceiling becoming a great step up in capabilities.
Sutton is 100% right (as he usually is) here btw and if you don't see it you have major skill issue and don't even understand bitter lesson properly
Karan Brar retweeted
The vibe type theory explosion is bound to occur. Axiom and others will realize their systems can make new type theories that are more expressive than the hand crafted ones we use.
Karan Brar retweeted
Post-train a formal theorem proving model to reach frontier performance to push the frontier of math 🔥🔥🔥
An in-house training stack for mathematical formalization.
Tobias Gessler from @axiommathai shares how the team approaches building training infrastructure for this specialized domain.
Karan Brar retweeted
What if a swarm of AI agents could learn to work together on its own? 🐝
No leader. No roles. No turn order. Just one shared reward.
Our new paper s³: multi-agent RL lets LLM swarms self-organize, beat best-of-N, and scale past their training size!
scalingswarms.com/ 🧵
Karan Brar retweeted
Using the Stacks formalization we built at Axiom, we formalized the Weil conjectures in a single day. With the foundations in place, formalizing research-level algebraic geometry is now within reach for anyone working in the field.
Karan Brar retweeted
The mask has slipped.
The heart of the pro-AI movement is contempt for humanity. Not progress. Not 'Abundance'.
Just frustration that ordinary Americans simply refuse to vote the way that AI devs and economics professors think they should.
Karan Brar retweeted
re: weight-space programming (WSP) craziness… just tried something that blew my mind..
i asked the question: is Astra + WSP a better optimizer than BP on simple extrapolation learning on any turing computable function.. (let alone interpolation)..
so far my answer is yes..
just tried a bunch of functions like:
x^2
sin(x)
x^x
tanh(x)
add
sub
rolling_sum
sort
fib
WSP gets to beyond machine precision in many cases for interp and extrap.. (exactly matched every tested 64-bit reference output) (DISCLAIMER: very prelim results so far mind you..)
and BP does not.
WSP beats BP on every task I gave it..
iso-architecture/iso-param/iso-initialization/iso-inputs..
I have been stuck on the following problem for 12 years... if UAT is true, why cant we find a decent root in the loss landscape..
its bc BP sucks.. WSP is the first optimizer that actually can in a lot of cases. this is a whole new area of optimization burgeoning..
Karan Brar retweeted
20 days ago we built a system that writes GPU code. Today its kernels outperform the state of the art. We just open-sourced 9 megakernels, which outperform NVIDIA’s stack on DeepSeek and run self-driving, robotics, protein folding and world models 2–3× faster than the current best systems. Every model runs in a single launch instead of thousands, and most of our kernels were generated in under an hour.🧵
We invented a recursive megakernel generator called Hyperkernel, the most complete solution to megakernels and GPU efficiency.
Hyperkernel-generated megakernels achieve state of the art performance on 11 different domains: inference (DeepSeek, Kimi and GLM), pretraining (Qwen), hardware fault recovery, robotics (π₀-FAST, ACT) diffusion and real time world models (Matrix-Game), full self driving (OpenPilot), voice models (Moshi), genomics (Evo 2) and protein folding (ESMFold).
All of this in a single kernel launch.
For context, Hyperkernel generated fully optimized megakernels for robotics, genomics and voice in under 50 minutes, and doubled the efficiency for >5 models generally using only about 60-100 lines of code per megakernel. This speaks to the unparalleled standard for code quality in the automated generations.
Our first release today open sources the megakernels themselves. Over the next couple weeks, we will release the entire Hyperkernel stack, along with the complete serving stack for Kimi and GLM (20-30% inference efficiency gains across all concurrencies).
Out of the box GPU machinery performs terribly for unorthodox research. We built Hyperkernel only 20 days ago for internal use; but given its multi-modality, we have decided to open source it.
Creative research should not be bottlenecked by the lack of dedicated infra teams. Hyperkernels offer a one stop shop infra stack for all research. We would love for the open source community to add to it :)
GitHub repo: github.com/Hyperstitioncc/Hy…
NOTE: We primarily use NVIDIA B300s, but our kernel generation extends to other hardware (B200s, AMD, TPUs).
1/9
We absolutely crush NVIDIA and achieve the best low latency inference for DeepSeek v4.1 Flash.
Compared with NVIDIA’s published vLLM results, Hyperkernel delivers:
• 36.7% higher per-user generation speed at concurrency 1.
• 28.2% higher per-user speed while serving two users, versus NVIDIA serving one.
• 15% higher per user speed while serving four users, versus NVIDIA serving one.
• 90.6% higher per-user generation speed at concurrency 4, compared with the highest published C4 AgentX interactivity for TP4 B300s.
• 28.5% higher per-user generation speed at concurrency 8, compared with NVIDIA's published result.
• 36% faster median per-request decode speed.
Karan Brar retweeted
Haven't seen this discussed elsewhere on X: Kim Morrison wrote a fantastic review of Axiom's Stacks
She identifies places where Stacks is not fully formalized; and lots of duplication of Mathlib
"I want to talk about ways in which we can do even better"
kim-em.github.io/blog/2026-1…
Karan Brar retweeted
I really love that any company is thinking about good proof engineering. Carina read the literature on this and it shows. And they hired some great people who also know the literature (I'm biased since one of my students is still there)
Karan Brar retweeted
These projects from Axiom are really interesting for a couple reasons. I'll list three, but I'm sure more will come to me later:
1. Coordination. There are fundamental issues with agent coordination which need to be solved to do stuff like this. I'm sure Axiom, Math, and similar are internally co-inventing some fundamental components for agent coordination which we will eventually learn more about.
2. Errata. I think that ironing out the small bugs in published results is important and reifies our understanding of them. (I am fundamentally a pedant.)
3. Agentic discovery. I'd like to see a new field emerge, in which we actually use formal infrastructure, and agents, to make new discoveries. (Note, I am coming at this as a computer scientist, so my aesthetic and functional goals are different from those of some mathematicians.)
Something I've learned about the Axiom team is that they have great mathematical taste. It's still in the air whether that mathematical taste will ultimately yield big dividends from a computer science perspective. Will canonization actually help me prove a chip is correct? I'm of the opinion that probably the answer is yes, but I don't know. This is really what makes the space so interesting to watch and engage with though -- not yet knowing what dividends these investments will yield.
Karan Brar retweeted
Obviously good stuff, but is the over the top marketing video really necessary? Casually making the comparison with Voyager seems a bit much...
Karan Brar retweeted
Beff Jezos speaks so strongly for a guy who raised $20M to ship what is functionally a toy and not the thing they originally promised and has 0 viability to do the thing it originally set out to do. If he had an IQ over 50 he’d be focusing on matmul instead of sampling
