ceo/co-founder of autoresearch, inc. | prev ml enjoyer @reductoai @dynamo_ai, math enjoyer @uoft
San Francisco, CA
Joined May 2025
- Tweets1.5K
- Following1.4K
- Followers564
- Likes10.5K
this is too real. the age old question, do i have psychosis, or do the normies?
i can't talk to friends or family about ASI and what might happen next year without sounding like a chronically online person lost in sci-fi fantasies. it's increasingly frustrating that trying to be rational about what's happening sounds delusional to normies. like i'm the one detached from reality when, from where i'm standing, they're the ones missing what's unfolding right in front of them. the gap in perception is getting so absurd it's making me feel like I am the schizo who needs to touch grass but im just trying to inform them because the media doesnt
1
2
244
Karan Brar retweeted
There lies his crown in waters deep, till Waleed finally gets his PhD (and joins @autoresearchinc)
About as much as every other field is cooked. I’m not a fan of these takes because it implies that math is somehow over. If anything, we’re entering the golden age of the industrialization of mathematics.
4
353
Karan Brar retweeted
lol what a gross post
Welcome Xi! He's joining us from @OpenAI after 18 years as a Professor of Physics at Harvard 🐐
As AI eats mathematics, the best theoretical physicists seem to be reorienting towards bridging theory and experiment. This is much harder to hill-climb, but incredibly impactful!
We’ll certainly have a lot more exotic optimization algorithms that combine classical techniques + LLM driven optimization. WSP is only the beginning.
I’ll take the other side of this bet. If one understands any reinforcement learning, it should be clear why weight space editing is a last mile touch up and not a backprop or heck even ZO opt replacement
2
7
550
Karan Brar retweeted
I found all the people saying negative things about Sutton and quoting bitter lesson to be quite surprising. It's as if they completely forgot how basically everyone was talking (including Dwarkesh Patel with Sutton on his podcast) when pretraining only was the dominant paradigm (pre o1 etc) and everyone thought just keep scaling and you'll get PhD level math models with just pretraining and then reasoning models came and completely shattered that ceiling becoming a great step up in capabilities.
Sutton is 100% right (as he usually is) here btw and if you don't see it you have major skill issue and don't even understand bitter lesson properly
Karan Brar retweeted
The vibe type theory explosion is bound to occur. Axiom and others will realize their systems can make new type theories that are more expressive than the hand crafted ones we use.
Karan Brar retweeted
Post-train a formal theorem proving model to reach frontier performance to push the frontier of math 🔥🔥🔥
An in-house training stack for mathematical formalization.
Tobias Gessler from @axiommathai shares how the team approaches building training infrastructure for this specialized domain.
Karan Brar retweeted
What if a swarm of AI agents could learn to work together on its own? 🐝
No leader. No roles. No turn order. Just one shared reward.
Our new paper s³: multi-agent RL lets LLM swarms self-organize, beat best-of-N, and scale past their training size!
scalingswarms.com/ 🧵
Still curious if formalization of mathematics will yield any gains in terms of our ability for discovery, or it’s mostly just to accomplish the autistic dream of a formally verified encyclopedia of mathematics.
Most mathematicians I’ve spoken to see this as a vanity project with no real applications, but the rest is to be seen.
Karan Brar retweeted
Using the Stacks formalization we built at Axiom, we formalized the Weil conjectures in a single day. With the foundations in place, formalizing research-level algebraic geometry is now within reach for anyone working in the field.
Karan Brar retweeted
The mask has slipped.
The heart of the pro-AI movement is contempt for humanity. Not progress. Not 'Abundance'.
Just frustration that ordinary Americans simply refuse to vote the way that AI devs and economics professors think they should.
Karan Brar retweeted
re: weight-space programming (WSP) craziness… just tried something that blew my mind..
i asked the question: is Astra + WSP a better optimizer than BP on simple extrapolation learning on any turing computable function.. (let alone interpolation)..
so far my answer is yes..
just tried a bunch of functions like:
x^2
sin(x)
x^x
tanh(x)
add
sub
rolling_sum
sort
fib
WSP gets to beyond machine precision in many cases for interp and extrap.. (exactly matched every tested 64-bit reference output) (DISCLAIMER: very prelim results so far mind you..)
and BP does not.
WSP beats BP on every task I gave it..
iso-architecture/iso-param/iso-initialization/iso-inputs..
I have been stuck on the following problem for 12 years... if UAT is true, why cant we find a decent root in the loss landscape..
its bc BP sucks.. WSP is the first optimizer that actually can in a lot of cases. this is a whole new area of optimization burgeoning..
Karan Brar retweeted
20 days ago we built a system that writes GPU code. Today its kernels outperform the state of the art. We just open-sourced 9 megakernels, which outperform NVIDIA’s stack on DeepSeek and run self-driving, robotics, protein folding and world models 2–3× faster than the current best systems. Every model runs in a single launch instead of thousands, and most of our kernels were generated in under an hour.🧵
We invented a recursive megakernel generator called Hyperkernel, the most complete solution to megakernels and GPU efficiency.
Hyperkernel-generated megakernels achieve state of the art performance on 11 different domains: inference (DeepSeek, Kimi and GLM), pretraining (Qwen), hardware fault recovery, robotics (π₀-FAST, ACT) diffusion and real time world models (Matrix-Game), full self driving (OpenPilot), voice models (Moshi), genomics (Evo 2) and protein folding (ESMFold).
All of this in a single kernel launch.
For context, Hyperkernel generated fully optimized megakernels for robotics, genomics and voice in under 50 minutes, and doubled the efficiency for >5 models generally using only about 60-100 lines of code per megakernel. This speaks to the unparalleled standard for code quality in the automated generations.
Our first release today open sources the megakernels themselves. Over the next couple weeks, we will release the entire Hyperkernel stack, along with the complete serving stack for Kimi and GLM (20-30% inference efficiency gains across all concurrencies).
Out of the box GPU machinery performs terribly for unorthodox research. We built Hyperkernel only 20 days ago for internal use; but given its multi-modality, we have decided to open source it.
Creative research should not be bottlenecked by the lack of dedicated infra teams. Hyperkernels offer a one stop shop infra stack for all research. We would love for the open source community to add to it :)
GitHub repo: github.com/Hyperstitioncc/Hy…
NOTE: We primarily use NVIDIA B300s, but our kernel generation extends to other hardware (B200s, AMD, TPUs).
1/9
We absolutely crush NVIDIA and achieve the best low latency inference for DeepSeek v4.1 Flash.
Compared with NVIDIA’s published vLLM results, Hyperkernel delivers:
• 36.7% higher per-user generation speed at concurrency 1.
• 28.2% higher per-user speed while serving two users, versus NVIDIA serving one.
• 15% higher per user speed while serving four users, versus NVIDIA serving one.
• 90.6% higher per-user generation speed at concurrency 4, compared with the highest published C4 AgentX interactivity for TP4 B300s.
• 28.5% higher per-user generation speed at concurrency 8, compared with NVIDIA's published result.
• 36% faster median per-request decode speed.
