
Provable @snarkify_ai Inference merakcompute | phd in operating systems | ex @Meta
Joined August 2010
- Tweets849
- Following789
- Followers1.9K
- Likes1.9K
The classic perf-engineering loop (profile, analyze, write code, experiment, measure, repeat) is now something AI agents run well, for hours, without us.
So the work moved. Performance engineering is now designing the harness the agents run in. In our GPU-kernel runs, three things decided whether the results were real.
* The score must match the end goal. Agents optimize exactly what you score, and the stronger the AI, the faster a skewed score goes wrong. A fast proxy score is fine only if it keeps being checked against real end-to-end throughput.
* Correctness is the harness's job, not the agent's. Agents will ship code that is faster and wrong. Every timed run also checks outputs.
* Measurement must be accurate, fast and consistent. Noisy numbers reward luck. Slow numbers starve the loop. And numbers that disagree between quick and full runs point the agents the wrong way.
The agents do the engineering. We design the referee.
Two weeks ago we announced Together Compiler: agents that tune inference to your model, chip, and traffic, for up to 2.6x more throughput.
Today, we’re sharing the journey behind it: the experiments that challenged our assumptions of megakernels, the surprising strengths of Torch Compile, and the lessons that shaped our development of a next-generation system for inference optimization.
Shout out to @dylan__lim @reyna_abhyankar @simran_s_arora and the team!
Read more: together.ai/blog/road-to-tog…
5
440
A lot rings true from our in-house runs: frontier models, an unreasonably ambitious target, then step away for 12+ hours.
Where we differ: the harness is not optional
1/ Agents will happily ship a kernel that's faster and wrong. Our loop only counts a win after output checks, integrity checks and a soak run, and every number goes through a noise floor.
2/ Cross-review earned its keep. On a Lean proof, a second agent caught premises that were vacuously true on the new model. Every test passed and nothing was proved.
3/ To know a result is reproducible, we give fresh agents only the test harness, never past notes, and time how fast they get back there.
btw we are building a compute infra that accepts agentic crypto payments!
Baking the GPU prover of a lattice-based zkVM 🔜
In this clip, @srinathtv categorises zkVMs into four types:
-> Elliptic curve-based,
-> Lattice-based,
-> Hash & more (which combine hashes with other assumptions like elliptic curves for memory checking)
-> Purely hash-based
Which zkVM are you?
1
112
Nodes switch between prefill and decode online in 6s (vs a 754s relaunch), KV eviction follows agent context. Up to 4.1x lower latency vs Dynamo. Impressive work!!
EvolvingInfra is a scheduling layer on top of the GPU inference engine, which is where our performance optimization work focuses.
We breathed life into LLM system.
Introducing Evolving Infra⚙️: an inference framework that adapts diverse and dynamic agent workload purely online and retires fixed PD disaggregation.
vs. NVIDIA Dynamo on up to 256 GB300 GPUs:
➡️ Up to 4.1× lower latency
➡️ Up to 14× lower P95 TTFT
➡️Up to 2.36× more revenue
on real agent traces(Computer Use, Coding and math agents) and opensource models.
🌐 Website: inference-rsi.com
📄 Tech report: alphaxiv.org/en/abs/2611.123…
💻 Code: github.com/mit-han-lab/Evolv…
Which agent(e.g. Pi/Openhands) do you want us to accelerate next? Tell us below 👇
drouyang retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Hard to overstate how big of a deal this is.
tl;dr — ANY unpatched Linux server with even a single TCP port exposed to the internet (say for HTTP or SSH) and MPTCP active (default for many Go web servers) is vulnerable to complete takeover.
What a disaster atlasum.net/blog/cve-2026-98…
We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence.
block.xyz/dessa
drouyang retweeted
.@apolynya is back with updated thoughts on ZK.
One line stood out: “The potential impact of ZKPs supercedes that of all blockchains.”
Early signs say he’s right. Our upcoming 2026 ZK Industry Report with @zeroknowledgefm surveys adoption across categories beyond blockchains.
drouyang retweeted
Introducing Magnetar from Microsoft Research, the first zkVM that is: (1) general-purpose (RV64), (2) post-quantum from standard hashes alone (no curves, no lattices, no Poseidon), and (3) built for recursion.
Despite its conservative security, it is the fastest zkVM: 7.2–8.7 MHz on a laptop, 10–39× faster than existing zkVMs across 15 workloads! An optimized version reaches 9.5–12.8 MHz.
Links in the thread below.
drouyang retweeted
1/ Today we're releasing Lattice Jolt: a post-quantum version of the Jolt zkVM, built on lattices instead of elliptic curves. It's _faster_ than curve-based Jolt and has the shortest proofs of any post-quantum zkVM — under 100 KB.
Post: a16zcrypto.com/posts/article…
drouyang retweeted
We believe developers should choose their own security configurations, but we made a mistake by allowing our DVN to act as a 1/1 DVN for high-value transactions. We didn't police what our DVN was securing, which created a risk we simply didn't see. We own that. Moving forward, LayerZero Labs will be more active in educating developers and monitoring how applications should build on the protocol to ensure they are configured safely.
DeFi composability is one of the most powerful things this industry has created, but it also means risk can compound across protocols in ways that are hard to detect until something breaks.
In my eight years in crypto, I’ve seen no shortage of infra projects pump a token, watch activity go to zero, and disappear. The LayerZero team is one of the rare groups of hardcore builders who genuinely care about the industry and hold themselves to a high standard. LayerZero has carried billions of dollars in crypto asset transfers and, because of that, has inevitably become a massive target.
What matters now is what the industry learns from this sophisticated hack, and how we take responsible action from here. Respect to Bryan and the LayerZero team for owning their part. We need more builders like the LayerZero team, not fewer.
I’ve been asking myself why has it taken me so long to write this? Ultimately I still carry a huge amount of cognitive dissonance here. In my mind LayerZero the protocol was like Gnosis Safe and the application was setting their config, and who the )@(!$ would secure billions in TVL on a 1/1? I even tweeted about it, literally 0, I would have bet almost anything on that because almost every major application we helped setup their configs. Someone then going and manually changing that to a 1/1 was outside of the realm of possibility for me.
I was wrong.
It’s easy to sit back and say ‘it’s just a protocol we have no control over how people use it’ but we have the opportunity to be better.
There was a conversation this week speaking to a customer and they just… screamed at me, at the top of their lungs, and swore for a solid ~3-5 minutes straight. We had implemented additional security measures of forcing a more stringent RPC quorum and forcing every chain to provide multiple RPCs and we had done it without telling them and it !)@$d with their business, which frankly is a deadly sin. We had messed with their business and they said communication wise we were completely blowing this. They were completely right.
This is something I care about a ton, it is both the result of a huge portion of my life and something that I fundamentally believe in. I literally gifted a copy of Unreasonable Hospitality to every single manager in the company. The entire point of what we’ve built is to enable others to build on top of it, it is to provide a protocol and a platform for people to build on, and we’ve been failing some of our largest customers.
The past two weeks have been unbelievably miserable. I’m incredibly grateful for all of the applications who have worked with us over the past 2 weeks, for @zeroshadow_io who has spent endless cycles with us tracking and seizing millions in attacker funds which will be returned to the rsETH team, and for all of the parties who brought together DefiUnited. Particularly @aave for leading and @MikeSilagadze for pushing everyone to get their !@)($ together and sort things out as quickly as possible, and putting himself in the frontline of acquisition talks and everything else to try to get to a solution quickly.
@LayerZero_Core is one of the most critical pieces of infrastructure in the industry, the LayerZero protocol has earned the trust of the largest and most important asset issuers in the space. We will do better and we will make the industry better for it.
My entire focus over the past 2 weeks has singularly been working with applications to harden their setup, building tooling to assist in tracking and freezing hacker funds, and the rsETH recovery efforts.
That is going to shift now to where LayerZero Labs spends it's time and effort. The only thing this company will spend time on is how we can better serve our asset issuers and the upcoming launch of Zero.
10
24
970
drouyang retweeted
Local coding models may have crossed a threshold to be viable. We compared several 30B class open weight models on Terminal-Bench 2.0 and they are close to what frontier models were around ~6 months ago.
- For consumer hardware: MoE > dense when memory is the bottleneck.
- We expect this can be improved a lot with some prompt/harness/llama.cpp tuning
Deepseek V4 dropped
We present a preview version of DeepSeek-V4series, including two strong Mixture-of-Experts (MoE) language models — DeepSeek-V4-Pro with 1.6T parameters (49B activated) and DeepSeek-V4-Flash with 284B parameters (13B activated) — both supporting a context length of one million tokens.
huggingface.co/deepseek-ai/D…
3
10
235
Congrats to Anne and the @Standard_Kernel team!
The proposed Standard Kernel Rubric is a great contribution. It’s getting increasingly difficult to reason about what has and hasn’t actually been solved in GPU kernel code generation. There’s rarely a simple answer.
Many of the ZK kernels we’ve built still lack good GPU STOA implementations. Most published work reports speedups, but in production, what matters to us is performance against the speed-of-light limit — the theoretical peak.
The fastest kernel >> a faster kernel
Excited to share @Standard_Kernel's seed round and some reflections on what we’ve learned about kernel generation and what we believe is next. Grateful to our amazing team, supporters, and the broader community pushing this space forward.
10
28
383





























