@ye_combinator
🇺🇸Account based in
About this account
- Account based in
- United States
- Connected via
- United States App Store
Account-level information from X, not a live location or the device used for a specific post.
Pinned Tweet
🍰 CAKE paper's out, the design bet: the compiler isn't a fixed black box the agent calls — it's part of the harness, and it's under evolution too.
CAKE didn't inherit existing abstraction layer. no tile/layout abstractions: the vocabulary was distilled by agents from a corpus of production kernels. every pattern the agent couldn't express pushed new primitives into the IR, and the analyses to keep them checkable. every barrier/layout bug that kept coming back became a verifier rule.
none of this can be designed up front. the IR has to co-evolve with the kernels, and the workload tells you what's missing, the corpus tells you if the fix broke anything.
the best language for an agent is the one that tells you what's illegal, what's slow, and which decision might made it faster.
arxiv.org/abs/2608.12629
🍰 another slice of CAKE just dropped: blackwell KDA kernels, ~2.05x geomean speedup vs FlashKDA:
github.com/flashinfer-ai/fla… by @avyyhuang
more cake kernels are baking 👀 not saying what CAKE actually is yet though, the full reveal is in the oven too.
Zihao Ye retweeted
🥐 Hi everyone! FlashAttention's backward pass has the same zero-sum bug
@jon_barron
just fixed in softmax. With FP4 stochastic rounding it's wrong on every single draw. We fixed it in our ICLR'27 submission, CroissAttn and unlock e2e FP4 training.
Zihao Ye retweeted
Miles v0.1.3 is out.
This release brings lighter weight updates, a smaller training footprint, and more resilient runs.
Rebuilt P2P transfer cuts the first weight update from 99.5s to 3.9s and pinned memory per sender from 28 GiB to 2 GiB in our benchmark.
Fully async RL can now share GPUs between training and rollout, even on a single GPU. The new fused log-prob backend cuts peak GPU memory by about a third in our long-response test.
Failures stay contained. With fault tolerance, a failed cell no longer blocks weight updates to healthy cells. Fully async runs can also resume where they left off.
LoRA now runs in Megatron raw mode across model families using a unified implementation.
Full release notes 👇
Zihao Ye retweeted
98%+ Tensor Core utilization btw
While keeping model weights in 4-bit, increasing activation precision from 4 to 16 bits lets us make the kernel up to 25% faster for decode vs FlashInfer’s optimized NVFP4 MegaMoE
This gives up to 10% more generation throughput and 7% faster latency overall for real workloads
humansand.ai/blog/faster-tha…
Zihao Ye retweeted
While keeping model weights in 4-bit, increasing activation precision from 4 to 16 bits lets us make the kernel up to 25% faster for decode vs FlashInfer’s optimized NVFP4 MegaMoE
This gives up to 10% more generation throughput and 7% faster latency overall for real workloads
humansand.ai/blog/faster-tha…
Zihao Ye retweeted
Gave a talk at PTC on flex_gemm which is a new gemm_epilogue API in pytorch built on Quack. I think its neat, not ground breaking but neat nonetheless :)
github.com/drisspg/transform…
planning to bootstrap this and continue to add fun examples
Zihao Ye retweeted
The team that launched this challenge took the @GPU_MODE OSS code, asked an agent to rewrite it in Go, copied it, didn't credit us, shipped a closed source binary to hide it all and are not committing releasing the entries.
Full receipts here github.com/msaroufim/openker…
We're launching a $100,000 challenge to see who can make Muse Spark run fastest: openkernelchallenge.dev/
Zihao Ye retweeted
Judgement Engineering for Agentic Coding
Running Claude Code or Codex on a long task for hours in, the agent says "all done," and then you find it is actually not ... That's why we built Humanize!
The idea is simple: two models, one builds and one judges.
1. A human signs off on a plan
2. Claude Code builds, round by round.
Using different models is important. A model tends to miss the same mistakes even the memory / context is reset. Pair it with a reviewer whose blind spots are different, and a bug only gets through if both miss it. (just like dicussing with labmates/workmates!)
What we've built with it so far:
🔧A 567-file gem5 build-system migration (PR under upstream review)
⚡Kernel Design Agents: top 3 in all three Full-Agent tracks at the MLSys 2026 FlashInfer contest
🧮 672/672 on PutnamBench, #1 on Lean-Eval (251/303)
🏅HOA: full scores on IOI, IMO, IPhO 2026 and IBO 2024; 418.5/437 on IChO 2026 (gold)
📄 Paper: arxiv.org/abs/2610.08900
💻 Humanize: github.com/PolyArch/humanize
🚀 Humanize2 (standalone runtime, in development): github.com/humanfia/humanize
By Sihao Liu, Ligeng Zhu, Zijian Zhang, Dongyun Zou, Zhengyang Zhang, Changye Li, Song Bian, Song Han, Tony Nowatzki
#humanize #humanfia #agents #claudecode #codex
Zihao Ye retweeted
🚀 Excited to introduce StreamRender-H3 — bringing real-time interactive worlds to life with diffusion models!
Built on Sol-Engine, we post-trained @MiniMax_AI H3 Ref2VA into a real-time streaming neural renderer.
🎮 Code drives the world. A game engine handles physics, object states, and player interactions, while H3 generates the visuals.
⚡ Real-time streaming: 2 denoising steps, 24 FPS, and ~273 ms per chunk on 8×GB200 GPUs.
🎨 One world, endless appearances: Change the reference image and prompt to render the same interactive world in entirely different styles.
🔥 Powered by streaming post-training, distillation, a fast causal H3 VAE, and full-stack inference acceleration.
Check out our demo and code! 👇
demo: nvlabs.github.io/Sana/Sol-En…
code: github.com/NVlabs/Sana/tree/…
Zihao Ye retweeted
As hardware × model combinations explode, bespoke serving systems are becoming the only viable path forward.
The pace of change makes it nearly impossible for incumbent, general-purpose engines to keep up.
🚀Agent-built systems for the win!
Zihao Ye retweeted
Vera Rubin is lots of fun! Thank god it still uses tcgen05 tho
vLLM now supports NVIDIA Vera Rubin. The early results show more than 7.8x the throughput of GB200 on MiniMax M3 on AgentX.
@inferact, @NVIDIAAI, @RedHat_AI, and the vLLM community have been bringing vLLM up on Rubin since it was announced. Here is where things stand.
🧵 1/5
Zihao Ye retweeted
New post: Is infra ready for recursive self-improvement?
An RSI loop runs at the speed of its verifier. Most of the discussion is about the proposer. I think the loop will stall at verification, and our infra was built for a world where verification is cheap.
Zihao Ye retweeted
Optimizing kernels can be time-consuming and require deep expert knowledge. KernelAgent is a multi-agent harness that can write and optimize kernels for you.
At #PyTorchCon North America 2026, @KaimingCheng (@Meta) will present “KernelAgent: Hardware-Guided GPU Kernel Optimization via Multi-Agent Orchestration.”
The session will cover a hardware-guided optimization layer that integrates GPU hardware-performance signals into a closed-loop multi-agent workflow to guide optimization for Triton kernels. Across all 100 KernelBench L1 tasks evaluated, KernelAgent achieved a 2.02x speedup over kernels generated by earlier versions and an average 1.56x speedup compared with default torch.compile.
Register for PyTorch Conference North America 2026: hubs.la/Q04v4SL60
Zihao Ye retweeted
using torch.compile, especially inductor, in RL is really risky, on both the trainer and the generator side (including vLLM’s torch.compile path). it can silently break train–inference consistency, and your RL run pays the price.
Zihao Ye retweeted
To get much faster NCCL comms at small-to-medium payloads, start using symmetric memory, which has been added recently to NCCL and PyTorch.
If you struggle to overlap comms and compute this could be your salvation in a few lines of code!
Details github.com/stas00/ml-enginee…
Zihao Ye retweeted
Girish is a really nice guy and I work with him quite often at NVIDIA, but I totally disagree with his opinion in this talk.
Compilers: The Central Nervous System of the AI Era
youtube.com/live/jX6QHjqU6-Y…
Girish Bharambe
Innovations in Compiler Technology (IICT) 2026
Zihao Ye retweeted
After ICLR, we finally got some time to top first again on lean-eval leaderboard with updated Humanize workflows 🤗
> LeanEval is the last unsaturated benchmark in mathematical formalization.
> New Humanize flow learnt a lot of insights from OpenAI and Anthropic's blogs, and Axiom's tech report.
> More progress on "big name theorem" is coming :)
xitter.cf/IlinVasily29521/status…
LeanEval is the last unsaturated benchmark in mathematical formalization. Saturating it will require formalizing Fermat's Last Theorem. @axiommathai is currently in the lead.
Leaderboard: lean-lang.org/eval/legacy/
Zihao Ye retweeted
🚀 Kernel Design Agents to Optimize Kimi Delta Attention (upto 2.96x)
When the KDA (Kernel Design Agents) project was initiated, its name happened to coincide with another KDA (Kimi Delta Attention). More than once, people asked us , "Can we use KDA to write a version of KDA?" Today, right at a moment with a full moon good for shooting, we are thrilled to share the latest results of KDA(gent)-v0.6: using KDA(gent) to optimize KDA(ttn)!
KDA-v0.6 achieves this through
* Humanize2 Flow & LLM Collaboration Optimization: Introducing multi-round iterative evolution with Flame Chase, GPT-5.6-Sol, and Fable-5, paired with an automated Workspace cleanup mechanism to significantly broaden the operator search space.
* Multilingual Domain-Specific Skill Stack Support: Fully supporting CuTe-DSL, CUDA C++, as well as recent Agent-Native CAKE IR and TIRx. Combined with customized diagnostic tools (such as CuTe's dedicated IKET pipeline analyzer and TIRx's CPU-side numerical simulation and static checks), achieving deep performance bottleneck diagnostics.
* Self-Evolving Kernel Wiki: Building a dynamically updated knowledge base that automatically cleans up invalid/erroneous implementations, corrects classification tags, and streamlines search indexing, significantly improving the Agent's knowledge retrieval quality and code generation accuracy.
KDAgent ultimately optimized the KDAttn kernel to 2.96x of FlashKDA, and we face a lot of unexpected hacks during the optimization. More details shared in our blogs nvlabs.github.io/kda/blog/20…
Release results of KDA for KDA: github.com/NVlabs/kda/tree/2…
Community & Production Deployment:
All kernels are open-sourced for community verification! For deploying NVIDIA Agentic CUDA acceleration in production environments, we recommend using the production-level version rigorously verified by FlashInfer.
By the Team: Dongyun Zou, Yixin Dong, Hongyi Jin, Junxian Guo, Yahui Cui, Avery Huang, Zihao Ye, Junru Shao, Changye Li, Zijian Zhang, Sihao Liu, Song Bian, Ligeng Zhu.
Zihao Ye retweeted
🔥 Optimizing LLM serving requires repeated experiments, and running them on real hardware is slow and costly.
Performance depends on how models, kernels, hardware, and serving policies interact. As models and workloads change, engineers must revisit their configuration choices.
🚀 Introducing ServingStudio: an integrated workbench for simulating, analyzing, and optimizing LLM serving systems.
🧵 More in the thread below. (1/9)
