MoonMath.ai

We fixed it - an easy 15% speedup. We’ll coordinate with AMD and send a PR to AITER later today. Thank you @__tinygrad__ ,here , you dropped this 👑

On this holiest day in the Jewish calendar, I want to share a list of PRs to @sgl_project and @vllm_project focused on low-level performance optimizations for @AIatAMD CDNA3 × Kimi K3 that we at @zroai_ opened over the past year. We touched everything from bugs to new attention kernels. We are now working on CDNA4 (thanks, AMD!). I’d also like to take this opportunity to ask forgiveness from AMD: you ship great hardware, and while your software is not as good, you do your best to improve it, and you are much more welcoming to startups than I gave you credit for. github.com/vllm-project/vllm… github.com/vllm-project/vllm… github.com/vllm-project/vllm… github.com/sgl-project/sglan… github.com/sgl-project/sglan… github.com/sgl-project/sglan… github.com/sgl-project/sglan… github.com/sgl-project/sglan…

1/ Here’s a demo showing how I generated tokens for free on @OpenRouter using the @FireworksAI_HQ endpoint for GLM-5.3

We implemented this banger, and it's now merged into @sgl_project main 🎉 PR: github.com/sgl-project/sglan…

Tldr- 1. For the model we tested, Fireworks appears to use off-the-shelf @vllm_project with a relatively simple routing setup 2. As a customer, you can exploit that routing behavior to improve your own inference economics

The updated dsv4-Flash-0731 feels unfairly good :) We added day-@zroai_ support serving it from our EU infra. And, just like with Kimi K3, we cut the price 🤠

We believe we’re one of the first non-Day 0 serving platforms to support Kimi K3. So we cut prices 🤠 We’ve built a full system around this amazing model. Proud of the team! and shout-out to @sgl_project and @AIatAMD !! Try it: zro.moonmath.ai Please help us get listed on @OpenRouter 🙏

We now serve GLM5.2 as well. privacy-centric, fast, almost free. GPUs are in the UK 👀 Ping me for access.

I reconstructed the @AIatAMD CDNA3 Compute Unit architecture. To my knowledge this is to most detailed/accurate CU description available (please let me know if you find a mistake or a way to make it more accurate). I used AMD/ROCm docs, older GCN diagrams, Accelerated Computing with HIP book, and mostly a lot of kernel-engineering pain from the team: The hard part was understanding what is private to a SIMD, what is shared across the CU, and what sits outside the SIMD execution path. Some immediate insights:  • A CDNA3 CU still carries the old GCN DNA: scheduler, scalar path, LDS, 4-SIMD structure, load/store path, and L1.  • For modern workloads, especially MFMA-heavy kernels, the important unit is often the SIMD slice. Each SIMD has its own VGPR slice, accumulator register slice, Vector ALU 16, and matrix core path.  • Scalar state is different. SGPRs and scalar memory/cache behavior sit on the scalar/control side, not inside every SIMD lane.  • LDS is not just “shared memory somewhere in the CU.” A more useful view is that SIMD 0/1 share one LDS access grouping, and SIMD 2/3 share another.  • L1 and load/store sit downstream of the execution cluster, and the details matter: line size, associativity, write-through behavior, and how VMEM traffic is fed. The closest NVIDIA analogy btw is still AMD CU ≈ NVIDIA SM, but that analogy breaks quickly. CDNA3 exposes a different mental model: explicit scalar/vector separation, per-SIMD register and matrix resources, and shared CU-level memory structures.