
making gpu's that go brrrrr @AgraniLabs
Joined December 2024
- Tweets972
- Following505
- Followers71
- Likes28.2K
Register files used to be the ultimate bottleneck for Tensor Core accumulators. Introducing Blackwell’s Tensor Memory (TMEM), a completely new address space inside the SM that isolates the accumulator entirely from the register file.
1
32
Hopper and Blackwell demand that you abandon standard thread copy loops. Introducing the Tensor Memory Accelerator (TMA) forklift and phase-based mbarrier synchronization. One thread dispatches; the hardware engine moves the data asynchronously.
1
19
Shared memory has 32 physical banks. If you access a tile via column-major strides, you hit a catastrophic 32-way bank conflict, dropping your throughput to 3.125%. Introducing the exact math behind Swizzle<B,M,S> to stagger your data like bricks.
"What I cannot create, I do not understand."
Introducing: The Feynman GPU Lectures.
Your H100s and B200s are running at a fraction of their peak utilization because your custom kernels are written with massive hardware bottlenecks. If you don't know what tcgen05. mma does at the wire level, you're lighting compute efficiency on fire.
1
2
67











