Fitsum Reda (@fitsumreda) profile picture

language models, multimodal understanding and world models

Santa Clara, CA
Joined July 2009
Photo by Fitsum Reda (@fitsumreda)Photo by Fitsum Reda (@fitsumreda)Photo by Fitsum Reda (@fitsumreda)Photo by Fitsum Reda (@fitsumreda)Photo by Fitsum Reda (@fitsumreda)Photo by Fitsum Reda (@fitsumreda)
Fitsum Reda retweeted
Lightning strikes for continuous and long-run agents! Nemotron 3.5 Lightning is smart, fast, efficient and open.
Introducing NVIDIA Nemotron 3.5 Lightning⚡ An open 30B MoE model with 3B active parameters, built for always-on agents to complete high-volume, specialized tasks faster. It delivers up to 4x the output speed of similar-sized models.
520
782
126
9,250
1,682,924
Fitsum Reda retweeted
Nemotron 3.5 Lightning: Same architecture as 3.0 Nano, with added speculative decoding, and with the intelligence of 3.0 Super. ⚡️⚡️⚡️
Introducing NVIDIA Nemotron 3.5 Lightning⚡ An open 30B MoE model with 3B active parameters, built for always-on agents to complete high-volume, specialized tasks faster. It delivers up to 4x the output speed of similar-sized models.
7
21
118
11,069
Fitsum Reda retweeted
The biggest question facing US AI leadership is whether we will treat AI models as infrastructure. Infrastructure (like the internet, electricity, transportation networks) is best built by lots of organizations and countries working together - in cooperation and in competition. The US knows how to build great infrastructure. Every time we have done it, we have opened new horizons of possibility. I believe it is inevitable that we will do the same with AI, and open models will be at the heart of this new infrastructure. Open models are enabling companies and institutions from brand-new startups to titans of every industry to build their own future, to capitalize on their perspective and ideas to better serve customers and solve problems. Open models make sovereignty possible. Policymakers that want to keep America AI at the forefront of this opportunity will understand how necessary open models are: they are now the critical infrastructure of the AI age.
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models. images.nvidia.com/pdf/Open-W…
23
41
4
229
26,806
Fitsum Reda retweeted
NVIDIA might just have solved the biggest tradeoff in LLMs. Every LLM makes you pick between speed and quality. Autoregressive models like GPT give you quality, but they're slow, because they generate one token at a time. Diffusion models do the opposite. They generate whole blocks in parallel, so they're fast, but that speed has always come at the cost of quality. Before we get to NVIDIA's fix, it's worth understanding why diffusion falls behind in the first place. A diffusion LLM starts with a block of masked tokens and unmasks them over a few passes. On every pass, it reads all the text written so far and predicts the masked tokens in that block together. So the same network is doing two very different jobs at once. It has to understand the context that's already there while also guessing at tokens that don't exist yet. That's one set of weights pulled in two directions, so it never gets good at either. TwoTower fixes this by not forcing one network to do both. It clones a pretrained autoregressive model into two towers. The first is a frozen context tower that reads everything written so far, exactly like the original did. It's never trained, so the model's intelligence stays perfectly intact. The second is a trainable denoiser tower that fills in each new block of tokens in parallel. To stay on track, it constantly cross-attends back to the context tower. Here's where it gets clever. The two connect layer by layer, so denoiser layer 5 talks to context layer 5, and so on. That gives the denoiser the backbone's full multi-scale understanding, not just a final summary of it. The results: ↳ 2.42x higher generation throughput ↳ Keeps 98.7% of the original model's quality ↳ Built on a 30B hybrid Mamba-Transformer MoE backbone ↳ Trained on just ~2.1T tokens, a fraction of the 25T used to pretrain the original That last number is the real story. Nothing here was trained from scratch. It's an adaptation you bolt onto an autoregressive model you already have, and it turns slow one-token-at-a-time decoding into fast parallel block generation, cheaply. So if you're shipping anything latency-sensitive, this is a blueprint worth studying. You no longer have to choose between the model you trust and the speed you need. Weights, code, and recipe are all open-sourced. Link to the paper: arxiv.org/pdf/2606.26493 That said, if you want to go deeper, I wrote a detailed article that explains LLM inference from first principles, step by step. The article is quoted below.
27
102
6
623
80,397
Fitsum Reda retweeted
Keeping up with AI news is becoming a full-time job. So my friend @ivan_bezdomny built HuggingNews, an AI-curated feed that surfaces the news actually worth reading. Soon, it will even personalize the feed using your Hugging Face profile. Been using it for weeks! Bookmark it, or ask your agent to send you the top 10 stories every morning or night. Less noise. More signal. More building! huggingnews.com
120
277
33
2,442
212,596
Fitsum Reda retweeted
Five AI models just changed what you should test next. This roundup covers June 27 to July 7, 2026. 1- Tencent Hy3 295B total parameters, 21B active per token, 256K context, Apache 2.0 weights. 2- OpenAI GPT-5.6 Sol, Terra, and Luna give a new price ladder, but access starts in limited preview. 3- Claude Sonnet 5 Default for Free and Pro, available in Claude Code, with $2/$10 intro API pricing. 4- Claude Fable 5 Restored after export controls, with Mythos-level capability and safety routing to Opus 4.8. 5- NVIDIA Nemotron-Labs-TwoTower 98.7% baseline quality at 2.42x wall-clock throughput by splitting generation into two towers.
7
4
2
12
1,792
Fitsum Reda retweeted
Really great to see NVIDIA experimenting with novel hybrid architectures for Nemotron family. Its becoming a major western lab! I just posted a video on Nemotron TwoTower (link in next post). Another interesting idea from NVIDIA to push the model throughput without full retraining.
The Nemotron family just passed 100M downloads! Huge thank you to the community building with us and showing what’s possible with open models. Cheers to OSS 🍾
1
1
4
819
Fitsum Reda retweeted
Why are open technologies for AI so important? How and why is NVIDIA building Nemotron? What can we learn from China’s AI efforts? Great conversation with Matt Turck. youtu.be/Oojrfdl42LI
9
23
4
87
21,341
Fitsum Reda retweeted
everyone's arguing about claude sonnet 5 while nvidia just quietly changed how models write text Nemotron-TwoTower: they took a 30B model, split it in half, and made it write tokens in parallel instead of one at a time > one half holds context, one half writes > both reuse the pretrained model, zero training from scratch > 98.7% of original quality > 2.42x faster generation the whole frontier race is about smarter models. this is about the same model, faster, for free SPEED IS THE FEATURE NOBODY IS SELLING YOU YET
We took a 30B model and split it in two to write tokens in parallel instead of one at a time. Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch. We found it kept 98.7% of the original model’s quality at 2.42× faster generation.
2
8
764
Fitsum Reda retweeted
Nvidia just shipped a diffusion model that doesn't diffuse from scratch. It clones an AR model. Twice. The setup: Every diffusion LM up to now made one network do two jobs. Represent the clean context. Denoise the corrupted tokens. Same weights. Two conflicting pulls. Nvidia's answer: stop sharing. TwoTower, in one line: Take a pretrained autoregressive model. Make two copies. Give them different jobs. - Tower 1: frozen, causal, reads clean tokens like a normal AR model - Tower 2: trainable, bidirectional, denoises a masked block No shared weights. The base model: Nemotron-3-Nano-30B-A3B. 52 layers - 23 Mamba-2, 6 self-attention, 23 MoE. Both towers start as identical copies of this checkpoint. Then their roles diverge. The detail everyone will miss: Most encoder-decoder diffusion setups broadcast one summary vector - the last hidden state - into the decoder. TwoTower doesn't do that. It connects layer by layer. Denoiser layer 6 cross-attends to context-tower layer 6. Not layer 52. Not a pooled vector. Same-index layers, same representation level. Because both towers came from the same checkpoint, that alignment isn't a hack - it's structurally guaranteed. Mamba stays causal. That's the counterintuitive part. They tried a bidirectional Mamba variant - run it left-to-right, then right-to-left, average the states. Doubles the SSM compute. Result: generation barely moves (72.94 → 72.96). Code and math get worse. So the final model keeps Mamba causal everywhere, and lets bidirectional attention - just 6 of the 52 layers - do all the within-block mixing. More capacity in the wrong place doesn't buy you anything. Sometimes minimal beats bidirectional. Training is cheap. That's the pitch. Backbone pretraining: 25T tokens. TwoTower adaptation: ~2.1T tokens. No re-pretraining. Take an off-the-shelf AR checkpoint, freeze one copy, train a denoiser on top. That's it. The generation loop: Not full-sequence diffusion. Block-wise autoregressive diffusion. - Context tower processes the prompt, builds KV cache + Mamba states - A block of S masked tokens gets denoised over T steps, cross-attending into that frozen cache - Once the block is clean, it's committed - the context tower ingests it, updates its cache - Repeat for the next block AR at the block level. Diffusion inside the block. The number that matters: 98.7% of AR baseline quality. 2.42x wall-clock throughput. Not because the model spits out more tokens per step uniformly - because commitment is front-loaded. Step 1 of the denoising loop commits the most tokens. Confidence-based unmasking clears the easy positions immediately and leaves the hard ones for later refinement. AR commits exactly one token per step, always. TwoTower commits a batch early, then tapers. The pattern nobody designed but showed up anyway: Look at which token positions get committed early vs late within a block - it's left-to-right. An upper-left triangular pattern, same shape you'd get from an AR model. Nobody engineered that. It falls out of the architecture: the context tower is fully causal, and 23 of the denoiser's 52 layers are causal Mamba too. Only 6 layers are bidirectional. The diffusion model inherited an autoregressive habit from its own frozen twin. Why this matters: Block diffusion has been trying to answer one question - can you keep AR's quality while beating its one-token-at-a-time bottleneck? Prior work (Arriola et al.) tried tied weights at 1.7B scale and left the "does full decoupling hold at real scale" question open. TwoTower answers it. 30B hybrid MoE. Frozen context, trainable denoiser, zero shared parameters. 98.7% quality, 2.4x speed, 12x less training data than a fresh pretrain. Two copies of the same model, pointed in different directions, is apparently all it takes.
We took a 30B model and split it in two to write tokens in parallel instead of one at a time. Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch. We found it kept 98.7% of the original model’s quality at 2.42× faster generation.
1
1
4
1,254
Fitsum Reda retweeted
Replying to @NVIDIAAI
This is a really strong systems-level innovation 🚀 Splitting a 30B model into two coordinated towers for parallel token generation is a clever direction, especially if it truly preserves ~98.7% quality while giving a 2.4× speedup. It’s the kind of architectural shift that can matter as much as scaling laws—same intelligence, better throughput.
1
1
207
Fitsum Reda retweeted
Two interesting pieces for this new diffusion project from @nvidia: 1 a 2.4x speed up v baseline with 1% accuracy loss is VERY good on MoE; MTP often yields minimal results with MoE, by comparison 2 autoregressive generation seems better suited for code/math (makes sense)
5
14
1,890
Fitsum Reda retweeted
Nvidia does cool stuff.
We took a 30B model and split it in two to write tokens in parallel instead of one at a time. Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch. We found it kept 98.7% of the original model’s quality at 2.42× faster generation.
4
3
113
13,159
Fitsum Reda retweeted
You now convert any LLM into a faster one without retraining from scratch. NVIDIA just did this to their 30B model. Here's the trick: 1. Duplicate the model into two copies 2. Freeze one copy, it just reads the prompt and remembers context 3. Train the other copy to write chunks of text at once instead of one word at a time 4. Run them together The frozen copy barely costs anything (it's already trained). The new copy only needed ~8% of the original training data to learn the new trick. Result: 2.4x faster generation, keeping ~99% of the original quality.
We took a 30B model and split it in two to write tokens in parallel instead of one at a time. Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch. We found it kept 98.7% of the original model’s quality at 2.42× faster generation.
17
88
1
1,043
134,746
Most diffusion language models make one network do two jobs at once — represent the clean context and denoise the noisy tokens. Those two goals pull the same weights in different directions. NVIDIA just split them apart. They released Nemotron-Labs-TwoTower — a block-wise autoregressive diffusion model built on the Nemotron-3-Nano-30B-A3B hybrid Mamba-2/attention/MoE backbone. It runs two towers: a frozen autoregressive context tower that processes clean tokens causally, and a trainable diffusion denoiser tower that refines noisy blocks via cross-attention to that context. Only the denoiser is trained — on ~2.1T tokens, a fraction of the backbone's 25T. Here's what's actually interesting: → Two towers, not one: a frozen AR context tower and a trainable diffusion denoiser, connected layer-by-layer — denoiser layer i attends to context layer i, not just the last hidden state → 98.7% of the autoregressive baseline's quality at 2.42× generation throughput (γ=0.8, block size 16, 2×H100) → It commits multiple tokens per denoising step early in decoding — that's where the one-token-per-step AR bottleneck breaks → One checkpoint, three decoding modes: mask diffusion, mock-AR, and standard AR → Ablations: causal Mamba beats bidirectional Mamba, and tying the two towers under a joint loss is substantially worse Full analysis: marktechpost.com/2026/07/01/… Paper: arxiv.org/pdf/2606.26493 Weights: huggingface.co/collections/n… @NVIDIAAI @NVIDIARobotics @NVIDIADeepLearn @nvidiadeveloper
2
4
1
19
46,528
Nemotron-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context "We propose TwoTower, a block-wise autoregressive diffusion model that decouples these roles into two towers: a frozen AR context tower that causally processes clean tokens, and a trainable diffusion denoiser tower with bidirectional block attention that refines noisy blocks via cross-attention to the context." "Nemotron-TwoTower retains 98.7% of the autoregressive baseline's quality while offering 2.42X higher wall-clock generation throughput."
3
29
196
12,043
AR pretraining won. Generation doesn’t have to stay AR. We’re releasing Nemotron-TwoTower: clone a pretrained AR LLM, freeze one copy as context, and train the other as a diffusion denoiser for parallel token generation. 30B model 98.7% AR quality 2.42× throughput 📄 🤗 💻 below 👇
7
6
1
35
65,729
Links 👇 📄 Report: arxiv.org/abs/2606.26493 🤗 Weights: huggingface.co/nvidia/Nemotr… 💻 Code/recipe: github.com/nvidia/megatron-l… Details: same AR checkpoint → two towers 30B hybrid Mamba-Transformer MoE frozen context tower diffusion denoiser trained on ~2.1T tokens, ~8.4% of the 25T backbone pretraining token budget
1
3
850
The goal isn’t to replace AR pretraining. It’s to reuse it. AR gives the model its knowledge and capability. Diffusion changes how we generate — boosting wall-clock throughput while preserving nearly all AR quality.
4
345
Why TwoTower? Most diffusion LMs ask one network to both represent clean context and denoise corrupted tokens. We split the jobs: 1⃣ frozen AR tower = causal context + KV cache 2⃣ trained diffusion tower = bidirectional parallel block generation Same pretrained starting point. Different roles
285