Sheila Brennan

I took a deep dive into the newly released Flint model and compared it to Qwen3.5-35B-A3B. The surprise? Their overarching architectures are strikingly similar, yet they make fundamentally different design bets under the hood. Here are 4 interesting architectural observations 👇 1. Layer Layout: The Repeating Pattern 🧱 Flint uses 49 layers, starting with 2 Full Attention layers, followed by a repeating [3 Sliding Window + 1 Full Attention] block.This strongly echoes Qwen3.5-9B’s [3 Gated DeltaNet + 1 Full Attention] rhythm. 💡 Hypothesis: Early Full Attention establishes global context before transitioning into highly efficient local processing. 2. The Skeleton: No Reinventing the Wheel 🦴 Both models share the exact same high-level Transformer block structure (RMSNorm → Attention → Add → RMSNorm → MoE → Add). They both use a 2048-dim hidden state and 256 routed experts.The real innovation isn't in the skeleton—it’s inside the Attention and MoE modules. 3. Attention: Two Different Bets on Memory 🧠 This is where the models diverge significantly: 🔹 Flint (Sliding Window): Directly accesses the most recent 512 tokens + periodic global retrieval. 🔹 Qwen3.5 (Gated DeltaNet): Maintains a continuously updated, compressed recurrent memory. 👉 TL;DR: Flint favors precise local recall. Qwen3.5 favors evolving, compressed long-range memory. 4. MoE: Routing and Compute Allocation ⚡ Both use 256 routed experts, but their allocation strategies differ: 🔹 Flint: Top-6 activated. Features a larger, ungated shared expert (1024 dim). 🔹 Qwen3.5: Top-8 activated. Features a smaller, dynamically gated shared expert (512 dim). 👉 TL;DR: Flint leans heavier on a robust shared knowledge path, while Qwen leans into dynamic, specialized experts. 🎯 My Biggest Takeaway: Flint and Qwen3.5 share a similar blueprint. Despite its impressive coding benchmarks, Flint has yet to report several important general-purpose evaluations, including MMLU-Pro, GPQA, HLE, and long-context benchmarks. Without them, it's hard to conclude that Flint's architectural choices are inherently superior to Qwen3.5's. As Recursive’s debut model, Flint’s architectural design feels surprisingly conservative. Instead of wild innovations, they played it safe with a masterful recombination of proven components. Architecture breakdown below 👇