Dynamic sparse attention reads fewer tokens per query. So why doesn’t the KV cache necessarily shrink to match? Sliding-window attention only looks at recent tokens. Once a token falls outside the window, that layer no longer needs its KV state to continue the sequence, so the runtime can release it. Dynamic sparse attention works differently. Each query selects which tokens to attend to from across the history. A token skipped by this query may be exactly what the next one needs. So the runtime may still need to keep the full KV history, even though each step reads only a subset. And selecting positions and gathering noncontiguous KV entries adds its own cost. Fewer attention reads ≠ a smaller cache. For long-context agents, compute and memory are two different budgets. Our deep dive covers what this, and hybrid models like Kimi K3, mean for prefix reuse, caching, and speculative decoding: friendli.ai/blog/llm-archite…