48＠💙💛

vLLM adopted a trick Strata uses. vLLM can now use a 51B-parameter part of Qwen without loading the whole thing into VRAM or system RAM. Qwen3.8-Flash-Next has a giant 51B-parameter PLE n-gram lookup table. It's basically read-only model memory indexed by the local token context. Normally vLLM keeps that table in 🎮 GPU memory or 🧠 pinned system RAM Now it can leave the table inside the original safetensors files. For each inference step it takes 🔎 token IDs determine which PLE rows are needed 💾 only those rows are fetched ➡️ they're copied into small GPU buffers 🎮 the GPU decodes and uses them So the entire 51B table never has to become resident in VRAM or pinned RAM. If you've followed Strata, the core idea is familiar. Strata already keeps Qwen's PLE table on SSD and reads only the rows needed per token. What's new here isn't the idea. 🎯 It's that this capability has now landed natively in vLLM, reading directly from the model's original safetensors checkpoint. And on one DGX Spark GB10 with Qwen3.8-Flash-Next NVFP4, the contributor reports 💾 99.09 → 72.09 GiB loaded memory ⚡ decode within ~3.5% at c=1 🧠 GSM8K 247/256 → 246/256 That 27GB matters even more on Spark. Its CPU and GPU share one unified memory pool, so moving the table from GPU → pinned CPU RAM doesn't really free that pool. Keeping the table file-backed avoids reserving that entire ~27GB. Recently used rows can sit in the OS page cache, but that memory is reclaimable when the system needs it. Here's the tradeoff. Cold rows have to come from storage so TTFT increases slightly. This is not a generic vLLM SSD model offload. Right now it's specifically Qwen4Exp PLE on CUDA. The interesting idea isn't another VRAM → RAM → SSD hierarchy. It's this, if each token only needs a tiny fraction of a 51B lookup table, why load the whole table into memory at all? Links in ALT