JustVugg

One person on a 12-core laptop with 25 GB of RAM built an inference engine that runs AI models with 2.8 trillion parameters. It's called Colibri, it has 24.8k stars on GitHub, and it runs open-weight models up to 2.8 trillion parameters on machines that have no business running them. Models like GLM-5.2 have 744 billion parameters, but they only activate about 40 billion of them for each word they generate. The rest sit idle. So the model doesn't need to fit in your computer's memory all at once. It needs to load the right 40 billion at the right time. Colibri treats your hard drive, your RAM, and your GPU memory as one system. It keeps the small always-needed piece in RAM (about 10 GB) and streams the rest from disk, loading each piece right before the model needs it. It watches which pieces your conversations use most and keeps those warmed up. The more you use it, the faster it gets. The engine is a single C file per model. It runs without Python, without external libraries, and without a GPU. A GPU makes it faster, but a hard drive is the only thing it needs. On six RTX 5090 GPUs it hits 5.8 tokens per second on a 744 billion parameter model. On a single laptop GPU it runs at about 1 token per second. On the 25 GB machine where the project started, it crawls at a tenth of a token per second. That was slow but correct, and the engine climbed from there. The name is Italian for hummingbird, a bird that weighs a few grams and visits a thousand flowers a day. This engine keeps a 744 billion parameter giant alive on 25 GB and patience.