I happily picked up two used Nvidia Tesla P40 GPUs. I knew the P40 was old and ridiculously overbuilt. What I did not expect was how long the GPUs would become once the coolers were attached. Tried putting them in the case. Nope. The cooler end would not fit. I thought about it for about 10 seconds. Turns out the bottleneck was not the human. It was the case. So I grabbed a hammer and pliers and destroyed the front section of the case. After that both P40 GPUs fit perfectly. Put the cover back on and you cannot even tell from the outside. Apparently hardware fits if you hit it hard enough. Good experience.

MiMo-V2.6 Pro w/ 8x DGX Sparks Concurrency 1 Decode TPS : 40 tps - Where is the limit to DGX Spark's performance? It’s only natural that the price keeps climbing.

LFM3 1.7B Uncensored by AtomicChat @atomic_chat_hq has an uncensored GGUF build of Liquid AI's LFM3 1.7B. The original model uses a hybrid architecture with convolution and attention layers. It supports a 256K context window and is designed for lightweight local agents, tool calling, and structured outputs. At 4 bit precision, the original model needs roughly 1GB for its weights. That makes it practical for laptops and even mobile devices. The uncensored version adds another option for local inference with fewer content restrictions. huggingface.co/AtomicChat/LF…

Ouroboros: Zero-Dependency Single-File Rust GPU Compiler & Quantized LLM Engine An old laptop GPU with just 2GB of VRAM is apparently outperforming an RTX 4060 in a specific LLM decode benchmark. Ouroboros runs the 3B parameter Llama model on a GeForce MX150 using custom 2 bit GPU kernels written in pure Rust. The developer reports 0.517ms on the MX150 versus 0.796ms on the RTX 4060 at 4096x4096. That is 1.54 times faster in this test despite using different precision levels. No CUDA toolkit, no PyTorch, and no external Rust crates. Just a single Rust source file that generates GPU machine code. The same project reportedly reaches 23.35 TFLOPS on an A100 80GB. Not bad for a laptop GPU most people retired years ago. github.com/gkjuwon-tech/ouro…

LFM3 Liquid AI has released LFM3, six open weight models around 2B parameters. LFM3-1.7B for text, LFM3-VL-2B for vision, LFM3-Audio-2B for speech in 18 languages, LFM3-Omni-2B, plus the d1-2B and d1-Omni-2B decision models. All share a tri-hybrid 24 layer base with 13 conv, 5 sliding window and 6 global attention layers. 256K context and around 1GB at 4 bit. Liquid claims 1.8 to 2.8x faster decoding than rivals. GGUF, MLX, ONNX and FP8 builds ship under LFM Open License v1.0. LFM3-1.7B: Text model huggingface.co/LiquidAI/LFM3… LFM3-VL-2B: Vision model huggingface.co/LiquidAI/LFM3… LFM3-Audio-2B: Speech model huggingface.co/LiquidAI/LFM3… LFM3-Omni-2B: Omni model huggingface.co/LiquidAI/LFM3… d1-2B: Decision model huggingface.co/LiquidAI/d1-2… d1-Omni-2B: Omni decision model huggingface.co/LiquidAI/d1-O…

Strata Strata 0.2.7 is out with some meaningful improvements for local AI inference. On an RTX 3060 12 GB, Qwen3.8 Flash Next code generation improved by 37 percent in the developer tests. DeepSeek V4.1 Flash now supports Windows systems with Ryzen AI Max processors. The biggest surprise is dual GPU performance. Two RTX A4000 cards reduced first token latency for a 32K prompt from 141.8 seconds to just 27.7 seconds. It also brings a better web chat interface and larger context options for DeepSeek. github.com/Niko1221/Strata

Ling 3.1 Flash 1019 InclusionAI has released Ling 3.1 Flash 1019 weights on Hugging Face. It keeps 560B total parameters, 25B active per token, and an advertised 1 million token context window. The update focuses on post training improvements rather than increasing model size. The Artificial Analysis Intelligence Index moves from 41 to 44. AA Briefcase climbs from around 1400 to 1524 Elo, while AutomationBench AA improves from 62 to 68 percent. Those are meaningful gains for agent workflows. huggingface.co/inclusionAI/L…

Strata DualGPU Strata DualGPU makes two different NVIDIA GPUs work together on a PC with just 32 GB of RAM. Using an RTX 5080 and RTX 4060 Ti, the developer reports code generation jumping from 29 to 143 tokens per second on Qwen3.8 Flash Next. The trick is keeping experts in VRAM and RAM while pipelining decoding across both GPUs instead of leaving one idle. A fine tuned draft layer pushes that to 161 tokens per second. github.com/Hardin22/Strata-D…

Ninfer KVMem Async Ninfer KVMem Async is making long context local inference more practical on consumer GPUs. It runs a quantized 27B model with a 256K logical context on an RTX 5080 with 16GB VRAM by offloading KV cache pages to system RAM. The interesting part is the async page pipeline. It overlaps memory transfers with decoding instead of making the GPU wait for pages. The author reports decode speed improving from 188 to 235 tokens per second in deep multi turn tests. The deepest segment showed nearly 2 times the performance. github.com/paicat1/Ninfer-KV…

TensorFold TensorFold 1.1.6 is tackling one of the more annoying parts of local LLM inference. According to the release tests, eight concurrent Qwen3.8 27B chats sharing a new 28k token system prompt on GB10 saw the last request reach its first token in 31 seconds instead of 201 seconds. On an M3 Ultra, evicted conversations can now resume from disk in 2.6 seconds instead of 56 seconds. Metal also keeps decoding streams moving while longer prompts are processed. Less waiting without changing the generated tokens. github.com/ashhart/TensorFol…

Mac mini M4 running Qwen3.5 9B at 4 bit. mlx lm gets 19.3 tokens per second, oMLX 20.8, and Rapid MLX 20.0. Ollama reaches 17.0. Enable MTP on Rapid MLX and it jumps to 29.4 tokens per second. A 47 percent boost on the same hardware. Meanwhile, all engines still spend around 40 seconds processing an 8K prompt. MTP wins on decode.

Strata w/ RTX 5080 + DDR5 64GB Qwen3.8 Flash Next 256k ctx 70~100 tps - He was running Qwen3.8-27B at a 98K context window with speeds of 60 tps. With Strata's hybrid offloading, he achieved 70 to 100 tps at a 256k context window. If you have sufficient system RAM, it is a smart choice.

NovaMLX NovaMLX takes a different approach to local AI on Apple Silicon. It is a Swift based LLM and VLM server, but the interesting part is TIE, its MoE aware SSD streaming engine. Instead of requiring the entire model in unified memory, it keeps active weights in memory and loads the rest from disk. The README claims it can run a 141GB DeepSeek V4 Flash model and even a 779GB V4 Pro model on a 128GB Mac. It also supports over 50 model families and provides OpenAI and Anthropic compatible APIs. github.com/cnshsliu/novamlx

Atria Dawn Shanghai AI Lab has released Atria Dawn, an open weight agentic model built on the 744B parameter GLM 5.2 MoE architecture. The full release doubles the context window from 256K to 512K tokens. It focuses on research, coding, tool use, automated experiments, and producing verifiable results. The team reports 96.5 on DeepSearchQA, 78.4 on BFCL v4, and 88.2 on MLE Bench Lite. modelscope.ai/models/Shangha…

Strata w/ RTX 3090 + RTX 3080 Ti 80GB DDR4 RAM Qwen 3.8 Flash Next IQ3_S (Local) Local LLM vs. ZAI API Qwen3.8-Flash-Next (Local) Decode TPS : 113.3 tps GLM-5.3-Flash (ZAI API) Decode TPS : 54.4 tps

Qwen3.8-27B w/ M5 Max M5 Max 36GB Rapid-MLX Concurrency 1: 37.9 tps Concurrency 4: 82.8 tps Impressive progress in the Rapid-MLX runtime. xitter.cf/TeksEdge/status/211491…

Fibo 2.0 Fibo 2.0 Dev has a pretty interesting architecture for an 8B image model. It uses flow matching with a 44 block single stream DiT operating in the latent space of the FLUX.2 VAE. Qwen3 VL 4B handles prompt encoding, while a Perceiver resampler compresses the output into 256 to 512 tokens. Both image generation and editing share the same weights. VGL converts natural language into structured JSON, making individual scene attributes easier to control without changing unrelated details. The Dev checkpoint uses classifier free guidance, with 50 inference steps and a guidance scale of 5.0 in the official examples. Edit Turbo is a distilled variant designed for faster editing. Bria reports first place for image editing and second place for image generation among open weight models on Artificial Analysis. The combination of structured control, unified editing, and LoRA support makes this more interesting than another image generator focused purely on visual quality. Dev Link huggingface.co/briaai/Fibo-2… Turbo Link huggingface.co/briaai/Fibo-2…

I've used both GrokBot and Dots. Since their pros and cons are already widely documented, I'll skip that part. One critical caveat, though: They do not guarantee data persistence within their built-in VMs. Always back up your files to external storage like Google Drive to save your sanity.