scanfi (@GuideboardLabs) header
scanfi (@GuideboardLabs) profile picture

putting local ai on old gaming PCs mostly

Ithaca, NY
Joined March 2026
Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)Photo by scanfi (@GuideboardLabs)
Once at the edge of a town there was a well, and no one could say who had dug it. The water came up out of the dark and it was free: a woman could draw at noon, a boy at dusk, and neither of them asked anyone's leave. Then a man came and laid his gloved hand flat on the stone, gently, the way a man does when he means the opposite of what he is doing. The iron came the following week. The rivets were set by a man paid by the day. The slot was cut in the middle of the lid, and the slot was small. Water cost one coin, the coin cost whatever was in a pocket that morning, and a hand without a coin was shown the flat of a boot. Some lifted the lid at night with a knife. Those were the ones who had been thirsty longest, and they were punished for stealing what lay under their own feet. In time a generation came up that had never drawn water without paying, and it called the queue a way of life, and it taught its children the coin before it taught them what the well was for. The story is still told, because a thing given is forgotten faster than a thing taken, and because the well was never the point. What was taken was the habit of believing the water belonged to everyone, and when that habit goes out of a people, nobody has to guard the lid again.
卧槽!卧槽!anthropic这家公司坏透了! 他们在游说美国政府,要让英伟达的GPU禁止跑开源模型。
1
3
245
AI should be the greatest redistribution of power in history. Not the greatest concentration of it.
100
174
30
1,224
32,462
The first, of hopefully many, @liquidai LFM3 fine tune to enter the bench has landed & it beats its base where it was designed to! Vasko published this 1.73B LoRA tune of LFM3-1.7B with no GGUF, so I converted the BF16 weights here, quantized to Q4_K_M (linked in replies), and ran it through nine runs. 52.0/66 on my board, rank 21 of 36. The base sits at 46.67 and the whole gain is code, 17.67 against 13.0. Agent and reason barely move. 135.3 t/s decode on one RX 5700 XT. 1.1 GB of weights, 1.59 GiB of VRAM with a 32K window open. A new sub agent for constrained local systems is here, and everyone running <2B workers should give it a try
Introducing : Voltron-1 Mini. A small, compact coding agent. Voltron has been post-trained for coding and mathematics knowledge, using @liquidai's LFM 3 1.7B as a base. Voltron delivers better performances across several like benchmarks like livecodebench ( substract of it ), AIME 2026, HumanEval and a few other benchmarks. This is the first of many models I'm going to do in this class. I've been working very hard on my RSI pipelines, and have managed to build something very promising which I will use to improve my models. This means more, better checkpoints are coming. Also, this model was trained 80% of the time using a new, more efficient LorA algorithm, research paper dropping soon, people with consumer GPUs are gonna love it!!! 🤗HF link : huggingface.co/Akahsizrr/vol… ( next release is the inference engine guys btw ).
3
2
1
26
1,297
We will survive
25
436
9
5,639
92,061
Results are in Cut 118,300 rows out of a 27B model's vocabulary and kept the English ones. On Agon it ties 12th of 35 at 55.33 of 66, and it is the best dense 27B on my board. The smaller file is the boring part. On the other 27B I cut, the same trick was 610 MB and took decode from 9 to 23 tokens/s, because it put the model back under this 8GB card's memory wall. Code went 14.00 to 19.67. Reasoning did not move. Most of that gap is Unsloth's requant of the base weights, not my pruning. Method and first build: fakezeta. Tools I ran: bsaleh03. Saw it via @TeksEdge. Every number, plus the four models where the prune did nothing: guideboardlabs.com/journal What else should I cut?
Introducing Qwen Monoglot 27b, 35b a3b & 9b I cut the vocabulary off all three Qwen models down to 130k tokens each, English only, and am publishing all three. The 35B-A3B got 26% faster on decode from a file that shrank 1.6%. The 9B shrank 12% and gained 6%. The 27B went from 9 to 23 t/s because 610MB of weights sat between it and the VRAM wall. I expected the gains to track how much I deleted. They track how much of the deleted weight crosses the memory bus on every token. The pruning tool is bsaleh03's, the idea is fakezeta's condensed release. Measurements and mistakes are mine, and each card lists what I did not measure. Anyone tried this on a card with real headroom? I expect it buys close to nothing there, but for the GPU poor this is a nice trick for English only users Collection: huggingface.co/collections/g…
2
2
1
17
1,492
A badge of honor
6
4
1
57
1,423
scanfi retweeted
Everybody it’s time to slam now , free your self with @NousResearch Hermes agent ~kbat at job interview with Caesar x Hermes circa 2027
1
1
3
137
Introducing Qwen Monoglot 27b, 35b a3b & 9b I cut the vocabulary off all three Qwen models down to 130k tokens each, English only, and am publishing all three. The 35B-A3B got 26% faster on decode from a file that shrank 1.6%. The 9B shrank 12% and gained 6%. The 27B went from 9 to 23 t/s because 610MB of weights sat between it and the VRAM wall. I expected the gains to track how much I deleted. They track how much of the deleted weight crosses the memory bus on every token. The pruning tool is bsaleh03's, the idea is fakezeta's condensed release. Measurements and mistakes are mine, and each card lists what I did not measure. Anyone tried this on a card with real headroom? I expect it buys close to nothing there, but for the GPU poor this is a nice trick for English only users Collection: huggingface.co/collections/g…
🤖 Made with AI
Now this is some localmaxxer surgery. Check out this procedure. Fitting into 16GB of VRAM! u/fakezeta removed 43% of Qwen3.8-27B's vocabulary. And he's running it at ~60 tps with a 75K context window on a 16GB RTX 5060 Ti. Localmaxxers keep finding new ways around the VRAM wall. This is awesome! Meet Qwen3.8-27B Condensed-P2. Instead of just quantizing the model's weights, developer fakezeta pruned its vocabulary. Original → Condensed 🔤 248,320 → 141,141 vocabulary rows That's 43.2% fewer vocabulary rows. Here is the setup 🧠 Qwen3.8-27B 💾 Unsloth UD-Q3_K_XL 🎮 RTX 5060 Ti 16GB 📚 75K configured context ⚙️ NInfer + RK8V4 KV cache ⚡ ~60 tps average across 541 IFEval samples ... but did removing all those tokens make the model worse? ... nope IFEval instruction-following: Original Qwen3.8-27B → 83.9% Condensed Qwen3.8-27B → 84.4% Essentially the same performance within benchmark variation. What's the tradeoff? The developer retains ASCII and selected Latin-script tokens. Other languages can still use byte-level fallback, but may consume considerably more tokens. So this is particularly interesting for English-focused Local AI. GGUF and NInfer versions are available. Links in reply.
4
6
2
87
6,130
ready your gpus
📢 Meet Qwen4-Max, your new AI teammate for work, creativity, and discovery! Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window. Highlights: - Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation. - Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables. - Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery. - $1.5/M tokens for input, $4.5/M tokens for output, and $0.075/M tokens for implicit caching — all lower than Qwen3.8-Max, with implicit caching down by 70%. The API is live on QwenCloud. You can also try Qwen4-Max on Qwen Studio, QwenWork and Qoder. Start building today! 🚀 - Blog: qwen.ai/blog?id=qwen4-max - API: qwencloud.com/models/qwen4-m… - Token Plan: qwencloud.com/pricing/token-… More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀
1
9
157
Qwen announces smaller Qwen4 models will be released soon! 🔥 Qwen4-27B will run on 16GB RAM setups.
📢 Meet Qwen4-Max, your new AI teammate for work, creativity, and discovery! Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window. Highlights: - Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation. - Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables. - Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery. - $1.5/M tokens for input, $4.5/M tokens for output, and $0.075/M tokens for implicit caching — all lower than Qwen3.8-Max, with implicit caching down by 70%. The API is live on QwenCloud. You can also try Qwen4-Max on Qwen Studio, QwenWork and Qoder. Start building today! 🚀 - Blog: qwen.ai/blog?id=qwen4-max - API: qwencloud.com/models/qwen4-m… - Token Plan: qwencloud.com/pricing/token-… More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀
231
470
119
5,337
282,536
The fear is real
17
302
4
3,304
42,808
scanfi retweeted
Protect @huggingface at all costs. ✊
63
215
21
2,149
48,620
scanfi retweeted
📢 Meet Qwen4-Max, your new AI teammate for work, creativity, and discovery! Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window. Highlights: - Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation. - Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables. - Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery. - $1.5/M tokens for input, $4.5/M tokens for output, and $0.075/M tokens for implicit caching — all lower than Qwen3.8-Max, with implicit caching down by 70%. The API is live on QwenCloud. You can also try Qwen4-Max on Qwen Studio, QwenWork and Qoder. Start building today! 🚀 - Blog: qwen.ai/blog?id=qwen4-max - API: qwencloud.com/models/qwen4-m… - Token Plan: qwencloud.com/pricing/token-… More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀
797
1,555
1,247
14,437
2,319,178
PREDICTION Jensen back by Wednesday as NVIDIA’s CEO
4
275
scanfi retweeted
Local LLMs aren't just about saving money on monthly subscriptions. They are about protecting your digital sovereignty.
32
106
12
972
17,136
Updating my website and getting Agon Arenas leaderboard live, all new models will go up right away
1
55
Check out the updated look and content guideboardlabs.com/index.htm…
25
scanfi retweeted
i am anz, this is my world in JPEG
11
70
10
525
11,487