educator, consultant and proponent of the open metaverse
Los Angeles
Joined June 2021
- Tweets3.6K
- Following373
- Followers2.4K
- Likes19.6K
rejoice
Qwen4-27B spotted in Intel intel-xpu-kernels repo !
Work for Qwen4-27B support is already underway, this only means one thing and you know what that is.
LFGG @Alibaba_Qwen
Give us Qwen4-27B soon !
github.com/analytics-zoo/vll…
do you get your agents to ask clarifying questions on complex requests?
or do you let the the agent take the wheel and refine?
really hoping the at-home-tuners can perform some magic here.
Today, we’re releasing Bonsai 2 2B, 4B, and 9B.
Based on Qwen3.5, these new ternary models bring stronger reasoning, visual understanding, and tool calling to compact footprints designed for everyday devices:
• 2B: 0.43GB
• 4B: 0.95GB
• 9B: 2.0GB
For this release, we focused on what makes small models useful as local assistants: following instructions, avoiding repetitive loops, calling tools on mobile devices, and giving useful answers without unnecessary output. The 9B model uses roughly 40% fewer tokens on standard instruction-following and everyday math tasks. Solving the Riemann hypothesis can wait.
These models are successors to our first-generation Bonsai 1.7B, 4B, and 8B, and provide a foundation for bringing practical agentic functionality to local applications on personal computers, phones, and wearables.
The Bonsai 2 family now includes four sizes spanning 2B to 27B parameters. Developers can choose the balance of capability, memory use, and responsiveness that fits their application and hardware.
Available today under Apache 2.0.
I'm still in awe with @CaryPalmerr's implementation of @trymirai's Qwen3.8-27B-S-experimental.
134t/s decode on a 3090, and still bringing quality like this.
For local agentic work, nothing beats the overall performance.
Repo here: github.com/professorpalmer/m…
thanks to @PeasantSmith for the amazing work on Maya and @ItsmeAjayKV / @needmorevram for the inspo.
For the local llm folks: what multi agent setups are you actually running at home? Who delegates what to who?
everyone doing local ai should build their own personal OS from scratch.
you unintentionally build so many other OS's features from first principles.
fwiw
After weeks of testing, this is the recipe for the GPU middle class.
if you have in the 24GB VRAM tier and are looking to run a local LLM through a harness, @CaryPalmerr's implementation of @trymirai's codec on @Alibaba_Qwen's 3.8 27B is what you need.
github.com/professorpalmer/m…
After weeks of testing, this is the recipe for the GPU middle class.
if you have in the 24GB VRAM tier and are looking to run a local LLM through a harness, @CaryPalmerr's implementation of @trymirai's codec on @Alibaba_Qwen's 3.8 27B is what you need.
github.com/professorpalmer/m…
Finally Bonsai that works as advertised. Tested in Hermes, doing great so far at real work. Thanks @CaryPalmerr!
Bonsai 2 27b PTQ1_0 from @PrismML is now smarter than ever!
12gb GPU, 100tok/s, and agentic workflow failures fixed!
After a week of experiments, I managed to find the two places that agentic work broke down in the model weights. Instead of adjusting the weights directly, I managed to bake the solution into the serve itself.
You can follow each experiment, failure, & finding in the docs. Get the patch : github.com/professorpalmer/b…
