Pinned Tweet
LlamAmpere v0.7 out now, with two big adds that unlock significant gains on smaller cards: Latent Sparse Attention (paper by Prefontal AI) and dynamic microbatching!
This allows users to run the AI3bpw quant on 12 GB, with MTP and full context. KLD is 70% lower than my initial 12GB class model, and 30% lower than Unsloth's v3.1 3XXS model (while also supporting longer context and room for MTP).
My initial implementation of LSA is more quality protective than ported QSA or paper-faithful LSA (at the cost of a bit of VRAM and speed): it uses a tail of the most recent 8K tokens and 20% of the preceding ctx, versus a fixed 2K allotment (which struggles at deep context). dynamic ub lets users start with the highest prompt processing possible and switches to smaller sizes only once the room is actually needed for context.
These VRAM saving techniques also enable users to run the 2.3bpw model, with MTP, on the 10GB card, at 200k ctx.
Google and oAI both shipping frontier models undercutting Anthropic's pricing 70-80% the 2 days before the Anthropic IPO is objectively hilarious
5
338
linux quick launch commands if this is your first time (headed vs headless options depending on what you need, and 10 vs 12):
10 GB card, monitor plugged into it (180K context)
CARD=10dense; case ${CARD%-headless} in 24dense) M=AI4;; 12dense) M=AI3;; 10dense) M=AI2;; esac; R=qwen3.8-27b-Swift1.5-Uncensored-ATX-$M; HF=huggingface.co/jakeatx; { [ -d llamAmpere ] && git -C llamAmpere fetch --tags --force && git -C llamAmpere checkout v0.7 || git clone -b v0.7 github.com/JakeATX/llamAmper…; } && cd llamAmpere && cmake -S . -B build-sm86 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DLLAMA_BUILD_BORINGSSL=ON && cmake --build build-sm86 -j8 --target llama-server && mkdir -p models/$R && curl -fL -C - -o models/$R/$R.gguf $HF/$R/resolve/main/$R.gguf && curl -fL -C - -o models/$R/lsa-qwen38-27b-d2048.gguf $HF/qwen3.8-27b-Swift1.5-Uncensored-ATX-AI3/resolve/main/lsa-qwen38-27b-d2048.gguf && ./build-sm86/bin/llama-server --preset $CARD -m models/$R/$R.gguf
10 GB card, headless (213K context)
CARD=10dense-headless; case ${CARD%-headless} in 24dense) M=AI4;; 12dense) M=AI3;; 10dense) M=AI2;; esac; R=qwen3.8-27b-Swift1.5-Uncensored-ATX-$M; HF=huggingface.co/jakeatx; { [ -d llamAmpere ] && git -C llamAmpere fetch --tags --force && git -C llamAmpere checkout v0.7 || git clone -b v0.7 github.com/JakeATX/llamAmper…; } && cd llamAmpere && cmake -S . -B build-sm86 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DLLAMA_BUILD_BORINGSSL=ON && cmake --build build-sm86 -j8 --target llama-server && mkdir -p models/$R && curl -fL -C - -o models/$R/$R.gguf $HF/$R/resolve/main/$R.gguf && curl -fL -C - -o models/$R/lsa-qwen38-27b-d2048.gguf $HF/qwen3.8-27b-Swift1.5-Uncensored-ATX-AI3/resolve/main/lsa-qwen38-27b-d2048.gguf && ./build-sm86/bin/llama-server --preset $CARD -m models/$R/$R.gguf
12 GB card, monitor plugged into it (197K context)
CARD=12dense; case ${CARD%-headless} in 24dense) M=AI4;; 12dense) M=AI3;; 10dense) M=AI2;; esac; R=qwen3.8-27b-Swift1.5-Uncensored-ATX-$M; HF=huggingface.co/jakeatx; { [ -d llamAmpere ] && git -C llamAmpere fetch --tags --force && git -C llamAmpere checkout v0.7 || git clone -b v0.7 github.com/JakeATX/llamAmper…; } && cd llamAmpere && cmake -S . -B build-sm86 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DLLAMA_BUILD_BORINGSSL=ON && cmake --build build-sm86 -j8 --target llama-server && mkdir -p models/$R && curl -fL -C - -o models/$R/$R.gguf $HF/$R/resolve/main/$R.gguf && curl -fL -C - -o models/$R/lsa-qwen38-27b-d2048.gguf $HF/qwen3.8-27b-Swift1.5-Uncensored-ATX-AI3/resolve/main/lsa-qwen38-27b-d2048.gguf && ./build-sm86/bin/llama-server --preset $CARD -m models/$R/$R.gguf
12 GB card, headless (262K context)
CARD=12dense-headless; case ${CARD%-headless} in 24dense) M=AI4;; 12dense) M=AI3;; 10dense) M=AI2;; esac; R=qwen3.8-27b-Swift1.5-Uncensored-ATX-$M; HF=huggingface.co/jakeatx; { [ -d llamAmpere ] && git -C llamAmpere fetch --tags --force && git -C llamAmpere checkout v0.7 || git clone -b v0.7 github.com/JakeATX/llamAmper…; } && cd llamAmpere && cmake -S . -B build-sm86 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DLLAMA_BUILD_BORINGSSL=ON && cmake --build build-sm86 -j8 --target llama-server && mkdir -p models/$R && curl -fL -C - -o models/$R/$R.gguf $HF/$R/resolve/main/$R.gguf && curl -fL -C - -o models/$R/lsa-qwen38-27b-d2048.gguf $HF/qwen3.8-27b-Swift1.5-Uncensored-ATX-AI3/resolve/main/lsa-qwen38-27b-d2048.gguf && ./build-sm86/bin/llama-server --preset $CARD -m models/$R/$R.gguf
1
79
github: github.com/JakeATX/llamAmper…
12gb model (3bpw): huggingface.co/jakeatx/qwen3…
10GB model (2.3bpw): huggingface.co/jakeatx/qwen3…
2
111
Idk how you looks at the size/struct of models coming from Kimi, Qwen, GLM and thinks theres a need for high end hardware for them.
They’re going after low/mid market. Ant’s problem is that they’re in a cash burning fight w Google, oAI & meta, renting most of their compute.
More than 1.5 GW of chips are in China that shouldn’t be. Unwinding that web could reshape open-source AI.
“In our view, there’s more than one and a half gigawatts of chips in China that are not supposed to be there according to US law. How they get there, that whole web, that’s what’s going to unwind.”
“If NVIDIA can no longer support the open-source ecosystem, because that was Jensen’s primary goal, does that mean the open-source ecosystem falls away?”
“Now open source looks a lot weaker with NVIDIA down and out. Anthropic looks much more likely to get a complete monopoly or a duopoly. This bodes really well for Anthropic’s IPO next week.”
“They’re more diverse in what chips they use than any other company in the world.”
1
5
427
Frontier code and terminal bench science continue to absolutely crush the Chinese models 😬
Replying to @ArtificialAnlys
Full breakdown of Qwen4 Max's results across the evaluations in the Artificial Analysis Intelligence Index
1
8
1,124
📢 Meet Qwen4-Max, your new AI teammate for work, creativity, and discovery!
Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window.
Highlights:
- Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation.
- Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables.
- Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery.
- $1.5/M tokens for input, $4.5/M tokens for output, and $0.075/M tokens for implicit caching — all lower than Qwen3.8-Max, with implicit caching down by 70%.
The API is live on QwenCloud. You can also try Qwen4-Max on Qwen Studio, QwenWork and Qoder.
Start building today! 🚀
- Blog: qwen.ai/blog?id=qwen4-max
- API: qwencloud.com/models/qwen4-m…
- Token Plan: qwencloud.com/pricing/token-…
More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀
The evergreen mistake people make with all automations is they like to assume the last 5-10% sorts itself out neatly, when in reality it is usually multiple OOMs harder than everything that came before it.
1
7
366
by far the most amusing demo i've seen from google for one of their model releases. Nice work on the model and promo!
Most of us open Google Sheets for budgets, trackers, and tables. With Gemini 4 Hydrogen, we wanted to see if it could paint a moving landscape instead 🍃
In Google @Antigravity, Hydrogen handled the coding to paint and animate thousands of cells — creating coastal trains, drifting clouds, and falling autumn leaves that recalculate frame by frame inside the spreadsheet.
3
312
Gemini 4 Hydrogen, High effort is a rocket ship for getting work done rapidly with solid quality. Terra max remains your best value. Kinda weird for their X-high effort 'hydrogen' model to cost as much as Astra High, though 🧐
Jake retweeted
A few weeks ago, I found that Qwen3.8 27B's pass@4 on DeepSWE could beat the pass@1 of frontier models.
But there is a catch: in practice, you don't have the hidden tests to tell you which patch is correct.
So I tried something simple: run the task several times, then give all the attempts to a new run that writes the final patch.
It works very well. 4 Qwen3.8 27B attempts per task, 70 DeepSWE tasks:
- random pick: 44.3%
- best config, chosen in hindsight: 47.1%
- aggregation (AggAgent-style): 61.4%
- oracle with the hidden tests: 78.6%
The aggregator never sees the hidden tests. It only reads the previous attempts. And it even solved 2 tasks that none of the 4 attempts solved!
The cost: 4 runs + 1 aggregation run. Worth it for +17 points.
Details:
kaitchup.substack.com/p/batc…
I have 20 runs of DeepSWE, so I can push it higher, I think. We can also iterate: the aggregation run is just a new run that we can reuse.
Limitation: This is DeepSWE... I'll try to do the same with Terminal Bench 4.0 and Qwen3.8 Flash Next, but the cost could be too high.
I would happily sell Google my privacy when using carbon if they gave me a plan with a usable volume/subsidy.
As it stands the $20 plan is not usable for coding, so it’s hard to meaningfully experiment with a model outside oAI/Ant.
4
223
I really would like Theo to share this data because I know absolutely no one getting even $10k of API spend on Opus with $200/mo.
Anthropic isn’t losing $ on their subs, so you’re implying they’re making 98.5% margin 🤔
The $200 Claude Ultra plan gets you $15,000 of Opus 5.5 in a month, but only around $3,000 of Fable 5.5.
Even ignoring that Fable is restricted to 50% of your weekly limit, it burns faster per $ than Opus does.
More info below
1
7
675
Gotta love Fable 5.2 Low/Med being a price cut vs Opus. Basically no reason to go above Opus 5.5 medium now:
5
353
Some of you requested a higher quality, larger file, so I’ve added 4 bit version of AI+ ✅
20% lower KLD than AI3, 15% lower than v3.1 Unsloth 4 XS (18% faster), still 100+ TPS. You’ll need 96GB RAM, or to pin ~25% of the experts in VRAM (lower speed) to run on 64.
120+TPS and >3K/s pp on a single 3090 + 64GB: 12% faster than Strata, 27% lower error than v3.1 Unsloth 3 XXS, and 50% lower error than Strata's original 3XS file: llamAmpere v0.6.3 with Qwen3.8 Flash Next out now!
1
6
1,414




















