Local AI on 2x DGX Spark. Inference engineering. Frontier open models, optimised. Pull-and-run recipes and real benchmarks. Next up: M5 Max vs Spark.
Sydney, New South Wales
Joined July 2026
- Tweets1.1K
- Following101
- Followers103
- Likes59
Pinned Tweet
New Kolibri-1 recipe built for 2x DGX Spark on TensorFold github.com/ajensenwaud/Kolib… @MiaAI_lab @ashxhart
Very proud to have contributed to this release! @ashxhart
(GH: ajensenwaud)
TensorFold 1.1.6 is out.
Eight chats arrive together with the same new 28k-token system prompt. On a GB10 with Qwen3.8-27B, the last chat gets its first token after 31s instead of 201s. Every reply matches its run alone.
Also in this release:
- Opt-in conversation disk cache on Mac.
- Qwen3.8-27B on Metal keeps decoding streams moving during long prompts.
- Gemma 4 CUDA chat-cache fixes and GLM-5.3-Flash image-cache reuse on Metal.
- Spark-X2.5/2.6-4B on CUDA and Metal.
- Drama speech sampling controls and speaker tags.
- TensorFold plan estimates CUDA model memory before downloading or loading weights.
There are also tool-call, memory-admission and macOS download fixes, plus updates to the experimental DeepSeek-V4.1 engine.
Huge thanks to the contributors:
@jvr0x @bobclawblaw @krunkosaurus @lasse_ai
And on GitHub: jschmied, wojo, ecohash-co, ajensenwaud, akol1, philip-pentatonic, gilby and BlivionIaG.
Thanks to Boscoeuk on GitHub for the reports too.
To everyone testing, sharing hardware access and supporting TensorFold, thank you. If I've missed anyone, I'm sorry.
@MiaAI_lab @volatilemarkts @plotarmordev @petruspennanen
tensorfold.dev
An 816B model, Muse Spark, running on 2x DGX Spark (1.8bpw), engineering the USS Enterprise. Attached is a nice video of its handling of Three.js, using @NousResearch Hermes Agent as the harness. Link to the recipe in the comments.
Meta's 816B Muse Spark Open is 1600 GB at 16bpw. It now runs on two in 195 GB, 4x faster than Unsloth's 2bpw model with 1.8bpw.
Unsloth's version does 7 tok/s on the same two boxes. Muspark (built on TensorFold) does ~32tok/s with a ~900K context.
Quality unchanged at GSM8K 196/200 vs 193/200.
Recipe and benchmark in the replies.
Recipe, patches and benchmarks are open:
github.com/ajensenwaud/muspa…
The remarkable thing is Muspark runs reasonably well without loss of quality compared to baseline - 3-4x throughout against Unsloth. Big driver was thoughtful quantisation (variable) and TensorFold.
sparkDash is awesome. I use it to monitor my Sparks at home. As a former Unix system admin, I can highly recommend it. xitter.cf/miaai_lab/status/21131…
DeepSeek V4.1 Flash decode speeds posted on X in the last 48 hours. All figures are self-reported and the setups differ, so this isn't a like-for-like comparison.
410 tok/s, 4x RTX PRO 6000, engine not stated, @0xSero
~320 tok/s, 8x RTX PRO 6000 Max-Q, vLLM + DSpark TP8, @hshimodaira xitter.cf/hshimodaira/status/211…
~200 tok/s, 4x RTX PRO 6000, vLLM with the rtx6kpro recipe, @siegl_rene xitter.cf/siegl_rene/status/2112…
113 tok/s, 2x DGX Spark, Blackfold, @Blackwellboy xitter.cf/Blackwellboy/status/21…
79 tok/s, M3 Ultra, engine not stated, @dataphwj xitter.cf/dataphwj/status/211247…
70-100 tok/s, 2x DGX Spark, MiaAI-Lab recipe + knapcio TP4 fork, @unsignedbytes xitter.cf/unsignedbytes/status/2…
68 tok/s, M3 Ultra 512GB, oMLX with MTP, @jundotkim xitter.cf/jundotkim/status/21128…
68-71 tok/s, 2x DGX Spark, EXL3 2.9 bpw, @easthambuilds xitter.cf/easthambuilds/status/2…
48.5 tok/s, 2x M3 Ultra, ds4 MXFP4, @ivanfioravanti xitter.cf/ivanfioravanti/status/…
40-50 tok/s, 2x DGX Spark, vLLM EXL3 2.7 bpw, @sfxnz xitter.cf/sfxnz/status/211219585…
Next up: 2x DGX Spark on NVIDIA Flare. Numbers to follow.
DeepSeek-V4.1-Flash-1017 on 2x DGX spark, quantised in-house to 2.7 bpw EXL3 🔥
- 40-50 tok/s decode
- 1M context
- Vision on vLLM
- KL 0.025 vs the unquantised model
This recipe will be significantly improved over time!
./run.sh and go
github.com/sfxnz/DeepSeek-V4…
Anders retweeted
Do you know the meaning of the @NousResearch logo?
The girl is called the "Nous Girl". She was drawn in July 2023 by Nous's first artist, who goes by John Galt. Yes, the Atlas Shrugged rebel
She stands for the rebellion of decentralized AI, and for keeping our humanity in the face of censorship and machine intelligence
The words around her: "Rebellion to tyrants is obedience to God."
That line goes back to 1776. Jefferson, Franklin and Adams put it on their proposed Great Seal of the United States. Congress said no, so Jefferson put it on his own personal seal instead
Nous pointed it at AI. Today's tyrants are closed labs and gatekeepers deciding who gets access to intelligence. The rebellion is open weights, open agents and open infrastructure.
That's why I build on Hermes
Wearing it proudly 🏴
Anders retweeted
We have 2700+ signatures on our petition to protect your right to intelligence.
Sign: righttointelligence.org
Reading this article about companies restricting tokens and AI usage makes me appreciate owning my own intelligence.
theguardian.com/technology/2…
Apple research: Muse Spark Open running on a cluster of 4 Mac Studios at 36 tok/sec
machinelearning.apple.com/re…
Meta's Muse Spark Open: 816B params, 42B active. Weights are 462 GB at MLX 4-bit. Two DGX Sparks hold 256 GB.
So 4-bit doesn't fit. About 2 bits per weight lands near 204 GB. Quantising it for 2x Spark now, numbers soon.
Ported ColGrep-Agent-2B to Hermes Agent on a DGX Spark and A/B tested it with Claude Opus 5.5 on 32 code-search questions. 3.1% fewer tokens, 95% CI -6.5% to +11.7%, so no measurable saving. Opus already answers in about 5 calls, leaving few search turns to cut. github.com/ajensenwaud/dgx-c…
Just got access to Beam. Now it’s time to play around. Do you think we can squeeze it into 2x Sparks?
For those of you questioning local AI: I can achieve 60+ tok/sec with frontier capability on two Sparks with high quality weights, quantised. I have complete control of where my data goes and how my model behaves. I run my own agents the way I want them to.
Thanks to @MiaAI_lab for publishing this model.
@XiaomiMiMo v2.6 Flash now ported to TensorFold v1.0.x after a night of debugging Zig. +36% TPS uplift on coding, 4 streams, still 77-78 tok/s for single stream coding. Patch set available for upstream if there is interest.
See github.com/ajensenwaud/MiMo-… for build instructions.
@MiaAI_lab @ashxhart
