๐จ๐ฆ Obsessed with all things Ai Agents, Loops, LLMs, Fine-Tuning https://teapawt.net/t.co/9thyt19bL5 - Entrepreneur. Builder.
Canada
Joined May 2026
- Tweets2K
- Following462
- Followers436
- Likes4.6K
GPTware retweeted
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context.
Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7ร faster and full attention 72.4x faster.
With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington
GPTware retweeted
Georgia Tech built ARK for the ugly part of disaggregated LLM serving: shipping a huge KV cache from the prefill GPUs to the decode GPUs without ECMP hashing two elephant flows onto the same spine.
My take: Those transfers last hundreds of milliseconds to seconds, so a hash collision can sit on one link while other spines stay idle. ARK has senders pick source ports whose hashes put concurrent flows on different spines, with no switch changes and no reordering on the receive side. In their RDMA packet simulations, mean flow completion time dropped by as much as 27.3 percent and the P95 by as much as 34.0 percent at moderate load, which cut mean time to first token by up to 12.9 percent. At higher load the fabric saturates and the gain shrinks. The caveat is that this is simulation, not a production fabric. Still, if you are splitting prefill and decode across boxes, the path the KV takes probably matters as much as the kernels.
arxiv.org/abs/2610.41160
GPTware retweeted
50% OFF service on medai.li! Pay @ashxhart's Mac 512gb 2Tb fund: ko-fi.com/ashxhart
Your top supporters from the last 90 days: ๐ฅ 1. iamMrDuncan ๐ฅ 2. Dennis Rye ๐ฅ 3. Adam 4. MedAI.li Natalia Sokolova 5. Ko-fi User ko-fi.com/ashxhart/leaderboaโฆ
GPTware retweeted
1/3
A CPU inside an LLM is one thing, but running another LLM inside an LLM would be taking things a bit too far, wouldn't it?
Anyways, here's an LLM running inside an LLM : QwenPU, an LLM (@karpathy's stories260k) running on an NPU compiled inside Qwen 3.5-4B's weights.
GPTware retweeted
Reddit's CEO said people are getting answers from Google's AI Mode instead of getting sent to Reddit. Google has already said AI Mode is cannibalizing its own search business. Millions of publishers and companies built their whole discovery path on that one handoff.
Once the intermediary answers the question instead of routing it, the traffic goes with it, and you find out how little of that demand you actually owned.
Traditional companies that treat Google as a permanent pipe are going to keep learning that after the quarterly report, not before.
There are many that will be happy to see Reddit go.
The collapse of Reddit will be studied.
After reporting earnings, its stock collapsed 30% in a single day.
It's down 44% in 2 weeks.
But the most important detail is what was revealed in that earnings call.
The CEO said AI is killing Google search.
People no longer get sent to Reddit from Google. They merely get their answers from the AI Mode itself.
Even Google confirmed this themselves with their disastrous results - AI mode is cannibalising their search business.
Pretty soon, no one will visit Reddit voluntarily. Its traffic will continue to crater.
The worst part?
Reddit isn't alone. There are millions of companies and publishers that rely on search traffic.
That's what happens when you rely on a single funnel for discovery.
GPTware retweeted
Some lessons learned networking DGX Spark's to DGX Station using Microtik CRS804:
al-engr.com/networking-dgx-sโฆ
GPTware retweeted
The question that is stopping me from buying an M5 Ultra: why does a Mac with 1.2 TB/s of bandwidth lose to DGX Spark at long context and with multiple users?
With one user, decode is memory-bound. Every token means reading all the active weights once, and the GPU mostly waits for data. More bandwidth means more tok/s, and that's where the Mac shines. ๐
Batching changes everything. With 8 users you read the weights once and use them for 8 tokens. Same bytes, 8ร the math. Throughput scales almost for freeโฆ until compute becomes the bottleneck. ๐
That's where the Mac hits a wall: good bandwidth, lower low-precision compute. The Spark's Blackwell tensor cores with native FP4 give far more FLOPs per GB/s of bandwidth, so it keeps scaling while the Mac stops.
Real numbers on DeepSeek V4.1 Flash with TensorFold, total tok/s from 1 to 8 users (chart below):
2ร Spark: 77 โ 105 โ 150 โ 194 (@rwojo)
1ร M5 Ultra: 63 โ 78 โ 98 โ 99 (@TechMDAI, first runs, capped at 4 slots)
Long context is even worse. Prefill is pure compute, attention grows with context length, and the KV cache is per-user: no way to share it. On DeepSeek V4 Flash the Mac's decode drops from ~60 to ~27 tok/s, while 2 Sparks hold ~51 (tests by @satsuki2694).
Bottom line: the Mac is unbeatable for fitting huge models (MoE) in a single box, but the Spark wins when you need compute: long prompts, many users, agents. Different tools, different jobs.
I run both, and I was planning a deep dive on the M5 Ultra to really understand its limits, but at this price I don't know if it makes sense spending all these money just to make tests and share data. I'll try to figure out something, otherwise I'll wait for others to test these concurrency limits in depth. ๐ช
GPTware retweeted
How much performance do you actually lose when capping a DGX Spark and where are the sweet spots?
As a follow-up to a previous post I decided to test it using my 2 x Sparks and likely the best model and quant currently available @MiaAI_lab's Deepseek V4.1 1017 EXL3.
Depending on your preference the sweet spots are 2200 MHz for close to original performance or 1900 MHz if you seriously want to reduce Temperatures and power consumption.
Anything below 1900 MHz and the Sparks start fighting with themselves.
GPTware retweeted
๐ณ้ฒธ้ฑผๅ
ๅผไปฌ๏ผDeepSeek ่ฆๆพๅคงๆๆฅๅฆ๏ผ
V5ใV5-openใV5-next้ฝ็ซฏไธๆกไบใ
deepseek Harness ็ๆๆฐไปฃ็ ไธญๅ็ฐๅทฒ็ปๆ็ธๅ
ณ็ไปฃ็ ไบใ
็ๆฅ็ฆปๅๅธไนไธ่ฟไบใ
github.com/deepseek-ai/deepsโฆ
GPTware retweeted
@AntLingAGI making moves!
Post training for Ling-Flash-3.1-1019 yielded a 3 point jump in the @ArtificialAnlys Intelligence Index (41 to 44) and puts this model ahead of 5.3-Flash and Haiku 5.5!
Weights available now: huggingface.co/inclusionAI/Lโฆ
GPTware retweeted
When you look at the spec of modern GPUs such as Blackwells, you'll see that it has FP4/FP8 tensor cores which older GPUs lacks. This means when we load a NVFP4 model on 3090, it cannot compute in FP4 and needs to go through extra steps to unpack it. So how much does this matter? Been wondering for awhile so put it to test
Bottom line is, negligible when it comes to decode, big impact on prefill. Also matters quite a bit when concurrency gets involved, so keep all this mind when you are selecting your next GPU.
GPTware retweeted
1/7 ๐ฅฅ New speculative decoding method!
Introducing DCOCO: Contextual Correction of Parallel Drafts.
Parallel drafters are fast, but lose context between predicted tokens.
DCOCO restores it without giving up parallelism.
Up to 6.6ร faster than vanilla decoding. ๐งต
GPTware retweeted
Qwen4-27B spotted in Intel intel-xpu-kernels repo !
Work for Qwen4-27B support is already underway, this only means one thing and you know what that is.
LFGG @Alibaba_Qwen
Give us Qwen4-27B soon !
github.com/analytics-zoo/vllโฆ
GPTware retweeted
๐จ @MiniMax_AI ๐ ๐ฏ.๐ญ ๐๐๐๐ฆ๐ โ ๐ฐ๐ฎ๐ด๐ ๐ฃ๐๐ฅ๐๐ ๐๐ง๐๐ฅ๐ฆ. ๐ข๐ก๐ ๐๐๐ซ ๐ฆ๐ฃ๐๐ฅ๐. ๐ต๐ณ% ๐ง๐ข๐๐๐ก ๐๐๐ฅ๐๐๐ ๐๐ก๐ง! ๐งช๐ฅ
I've done it AGAIN, and this time I'm releasing TWO new SAGE-EXL3 quantizations of the brand-new MiniMax M3.1 Flash!
428B total parameters.
23B active.
Native 1M-token context capability.
My goal was simple:
๐ฆ๐ค๐จ๐๐๐ญ๐ ๐ง๐๐ ๐ ๐ข๐๐๐, ๐ก๐ข๐ง ๐๐ง๐ฆ ๐๐ก๐ง๐๐๐๐๐๐๐ก๐๐.
And the fidelity measurements are some of my most exciting yet! ๐คฏ
โโโโโโโโโโโโโโโโโโโโ
๐ข ๐ฅ๐๐๐๐๐ฆ๐ #๐ญ โ ๐ข๐ก๐ ๐๐๐ซ ๐ฆ๐ฃ๐๐ฅ๐
๐ ๐ถ๐ป๐ถ๐ ๐ฎ๐
-๐ ๐ฏ.๐ญ-๐๐น๐ฎ๐๐ต-๐๐ซ๐๐ฏ-๐ฆ๐๐๐-๐ฎ.๐ญ๐ฑ๐ฏ๐ฝ๐
๐ฆ 108.64 GiB checkpoint
๐ง 96.89% Top-1 Token Agreement
๐ฏ 99.75% Top-5 Agreement
๐ Mean KL Divergence: 0.02528
๐๐๐ ๐ข๐ ๐ง๐๐๐ง ๐๐ก ๐ ๐ฆ๐๐ก๐๐๐ ๐ญ๐ฎ๐ด๐๐ ๐๐๐ซ ๐ฆ๐ฃ๐๐ฅ๐!
And YES, I already have a working vLLM serving configuration!
But there's a catch, and I'll explain why shortly. ๐
โโโโโโโโโโโโโโโโโโโโ
๐ฃ ๐ฅ๐๐๐๐๐ฆ๐ #๐ฎ โ ๐ง๐ช๐ข ๐๐๐ซ ๐ฆ๐ฃ๐๐ฅ๐๐ฆ
๐ ๐ถ๐ป๐ถ๐ ๐ฎ๐
-๐ ๐ฏ.๐ญ-๐๐น๐ฎ๐๐ต-๐๐ซ๐๐ฏ-๐ฆ๐๐๐-๐ฐ.๐ฎ๐ฒ๐ฏ๐ฝ๐
๐ฆ 209.88 GiB checkpoint
๐ง 98.44% Top-1 Token Agreement
๐ฏ 99.94% Top-5 Agreement
๐ Mean KL Divergence: JUST 0.00592!
๐ก๐๐ก๐๐ง๐ฌ-๐๐๐๐๐ง ๐ฃ๐ข๐๐ก๐ง ๐๐ข๐จ๐ฅ ๐๐ข๐จ๐ฅ ๐ฃ๐๐ฅ๐๐๐ก๐ง ๐ง๐ข๐๐๐ก ๐๐๐ฅ๐๐๐ ๐๐ก๐ง! ๐คฏ
That's exceptionally close to the unquantized reference in my held-out token-level evaluation.
Both releases were measured against the original unquantized MiniMax checkpoint using full-vocabulary teacher-forced evaluation.
These are quantization-fidelity measurements, not task pass rates or proof of mathematically lossless behavior.
But THESE are the kinds of quality-per-bit results I'm chasing!
โโโโโโโโโโโโโโโโโโโโ
โ ๏ธ ๐ง๐๐ ๐ฃ๐ฅ๐ข๐๐๐๐ ๐๐ฆ๐ก'๐ง ๐๐จ๐ฆ๐ง ๐ง๐๐ ๐ ๐ข๐๐๐ ๐๐ก๐ฌ๐ ๐ข๐ฅ๐.
It's the inference engine.
My current ONE-SPARK vLLM recipe WORKS, but it's still a WORK IN PROGRESS.
The measured configuration currently reaches 32K context using a quantized KV cache.
MiniMax M3.1 Flash supports up to 1M native context, but vLLM's memory overhead on this hardware is consuming headroom I'd rather dedicate to the model and its context.
I could shrink the weights further to compensate.
๐๐จ๐ง ๐ช๐๐ฌ ๐ฆ๐๐๐ฅ๐๐๐๐๐ ๐ ๐ข๐๐๐ ๐๐๐๐๐๐๐ง๐ฌ ๐ง๐ข ๐๐๐๐ข๐ ๐ ๐ข๐๐๐ง๐ ๐ฅ๐จ๐ก๐ง๐๐ ๐ ๐ข๐ฉ๐๐ฅ๐๐๐๐?!
That's the wrong trade-off for what I'm trying to accomplish.
I'm optimizing for the BEST possible balance between model quality, memory usage and practical inference speed.
And that's where something else I've been building comes in...
โก ๐ฅ๐๐ซ๐๐ฏ.
Our upcoming public inference engine and dedicated recipe are aimed at removing these limitations and making the model's full native context practical on Spark.
We're continuing to validate that work before release.
๐ฆ๐ง๐๐ฌ ๐ง๐จ๐ก๐๐. ๐ง๐๐ ๐ฃ๐จ๐๐๐๐ ๐ฅ๐๐๐๐๐ฆ๐ ๐๐ฆ ๐๐๐ง๐ง๐๐ก๐ ๐๐๐ข๐ฆ๐๐ฅ! ๐
โโโโโโโโโโโโโโโโโโโโ
๐ค ๐๐๐๐๐๐ก๐ ๐๐๐ ๐ง๐ช๐ข-๐ฆ๐ฃ๐๐ฅ๐ ๐๐๐ฉ๐๐๐ข๐ฃ๐๐ฅ๐ฆ!
Here's where I'd love some help from the community!
The 4.26 bpw model is already available on Hugging Face, but I haven't had the time to properly build and validate its TWO-SPARK serving recipe.
If you have two DGX Sparks and experience with distributed inference, I'd LOVE to collaborate!
Fork my GitHub recipe.
Experiment.
Submit a PR with your improvements and measurements.
๐ ๐ช๐๐ก๐ง ๐ง๐ข ๐ช๐๐๐๐ข๐ ๐ ๐๐ข๐ก๐ง๐ฅ๐๐๐จ๐ง๐ข๐ฅ๐ฆ ๐๐ก๐ ๐๐๐ฉ๐ ๐ง๐๐๐ ๐ฃ๐ฅ๐ข๐ฃ๐๐ฅ ๐๐ฅ๐๐๐๐ง ๐๐ข๐ฅ ๐ง๐๐๐๐ฅ ๐ช๐ข๐ฅ๐! ๐
I want these releases to push the quality-per-bit frontier for EXL3, while making it easier for the community to reproduce and extend the inference results.
โโโโโโโโโโโโโโโโโโโโ
๐ ๐ข๐ก๐ ๐ฆ๐ฃ๐๐ฅ๐ โ ๐ฎ.๐ญ๐ฑ๐๐ฃ๐ช
huggingface.co/vcruz305/Miniโฆ
๐ ๐ง๐ช๐ข ๐ฆ๐ฃ๐๐ฅ๐๐ฆ โ ๐ฐ.๐ฎ๐ฒ๐๐ฃ๐ช
huggingface.co/vcruz305/Miniโฆ
๐ ๏ธ ๐ฅ๐๐๐๐ฃ๐ & ๐๐ข๐ ๐ ๐จ๐ก๐๐ง๐ฌ ๐ฃ๐ฅ๐ฆ
github.com/vcruz305/MiniMax-โฆ
๐ง๐ช๐ข ๐ค๐จ๐๐ก๐ง๐ฆ. ๐ง๐ช๐ข ๐ ๐๐ ๐ข๐ฅ๐ฌ ๐ง๐๐ฅ๐๐๐ง๐ฆ. ๐ข๐ก๐ ๐ฃ๐ฅ๐๐ข๐ฅ๐๐ง๐ฌ: ๐๐๐๐๐๐๐ง๐ฌ.
And the next chapter is getting very interesting. ๐งช๐ฅ
MiniMax M3.1 Flash weights are now public. 428B parameters total, 23B activated per token.
Agentic performance: A generational leap in agentic tasks, coding, and cybersecurity โ 76.2 on BankerToolBench, 73.7% on DeepSWE v1.1, 61.7 on OSWorld V2, 29.6 on Agents' Last Exam, and 82.2 on CyberGym.
Efficient by design: MiniMax Sparse Attention and DSpark speculative decoding improve inference efficiency.
Hugging Face:
huggingface.co/MiniMaxAI/Minโฆ
ModelScope:
modelscope.cn/models/MiniMaxโฆ
API: platform.minimax.io
M Plan: minimax.io/m-planMiniMax
MCode for desktop: agent.minimax.io/download
GPTware retweeted
๐จThinkingCap: Qwen3.8-Flash-Next is out!
The most token-efficient Qwen3.8-Flash-Next available.
๐Up to 51% fewer thinking tokens, 38.5% on average
๐ฏNegligible accuracy loss 1.03pp
๐ขThinkingCap PRO: 60% fewer thinking tokens on average - Contact our enterprise[at]bottlecapai [dot]com
๐Blogpost: bottlecapai.com/post/thinkinโฆ
๐คHugging Face: huggingface.co/bottlecapai/Tโฆ
๐GGUF: huggingface.co/bottlecapai/Tโฆ
GPTware retweeted
Two years of work, and PySR v2 is finally out!
If you haven't used it before: PySR is a machine learning library that finds short equations that fit your data.
I've always wanted a PyTorch for symbolic learning. PySR v2 is a big step towards that:
GPTware retweeted
Mac Studio M5 Ultra vs DGX Spark ร2ใไปๅบฆใฏ้ทๆๅ
ฅๅใงๆฏ่ผ๏ผ๐ฌ
Context capใ200Kใซๅบๅฎใๅ
ฅๅใๅขใใใฆใฟใใ
QwenใฏMacใฎTTFTใๅงๅ็ใซ็ญใใใฉใDecodeใงใฏSparkใ้ทๆใงใๅฎๅฎใDeepSeekใฏ้ทๆใซใชใใจDecodeใงSparkใ้่ปข๏ผ
ใขใใซใจๅฎ่ก็ฐๅขใฎ็ตใฟๅใใใง็ตๆใๅ
จ็ถ้ใใฎใ้ข็ฝใใญ๏ผ
