GPT-QModel, QvQ ultra low bit quants. Contributor to stuff you are probably using like sglang, vllm, transformers. ucla alum

Earth
Joined February 2020
You need to have extreme curiousity if you want humans to sustain this war with AI. Either out monkey the AI, or retire. That's my motto. I am not retiring, yet. Gonna out-curious the Ai until that day comes. Curiosity is my #1 weapon. Not lemming is my #2.
7
1,453
Documentation is code, it is oracle. Technical writers sometimes needs to be taken out to the sheds. Nasdaq listed hw co still can't find itself to align cpu0/1 as 0 offset based or 1 offset based in its own docs. These two pdf diagrams are back to back next to each other.🫠
1
6
399
I also get a headache reading these docs because almost all hw companies would randomly flip the motherboard 180 degrees in various diagrams. Hello! My head can't spin around like an alien.
1
66
Many Python programs using env variables for flow control/config is a festering disease that should be cured. Just my two cents.
151
🥳 GPT-QModel 7.7.0 dropped! I appreciate the continued contributions from @IntelAI and @positron_ai 👊 * VTPQ added. * Extensive CPU kernels optimized for Zen5 @AIatAMD * Lots of micro optimizations in the quantization pipeline * Many more model support added/clean-ups.
1
1
4
309
🥰 Got this dm. Feels good to be acknowledged to have contributed to the advancement of ai in however tiny ways. Many startups, from m to b $ in vc using GPT-QModel,mostly in secret. I wish they upstream their own modifications but competition is so fierce, many don't. 😎
1
4
369
DeepSeek v4.1 Flash 1017 just reached batch 1 peak decode of 314 t/s on real coding work, not simulation. Avg 200 t/s on my coding agents. Setup is 4x 2019 era gpu. Full native, unmodified weights. Imagine what I can do with 2025 gpus. 😎
3
1
8
516
Cold prefill is peak sustained 10.2k t/s on large enough ctx.
1
1
94
I spoke to an environmentalist who wrote a book on China who thinks this did not lead to env/food crises and it was industrialization that is root of all evil.
In December 1958 the Polish embassy in Beijing refused to let the sparrow killers onto its grounds, so the Chinese surrounded it with drummers and kept the drums going for two days until every bird sheltering in the garden had dropped dead from the air, and the diplomats cleared them up with shovels. That is how thorough it was. The campaign had been running since the spring. Every village, school and factory was turned out with gongs, pans and firecrackers to keep the sparrows flying until their hearts gave out. Nests were torn from eaves, eggs smashed, chicks killed, and children went through the fields with slingshots and were paid by the bird. The dead came in by the basket, strung on wires, piled on carts, and the totals went up the chain as a harvest. Something close to two billion sparrows in two years. The embassy garden was the last place in the capital with birdsong in it, and the drums got that too. A zoologist called Zhu Xi had told them before it started that a king had tried this and lost. Frederick the Great of Prussia had put a bounty on sparrows in the 1740s to save his grain and his cherries, and the sparrows went, and the caterpillars they had been eating stripped his orchards to the stalk, and the king cancelled the bounty and paid to have sparrows brought back from neighbouring states and let loose. The whole episode was in the library. The ministry had a figure for what a sparrow ate and no figure for what it did, and a man with the second figure was told to sit down. The locusts arrived on schedule. Then the grain borers. A sparrow takes a few grains from a field in autumn and feeds its chicks nothing but insects all spring, and with the sparrows gone the insects had the rice and the wheat to themselves. The counties with the most sparrows to lose saw their grain fall the furthest, and the rice and wheat that had been holding the country up went down together. That was 1959, and the state took its grain quota anyway, on the harvest that had been reported. What was left in the villages was not enough, and the famine that followed over the next three years is the largest in human history: tens of millions dead, the estimates running from fifteen million to forty-five, whole counties in Anhui and Sichuan where the living could not bury the dead. The sparrows were one cause among several and the economists who have gone back to the county records put the bird's share at something over two million people, with another four hundred thousand never born. Two million, from a campaign against a thing that weighs an ounce. By 1960 the ornithologist Tso-hsin Cheng, who had been cutting open dead sparrows and counting the insects in their stomachs, got the bird struck off the list and bed bugs put in its place. There were almost no sparrows left in China to pardon. So the government did what Prussia had done two centuries before: it bought a quarter of a million sparrows from the Soviet Union and shipped them east in crates, to be released into a country that had drummed every one of their relatives out of the sky. The animal that does the invisible job is always the one on the ledger as a cost. Take the sheep off a chalk down and the orchids are under scrub in twenty years. Fence the cattle off a Welsh hill and the bracken takes it. Empty a savannah of its herds and the grass goes to fire. All of it written down, all of it in the library, all of it available to the next committee with a figure for what a cow eats. Beijing paid Moscow for the birds in hard currency. The men who had said so in 1958 were still not allowed to say so in 1960.
145
MiniMax M3.1 flash vs DeepSeek v4.1 flash 1017 has no data for many tests. Can you guess why? 🐋
MiniMax M3.1 Flash: Space Performance + Bunny Speed 🚀🐰 Yes, Space Bunny - which has topped global leaderboards with 200T+ tokens - was an earlier version of M3.1 Flash. 💰Redefining the performance economics of agentic AI. M3.1 Flash achieves frontier-level intelligence across key tasks such as Cowork (Automation-Bench: 62%; ALE: 30%), Coding (DeepSWE: 74%) and Visual Reasoning (BabyVision: 91%), at 1/3 the cost of M3 and 3x the speed. API pricing (flat rate): $0.15 / $0.60 per 1M input/output tokens, with cache reads at $0.015 / 1M tokens. Model weights will arrive on Oct 23. API: platform.minimax.io M Plan: minimax.io/m-plan MiniMax Code for desktop: agent.minimax.io/download
205
QUIC is a protocol hack around latency issue and introduces 1000x increase in code/buffering complexity. Don't use it unless you have to.
208
What ever it takes. Down the rabbit hole I go.
66
Today was my dremel gpu unlock. You do not want to know. 😂 Yes, on gpu.
1
138
Maybe gitlab should've used Pcre2. Fafo. Btw, I have the fastest python binding for pcre2 in PyPcre. So much faster than anything Python has and it re-uses system libs for instant cve mitigation pushed by OS vendor!
1
398
Why in the world aren't more projects using Pcre2? It's battle tested to the max and offers 10x the features other engine does. There is nothing better. "It needs to be ruby/golang native!" Nothing needs to be "native" if it sucks! github.com/modelCloud/PyPcre
71
Replying to @AIatAMD
@AIatAMD @IntelAI New AVX512/BF16 optimized kernels (originally written by Intel) to GPT-QModel and it's faster than ever. ~2x faster Aten int4. Tested on Zen5. Now you know why AMD is partnering with Intel on CPU inference.
1
6
256
I am not using full dims (max bw) on zen5 and have not yet validated on Granite Rapids. I literally can't afford ram for my mobos. Got cpu/mobo with no ram! If you like my work, donate some spare change via x money so I can populate those empty mobos with dims. github.com/ModelCloud/GPTQMo…
44
One thing I have to say about Tom Brady. I hate the guy. Destroyed my Raiders. One thing amazing about him is his ball velocity. The mofo rarely zips the ball. All the deliveries are so smooth, on-time with perfect spirals, all without the turbo zip you see from other qbs.
88