😭😭😭

How is concurrency and quality of answers on this vs something smaller but at q8?

Hey could you run a test with the same unlock that got you those stats, but this time power limit it to 200W or less? Would be interesting to see how much power impacts the performance unlock

Qwen 27b BF16 or even q8 seems to perform better than these 100-200B models (Qwen flash, deepseek flash, GLM 5.3 flash) running at low quants like 2 bpw or q2 On currency is way better on it on my machine as well

tried the gpu unlock. smaller bump than everyone else is posting, which makes sense when every card is capped at 160w until the house wiring gets upgraded

😂😂😂

Whatttttttttttttt

nothing says new season like a map from 2019

props for giving ollama its mtp build too. a 2.7x win over a handicapped baseline wouldn't mean much

I can’t wait till I’m done with all these tests on my machine so I can actually use it haha

agree, and post time to first token at that length too. decode at 128k can look fine while you sit waiting on prefill

Reran this with some other stats + 3JS game at the end Model at the top

Yes I have to cap it because of electric wiring

I have ran some strata tests will post later. Also did Maya tests. Multiple agents test, etc

Hmm for me it's when tasks a long for example with /goal. A lot of tool calling can happen and I think a better quant will be able to do the job in significantly less tool calls than a smaller quant. I also stress test quants against tools i use often to see to decide

Please compare quants you’re running vs seeing when comparing speeds

Does it work with strata or exlama?