AI systems, Local LLMs, Agents, GPUs I test models, read code, and post the numbers. Programmer / DJ
Seoul, KR
Joined September 2013
- Tweets2.2K
- Following237
- Followers629
- Likes6.1K
dextune retweeted
If you have spare RAM, increase --conversation-cache for Strata now. With 64GB RAM, I cached 32 conversations. Token use doubled vs. yesterday, but power draw fell 30%—so energy cost per token dropped to 35% of yesterday’s, a 65% saving. That matters for local LLMs.
dextune retweeted
Introducing Code2Video-AA, our independent implementation of HeyGen’s Code2Video Bench, which evaluates how well models create product and brand marketing videos in code
Code2Video-AA tests models across 114 design briefs, tasking them with turning each product or brand marketing brief into a @HyperFrames_ HTML composition, which we render to video. Model outputs are evaluated through pairwise comparisons with other model outputs across five axes: Engagement, Prompt Intent, Composition, Temporal Consistency, and Craft. We use the outcomes of these comparisons to calculate an Elo rating for each axis, which are then averaged to produce the headline Code2Video-AA Elo. Code2Video-AA was built in partnership with @HeyGen.
Claude Opus 5.5 (max with fallback, 1536) leads Code2Video-AA by 93 points, but costs $3.08 per task, ~10x more than GPT-6.1 Sol (max, 1369), which trails by 167 points.
Key findings:
➤ Claude Opus 5.5 leads on all five judge axes: Models are judged across 5 axes: Engagement, Prompt Intent, Composition, Temporal Consistency, and Craft. Claude Sonnet 5.5 (max with fallback, 1443) and GPT-6 Astra (max, 1427) are statistically tied for second in overall score
➤ Cost per task varies more than 300x, from $0.01 to $3.89: GPT-6.1 Sol (max), fourth, has a Cost per Task of $0.31, about 90% lower than Claude Opus 5.5 (max with fallback) at $3.08
➤ Open weights models trail by at least 483 points: DeepSeek V4.1 Flash (max, 1053), MiMo-V2.6-Flash (1052) and MiMo-V2.6-Pro (1052) are the top open weights models, at $0.01 to $0.08 per task
Code2Video-AA details:
➤ Models are tested on a single response without tools. We render it with HyperFrames, with no network access
➤ A judge (Gemini 4 Argon, high) compares pair of videos across five axes: engagement, prompt intent, composition, temporal consistency and craft, with the final score being calculated from a combination of each individual axis Elo
➤ Over 35,000 verdicts rank 27 models by Elo, with 95% confidence intervals
More results, example videos and how Code2Video-AA works follow in the thread below
dextune retweeted
Qwen4-27B might be closer than we think.
hzjane, a developer contributing to Intel linked analytics-zoo, has already published FP8/INT4 inference benchmarks and kernel optimizations targeting Qwen4-27B on Intel B70 GPUs.
The code reveals clues to a dense architecture with a 5,120 hidden size and 4-stream hyper-connections.
No official release date yet, but the groundwork is clearly underway.
github.com/analytics-zoo/vll…
dextune retweeted
KRAFTONによるRaon-1-mini-previewのGGUFフォーマット版が公開されているHugging Faceリポジトリ
huggingface.co/KRAFTON/Raon-…
Reddit（r/LocalLLaMA）におけるRaonとFlintの性能比較やエージェントタスクでの実用性に関する議論スレッド
reddit.com/r/LocalLLaMA/comm…
dextune retweeted
Kimi K3.1 Pro is not beating the API wrapper allegations
Gave it and Claude Opus 5.5 another task. This time a full underwater scene: a research station, a bathyscaphe, and a robotic arm collecting a sample from the seabed
Claude on the left. Kimi on the right
I'm supposed to believe this is the model that made those launch demos?
Or am I doing something wrong?
What the f**ck is Kimi K3.1 Pro even making?
This is the quality gap I was talking about. I gave Kimi and Claude Opus 5.5 the same task: create a 3D bathyscaphe, but disabled any skills
Claude on the left: $0.51
Kimi on the right: $1.08
Somehow the model that needs another model to do its job also costs twice as much
dextune retweeted
MiniMax M3.1 getting more love ❤️
@MiniMax_AI Flash (Space Bunny) for 2x DGX Spark.
Quant is done, now working on the recipe. Based on a baseline recipe provided by @MiaAI_lab! Thanks, Mia!
Porting in @ashxhart latest TensorFold version to be able to build on that and then contribute upstream PR's.
Already leveraged several improvements based on my previous work on other recipes.
dextune retweeted
Upcoming models: Claude Fable 5.5 (routed) vs. Gemini 4 Carbon checkpoint in the Arena.
>As of now, Fable 5.5 edges out Carbon.
>Still too early to tell,excited to see how they perform at launch.
dextune retweeted
Cleared 40 tokens/s.
DeepSeek-V4.1-Flash running locally on 2× RTX 3090 + 128 GB RAM, using dsx and T16.
41 tok/s measured in free generation with expert deferral + FP8 projections. Rust, CUDA, and a lot of profiling.
github.com/lxe/dsx
dextune retweeted
Same two RTX 3090s. Same 280W per card. Same 27B INT4 model.
One change to how the GPU comes up, applied before the driver loads. No extra watts!
BF16 GEMM: 63.7 to 93.1 TFLOPS per GPU (+46%)
TF32: 34.9 to 54.8 (+57%)
FP16: 62.3 to 88.3 (+42%)
Prefill, fresh prompt, no cache:
256K: 666 to 829 tok/s (+24%)
63.5K: 913 to 1,094 tok/s (+20%)
7.4K: 1,025 to 1,198 tok/s (+17%)
Decode:
256K: 95.4 to 104.8 tok/s (+10%)
63.5K: 121.9 to 126.7 tok/s (+4%)
7.4K: 132.7 to 131.7 tok/s (flat, inside the sample range)
First token on a cached 256K prompt: 4.38s to 3.62s (-17%)
Repo: github.com/Zagitta/gpu-unloc…
Credit: @superalesha
If you run 3090s, what are you seeing?
dextune retweeted
🚨New contender for the two Spark category🚨
DeepSeek-V4.1-Flash-1017 on two DGX Sparks: 70.6 tok/s with @bertholomusai's TensorFold recipe.
bertholomus' own EXL3 2.5 bpw quant, one model split across both Sparks (TP2).
The recipe v0.7.2 as of Oct 24, commit 30a5bf9, with the author's sampling and context.
Thinking off for every number here, except the GPQA line marked with thinking.
The recipe wants RDMA GID index 5, which my Sparks did not have, so I gave the link ports IPv4 addresses. One Spark, an ASUS GX10, has an older driver.
Decode, code: 70.5 tok/s
Decode, prose: 70.6 tok/s
500 words: about 10.0 s
Reading a 32K-token prompt: 1,830 tok/s
First token after a 32K prompt: 17.9 s
Four requests at once: 107.4 tok/s in total
Eight requests at once: 106.3 tok/s in total
Context: 1,048,576 tokens, the recipe's setting
Largest context I tested: 131,072 tokens
The recipe runs 4 slots, so with eight requests the rest wait for a free slot.
For scale, bertholomus' 4.0 bpw quant on four Sparks:
Decode, code: 82.8 tok/s
Decode, prose: 80.8 tok/s
Eight requests at once: 141.2 tok/s in total
MMLU-Pro: 62.0% ± 4.7 points
GPQA Diamond: 43.9% ± 6.9 points
IFEval: 92.0% ± 3.8 points
LiveCodeBench: 60.0% ± 14.5 points
Long text 8K: 67.5% ± 14.0 points
Long text 32K: 57.5% ± 14.7 points
GPQA with thinking: 80.0% ± 13.9 points
Open: MMLU-Pro and GPQA answers were capped at 48 tokens (one letter). Answers that hit a cap count as wrong:
MMLU-Pro: 49 of 400 cut
GPQA Diamond: 46 of 198 cut
Long text: 3 answers cut
LiveCodeBench: 2 answers cut
GPQA with thinking: 5 of 30 cut
Quality and tool calls in Mia's sparkDash 2.50.0:
QA: 89.3% ± 5.0 points
MMLU: 93.0% ± 3.0 points
Tool Eval: 93/100
Every number, the exact command and the raw output: niklaslenz.com/v2/deepseek-v…
You can play the world it built there in your browser.
Recipe, quant and TensorFold fork by @bertholomusai. TensorFold engine by @ashxhart. sparkDash by @MiaAI_lab. Model by DeepSeek.
dextune retweeted
24gb of VRAM ? Not any more
Single 3060 running Qwen3.8-27b
65 t/s code and 700 t/s prefill
128k context at Q4
vision on gpu at Q8
cache and checkpoints working,
Dflash2 Q4 drafter in vram.
using a hybrid Escha W2 model I made.
Now has a gpu arena for KV and vision to offload when not in use. Hopefully will be the best 12gb card runtime.
Flys on the 4070 ti.
Next up is a new quant using same codebook with a RCO or Qtip approach to the weights targeting same 8.6gb size.
dextune retweeted
Trending on Hugging Face today (84 upvotes):
IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems
huggingface.co/papers/2610.3…
dextune retweeted
Un developer armo su propio laboratorio de ESP32 que corre gratis en el navegador y lo puedes usar tu tambien
Se llama ESP Labs y lo hizo Abderrahmen Gharsallah con Opus 5.5. Está pensado para el ESP32-S3: ves cómo se conecta todo y entiendes qué hace cada cosa antes de tocar la placa.
Tiene un pinout en vivo, diagramas de cableado, simuladores en cada lección y un code reader que te explica el firmware línea por línea. No solo ves qué hace la placa, ves qué parte del código lo provoca.
Lo que aprendes:
→ Tu primer programa y tu primer circuito con el LED RGB
→ Manejar la pantalla de 1.9″ y los botones
→ Wi-Fi: escanear redes, conectarte, servir páginas y updates over the air
→ RFID y NFC, Bluetooth LE y sonido
Para qué sirve:
→ Pasar de Arduino a ESP32 entendiendo qué cambia
→ Leer firmware real antes de flashearlo a tu placa
→ Tener una guía si ya tienes un ESP32-S3 y no sabes por dónde empezar
No es un curso en video. Es un laboratorio donde lees, simulas y conectas antes de gastar en componentes.
