Accelerating AI by Learning Optimal Datatypes
Canada
Joined October 2024
- Tweets40
- Following7
- Followers180
- Likes44
ByteShape retweeted
Trying out a handful of Qwen 3.8 27B quants tonight on my 4090 (custom lovelace runtime). Tests included HumanEval+ and a custom suite from my local harness sessions.
@ByteShape came out on top. Well done!🎉
Any other quants I should check out?
ByteShape retweeted
In the past week or so alone, ByteShape’s optimized AI models were downloaded more than one million times.
The pace is clearly accelerating.
ALT ByteShape Is Taking Off banner
ByteShape retweeted
These @ByteShape models likely don't get enough attention (they don't seem to be loud like many others!). I've been running for a couple weeks, so well tested. Worth trying if you run Qwen3.6. Quite a lot faster for me, on my setup as well.
huggingface.co/byteshape
ByteShape retweeted
+ Hermes Agent
+ Qwen3.5 35B A3B
+ 4x parallel agents with 262k context window (each)
+ Over 200 t/s token generation + 3000 t/s prefill
+ 23.2GB total VRAM consumption on RTX 5090
It can take 5 parallel agents, 4 was the sweet spot with 2x in completion time vs 1.74x.
Dream inference.
@NousResearch
@ByteShape
@ByteShape Qwen3.5-35B-A3B-IQ4_XS-4.12bpw.gguf is lethal.
It was the most efficient Qwen3.5 35B-A3B, consuming only 23GB VRAM at FULL native context, on a RTX 5090 with -b and -ub 256.
Just shy of 20 noisy points from UD-Q6_K_XL and a minute slower. However, Unsloth's was running at 131k.
This is a custom Rust/Next.JS bench. It doesn't reflect all other categories and use-cases.
ByteShape retweeted
ByteShape was quietly launched just before the year end. Two weeks ago, we announced our investment in the company. Since its launch, and with minimal fanfare on purpose, @ByteShape cumulative downloads have easily blown past 100,000. No small feat for a new startup!
ByteShape retweeted
Announcing @twosmallfishvc's investment in @ByteShape.
In short, ByteShape is delivering step-function gains in AI efficiency, including up to 7x faster training, up to 10x faster inference, plus up to 40% compression to reduce model size.
ByteShape retweeted
Excellent quantisation on Devstral Small 2
We released ShapeLearn-optimized GGUFs for:
• Devstral Small 2 24B, tuned for RTX 40/50 GPUs
• Qwen3 Coder 30B, runs everywhere, yes even the Pi
Maximum quality. Fastest TPS. Minimal compromise.
GGUFs + interactive plots are live: byteshape.com/blogs/Devstral…
ALT Every Hardware Deserves a Coder: Devstral Small 2 24B & Qwen3 Coder 30B
ByteShape retweeted
Raspberry Pi has a new AI HAT. This time with built-in 8 GB of RAM, so you can run machine vision + LLM inference all without touching the Pi's CPU. It's $130 and a little bit of a niche item. Find out why in my video: youtube.com/watch?v=jRQaur0L…
ByteShape retweeted
A 30B Qwen Model Walks into a Raspberry Pi and Runs in Real Time #HackerNews
byteshape.com/blogs/Qwen3-30…
