8×3090s + 256GB RAM · EXL3 local AI Full-stack SaaS, weird infra, consulting, gaming, movies, thoughts

USA
Joined October 2022
Reran this with some other stats + 3JS game at the end Model at the top
one prompt, one file, a full asteroids game. running locally on my 8x 3090 rig qwen 3.8 flash-next on exllamav3 (6.05bpw + MTP draft) 124 tok/s 0.44s to first token ~1,270W while generating (8 cards, each capped at 160W) @Alibaba_Qwen @turboderp_
2
232
one prompt, one file, a full asteroids game. running locally on my 8x 3090 rig qwen 3.8 flash-next on exllamav3 (6.05bpw + MTP draft) 124 tok/s 0.44s to first token ~1,270W while generating (8 cards, each capped at 160W) @Alibaba_Qwen @turboderp_
8
2
2
24
4,374
It may be ugly until I get a new rack but the 8x3090 build is finally done!! 192GB VRAM + 256GB RAM
132
33
17
1,242
68,649
low poly deer off the printer. one print before this turned into spaghetti, so this one feels earned
3
71
Managed to get Maya L faster Going to be my daily model now!
16
Replying to @PeasantSmith
This is what I am getting so far, I see others getting more prefill than me on glm 5.3 flash maya so if I can figure that out I'll probably use Maya Also did a quality test and qwen 3.8 flash 6.05bpw is better than all GLM 5.3 flash quants besides the 4.05 bpw But the smaller IQ3 models are pretty close to the 4.05 bpw
1
32
Been going ham with codex /goal 👀👀
29
I wish 35B A3B was coming to Qwen 4 a bit now... Just got done testing different models on my m1 mac and found 35B A3B 6 Bit to be the performing one
13
Finally got the second PSU and connected the 6th 3090
16