Shipping to prod in a $100M+ ARR unicorn startup w/ local AI. 4x DGX Spark (GLM 5.3 Flash), 1x M5 Max 128GB (Qwen 3.8 Flash Next), 1x RTX 5090 (Qwen 3.8 27B).
Joined February 2018
- Tweets33K
- Following81
- Followers569
- Likes13.1K
It's really cool that I've got Astra and Opus continually finding interesting and meaningful ways to speed up inference for local DeepSeek V4.1 Flash 1017.
But I don't see any signs that they will ever stop, and I might never actually get to use the model if they keep working on it lol.
Speculative drafters have different ideal configuration shapes, depending on the content of what they're speculating.
What if we discovered the ideal shape per category of content, and then shifted the speculative drafter's strategy in real time depending on that category? 🤔
Codex is working on a Rust inference stack for DeepSeek V4.1 Flash 1017, with the goal of slightly higher speeds (and much more consistent prefill speeds) for native MXFP4 weights across 4x DGX Sparks.
Been working on this for a few days. It will probably be a few more days before it's ready. Would like to push performance a bit further, but also need to double-and-triple-check that there's no quality degradation, and address a wider range of performance/quality edge-cases.
Lol.
The things Opus 5.5 in Claude Code worries about during 100% pure technical LLM inference stack optimization work.
It's really hard to get people to understand how multifaceted models are.
Not to anthropomorphize them, but it's a useful metaphor to say that they're similar to people in this regard: It's hard to boil people into a single score saying "this is how good you are as a person".
I seriously doubt this result on my Rust inference stack for DeepSeek is consistently-reproducible, almost certainly a fluke... but it would be cool if it were.
> 69.3% of the web runs on PHP
And only 30.7% of the web gets more than 1 visit per month.
I pre-emptively disconnected ALL my sites / VPS servers from the web
I'll wait for the patch to hit at 2 PM UTC
If you have a PHP website, probably good to take them down until you patch PHP
69.3% of the web runs on PHP so this is going to be intense!
Good time maybe to buy cybersecurity stocks? (not financial advice)
Was the idea of the hackathon to build games, or was it to build graphics engines? If it was to build games, then spending the entire time "building a raycaster by hand" feels like it's missing the point, and the vibe coder might've stuck better to the spirit of the meetup.
Recently attended an in-person handheld game console hackathon. I wrote a raycaster by hand. Someone wearing a “I ❤️ slop” tshirt saw mine and said “oh I wrote a ray caster too!”. They showed me. It was a ray *tracer* and claude did all the work. The person didn’t know basic graphics terms. They couldn’t answer basic questions about how it worked, could only talk about their prompting harness. Sad how AI is ruining the craft and community for people that actually like to program and have spent their lives getting good at it.
