This should be the first diffusion model you try internally! Finally a diffusion model that will be near the Pareto frontier on price/performance, at fantastic speed. Lots of low latency applications for this one. I personally had doubts about scaling diffusion so very happy to be wrong about this. Would love to hear from people using this where they’ve found deficiencies.

I tried this new vision model yesterday on a few hundred prompts. This is not cheap at all, do not be fooled by the input and output price. It spends 20-45x the amount of tokens as models like Gemini Flash 3.8. It is also extremely slow. Like p50 was like 40s. Fastest request was 21s, with a trivial question. This is on the minimum low reasoning. It does really well and is a good model if you care only about accuracy! But it is not good/fast/cheap. It is good/slow/expensive.

I feel attacked

I just opened LinkedIn for the first time in ~20 days. The feed is a Bacchanalian slop apparatus. It is painful to scroll through. There is neither pleasure nor valuable information, and it feels alien and weird. I go to LI now only for a quick targeted person search.

I got access to Jev earlier today (thank you @hackgoofer). I have run ~5,000 requests so far, (which cost me around $2!), across classification, model routing, intent, steering, and many other things. tl;dr, Jev enables a new intelligent decision-making primitive, separate from deterministic code and LLM calls. This allows a class of decision-making that was neither suited to dumb, unintelligent code, nor to slow, expensive LLMs. It is super fast and cheap, and I think I will likely end up making a few Jev calls to every LLM call I make in my product. I think probably any company using LLM requests today can probably add a Jev call pre and/or post LLM calls to quite literally make their product much better for free, and have better tool calling behavior in many cases. I happened to have a personal benchmark for this as I’d been working on a ton of proactivity and classification tasks. I have been using the deepseek flash and more recently gpt 5.6 luna family of models as reasonably smart classifiers with low latency. Think questions like: - Did this conversation output contradict something they’ve mentioned before? - Should we send a followup message to this user based on our rules? - It’s been a few seconds of silence. Should we proactively send a message? In the past, I’ve been forced to write a bunch of what I call decision chains, mostly because an LLM classification call is very expensive in TIME (avg 4s), and less importantly can cost quite a bit if run on every message. Imagine a normal chat app. If you added 4s to every response to figure out if the response is good before sending it out, that ends up being pretty bad. So instead, I usually have to write some code that is a crude heuristic that runs quickly and decides whether to run the classifier. Obviously this sucks because you call the classifier many times that you don’t want to, which makes your p95 bad, and you also miss cases with the heuristic, and you also have to manage all of these weird chains. With Jev, it’s cheap enough, and fast enough (p50 ~150ms, p95 ~350ms in my testing!) that you can easily run it every turn. Heck you can reasonably run it before generation AND post generation, for any application that isn’t realtime voice, and still feel snappy. But this is just one use case. Think: smarter model routing, better context packing, better responses, better observability for intent/tags/safety, smarter retries and so much more. By simply thinking about the inputs and outcomes you want to enforce, you can use Jev to supercharge most model calls and reduce bad user outcomes. The more “quirks” a model has, the more valuable it ends up being. It’s a bit weird and unintuitive using Jev. Generally, you want to decrease the # of questions you ask a classifier, or it makes more mistakes. In fact, you might want to ask your questions kind of in a compound way, because the reasoning happens in a shared scratchpad of sorts. Adding questions muddies the scratchpad and makes it take longer. With Jev, you feel incentivized to go the other way, to formulate your query as a set of independent questions. It doesn’t feel like adding more questions decreases your performance on others. You can go a bit deeper to improve tool calls. Many model tools are things like turning on settings, or other things. You can easily improve models that are not very good at tool calling with Jev, by simply figuring out when to run them. You can do a pre-LLM call to figure out when to unfurl different tool definitions, in order to make your main LLM run better, you could run a background task with Jev + another LLM to reduce tool and context burden on your main LLM, and free it to be responsive. I’ve only scratched the surface of my testing but very excited!

If what they say in the research report is true, this will instantly replace at least a quarter of ALL LLM calls, basically capturing all tokens currently used for decision-making and scoring. Insanely excited to see if they come close to the level of usefulness, speed, and reliability they claim here.

I skimmed the DS 4.1 Flash report, and I feel fairly certain this will be the most used model on OpenRouter for the next few weeks. I had quite a few thoughts while reading. 1. This is clearly a good, if not great, model, and changes what I thought a model of this size could do. Reshapes the Pareto frontier. 2. This thing is great to serve! Insane KV cache reduction. Fewer active params. Will be huge on value once neoclouds compete the price into the ground. 3. This strikes me as a reaction to massive demand. This is not a "prioritizing AGI" model. They got slapped with infinite demand, and ran out of (finite) compute. They're switching DS V4 Pro over to serving this, and everything is optimized for big throughput on fewer chips. 4. Thought multiple times while reading: "Oh god I wish OpenAI/Anthropic published papers" 5. I will caveat that I think it is slightly benchmaxxed and does not perform as well as large models on my benchmark (not close to Sol, for example). DS is quite forthcoming with perf gaps though, and considering how bearish I have been on small models it surprised me. 6. DGX Sparks get more valuable lol... 7. They say pretty overtly that their verifiable task generation drove all the performance gain. Same vanilla SFT, RL with synthesis of verifiable tasks. Wider, sparser model. 8. Only 1/2 the prefill params active as during decode? I skimmed this portion but had not seen this idea before.

Ultra-fast, cheap models allow you to do cool things! For example, here's a demo to visually prompt designs in real-time, and iterate faster than typing out changes...

The funniest thing to me about this is that she listened to her own words and thought she sounded like an idiot

Amazing piece that everyone at all interested in chip throughput should read, best article I've read this month. The design could look less like Claudeslop but incredible content written so clearly and earnestly.