Steve White (@jstevewhite) header
Steve White (@jstevewhite) profile picture

Old Guy, geek, co-creator of the RailCore2 3d printer, thinking and joking @jstevewhite@mastodon.online

Joined April 2007
Photo by Steve White (@jstevewhite)Photo by Steve White (@jstevewhite)Photo by Steve White (@jstevewhite)Photo by Steve White (@jstevewhite)Photo by Steve White (@jstevewhite)Photo by Steve White (@jstevewhite)Photo by Steve White (@jstevewhite)Photo by Steve White (@jstevewhite)Photo by Steve White (@jstevewhite)Photo by Steve White (@jstevewhite)
Using opencode and qwen3.8-next-flash I got prefix caching working and it made a huge difference in performance, with ttft dropping from ~40seconds to more like eight to ten seconds. Furthermore I'm blown away by the coding it does. Not that Fable and Astra aren't better -
1
2
67
But because it's a local model and does things that Opus 4.5 struggled with. I can hand it a full repo and ask it to make a change and it can do it reliably. The cloud is faster, but this is mine.
1
31
qwen3.8-next-flash on my DGX-spak is an actually useful local coding model. 40 tokens/sec is slower than the cloud, but it's on my hardware. I use it for sensitive stuff (e.g. "set up this encryption,here's the key" and it doesn't go to the cloud ). It's gettin crazy out there.
4
12
927
I periodically go back and listen to the discographies of bands I've known but not done that with. I started on Bob Dylan a couple of days ago, and I'm astonished. I love this guy. It's crazy. Not a single song I wanted to skip and I'm eight albums in.
3
10
187
Got an ASUS Ascent GX10; it's the low rent DGX-Spark, though John Carmack was saying his Spark throttles, and I haven't seen that with my GX10. Anyway, it's an interesting case study in AI inference.
3
3
280
But today I found a GLM-4.6 at Q2_K_XL, so 90GB, and it runs pretty well. 18 tokens/sec, and it still manages to call tools successfully! I'm absolutely amazed at the response quality from a 355B with 32Bn active - not 4.5 Air, but the full 4.6.
1
4
195
Also, qwen3-235-a22b runs in 3bit - and actually runs very well. No gibberish even with long context. I'm pretty impressed.
3
91
PSA: They *ALL* blow smoke up your ass. Always make the LLM accountable if the answer matters.:
1
1
63
I cannot *tell* you how many times not asking this question has led me down garden paths. Explicitly tell them to look it up. If you have them, provide links.
1
43
My hints for coding with AI tools (Vibe Coding). Check it out. It works for me. Medium - Vibe Coding That Works medium.com/p/vibe-coding-tha…
1
90
This does not surprise me. All the high-scoring medical diagnostic AI work I've seen was with models trained to the job. I wouldn't expect general purpose LLMs to do well at this.
Bad news for AI-based radiology. 🤔 It checks if chatbots can diagnose hard radiology images like experts. Finds that board-certified radiologists scored 83%, trainees 45%, but the best performing AI from frontier labs, GPT-5, managed only 30%. 😨 Claims “doctor-level” AI in medicine is still far away. The team built 50 expert level cases across computed tomography (CT), magnetic resonance imaging (MRI), and X-ray. Each case had one clear diagnosis and no extra clinical history. They tested GPT-5, OpenAI o3, Gemini 2.5 Pro, Grok-4, and Claude Opus 4.1 in reasoning modes. Blinded radiologists graded answers as exact, partial, or wrong, then averaged scores. Experts beat trainees, and trainees beat every model by a wide margin. More reasoning barely helped accuracy, but it made replies about 6x slower. Models did best on MRI and struggled more on CT and X-ray. To explain mistakes, the authors built an error map covering missed or false findings, wrong location or meaning, early closure, and contradictions with the final answer. They also saw thinking traps like anchoring, availability bias, and skipping relevant regions. General purpose models are not ready to read hard cases without expert oversight.
2
4
144
Here's my research pipeline, if anyone is interested: github.com/jstevewhite/rsrch It's python based. Largely produced with Claude 4.5 and GPT5; I typed maybe 5-10%, but the design is all mine.
1
47
Requires LLM API key or local LLM (I've tested it with qwen3-30b-a3b on my mac studio and it works), an API key for SERPER or Tavily, and a Jina.ai key for reranking. Not hard to hack if you want to make it all local.
1
83
So I keep building research agents - as a kind of learning exercise; a system that accepts a general query and produces a report; like the DeepResearch offered by most frontier models. It's an interesting exercise, attempting to codify the heuristics we use for our own process.
2
2
50
I say... no, it did shut down. here's a link to an CNN article about it. Gemini says "Oh, shit, let me check. Let me re-evaluate your report. Your report is *highly accurate*." LOL
2
1
2
71
At least it didn't double-down, like qwen3-next, which assured me that the NYT, CNN, AP, snopes, and reuters were all compromised by an AI misinformation campaign because Charlie Kirk was alive and well.
1
68