I ranked my AI subscriptions per dollar. Then I changed the unit, and the winner changed too
API dollars: Claude wins
Tokens and benchmark tasks: OpenAI wins
Output tokens: Claude again
But they didn't do the same work: the Claude seat handled corporate tasks with my review, OpenAI my own projects in more autonomous way. Measuring is harder than it looks
khalymon.dev/lab/what-ai-sub…
25
Serhii Khalymon retweeted
the most import skill in 2026 is still being able to tell your agents "wtf is this crap, no, not like this"
if you stop doing that, you're basically fucked, despite what some devfluencers would like you to think.
I built a personal tool that pulls facts from a log, gives each one an ID, and lets an investigation agent cite them
Telling the model to leave a checked section unchanged failed: it edited the section in all three runs and reported it as unchanged
When code delivered that section, all three runs kept it intact. When the tool also built the evidence, the model only picked claims by short ID
Against my first compact version: 15% fewer tokens and 19% lower estimated model cost
The first verification workflow was a regression: +74% tokens, +36% estimated cost, and review still found errors in all three reports
The catch: code checks the facts the model selects, not the prose around them. That prose still had errors
Before you call a check a guarantee, write down the exact object and property it covers
khalymon.dev/lab/what-a-veri…
4
4
148
Serhii Khalymon retweeted
ABSOLUTE INSANITY -EVERY RECORD DESTROYED
NOVEMBER NATIONAL RECORDS
IVORY COAST
38.0 Mahandiana
CAMEROON
41.5 Garoua
CHAD
42.4 Mongo
42.1 Bokoro
NIGER
43.5 Tillaberi
42.1 Birni N Konni
41.3 Dosso
41.1 Gaya
41.0 Tahoua
NIGERIA
41.0 Sokoto
ALGERIA
39.8 In Guezzam
HIGH MINIMUMS
EGYPT
23.2 Dabaa
TOGO
26.0 Mango
I've been experimenting with my home setup. It's my dev environment and the place where I run test projects in containers
What I'd really like there is a persistent agent that keeps an eye on the system and fixes nodes when something goes wrong
Things seem to be heading that way anyway. Systems built with coding agents are also easier to maintain with coding agents, and that's already fairly cheap
At OpenAI DevDay Exchange Berlin, @whoiskatrin shared that OpenAI's infrastructure is now monitored by an agentic system. If I remember correctly, the cost was around $16 an hour. At that scale, it's next to nothing
The direction is pretty clear now. Persistent agents that live with your system and keep it running are where this is going
4
2
116
Could the on-device model on my Mac mini replace a cheap cloud call for a small image-sorting job? No
30 COCO 2017 val grids, 270 photos, 90 positives, one yes/no question per photo:
- Apple Foundation Models missed 30 of 90 positives, 5/30 grids fully right
- Decisions API + GPT-6 Luna: 23/30, ~$0.23 per 1,000 grids
- GPT-6.1 Terra: 26/30, ~$2.12
- GPT-6.1 Sol (later run): 24/30, ~$4.26
At n=30 the cloud setups can't be told apart. Apple's gap is the only clearly real one
Also: the prompt cache was written on every request and never read, so caching made the run more expensive
I picked Decisions alone. Costs are estimates from token usage, not a bill
khalymon.dev/lab/coco-vision…
74
In an agentic dev pipeline, verification isn't something you set up once. It rots like code
This week I froze changes to the dev workflow of a personal project where I build everything through an agent pipeline, and looked back at recent coding sessions. Two findings. The security audit and a GraphQL contract check had been failing on main for a while. My CI budget had run out, so they simply weren't running, and I was relying only on the local checks the agent runs. And the workflow itself had too many steps the agent didn't need to take
Neither shows up while you're building. You only see it in a periodic retrospective of the sessions
So the job is shifting. The logic is still mine, but more of my time goes into maintaining the pipeline that produces and checks the code. The checks need an owner too
2
259
Sometimes I feel exactly like this. Not because the credits are free, but because they run out way too fast, so I keep moving to the next provider
24
Where did we take a wrong turn?
Hetzner's new CPX02: 1 vCPU, 1 GB RAM, 20 GB SSD, €5.99/mo.
Not long ago that kind of money got you 4 cores and 8 GB.
RAM prices went up, GPUs are eaten by AI, and the cheap VPS that every side project started on is quietly turning into a luxury.
3
1
89
Didn't think a 27B model would run on 8 GB of VRAM. Today it did, on my home lab.
Bonsai 2 27B (1-bit PTQ) through PrismML's llama.cpp on an RTX 2070 Super:
• 64K context
• all 65/65 layers on GPU, peak VRAM 7.16 GiB
• 49 tok/s decode on short context
• at 60K tokens: 240 tok/s prefill, ~21 tok/s decode
Next step: plugging it into my dev workflow to see what a local model can actually handle there. No verdict yet, just starting the experiments.
6
1
99
the /statechart skill in pstack v0.15.36 is now stable and ready for use! it's best combined with /create-formal-verification-skill to abstractly model state machines and distributed systems to find bugs.
the skill contains a mini tlc compiler that parses, lowers to a small IR, and has a backend that outputs mmd syntax. if you want you can also ask your agent to create charts in whatever style you like, the skills tells it to use pele.run/ which is very themeable so you can make blueprints, hand drawn encyclopedia illustrations or whatever you like
here's a chart of the /statechart skill and 2 real bugs i just found. try it out and let me know what you think!
*ehm ehm* BIG ANNOUNCEMENTS DAY 📣
two new things cooking in herdr labs 🐏🧪
a brand new herdr-core, and a desktop app built on it. open source.
first, the app: software that fits you. every part of it is a file your agent can change. you just ask.
My biggest fear used to be losing the roof over my head.
Now it's my card getting declined on AI subscription renewal day.
I could probably live without a roof. Living without agents reviewing my PRs at 2am is another story.
Seriously though: I'm not afraid of losing things anymore. I'm afraid of falling behind. Every week something new opens up, and that's where my real edge is now. Priorities have shifted.
1
34
My release assistant cut model cost by 59% and took 51% longer. I kept both numbers in the title.
I compared two setups on 8 historical PRs, with the same model and budget cap: read-only GitHub tools alone, and GitHub tools plus Release Manager.
Both made all 8 hold/ship calls correctly. The difference was depth: with GitHub tools alone, the model explained 68 of 128 failed jobs. With Release Manager, 120. A binary readiness check would have hidden most of that.
Some evidence was there and still got missed. One 384,702-character bundle had the error messages past the parts the agent read. In an offline replay, a 6,319-character per-job index replaced that bundle: 98.4% smaller, with evidence for all 20 jobs available.
Cost: $14.19 down to $5.81. Summed runtime: 18.0 up to 27.1 min.
Eight PRs, one attempt per arm, unblinded grading. Not a benchmark.
Data + method notes:
khalymon.dev/lab/release-man…
63
What "targeted tools on top of existing harnesses" means in practice:
Don't let the model grep logs and guess. Give it a tool that builds the evidence (parsed records, checked facts), and let the model do the investigation.
And every "verified" claim should name exactly what was checked. Nothing more.
16









