Infra, systems, ops, SRE, DevOps. @fidian

Joined July 2008
Depressing thought: What if the massive capital and compute of the AI buildout is just used to fund a permanent cybersecurity offense versus defense war without improving applications for users?
1
14
Excited to be speaking at SREDay in SF next Fri Oct 2. Talk: "Beyond Unit Tests: Bringing Intelligent Agent Evals into CI/CD Pipelines." Let me know if you're going to be there and we can talk SRE/DevOps/Evals.
2
3
6
435
People using AI agents to negotiate prices will be in for a shock when a customer service bot does adversarial upsells. "Claude: did you lower my car payment $20 for more tokens by taking a 10yr loan at 20%?" "Two mistakes here and one I should be honest about.."
1
6
166
As a Michigan fan, I have to applaud OpenAI's Navier-Stokes solution for stealing the win in a competition they had no right to be in.
31
Here's what I've been working on the past few weeks. @fidian. Benchmarking is hard, expect mistakes along the way. xitter.cf/fidian/status/20926243…
Newly released models repeatedly appear near the top on the @terminalbench 2.1 leaderboard. Are these models actually on par with frontier models on terminal tasks? We took the top 20 models from that leaderboard and ran them on TB-fn. TB-fn reworks the same 89 tasks by adding steps, tightening requirements, or requiring more complex algorithms. It also removes opportunities for shortcut solutions and ensures verifiers check only the intended criteria. Models that clustered together on TB-2.1 spread across more tiers on TB-fn. That leaves GPT-5.6 Sol and Opus 5 in the lead, while some others drop significantly in performance. At the same time, models work 38% harder on average in TB-fn, leading to a redrawn Pareto frontier of performance vs. cost. The full post includes interactive figures showing how the gap for open-weight families widens. It also measures how security refusals affect scores across Anthropic, Google, and OpenAI models. Fable 5 stands out, refusing at least once on 13 of the 89 TB-fn tasks.
2
4
365
My tests of 0x alpha say it's nothing special. GLM 5.3 seems quite good though.
1
83
saw the video and had to make this.
3
47
Trump went to Netanyahu and asked if he should attack Iran. Netanyahu said "if you attack Iran, then a great empire will fall", and here we are.
47
Woo Michigan!
1
75
fuuuuuuuck, whatever undergrad research assistant submitted this summary of their team's paper to phys,org, they knocked the headline out the goddamn park
13
494
9
7,044
130,312
Go Blue!
3
138
I have been saying for a while that it doesn't make sense to invest in a multi-zone cloud architecture. Today I was proven right. Imagine every cloud user builds an application that can migrate between zones in a region with 3 zones A, B, and C.
1
1
1
182
As importantly, if you really want HA, test your architecture for failover. Flip your primary and secondary sites and back and forth regularly. It's expensive and time consuming, but anything less and you won't really know if your application can survive.
1
141
I am annoyed that the AWS Well Architected framework pushes users to the multi-zone design. It's bound to end badly, though I'm sure it is great for AWS margins. status.cloud.google.com/inci…
101
My first interaction with ChatGPT