strawgate retweeted
I’m glad @strawgate's standards for coffee are lower than his standards for code review because many coffees were brewed by yours truly in the creation of this video. I was thrilled to sit down with Bill, who is the head of product at @pydantic, to hear about how he and the team use @Macroscope. It’s an absolute joy to work with Pydantic and to have a small part in supporting their work.
strawgate retweeted
"Big brains trust Braintrust."
Big brains check the pricing page.
We checked it for you: one night of serious evals uses the entire monthly score allowance. Nights two through thirty bill at $1.50 per thousand. That's $26,100 a year to score one agent, nightly, using OpenAI's own public eval suite — before data and retention.
Our pricing page is shorter. Scores are spans. Spans are $2 a million. Same year of scoring: $36.
$36 vs $26,100.
Big brains read the article pydantic.io/qwbaV
strawgate retweeted
Our social media team is off for the week, so @strawgate's in charge. Separately, his "no making fun of competitors" pledge expired at midnight.
We resume operations with a pricing observation: 50 million eval scores on Braintrust Pro costs $75,174. In Pydantic Logfire, a score is just a span. Same 50 million: $100.
The meter doesn't just cost money. It's why your sampling is set to 10%, and why last week's failure lived in the other 90%.
Full arithmetic: pydantic.io/5Yn3V
