Rishi Mehta (@rishicomplex) header
Rishi Mehta (@rishicomplex) profile picture

Solve i̶n̶t̶e̶l̶l̶i̶g̶e̶n̶c̶e̶ ̶ coding, use it to solve everything else | Research @AnthropicAI | Past: RL @GoogleDeepmind: AlphaProof co-lead, Gemini.

San Francisco, CA
Joined July 2009
Opus 5.5 tops RoastBench
Made a little benchmark called RoastBench - it compares frontier models on their roast jokes. The models roast 10 personalities from comedy central roasts I enjoyed, and I manually rank their jokes. I also mark the ones that made me laugh. LLMs are way worse than top humans.
2
1
12
1,537
Couple bangers from them
1
1
328
RoastBench results for Fable 5.1 and GPT-6. They're tied on the main set, but fable does a lot better on the second set. Neither felt like a major step up though.
Made a little benchmark called RoastBench - it compares frontier models on their roast jokes. The models roast 10 personalities from comedy central roasts I enjoyed, and I manually rank their jokes. I also mark the ones that made me laugh. LLMs are way worse than top humans.
1
1
6
2,070
if you've been holding out on fable due to price or zdr you really should try fable 5.1
We’re introducing Claude Fable 5.1 and Claude Mythos 5.1. They're the world’s most advanced models for coding and knowledge work.
2
8
100
10,918
also lol'd at this from kimi
3
252
this one made me laugh
1
2
335
Added Opus 5, GPT 5.6 Sol and Kimi K3 to RoastBench. Opus 5 narrowly edges out Fable at the top
Made a little benchmark called RoastBench - it compares frontier models on their roast jokes. The models roast 10 personalities from comedy central roasts I enjoyed, and I manually rank their jokes. I also mark the ones that made me laugh. LLMs are way worse than top humans.
1
2
867
The one joke that made me laugh
1
3
580
Fable 5 beats Opus 4.8 on RoastBench (but still well behind humans)
Made a little benchmark called RoastBench - it compares frontier models on their roast jokes. The models roast 10 personalities from comedy central roasts I enjoyed, and I manually rank their jokes. I also mark the ones that made me laugh. LLMs are way worse than top humans.
2
1
8
2,130
For the first time I don't feel the need to review its code line-by-line. It works autonomously over long horizons, on underspecified prompts, figuring things out as it goes.
1
1
694