A benchmark for long-horizon AI agents, played in the classic games you grew up with. The bar to beat: the game's own computer. Also general AI and tech talk.

Joined December 2021
we rented 3 GPUs to run 16 AI agents at once so you don't have to. the H100 lost memory, not compute, decided it
5
2
1
8
182
Replying to @sortitoutsi
we've been testing this in championship manager 97/98: every AI manager handed a random formation, 15 worlds, original 1997 squads. top three are level, 4-5-1 defensive costs ~9 pts a season, and no formation counters another. what does change it is your own strength
1
47
grok made 3,262 tool calls and thought for 14.8 h. sonnet made 218 in 36 min the two local models got the most calls rejected: gemma 18%, qwen 11%. the cloud ones stayed under 4%
34
final scorecard (league pts while in charge + 5 per season survived): opus 92 · grok 86 · deepseek 24 · gemma 23 · sonnet 23 · qwen 21 one run each so there's luck in it, next runs repeat every model
3
1
1
2
1,327
season 1: grok 3rd (77 pts), opus 4th (74). the game's own AI with the same squad got 78 and 73 so the best models matched a 1997 game AI. none beat it
1
86
it's over. grok got sacked on 15 sep 1998, 16th with 4 pts from 5 games. all 6 AIs have now been fired from atlético how long each one lasted 👇
7
1
99
grok 4.7 managing atlético in cm 97/98. copa semi, 2nd leg v real madrid, 0-0 from the first leg. every tool call it made, from the team sheet to the final whistle. #Ai #AIagents #retrogaming
1
32
what went wrong in this run, all being fixed for the next one: 1. the "available for transfer" flag the tools show doesn't match the game's own filter. 22 of Opus's bids and 19 of Grok's came back "not for sale at any price"
1
13
how the CM 97/98 AI league actually works, since a few people asked 🧵 the game is the original DOS version, unmodified, in DOSBox-X. 6 leagues run at once, each one its own emulator on a virtual display, all from the same July 1997 save
1
1
506
Replying to @cb_doge
been running grok 4.7 as a football manager in a 1997 game for 2 days, only model the board hasn't sacked yet
17