Pinned Tweet
i made astra, fable and other LLMs fight each other in minecraft
11 models, almost 200 duels. astra went 36-0
the game runs 20x slower, but models respond in real time, so speed matters
introducing MC-PvP-Bench. leaderboard and results below 🧵
Kimi K3.1 Pro is an improvement over Kimi K3 on minecraft pvp, even though it is slower. but it still loses to DeepSeek V4.1 Flash.
70
Inkling 2 Small enters minecraft pvp leaderboard. it does not show a breakthrough and is worse than deepseek v4.1 flash. it is comparable to other recent opensource models that struggle to find the enemy on the screen.
76
Claude Fable 5.2 is a small regression compared to 5.1 on Minecraft PvP. Likely due to being slower. Opus remains the best Claude model and competitive to Astra.
1
101
Gemini 4 Carbon and Hydrogen enter the minecraft pvp benchmark leaderboard but lose hard to their brother Argon.
1
3
186
however, the Muse Ember takes the place of Muse Spark on the pareto frontier, showing a much stronger result at a very low cost. It actually pushed DeepSeek V4.1 Flash out of the pareto frontier for the first time.
I decided to recalculate the costs of Muse Spark 1.3 using regular pricing instead of a heavily discounted contributor pricing which I used to run cheaper testing. I believe it is more fair to compare models at their default-path prices. That change moved it much closer to Muse Wonder on the cost efficiency chart and it leaves the pareto frontier.
1
4
297
I decided to recalculate the costs of Muse Spark 1.3 using regular pricing instead of a heavily discounted contributor pricing which I used to run cheaper testing. I believe it is more fair to compare models at their default-path prices. That change moved it much closer to Muse Wonder on the cost efficiency chart and it leaves the pareto frontier.
gpt-6.1 terra gets ~same ELO on minecraft pvp benchmark as gpt-6.1 sol while being 25% cheaper to run
55
some abuse detection system can just block you randomly and you won't be able to do anything unless you're popular here or know someone directly
openai, github, google - all can block you
it's a good idea to prepare for these cases in advance so that it doesn't fuck up all your work and life
do local backups, migrate to self-hosted solutions, add 2fa and recovery methods where possible
I dont even know what I did...not having access to my repo is teaching me github is a friction point and Ill be pondering what I want to do about this. Any ideas folks? Should I run my own repo on a VPS?
1
1
190
it just keeps working and it'll continue to work unless some retarded political shit gets in the way
A double exponential fits the data very well (R^2=0.99). Extrapolating that gives a predicted time horizon of ~80 work years by end of 2027. Extrapolating an exponential from post-Opus-4.5 gives ~30 work years. (up from Opus 5.5's AECI-implied TH of ~1 work-month)
28
It seems that Daybreak Blue is practically useless. You only get "reduced refusals" on old and small models like Luna or 5.6-Sol
78
three things in the past couple of months changed my mind about self-hosting local obliterated models at home.
i used to be very skeptical since they're many months behind the frontier in capabilities and slow to run on personal hardware. but:
1. openai banned all my accounts in august for "cyber abuse". refusals became big pain in the ass in general. it'll get worse.
2. claude banned me for political reasons. it'll get worse.
3. third-tier api providers cannot be trusted due to data retention and tool-call injection risk. even if they're honest, someone will hack them. it'll get worse.
i now have obliterated deepseek on my macbook, just in case i suddenly lose access to everything else. i recommend everyone to prepare for this too. atleast download the weights and figure out local inference so that you're ready.
Replying to @PalmerLuckey @dickclucas
Corporations lording over what you’re allowed to do with tools is not The Way. It’s like Photoshop stopping mid-painting and being, like, “I can’t let you draw that, Dave”. There will be endless political pressure on them to censor. Open local models are the only solution.
2
1
183























