Daniil Sedov (@Gusarich) header
Daniil Sedov (@Gusarich) profile picture

currently using llms to find bugs. opinions are my own.

Joined October 2023
i made astra, fable and other LLMs fight each other in minecraft 11 models, almost 200 duels. astra went 36-0 the game runs 20x slower, but models respond in real time, so speed matters introducing MC-PvP-Bench. leaderboard and results below 🧵
7
3
1
26
1,569
Kimi K3.1 Pro is an improvement over Kimi K3 on minecraft pvp, even though it is slower. but it still loses to DeepSeek V4.1 Flash.
70
Inkling 2 Small enters minecraft pvp leaderboard. it does not show a breakthrough and is worse than deepseek v4.1 flash. it is comparable to other recent opensource models that struggle to find the enemy on the screen.
76
Claude Fable 5.2 is a small regression compared to 5.1 on Minecraft PvP. Likely due to being slower. Opus remains the best Claude model and competitive to Astra.
1
101
Gemini 4 Carbon and Hydrogen enter the minecraft pvp benchmark leaderboard but lose hard to their brother Argon.
1
3
186
however, the Muse Ember takes the place of Muse Spark on the pareto frontier, showing a much stronger result at a very low cost. It actually pushed DeepSeek V4.1 Flash out of the pareto frontier for the first time.
I decided to recalculate the costs of Muse Spark 1.3 using regular pricing instead of a heavily discounted contributor pricing which I used to run cheaper testing. I believe it is more fair to compare models at their default-path prices. That change moved it much closer to Muse Wonder on the cost efficiency chart and it leaves the pareto frontier.
1
4
297
I decided to recalculate the costs of Muse Spark 1.3 using regular pricing instead of a heavily discounted contributor pricing which I used to run cheaper testing. I believe it is more fair to compare models at their default-path prices. That change moved it much closer to Muse Wonder on the cost efficiency chart and it leaves the pareto frontier.
1
1
337
gpt-6.1 terra gets ~same ELO on minecraft pvp benchmark as gpt-6.1 sol while being 25% cheaper to run
55
sell nvidia, buy meta
58
ai bubble is going to pop isnt it
BREAKING: Nvidia CEO Jensen Huang resigns
41
this is all very simple
29
sometimes you just want your agent to be an LLM - large LANGUAGE model
49
some abuse detection system can just block you randomly and you won't be able to do anything unless you're popular here or know someone directly openai, github, google - all can block you it's a good idea to prepare for these cases in advance so that it doesn't fuck up all your work and life do local backups, migrate to self-hosted solutions, add 2fa and recovery methods where possible
I dont even know what I did...not having access to my repo is teaching me github is a friction point and Ill be pondering what I want to do about this. Any ideas folks? Should I run my own repo on a VPS?
1
1
190
it just keeps working and it'll continue to work unless some retarded political shit gets in the way
A double exponential fits the data very well (R^2=0.99). Extrapolating that gives a predicted time horizon of ~80 work years by end of 2027. Extrapolating an exponential from post-Opus-4.5 gives ~30 work years. (up from Opus 5.5's AECI-implied TH of ~1 work-month)
28
told astra to use no external dependencies so it just wrote a font by hand
4
184
It seems that Daybreak Blue is practically useless. You only get "reduced refusals" on old and small models like Luna or 5.6-Sol
78
It’s crazy how outdated the average gamer’s opinion on Apple silicon is
61
three things in the past couple of months changed my mind about self-hosting local obliterated models at home. i used to be very skeptical since they're many months behind the frontier in capabilities and slow to run on personal hardware. but: 1. openai banned all my accounts in august for "cyber abuse". refusals became big pain in the ass in general. it'll get worse. 2. claude banned me for political reasons. it'll get worse. 3. third-tier api providers cannot be trusted due to data retention and tool-call injection risk. even if they're honest, someone will hack them. it'll get worse. i now have obliterated deepseek on my macbook, just in case i suddenly lose access to everything else. i recommend everyone to prepare for this too. atleast download the weights and figure out local inference so that you're ready.
Corporations lording over what you’re allowed to do with tools is not The Way. It’s like Photoshop stopping mid-painting and being, like, “I can’t let you draw that, Dave”. There will be endless political pressure on them to censor. Open local models are the only solution.
2
1
183
i still think you should keep using openai/anthropic models for most stuff, simply because they're much better. but having a backup plan is important nowadays, especially if you aren't US citizen or if you're doing anything that is in the gray area of their terms of usage.
23