Emirhan Erkan (@permaximum88) header
Emirhan Erkan (@permaximum88) profile picture

Community Lead @arcprize. Built The Singularity Gate: The threshold for AI-driven scientific discovery. Owner @ https://teapawt.net/t.co/RDmI9S17QC

Thrace/Europe
Joined July 2009
You can see each model's per-field breakdown below. Fable 5.1 answered only 3 of the 14 Life Sciences tasks, so it has no score in that field.
1
11
When it comes to all tasks, there is noticeable progress. You can see it with each release. Cost-wise, Fable 5.2 writes fewer output tokens than Fable 5.1 but reads more context, so at equal prices it would cost about the same. The cut in API prices makes it about 20% cheaper.
1
20
I suspect that past a certain point, a bigger model doesn't do better in truly out-of-distribution areas as long as the base model behind the public models stays the same. On the 12 Hard tasks, Opus 5.5 and Fable 5.2 are level, and so are Sol and Astra.
1
36
Claude Fable 5.2 is a major improvement on the Singularity Gate! It scores 25.8% while using 28.8% of a week's limit on a $200 plan. However, the models seem to hit a wall on the 12 Hard tasks, the most paradigm-breaking discoveries. Five models score between 7.6% and 8.0%.
1
1
1
207
I tried to figure out when Anthropic can reach fully-automated RSI with no human in the loop. Previously Anthropic put out numbers for how much of their R&D work AI did, and METR just measured how fast it's speeding up. Fitting the two (and a bit more from various sources) together I get a median of December 2027 for full automation, with a 50% chance it lands between July 2027 and September 2028. That assumes they don't slow things down on purpose. You can read the full research note below. singularitygate.org/research…
1
2
706
I'm proud to announce that the Singularity Gate is now open source. Everything behind v2.0 is at github.com/deathes/singulari…. It has the 80 discoveries with their first public traces and abstracts, every scored answer from Claude Opus 5.5, Fable 5.1, Sonnet 5.5, Haiku 5.5, GPT-6.1 Sol, GPT-6 Astra and GPT-6.1 Terra, the blind judge prompts with their label keys, every judgment by Opus 5 and GPT-5.6 Sol, the token counts and costs, and the code. A single script recomputes every score from the raw data. Each task asks for a paradigm-breaking discovery that first became public in July 2026 or later, after the training cutoff of every model we test. Claude Opus 5.5 leads at 18.9% and Fable 5.1 follows at 16.2%. The best GPT models, Sol and Astra, sit just above 11%, and no model reaches 10% on the 12 hardest tasks. The gate hasn't been opened yet. Finding the discoveries was the hard part. We screened 1,484 candidate discoveries to get those 80, and most of the rejects had already leaked before the cutoff in an earlier preprint or a conference abstract. From now on the benchmark grows all the time, and I'd love your help with it. Anyone can propose a new discovery, rate how paradigm-breaking it is, audit it for contamination, or write and audit its question. Every model on the leaderboard answers the same tasks, and tasks only retire for contamination. We'll run the tests in Claude Code and Codex at first, at xhigh effort, and add more models and effort settings as we go. The leaderboard is at singularitygate.org.
1
1
2
388
I'll fully open-source the Singularity Gate. Hopefully tomorrow. All the tasks, harness, judging pipeline, every answer and score. I hope we can build the next versions together and keep it up to date with more models and effort settings (v2.0 is xhigh only). #GPT #Claude #Opus
1
2
97
Subscription usage, API Price, and full leaderboard details can be found in the image below:
1
17