Figuring out better plans @AI_Futures_ Co-author of AI 2040: Plan A and the AI Futures Model.
Joined July 2026
- Tweets36
- Following0
- Followers109
- Likes45
The elasticity of software progress to researcher capability is maybe the most important quantity in RSI forecasting. IMO TasteVal provides the best evidence yet on the component of this quantity coming from *selecting better experiments* using fixed compute and coding labor.
How fast is AI's research taste improving?
We find that the experimental research taste of frontier models has doubled every ~3 months since December 2025. The best model, Opus 5.5, now exceeds our expert human baseline. Our human experts are experienced researchers, but most haven’t worked at a frontier lab.
Why measure research taste? In the AI Futures Model, it largely determines how quickly artificial superintelligence is reached once coding is fully automated.
Hence, if the TasteVal doubling time applied to the situation where AIs are running the entire frontier lab, a "taste-only singularity" (or, an acceleration that's self sustaining solely on the basis of the research taste feedback loop) sole seems likely.
Dario writes "since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI."
But as of September 1, the AECI has not been "advancing drastically faster", so what does this mean?
to the extent that "ai has been advancing drastically faster" in research *outputs* rather than research *inputs*, it seems valuable to publish a chart that demonstrates that.
We updated the AI Futures Model to incorporate coding uplift trends and revenue trends into our timelines forecasts. Read about it here! blog.aifutures.org/p/q25-202…
Brendan Halstead retweeted
One of my favorite proposals in this piece is:
Limit capabilities of models used for AI R&D
It seems relatively simple to enforce to me, and has some nice properties:
> It delays the point at which AIs have the capability to sabotage AI research and align the next model to themselves.
> It likely introduces a “negative internal-public gap,” meaning that models are deployed to the public before being used for internal AI R&D. This is good for societal response.
> It has larger effects in worlds where takeoff is fast and therefore more risky. This also means that those who are skeptical of rapid recursive self-improvement may be open to this policy, because on their worldview it wouldn’t have as much of an effect.
This policy could optionally be combined with a minimum transparent safety allocation
Plan A is an ambitious international plan for AI, but it's also good to consider medium-ambition options for domestic regulation. Especially after the Pacing the Frontier employee letter! So, we brainstormed a bunch of options and articulated them here: blog.aifutures.org/p/how-to-…
Here are some concrete proposals for how the US could moderate the pace of RSI.
blog.aifutures.org/p/how-to-…
At the core of our mission is working through how to ensure increasingly powerful AI benefits everyone.
We believe that, at some point in the future, AI acceleration for frontier model development may be so high that the world will need to pace the rate of AI advancement.
We hope to contribute to work led by the U.S. government, alongside other labs and the open-source community, to develop the tools and mechanisms that could make that possible.
pacingthefrontier.com
