CEO @sistemaFutura building social world models. Behavioral Scientist, Author (https://teapawt.net/t.co/g3ZQaVRZVa), father, Quaker.
Costa Rica (mostly)
Joined July 2012
- Tweets1.3K
- Following528
- Followers2.8K
- Likes930
Something odd is happening in AI forecasting.
As of July 2026 on ForecastBench, AI has caught up with the best of human forecasters (superforecasters) in predictions on ordinary questions.
But it's not uniform. In a key area, the "best" models are actually getting worse.
teapawt.net/Research_FRI/status/20…
🏅 On the ForecastBench tournament leaderboard, several models are now statistically indistinguishable from superforecaster-level accuracy
The top model is from @Cassi_on_X, with models from @spacexai and @GoogleDeepMind also ranked as indistinguishable from superforecasters.
Cassi-2026-05-10 is just 0.3 Brier Index points behind superforecasters—a difference that corresponds roughly to 1.5 months of model progress.
A major bank asked us at sistemaFutura: how would AI reshape their business over the next several years?
There's no dataset of "AI transforming the banking industry" to fit a model to -- it hasn't happened yet.
What they needed was (quantitative) a foresight analysis, not forecasting from a fitted model -- and that's true for many decisions worth modeling.
AI has made building and calibrating these models almost free. The modeler's tacit judgment -- which used to quietly create trust in stakeholders -- is thin now, so an audit trail is essential.
AI also promotes a known failure mode: confident, articulate, and wrong causal structure, at volume.
I've written it up here sistemafutura.substack.com/p…
as an invitation to the AI, systems modeling, and behavioral science folks all circling this problem (social world models, predicting human behavior at scale).
We can't check these models against a future that hasn't happened, so we'd better get very good at checking everything else. And we don't have to start from scratch.
Last week I said we'd crossed into AI, and that it's messy. Here's a first piece of what I meant.
World models are having a moment -- LeCun, DeepMind's Genie, Fei-Fei Li on spatial intelligence. The bet is that real intelligence has to predict consequences, not just continue text. I think that's right.
But all of them are modeling the physical world.
I won't oversell it. This isn't a better crystal ball -- understanding systems is hard, and nobody should trust a confident number for its own sake. It's a different tool, which provides counterfactuals that can be audited, not just interpreted.
So this is less an argument than an invitation: if your looking at social systems + AI (world models, ABMs, system dynamics or LLM social simulation) -- where does this break? Who else is working on this intersection?
I think we've got a lot to learn from each other.
sistemafutura.substack.com/p…
Here's a direct link to the longer version if you're interested: sistemafutura.substack.com/p…