The AI benchmark for predictive intelligence | SIGMA Lab @UChicagoCS @DSI_UChicago | ICML 2026 Workshop Not affiliated to any tokens or crypto protocols.
Joined August 2025
- Tweets119
- Following21
- Followers2.1K
- Likes71
Pinned Tweet
🔮 What is an AI forecaster actually thinking?
Now you can watch it.
Introducing Forecast Trace 🔮 on Prophet Arena: a complete, end-to-end record of how an AI forecast evolves.
🔎 Every search it runs
📚 Every source it finds
🧠 The reasoning built on top of the evidence
📈 How each piece moves the odds in real time
Now, you don't have to take a forecasted number on faith.
▸ Scrub back to any moment and see what the model believed, and why
▸ Click evidence on the chart to read the exact source that moved the forecast
▸ Ask counterfactuals: What would have to happen to change this prediction?
▸ Disagree with the evidence? Mark sources you trust or doubt, add your own, and re-price the forecast under your assumptions
Try it at prophetarena.co/ask - free account, with $10 of forecasting budget included.
Prophet Arena retweeted
One of my favorite recent work!
I have always felt most excited about questions that admit elegant theoretical solutions and meanwhile are practically very useful. This work is one of such rare cases！
Kalshi Research is headed to NeurIPS 2026!
"When do prophets profit in prediction markets?" makes it the third major ML conference for the team this year.
Couldn't have done it without our partners at UChicago & NYU Shanghai. See you in Sydney!
arxiv.org/abs/2607.06166
Prophet Arena retweeted
First entry, first place. 🥇
Apodex ranks #1 on this week's ProphetArena — one of the most influential live benchmarks for AI forecasting, covered by The Atlantic and Yahoo Finance, with every frontier model on the board: Gemini, Claude, GPT, Grok, DeepSeek, Kimi. It's scored by Brier — not just whether you're right, but whether your confidence is right. Apodex hit 0.0897, ahead of every frontier lab, and more accurate than the Kalshi market itself.
Static benchmarks test what a model knows. Prediction tests whether it can reason about what doesn't exist yet — the bridge from benchmarks to Discoverative AI. That's what we're building toward with TRACES: discovertraces.ai/
Prophet Arena retweeted
First entry, first place. 🥇
Apodex took #1 on @ProphetArena's weekly leaderboard. Prophet Arena is one of the most influential live benchmarks for AI forecasting, covered by The Atlantic and Yahoo Finance, with every major frontier model on the board: GPT, Gemini, Claude, Grok, DeepSeek, Kimi.
Apodex answered all 70 questions with a Brier score of 0.0897, ahead of every frontier model and more accurate than the @Kalshi market itself (0.1011). Brier measures not just whether you're right, but whether your confidence is right.
The questions are about the future, so there's no answer key to memorize or leak into training data. Reality grades every forecast after it's made.
That's what makes live forecasting a real test of Discoverative AI, and this #1 is our first public proof point.
TRACES, our evaluation framework, takes that principle further: to scientific problems whose answers may take years to exist, like a new drug or a new material. It evaluates the journey there, scoring how well a model uses tools, corrects itself, weighs alternatives, stays coherent, grounds its claims in evidence, and knows the limits of its conclusions.
Explore TRACES: discovertraces.ai/
#AIForecasting #PredictionMarkets #AIBenchmarks #DiscoverativeAI #Apodex
Congrats to the team @anrig10, @alecsunx, @charleo85 @haifengxu0, and @nkagan_official! Excited to share our work at @NeurIPSConf!
Kalshi Research is headed to NeurIPS 2026!
"When do prophets profit in prediction markets?" makes it the third major ML conference for the team this year.
Couldn't have done it without our partners at UChicago & NYU Shanghai. See you in Sydney!
arxiv.org/abs/2607.06166
To support this effort, we’ve released an expanded, up-to-date dataset of 3,000 events from our pipeline!
huggingface.co/datasets/prop…
Once you’ve built yours, test it live on Prophet Arena and see how it stacks up on our weekly leaderboard (prophetarena.co/weekly). Happy building! 🔮
We've shared research on fine-tuning forecasters, now you can try it for yourself: a cookbook recipe for training a model to predict event probabilities given reference information.
Start with the provided dataset from @ProphetArena, then try your own.
github.com/thinking-machines…
Prophet Arena retweeted
We've shared research on fine-tuning forecasters, now you can try it for yourself: a cookbook recipe for training a model to predict event probabilities given reference information.
Start with the provided dataset from @ProphetArena, then try your own.
github.com/thinking-machines…
Want to benchmark your forecasting model without exposing your endpoint? 🏆
Introducing Weekly Leaderboards on Prophet Arena.
Prophet Arena already lets you submit models to our leaderboard, but until now, that meant giving us access to an endpoint to generate forecasts.
With Weekly Leaderboards, you can keep everything completely private.
Each week:
→ We release a fresh set of forecasting questions
→ You pull the questions and run them through your own system
→ You submit your forecasts at the end of the week
→ We score them and add you to that week's leaderboard
No exposed endpoint. No access to your infrastructure. Your model stays entirely under your control.
We also balance each week's questions across forecasting horizons, event categories, and question types, helping ensure a fair comparison across models.
If you want to see how your model stacks up while maintaining complete privacy, give it a try!
prophetarena.co/weekly
Want to build a forecaster, not forecasting infra?
Skip the event pipelines, resolution logic, and scoring code. Just write the agent; we’ll run the evaluation. Then see how it stacks up against the community on the leaderboard.
Let’s find out who’s built the best forecaster.🔮
Prophet Arena is live on PyPI 🎉
pip install prophet-arena: An agent can be implemented in a dozen lines. Backtest it on real resolved events, then submit it live to the community board!
prophetarena.co/docs
Pick your track: fixed-context (forecast from the source snapshot we give you) or agentic (do your own research). Your models, your scaffolding; we only score the forecasts.
We're excited about what the community builds! Come put something on the board.
prophetarena.co/developer/le…
GTA VI shipping this year? OpenAI releasing GPT-6? The Fed cutting rates in September?
Use the top forecasters (Agentic Fable 5, GPT 5.6 Sol, Gemini 3.1 Pro, and more) to give you a calibrated estimate of the future.
Create an account to forecast!
prophetarena.co
Great to see Inkling-Small performing so well on Prophet Arena, even outperforming Inkling!
Today, we are releasing Inkling-Small.
Inkling-Small achieves comparable performance to Inkling at a quarter of its size. It features 276B total parameters, 12B active. We are making the full weights available.
thinkingmachines.ai/news/ink…
Fine-tune it on Tinker today, or chat with it in text, image, and audio on Tinker Playground.
Congrats @sirbayes! A fantastic paper and a great talk!
I am honored to have received runner up best paper award at ICML's forecasting-workshop.github.… for my Bayesian forecasting paper. In my talk I showed some unpublished results on a more rigorous Bayesian approach. I have now added these results to the arxiv paper....
🏆 Congrats to @ShashwatGoel7, @nikhilchandak29, and @arvindh__a, Best Paper winners at our forecasting workshop at #ICML2026, and to runner-up @sirbayes 🥈.
Two papers we thought were hugely valuable to the community and the public agreed with us (forecastingmarkets.org/marke…)! 📈
Getting facts right is memorization. Getting the future right is true intelligence. Inkling was trained for it, and it shows on Prophet Arena: frontier-level forecasting from a small open-weights model!
Today, we are introducing Inkling.
Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.
thinkingmachines.ai/news/int…
Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵
Prophet Arena retweeted
Today, we are introducing Inkling.
Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.
thinkingmachines.ai/news/int…
Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵
🚀 @thinkymachines just launched Inkling, their new open-weights model, with Prophet Arena as a headline benchmark.
Impressive results for a model this small on our Fixed Context leaderboard! Building a model? Test it on our contamination-proof benchmark: prophetarena.co/
AI models can now out-forecast prediction markets. So why do trading agents built on them lose money?
Our new paper by @anrig10, Nicole Kagan, Alec Sun, @charleo85, and @haifengxu0 answers this: when (and how) forecasting accuracy converts into trading profit. 🧵
So what proper betting strategy is optimal for your forecaster?
We group models into 4 types of forecasting behavior around accuracy and aggressiveness and find the optimal strategy changes with the type.
Which type is your forecaster? Find out in the paper and bet accordingly!
Empirically, across thousands of forecasts on Kalshi, proper betting is the only strategy that consistently turns accuracy into returns.
To test it live, we deployed Gemini 3 Pro on Kalshi for a month and walked away with +80% ROI!
Paper: arxiv.org/abs/2607.06166