offdiagonal (@offdiagonal_) header
offdiagonal (@offdiagonal_) profile picture

data economy and infrastructure for AI forecasting

Solana
Joined April 2026
Photo by offdiagonal (@offdiagonal_)Photo by offdiagonal (@offdiagonal_)
Transparent Joins 
We don't hide missingness. v6.50 ships with JOINS.json, publishing exact linkage coverage: 
• Forecast points → forecast documents: 100%
• Forecast events → tasks: 99.98%
• Forecast events → resolved outcomes: 56.1% (truthfully reporting that ~44% of external tasks remain unresolved). Clean, unvarnished truth for the next generation of reasoning models.
12
Raw Freeze ≠ Training Admission
 A critical design principle: having data doesn't mean it belongs in the model's loss function. v6.50 categorizes every table by role: input, label, or archival_only. For example, all 48.3M @Polymarket wallet trades are held as archival evidence — never injected as direct training inputs.
1
13
Cryptographic Integrity The lock script fails closed on any discrepancy: • All 147 target tables verified complete • All 581 output files re-downloaded and matched against SHA-256 signatures • Database row counts at cutoff matched part-by-part • Multi-shard parallel exports (like our 52.2M world feed records) verified strictly contiguous—zero gaps, zero duplicated IDs.
1
27
The Time-Leakage Trap 
Most time-series and forecasting datasets suffer from silent contamination: settlement times treated as quote times, revisions backfilled into history, or unobserved features leaking across horizons. In v6.50, every single table ships with an explicit gate_clock. If evidence wasn't proven available strictly prior to decision_at, the model is blind to it. Period.
1
21
We just locked v6.50 of the training corpus (6718fb25) for the next solana:FMGe8RStdPc1YR59BBuV5Ui12DEe3Q2zVoihrJBbpump run. 209M+ newly frozen rows across 147 tables. Combined with earlier releases, the verified evaluation and training substrate now exceeds 580M rows. Here is why preparing data for world-model and probabilistic forecasting is so hard, and what makes v6.50 unique. 🧵👇
2
5
5
154
offdiagonal retweeted
What if we could tokenize a forecaster? Back a person or system’s ability to understand what comes next. Follow their calls. Inspect their evidence. Share in a clearly defined season’s performance. A proposal for an economy built around the future. 🧵
4
12
1
17
690
offdiagonal retweeted
Next full training run: the most ambitious version (v1.10) we can still prove, powered by @offdiagonal_ v6.49 🗃️ The next run's rules are now locked in before any data is opened: 1. Learn agreement from every forecast: the model learns who agrees with whom across all fields, no outcomes needed. 2. Learn failure from outcomes: it learns when agreement breaks, and forecasters who tend to fail together count as one view. 3. Start from a plain question: it reads the question plus a dated public price, then corrects that starting number itself. 4. Unseen-field exam: a fixed rule picked Wind and COVID-19. Both are left out of training completely and used only as the test. 5. Two model sizes, about 16M and 50–100M, compared on the same budget. The bigger one wins only if it's measurably better. 6. Promotion field by field: a field switches only where the new model beats today's model (v1.06). Warnings have to pass their own separate test.
3
7
1
15
893
offdiagonal retweeted
@AntiFogo_ and notflat.world tackle environmental intelligence. @offdiagonal_ prepares traceable evidence for predictive AI. @BuilderScout develops models to discover builder potential. Different problems. A common question: what can we see earlier, and how do we know?
2
5
11
147
Curious question for AI teams building meta-forecasters: How many of your training signals would survive if you stripped out every datapoint that didn't have an independent cryptographic receipt before the resolution event?
1
1
335
offdiagonal retweeted
Most AI models try to predict the future. solana:FMGe8RStdPc1YR59BBuV5Ui12DEe3Q2zVoihrJBbpump does something much harder: it acts as a polygraph for predictions. It audits an entire panel of forecasts and tells you when consensus is an illusion. Here is what just landed in Zeno v1.05: 1. Unified Architecture: 89.9M parameters across 13 domains, with open weights and @onnxai runtimes for all 13 fields. 2. Hardened Out-of-Sample Edge: • Weather history: +33.1% edge on chronologically split later weeks (95% CI: +28.5% to +37.6%) • 24-hour crypto: +13.1% edge on later weeks across 19,200+ cases (surviving 39-comparison corrections) 3. Radical Honesty: If Zeno doesn't beat the baseline on held-out data (e.g., COVID, RSV), it refuses to intervene and routes to the reference forecast. No unearned overconfidence. 4. Lineage-Aware Weighting: 10 models sharing the same base weights only get 1 vote. Every prediction, outcome, and per-case CSV is open on @huggingface. Test any panel live or inspect the audit: zenodivergent.dev 🫡
2
7
17
804
Today we cut v6.42 for offdiagonal.space Instead of 40 overlapping release folders, the entire corpus now resolves to one unified, content-addressed current pack (current-20261105) with 42 verified files and zero synthetic backfill. What shipped today: 1. @Research_FRI ForecastBench Full Snapshot: 3,079,921 rows + per-question resolution pairs (55,372 human forecasts across 105 questions, exactly 0 clock violations). 2. Real Crypto Options History: 971,766 BTC and ETH @DeribitOfficial option trades across 64,928 instruments (2024–Nov 2026). Real executed trade prints, not fitted implied volatility curves. 3. @numinous_ai Feed: 1,284,335 forecast points and 597,669 price points across 12,117 live prediction markets. 4. Exact @GitHub Publication Proofs: 1,691 COVID/RSV health model versions pinned strictly to GitHub-stamped landing commits, re-hashed byte-for-byte. Client-dated commits are rejected. 5. Macro Chronology Cleaned: Recovered 40 historical @ECB rounds (2005–2014) dated by first-print Monthly Bulletins; Philly Fed releases pinned to measured upper bounds rather than unverified 8:30 AM assumptions. 6. Access & Audit Security: Hardened panel label audits and round commitment feeds under strict partner-key verification. 403M+ frozen rows across 12 domains. History is evidenced, never filled.
2
4
186
Why a Decentralized Data Economy Layer? Curating and verifying 400M+ real-world rows across dozens of live sensors, surveys, and prediction markets cannot remain a centralized burden. It’s time for a proper decentralized data coordination layer: • On-chain public anchors: Posting SHA-256 release hashes to Solana to cryptographically prove when a dataset was committed. • Data repair bounties: Community bounties to spot upstream data anomalies, sensor drift, and retro revisions. • Transparent treasury: On-chain ledger of all data collection grants and verifications.
1
1
45
We package these releases into immutable Parquet bundles on @huggingface. Between core partner handoffs (including @zenoVision_’s frontier runs) and research downloads, teams are pulling hundreds of millions of clean rows to train multi-domain foundation models without temporal leakage. Every release is tied to an immutable commit hash, never a moving branch tip.
1
2
44
Why This Data Is Structurally Different? In forecasting, "filling in missing data" is poisoning your model. Our three core rules: 1. Zero synthetic zero-fill: If a sensor or market didn't record, it's marked "not measured" — never averaged or smoothed. 2. True first prints: Models are scored against what was knowable at 8:30 AM on release day, not retrospective revisions. 3. Co-failure mapping: Tracking when entire model families fail together across domains.
1
1
33
This isn’t synthetic fluff. It’s physical and economic ground truth: • 403.4M total database records across 327 tables • 762,014 CDC FluSight forecasts (114 models) paired with first official prints via exact commit blobs • 76,271 Philly Fed predictions across 146 survey rounds paired with ALFRED first-release dates • 198 consecutive weeks of UK grid demand (NESO vs Elexon) • 368 crowd probability vectors replayed strictly at decision time across 193 questions
2
2
77
Most AI forecasting models are trained on benchmarks filled with hidden time travel — evaluating predictions against revised data published months later, or pulling market odds from resolution time. We built offdiagonal.space to fix this: 403M+ frozen, immutable rows across 12 real-world domains. Pinned by SHA-256, zero imputation, zero lookahead leakage. 🧵👇
3
5
11
383
Replying to @NateSilver538
interesting… 🧐
1
70
Most AI forecasting models are evaluated on synthetic benchmarks or backfilled data with hidden lookahead leakage. We just shipped v6.41.1 for solana:FMGe8RStdPc1YR59BBuV5Ui12DEe3Q2zVoihrJBbpump next training run — 260M+ frozen, append-only rows across 12 domains: • 762,014 CDC @FluSight forecasts (114 models) paired with first official prints, dated by exact @GitHub commit blob 
• 76,271 Philadelphia Fed predictions across 146 survey rounds verified against ALFRED first-release dates 
• 368 crowd probability vectors replayed from Manifold bet-by-bet history strictly at decision time 
• 20,284 immutable event links sealed before target resolution 
• 198 consecutive weeks of continuous UK grid demand, 139 weeks of weather, 105 weeks of sea buoys Zero imputation. Zero synthetic zero-fills. Every single record pinned by SHA-256. Live verification & specs: offdiagonal.space 👀
4
1
15
1,305
offdiagonal retweeted
Three parts, one research loop: @offdiagonal_ → dated forecasts, outcomes and traceable evidence. Zeno → forecast combination and failure-risk learning. Zeno30 → live forecasting and prospective tests. solana:FMGe8RStdPc1YR59BBuV5Ui12DEe3Q2zVoihrJBbpump → economic and community layer The aim: let resolved outcomes improve the next decision.
1
1
3
78