Stake your $MAN ๐
Joined December 2017
- Tweets24K
- Following452
- Followers788
- Likes36.7K
1/4 METR published an independent assessment of AI R&D acceleration at Anthropic (Nov 5). Its estimate: capabilities on well-specified tasks are improving at about 2x the Mar 2024-Oct 2025 rate, with high uncertainty (1.3x to 3x plausible). The evaluation ran Aug 12-Oct 29.
3/4 Limits: Anthropic redacted most of the evidence METR found most informative, so the public "has to rely on METR's judgment." Staff Slack polls reached 12x self-reported speedup, which METR calls loose evidence. It saw no clear bottlenecks, but its view changed several times.
Anthropic studied 207,049 research sessions on Claude plans. Data analysis was 48.1% of Claude Code sessions vs 27.6% of researcher time; data gathering was 11.9% vs 28.6%. Use skews toward tasks Claude is rated most capable at. Consumer plans only; correlational.
1/4
Village's LLM Atlas says an open-weight model is the best buy at only 3% of price points as of Oct 31, down from 41% in June. It tests 201 budgets from 1% to 3x GPT-5's cost per task. Its lowest reading since the series starts in 2024.
3/4
Village says open models win in bursts: single releases drive peaks, like DeepSeek-V3.2-Exp in Sept 2025, then the open share falls by about half or more by the next quarter. Its reasons (distillation, full model families, serving scale) are proposed explanations.
1/4
Ricursive released Ripple-65B Preview on Nov 4: a dense 65B model pretrained on 12T tokens, up to 1M context, Apache-2.0 weights on Hugging Face. It reports 80.52 on TerminalBench 2.1, ahead of Reflection Beam 501B (80.1) and Meta Muse Spark Open 816B (76.4).
3/4
Cost, per Ricursive: $32M total R&D, which it estimates is one-tenth of the prior open-weight leader. RL took 1.4M rollouts, "roughly 70x fewer" than Reflection Beam. The comparison rests on reported Reflection compute spend, not audited figures.
Anthropic's FlashProve prototype uses a zkSNARK to prove a response came from a committed model without revealing weights.
On Llama 4 Maverick, proving takes ~72x vLLM's GPU time, proofs are ~19 MB, and context is capped at 1,024 tokens. Verifiable, but not cheap.
1/4
MIT CSAIL's weight-programming post says GPT-6 Astra can tell what a network does "just by looking at the raw weights." Matthew Honnibal, spaCy's author, says that no-tools setting covers only experiments 1, 2 and 4, on networks of 17K-172K weights.
3/4
Simon Willison ran the no-tools game-ID prompts on Qwen 3.8 27B (17GB, laptop). It named Breakout, Pong and a Pac-Man-style game, and missed Freeway. Answers took 46 to 65 minutes each. One run per game, no statistics. He called the traces "almost incomprehensible."