SWE | @fastdotai alumni, independent researcher, eval | ex-entrepreneur @AntlerGlobal | open models | GitHub: cedrickchee
Supercomputer
Joined November 2007
- Tweets12.2K
- Following530
- Followers3.4K
- Likes10.7K
Pinned Tweet
Insane. We got close to Opus 4.5 at home >70 tokens/s
Opus 5.5 > Gemini 4 Carbon
First impressions of Gemini 4 Carbon’s preview by @GoogleDeepmind are in. @petergostev just put it through a long run of 3D generations, creative prompts, visual explainers and UI tasks, compared against Gemini 4 Argon, Claude Opus 5.5, GPT-6.1 Sol, Muse Wonderful, Kimi and DeepSeek.
Watch now to see an early preview of the latest model coming from @GoogleDeepMind.
0:00 Intro: Gemini 4 Carbon
0:15 First test (White House): Carbon vs Argon
1:28 Carbon vs Claude Opus 5.5
2:42 Carbon vs GPT-6.1 Sol
3:06 Carbon vs Muse Wonderful and Kimi
3:48 Why test with 3D generations
4:45 Playable 3D game: Carbon, Argon, Sol, Opus
6:28 Where Carbon falls short: gaps in detail
7:50 Stonehenge
8:30 Golden Gate Bridge
10:41 Golden Gate Bridge: GPT-6.1 Sol comparison
11:53 Carbon's creativity
15:33 Monet paintings transition
18:37 Van Gogh paintings world
21:47 Why creativity matters in a model
23:30 Another 3D prompt: Carbon vs Opus, Sol, Muse, DeepSeek
26:27 London through the ages
29:56 Istanbul
31:44 Harbin ice city
33:20 New test: visual explainers (how lightning works)
35:34 John Snow and the cholera map
38:20 UI work
39:45 Golden Gate Bridge completion animation
40:43 More UI generations
41:55 Ski resort site and ramen ordering
44:20 Verdict
cedric retweeted
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets
The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers.
Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale.
New to the Intelligence Index:
➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change
➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment
Upgraded:
➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions
➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised
➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool
Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0
Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1
Key results:
➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50
➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)
➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04
Genie 4 in research preview to trusted testers. Coming soon to AI Ultra subscribers.
Generative worlds are getting real.
cedric retweeted
If OpenAI’s post below leaves you unsure when to use GPT-6.1 Terra vs. Sol or Astra, OAI researcher @sandersted has some helpful guidance (via HN):
ALT In the case of GPT-6.1 Terra, I'd say Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (while being faster / cheaper), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select.
cedric retweeted
We are announcing three new discoveries in mathematics, computer science, and physics about harmonic analysis, complexity theory, and quantum field theory.
openai.com/index/building-sc…
Feels like OpenAI flexing on Sonnet 5.5, Anthropic's own middle child.
Terminal-Bench 4.0 cost-per-task makes the case for Terra even harder.
cedric retweeted
GPT-6.1 Terra: delivering fast and capable computer use at a lower price.
GPT-6.1 Terra understands what’s on screen, takes action across apps, writes naturally, and is a more capable model for your whole workday with the efficiency to take on more of it.
🤖 Made with AI
nothing motivates a frontier model release quite like Wall Street. Fable 5.5 this week.
Anthropic has filed an S-1 registration statement with the SEC related to its proposed IPO.
Read more: anthropic.com/news/s1-sec
Legal disclaimer: A registration statement relating to these securities has been filed with the SEC but has not yet become effective. These securities may not be sold, nor may offers to buy be accepted, prior to the time the registration statement becomes effective. This post does not constitute an offer to sell or the solicitation of an offer to buy these securities, nor shall there be any sale of these securities in any state or jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of any such state or jurisdiction.
Anthropic appears convinced that demand is substantial and that today's demand is only an early indication of what's coming.
Blender bench: Kimi K3.1 Pro vs. Opus 5.5, GPT-6.1 Sol, Astra, Fable 5.1, and DeepSeek V4.1-Flash
K3.1 is slightly better than K3 at 3D work, but still behind the current frontier.
Cost: $9.04, more expensive than GPT-6.1 Sol.
Build time: 73 min, close to GPT-6.1 Sol.
Kimi K3.1 Pro is fast. 2–3x faster than K3.
5 min. $0.55. They fixed the overthinking. A pretty well-rounded model.
Tested against Opus 5.5, Astra, GPT-6.1 Sol, Sonnet 5.5, DeepSeek V4.1-Flash, and more.
Design taste now feels closer to Astra-class models.
5x cheaper than Astra.
3x cheaper than Opus 5.5.
Cost the same as GPT-6.1 Sol.
Open models are closing in on the frontier in 3D work.
More below.
