
Professor @Wharton studying AI. New book, Co-Existence, is out now: https://teapawt.net/t.co/hsDYf1xU9K My Substack: https://teapawt.net/t.co/UIBhxu4bgq
Philadelphia, PA
Joined May 2009
- Tweets37.4K
- Following596
- Followers42K
- Likes7.4K
Pinned Tweet
My new book, Co-Existence, is out today. It is a guide to how to live & work in a world where AI is sometimes, but not always, better than you.
(And, for the release, I worked with Opus 5.5 to make a quite interesting experience for the book's website: co-existence.ai/)
It is a very funny time where the markets are basically assuming the future will look like the past even as many CEOs & AI observers & government officials keep yelling that the future is coming and it will change everything.
Efficient markets versus radical discontinuities?
A lot of folks don't realize how much of our computer infrastructure is an improvised system prone to cascading failures. You didn't need offensive AI cyberattacks to have big outages like Microsoft/FAA. Of course, now we DO have to deal with offensive AI (but also defensive AIs)
I say this as I see every outage of the last five days get picked up by people who did not pay attention to cybersecurity before, usually with the implication that "this is the big one," which, in turn, leads people on both X and real life to panic and spread further rumors.
OpenAI’s Dot is capable, but seems to be falling further behind in its interface compared to Muse and Duet. The single-threaded conversation, lack of inline generative flourishes (like maps), & the inability to see what tasks it is assigned makes it harder to use than the others.
I think contrary to many of our expectations, we have not seen a Fable class open model more than 8 months after Fable/Mythos was first made public.
Alibaba has launched Qwen4 Max, gaining 9 points over Qwen3.8 Max on the Artificial Analysis Intelligence Index and landing 3 points behind Kimi K3.1 Pro at ~11% lower Cost per Task
Qwen4 Max, using xhigh reasoning effort, scores 43 on the Intelligence Index, 9 points ahead of Qwen3.8 Max (xhigh) and 3 points behind Kimi K3.1 Pro (max, 46)
Key takeaways:
➤ Qwen4 Max improves on every evaluation in the Index vs Qwen3.8 Max (xhigh, 34): HLE-Diamond +21 points (24% to 45%), AA-Omniscience +18 points, AutomationBench-AA +17 points, CritPt v2 +13 points, Terminal-Bench-Science +11 points, Terminal-Bench 4.0 +9 points, GDP.pdf +3 points, FrontierCode 2.0 +3 points and AA-Briefcase +82 Elo.
➤ Qwen4 Max costs $3.27 per Intelligence Index task, ~3x less than Qwen3.8 Max ($10.02) and ~11% less than Kimi K3.1 Pro ($3.68): This is driven by 25% lower per-token pricing than Qwen3.8 Max, 70% cheaper cache hits and lower token usage. Tencent Hy4 costs ~3x less per task ($1.03) than Qwen4 Max but scores 8 points lower on the Index.
➤ Qwen4 Max uses 39% fewer output tokens per task than Qwen3.8 Max: ~105k output tokens per Intelligence Index task vs ~171k for Qwen3.8 Max, ~72k for Kimi K3.1 Pro and ~78k for Tencent Hy4. Qwen4 Max uses fewer tokens, while increasing 9 points on the Intelligence Index
➤ Improvements in AA-Briefcase (knowledge work): On AA-Briefcase, our evaluation of agentic work across real business files, Qwen4 Max reaches 1686 Elo vs 1674 for Kimi K3.1 Pro and 1446 for Tencent Hy4.
➤ Qwen4 Max trails Kimi K3.1 Pro on Terminal-Bench 4.0 and Tencent Hy4 on FrontierCode 2.0: Qwen4 Max scores 48% on Terminal-Bench 4.0 vs 67% for Kimi K3.1 Pro and 37% for Tencent Hy4. On FrontierCode 2.0, @cognition’s private evaluation of real-world coding tasks, it scores 26%, level with Kimi K3.1 Pro (26%) and behind Tencent Hy4 (31%).
Key model details:
➤ Context window: 1M tokens
➤ Multimodal: Text and image input with text output
➤ Pricing: $1.50/$4.50 per 1M input/output tokens on Alibaba's first-party APIs, with a $0.075 cache hit price. This is lower than Qwen3.8 Max across the board ($2.00/$6.00, with a $0.25 cache hit price)
➤ Availability: @Alibaba_Qwen 's first-party APIs
I think often about the fact that almost everyone working in the West has lived in a quite stable socio-technical period
Only 1 job out of 270 in the 1950 census is now obsolete (elevator operator)
2026 is similar to 1957 in a way that 1969 was not from 1900.
That may change.
But it also means that no leader of a company, government, or any other institution has ever had to deal with a radically different socio-technical regime. The old approaches that worked through the ups-and-downs of the previous 75 years may not work for the years ahead.
The paper I got the elevator operator stat from. Which doesn’t mean many jobs haven’t changed dramatically as a result of technology, of course. cepr.org/voxeu/columns/how-c…
A good frontier model can reason about the state of the house; point out anomalies, safety issues, or things that are wrong; and solve problems for you. They have increasingly good spatial understanding from images. All of this is the things you want.
I was watching the new Apple HomeView reviews and seeing that they hallucinate and fail to understand what's going on around them. Feels very bitter lessonable compared to whatever expert system they're trying to build.
Apple made big strides with Siri AI this year, but Siri still struggles with the basics. And nowhere is that more obvious than on the new HomeView. This clip from my review sums up the problem.
5
44
12,235
Another one of those things that would have seemed impossible a few months ago: "Fable & Opus, find video of the Starship catch. Render the last 40 seconds or so in 3D in 3js with extreme detail. Let me switch views. No need to include the crowd."
Play: ship-42-catch.netlify.app/
Full prompts:
Find video of the starship catch today. render the last 40 seconds or so in 3D in 3js with extreme detail. Let me switch vision from visual to cut-away blueprints to other useful views. the main view should be stunning, including mecha arms, the cloud deck, the rocket, etc. find images and video online to work with
make sure the images and video match exactly. launch a fable 5.2 high thread to polish when the opus 5.5 thread is done
lets not worry about the crowd, remove them from the cast. focus on getting the in-the-cloud scene through breakout looking better. there is also a bit of flashing/strobing on the ocean in some view
also host it synched to the webcast replay so you can see both at the same time. anmd make sure the exhaust/engine plume looks right
This is apparently all from the video of the webcast. When I asked for sources, Claude replied:
"what the recreation was built from:
SpaceX’s official webcast replay of Flight 15 (YouTube): seconds 74 to 116 drive the timing, camera shots and catch moment, and it’s the footage in the side-by-side and the comparison video.
Wikipedia’s “SpaceX Starship (spacecraft)” article: the ship’s layout for the blueprint view, including tanks, header tanks, downcomers, pressure bottles and engines.
Guinness World Records: the 146 m tower height.
Community cut-away drawings: which way the tank domes bulge.
Everything else is estimated, including heights along the hull, tank sizes, pipe routing and catch rail height. No SpaceX engineering drawings were used."
1
19
8,689
As a side note, a couple years ago, making this benchmark & analysis would’ve taken a considerable amount of time in STATA, likely several days, and making it look this nice would have been quite a while as well.
This was like 4 prompts to Fable 5.2
AI is getting better faster than our measurements are
I threw together SatBench, which measures how quickly major AI benchmarks become saturated (effectively beaten). Of 173 benchmarks cited by the Labs since 2023, 63% are now saturated. Saturation is happening faster each year.
Here: github.com/emollick/satbench
Also thanks to The Benchmark Almanac: github.com/brianlzhou/benchm…











