eric (@ericlim) header
eric (@ericlim) profile picture

applied @openai | mentoring, teaching @chapmanu 🇰🇷

California, USA
Joined October 2011
Photo by eric (@ericlim)Photo by eric (@ericlim)Photo by eric (@ericlim)Photo by eric (@ericlim)Photo by eric (@ericlim)Photo by eric (@ericlim)Photo by eric (@ericlim)Photo by eric (@ericlim)Photo by eric (@ericlim)
Pinned Tweet
Ready for DevDay
9
15
3
428
18,596
is star wars happening soon? SpaceXwing wen
8
484
misleading headline. robotics team is making insane progress. astra and our newer models show much better generalization and scaling on robotics than other approaches so we are leaning into that and working more closely with the rest of research. same program, roadmap, goals.
This quoted post is unavailable.
7
15
7
166
14,838
this guy is the GREATEST man to do it to work for his bags. what a bullpost. respect
This slide from the Anthropic IPO roadshow was a big factor in my decision to leave. It’s an insane plot and I’m surprised they made it public. It shows the compute efficiency of pretraining over time. The y-axis is a **log scale**: they are seeing continued superexponential algorithmic progress. Mythos was not a one-time deviation, but a point on a superexponential. They could already retrain Mythos with a fraction of the compute originally needed. It doesn’t strictly imply a software-only intelligence explosion- they have been spending more on R+D. But algorithmic improvements do not require ridiculous creative genius and are on the verge of being automated. You do the math.
5
20
12,121
I have a major result to share. I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China. This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model. Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves. I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems. I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used. It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20. These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs. I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo…. There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
1,812
6,395
2,185
41,097
6,621,351
(1/10) Today, we’re sharing new research from @OpenAI: How does CoT monitorability (no-CoT capability and CoT controllability) evolve over the model training lifecycle? We emerged with unsurprising but sobering findings: as we scale toward more intelligent systems, models are becoming harder to monitor through their CoT. 🧵
126
188
73
1,469
263,573
The Artificial Analysis Intelligence Index is currently made up of 9 benchmarks with arbitrary weights. I wanted to know how these weights varied over time. They vary quite a bit!
47
13
7
185
21,922
this app is free
Replying to @lulumeservey
There is another possibility, Lulu, one that dominates my group chats. Jensen is sleeping with all his customers. When they say the interactions were not unusual or unexpected, they mean he is a sex machine on a rampage, seducing thousands of AI hardware execs with a glance.
1
6
1,309
me watching computer use ordering me some mcdonalds on uber eats at 11:48pm
watching AI navigate the browser for me
4
11
834
We are now allocating a significant amount of our frontier compute towards discovering and patching vulnerabilities in open source codebases, fully pro bono. In one case, we studied the Linux kernel, which is a particularly load-bearing piece of software at OpenAI. Before AI, security researchers might discover ~20 local privilege escalation (LPE) vulnerabilities in Linux in a typical year. In one week's time, our upcoming frontier cyber model was able to find ~1000 LPE vulnerabilities using about $1M of tokens. In the process, it was able to discover at least 10 novel classes of Linux exploitation techniques.
Our security researchers built an autonomous cyber vulnerability discovery, exploitation, and patching pipeline. In a single run with an upcoming cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in the Linux kernel at a total API cost of $1.2 million.
105
128
48
1,393
207,085
Levent Alpoge is directly responsible for this
anthropic just stole my best friends research he spent the last 10 months developing zero knowledge proofs for AI accountability, only for @AnthropicAI to announce remarkably similar research days after he applied for compute support. Can someone at Anthropic at least let him collaborate instead of jacking his research and calling it their own? proof attached below
5
12
1
273
26,496
just ask chat
In many circles, “Ask Chat” has replaced “Google it,” and that’s reflected in a new announcement from OpenAI: the volume of search queries run through ChatGPT now equals 20% of the volume run through Google. theverge.com/ai-artificial-i…
1
3
403
Hi folks, I wrote half the materials science benchmark on this one. It tests the whole arc of the matsci workflow from candidate proposal to characterisation, and I'm bewildered by how performant the models already are at some of my most complex tasks, especially Astra.
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
24
19
8
385
33,539
this is one of the biggest reasons why i decided to study computer science
Kids today will never understand the power this little icon held back in 2014
25
72
2
1,527
40,599
eric retweeted
There are many futures. Which do you want to build?
1,008
827
614
11,757
2,836,560
Anthropic: We have the best model. Google:
7
2
177
21,004
we heard the violent feedback (and also heard OpenAI is so over for the 837492th time this year)
Day 25/ We are doubling all usage limits for all Pro 100/200/500 tiers and Business Premium, effective today. We heard your kind feedback and won’t be reducing usage limits for Pro 200, so your Work and Codex usage will stay the same. We’re also going to continue offering extended limits for dots for now, and usage of your dot will still continue to be separate from your default usage. And we'll of course also keep shipping better models and other efficiency improvements that increase your effective usage over time.
14
5
2
147
6,042
google benchmaxxed for their ai numbers on earnings and it doesn’t look good probably wont look good for a bit
1
4
654