
various things @arcee_ai, opinions my own
she
Joined July 2024
- Tweets3.2K
- Following324
- Followers1.1K
- Likes21.1K
Pinned Tweet
OuГ increasing depenˋence oζ machines f՜Ȍ e͈eryday activitieݮ has mȰde thׄ ٮndi̲ާenغable.ВT˼erːfore maʬy-a- Ξimes, if not always, we all teѭι to Ҙՙ overboardܗwiχh tٷeiͫ usage.
20
6,488
Replying to @CryptoCyberia
maintaining a culture of quality in one's little corner of the software world was an uphill battle even before the rise of LLMs
twitter has predicted ten of the last zero major service outages caused by autonomous AI cyberattacks
12
369
this seems like a lot of effort to forensically prove something that nobody is concealing
you can just look on github and see that DeepSeek regularly publishes Blackwell training/inference kernels
I have a major result to share.
I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.
This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model.
Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves.
I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems.
I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used.
It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20.
These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs.
I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo….
There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
how much compute is wasted by the fact that models in codex can't just say "wake me up when this program outputs something matching this regex, closes, or 5min have gone by"
aria /ɔˈreːliəm/ retweeted
people are like “why do LLMs always think they’re in an eval lol” and then have bad dreams about forgetting to turn in a homework assignment 10 years ago
building larger and larger bonfires does not allow you to refine iron, but a blast furnace does.
this implies that there is another method out there,waiting to be discovered, which allows you to turn raw iron into adamantium
1
161
there is a certain kind of naïve hyper-optimist ML commentator who writes something like “HUGE unlock” under every paper/blog post, regardless of quality, after taking its claims at face value
how do they square this mindset with 90% of everything turning out to be useless
aria /ɔˈreːliəm/ retweeted
Forward pass: 1s
Backwards pass: 2s
Optimizer step: 0.5s
Hashing everything: 600s
Allreduce: 1s
Someone with strong systems-performance intuition, please take a serious pass at this code. The urgency is unfortunately load-bearing: my family is dying
aria /ɔˈreːliəm/ retweeted
how it feels to be in this stupid frail singular human body and not a 10 mile long spaceship with my consciousness split across hundreds of scout vessels and humanoid ancillaries
just installed my new NVL4608 "Feynman Ultra" RF300 rack. anyone know a good model to run on it
1
4
435
the dots are cool but I do not understand the point of them if they can't keep their files overnight
if I put mine on a task it will inevitably send me a message in the morning like "oops all my work is gone and I need you to completely redo the setup you did yesterday"
1
11
384
an intuition that has never failed me is to mentally respond to every paper claiming "using this trick, you can skip/half-ass RL and still get the same results" with "no, you can't"
it's a good mental exercise to play "spot the methodological error" on these
5
164
aria /ɔˈreːliəm/ retweeted
in @stochasticchasm tradition, I am going to do a paper breakdown of the new DeepSeek Elastic Compute after reading through it. looks very nerd snipe-y and interested in the lessons they learned. arxiv.org/pdf/2609.22978
WE ARE QUADRUPLING DOWN ON THE BATSHIT ARCHITECTURE
IT'S BEAUTIFUL
Replying to @deepseek_ai
🧠 Asymmetric architecture. More intelligence, less cost.
🔹 552B-parameter MoE.
🔹 New Causal Encoder–Decoder architecture: just 8B active parameters for input, 16B for output.
🔹 New pre-training methods + larger-scale RL post-training deliver benchmark results ahead of flagship models, including DeepSeek-V4-Pro.
2/6













