distributed machine learning @ google | sometimes mathematician
Seattle
Joined September 2012
- Tweets992
- Following438
- Followers1.4K
- Likes1.4K
Zachary Charles retweeted
I've tweeted before about how architecture improvements usually improve the constant factor in loss scaling laws, keeping the same exponent. The scaling ladders in the Ripple-65B release from @RicursiveAI are a nice illustration of this at scale! 🧵
Zachary Charles retweeted
this is cool but it's extremely weird framing imo to call the stacks project "critical infra," that feels like it's importing intuitions from engineering that just don't apply here. if you deleted the stacks project tomorrow it's not like planes would fall out of the sky
Zachary Charles retweeted
Hi! this is Rohan Anil, I develop training algorithms among other things - my work trains various frontier models.
At CoreAuto, I was fascinated by the problem of training on compute islands, eventually in space training.
I hope you all enjoy this write up!🪐🛰️
coreauto.com/hitchhikers-gui…
Zachary Charles retweeted
For today's Sunday Review, we're reconnecting with a 2004 cult-classic video game whose postmodernist soundtrack doubles as a treatise on the meaning of life
pitchfork.com/reviews/albums…
Zachary Charles retweeted
Tinker’s Dimuon is Muon with delayed diagonal conditioning, Polar Express and a rank-dependent update norm. I recovered its update rule through the API, borrowing @mrtnm’s approach to StableReinforce. Here’s how 🧵
Dimuon is a new optimizer we built for LoRA fine-tuning. In our SFT tests, it often reaches lower loss in fewer steps than AdamW. We're still experimenting with Dimuon; you can Tinker with it today and share your feedback!
See our optimizers guide at tinker-docs.thinkingmachines…
Zachary Charles retweeted
I agree with this and adopted this stance unilaterally on a recent review. I encourage you to adopt it in your reviewing — it’s reasonable to write reviews that say “this is dense unreadable slop, my review isn’t detailed and I recommend rejection solely on that basis.”
Some thoughts on reviewing: spylab.ai/blog/reviewing/
First for @satml_conf where we trialed an early reject phase which cut submissions by ~40%.
Then more generally about how not everything deserves peer-review, and why we should reward writing things that people actually read.
Zachary Charles retweeted
People really have no idea what string theory is, most have never talked to a string theorist nor bother to read up on it. Its public perception is driven by remarkably confident statements constructing a bogeyman that has nothing to do with the actual research
Zachary Charles retweeted
If you want evidence that capitalism is a force for good, just visit a grocery store.
My ode to the humble American supermarket, for @WSJFreeEx
Zachary Charles retweeted
An idea: any paper where all creative mathematical contributions are due to an AI model should be positioned as "an exposition of a proof by Model X". That'd be intellectually honest. It would also add value, rather than DDoS-ing arXiv with daily conjectures. Example below.
