mechinterp, alignment & safety @flockcap, part-time independent journalist opinions my own 💍 @computerphile
Seattle / San Francisco
Joined March 2020
- Tweets11.3K
- Following1.1K
- Followers23K
- Likes32.3K
Pinned Tweet
we made sparse autoencoders compete with tiny softmax classifiers and the classifiers won on bits 🐐
a pure discrete code (no coefficients!) beats a top-k SAE at matched bits on SONAR, and an 800-bit hard code recovers 97.6% of gpt-2's loss
paper out now! (link in replies)
fun demo! this runs on the ontologizer work we’ve been doing.
sparse autoencoders can’t make this chart. they give you a bag of 16k latents per person. an ontologizer gives every attendee a discrete code word, which is why you get a typology instead of a feature dump.
At @WakaruLabs, we needed to decide who to prioritize talking to at EA Global. I used a truebase's residual stream activations and our library to make a color-coded ontologizer of conference attendees
if you want the boring version with 77 pages of ablations:
flock.holdings/research/onto…
celeste retweeted
At @WakaruLabs, we needed to decide who to prioritize talking to at EA Global. I used a truebase's residual stream activations and our library to make a color-coded ontologizer of conference attendees
celeste retweeted
Temple
She loves Factorio
Crack for autistic systems thinkers and engineers hyperoptimizers
Eventually she figured out in the lategame that the ultimate bottleneck is your computer’s memory and processing power to “hold state” on the entire map and gigabase, and you’d have to optimize for that
Every bottleneck you solve…creates the next bottleneck somewhere else
celeste retweeted
CLAUDE FOUND THE SEEDS
anthropic.com/research/ecdsa…
celeste retweeted
I never ~announced~ this, but for the past few months I've been working at @midjourney, learning a ton about operations from @aalleexx
Alex organized the Ghost in the Shell screening at Alamo Drafthouse on Monday evening, so I got to pitch in with planning the event
it was gratifying how much people appreciated seeing this beautiful philosophical movie on the big screen! I took the liberty of turning your reactions into a video. I hope you like it
quotes from @majamediaco (especially!) @PrinceVogel @alder_riley @DavidSHolz @potterlab
photos from @snnneee @lishali88 @maxwong @natalieyeo @vmfunc @thedataroom
P.S. to be clear this is not Official Midjourney Content, just something I had fun putting together!
celeste retweeted
all those inaccurate portrayals of computers from hacker movies were just early
Were you thinking “I wish I could explore the OpenAI math repo like playing Breath of the Wild?”
mts.now/mathofthewild
celeste retweeted
I was hesitant to open source this out of concern that it could make it easier to build a neuralese CoT. Since OpenAI is set on building neuralese CoT models anyway, maybe I can at least encourage them to use something more interpretable than looped transformers.
(repo link)
celeste retweeted
If you are the only person with nukes, you take over the world. If your opponent is the only person with nukes, you die.
If you are the only person with artifical superintelligence, you die. If your opponent is the only person with artifical superintelligence, you die.
celeste retweeted
I support President Trump in proposing nuclear attacks on Los Angeles and San Diego and i urge him to include San Francisco as well . perhaps a chemical attack for Berkeley — something slow and painful
