Pinned Tweet
New blog post! In which I talk about dynamics in physics that amplify information, i.e. cause a small localized change to propagate to a larger scale. Example: a spark that sets off a forest fire. thermontology.com/posts/phys…
this video is the best 1-minute depiction I know of the appeal of science and math youtube.com/watch?v=BWZTlfrn…
I pretty much buy what nick says in his post about why superposed stuff would be hard to learn. OTOH there clearly IS superposed stuff in a weak sense at initialization -- the whole network is like a gaussian process.
maybe instead of generating a list of concepts, the goal of advanced interpretability should be automatically generating a *curriculum* which guides you to understanding how a model thinks, step by step.
our ability to understand the apparent function of neurons in image models depends on our painstakingly built-up model of the visual and physical world. understanding domains we don't already know will likely require a similar build up.
I like Danny Calegari's reaction to the OAI proof dump. if your field got obsoleted by AI maybe you're just too basic?
interstice retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
interstice retweeted
Replying to @billionsmustliv
A monk asked Joshu: does the dog have buddha-nature?
Joshu replied: 🙄🙄🙄🙄🙄🙄
it was then that the monk achieved enlightenment.
Replying to @lu_sichu
"yeah sucks about your fields getting destroyed. anyways what do you think of my zippers?"
interstice retweeted
We discovered a sophisticated recording device that was hidden inside one of our offices. We used our models to understand the device and where it had come from, identifying that a specific employee had planted it a few days earlier. We referred the matter to the FBI, and we’re grateful to the agents and prosecutors who moved quickly to make an arrest. The FBI’s investigation is ongoing and we will continue to support their work.
At this time, we have no indications of this individual inappropriately accessing or exfiltrating model weights, source code, or customer data. The employee was terminated.
We take protecting our technology, data, and people seriously, and as a matter of national importance. Our security program is designed to protect against a wide range of threat actors, from insider risks to cybercriminals to nation states, and this case is an example of how AI can help address these sophisticated security threats.









