AI Institute Postdoc @HarvardHBS. @Stanford PhD. Generative agents, deep learning, behavioral data, enterprise AI, computing infra.
San Francisco, CA
Joined February 2019
- Tweets585
- Following1.4K
- Followers362
- Likes2.9K
1/ Goodfire's new essay "Neural Networks Are Already Clean" by Nick Cammarata estimates that roughly 90% of neurons in Qwen3-32B have one main job. The essay links neuron cleanliness to training choices. Quantization-aware training (QAT) offers a natural next test. 🧵
7/ The comparison also informs safety work, because interpretability audits usually examine full-precision weights while users often interact with compressed versions. A clean audit on one version deserves confirmation on the deployed version.
Sumer Vaid retweeted
PSA to everyone doing neuron interp after the recent monosemantic neuron posts: the model's training recipe matters a lot.
e.g. GPT2 seems to have been trained WITH dropout (modern LLMs are not), which seems to increase redundancy and (maybe) polysemanticity but reduce superposition (see sources in comments).
Other choices probably also have an effect (e.g. SwiGLU vs GELU MLPs, optimizer, amount of weight decay, etc.) and I think we should be careful not to generalize too much.
Currently working on a project to characterize the effects of these choices on neuron interpretability (a lot of the existing work is done on older/toy models which I'm not sure are representative), feel free to reach out if interested!
Sumer Vaid retweeted
I was looking at whether Apple Foundation Models could fit into Reflex and realized the interesting part isn’t that the model is local, it is that the model can belong to the operating system.
Reflex can match coding sessions by meaning. today, doing that locally still means carrying an inference runtime and downloading a separate model and that is basically how local AI works right now. Every app brings its own little AI stack.
macOS 27 changes that assumption. fm comes with the OS. apps and scripts can use the system model without setting up another model runtime or paying per token. If this pattern holds, a lot of software may stop shipping its own model at all.
The app asks the computer to understand text, extract structure or classify something, the same way it already asks the computer for networking or storage but there is a weird trade.
If the OS owns the model, an OS update can change how your app understands the exact same input even when your app didn’t change, so maybe software ends up shipping fewer model weights and a lot more tests for what “understood correctly” means.
The model becomes part of the computer. The app owns the behavior around it.
try reflex: npmjs.com/package/reflex-ski…
6/n System 2 agents persist across days and weeks. Instinct keeps a persistent cloud computer for each user. Town refreshes a wiki about each user every night. Those workloads depend on memory capacity, KV cache reuse and cheap storage for long-lived agent state (6/n)
5/n A system 1 model like Jev answers every question in a single parallel pass. Jev-style inference is largely prefill, the compute-bound phase of serving. Prefill rewards dense math units and large batches (5/n)
2/n Yes much has been said about Jev-like decision models being the system 1 of AI cognition but less has been said about persistent AI agents like Instinct / Town as the system 2
3/n My bet is that we are still in the very formative days of this bifurcation where we are only really seeing it play out at the model layer
4/n Listening to @neilmovva i cannot help but wonder if the system 1 / system 2 division will reverberate through the entire inference stack down to the chip memory coastline
1/n Quite interesting seeing deep learning / AI models bifurcate into system 1 and 2 thinking much like psychology did aeons ago
Tl:dr — there are tons of agent traces sitting on your laptop, we made a tool to automatically turn them into skills to give your agents “reflexes”. All on prem, fully open source, nothing ever leaves your computer. Take it out for a spin.
I use Claude Code, Codex and Cursor every day. The record of what it did is still on your disk.
~/.claude/projects
~/.codex/sessions
~/.cursor/projects/*/agent-transcripts
Every command the agent ran and whether it passed. Every correction you typed. Every token. None of them learn from it. The next session starts from zero.
I scanned my last 30 days: 94 sessions across 26 repos, sitting there unread.
So we built Reflex. A local CLI that reads that history and keeps only what has repeated across separate sessions:
- a skill for a task you keep redoing
- a handbook per repo: commands that work here, commands that fail here
- a memory block for CLAUDE.md
- a model pick per kind of task, from your own runs
Every line shows the sessions behind it. Token savings are measured before and after, or not shown. Nothing leaves your machine.
Built with @sumervaid1 and @gio_dano_ . Repo and install in the replies.