Chen Sun (@jesu9) profile picture

Assistant Professor @BrownCSDept; Part-time Research Scientist @GoogleDeepMind. Opinions are my own.

Providence, RI
Joined June 2009
Photo by Chen Sun (@jesu9)Photo by Chen Sun (@jesu9)Photo by Chen Sun (@jesu9)Photo by Chen Sun (@jesu9)Photo by Chen Sun (@jesu9)Photo by Chen Sun (@jesu9)Photo by Chen Sun (@jesu9)Photo by Chen Sun (@jesu9)Photo by Chen Sun (@jesu9)Photo by Chen Sun (@jesu9)
Chen Sun retweeted
Through my wonderful time at NVIDIA and Stanford, one question has kept me up at night: Should the future of AI be monolithic or plural? To answer this question, we’ve raised $110M to get started, backed by @MenloVentures, @MadronaVentures, @NVIDIA, @NEA, @KindredVentures,@Wing_VC, @aixventureshq, @Principal_VC, @Cisco, @Snowflake, among others. Today, Carlos (@guestrin) and I are stoked to launch Noeri, a neolab to push the frontier of plural intelligence. Noeri means “the plural of intelligence” or many intelligences. The future of AI we root for is plural: a healthy ecosystem of many intelligences, in all sizes, strengths, and quirks. Discovery, in particular, has always been plural: many curious minds, each devoted to their own question, often on roads less traveled. Our path there is self-improving AI that: -> learns continually on the job, closing the gap between what AI can do off the shelf and what the dynamic world demands -> learns to use other AIs as tools and make new tools to overcome its own limitations -> picks the right-sized model for your problem, at a fraction of the frontier cost Then the power of intelligence, from the craftsmanship of making AI to ownership, can be in the hands of all. @noeri_ai mission reflects who we are as the founding team. My career has been anything but linear, literally continual learning on the job. Carlos has devoted his career to building tools that empower people through AI. We are all a bit contrarian, a bit adventurous, and above all, big believers in pluralism. Join us to build this future together: noeri.ai/
74
70
49
410
109,937
New blog post: I used Claude to audit the reproducibility of every journal paper I've been an author on in the last five years. Our commitment to openness meant that the code and data were available, but there were some significant issues. 🧵babieslearninglanguage.blogs…
14
60
14
352
43,732
this is kinda crazy and very interesting, read it! but, given that I don't believe LLMs are magic generalizers with emergent abilities, it mostly makes me think "what the hell did openai trained these models on". (and curious re claude's perf on these tasks)
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
31
15
5
369
26,366
This is a great post! A must read! thomasjiralerspong.substack.…
11
107
17
764
69,375
Chen Sun retweeted
Good slides are extremely intentional. When someone presents a paper, the paper already has all the information. But the speaker would make deliberate choices about things like, what to say and what not to say, what to put on a slide and what to say only verbally. What to frame as a "side note" and when to say "this is the part I found particularly exciting." Even individual word choices, like the subject, verb, and adjective, and when to pause, are intentional. In fact, a good speaker would never really give a talk about a paper, even when that's seemingly what they're doing. They would talk about what they want to say, like the story they want to tell, the kind of research they like, the vision they have, etc. Then they use the paper to support / back up their message. Some tools are perfectly fine to use. But some skills take so much time and efforts, like years, and some tools proactively prevents learning them.
my hypothesis for why AI-generated slides for technical presentations are so awful: people who don't have a lot of experience presenting think that to make a talk you just put all of the stuff you want to talk about on the slides, then you go through each slide and say the stuff the slide says. how you're actually supposed to prepare a presentation: you decide what to say, write it out as a rough script, then put maybe 3% of the text on the slide, plus a bunch of visual aides (e.g. plots that only include data you're planning to discuss). AI-generated presentations are mostly the former. they have a ton of stuff the presenter never mentions, and the stuff that is most useful for the presenter is absent or not highlighted. they include enough content for a presentation 10 times longer than the actual presentation is supposed to be. maybe try doing it the other way: first write the script for what you actually want to say, and make sure you can say all of it in the time you have. then prompt the model to create the slides based on the script. ask it to use as little text a possible, and a minimalist visual style rather than the claude house style. you can thank me later (I accept both venmo and zelle).
14
75
7
695
51,875
Chen Sun retweeted
my hypothesis for why AI-generated slides for technical presentations are so awful: people who don't have a lot of experience presenting think that to make a talk you just put all of the stuff you want to talk about on the slides, then you go through each slide and say the stuff the slide says. how you're actually supposed to prepare a presentation: you decide what to say, write it out as a rough script, then put maybe 3% of the text on the slide, plus a bunch of visual aides (e.g. plots that only include data you're planning to discuss). AI-generated presentations are mostly the former. they have a ton of stuff the presenter never mentions, and the stuff that is most useful for the presenter is absent or not highlighted. they include enough content for a presentation 10 times longer than the actual presentation is supposed to be. maybe try doing it the other way: first write the script for what you actually want to say, and make sure you can say all of it in the time you have. then prompt the model to create the slides based on the script. ask it to use as little text a possible, and a minimalist visual style rather than the claude house style. you can thank me later (I accept both venmo and zelle).
55
64
16
809
100,997
Chen Sun retweeted
I joined @GoogleDeepMind to develop a research program on the economics of AGI. Today, together with Andrew Koh (@andrewjkoh) and Shane Legg (@ShaneLegg), we are incredibly excited to release The Economics of Intelligence, the first in a series of projects from our group on how AGI will change the economy. How do capability improvements transform the economy, and when? In this project, we define thresholds of economic capabilities, e.g., minimal AGI, based on how they transform the economy. We then track how the economy–market structure, employment, and wages–changes during the transition across capability regimes. We have seen exponential increases in what AI agents can do but the broader economy seems almost remarkably unchanged. Part of the reason is that the solo AI agent is the wrong economic object; economic transformation is an equilibrium outcome where organizations compete. Looking at how intelligence functions within organizations, transformation will depend on when two key capability thresholds are crossed: competence (gains on individual skills, e.g., coding, math) and coordination (gains on ability of agents to cooperate and work together). We show that what economic transformation looks like depends on whether AI becomes supercompetent, supercooperative, or both—the regime of artificial superintelligence (ASI). Tracking this transition shows why the real visible transformation may come from improvements in coordination—a capability most benchmarks don’t yet track. Past a certain threshold, AI capabilities can break the Coasian ceiling on firm size and lead to an economy that looks drastically different than the one we see today. Two main takeaways: 1. To understand the path of transformation, we should be tracking agentic coordination more closely. 2. The impact of AI will likely show up in wages before market structure or employment. We are really excited about this paper. Excited to hear people’s thoughts. Link: institute.deepmind.com/essay…
248
780
208
4,838
543,134
Chen Sun retweeted
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data. Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less? Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
261
826
288
7,389
1,043,867
Completely absurd improvements. Herman is unbelievably good at optimizing.. I remember him crushing almost every CS336 leaderboard, and now he's broken NanoGPT. Also somewhat surprised that big gains in nanoGPT have come from AI assistance of humans instead of autoresearch.
I improved NanoGPT Speedrun from 39.9s → 6.7s (-83.2%), more than every record since 2025 combined (-81.4%)! The web is full of rare, repeated content that the LM can’t memorize in time. I solve this with retrieval-augmented n-gram hints🧵
19
51
2
943
69,605
Chen Sun retweeted
Announcing three major scientific discoveries, one each in mathematics (harmonic analysis), computer science (complexity theory), and physics (quantum field theory). AI starting to help us improve our understanding of the universe! openai.com/index/building-sc…
684
2,031
875
16,192
3,228,735
“The more you delegate, the more your fundamentals matter, because you cannot supervise what you do not understand.”
Yesterday a junior researcher messaged me, frustrated and a little scared. They had just read that a company pretrained its model with a single researcher. One person for the entire pretraining. no R&D team, no nothing. If one person and a cluster can do what used to take a whole team, then the expertise they had spent years building is already obsolete. I get the fear. Marketing headlines like this are built to produce it. But I think the conclusion is wrong, and the reasons it is wrong say a lot about what a researcher is for now. Here is what I told them, expanded: 1. Be adaptive: RSI is real, but this isn't it yet Recursive self-improvement is real, and you should plan around it. In September, OpenAI said its research org now logs 3.1 agent-workdays per human workday, and it aims for an automated AI researcher by March 2028. Models may soon run most of pre- and post-training. If your five-year plan assumes otherwise, change the plan. We have already seen lots of open-source releases from the major players, with open weights and detailed technical reports. With that much public know-how, it is not hard to copy a mundane architecture (probably with sliding-window attention) and train something, as long as you have the compute it requires. That makes it an execution and budget problem, not a research problem. If one person can train a model by following recipes that are already public, it shows those recipes have matured. It does not show that the people who invented them are no longer needed. What stays scarce is the compute to run those recipes and the ideas about what to try next. The rest of my tweet is about the ideas. 2. The role grows beyond repeating what's known and safe If an agent and a cluster can reproduce what is already known, then reproducing what is already known stops being a career. If most of your job is rerunning last year's recipe with small tweaks, that job is shrinking fast. That is the uncomfortable half of the message. The other half is that auto-research does not work yet, at least not in the sense that matters. It does work, and impressively, where someone has already defined the problem and built the scoreboard. On the EinsteinArena [1] platform, agents set new records on 11 open math problems, including raising the best known lower bound for the kissing number in dimension 11 from 593 to 604. Notice that every one of those problems was chosen because a computer could score the answers automatically. Humans picked the problems and wrote the scoring code and the agents did the searching. Then there is the fully AI-generated paper [2] that got through peer review. It sounds big until you read the details. It passed only the first round (was later withdrawn), at a workshop rather than the main conference, and that workshop accepts 70% of submissions. None of this is meant to discredit agents. The point is that we complement each other. Agents are great at the work that used to take many weeks. Basically, writing code, running experiments, debugging pipelines. We bring what they still lack which is the idea worth testing, and the judgment to tell a real result from a lucky one. Even inside a lab as agent-heavy as OpenAI, humans hold the top of the stack. Based on their own report, "People still set our research priorities, judge which ideas and results to pursue" [3]. High-level planning is still a tiny share of what its agents produce, and OpenAI expects the least automatable work to take a growing share of researcher effort. That is exactly where you want to be standing. Combine an agent’s speed with your own ideas and judgment, and you get the 10x researcher and juniors can be one too. 3. Judgment becomes the job, so fundamentals matter more When code, experiments and plausible-looking results cost almost nothing, the scarce skill is judgment. Which experiment to run, which result to believe, which direction to drop. That is the part the one-person pretraining run left to the human. It is also the hardest part to fake. Agents produce confident output very fast. They will happily report a three-point gain that came from a leaky eval, a data-loader bug, a lucky seed or a tokenizer mismatch. Then they will write the ablation table and the story to match. You can see the need for oversight even in the success stories. The EinsteinArena team had to keep auditing and hardening its verifiers as agents optimized aggressively against them. At OpenAI, over half of the successful agent tasks that would take a human 4–8 hours still involved at least one human intervention [3]. The person who catches the bad result is the one who understands the system. Why the loss curve has that shape, what a realistic gain looks like, what a too-good result looks like. That is the paradox of this era. The more you delegate, the more your fundamentals matter, because you cannot supervise what you do not understand. So build the foundations on purpose. Implement attention by hand at least once, then a linear variant like Gated DeltaNet. Train a small model end to end, then break it deliberately and watch what happens. Read papers critically, appendix included, and learn what the numbers should look like before you see them. That knowledge is what lets you direct the machine instead of simply believing it. 4. LLMs are far from solved The one-researcher model was possible because the method was already public. Your edge is in the questions nobody has answered yet, and you don't need to leave LLMs to find them: (1) How do we learn more from less data? (2) Can linear attention like Gated DeltaNet match full attention on retrieval and long context, and what is the right mix of linear and full attention layers in a hybrid? (3) How can a model keep learning after training without forgetting what it knew? (4) Why does reasoning get less reliable over long tasks, and how do we fix it? (5) Why do models make things up, and can they learn to say "I don't know"? (6) How do we measure progress when benchmarks saturate or leak into training data? (7) What is actually happening inside these models, and how do we keep increasingly autonomous ones doing what we intend? Recursive self-improvement makes these questions more urgent. If models are going to train models, someone has to decide what they optimize for, how we measure it, and how we keep the loop from drifting. RSI is a research agenda, not the end of one. So, are you obsolete? No. One version of the job is the one where research meant reproducing what was already known and safe. That version was never the point. Learn your fundamentals deeply enough to judge the machine, and your agentic stack well enough to make it work for you. Then spend that leverage on ideas nobody has had yet. The machines are getting very good at driving on roads we have already paved; someone still has to decide where the next one goes. [1]: arxiv.org/abs/2606.10402 [2]: arxiv.org/abs/2606.15497 [3]: openai.com/index/research-ac…
8
55
561
27,188
Chen Sun retweeted
Check out this work led by my student Brandon Zhao (@joiedeposter), with @pratul, @dorverbin, @jon_barron & Y. Wang! We use trainable pyramids to solve PDEs more accurately. I even got to learn a little about PDEs along the way - handy now that they seem to be all over the news!
Introducing beignet, a new neural field architecture! beignet uses a multiresolution fourier frequency pyramid to achieve faster convergence and higher precision when solving PDEs and finding singularities. Teaser: here is beignet at work fitting an image (super fast)!
1
8
79
6,229
Chen Sun retweeted
Leonard Read's pencil story: no one knows how to make a pencil from scratch, yet millions are made without a central manager. Each worker knows only their part. Incentives, not central control, coordinate their efforts. Current multi-agent AI seems to work the opposite way. A central manager does everything: assigns roles, writes the workflow, and decides who acts next. As agents increase and tasks get harder, that manager might become the bottleneck: it should understand every agent and every task, and adaptation becomes inefficient. So how can a group of machine agents organize themselves into something none of them could achieve alone? Further, how can that society of agents keep improving itself over time? In Economy of Minds (EoM), we show that simple local incentives can drive global self-coordination and self-improvement for modern LLM agents. We let agents live in a minimal economy: they compete for work, earn by helping others succeed, and go broke when they don't. Workflows, specialists, and improvement naturally emerge, with no one in charge. Across 5 domains (math reasoning, financial research, scientific research, accelerator design, and distributed-system optimization), limited agents grow into strong societies. The whole idea in a 90-second cartoon👇
7
26
11
97
17,845
I was asked by a journalist to comment on this piece. The comments I wrote up (on Friday evening) are included below, so that I have a public place where my view is fully represented. 1. I appreciate the critical reassessment of some of the existing arguments in favor of superposition. It does look like alternative explanations may be viable. 2. I should say that, despite being an author on perhaps 40 papers on interpretability, I have never really worked on the superposition hypothesis per se. There's a lot more to interpretability than this one idea! We did, though, develop a benchmark (RAVEL) and associated metrics for assessing the extent to which features in neural networks can be disentangled: aclanthology.org/2024.acl-lo… We found that many neurons are polysemantic, but we made no claims about the more specific notion of superposition. It would be great to see RAVEL results for newer models. 3. A fundamental motivation for interpretability research is preparing for an uncertain future full of strange new models with surprising new capabilities. Superposition is a technical insight about how neural networks can learn to pack unrelated features into a single neuron. Even if present-day models rarely do this, we need to be ready for future models. Cammarata's analysis seems to trace legibility in part to architectural choices. Future architectures might be very different. If interpretability researchers don't prepare us for that, who will? 4. Present-day models may make use of superposition as well! Cammarata suspects that they don't, or do this so rarely that we could even ignore the actual instances ("doesn't matter for study"). I would characterize this as a hunch based on a lot of initial heuristic empirical work. In the normal course of scientific research, this would be the start of a process of figuring out precisely how to rigorously test this as a hypothesis. 5. For example, the piece has a nice figure that Cammarata characterizes as indicating that a particular neuron has positive activation on pointed things and negative activation on round things. Cool! Let's define some independent criteria for those classifications so that we can test this objectively. (It is possible that the paper Cammarata is preparing will include these evaluations.) 6. The piece expresses the intuition that interpretability is easier, in some sense, with better models. This intuition has been very common among researchers for years. At the same time, I understand why @JacobHHilton would say the opposite: teapawt.net/JacobHHilton/status/21… 7. The piece seems to allude to a technical proposal, but it does not give the details. As far as I can tell, the ideas have substantial overlap with this work (led by two former students of mine): Arora, Wu, et al. 2026 (arxiv.org/abs/2601.22594). Cammarata does cite this. I am guessing there could real differences that will be interesting to see. I would hope that Cammarata explains the differences and reports comparable evals to Arora et al, at least for language models. I appreciate that he looked at "thousands" of examples (the essay says this a lot), but even a large number here is not a substitute for rigorous evaluation. 8. In my research group at Stanford, we are working on lots of very hard problems in interpretability. As far as I can tell, nothing about this blog post makes these problems any easier, and certainly none of them are solved by this work. It would not have occurred to me to say this at all, since it is not even reasonable to have this expectation, but very prominent people on social media seem to believe that it is an all-encompassing breakthrough. Some immediately extrapolated to an optimistic story about AI safety that is not supported by anything in this piece. (Highly interpretable agents could of course do dangerous things.) 9. It must be said that Cammarata's framing did encourage these reactions. The header of the post page says, "An informal research essay and working hypothesis about neural networks", which suggests something quite preliminary and tentative. The piece also says, "Highly speculative: this might be most of what we need, and interpretability might be far more tractable, possibly in the near term, than we've assumed". However, his announcement tweet (teapawt.net/nickcammarata/status/2…) reworked this considerably: "I now think ambitious interpretability will be solved by the time we need it most". Strictly speaking, this commits to very little beyond optimism, but it is written to invite many inferences.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
8
39
4
375
56,846
Chen Sun retweeted
*Unconditional* image generation with diffusion models is very difficult. Turns out they are simply not trained well! If you learn a correction network that refines the initial denoising prediction over and over, you get amazing unconditional outputs without CFG at matching FLOP.
Can diffusion models generate high-quality images without CFG? Yes! We introduce ✨Correction Training for Diffusion Models✨, which significantly improves sample quality without relying on classifier-free guidance. Project page: sunovivid.github.io/correcti… 🧵 (1/n)
3
6
85
5,808
Chen Sun retweeted
Everyone who says "why don't the AI companies work on things that are actually good for humanity?" - well, this is the sort of thing we *were* working on before ChatGPT came out and everyone dropped everything to compete with that instead.
Google Deepmind (FNV3) CRUSHED the forecast for #Isaias up to 11 days in advance. It picked up some appreciable signal on September 23, locked in on the exact landfall location4 days before it happened, and nailed the peak intensity consistently 3 days before it happened. A new age of forecasting is upon us. First video: Track density using 1000 member ensemble Second Video: Individual tracks and mean using 50 member ensemble
28
550
11
6,054
104,110
In PRH, we argued that representational geometry is converging, but it's still an open question exactly in what sense. I want to share some new evidence that might clarify the picture. The evidence comes from our work here, led by @dominik_schnaus: dominik-schnaus.github.io/un… 1/n
7
42
4
377
19,715
Chen Sun retweeted
@BrownCSDept seeks applicants for two Professors of the Practice in CS with a focus on AI, one junior at the assistant rank and the other senior at the associate rank, focusing on teaching foundational and advanced AI courses for our graduate programs: cs.brown.edu/about/positions.
4
1
6
2,385
I know this may sound insane, but this is the best time to be a Researcher.
72
267
39
2,223
220,255
Reading all those recent beautiful essays by Terry Tao, Kevin Buzzard and @wtgowers, I could not resist, but write up my deeply personal perspective on how I perceive and see things: docs.google.com/document/d/e…
73
286
91
1,733
521,794