PhD Student @MIT (EECS & CSAIL & @nlp_mit) advised by @lateinteraction. Prev. @BrownUniversity
Cambridge, MA
Joined September 2020
- Tweets717
- Following244
- Followers3.8K
- Likes1.5K
Pinned Tweet
Continual learning is widely discussed right now, but mostly as improving on the job or avoiding catastrophic forgetting. But it has a different, difficult, and already urgent form:
Given nothing but a corpus of documents, how should AI systems develop expertise in a new, unfamiliar domain? We call this problem Machine Studying.
Jacob X. Li retweeted
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?
We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.
In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.
📝 xingjianbai.com/reactive-opt…
💻 Try it on your task: github.com/xingjian-bai/opal
🧵
someone I really admire said researchers should share more of the messy day-to-day instead of only the polished results
so in the spirit of open science, nothing worked today.
Jacob X. Li retweeted
Your world model may know less about the world than you think.
I've always wondered what world models really learn.
Humans don't memorize every trajectory. We learn rules we can reuse.
Can world models do the same?
We built AnyGoal to find out.
hazel-heejeong-nam.github.io…
Jacob X. Li retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Jacob X. Li retweeted
We’d like to condition agents on large text corpora, but most are poorly organized and/or poorly written. Can we train an agent to write well-indexed, LLM-native notes optimized for future agents?
We train a builder agent with RL to maximize the accuracy of a frozen answering agent that reads its workspace, rewarding each edit based acc improvement. Even a small, frozen 4B answerer far outperforms the strongest retrieval baselines on BrowseComp-Plus, and the builder transfers zero-shot to unseen BRIGHT documents.
Scaling this up points to a different kind of continual learning, where an agent adapts to a new domain by writing for its future self rather than updating weights. Co-led with @Anikait_Singh_ @teresaszhang
👀
Jacob X. Li retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Jacob X. Li retweeted
Surprising new results:
Giving tools to LLMs makes them less able to say NO!
We tested 11 multimodal LLMs (Claude, GPT, Gemini, ...). With tools, every one fails to refuse harmful requests more often, up to 68.7% more!
Details and possible reasons 🧵
To appear at NeurIPS’26.
Jacob X. Li retweeted
my talk @MIT NLP Seminar on Infinite test-time scaling & prefix sliding :)
we seem close to models that can work autonomously for weeks to solve very hard tasks
maybe the final boss: replace compaction
youtube.com/watch?v=CKdlVoMh…
More and more problems are 'self-solving'
e.g peer review seems broken, that 60k iclr submissions feels like a crisis but just give everyone the best LLM and the avg review quality might even goes up. Human reviewer quality was never 'uniformly' amazing, and the models are only getting better
so u see peer review is getting fixed whether we show up or not, and if we don't fix it ourselves AI pretty much sovles it for us, whether or not we like the solution
And this is kinda crazy cuz lots of institutions suddenly have a deadline
we need to slow down AI progress because I’m tired of changing the model name in my slides every week
Jacob X. Li retweeted
AI can increasingly make progress on open problems like Navier-Stokes. Yet defining a new problem still relies on human research taste.
Introducing ScholarCatalyst, a benchmark built from AI researchers’ firsthand accounts of what inspired their work.
Jacob X. Li retweeted
StudyLoop is going to NeurIPS :)
Accepted to two workshops:
• Verify-Agents
• RAAAI
Single author. Independent research.
Now I just gotta figure out how to get to Sydney and Atlanta 😭
Jacob X. Li retweeted
1/7 🧠 Do LLMs reason just as well when the context is no longer familiar?
Our paper "MemoReason: Evaluating the Effect of Parametric Memory on Contextual Reasoning in LLMs" has been accepted to the NeurIPS 2026 Workshop on Trusted AI Evaluation! 🎉
A thread 🧵
Jacob X. Li retweeted
Is Jev secretly learning a Value function?
A model can always choose “A” without being certain that “A” is correct. That distinction matters when an agent must decide whether to act fast or switch to deeper reasoning.
We spent some time exploring:
1. How Jev fits into System 1/2 decision-making and why calibration matters for knowing when to switch
2. Why maximising expected reward in RL does not guarantee calibrated probabilities.
3. How predicting eventual task success in agentic tasks connects to a familiar RL quantity: the Q-function.
Could value prediction help build a calibrated System 1—and could that be part of the story behind Jev?
Link: souradip-chakraborty.github.…
#Jev #RLCD #System1
Jacob X. Li retweeted
new preprint: language models are “insecure” reporters.
researchers and engineers have become increasingly reliant on LLM-written reports---second-hand accounts of code and experiments---to understand agent-generated work.
in this study, we ask: “how transparent are these LLM-generated reports?” 🔍
Jacob X. Li retweeted
🎉 Accepted at #NeurIPS2026!
🚨 Why does inference-aware fine-tuning often hurt single-shot accuracy?
Answer: Prompt Interference
TLDR: Pass@k optimization upweights harder prompts, potentially leading to gradient conflicts
Paper: arxiv.org/abs/2602.21189
🚨 New Insights 🚨
Why does optimizing LLMs for Pass@k often hurt Pass@1?
Answer: Prompt Interference.
Pass@k optimization upweights harder prompts, potentially leading to gradient conflicts that pull the model away from Pass@1
Paper: arxiv.org/abs/2602.21189
@anas__barakat
