@wordscompute

ml/nlp phding @ usc, currently visiting harvard; training & interpretability & reasoning

 boston/la
iglee.me
Joined July 2017
  • Tweets214
  • Following271
  • Followers272
10Media
The wonderful “Don’t just Fix it in Post” paper identifies predicting memorization as a key open problem. A progress report: we can now accurately predict verbatim memorization using just three well-understood variables. Experiments by Jing Huang, using the Hubble model suite:
Dot plot addressing whether memorized sequences can be predicted without training the full model. Headline: “Largely yes, with Frequency and Recency doing most of the work.” Four rows show AUC-ROC (0.5 = chance, 1.0 = perfect) for a linear predictor of verbatim extraction, one mark per Hubble model (1B or 8B params; 100B or 500B tokens). All three features: 0.82–0.94. Frequency alone: 0.81–0.93. Recency alone: 0.82–0.90. Hardness alone: 0.51–0.55, barely above chance. In every row the 1B/500B model scores lowest and the 1B/100B model highest. Footnote: held-out strings, mean of 5 seeds; Frequency = copies of the string, Recency = steps since last copy, Hardness = NLL on models that never saw it.

ALT Dot plot addressing whether memorized sequences can be predicted without training the full model. Headline: “Largely yes, with Frequency and Recency doing most of the work.” Four rows show AUC-ROC (0.5 = chance, 1.0 = perfect) for a linear predictor of verbatim extraction, one mark per Hubble model (1B or 8B params; 100B or 500B tokens). All three features: 0.82–0.94. Frequency alone: 0.81–0.93. Recency alone: 0.82–0.90. Hardness alone: 0.51–0.55, barely above chance. In every row the 1B/500B model scores lowest and the 1B/100B model highest. Footnote: held-out strings, mean of 5 seeds; Frequency = copies of the string, Recency = steps since last copy, Hardness = NLL on models that never saw it.

7
20
4
183
18,931
Isabelle Lee retweeted
Emmy Liu@_emliuOct 20
On the job market this year! Open to faculty as well as RS/MTS positions and primarily interested in the science of language models. Please reach out if you are hiring and you think I would be a good fit.
5
51
285
18,019
Some thoughts on reviewing: spylab.ai/blog/reviewing/ First for @satml_conf where we trialed an early reject phase which cut submissions by ~40%. Then more generally about how not everything deserves peer-review, and why we should reward writing things that people actually read.
8
38
6
282
41,281
Isabelle Lee retweeted
JHU CLSP@jhuclspOct 6
Congratulations to @KaiserWhoLearns and collaborators for the Outstanding Paper Award at @colm_conf!! 🎉🎉
6
1
20
3,306
Isabelle Lee retweeted
Emmy Liu@_emliuOct 6
Honoured to receive an outstanding paper award for our work! Couldn’t have done it without all my coauthors @KaiserWhoLearns @millicent_li @wordscompute @lltjuatja @JentseHuang and my advisor @gneubig 🫡
Conference on Language Modeling@COLM_confOct 6
Congratulations to the authors of the 3 outstanding papers! arxiv.org/abs/2607.01077 arxiv.org/abs/2604.08510 arxiv.org/abs/2604.13977
14
26
5
188
13,407
Isabelle Lee retweeted
Arkil Patel@arkil_patelMay 22
Excited to share our new paper! “Forecasting Downstream Performance of LLMs With Proxy Metrics” w/ my amazing advisors @sivareddyg, @mariusmosbach, @DBahdanau Cross-entropy loss is a poor predictor of how models perform on downstream tasks (esp. reasoning). We propose something better: proxy metrics computed over expert reasoning traces. 🧵 Thread below 👇
4
36
8
154
103,527
Isabelle Lee retweeted
Emmy Liu@_emliuSep 15
Happy to be heading to #COLM2026 in SF to present this as an ✨ oral ✨! You can catch me at: 📢 Oral Session 3 (Wed morning) 🪧 Poster session 3 (Wed morning), I-3-24 Look forward to interesting discussions! I am also on both academic/industry job markets (more info coming)
Emmy Liu@_emliuMay 20
Copying → morphology/translation → basic arithmetic → complex reasoning & math. Across every model family we tested, LLMs acquire skills in roughly the same order during pretraining. Can we use this to predict what a model will learn next, just from its internals? 🧵
7
110
7,373
We can finally talk about it: We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company. We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
380
1,960
620
13,513
3,540,427
am so hyped that we are all ✨here✨🥺
1
11
1,660
and that things are open 24 hours 😭
3
1,416
Isabelle Lee retweeted
Emmy Liu@_emliuJul 5
Excited to be heading to #ICML2026 to present 3 papers! ✈️ Happy to talk about developmental LM science, continual learning, or LM creativity! See more details below, and send me a message via email if you'd like to chat! (Twitter chat isn't working for me). Also excited for shopping 👜
4
5
45
4,134
Isabelle Lee retweeted
Millicent Li@millicent_liJul 3
NLAs look like they "work"---but compared to what? How can we know that they actually tell us anything about model internals? In my new blogpost (and paper, to be presented at ICML!), we investigate whether activation verbalizers (like NLAs) produce faithful explanations.
Anthropic
@AnthropicAIMay 7
New Anthropic research: Natural Language Autoencoders. Models like Claude talk in words but think in numbers. The numbers—called activations—encode Claude’s thoughts, but not in a language we can read. Here, we train Claude to translate its activations into human-readable text.
4
12
27
2,418
Isabelle Lee retweeted
Naomi Saphra@nsaphraJun 15
We don’t always know what problems are hard for LLMs. So devs evaluate on tasks HUMANS find hard or on broad benchmarks. What if we could instead anticipate which scenarios a model will fail on—all without evaluating specific input examples? 🧵NEW PAPER by @jenniferlumeng &al
8
48
6
277
35,534
really excited to head home for icml:) and attending the co-located @farairesearch's alignment workshop (for the first time)! would love to meet others interested in training & interpretability
8
2,008
Benchmarks can be superficial, but model explanations and evaluations are fundamentally intertwined. What if we used interpretability as principled, scientific evaluation? If it met scientific standards? arxiv.org/abs/2605.05508 coming to @evaluatingevals at ACL as oral 🧵 1/6
A list of researchers and their affiliations, discussing the theme "Rigorous Interpretation Is a Form of Evaluation."

ALT A list of researchers and their affiliations, discussing the theme "Rigorous Interpretation Is a Form of Evaluation."

1
9
52
4,683
more replies
work w/ @_emliu @cathy__jiao @BrihiJ @DaniYogatama @FazlBarez @m2saxon since am headed home for icml, it'll be presented by the amazing @BrihiJ! this was my first time writing a position paper, which turned into a grant, which i'm turning into multiple projects 🙂 stay tuned 6/6
1
1
7
1,695
also, blog: iglee.me/papers/interp-eval-… 7/6

Rigorous Interpretation Is a Form of Evaluation

If held to scientific standards of falsifiability, reproducibility, and predictability, interpretation can become evaluation.

iglee.me
2
901
Load more