First, a bit more detail about the evaluation in this paper. Given an input X on which a model displays odd behavior Y, we ask questions like "If the input had instead been X', would the model have still done Y?" We study whether interpretability tools that read model activations (but don't edit them) make it easier to answer these questions. We find no uplift from any of the tools studied (SAEs, activation oracles, and NLAs). My overall take is: * This is a fair testbed where I thought interp tools could plausibly provide uplift. Seeing that they didn't was a negative update on the usefulness and reliability of the tools studied. * That said, this setting is not the place where I was most optimistic that interp tools would provide value. So it's not a direct test of (IMO) the strongest case for interp being useful. In more detail, I think the cases where interp is most likely to provide value are those where it's difficult or intractable to understand something about the model based solely on its input/output behavior. For example: 1. The model refrained from substantially harming someone because it knew it was in an alignment evaluation (but it didn't explicitly verbalize this suspicion). Ideally you'd be able to address this by varying the level of realism in the evaluation scenario and measuring rates of harmful behavior. However, in practice it can be extremely difficult to make realistic-looking evaluation that still provide opportunities for high-stakes harm. (E.g. imagine trying to make a realistic-seeming alignment eval that gives a model an opportunity to kill a human—seems hard!) I'll note that studying sensitivity of harmful behavior to eval awareness is one of the primary use cases of interp in Anthropic system cards. 2. The model dishonestly provided responses which are very subtly flawed. Here I'm imagining responses which are "beyond our kin" such that it's intractable to notice the flaws ourselves, but such that the model knows the response is flawed (such that interp tools could pick up on this knowledge in principle). I've intentionally chosen these settings to be very "unfair" to behavioral methods(=methods the only rely on model inputs/outputs). It's these settings—settings where it's structurally difficult to apply behavioral methods—where I think interp could provide uplift. I discuss my views on this topic in more detail here: lesswrong.com/posts/DqaoPNqh… In the evaluation in our paper, it's—by construction—possible to perform well using behavioral methods. Namely, the ground truth is constructed by testing counterfactual behavior under prompt edits. This means that the behaviors are tied to immediate, editable features of the input. (Compare to the eval awareness setting I discussed above, where it's difficult to make prompt edits that make the scenario look "realistic.") This makes this evaluation more amenable to behavioral methods than the mainline settings where I'm most optimistic interp could be useful. But I did think it was plausible that interp could have helped here. In some examples from the eval, it's not obvious which feature of the input was decisive for the weird behavior. E.g. in this example xitter.cf/a_karvonen/status/2090…, it's really not obvious that list ordering would be decisive for the model's choice. You could imagine interpretability highlighting "this option is first in the list" feature as abnormally salient to the model, and thereby providing uplift on the eval. In fact, our qualitative results suggest this might sometimes happen. But insofar as it does, it was offset by interpretability tools also injecting misleading hypotheses and noise. So I think interp tools could have helped here but empirically didn't.