🚨 One week left to submit your AI-gov-related research to the TAIGR workshop. @taig_icml

🚨The 2025 AI Agent Index is out! 🚨 Amidst recent buzz over 🦀 and @NIST’s new agent initiative, we find: - Selective reporting – esp. on safety - Almost all agents backend just 3 model families - Many agents don’t ID themselves as bots online - Big US/China gaps - And more…

🚨New paper🚨 From a technical perspective, safeguarding open-weight model safety is AI safety in hard mode. But there's still a lot of progress to be made. Our new paper covers 16 open problems. 🧵🧵🧵

🚨New paper led by @aribak02 Lots of prior research has assumed that LLMs have stable preferences, align with coherent principles, or can be steered to represent specific worldviews. No ❌, no ❌, and definitely no ❌. We need to be careful not to anthropomorphize LLMs too much.

🚨 📣🧵 Introducing LAT-Chat -- abhayesian.com/lat-chat We now have an interface to help red-team our Llama3-8b model trained using latent adversarial training. Happy jailbreak hunting! If you find a novel attack, consider sharing it via our form!

🚨New paper: Targeted LAT Improves Robustness to Persistent Harmful Behaviors in LLMs ✅ Improved jailbreak robustness (incl. beating R2D2 with 35x less compute) ✅ Backdoor removal (i.e. solving the “sleeper agent” problem) ✅ Improved unlearning (incl. re-learning robustness)

📢 Announcing the #SaTML2024 CNN Interpretability Competition results: 🥇 *New record* - Yun et al. - RFLA-Gen2 🥈 Moore et al. - FEUD 🥉 Nicolson - TextCAVs 👏 Tagade and Rumbelow - Prototype Gen Report: arxiv.org/abs/2404.02949 All were impressive!🧵 on new innovations below.

🚨 New paper: Defending Against Unforeseen Failure Modes with Latent Adversarial Training We argue that LAT can be a key tool for safer AI because it can help address the gap between failure modes that developers identify 🎯 and ones they miss 🤔. arxiv.org/abs/2403.05030

🚨New paper🚨 8 Methods to Evaluate Robust Unlearning in LLMs Unlearning is promising for safer LLMs, but it’s tricky to evaluate. Here, we (1) overview eval techniques, (2) red-team a popular method, and (3) show that ad-hoc evals can be misleading. arxiv.org/abs/2402.16835

🚨New paper🚨 Black-Box Access is Insufficient for Rigorous AI Audits AI audits are increasingly seen as key for governing powerful AI systems. But to be effective, audits need to be high-quality, and to produce high-quality audits, auditors need access.🧵 arxiv.org/abs/2401.14446

🧵📣Do LLMs "lie"? In a new EMNLP paper, we study what happens when what language models say seems to disagree with internal representations of truthfulness. The story is a little nuanced. Thanks to coauthors Kevin Liu, @dhadfieldmenell, and @jacobandreas arxiv.org/abs/2312.03729