UNC Computer Science

🎉 Excited to share our #NeurIPS2026 paper, "Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge"! Latent Safety Signals in LLMs: Can we detect a harmful query before the model generates even one token? We show that the answer lies in the model's dark knowledge, i.e., the first-token probability distribution contains safety signals that transfer across different LLMs. We use this insight to build LADE (Latent Safety Signals for Defense), a model-agnostic defense against jailbreak attacks that requires no access to internal hidden states. 🛡️ Key takeaways: 1️⃣ We discover latent safety signals in the first-token probability distribution: contrasting harmful and benign queries reveals safety-discriminative tokens that transfer consistently across safety-aligned LLMs, despite different architectures, tokenizers, and refusal styles. 2️⃣ We introduce LADE 🔍: a lightweight framework that identifies harmful queries before generation begins by applying a simple kNN classifier to latent safety signals in the first-token probability distribution. 3️⃣ LADE also transfers across models 🔄. Across all reference–target LLM pairs, it achieves 96.6% average harmful-vs-benign classification accuracy. It also performs well against several jailbreak attacks while keeping over-refusal low.