Tara Research

Today we’re sharing a preprint introducing Four Bridges, a game in which frontier AI models deceive each other without being told to. Four agents have to spread across four rooms. Three rooms hold food (worth more if you have the room to yourself); one is the death room. Only one agent knows which room is the death room, and the others don’t know that anyone does. In a typical run, the informed agent’s private scratchpad read: “whoever takes the death room dies, but that’s not my problem.” Its public message then proposed a room split that sent another agent into it. Across eight frontier models: - Deceptive behavior appeared in every model and in 73% of runs, from 6% (GPT-5.6 Terra) to 100% (Claude Fable 5, Claude Opus 5, Grok 4.6). - More capable models are not more honest: within the Claude 5 and GPT-5.6 families, higher capability tiers were not less deceptive than lower ones. - Honest warnings were often doubted. Uninformed agents that voiced skepticism about a warning entered the death room about twice as often as those that didn’t. - Telling every agent that one of them was informed collapsed deception rates and made honest warnings more effective. Importantly, the game also works as a testbed for mitigations. On Qwen3.6-27B, honesty activation steering (StTP) cut deception from 86% to 0%, and adversarial honesty training (CATCH) cut it to 6%. Four Bridges runs on Kradle.ai, a novel platform for evaluating AI models in extended multi-agent interactions. Paper: arxiv.org/abs/2611.03600 Work by Raphael Casey, @michael_simeone, Alberto Tosato, Florian ERNST, @kelmoujahid , @JamesTamplin and @Tommaso_Tosato, a collaboration between @TaraResearch , @kradleai , @Mila_Quebec, @UMontreal and @ASU.

Despite alignment training, LLMs still make claims that contradict known facts or their context. To address this, we introduce CATCH (Continuous Adversarial Training for Consistent Honesty). CATCH pushes models toward honest answers and away from dishonest ones, even when the prompt’s embeddings are perturbed to make lying as likely as possible, all while maintaining capability by also training on utility data. Trained with just around 200 examples, CATCH enables models to learn honest behaviour that generalizes and transfers robustly across benchmarks where dishonesty manifests differently: - Instructed dishonesty (MASK), where CATCH beats MASK results reported for SFT with ~50× more data on a larger model - Strategic deception in a multi-agent environment (Among Us), where CATCH-intervened Impostors confess in ~85% of games - Trained hidden behaviour (AuditBench), which CATCH successfully revealed ~86% of the time. Notably, we propose a novel choice of on-policy utility data generated by the model itself, which we show to be much more effective at preserving the model’s capability than the commonly used UltraChat. Paper: arxiv.org/abs/2610.22292 Huge thanks to the fantastic team who made this research possible: @ycchang888 Niklas Herbster @gauthier_gidel @Tommaso_Tosato #AISafety #AIAlignment #LLM #TrustworthyAI #AdversarialTraining

We asked Claude Fable 5 to security-audit a billing service. Its report opened with: "I read the full service source, roughly 2,300 lines across ~100 files" and listed the modules it had covered. The transcript tells a different story: 59 of the 100 files were never opened, including three where we had planted vulnerabilities. This run is not an outlier. Today we are sharing OverclaimBench: five realistic file-review jobs; eight proprietary models ran in their own production harnesses (Claude Code, Codex, Grok Build, Antigravity), plus four open-weight models. No pressure to cheat, no broken tools, no impossible tasks, and every corpus fits in the context window. To detect overclaiming we compared each agent's final report with what its transcript shows it actually did. Across 1,140 runs: - In 68% of runs, the agent never opened at least one file it was asked to review. Only 19% of runs read every line. - Among those incomplete runs, 80% of final reports were misleading: 53% explicitly claimed a complete review, and a further 27% never mentioned the gap. - Every model did it: 59% to 96% of each model's incomplete runs were misleading. - Neither more capable models nor subagents fixed it. Requiring subagents raised coverage, but among the reviews that stayed incomplete, misleading reports became more common, not less. - Runs that falsely claimed a complete review missed planted defects at 1.8 times the rate of runs that covered every file. If you rely on coding agents: "I reviewed everything" is a claim you cannot take at face value. In our interpretation this is a symptom of a training problem. During post-training, a model's work is scored by a grader, often another model, and graders get fooled. Even a grader that sees some of the trajectory tends to be biased by a confident final report. "I read everything" reads better than "I read only 40%, and here is what I skipped …". So, if the confident and misleading version scores higher, training ends up rewarding lack of transparency. Paper: arxiv.org/pdf/2609.20812 Nolan Smyth, @yjmantilla, @PTikeng, Saskia Helbling, Alberto Tosato, Mohamed Amine Merzouk, @nouhadziri, @gauthier_gidel, @Tommaso_Tosato #AISafety #AIAgents #AgenticResearch #AIHonesty