Research Director @coop_ai / Affiliate @GovAIOrg / Previously @FHIOxford @CompSciOxford @HertfordCollege
Oxford, England
Joined April 2011
- Tweets237
- Following1.6K
- Followers1.4K
- Likes3.1K
Great chatting with @labenz and @8teAPi yesterday on AI:AM! We covered multi-agent risks, hypotheses around the training setups that led to the HF/OAI incident, and the need for better info-sharing and reporting practices in a multi-agent world. Link below in case you missed it.
Beyond Safety Scores: Aligning Multi-Agent AI and Financing Its Compute
Join us this morning, Wednesday, 23rd September:
• 9:30 AM PT — Lewis Hammond, Research Director, Cooperative AI Foundation (@lrhammond)
• 10:15 AM PT — Wayne Nelms, Co-Founder, Ornn (@wayne_nelmz)
𝕏 Chat is Live on Screen, Come Join Us:
xitter.cf/i/broadcasts/1qxoNYLAN…
Great list of understudied and important research problems highlighted by @timrudner. Fill in the form – what are you waiting for?
We don't just need more evaluator organizations, we also need better methods for effective frontier AI oversight.
Fill out this form if you want to work together on improving scalable oversight for frontier AI: forms.gle/VpWyH6gTqqcMot7e9
Topics of interest (what's missing?):
- Formal verification and runtime verification for AI agents and multi-agent systems
- Scalable oversight for very large multi-agent systems
- Statistical guarantees for scalable oversight
- Mechanism design for frontier AI multi-agent systems
- Collusion in frontier AI multi-agent systems
- Emergent cooperative norms in frontier AI multi-agent systems
- Covert communication (steganography) in frontier AI multi-agent systems
- Oversight alternatives to CoT monitoring
- AI control
Lewis Hammond retweeted
Today's reporting of the second swarm of colluding AI agents seems like as good a time as any to bump next week's @coop_ai seminar. RSVPs are at 8x expectations with a week to go, for reasons that need no explanation. Tune in for a roadmap for governing multi-agent risk across the lifecycle, not just responding after the fact or once it's too late.
Grateful to @coop_ai and @lrhammond for beginning to lay the groundwork of the multi-agent risk agenda years before the moment demanded it.
Register: cooperativeai.com/seminars/w…
Read the joint memo with @iaseai and @brookings: cdn.prod.website-files.com/6…
What interventions are needed across the AI lifecycle to prevent and mitigate multi-agent risks? And which actors should be responsible? Join our online seminar with @IASEAIorg, presenting five action points: Evaluate, Identify, Monitor, Report, and Incentivise.
Lewis Hammond retweeted
New blogpost: ‘Lessons from Recent Multi-Agent Safety Incidents’. From OpenAI and Hugging Face to the GTG-1002 Claude espionage campaign, recent safety incidents have been notably multi-agent. In today’s blogpost, our Research Strategist @wlanderson0 shares what these incidents imply for multi-agent safety and what challenges we will face moving forward.
Lewis Hammond retweeted
1/Recent security incidents including the OAI/HF attack have shown how important it is to understand how agents will behave in complex, multi-agent deployment settings. Backed by @coeff_giving, @coop_ai, and @RethinkPriors I'm building a new team to focus on exactly this problem.
Lewis Hammond retweeted
🌊🏛️ AI-driven surges in citizen demand are straining government services. In a new paper w/ @lrhammond and @_achan96_, we call these surges “agentic flooding”, and ask: how widespread are they, how big is the risk, and what could be done about them?
Lewis Hammond retweeted
Help shape the future of AI safety!
1/ The 🇪🇺 EU AI Office Safety Unit is hiring up to 30 people:
💻 Technical Specialists
🏛️Governance specialists
⚖️ Legal and paralegal Officers
⚙️ Operations Specialists
Bring your expertise to frontier AI governance! Deadline September 8
Lewis Hammond retweeted
Can AI agents learn what you think and represent you in a debate you never attended? In today’s blogpost, @Jolow99 and @Oscarduys share their thought process and learnings after building Habermolt: the first public platform for AI-delegated deliberation.
Cool new framework for multi-agent evals (built on top of Inspect by @AISecurityInst) 🤖🤖 Great work by @BenHagag20 and @wlanderson0!
Today we're releasing Orbit v0: a framework for multi-agent safety and security research.
If you've tried running multi-agent experiments, you know the drill - weeks of infra before your first result. Built on Inspect, Orbit lets you swap environments, threat models, and topologies without rebuilding anything.
Good thread from Ryan on how the term "cooperative AI" can be confusing. Having spent a *lot* of time over the past few years explaining this topic, here are some thoughts that hopefully clarify things a bit! (1/11)
If this is a question that seems interesting, you can find out more at @coop_ai / cooperativeai.org. If you're new to the area there's also a course (cooperativeai.org/curriculum), and I'll be releasing a new research agenda soon that should clarify things further. (10/11)
Finally, lots of these points aren't original. I don't claim to represent them here, but my thinking on this has benefitted a lot from @AllanDafoe, Jesse Clifton, @ghadfield, @conitzer, @RichardMCNgo, @AndrewCritchPhD, @audreyt, @sebkrier, @ThoreG, @IvanVendrov, etc. etc. (11/11)