@FOCAL_lab

Foundations of Cooperative AI Lab @CarnegieMellon. Creating foundations of game theory for advanced AI w/ focus on achieving cooperation. Directed by @conitzer.

 Pittsburgh, PA, USA
cs.cmu.edu/~focal/
Joined January 2022
  • Tweets141
  • Following1
  • Followers219
New positions at FOCAL! cs.cmu.edu/~focal/index.html
Vincent Conitzer@conitzer22 Nov 2023
We are recruiting postdocs at the Foundations of Cooperative AI Lab (@FOCAL_lab) at @CarnegieMellon (cmu.edu)! Please retweet / share / send great applicants our way! For different positions please reach out. @SCSatCMU @CSDatCMU @mldcmu apply.interfolio.com/136899
832
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerOct 21
Our new paper led by Ida Mattsson on the game-theoretic limits that an assistant faces when trying to be both helpful and not harmful, when it doesn't know who it is talking to. arxiv.org/abs/2610.27373

Fundamental Impossibility Results in AI Safety: Trading Off...

Is there a fundamental tradeoff between desiderata for safe AI systems, such as harmlessness and helpfulness? We answer by formalizing the interaction between AI systems and users as a game with...

arxiv.org
1
7
1
60
5,310
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerOct 20
Congratulations to Emin Berker, Ioannis Anagnostides, and @EmanuelTewolde for receiving a WINE 2026 Best Student Paper Award for our paper "What You Earn from What You Learn: Characterization and Complexity of the Value of Information"! arxiv.org/abs/2610.19488

What You Earn from What You Learn: Characterization and Complexity...

We examine the value of information in extensive-form decision problems: the largest multiplicative gain that a decision maker can obtain from an information refinement, taken over all payoff...

arxiv.org
3
27
1,793
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerOct 4
Congratulations to our 2026 Cooperative AI Research Fellow Akash Kundu @Akash190104 on his new role as an Evaluations Engineer at Apart Research @apartresearch! apartresearch.com/about

About

Meet the people behind Apart Research and learn how our independent research, evaluations and programs contribute to AI safety.

apartresearch.com
3
13
1,471
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Cooperative AI Foundation
@coop_aiSep 9
Designers of multi-agent systems need to know which infrastructure choices can support mutually beneficial outcomes between AI agents. Today's blogpost by @EmanuelTewolde introduces CoopEval, a framework for comparing cooperation mechanisms in practice, and outlines a broader research agenda for understanding when and why they work.
2
5
21
1,223
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerAug 17
Now on arXiv: our paper on whether LLM agents are more likely to cooperate when given various signals that the partner agent is similar -- led by @Akash190104 and @EmanuelTewolde. (Honorable Mention at the 2026 ICML AI4GOOD Workshop!) arxiv.org/abs/2608.12125

Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperation

As LLM-based agents with user-instructed goals are becoming widely deployed, they increasingly encounter each other in strategic interactions, and face challenges of finding mutually beneficial...

arxiv.org
3
7
610
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerJul 15
Two honorable mentions for papers at the ICML AI4GOOD workshop! Paper led by @EmanuelTewolde and Xiao Zhang: ‘CoopEval’ arxiv.org/abs/2604.15267 Paper led by @Akash190104 and @EmanuelTewolde: ‘Do LLMs Take Care of Their Own?’ openreview.net/forum?id=neTp…

CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM...

It is increasingly important that LLM agents interact effectively and safely with other goal-pursuing agents, yet, recent works report the opposite trend: LLMs with stronger reasoning capabilities...

arxiv.org
4
1
5
639
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerJul 10
The AI4Good workshop is today (Korea) @ ICML! @EmanuelTewolde is presenting the CoopEval paper, but also follow-up work with CAIRF fellow Akash Kundu about whether LLM agents cooperate with others that they perceive as similar. openreview.net/pdf?id=neTpZb… trustworthy-ai-for-good.gith…
2
4
497
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerJul 8
Today (Korea time) at ICML in the 5pm session, Vijay Keswani is presenting our position paper "We Need Practical AI Alignment Methods that Mirror Human Reasoning!" presentation: icml.cc/virtual/2026/poster/… paper: openreview.net/pdf/a895d4cfe…
2
10
839
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Emanuel Tewolde@EmanuelTewoldeJul 8
Standing by the CoopEval poster right now at #ICML2026! 📊👋 If you are interested in cooperation and multi-agent safety, game-theoretic mechanisms, or modern LLM agents defaulting to defection, come by board #4008 and let's chat! 🇰🇷🫱🏿‍🫲🏾
3
10
592
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerJul 5
Replying to @EmanuelTewolde
@EmanuelTewolde is presenting our CoopEval work at ICML on Wednesday 10:30am session (or catch him at the alignment workshop today)! presentation: icml.cc/virtual/2026/poster/… arXiv: arxiv.org/abs/2604.15267
3
6
205
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerJun 29
Congrats to our recent undergrad+MS grad Jerick Shi on receiving the best paper award at ICML NExT-Game for our paper "When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games" w/ Terry J. C. Zhang, Bernhard Schölkopf, @ZhijingJin! openreview.net/forum?id=v8nY…
1
4
488
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerJun 20
our recursive joint simulation paper (w/ Vojta and @C_Oesterheld) accepted to Synthese! TLDR: When players in a game run a simulation of themselves (incl. further subsimulations), that's equivalent to an infinitely repeated game and so allows cooperation. arxiv.org/abs/2402.08128

Recursive Joint Simulation in Games

Game-theoretic dynamics between AI agents could differ from traditional human-human interactions in various ways. One such difference is that it may be possible to accurately simulate an AI agent,...

arxiv.org
3
13
1,168
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerJun 13
One of my open math problems apparently got resolved by ChatGPT 5.5 Pro (Ryan O'Donnell prompted it better than I did!), though the proof was so hard for me to read that it seemed easier to just prove it myself. More thoughts on implications for math here: aifails.substack.com/p/even-…
10
24
4
177
23,690
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerJun 12
Replying to @conitzer @FOCAL_lab @EmanuelTewolde
arxiv.org/abs/2604.15267

CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM...

It is increasingly important that LLM agents interact effectively and safely with other goal-pursuing agents, yet, recent works report the opposite trend: LLMs with stronger reasoning capabilities...

arxiv.org
2
1
4
1,448
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerJun 12
Replying to @FOCAL_lab
@FOCAL_lab member @EmanuelTewolde is presenting his CoopEval work at EPFL on Monday (14:30) and it will be on zoom! (paper link in comment) memento.epfl.ch/event/ai-cen…
1
2
5
142
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Jiayuan Liu @ COLM 2026@jiayuan_liu_May 15
(1/4) Can remembering more of the past make AI agents less cooperative? In our new paper, we study LLM agents in repeated social dilemmas. The key variable is not how many rounds they play, but how much prior interaction history they can access when making each decision.
1
3
9
688
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Jiayuan Liu @ COLM 2026@jiayuan_liu_May 15
(2/4) Surprisingly, longer recall often degrades cooperation. Across 7 LLMs and 4 repeated social dilemma games, agents with longer histories often shift away from forward-looking cooperation and toward retrospective grievance-tracking.
1
2
3
563
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Jiayuan Liu @ COLM 2026@jiayuan_liu_May 15
(3/4) The mechanism is not just “too much context.” It is what the agents remember: replacing histories with synthetic cooperative records restores cooperation, and ablating explicit CoT reasoning often reduces the collapse. We call this the memory curse.
1
2
5
482
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Jiayuan Liu @ COLM 2026@jiayuan_liu_May 15
(4/4) Paper link: arxiv.org/abs/2605.08060 Big thanks to my collaborators! @Jack_Litq, @zzzoooeee321, Xin Luo, Haoxuan Zeng, @EmanuelTewolde, Tai Sing Lee, @tonghanwang, @ckingsford, @conitzer

The Memory Curse: How Expanded Recall Erodes Cooperative Intent in...

Context window expansion is often treated as a straightforward capability upgrade for LLMs, but we find it systematically fails in multi-agent social dilemmas. Across 7 LLMs and 4 games over 500...

arxiv.org
3
7
872
Foundations of Cooperative AI Lab (FOCAL) at CMU retweeted
Vincent Conitzer@conitzerMay 11
In this paper led by Jerick Shi @Jerick1380, we propose a taxonomy of LLM deception. arxiv.org/abs/2604.04788

From Sycophancy to Deception: A Unified Taxonomy for LLM...

Large language models (LLMs) could produce systematically misaligned output, from hallucinated citations to strategic deception of evaluators, yet these phenomena are studied by separate...

arxiv.org
1
2
8
1,272
Load more