Zoë Hitzig retweeted
David’s right.
I regret helping spread the idea of “iterative deployment” which maybe briefly made sense in the GPT-3 era but makes no sense at all after many deaths have been tied to AI and as we’re careening towards extinction-level risks.
The industry needs to mature ASAP.
.@dgrobinson, who worked on OpenAI’s safety team, resigned from the company this week. “I believe we need to look deeper than specific rules or new laws. We need to talk about culture,” he writes: theatlantic.com/technology/2…
Zoë Hitzig retweeted
Incredibly cool research from my colleague @zhitzig et al. Although it's focused on a fascinating market, it contains lessons for governance, too: it's hard to get agents to understand their humans' preferences. This creates slippage in the market outcomes that these economic agents were able to achieve for their humans, and suggests we'd see similar slippage if we asked agents to carry out governance tasks for people, too.
There's a huge open space to study different ways of getting agents to elicit the preferences their human, to remember them, and to act consistent with them. Would love to see polisci work doing this for political agents---trying out different ways of having agents survey their humans about politics, then having the agents carry out political tasks, and then have the humans evaluate them for their fidelity.
➕➕ New research ➕➕
On the design of agents for markets and markets for agents.
AI agents are increasingly entering markets, acting on behalf of people—
How do we make sure they represent their people well, and that these markets are set up safely and fairly for everyone?
Some policy takeaways:
1 / For agent design: We need protocols for checking how well an agent understands the person it is acting for.
2 / For market design: We need rules about who can enter, who can see what, and what happens when a deal fails.
Link: anthropic.com/research/proje…
So much more research and policy to work on here... !
Very grateful for the many convos and papers that informed this project—with people like @ghadfield @sethlazar @erikbryn @sjgadler @divyasiddarth @andrewjkoh @alexolegimas @johnjhorton @AndreyFradkin @annieliang, and more!
I wrote this poem several years ago when I first heard the coil whine of a GPU
"a voice so thin / and distant / as to be almost / human"
Zoë Hitzig retweeted
Gillian and I wrote this chapter in mid-2025 but the technology wasn't quite there yet so we were forced to do some 'sci fi economics'.
We now have enormously powerful AI agents but there are still many open questions. Here's a few:
Great to see this out! In addition to a lot of great work from others, @andrewjkoh and I have a chapter in here on “An Economy of AI Agents” asking what fundamental questions of economic theory (like: what happens to the boundary of the firm if governance costs become AI alignment costs?) need rethinking.
Zoë Hitzig retweeted
This July, we gathered economists and tech industry leaders to talk about what "success" means when it comes to transformative AI, and how we can guide policy to get us there.
@erikbryn, @alexolegimas, Fabien Curto Millet and @zhitzig here, with more in the full YT video
Zoë Hitzig retweeted
.@_sholtodouglas and @DarioAmodei are right to be con concerned about AI driving an increase in the concentration of economic power.
@zhitzig and I write about the forces toward and against centralization in "AI's Use of Knowledge in Society," nber.org/books-and-chapters/…
I discuss some alternative approaches in The Turing Trap: digitaleconomy.stanford.edu/…
Completely false.
I like Gavin's takes, but whoever he heard this from is lying so that it fits the narrative some people so desperately want you to believe.
The same people will try to convince you Anthropic has no moat, and a sentence later that it might become so powerful it could be the only company left.
In fact, one of the things we are _most_ worried about is economic concentration of power. There is no world where the government should let any company have that much influence. We need competition and capitalism. darioamodei.com/essay/the-ad…
The AI market is literally the most competetive market in the world right now - every single one of the largest companies on earth is singularly focused on getting you smarter, cheaper models. If it all works out, we'll succeed in reducing the cost of everything to the cost of energy. This is awesome, but it threatens a lot of people's old moats. They are frightened.
For sure with AGI capitalism gets _super_ weird and what a company even is might look different. Good takes here: dwarkesh.com/p/ai-firm.
New paper headed to ICML!
LLMs love to contradict themselves.
We construct a new measure of inconsistency and study its connection to real-world failures––
(Led by the talented @MMancoridis!)
Zoë Hitzig retweeted
In agentic pipelines, one LLM can play many roles: writer, reviewer, judge, editor, and more. Does it use the same understanding of a concept across them?
In our new ICML paper, we measure conceptual consistency and test whether more consistent models make fewer mistakes.
We studied seven months of Claude Code usage —
for now, success on coding tasks depends little on whether you have a coding background …
1/
Zoë Hitzig retweeted
How might agentic coding tools, like Claude Code, alter the returns to expertise?
In this report, we find evidence that domain expertise leads to better outcomes: more successful sessions and better recoveries after Claude struggles.
