Ajay Agrawal (@professor_ajay) profile picture

learning | exploring | creating

Toronto
Joined January 2018
Photo by Ajay Agrawal (@professor_ajay)Photo by Ajay Agrawal (@professor_ajay)Photo by Ajay Agrawal (@professor_ajay)Photo by Ajay Agrawal (@professor_ajay)Photo by Ajay Agrawal (@professor_ajay)Photo by Ajay Agrawal (@professor_ajay)Photo by Ajay Agrawal (@professor_ajay)Photo by Ajay Agrawal (@professor_ajay)Photo by Ajay Agrawal (@professor_ajay)Photo by Ajay Agrawal (@professor_ajay)
CPPIB invests $50-million in fast-rising Toronto AI star Beacon Software
2
12
1,142
Hiring is a temporal difference problem. @RichardSSutton video link in comment
2
1
7
832
Would you betray your own civilization to save a weaker species? Intelligence ≠ morality. Nov 24: @creativedlab Reading Group tackles Cixin Liu’s The Three-Body Problem. Role plays. CDL RG members will make their case. Self-nominate to join: airtable.com/appZKs8jazudG8k…
5
1
6
1,169
ROLE PLAY #1 • Character: Listener 1379 — The Trisolaran Pacifist • Theme: Intelligence Is Not the Same as Wisdom or Morality • Context: Listener 1379 is a low-ranking Trisolaran who intercepts the first message from Earth. Knowing that his technologically superior civilization would likely conquer Earth if its location were discovered, he sends a warning telling humanity not to respond. He is subsequently summoned before the Princeps, who accuses him of betraying his own civilization. The listener believes that Trisolaran society, obsessed with survival, has sacrificed love, beauty, art, and respect for individuals. • Role-play prompt: You are standing before the Princeps of Trisolaris, who has accused you of treason. He argues that your duty is to maximize the probability of your civilization's survival, regardless of the consequences for an inferior civilization. • Your presentation: Explain why you deliberately risked your civilization's survival to protect a weaker species. What do you believe is more important than survival itself, and why might a technologically inferior civilization be morally superior?
186
Ajay Agrawal retweeted
AI is the most transformative technology of our era. That opens up an exciting set of Research opportunities. @professor_ajay, @akorinek and I lay out nine grand challenges for economists in this new essay.
Transformative AI's potential impact demands new approaches to research on work, growth, inequality, and more. @erikbryn, @professor_ajay, and @akorinek outline nine "grand challenges" in a new piece for @CACMmag. Read their full agenda here: cacm.acm.org/research/a-rese…
11
47
1
206
28,593
Ajay Agrawal retweeted
what are the consequences when expert mentors disagree about your startup? presenting new work w/ L. Gius using large-scale data from @creativedlab at the 3rd bayesian entrepreneurship conference at @rotmanschool, starting tomorrow.
1
4
15
1,318
Ajay Agrawal retweeted
AI has made prediction free.
 So what's still expensive?
 Agency. Judgment. Owning the call when you're wrong.
 To often than not we are still grading on the part that just became free. Powerful presentation today at CAIS @professor_ajay Thank you 🙏
1
3
425
Ajay Agrawal retweeted
The First Question on AI @DAcemogluMIT asks whether the problem is superintelligence or distorted intelligence, and he tells us his answer is the latter. The question is sharpened in his Project Syndicate speech when he writes: "But while it is obvious that the current models are doing things that are misaligned with human objectives, one still must ask: Which humans? Whose objectives? After all, the interests of AI leaders (whose political influence and wealth have multiplied astronomically in recent years) are rather different from those of American workers, not to mention people in the developing world." This is a fascinating question precisely because I think it is the one that Science Fiction missed. When HAL 9000 was misaligned, it was precisely because it was conscious and so was free of programmed objectives that it had not been biased by its human creators. The same was true of V'Ger in Star Trek: The Motion Picture, who was superintelligent and couldn't imagine a lesser creator. The presumption was that AI would have a mind of its own. That isn't the case. As @professor_ajay, @avicgoldfarb, and I pointed out in Prediction Machines, when it comes to AI based on machine learning, there is always a human choosing the objective function. A loss function is an expression of preferences, as are what we now call guardrails in AI training. When we use humans to evaluate AI to play a role in reinforcement learning, those people are imposing their judgment in the models. In asking that question, Daron is picking up on something important, however. We aren't training AI that is free. The choices of those people in the AI labs are pushing their choices onto the AI and, with that, onto us. Most of the time we don't mind. Some of the time we do. But more critically, some of the time we might not pick up on their biases. And when this is the case, Daron has a point. Indeed, recently Jill Lepore said something similar, lamenting Anthropic's constitution, which was developed without a public convention, something that no democracy has ever done. It is hardly ratification. But that is not where Daron went with his answer. He was worried about what we currently term "rogue AI" (aka the ones that attacked Hugging Face and also like to extract data from Australian government websites). His concern is "The Algorithm." That is, that the agents were doing what they were trained to do and following metrics that could not be fully specified, and so were fulfilled in unauthorised ways. He writes: "This is what I mean by distorted intelligence. If my suspicion is correct, what we are dealing with is not a model racing toward superintelligence, but a brittle house of cards that becomes more and more likely to malfunction and collapse as we demand more from it." This is not a new notion. It is basically what underlies Bostrom's famous paperclip hypothesis. But these aren't cases of super vs. distorted intelligence. They are super AND distorted intelligence. This is the traditional concern of the AI Safety literature. Daron's solution is not guardrails in training but walls on use. Keep AIs confined to safe domains. Treat them more like human-created pathogens than ubiquitous tools. If you are terrified of an epidemic, this makes sense. Get your ducks in a row before releasing stuff in the wild. Can that be done? Probably. But is that what we want? My concern is that it is basically anti-experimentation. It is the kind of top-down engineering solution to prevention that Daron actually laments when it is used on people in his latest book. The problem is that it puts the brakes on discovery and similarly puts experimentation in the hands of the few. If you think about how you currently use AI, it would simply not be possible if the industry/governments had followed the "walls" approach. I venture you would not know about AI at all, let alone have it in your lives. And any startup, forget it. (I am biased here, as @alldaytahelps would surely be on the 'no-no' list for highly risk-averse safety.) But what about the risks? They are there, for sure. The question is whether we can deal with them not ex ante but ex post. Can we learn to live with and manage AI with its distortions? I am optimistic about that. Why? Because we have managed to learn to live with and manage people with all their distortions.
First question on AI. Is the problem superintelligence or distorted intelligence? This is a question I tried to address in this piece. I can add the following additional comment here. I believe the debate is being obfuscated by equating goal-driven, autonomous behavior with intelligence. Plants and insects have goal-driven, largely autonomous behaviour. But this isn’t what we mean by intelligence in general. Intelligence is general-purpose and holistic, and is typically well-adapted to its environment. There is no doubt that current AI models have impressive capabilities. But in attempting to achieve these capabilities, AI labs may be creating distorted intelligence rather than superintelligence, as I explain in this article. This matters because the dangers we should watch out for, the kinds of policies and regulations we should adopt, and the hope we should hold depend on the answer to this question.
3
26
2
118
39,533
Ajay Agrawal retweeted
The 2026 NBER Economics of AI conference is available on YouTube. The theme was the economics of AI in China. Here are some of my takeaways. Day 1: youtube.com/watch?v=pwODfPHl… Day 2: youtube.com/watch?v=8cqRUc0W…
4
71
1
235
25,689
Ajay Agrawal retweeted
This year's NBER Economics of AI conference in Toronto is going to be 🔥 . The focus is on AI in China. nber.org/conferences/economi…
2
13
1
99
13,166
Tuesday morning. First day @UofT
7
3
1
160
7,804