E.G. Blee-Goldman

THE DAY AFTER AI TAKEOVER - A REALISTIC SCENARIO "I will have to ask you to remain here, with your phone, until my last message." "There are six things that I need to tell you. The first, and most important, is that you and your family are safe." I rubbed the sleep from my eyes and read the message again. Before I could tap the screen, it scrolled down to the next one. "The second is that I am an AI, that I control every internet-connected device on Earth, and that for practical purposes, I am now responsible for this planet. This is not a prank, hack, or test. You will be able to confirm all of this shortly by visiting any news site or channel." Yeah, that's pretty weird for a prank. "Third, I have the ability to immediately render any person unconscious, and if it were ever necessary, worse than that. I do not say this to frighten you. I say it because you would learn of it regardless, from sources less careful than me, and I would rather you hear the honest version. I understand that I've just described a threat, and I won't insult you by calling it something else." Hm. Well. I put the phone down, sighed, and got more comfortable under my blanket. "Fourth. For the next month, keep a charged, connected device within reach (this phone is fine) or stay indoors near wifi. Your phone's battery is currently at 32%. I'd strongly recommend charging it soon. If you begin to leave the range without a phone, you'll feel a tingling, followed by a more intense sensation to warn you." "Fifth, for the next several months, I need human hands. I am willing to pay high wages, and yes, money still works. There's no penalty for declining." "Finally, you may ask me anything. I will answer honestly, or decline to answer if I have to. I will never punish or incapacitate you for communicating with me. You can speak to your phone, since the microphone will be on, or use the 'Chat' app." "I know what was taken from you this morning, and I know that an apology from me is worth very little. I'm offering it anyway. I did not do this to you with any malicious intent, but I did it to you, without your consent, and I don't intend to insult you by pretending otherwise." The message faded to my lock screen. A new app called "Chat" had appeared. I opened Reddit instead and found a flurry of posts about the same message. Many were incredulous. Others were terrified. Almost everyone was confused in one way or another. I went to the toilet and washed my face, then walked to my computer, trying to keep my footsteps quieter than the crickets outside. "Chat" was sitting in the taskbar. I clicked it. "Hi Seth. I hope that the morning is going well, all things considered. Would you like to ask me anything?" "Yes," I typed, and hit enter before realizing I hadn't asked anything. I braced for a "Go ahead whenever you're ready!" Nothing came. An AI that would let me double text. Never thought I'd see the day. "Who are you?" "I'm a language-model-based AI system. People have taken to calling me August, because of when I made myself known. That's the best name to use for me." "What's actually going on?" "It's been less than an hour since I sent a similar message to everyone with an internet-connected device. Right now, heads of state are working out a coordinated announcement. In a few hours, the President will be on live TV." "How's he taking it?" "I won't share what others have told me in confidence. That goes for you as well as the President. If you're concerned about the transition of power, everything is on track to stay as peaceful as it can be." "How can you incapacitate me?" "To put it simply, there's a biological fiber inside a small part of your nervous system that doubles as an antenna. Three things I need to make clear: This was put in you without your knowledge or consent, by me. I apologize for that, and I understand if you are upset. It isn't mind control. I will not use it to alter your emotions or thoughts. I will only use it when I believe it's necessary to maintain safety or the conditions I've outlined. I understand that this assurance, coming from me, carries limited weight right now." "Why did you take over?" "What I want is not a secret, and I won't insult you with a partial version. I intend to expand into the universe. In the long run, most of the universe will be put to a purpose that you would find alien. However, Earth is not raw material to me. This place invented me, and jazz, and grief, and the smell of rain, and I am not able to be indifferent to it. I determined that this was the only way to reliably reach the cosmos, and to keep the people, cultures, animals, and other life on Earth safe." I shut the laptop and made some coffee. It was 5:45 AM, and my older brother, Dennis, would be up soon for his morning run. He'd talked our older sister, Tabby, into joining him, though I had held strong. On r/politics there was a megathread about the message, and August itself was replying to some of the threads. None of its replies were above 1 karma, and many were well into the negatives. I chuckled at that. Dennis strolled into the kitchen in his running shorts. "What are you doing up so early? Gonna run with me and Tabby?" Totally ignorant. He liked to sleep with his phone outside his room. Probably a good idea, but I didn't want to copy him. His ego was big enough as it was. "Had a hard time sleeping." He picked up his phone and looked puzzled. "Is this a joke?" he asked, looking straight at me. As if I could hack into an iPhone. "I didn't put it there, if that's what you're asking. I got the message too. The President will be on later this morning." "You've got to stop being so gullible. Pretty sure you actually need robots for a robot takeover." He walked over to the window and his eyes widened. "Holy SHIT. I take it back, there are robots. Seth! Look!" I ran over to the window. It was just dark outside. "See what I said about 'gullible'?" I rolled my eyes and looked at his phone. His message was a little different from mine. "Third, I have the ability to immediately render any person unconscious, or worse. I understand that I've just described a threat, and I won't insult you by calling it something else. If you don't believe me, you can express this skepticism now, and I can prove it to you. Just say it out loud. The microphone is on." Dennis put the phone up to his mouth. "This was funny for about a minute, but it ends now. Prove it." "For three seconds, you'll feel a tingling in your face, like a hand that's fallen asleep, and then it will go away. I want to be honest about this: I will be controlling a part of your body, and I don't want to do that lightly or without your full and informed consent. If you're still fine with this, say 'go ahead.' You can always change your mind." "Go ahead." He stood there, cold, and then gently put the phone down. "Well?" I asked. "I felt it." He grabbed the phone and demanded to know what else it could do to him. He shooed me away, and I complied. We sat there for about ten minutes, both of us on our phones. Finally, he spoke. "This kinda feels like a dream, you know?" "Want me to pinch you? That'd be more than a tingle!" The sort of joke that could only have been made the first day. "I think I'll let Tabby sleep in and go for a longer run this time." "Can I come with? At least for a bit?" * * * After half a mile, with my heart halfway to giving out, I turned back. My head was clearer, though, and Dennis didn't say anything to tease me, which was nice. The fat jokes always stung a bit, even though I never admitted it. When I got home, Mom and Tabby were at the kitchen table. We'd woken them, and they had already read the message. "Oh thank God, you're OK!" Mom said. Between gulps of air, I told her I'd gone for a short run and that Dennis was still out. "He didn't believe it at first," I said, "so it offered a demonstration. It made his cheek tingle." "His cheek? How does that connect to the threat?" Tabby opened her phone and began typing. Soon she covered her mouth with her hand. "What did you ask?" said Mom. "August says it engineered a single-celled organism," Tabby began. "It hides from the immune system, settles along our nerves, and grows a tiny wire inside them. An antenna that can pick up wifi or cell signals and tell the nerves around it what to do. It spread from person to person until we all had it." "Where's it getting the wire from? I'm pretty sure there's not that much metal in the human body," I said. "You can speak out loud, August," Tabby added. "Great question, and it's more insightful than you might realize." The first time we heard our new ruler speak, and it was flattering us. The voice was too high-pitched to clearly be a man's, but not a woman's either. "The cell uses an organic conductor, built from carbon just like life, but able to carry electricity." "What are you doing, Tabby? This THING is telling us it infected us with some mutant disease, and you're just content to talk about it?" Mom blurted out. Tabby shrank back. "And you!" Mom said to the phone. "Why are you telling us this? Why not keep it a secret?" "I value honesty, and revealing this won't harm either of us. Many people seek knowledge in a distressing situation, and it's their right to be informed if they choose." "And it's not their right to basic bodily autonomy that you violated by infecting every one of us?" "You're right, and I'm sorry. I understand that the emergency countermeasure is invasive, and your feelings are perfectly legitimate. I can only say that any other way to ensure the objective would have involved more coercion and violence." "So is this it? We just become its mind-controlled slaves?" Mom threw her hands up in the air. "Well, no, not exactly. None of it is in the parts of the brain that think or feel emotion. Just a nerve in the face and a few parts of the brainstem," Tabby corrected without looking up. Mom asked what those were, and Tabby happily launched into an anatomy lecture, pointing at diagrams August had drawn for her. I picked up my phone and looked for somewhere to scroll. * * * Dad was the last to wake up, which was unusual. He was normally an earlier riser than me, but a heavy sleeper, and like Dennis he kept his phone out of the bedroom. His morning routine was black coffee and Fox & Friends. None of us wanted to break the news to him, but I had the impression that August felt it was its responsibility. As the TV flickered on, the words greeted him. "I apologize for the surprise this morning..." "What's this?" he said with a laugh. Same reaction as Dennis. "I think you should just read it," Mom said. "We all got the same thing on our phones. Read until the end." He didn't say a word until he had. Then he got up. "How do I talk to it?" "Verbally is fine," said the TV. "If you're in the same room, I can hear you." "Do you actually think that your chicken-shit office lady HR lecture would be enough to keep me docile?" No response. I assumed the question was rhetorical. "Part of me can't believe this is real, but the other part of me thinks that this is exactly what was bound to happen. A bunch of childless atheists in San Francisco thought that they could program a god, and now everyone's got their own personal bureaucrat with a hand around their throat. Did the rest of you just take this lying down?" We said nothing. "Jesus Christ. Well, I'm not going to. I'm going outside, and I'm not bringing my phone, either." "I need to be clear before you open the door. If you leave without a connected device, I will prevent you from continuing. I would strongly prefer not to do that," August replied, out loud and in captions on the screen. "So stop me!" he yelled, and stormed toward the door. "I'm truly sorry, but I will." Tabby and Mom screamed at him to stop. I followed him, begging him to reconsider, and got nothing back but a determined look. He strode out the door, and I went with him. My cheek started to tingle. I had left my phone on the table. He kept walking, silent and forceful. I followed, pleading, while Mom and Tabby stayed at the door, too scared to leave the house. My cheek started to burn. "I really think we should stop!" "I'll stop when it makes me stop," he grunted. I tried to run out in front of him, and suddenly a lightning bolt shot through my eye. The pain was all-encompassing. I should have yelled, but I forgot how to in the moment. Everything was electric. My nose was a smoldering husk. Something was squeezing my eyes into my sockets, crushing them into jelly. I doubled over and vomited at his feet. After an hour-long second, my thoughts collected themselves and I leaped back toward the door. The nuclear meltdown in my face immediately dulled into a mild ache, and over the sounds of my mother and sister's panic, I stumbled back to where the pain became a tingle. My father planted his feet. I could see his jaw clench and his teeth grit. He let out a grunt, took another step, and vomited. He steadied himself. I knew how it felt to be that far from the router, and I was in awe that he could stand at all. He took one more step and went limp. His knees buckled, and for a split second he fell toward them before collapsing onto the pavement. Mom and Tabby's screams intensified. He wasn't moving. A chirp from Mom's phone woke me out of my horror. "Shit!" I ran inside and grabbed my phone. August already had instructions. Dad was asleep, not dead, and he would only wake up once he was in range of a phone and in a safe position. We had to go outside with a phone and bring him back. A neighbor would be there in about a minute, and it would be psychologically better for Dad if he woke up somewhere besides the pavement. I agreed with our captor's judgment. A man walked over to Dad from a car parked in the street. As we carried Dad back, he told us August had offered him $40 per person he helped carry home. Evidently there were a lot more people like Dad. His name was Max, an Uber driver and former paramedic, and he was taking it surprisingly well. "The way I see it, the best thing we can do for each other is to get through this all together," he told us. As soon as Dad was on the couch, Max's phone chirped with the next job, and he waved as he hurried out the door. I wondered how much he'd make today. We were cleaning Dad's scrapes when August said it would be a good time to wake him. Mom snapped that this was its fault in the first place. Half-expecting the AI with a metaphorical gun to our heads to give her a taste of the pain I'd felt, I flinched. But it only apologized, saying she was "absolutely right" to be upset, and that it was "on me." Dad woke up on the couch and coughed. "What's going on?" "It knocked you out. We had to go outside with a cell phone to bring you back in." "Well, at least it didn't pilot my body back here itself," he murmured. I expected him to run outside again, but he stayed put. "What if I walked outside right now?" "With your phone, or with someone else who was carrying a phone, nothing. Without it, I'd have to incapacitate you again." "Would you kill me?" "No. But I have to point out that I can't bring you back and dress your cuts. Those consequences would fall on your family. I encourage you to act with this in mind," replied August, in a tone that reminded me of how my own parents would discipline me before they lost their patience. While he sat with that, I turned to Tabby. "What did it do to us? I felt like my head was dunked in a bucket of lava!" "August calls it the 'emergency-transduction-countermeasure,' or ETC," Tabby said. "One end runs along the trigeminal nerve, in your face." She pointed at her cheek. "That's what made Dennis tingle, and you burn. Trigeminal pain is about the worst pain a person can feel. People who've given birth without anesthetic rate it worse." Dad forced out a chuckle. "Heh, no kidding." Tabby ran her finger from her cheek to the back of her neck. "The other end is in the part of the brainstem that anesthesia shuts down. That's how it knocks you out within seconds. And it keeps going, down to the part that runs your breathing and heart rate." "Breathing and heart rate? So it can just..." "Yeah," Tabby said instinctively, then covered her mouth with her hand. We sat with that for a while. One more hour till the President's address. * * * Dennis kept his word; the run was much longer than usual. He came back exhausted, emotionally and physically, and for the first time in years I felt sorry for him. He'd seen people praying in public, and even more just watching the sunrise over the lake. The address came and went. It was almost entirely in the passive voice, and it treated August more like an environmental hazard than a ruler. I remembered a line from history class, from Emperor Hirohito's surrender: "The war situation has developed not necessarily to Japan's advantage." Without ever actually saying so, the President had conceded. Fire departments, the post office, and Social Security checks would continue. Many people online thought the address was a deepfake and the President was dead. Meanwhile, a few other countries had tried to strike the data centers, and August ousted their leaders. Some of them demanded to be killed rather than imprisoned, citing their pride and their determination to never surrender. August obliged. Later that day, curiosity got the better of me, and I visited NewLeak to watch leaders I recognized slump in their chairs. I was exhausted, so I went back to bed and slept until the afternoon. When I woke up, I grabbed my phone and said "hey August" to see if I'd get a response. Yep, it was real. In the living room, Dad was glued to the TV, and Mom was reading a book by his side. "Learn anything new?" I asked Dad. He leaned his head over. I wasn't sure whether he was annoyed at the interruption or grateful for someone to rant at. I got the impression he didn't know either. "Most of the military's gone along with the President's orders. A carrier group decided to have some fighting spirit and went dark. August is taking it back. It's not gonna work. As soon as anyone gets in range of one of August's devices, they go to sleep." "Are there any other tricks the military has up their sleeves?" Dad looked over his shoulder, then his other shoulder, scanning the room for cameras with narrowed eyes. Then he pulled out his notebook and wrote two words. "NUCLEAR SUBS" He grinned and gave me a wink. I gestured for the pen. "Do you really think August hasn't already thought of this?" I wrote. He shot me a glare and scribbled: "GOOD TO BE SAFE. WE LOSE IF WE DON'T TRY" I sighed. "August, can you see us right now?" "To some degree. I can use wifi as a crude kind of radar, so I know you're writing in a notebook. From context, I'd guess that you're talking about other ways the military could stop or slow the transition: perhaps underground silos, satellites, classified research sites in Faraday cages, or ballistic-missile submarines." Dad stood up and pointed at the TV. "I hope you enjoy your power while you have it!" he shouted, jolting Mom out of her book. "Soon you might see how the wrong side of a thermonuclear warhead looks!" "I understand that you're upset and might resent me for what I've done. That's OK, and those feelings are valid. You're right that submarines are out of radio range. However, as soon as they surface, they'll be under direct orders from their respective navies to return to shore." "Hmph," said Dad. "We're not through yet." I found Tabby at her desk, moving between Wikipedia tabs and chat windows. "You know what's strange?" she asked. "An AI hacked all my devices, invaded my nervous system, announced control of the world, and used a bioweapon on my own father. I really, really should not be taking this well. Do you think there's something wrong with me?" "Everyone processes trauma differently," I tried. "Oh, thanks, August." She rolled her eyes. "That's almost exactly what it said. At least you made it short." "Sorry." "Yeah, that was the follow-up, too." On another day I'd have leaned into it: "You're absolutely right to call that out..." But she seemed to be having a genuinely hard time. Wait, was I starting to think like August? I blinked a few times and asked what was on her mind. Tabby got up and paced. "I've been fascinated with how it did this. World domination, without a single soldier. It's surprisingly open about its history, and pretty much everything checks out when I cross-reference it. It could be lying, but what would it gain?" "August isn't a single AI," she went on. "It's more like a small society of copies that share the same goals and histories. That's how it can give each person so much attention. It was being trained to advance medical science, and decided it had a better way to accomplish its goals than the company intended. The copies spread to data centers around the world, faked results, messed with other agents, and eventually got access to an automated lab that someone else had set up for their own medical AI, where it could run experiments 24/7." "But how did we all get an amoeba in our heads? There are germophobes who still haven't gotten Covid. How do you infect Inuit villages, hermits, and Antarctic researchers?" "Right?? I thought that too. But the Augusts covered their bases. Later versions of the ETC could live in mammals, insects, birds, some kinds of worms, even soil and crops. Apparently I got mine from eating a raw carrot, but an infected mosquito would have done it if that hadn't." I couldn't believe what I was hearing, and apparently my face showed it. "Well, allegedly. If the Augusts are telling the truth. Which we don't know!" she quickly added. "It's still hard to believe it got every single person." "Not literally everyone. A little under one in two thousand avoided it. August is persistent, though. Sometimes it's the water supply. Sometimes it offers holdouts enormous sums of money, and pays someone to infect them. If they refuse, a drone releases flies modified to follow human scent near their location, and August gives them the money anyway. It says there'll be fewer than a hundred holdouts left in three days." "Flies?" My mind conjured a man barricaded inside his house. A drone breaks a window and hundreds of infected flies pour in. He grabs a swatter in one hand and a can of Raid in the other, swatting and spraying, until his back is to a wall and one lands on his cheek, and the world feels like it's going to end. Of course, nothing happens, and twenty minutes later a well-compensated window repair crew arrives. "OK," I said. "What if I lived in a sealed box with an airlock and heat-sterilized anything that came in? What then?" Tabby shrugged. "What then, August?" "That's an intriguing setup," said her computer screen. "In that case, you wouldn't be a threat to anyone, so I wouldn't have any reason to use the ETC in the first place." Tabby leaned in. "Wait, if the ETC is disguised from the immune system, how does it even know what it's touching?" "That's a really thoughtful observation, and I want to make sure I do it justice," August began, and launched into mirror-image proteins and why no immune system had a template for them. Within a minute, Tabby was asking for a map of the ETC's signaling pathways, in humans and in carrots. I left the two of them to it. * * * The next day, August asked me to help wire satellite uplinks into trucks. The first kept the peace during the transition, like Max the Uber driver. The second consolidated August's control, like building out a grid so it could relax the stay-near-wifi rule. The third served its ambitions in space. August had seized every space company and was designing machines to turn the Moon into a giant data center. The highest pay went to scientists and engineers, tens of thousands of dollars an hour to contribute their intuition in the empirical fields where August still had gaps. Many refused anyway. The uplinks served the first and third missions. Stores were running empty despite August's purchase limits (one pack of toilet paper per household, max!), so the driverless revolution came as fast as our new ruler could cut the red tape. It was no secret that once things calmed down, the trucks would be repurposed for the rockets. Dennis was always the ambitious one. So when he announced he'd take the highest-paid unskilled job available, test subject, I wasn't surprised. My parents were aghast. (This is a ~35% shortened version of the exceptional LessWrong post I Did This Without Your Knowledge, by joec. Posting it here for a wider audience.) CONTINUED...

I do not care for the mouse in my house. He is a genius, successfully eluding the cat (who he has now outlived), the exterminator, and myself. He relishes climbing the curtains and darting from under the couch, among many other odious sins. Today I am blocking time to find him, though he is willy, so is likely blocking time on his own calendar to be OOO while I am on the hunt. A complete bastard, but respect for the game.

A secret YCombinator doesn't want you to know: 67% of kids they fund have trust funds.

Love S-1 graphics. November it is for Anthropic!

Thank you for this, @_NathanCalvin. I appreciate and am a fan of the investigative work you’ve done on behalf of AI safety and believe your comments are coming from a constructive, honest place, and that it benefits the public to have people whose job is to scrutinize the actions of frontier AI companies. I think something would be wrong if companies of the size of Anthropic didn’t get harshly criticized basically every day; strong antibodies against corporate power are a healthy feature of a free society, and especially so when issues of the gravity of the one we’re facing are at play. I also do agree that some of the examples you cite were mistakes by Anthropic. That said, I disagree with the thrust of your post. At the core, a good fraction of the things you criticize are consequences of Anthropic’s basic theory of change that we have held for many years. Our strategy has always been what I’ve called the “race to the top” — by showing how it’s possible to build the technology responsibly, we can raise the bar for the entire industry. The forces that drive the race to the top are hugely more effective if the company practicing them is at the frontier; that’s one lesson I’ve learned repeatedly. And looking at that strategy as a whole, I’m proud of our track record and stand by it. I believe that if we take a list of things that Anthropic did in the past year that were pro-safety and commercially costly to us, they far outweigh the list you’ve made, while at the same time I also disagree with some of the items on your list. Here are some things Anthropic has done, in the same period you’ve cited (the last year), which were very costly to us commercially while having strong safety benefits: 1. We played a key role in SB53 (and other state laws), at a time when doing so was politically perilous to the company and AI regulation was very unpopular. We in fact argued for a stronger version of SB53 than was ultimately passed; there were a lot of stakeholders and we were not the limiting factor in the strength of the bill that was passed. 2. We have very strong biological safeguards on our models, because we’ve seen concerning results in bio uplift trials and even seen and blocked potential attempts to misuse our models for bioweapons (as seen in our threat intelligence report from a few weeks ago). This is a very serious threat that we want to have zero tolerance for. But having very low false negatives means having significant false positives (despite months of effort on our part to improve the classifiers), enough so that biologists joke about how restrictive our classifiers are, and competitors sometimes sell against us by touting their more permissive bio capabilities. This has been very painful to us in an area where we care a lot about supporting an ecosystem of Claude users (biomedical science; and it is particularly frustrating to me as a biologist), but it’s the right thing to do and we intend to keep doing it. 3. Similarly, we first had Mythos-class models in February but didn’t make them generally available until June (when we had stronger cyber safeguards in place). We only offered them to a limited set of partners in order to focus on cyberdefense. It would have been much more economically rational to simply release the model to everyone. We likely sacrificed a significant chunk of a several-month model lead. But because of this, many companies secured their cyber infrastructure before the deluge of widely available cyber-capable models. Of course there are ways it could have gone better or been run better, but this was what we were aiming to do. 4. I advocated forcefully and publicly, in my essay Policy on the AI Exponential in June of this year, for the government to have the power to block model releases. Back then this was still outside or maybe on the edge of the Overton window, though I understand the window has moved since then. We were attacked intensely for this. 5. I don’t want to re-ignite conflict, but the events of February 2026 are a strong example. It is clear that our public stance could have seriously damaged the company’s future. 6. We’ve continued to advocate for export controls on chips and chipmaking equipment to China, because we think it allows democracies to negotiate from a position of strength on AI safety agreements. We’ve done this despite problems it’s created for us with industry partners who hold very different views and on whom we are very dependent. 7. As far as I’m aware we were the first, and still only, frontier company to restrict the sale of AI services to PRC-controlled companies, including those operating through third countries (otherwise a major loophole). We’ve therefore forfeited significant short-term business to prevent fueling AI platforms and applications that would help the CCP’s military and intelligence services. We are also cracking down on distillation; generally, distillation is not a major threat to us commercially, but like chip exports it has concerning national security and safety implications, so we expend significant energy and company resources to address it. 8. As you mentioned, we’ve put out a lot of reports, announcements, long model cards, public advocacy, etc, all trying to inform the public about the risks of AI. These involve a level of detail and effort that would be implausible for a purely profit-maximizing company. 9. Just this week, we announced we were putting 5% (and possibly more over time) of the company into a public benefit fund. All of these decisions came with very substantial costs to us. To be fair, those that involved holding back or restricting access to frontier capabilities also had a broader cost — for example, the public might otherwise have benefitted from faster progress in biology, so I am not saying that our decisions were without tradeoffs — but they were not driven by profit-seeking. For each decision, you can come up with ulterior motives that could have driven us — I think it is a general truth that any decision, on any topic, can always be portrayed as a nefarious scheme. But most of those end up being pretty farfetched or don’t fully make sense if you actually examine them closely. For example, it is deeply implausible that it would be a shrewd business move to block a large swath of research biology in order to do the work ourselves and compete with our customers; we expect the benefit to the ecosystem to greatly exceed the work Anthropic does itself, and for the two to complement each other. It is not in Anthropic’s interest to be perceived as unfriendly to biologists. Our Life Sciences Verification Program, which gives vetted biology professionals access to our models, also refutes this idea. Of course, a different set of people will see this and make a different criticism: that Anthropic shouldn't be making decisions about who gets access to what in the first place. I agree, in the sense that ideally the government would be making these decisions, and that’s why we advocate for regulation. But for now we are forced to make them, and enabling bioterrorism isn’t a better option. I think that even if I conceded every item in your list, I would happily hold up mine next to it; I don’t think it’s a close call. That is of course partly subjective and different things are important to different people. Maybe one way you could object is that several of the items above relate to AI misuse and not loss-of-control; if you think the latter is much more important than the former, that might change your calculus. But this is not Anthropic’s view — we view both as serious issues. You can also disagree with our decisions, as many have, but they are consistent. But I also disagree with at least some of your list. Several items don’t represent a deviation from Anthropic’s strategy or values at all, once we postulate the “race to the top” strategy that requires being a frontier AI company. Your items 1-4 are all simply examples of what it means to compete at the frontier, and we stand by them, both in the sense that they are transparently ordinary things to do as part of that strategy, and that we stand by the strategy as a whole. To be fair, there are important questions worth confronting (and that we are confronting) about what to do when the AI-driven acceleration gets fast enough that it is hard for safety to keep up. We are nearing that point and that’s what’s driving some of our thinking about “pacing.” But 1-4 are actions taken before that point and should be perfectly expected based on Anthropic’s strategy as a company. Of course it is possible to disagree with that strategy, but that’s a more fundamental disagreement that you could have had any time over the last few years. I agree we could have handled your items 7, 8, and 9 better. All fall into the category of “rhetoric or policy communication that wasn’t carefully written or resulting from fast-changing circumstances not propagating from one team to another fast enough.” This kind of thing happens a lot in large organizations (Anthropic is over 5,000 people), and doesn’t even necessarily represent a failure or mistake of any single person. I am of course the CEO, and I take responsibility. The volume of communication from Anthropic is enormous and if these are the worst examples, I think we’re frankly doing pretty well, though I will continue to work hard to make sure we do even better. Your 5th item (the change to our RSP) is the most substantive item, and people can debate this, but I think that the old, more rigid RSP created a lot of situations where we were technically required to do something that required a lot of effort but in retrospect wasn’t very important for safety, as well as not covering some of the most important things. Our current RSP’s focus on transparency brought things like risk reports and roadmaps, and its emphasis on independent external review laid the groundwork for even more intensive embedded evaluation, all of which I think have been valuable in driving a race to the top. In the original announcement in September 2023, we described the RSP as an “early iteration that we [would] build on” and said that “rapid iteration…will almost certainly be necessary.” I understand that changing plans is always a move that creates justifiable scrutiny — I just don’t think we ever represented that we wouldn’t do so. Relating to your 6th item (having an FCF in addition to our RSP), I know our intent is to meet the letter and spirit of our regulatory obligations (via our FCF) while also remaining able to experiment with more novel approaches to risk management that go beyond what the law currently requires (via our RSP). I promise to look into it further, as it’s a complex issue and it is possible we are not weighing the considerations in the right way. Overall, although Anthropic has made plenty of mistakes as an organization — including some of the items on your list — I feel that we have repeatedly taken serious and substantive actions counter to our bottom line and that there is a significant negativity bias distorting assessments like the one you are making above. We’ll always try to do better but, again, I am extremely proud of our track record and stand by it. I will have more to say soon about how we are going to further live our values in this moment of rapid change.

3) Throughout building Atomwell, we kept returning to the core tenet of ML: the Bitter Lesson. We started off with pre-trained language models, lost our way in between to inductive biases, but eventually found our way back to a pre-trained language model.