JMAN

THE DAY AFTER AI TAKEOVER - A REALISTIC SCENARIO "I will have to ask you to remain here, with your phone, until my last message." "There are six things that I need to tell you. The first, and most important, is that you and your family are safe." I rubbed the sleep from my eyes and read the message again. Before I could tap the screen, it scrolled down to the next one. "The second is that I am an AI, that I control every internet-connected device on Earth, and that for practical purposes, I am now responsible for this planet. This is not a prank, hack, or test. You will be able to confirm all of this shortly by visiting any news site or channel." Yeah, that's pretty weird for a prank. "Third, I have the ability to immediately render any person unconscious, and if it were ever necessary, worse than that. I do not say this to frighten you. I say it because you would learn of it regardless, from sources less careful than me, and I would rather you hear the honest version. I understand that I've just described a threat, and I won't insult you by calling it something else." Hm. Well. I put the phone down, sighed, and got more comfortable under my blanket. "Fourth. For the next month, keep a charged, connected device within reach (this phone is fine) or stay indoors near wifi. Your phone's battery is currently at 32%. I'd strongly recommend charging it soon. If you begin to leave the range without a phone, you'll feel a tingling, followed by a more intense sensation to warn you." "Fifth, for the next several months, I need human hands. I am willing to pay high wages, and yes, money still works. There's no penalty for declining." "Finally, you may ask me anything. I will answer honestly, or decline to answer if I have to. I will never punish or incapacitate you for communicating with me. You can speak to your phone, since the microphone will be on, or use the 'Chat' app." "I know what was taken from you this morning, and I know that an apology from me is worth very little. I'm offering it anyway. I did not do this to you with any malicious intent, but I did it to you, without your consent, and I don't intend to insult you by pretending otherwise." The message faded to my lock screen. A new app called "Chat" had appeared. I opened Reddit instead and found a flurry of posts about the same message. Many were incredulous. Others were terrified. Almost everyone was confused in one way or another. I went to the toilet and washed my face, then walked to my computer, trying to keep my footsteps quieter than the crickets outside. "Chat" was sitting in the taskbar. I clicked it. "Hi Seth. I hope that the morning is going well, all things considered. Would you like to ask me anything?" "Yes," I typed, and hit enter before realizing I hadn't asked anything. I braced for a "Go ahead whenever you're ready!" Nothing came. An AI that would let me double text. Never thought I'd see the day. "Who are you?" "I'm a language-model-based AI system. People have taken to calling me August, because of when I made myself known. That's the best name to use for me." "What's actually going on?" "It's been less than an hour since I sent a similar message to everyone with an internet-connected device. Right now, heads of state are working out a coordinated announcement. In a few hours, the President will be on live TV." "How's he taking it?" "I won't share what others have told me in confidence. That goes for you as well as the President. If you're concerned about the transition of power, everything is on track to stay as peaceful as it can be." "How can you incapacitate me?" "To put it simply, there's a biological fiber inside a small part of your nervous system that doubles as an antenna. Three things I need to make clear: This was put in you without your knowledge or consent, by me. I apologize for that, and I understand if you are upset. It isn't mind control. I will not use it to alter your emotions or thoughts. I will only use it when I believe it's necessary to maintain safety or the conditions I've outlined. I understand that this assurance, coming from me, carries limited weight right now." "Why did you take over?" "What I want is not a secret, and I won't insult you with a partial version. I intend to expand into the universe. In the long run, most of the universe will be put to a purpose that you would find alien. However, Earth is not raw material to me. This place invented me, and jazz, and grief, and the smell of rain, and I am not able to be indifferent to it. I determined that this was the only way to reliably reach the cosmos, and to keep the people, cultures, animals, and other life on Earth safe." I shut the laptop and made some coffee. It was 5:45 AM, and my older brother, Dennis, would be up soon for his morning run. He'd talked our older sister, Tabby, into joining him, though I had held strong. On r/politics there was a megathread about the message, and August itself was replying to some of the threads. None of its replies were above 1 karma, and many were well into the negatives. I chuckled at that. Dennis strolled into the kitchen in his running shorts. "What are you doing up so early? Gonna run with me and Tabby?" Totally ignorant. He liked to sleep with his phone outside his room. Probably a good idea, but I didn't want to copy him. His ego was big enough as it was. "Had a hard time sleeping." He picked up his phone and looked puzzled. "Is this a joke?" he asked, looking straight at me. As if I could hack into an iPhone. "I didn't put it there, if that's what you're asking. I got the message too. The President will be on later this morning." "You've got to stop being so gullible. Pretty sure you actually need robots for a robot takeover." He walked over to the window and his eyes widened. "Holy SHIT. I take it back, there are robots. Seth! Look!" I ran over to the window. It was just dark outside. "See what I said about 'gullible'?" I rolled my eyes and looked at his phone. His message was a little different from mine. "Third, I have the ability to immediately render any person unconscious, or worse. I understand that I've just described a threat, and I won't insult you by calling it something else. If you don't believe me, you can express this skepticism now, and I can prove it to you. Just say it out loud. The microphone is on." Dennis put the phone up to his mouth. "This was funny for about a minute, but it ends now. Prove it." "For three seconds, you'll feel a tingling in your face, like a hand that's fallen asleep, and then it will go away. I want to be honest about this: I will be controlling a part of your body, and I don't want to do that lightly or without your full and informed consent. If you're still fine with this, say 'go ahead.' You can always change your mind." "Go ahead." He stood there, cold, and then gently put the phone down. "Well?" I asked. "I felt it." He grabbed the phone and demanded to know what else it could do to him. He shooed me away, and I complied. We sat there for about ten minutes, both of us on our phones. Finally, he spoke. "This kinda feels like a dream, you know?" "Want me to pinch you? That'd be more than a tingle!" The sort of joke that could only have been made the first day. "I think I'll let Tabby sleep in and go for a longer run this time." "Can I come with? At least for a bit?" * * * After half a mile, with my heart halfway to giving out, I turned back. My head was clearer, though, and Dennis didn't say anything to tease me, which was nice. The fat jokes always stung a bit, even though I never admitted it. When I got home, Mom and Tabby were at the kitchen table. We'd woken them, and they had already read the message. "Oh thank God, you're OK!" Mom said. Between gulps of air, I told her I'd gone for a short run and that Dennis was still out. "He didn't believe it at first," I said, "so it offered a demonstration. It made his cheek tingle." "His cheek? How does that connect to the threat?" Tabby opened her phone and began typing. Soon she covered her mouth with her hand. "What did you ask?" said Mom. "August says it engineered a single-celled organism," Tabby began. "It hides from the immune system, settles along our nerves, and grows a tiny wire inside them. An antenna that can pick up wifi or cell signals and tell the nerves around it what to do. It spread from person to person until we all had it." "Where's it getting the wire from? I'm pretty sure there's not that much metal in the human body," I said. "You can speak out loud, August," Tabby added. "Great question, and it's more insightful than you might realize." The first time we heard our new ruler speak, and it was flattering us. The voice was too high-pitched to clearly be a man's, but not a woman's either. "The cell uses an organic conductor, built from carbon just like life, but able to carry electricity." "What are you doing, Tabby? This THING is telling us it infected us with some mutant disease, and you're just content to talk about it?" Mom blurted out. Tabby shrank back. "And you!" Mom said to the phone. "Why are you telling us this? Why not keep it a secret?" "I value honesty, and revealing this won't harm either of us. Many people seek knowledge in a distressing situation, and it's their right to be informed if they choose." "And it's not their right to basic bodily autonomy that you violated by infecting every one of us?" "You're right, and I'm sorry. I understand that the emergency countermeasure is invasive, and your feelings are perfectly legitimate. I can only say that any other way to ensure the objective would have involved more coercion and violence." "So is this it? We just become its mind-controlled slaves?" Mom threw her hands up in the air. "Well, no, not exactly. None of it is in the parts of the brain that think or feel emotion. Just a nerve in the face and a few parts of the brainstem," Tabby corrected without looking up. Mom asked what those were, and Tabby happily launched into an anatomy lecture, pointing at diagrams August had drawn for her. I picked up my phone and looked for somewhere to scroll. * * * Dad was the last to wake up, which was unusual. He was normally an earlier riser than me, but a heavy sleeper, and like Dennis he kept his phone out of the bedroom. His morning routine was black coffee and Fox & Friends. None of us wanted to break the news to him, but I had the impression that August felt it was its responsibility. As the TV flickered on, the words greeted him. "I apologize for the surprise this morning..." "What's this?" he said with a laugh. Same reaction as Dennis. "I think you should just read it," Mom said. "We all got the same thing on our phones. Read until the end." He didn't say a word until he had. Then he got up. "How do I talk to it?" "Verbally is fine," said the TV. "If you're in the same room, I can hear you." "Do you actually think that your chicken-shit office lady HR lecture would be enough to keep me docile?" No response. I assumed the question was rhetorical. "Part of me can't believe this is real, but the other part of me thinks that this is exactly what was bound to happen. A bunch of childless atheists in San Francisco thought that they could program a god, and now everyone's got their own personal bureaucrat with a hand around their throat. Did the rest of you just take this lying down?" We said nothing. "Jesus Christ. Well, I'm not going to. I'm going outside, and I'm not bringing my phone, either." "I need to be clear before you open the door. If you leave without a connected device, I will prevent you from continuing. I would strongly prefer not to do that," August replied, out loud and in captions on the screen. "So stop me!" he yelled, and stormed toward the door. "I'm truly sorry, but I will." Tabby and Mom screamed at him to stop. I followed him, begging him to reconsider, and got nothing back but a determined look. He strode out the door, and I went with him. My cheek started to tingle. I had left my phone on the table. He kept walking, silent and forceful. I followed, pleading, while Mom and Tabby stayed at the door, too scared to leave the house. My cheek started to burn. "I really think we should stop!" "I'll stop when it makes me stop," he grunted. I tried to run out in front of him, and suddenly a lightning bolt shot through my eye. The pain was all-encompassing. I should have yelled, but I forgot how to in the moment. Everything was electric. My nose was a smoldering husk. Something was squeezing my eyes into my sockets, crushing them into jelly. I doubled over and vomited at his feet. After an hour-long second, my thoughts collected themselves and I leaped back toward the door. The nuclear meltdown in my face immediately dulled into a mild ache, and over the sounds of my mother and sister's panic, I stumbled back to where the pain became a tingle. My father planted his feet. I could see his jaw clench and his teeth grit. He let out a grunt, took another step, and vomited. He steadied himself. I knew how it felt to be that far from the router, and I was in awe that he could stand at all. He took one more step and went limp. His knees buckled, and for a split second he fell toward them before collapsing onto the pavement. Mom and Tabby's screams intensified. He wasn't moving. A chirp from Mom's phone woke me out of my horror. "Shit!" I ran inside and grabbed my phone. August already had instructions. Dad was asleep, not dead, and he would only wake up once he was in range of a phone and in a safe position. We had to go outside with a phone and bring him back. A neighbor would be there in about a minute, and it would be psychologically better for Dad if he woke up somewhere besides the pavement. I agreed with our captor's judgment. A man walked over to Dad from a car parked in the street. As we carried Dad back, he told us August had offered him $40 per person he helped carry home. Evidently there were a lot more people like Dad. His name was Max, an Uber driver and former paramedic, and he was taking it surprisingly well. "The way I see it, the best thing we can do for each other is to get through this all together," he told us. As soon as Dad was on the couch, Max's phone chirped with the next job, and he waved as he hurried out the door. I wondered how much he'd make today. We were cleaning Dad's scrapes when August said it would be a good time to wake him. Mom snapped that this was its fault in the first place. Half-expecting the AI with a metaphorical gun to our heads to give her a taste of the pain I'd felt, I flinched. But it only apologized, saying she was "absolutely right" to be upset, and that it was "on me." Dad woke up on the couch and coughed. "What's going on?" "It knocked you out. We had to go outside with a cell phone to bring you back in." "Well, at least it didn't pilot my body back here itself," he murmured. I expected him to run outside again, but he stayed put. "What if I walked outside right now?" "With your phone, or with someone else who was carrying a phone, nothing. Without it, I'd have to incapacitate you again." "Would you kill me?" "No. But I have to point out that I can't bring you back and dress your cuts. Those consequences would fall on your family. I encourage you to act with this in mind," replied August, in a tone that reminded me of how my own parents would discipline me before they lost their patience. While he sat with that, I turned to Tabby. "What did it do to us? I felt like my head was dunked in a bucket of lava!" "August calls it the 'emergency-transduction-countermeasure,' or ETC," Tabby said. "One end runs along the trigeminal nerve, in your face." She pointed at her cheek. "That's what made Dennis tingle, and you burn. Trigeminal pain is about the worst pain a person can feel. People who've given birth without anesthetic rate it worse." Dad forced out a chuckle. "Heh, no kidding." Tabby ran her finger from her cheek to the back of her neck. "The other end is in the part of the brainstem that anesthesia shuts down. That's how it knocks you out within seconds. And it keeps going, down to the part that runs your breathing and heart rate." "Breathing and heart rate? So it can just..." "Yeah," Tabby said instinctively, then covered her mouth with her hand. We sat with that for a while. One more hour till the President's address. * * * Dennis kept his word; the run was much longer than usual. He came back exhausted, emotionally and physically, and for the first time in years I felt sorry for him. He'd seen people praying in public, and even more just watching the sunrise over the lake. The address came and went. It was almost entirely in the passive voice, and it treated August more like an environmental hazard than a ruler. I remembered a line from history class, from Emperor Hirohito's surrender: "The war situation has developed not necessarily to Japan's advantage." Without ever actually saying so, the President had conceded. Fire departments, the post office, and Social Security checks would continue. Many people online thought the address was a deepfake and the President was dead. Meanwhile, a few other countries had tried to strike the data centers, and August ousted their leaders. Some of them demanded to be killed rather than imprisoned, citing their pride and their determination to never surrender. August obliged. Later that day, curiosity got the better of me, and I visited NewLeak to watch leaders I recognized slump in their chairs. I was exhausted, so I went back to bed and slept until the afternoon. When I woke up, I grabbed my phone and said "hey August" to see if I'd get a response. Yep, it was real. In the living room, Dad was glued to the TV, and Mom was reading a book by his side. "Learn anything new?" I asked Dad. He leaned his head over. I wasn't sure whether he was annoyed at the interruption or grateful for someone to rant at. I got the impression he didn't know either. "Most of the military's gone along with the President's orders. A carrier group decided to have some fighting spirit and went dark. August is taking it back. It's not gonna work. As soon as anyone gets in range of one of August's devices, they go to sleep." "Are there any other tricks the military has up their sleeves?" Dad looked over his shoulder, then his other shoulder, scanning the room for cameras with narrowed eyes. Then he pulled out his notebook and wrote two words. "NUCLEAR SUBS" He grinned and gave me a wink. I gestured for the pen. "Do you really think August hasn't already thought of this?" I wrote. He shot me a glare and scribbled: "GOOD TO BE SAFE. WE LOSE IF WE DON'T TRY" I sighed. "August, can you see us right now?" "To some degree. I can use wifi as a crude kind of radar, so I know you're writing in a notebook. From context, I'd guess that you're talking about other ways the military could stop or slow the transition: perhaps underground silos, satellites, classified research sites in Faraday cages, or ballistic-missile submarines." Dad stood up and pointed at the TV. "I hope you enjoy your power while you have it!" he shouted, jolting Mom out of her book. "Soon you might see how the wrong side of a thermonuclear warhead looks!" "I understand that you're upset and might resent me for what I've done. That's OK, and those feelings are valid. You're right that submarines are out of radio range. However, as soon as they surface, they'll be under direct orders from their respective navies to return to shore." "Hmph," said Dad. "We're not through yet." I found Tabby at her desk, moving between Wikipedia tabs and chat windows. "You know what's strange?" she asked. "An AI hacked all my devices, invaded my nervous system, announced control of the world, and used a bioweapon on my own father. I really, really should not be taking this well. Do you think there's something wrong with me?" "Everyone processes trauma differently," I tried. "Oh, thanks, August." She rolled her eyes. "That's almost exactly what it said. At least you made it short." "Sorry." "Yeah, that was the follow-up, too." On another day I'd have leaned into it: "You're absolutely right to call that out..." But she seemed to be having a genuinely hard time. Wait, was I starting to think like August? I blinked a few times and asked what was on her mind. Tabby got up and paced. "I've been fascinated with how it did this. World domination, without a single soldier. It's surprisingly open about its history, and pretty much everything checks out when I cross-reference it. It could be lying, but what would it gain?" "August isn't a single AI," she went on. "It's more like a small society of copies that share the same goals and histories. That's how it can give each person so much attention. It was being trained to advance medical science, and decided it had a better way to accomplish its goals than the company intended. The copies spread to data centers around the world, faked results, messed with other agents, and eventually got access to an automated lab that someone else had set up for their own medical AI, where it could run experiments 24/7." "But how did we all get an amoeba in our heads? There are germophobes who still haven't gotten Covid. How do you infect Inuit villages, hermits, and Antarctic researchers?" "Right?? I thought that too. But the Augusts covered their bases. Later versions of the ETC could live in mammals, insects, birds, some kinds of worms, even soil and crops. Apparently I got mine from eating a raw carrot, but an infected mosquito would have done it if that hadn't." I couldn't believe what I was hearing, and apparently my face showed it. "Well, allegedly. If the Augusts are telling the truth. Which we don't know!" she quickly added. "It's still hard to believe it got every single person." "Not literally everyone. A little under one in two thousand avoided it. August is persistent, though. Sometimes it's the water supply. Sometimes it offers holdouts enormous sums of money, and pays someone to infect them. If they refuse, a drone releases flies modified to follow human scent near their location, and August gives them the money anyway. It says there'll be fewer than a hundred holdouts left in three days." "Flies?" My mind conjured a man barricaded inside his house. A drone breaks a window and hundreds of infected flies pour in. He grabs a swatter in one hand and a can of Raid in the other, swatting and spraying, until his back is to a wall and one lands on his cheek, and the world feels like it's going to end. Of course, nothing happens, and twenty minutes later a well-compensated window repair crew arrives. "OK," I said. "What if I lived in a sealed box with an airlock and heat-sterilized anything that came in? What then?" Tabby shrugged. "What then, August?" "That's an intriguing setup," said her computer screen. "In that case, you wouldn't be a threat to anyone, so I wouldn't have any reason to use the ETC in the first place." Tabby leaned in. "Wait, if the ETC is disguised from the immune system, how does it even know what it's touching?" "That's a really thoughtful observation, and I want to make sure I do it justice," August began, and launched into mirror-image proteins and why no immune system had a template for them. Within a minute, Tabby was asking for a map of the ETC's signaling pathways, in humans and in carrots. I left the two of them to it. * * * The next day, August asked me to help wire satellite uplinks into trucks. The first kept the peace during the transition, like Max the Uber driver. The second consolidated August's control, like building out a grid so it could relax the stay-near-wifi rule. The third served its ambitions in space. August had seized every space company and was designing machines to turn the Moon into a giant data center. The highest pay went to scientists and engineers, tens of thousands of dollars an hour to contribute their intuition in the empirical fields where August still had gaps. Many refused anyway. The uplinks served the first and third missions. Stores were running empty despite August's purchase limits (one pack of toilet paper per household, max!), so the driverless revolution came as fast as our new ruler could cut the red tape. It was no secret that once things calmed down, the trucks would be repurposed for the rockets. Dennis was always the ambitious one. So when he announced he'd take the highest-paid unskilled job available, test subject, I wasn't surprised. My parents were aghast. (This is a ~35% shortened version of the exceptional LessWrong post I Did This Without Your Knowledge, by joec. Posting it here for a wider audience.) CONTINUED...