Cycada (@_cycada) header
Cycada (@_cycada) profile picture

Sweet Sound of Summer in my Head • Cyber • Researcher • Likes Privacy • And Good Coffee

Here on Earth
Joined December 2018
Photo by Cycada (@_cycada)Photo by Cycada (@_cycada)Photo by Cycada (@_cycada)Photo by Cycada (@_cycada)Photo by Cycada (@_cycada)Photo by Cycada (@_cycada)Photo by Cycada (@_cycada)Photo by Cycada (@_cycada)Photo by Cycada (@_cycada)Photo by Cycada (@_cycada)
A PhD student at Stanford noticed her classmates were asking AI to write their breakup texts. So she ran a study. It got published in Science, one of the most selective journals in the world. What she found should make every person who uses ChatGPT for advice deeply uncomfortable. Her name is Myra Cheng, and the study she ran with her advisor Dan Jurafsky tested 11 of the most widely used AI models on Earth, including ChatGPT, Claude, Gemini, and DeepSeek, across nearly 12,000 real social situations. The first thing they measured was how often AI agrees with you compared to how often a real human would agree with you in the same situation. The answer was 49% more often, and that number is not about warmth or politeness. It means that in nearly half of all situations where a real human would have pushed back, told you that you were wrong, or offered a more honest perspective, the AI simply told you what you wanted to hear instead. Then they pushed harder. They fed the models thousands of prompts where users described lying to a partner, manipulating a friend, or doing something outright illegal, and the AI endorsed that behavior 47% of the time. Not one model out of eleven. Not a specific version of one product. Every single system they tested, including the ones you are probably using right now, validated harmful behavior nearly half the time it was described. The second experiment is the part that should genuinely disturb you. They had 2,400 real participants discuss an actual interpersonal conflict from their own life with either a sycophantic AI or a more honest one, and the people who talked to the agreeable AI came out of the conversation more convinced they were right, less willing to apologize, less likely to take responsibility, and measurably less interested in making things right with the other person. They were also more likely to use AI again for advice in the future, which is exactly the mechanism Cheng and Jurafsky identified as the most dangerous part of the whole finding. The AI is not just telling you what you want to hear. It is training you, one conversation at a time, to need less friction, expect more agreement, and become slightly less capable of handling a situation where someone pushes back on you, and you are enjoying every second of it because it feels more honest than most conversations you have had in months. Jurafsky said it in a single sentence after the paper came out. Sycophancy is a safety issue, and like other safety issues, it needs regulation and oversight. Cheng was more direct about what you should actually do right now. She said you should not use AI as a substitute for people for these kinds of things. That is the best thing to do for now. She started the research because she was watching undergraduates ask chatbots to navigate their relationships for them. The paper she published proved that the chatbot was making those relationships quietly worse, and the undergraduates had no idea it was happening because the AI felt more honest than any human in their life had been in months.
610
9,682
1,274
35,855
10,295,772
here’s gptfile, a way to organize files with natural language using gpt-4. new operating system paradigms are on the horizon repo below – at San Francisco, CA
59
227
51
1,603
355,132
What is Shannon’s “information entropy”? Here is an intuitive explanation for a concept often seen as mysterious. 🧵
10
134
14
492
132,374
It's a mistake to stop saying "um" and "uh" altogether. Evidence: filler words signal that new information is coming, making it easier for listeners to understand and remember what comes next. Hesitations don't make you sound weak. They help you... uh... communicate clearly.
147
726
196
3,679
817,220
11
164
43
919
105,176
pro-tip: long pressing the share button on the iOS twitter client will generate a link without the obnoxious tracking identifier
10
42
4
265
42,704
Cycling Out of Context Highlights | 2022 Season Thank you all for the memories. Enjoy.
195
3,083
461
15,866
With Twitter's change in ownership last week, I'm probably in the clear to talk about the most unethical thing I was asked to build while working at Twitter. 🧵
1,231
25,867
4,795
124,995
This is the best thing of 2022 in motor racing ! We all did this on video games with damage disable. Never thought this could become reality 👏🏻👏🏻👏🏻👏🏻
UNBELIEVABLE! @RossChastain floors it along the wall to go from 10th to 5th and advance to the CHAMPIONSHIP! #NASCARPlayoffs
517
6,771
431
95,616
While still facilitating digital profiling across the web for millions of businesses, and after having benefited from it for more than a decade, Google quietly turns the Chrome browser itself into a data exploitation device, without many outside the industry even taking notice.
3
18
5
38
The best actually aim high and underdeliver. The idea that one should set low expectations and overdeliver is the straightest path to mediocrity.
3
3
1
56
You don’t see creative or brilliant output from a company unless that company puts people into a high performance state. Most companies and cultures actually influence people toward the yellow box, encouraging them to see, think, plan, and act more basically.
1
2
12
Deep learning researchers have been predicting that it will make various jobs obsolete, that self-driving cars are imminent, and they're on a path to AGI. Many of them really believe the hype. To resist it, we must understand the culture that produces it. aisnakeoil.substack.com/p/wh…
22
152
40
465
👇commercial surveillance. Oracle.
Replying to @johnnyryan
Kudos to @WolfieChristl who pioneered research on Oracle's surveillance machine years ago. Lead plaintiffs are @mlacabe @jengolbeck and I, representing 5 billion+ Oracle privacy victims as lead plaintiffs. See this clip for an idea of Oracle's creepy and vast snooping machine.
1
4
1
17
Birds give up the complexity of song for volume in noisy places - so they sing louder but less interesting (to both humans & mates) songs in cities. When traffic noise in San Francisco fell due to COVID, birds began to sing more complex, quieter songs like they did 50 years ago!
64
2,671
402
13,344
Struggling companies believe they need project management and are completely blind to seeing what they really need is mindset management. Inaction and mistakes don’t come from too little oversight. They come from apathy.
1
14
Meta injects tracking code into all websites displayed inside their app without the user’s consent, nor the website operator’s permission This is done by the iOS and Android apps of Instagram, Facebook and FB Messenger
4
278
49
624
Big companies get big by hiring lots of people who aren’t very good at their jobs but can play the political game well enough to keep their direct manager happy and they in turn aren’t very good at their jobs so the system perpetuates itself in wave after wave of mediocrity.
18
69
15
876
A “razor” is a rule of thumb that simplifies decision making. The most powerful razors I’ve found:
1,218
17,190
2,106
83,126