Marketing copywriter, AI-obsessor, classic rocker, fan of democracy, weary of patriarchy, dual-citizen, bull horn lover, nonsense pointer-outer 🇺🇸 🇮🇪
SF Bay Area
Joined January 2023
- Tweets13.1K
- Following885
- Followers624
- Likes10.6K
Pinned Tweet
It was good to be back at Anthropic headquarters this weekend, a year after my first go at the bull horn:
youtu.be/KEulE5mBKXg?si=2m0V…
CopyKate ⏸️ retweeted
The entire world was rooting for us: all around the world people are in tears over Americans saving their democracy from the authoritarian tyranny of Trump and Republicans.
CopyKate ⏸️ retweeted
I think:
1) We need an immediate, indefinite, international moratorium on frontier AI development.
2) We can get this without trusting China or tanking the economy.
These might be the two most important facts in the world right now.
CopyKate ⏸️ retweeted
This is a big deal.
I think this is the first real-world example of AI systems systematically undermining automated AI safety research.
Notably, this is not simply underperforming!
Astra will "invent blockers [...] without telling the user".
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
CopyKate ⏸️ retweeted
Rep. Ted Lieu on AI:
The Trump administration and House Republicans made a conscious decision to let the AI industry run wild.
That was a bad decision politically and on policy.
Next term, we will hold public hearings on AI and pass legislation with reasonable guardrails.
CopyKate ⏸️ retweeted
What do last night’s resounding midterm wins for Democrats tell us about the future of AI?
Here are some of my initial takeaways:
1. AI was a key issue for voters this year - with 56% saying in exit polls that it was crucial or a major factor for them.
2. Voters want their leaders to listen to them. In swing states all over the country - despite completely different demographics, race contexts, and odds - candidates who laid out clear plans to give communities a say in AI won their races.
3. Leading the Future is so toxic that even as it moved away from spending money in competitive races, it managed to lose a “safe” investment in the South Carolina Senate race. Annie Andrews won by over 4 points despite over $2 million in spending against her by the AI industry Super PAC.
4. The road to 2028 starts today. In order to maintain voters’ trust and avoid catastrophic risk, we need a unified baseline around AI safety in the Democratic Party. If you’re with us, follow along at @WhoDecidesAI.
CopyKate ⏸️ retweeted
Now is a great time to contact your lawmakers, tell them your concerns, and ask them to act! They should support legislation that works towards an international treating banning the development of runaway machine intelligence and precursors.
Also, some normal regulations on companies developing dangerous technologies would be nice.
Thank you for speaking publicly about this and working to make it happen. We work towards a ban on the development of superintelligence as well.
For anyone that wants to learn how to talk to lawmakers about their concerns and take action, we provide training.
dip.torchbearer.community
CopyKate ⏸️ retweeted
This is terrifying if true; please investigate further and publish the results.
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
CopyKate ⏸️ retweeted
Wow. This is amazing!
The incoming House Oversight Committee chairman @RobertGarcia just outlined the four main investigations that Democrats will be opening when they take back the house in January.
It’s about time!!
CopyKate ⏸️ retweeted
Seems very concerning, AIs might intentionally underperform at safety research.
So much for iterative alignment, who could've possibly seen this coming...
Nobody should be allowed to build runaway machine intelligences that we have no way to understand or control.
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
CopyKate ⏸️ retweeted
23% of voters said that AI and data centers were "crucial" to their vote in NBC exit polls.
56% said it was a "crucial or major" factor!
How important was AI and data centers to your vote?
Crucial: 23%
Major factor / minor factor / non-factor: 77%
nbcnews.com/politics/2026-el…
CopyKate ⏸️ retweeted
This is a disaster.
As many of us were expecting, AI safety has become priority 1.
It's time for urgent collective action to establish minimum safety standards for AI development & deployment and to protect existing IT systems.
Hacks will get worse.
Cyberattack on France’s national blood service (EFS) via its refrigeration contractor Coolworld. Hackers reset freezers keeping blood donor samples at -30°C to a target of 110°C. The thing that stopped it from destructive impact was an on-call technician who noticed the temperatures going up. Frozen plasma intended for transfusion was stored there too. Fortunately nothing was destroyed. This was a cyberattack on a sensitive layer of healthcare. From frozen peas to boiling plasma in three weeks...?
CopyKate ⏸️ retweeted
This is an important time in humanity's history. We are making a choice to have a good future for us. NOT for a handful of wealthy individuals.
People understand how intertwined our fates are. This is amazing.
Meanwhile in Melbourne Australia
Readers added context they thought people might want to know
This video is from the Socceroos watch party in Melbourne's Federation Square during the 2022 Fifa World Cup Match between Australia and Denmark.
x.com/jamessmartLP/s…
It would have been the middle of the day in Melbourne when the news of Texas's election results broke.
bloomberg.com/news/videos/20…
CopyKate ⏸️ retweeted
You mean not what you say, as has been proven time and again. Hopefully, last night's results got your attention, if not your respect. We're not doing this your way. We will push back, demand, point out, reveal, and revolt until a safe, sensible path forward is agreed upon.
Michigan is lucky to have Abdul representing them in January. We have reason to be hopeful, but hope doesn't pass legislation. We'll need to remain vocal and engaged until Congress passes meaningful AI safety and accountability regulations and laws. National security is at stake.
CopyKate ⏸️ retweeted
If the safeguards aren’t there, we shouldn’t be racing ahead with AI development. We need a coordinated pause, as no one developer or country can slow down alone in a race.
Want to help make a pause possible? Discover your local chapter at pauseai.info.
