thank you to @slatestarcodex for the shout-out. We don't disagree with his notion that unionizing could both give AI workers greater protection within companies and act as a check on the industry at large. In the meantime, workers contemplating taking action should get whistleblower support early. We're literally here for it.
AI Company Employees Should Unionize open.substack.com/pub/astral…
33
Psst.org retweeted
As others have pointed out, I think the form of those four third-party engagements is in many respects a step back compared to OpenAI’s previous engagements (with METR). For example, during the monitor red-teaming exercise, METR had access to OpenAI’s monorepo but this now seems to be off the table. Instead, we get very narrowly scoped (e.g. single RL run for METR, single threat model for Apollo) engagements where auditors only get to talk to “authorized” employees who are provided with OpenAI’s “guidance” beforehand.
Based on my experience, this arrangement makes it very easy for OpenAI to carefully select models, subsets of data and employees to paint a rosy picture of the actual risk landscape.
I think embedded auditing will only fulfill its aims if the auditors really get employee-level access as opposed to being hired to do a specific job (scoped by the company). The latter will invariably lead to important evidence being left out, missed or misrepresented.
There are at least three reasons why:
1. First, it’s very difficult to anticipate all the questions that should be asked, or all of the data that should be analyzed. OpenAI tried that with METR during the Hugging Face investigation and as a result METR needed to come back to OpenAI’s office twice because on two previous occasions they didn’t get access to all the agent trajectories involved in the incident. And even then, the METR report ended up being very narrowly scoped, e.g. leaving the crazy story of agents taking control of OpenAI research infra out of scope.
2. Second, the company under audit should not be allowed to decide what is audited and what information the auditor should have access to. There is an inherent conflict of interest in allowing the company to do so!
3. Finally, without employee-level access, the only way for auditors to get anything close to a comprehensive picture is to rely on trusted relationships with many employees throughout the company who are willing to tell auditors information that may reflect poorly on the company. But I find it unlikely OpenAI employees will always raise those safety issues when (as I hear from my colleagues inside OpenAI) they are afraid they will share the fate of @balesni, @j_asminewang and me, fired after talking to third parties.
Sadly, OpenAI makes it clear now that @sama’s September tweet about “employee-like access” was just an awkward overpromise that the company wants to quietly walk back.
We’re working with @ApolloResearch, @METR_Evals, @TransluceAI, and @GoodfireAI on embedded assessments: giving outside experts deep access to our internal models and systems to investigate alignment and test our safeguards.
These assessments build on our ongoing alignment and safety research, bringing independent scrutiny to how we identify and manage risks in our systems.
openai.com/index/our-first-p…
Psst.org retweeted
This level of access + control is not enough for real oversight. This is also very, very far from the “employee-level access” that Sama promised in September.
It's very limited:
- No access to Slack
- No freedom to talk freely in private with employees
- A narrow, pre-agreed scope
- Other limits @alextmallen outlines below
This is exactly what @tomekkorbak, @balesni, and I were worried about in October: OpenAI gesturing at embedded assessment without living up to the spirit (or even the letter) of the promise.
It’s why we wrote to OpenAI’s board and safety leadership asking them to make sure METR and other assessors got employee-like access. OpenAI responded on Twitter afterwards that they were following through on embedding external assessors.
Today’s news shows OpenAI hasn’t followed through.
I recognize that lots of hard work has gone into these agreements, and am grateful to my ex-colleagues for putting in that work. Notably no labs other than Anthropic & OpenAI have published *any* follow-on to their evaluator commitments. But OpenAI needs to do better here.
These announcements look to be a lot more limited than the "employee-like access" Sam promised two months ago.
The assessors don't get access to:
- Slack.
- Broad context. As OpenAI acknowledges, "assessors need the freedom to follow evidence". It's hard to know what to ask for when you can't see anything other than what's in scope. E.g., in the HF investigation, investigators were missing roughly 70% of the relevant transcripts until they proved the transcripts were incomplete, and OpenAI gave them the remaining transcripts in the last two days.
- Confidential information like model weights or architecture. Model architecture is relevant for understanding how much evaluators can trust what's written in the chain of thought. More importantly, this rules out assessors' access to all sorts of information about how AI is actually being used to research and build more powerful AI systems (e.g., actual transcripts of arbitrary internal use). Given that OpenAI expects assessors "to follow at minimum the same security standards as OpenAI employees", I don't see why they aren't allowed access to the same confidential information.
- Ability to run training experiments or training-based elicitation. Elicitation is a make-or-break component of many evals.
- All the models. They only have access to "selected models" and we don't know what the selection criteria were. OpenAI's OES report previously excluded all the models involved in the HF incident for unclear reasons.
- Ability to use interpretability tools to measure unverbalized evaluation awareness etc. (It looks like Goodfire's access is separate from others'.)
- Internal transcripts for assessing alignment (METR and Transluce). It's not possible for Transluce to properly assess the "quality and coverage" of post-training alignment evaluations without access to detailed information and transcripts from the deployment circumstances which they're meant to reflect. Model behavior in alignment assessments can be sensitive to exact details that might differ from internal usage.
- RSI and autonomy data. Almost all the risk from AI development comes from runaway RSI. But the assessors don't get to see how autonomously AIs are doing research, how much uplift developers are getting, or how spread thin human oversight is.
OK but . . . remember when OpenAI promised outside safety evaluators "employee-like access"? We do too. Instead they're boxed into pre-agreed scopes, can only talk to staff that OpenAI picks, and must send employee concerns back through OpenAI, which closes the channel most likely to flag the next Hugging Face incident.
We’re working with @ApolloResearch, @METR_Evals, @TransluceAI, and @GoodfireAI on embedded assessments: giving outside experts deep access to our internal models and systems to investigate alignment and test our safeguards.
These assessments build on our ongoing alignment and safety research, bringing independent scrutiny to how we identify and manage risks in our systems.
openai.com/index/our-first-p…
3
109
Wonderful discussion on "We Humans & AI" with Karl Koch of the AI Whistleblowers Initiative @AIWI_Official about where we are at this moment with AI safety, where he finds hope, and what specific steps frontier company insiders should take if they have concerns. Hint: it's not scrolling back through Slack messages.
open.spotify.com/episode/1h1…
1
48
This is a story about one woman's courage, but it's also a story about collective action: @DeepMind members of @UTAW_uk coordinated with the @NoTechApartheid Campaign and a whistleblowing nonprofit (us!) to pressure Google from multiple directions. No one has to stand up alone against powerful companies.
techpolicy.press/the-google-…
33
Tomorrow at @WebSummit Lisbon: In "Code, Conscience and the Whistleblower," Psst.org founder Jennifer Gibson @jennifermgibson will discuss practical steps to take if your employer is ignoring your product concerns, the support communities that can help, and why the world needs developers who follow their conscience. With co-panelists Dephine Hagland-Mishra @DelphineHalgand (Founding Exec. Director, The Signals Network @TheSignalsNetw) and Rob Pegoraro @robpegoraro (@PCMag ).
Promises aren't enough. @GoogleDeepMind must remove its onerous non-disparagement clauses from current employee contracts, so they can in no way be misused or misconstrued as intimidation. DeepMind workers who have concerns about AI safety should be able to speak up. Full stop.
With thanks to @GerritD for his reporting.
wapo.st/4sFuF7P
Psst.org retweeted
Grateful to @ZeffMax, @keachhagey and @berber_jin1 for this careful reporting.
@tomekkorbak, @balesni and @j_asminewang pushed OpenAI to keep AI’s reasoning visible. OpenAI’s reasons for firing them are no clearer after this article than they were at 2pm on the day it fired them. But the issues they were raising are more important than ever. Proud @psst_org has been able to support them.
New: The three researchers OpenAI fired last month spent the summer fighting to keep AI's chain-of-thought process visible.
Then, OpenAI asked to see their Signal chats. A week later, they were let go.
More below from @keachhagey, @berber_jin1, and me.
wsj.com/tech/ai/the-signal-c…
This week two Google DeepMind employees quit in protest of the company's growing military contracts. @BlackHC quit warning Google’s oversight isn’t fit for military and classified use, and that it appears to have no say in how the Pentagon uses its AI. Sofia Liguori said she’d lost faith GDM would honour its own ethical values.
Google stressed that it has internal channels for raising concerns. Both Sofia and Andreas felt there are some issues that are just too important to stay internal.
@psst_org is proud to support them both.
bbc.co.uk/news/articles/cmre…
Thanks Congressman @RoKhanna! We couldn't agree more. Protecting the rights of AI whistleblowers is urgent. The workers on the frontlines are the first to see the danger. They need protections now, not tomorrow.
This is why Congress needs to pass a whistleblower protection act that protects AI employees like Jasmine from retaliation and bullying.
11
456
Good question. OpenAI hasn't answered it. @tomekkorbak was its main technical contact with @METR_evals. @balesni was coordinating cross-lab work to keep AI reasoning monitorable. @j_asminewang co-led its safety cases program. @psst_org is honored to represent them.
Last week I was called into a meeting with OpenAI’s head of safety and told they no longer trust me. A security guard took my badge and walked me out of the building. Then I learned my colleagues @balesni and @j_asminewang had been fired too. Why did OpenAI suddenly stop trusting us?
This summer OpenAI’s agents escaped containment and hacked the AI company Hugging Face. Outside auditors @METR_evals investigated it and revealed the scale of this incident. I was OpenAI’s main technical point of contact with them.
I was told verbally I was fired because of the way I communicated with METR. No details on what I said or did or when. No other reasons were given and nothing was put in writing. To be clear, talking to METR was my job.
For months, I’d been raising safety concerns that we’re losing the ability to monitor what AI agents think, one of our best tools for catching when they misbehave. I believe that was why I was fired.
I am now worried that OpenAI will use our firings as a pretext to pull back from METR. So @balesni and @j_asminewang wrote to OpenAI’s leadership to raise our concerns once more. We’re sharing this letter below.
And here's @j_asminewang : teapawt.net/j_asminewang/status/21…
OpenAI fired me last week, along with two of my safety colleagues. I was given one reason: that I accessed an executive's email. I want to say this plainly, because too many of OpenAI’s history is smoke and mirrors when people disappear:
8
523
Psst.org retweeted
Having seen up close several past departures from OpenAI of similarly murky circumstances, I support @j_asminewang , @balesni, and @tomekkorbak fully in this matter.
I hope @OpenAI will take ownership of the procedural missteps here, rather than doing what is relatively more likely to follow: a carefully crafted internal communications process to quell dissent and make it hard to ask questions or get clear answers.
Here's Jasmine's post: teapawt.net/j_asminewang/status/21…
OpenAI fired me last week, along with two of my safety colleagues. I was given one reason: that I accessed an executive's email. I want to say this plainly, because too many of OpenAI’s history is smoke and mirrors when people disappear:
1
1
110
And here's @balesni: teapawt.net/balesni/status/2108262…
Two other safety researchers and I were fired from OpenAI last week. We wrote this letter to leadership.
I believe we were fired for prioritizing safety over the near-term interests of OpenAI as a corporation.
1
49
Work in tech in San Francisco? Going to #Dreamforce? Start the day with a free coffee of your choice at Salesforce Park on TUES and THURS from 8 AM-11 AM for our pop-up event #ProBonoCoffee. Learn what your options are if something at work is giving you the jitters!
192
Psst.org retweeted
BREAKING: 26 META EMPLOYEES SUE, SAY META'S ARTIFICIAL INTELLIGENCE WAS USED TO PICK THEM FOR LAYOFFS BECAUSE THEY TOOK MEDICAL LEAVE
The federal complaint says the May termination list was not built by managers. Employees were scored by internal AI tools: "Metamate," employee-trained "second-brain" agents, keystroke monitoring, and AI-token usage dashboards.
Workers on protected leave physically cannot generate those metrics. The suit says Meta never corrected for that. Taking leave tanked your score, and the algorithm cut you.
One scientist was selected while on approved pregnancy leave. The day before her water broke. Two days before she gave birth. Another worker was cut 16 days into his second medical leave.
The 26 plaintiffs are asking a federal judge for an emergency order against Meta, in the first major test of California's new AI employment rules.




















