sanjana (@sanjanayed) header
sanjana (@sanjanayed) profile picture

@openai safety!

Berkeley, CA
Joined March 2025
Photo by sanjana (@sanjanayed)Photo by sanjana (@sanjanayed)Photo by sanjana (@sanjanayed)Photo by sanjana (@sanjanayed)Photo by sanjana (@sanjanayed)Photo by sanjana (@sanjanayed)Photo by sanjana (@sanjanayed)Photo by sanjana (@sanjanayed)Photo by sanjana (@sanjanayed)Photo by sanjana (@sanjanayed)
klein’s at sfo is good
3
112
sanjana retweeted
Welcome to the real world. Today we’re introducing FrontierEE Bench, the first benchmark to test frontier models on electrical engineering tasks at a physical lab bench. Real-world engineering benchmarks should involve real hardware. At our lab, models control instrumentation, run experiments on physical hardware, and decide what to test next. The best models scored 36.7% across a range of EE domains including mixed-signal circuits, power systems, & electromechanics.
84
53
12
492
36,748
sanjana retweeted
We’re working with @ApolloResearch, @METR_Evals, @TransluceAI, and @GoodfireAI on embedded assessments: giving outside experts deep access to our internal models and systems to investigate alignment and test our safeguards. These assessments build on our ongoing alignment and safety research, bringing independent scrutiny to how we identify and manage risks in our systems. openai.com/index/our-first-p…
672
308
347
3,248
1,096,820
does anyone have a robot I can hangout with this weekend?
1
3
375
tech with Hannah strikes again
Replying to @sanjanayed @OpenAI
hit the download canon powershot g7x mark ii photos button in the passenger seat of the prius otw home 😋😋 also for vis no action needed @sanjanayed
1
4
498
this eval is so fun
At Blood on the Clocktower, Fable 5.2 is still worse than GPT-6 Astra. Though much better than Opus 5.5 or Fable 5.1
3
201
I spent 20,000 dollars in API credits having different AI models play each other at Blood on the Clocktower (a Mafia or Werewolf type social deception game). The results were fascinating. Models have gotten much better over the course of this year at creative on-the-spot scheming and bluffing, and also at reasoning as a team to break apart these underground conspiracies. I go through some of the most standout games these AIs played in this video. The new models are getting pretty good. If my friends pulled off some of these moves while we were playing, I’d give them a fist bump. Fwiw, OpenAI’s newer models are much better than Anthropic’s at this game. You can see the full benchmark at the website, and I’ve also made it so that you and your friends can play again the AIs: clocktowerbench.com
189
181
134
3,150
654,915
blows my mind that there is no frozen yogurt shop in San Francisco
91
7
10
338
52,542
sanjana retweeted
We’ve updated the Model Spec, our public description of how we want our models to behave. This revision makes clearer what we want from models as they take on more autonomous work in the world and become capable of helping us in new ways. 🧵 openai.com/index/updated-mod…
89
103
73
824
213,404
extra beautiful today
4
200
I have to know what this sounds like
Do a kickflip!
2
150
🧵 Excited to share the first batch of 6 misalignment reports from OpenAI's new disclosure process for misalignment incidents. We want to be more transparent about the misalignment we see during training, evals and deployment, this is an important step in that direction.
39
79
20
647
70,821
thank you anthropic for sharing! important information for the world here. eg: a freelance threat actor tried using frontier AI to engineer and test a coordinated drone swarm that could select a human target and detonate without human oversight more examples in their report
We're publishing our most detailed threat intelligence report to date. It covers how people tried to misuse Claude—for cyberattacks, influence operations, surveillance, biology, and building weapons—and how we found and stopped them. We disrupted every operation in the report, and used the lessons from them to strengthen our safeguards. Where appropriate, we also shared what we found with authorities and other AI companies. These cases are not typical: we’re highlighting some of the most sophisticated misuse we’ve seen. But they’re especially important to discuss, because they show us where AI misuse is headed, where our safeguards work, and where they need to improve. We’re publishing this report so others can spot the same activity on their own platforms, and so we can give the public a clearer view of how emerging threats develop. Read the report: anthropic.com/threat-intelli…
5
7
146
16,604
the super important ground breaking thing that happened at work today was that they stocked up cheez its in the micro kitchen
6
232
💫💫💫
This is GPT-6 Astra. Anything you can do on a computer, Astra can do for you. Fast.
2
356
sanjana retweeted
Introducing GPT-5.5 A new class of intelligence for real work and powering agents, built to understand complex goals, use tools, check its work, and carry more tasks through to completion. It marks a new way of getting computer work done. Now available in ChatGPT and Codex.
2,436
6,627
3,415
50,653
13,434,193
In the coming weeks we'll be enabling basic telemetry by default in Phoenix over the coming weeks. Our commitment: privacy and security are foundational What we're collecting: Simple, anonymous usage stats Opt out: Set PHOENIX_TELEMETRY_ENABLED=false—no questions asked. github.com/Arize-ai/phoenix/…
3
2
8
309
sanjana retweeted
In case you missed it: we gathered with builders in NYC last night to dive into what really moves the needle after an agent is deployed. A lot of progress starts in production. This means learning from how your agent behaves in the wild and feeding those findings back into your development pipelines. We walked through how continuous evaluation surfaces failures, closes the feedback loop, and turns real insights into a better performing agent. With Arize AX, online evals and real-time tracing makes every production run part of an iterative cycle. This helps teams ship smarter agents that are consistently evolving. Check out the video recording of the session: youtu.be/qF2XQ3WSyrE
4
2
3
491