SecureBio is a research non-profit that works to advance biotechnology safely and prevent catastrophic pandemics
Joined January 2025
- Tweets232
- Following1
- Followers1.1K
- Likes12
SecureBio retweeted
Today, more than 100 leading AI experts endorsed a set of minimum requirements to take seriously AI companies' recent call to embed external evaluators.
These evaluators need to be genuinely independent, transparent, and represent a range of expertise areas. They also need to be guaranteed employee-level access and to be protected from retaliation for findings that make companies look bad.
We welcome model developers’ recent calls for independent oversight, but it’s what they do next that matters. The labs must be accountable for ensuring these requirements are met, so that the public can have faith in the process and the outcomes.
Over the past week, the AI community has debated the appropriate role of external evaluation, including who should do it and on what terms. We may not agree on everything, but there is a lot of common ground.
To make embedded evaluations credible, more than 100 experts with varying backgrounds and ideas about AI risk agree in today’s letter that frontier AI developers should:
1. Guarantee embedded evaluators full editorial independence and mitigate conflicts of interest
2. Rely on multiple evaluators with differing viewpoints and areas of expertise
3. Publicly document the terms under which evaluators operate, as well as facilitating permissive publication of methods and findings
4. Shield evaluators from retaliation
5. Grant access equivalent to that of highly privileged employees
There is a thriving and growing ecosystem of independent AI evaluators who are advancing this science every day – but we need aligned standards, guaranteed protections, and independent funding. That’s why we created the AI Evaluator Forum.
Today we are entering our next phase. We’re launching an open call for new members, collaborators, and independent funding sources to help evaluators meet this moment and demand accountability from developers. Join us in building the evaluator ecosystem.
See the public letter here: aievaluatorforum.org/initiat…
Learn more at aievaluatorforum.org/path-ah…
Anthropic and OpenAI need truly independent safety evaluators, experts say in public letter cnbc.com/2026/09/18/ai-safet…
SecureBio retweeted
A non-exhaustive list of more orgs doing frontier eval work (I make no claims in either direction re their engagements with labs): @ApolloResearch @farairesearch @SecureBio @Irregular @ActiveSiteBio @redwood_ai @SaferAI_org @ValsAI @EpochAIResearch @AVERIorg
AI Evaluator Forum has released AEF-one, Minimum Operating Conditions for Independent Third Party AI Evaluations: aef.one/aef-one.pdf / aievaluatorforum.org/
Guidelight AI has rolled out assessment standards: guidelight.ai/
@Fathom_org is working on building the auditing ecosystem, including through the newly launched @pact_ai_org
So many hands on deck! So many more needed!
SecureBio retweeted
Very glad to see government officials at the highest level taking the threat of AI-engineered bioweapons seriously
NEWS from my @WSJopinion interview with @SecScottBessent
Treasury officials say that at the Trump-Xi summit in Beijing next month, history will be made. For the first time ever, the leaders of earth’s two greatest powers will discuss AI as an agenda item. Messrs. Trump and Xi will look for areas of mutual cooperation and explore ways to work together on security and threats from nonstate actors.
Full interview here:
wsj.com/opinion/scott-bessen…
@lizalinwsj @stuartlauchina @danstrumpf @yuanli233 @paulmozur @trippmickle @kateconger @cade_metz @evadou @christian_shep @Cat_Zakrzewski @GerryFShih @DemetriSevast @EleanorOlcott @AnnaNicolau @tabbyleung @parmyolson @daveyalba @jackiewattles @iansking
SecureBio retweeted
Glad to see AI bio-risk entering the public consciousness. It really is existential.
Between Mythos, Anthropic v Pentagon, and the drumbeat of biosecurity concerns, I think we're entering a period where
(a) AI as a consumer product (chatbot + coding assistant) will continue to behave like a "normal" technology—a very powerful tool that does all sorts of useful things alongside workers without imminently wiping out tranches of the labor force; while...
(b) as bio-/cyber-/national security threat, AI is becoming something else entirely, a sort of hydra of existential risk that's going puncture ppl's confidence that "AI is a normal technology" and force both the labs and the federal govt to establish rules on the fly, while open models race to catch up to the frontier
SecureBio retweeted
Today we are announcing the creation of the AI Evaluator Forum: a consortium of leading AI research organizations focused on independent, third-party evaluations.
Founding AEF members: @TransluceAI @METR_Evals @RANDCorporation @halevals @SecureBio @collect_intel @Miles_Brundage
SecureBio retweeted
8) SecureBio @SecureBio
Remember the pandemic? That sucked...
We're doing MUCH less than we should be to prepare for the next one, but we do have a few heroes out there doing early detection wastewater monitoring 🙏
xitter.cf/Simon__Grimm/status/19…
SecureBio retweeted
How can we verify that AI ChemBio safety tests were properly run?
Today we're launching STREAM: a checklist for more transparent eval results.
I read a lot of model reports. Often they miss important details, like human baselines. STREAM helps make peer review more systematic.
SecureBio retweeted
AI might increase biological risks substantially.
But it could also transform our ability to detect pathogen outbreaks.
In a new piece, I lay out how the US government can generate the data necessary for AI-enabled pathogen detection.
THE LAUNCH SEQUENCE
A new collection of 16 essays on how to accelerate AI for science & security: ifp.org/launch
The AI revolution is already delivering enormous consumer benefits. But AI progress won't automatically solve humanity's most important problems first. To get the future we want, we need to shape the trajectory of AI progress.
This series is a step toward that future…
