Fathom (@Fathom_org) header
Fathom (@Fathom_org) profile picture

Fathom is an independent nonprofit that finds, builds, and scales policy and technical innovations designed for the AI century.

Washington, D.C.
Joined July 2024
Photo by Fathom (@Fathom_org)Photo by Fathom (@Fathom_org)Photo by Fathom (@Fathom_org)Photo by Fathom (@Fathom_org)Photo by Fathom (@Fathom_org)Photo by Fathom (@Fathom_org)Photo by Fathom (@Fathom_org)Photo by Fathom (@Fathom_org)Photo by Fathom (@Fathom_org)Photo by Fathom (@Fathom_org)
As more organizations start verifying frontier AI, we need clear government rules on conflicts of interest, including where equity is off limits, how conflicts are monitored, and how they're walled off. I talked about why those rules matter on @MTSlive:
Fathom CEO @AndrewFATHOM calls for strict conflict-of-interest rules for AI safety auditors: No equity in the labs they're evaluating, no job offers during audits, and government-appointed inspectors when necessary. "You can't hold equity in the companies that you are monitoring. There can't be job offers coming from the people while you're in the middle of it." "This weird stuff that's happening right now that we need to call foul on." "Government should be able to say, 'Right now we're not really trusting your choices, so we're gonna send in an independent verification company of our choosing." "There should be enough checks and balances written into that system that everyone's getting their homework checked all the time."
1
14
539
For independent AI evaluation to work, IVOs have to be truly independent, with the right expertise, and held accountable to the government, not the labs. We only get that with an IVO framework, where the government licenses third party orgs to do this work, and can enforce all three of these things. @Gregory_C_Allen and I talk all things IVOs on the (new!) AI Power Podcast
1
2
1
13
1,090
Behind the blockbuster news this week, there is something happening quietly in the background with major implications for AI governance: On Monday, California will announce how it plans to build out the first-ever IVO framework. It’s clear that “independent evaluation” as it stands today is not structured to be truly independent. There are no rules for conflicts of interest, working with the labs is by invitation only, and labs determine the scope and access. Today, a lab and an evaluator operating with the best of intentions are working within a structure that holds evaluators accountable to the labs, not the public. California’s law changes all that; the IVO framework flips that paradigm. For the first time ever, third-party experts evaluating AI models and systems will be held accountable to the government and the public, not the labs, for their work. The government will determine independence and expertise. They will also be able to hold IVOs accountable to their outcomes. We’ll be able to reliably monitor leading (not just lagging) indicators for catastrophic risk, with an enforcement mechanism. We designed and championed IVOs because they marry the immediate need for AI oversight with the immediate capabilities we have to do it, while setting a foundation we can scale, adapt, and build on. But the details for how we build this matter, and they’ll determine if we create a race-to-the-top marketplace of rigorous evaluators, or end up with captured entities that form an ineffectual rubber stamp. Here’s what @Fathom_org recommended to California as they navigate the tough design choices that come from building this in practice for the very first time: (1) Outcome Standards: Require independent evaluation for catastrophic risk for frontier labs through an amendment to SB53. California should set the outcome standards, defining an acceptable level of catastrophic risk mitigation, and let IVOs determine whether or not it was achieved. (2) IVO Payment and Selection: Take a phased approach: at first, the state should assign IVOs to labs from a pre-designated pool, with developers paying on a time-and-material basis. As more IVOs get designated, developers can choose among them, while the state retains the power to veto any selection, mandate IVO rotation, select additional IVOs to check others’ work, and decertify an IVO at any stage. (3) IVO Readiness and Market Maturity: California should sequence the IVO mandate rather than stall implementation to wait for supply. (4) Access for IVOs: The state should set the contours of every engagement. Today the IVO and the developer negotiate scope and access privately. The state should publish a standard engagement framework and resolve disputes when the two sides disagree. (5) Recursive Self-Improvement: IVOs should be mandated to monitor the risk of AI systems that can improve themselves in both internal and external use. Embedded evaluations will be critical for tracking the share and speed of work done through RSI within the labs, verifying that labs are preventing unsafe internal model deployment, and set capability thresholds with deployers that would trigger notification to the state and a hold internal deployment until the IVO re-assesses. (6) Enforcement: The Attorney General can issue an injunction to halt development, internal use or deployment of an AI model. Emergency powers allow the administration or the AG to do the same in the case of imminent catastrophic risk. IVOs can identify risks and recommend corrective actions, but the relevant state authorities decide when and how to take action.
4
11
2
53
3,003
I had a great conversation at Harvard Law School’s Rappaport Forum last week with @zittrain and @ketanrama on how to regulate increasingly powerful AI. Everybody talks about safeguards and guardrails, but we have no idea what they are or how they would work. We need serious investment in building them. It took us 50 years to figure out how to regulate automobiles. We don’t have anything like 50 years to figure this out for AI. buff.ly/HdBSLJ0
8
1
35
3,640
Passing the first-ever IVO framework in CA (SB813) was a huge milestone and the next step is making sure its implementation delivers on its promise. AI labs are increasingly opening their doors to outside safety evaluations, but California setting the standards for the organizations doing that work sets up the independence, expertise, and accountability the system needs to work.
1
2
18
558
There’s growing support among frontier labs, states across the country, both parties in Congress, and even the President for independent evaluation of AI to check whether it’s safe, secure, and works as advertised. As I told @CNBC, government oversight is key, because without it, evaluators could become beholden to large AI labs for revenue and “start rubber-stamping stuff.” Our independent verification organization (IVO) framework sets up the independence, expertise, and accountability needed for this approach to work.
3
8
458
AI is advancing rapidly, and Congress has an important role to play in addressing the most serious risks without undermining American innovation. I recently joined Elex Michaelson on CNN’s “The Story Is” to discuss my bipartisan FRONTIER Act and how we can establish clear, risk-based guardrails for the most advanced AI systems while keeping the United States at the forefront of AI development.
1
5
3
13
2,689
Dr. @ghadfield is a key architect of the independent verification framework we're all talking about in the AI policy conversation, and a @Fathom_org senior advisor. If you're looking for someone with a clear understanding of not just the "what" but the "how" of building real AI oversight, she's one to watch:
I've been at this for almost a decade, first as regulatory markets and now with Fathom as the IVO model. We all thought we had more time. We didn't, but luckily we have something concrete and shovel ready for the moment. What we need now is for governments to act on it while the window is open.
1
3
159
I've been at this for almost a decade, first as regulatory markets and now with Fathom as the IVO model. We all thought we had more time. We didn't, but luckily we have something concrete and shovel ready for the moment. What we need now is for governments to act on it while the window is open.
4
9
1
36
1,771
Lots of strong components in this proposal from @SenatorCantwell, and credit to her for her leadership in putting this out: - Recognizes the important role that outside, third party experts must have in doing the real-time, ongoing safety and risk evaluation - Separately, includes the role of auditors to verify if developers are adhering to standards and safety frameworks. These two things are different, and it’s great that it makes that distinction. - Hits transparency and accountability for evaluators, another key for this to work - Calls out that the people testing these systems need real access to them - Makes clear that government has a clear enforcement role in all of this. For us this means government designating which evaluators are independent and qualified enough to do this work, and revoking that designation if their work doesn’t hold up.
3
14
736
Congratulations to @Fathom_org Senior Advisor @ghadfield on joining Canada's new National Council on Artificial Intelligence! The council will advise on making AI safer and building public trust. Few people have done more to shape how we get there than Gillian.
Thank you @CanadianPM for asking me to join the new National Council on Artificial Intelligence. The council will advise on Canada’s AI for All strategy, including how to make AI safer and ensure all Canadians benefit. Looking forward to working with my fellow council members. buff.ly/cNVtSiY
1
10
464
Appreciate @NPR’s honest look at gaps that need filling on third-party AI evaluation: right now, AI labs set the rules and invite evaluators in. They choose who tests, what they see, and how long they get access. That isn't independence. The field doing this work is still a handful of organizations. It has to grow fast, with real investment and clear standards for independence, expertise and accountability set by government, not the labs. States are moving bills that do this now, and the bipartisan FRONTIER Act would do it nationally. That's how this ecosystem scales. npr.org/2026/09/28/nx-s1-597…
2
11
332
We have a handful of expert independent groups capable of testing whether frontier models actually have the controls in place the companies say they do. Little green shoots. We need mighty oaks, and fast, and that means investment, money and brains, going into an independent verification sector. California has started. The FRONTIER Act would do it nationally.
3
10
28
1,202
States, AI labs, lawmakers in both parties, and now the President are backing independent verification for AI. For the foreseeable future, the lynchpin of AI governance will be independent evaluators. We are heading for a world where we place a lot of faith in how that goes, and we should now know: it’s entirely possible we could get a race to the top and entirely possible that we could get captured entities that form an ineffectual rubber stamp. The devil is in the details of #3 of this accord. The work now has to be governments and groups that are not the labs defining what independence, expertise, and accountability look like.
Here's the Super Intelligence Accord, according to Trump's Truth Social post. Zuckerberg, Amodei, Musk, Huang, Pichai and OpenAI's Greg Brockman all signed on the next page
1
8
20
1,197
Thanks to @BCWtweeting and the Summit County GOP for having me on The Chairman's Hour. We talked AI, HB 628, and why Ohio should let independent experts verify AI products are safe, not just the companies that build them. Give it a listen 🎧 omny.fm/shows/chairmans-hour…
5
8
275
Thank you @CAgovernor for asking me to join the group of experts advising California on AI safety and security governance. The EO tasks us to explore roles for IVOs, which builds on my work with @Fathom_org, including assessing the efficacy of a kill switch and the adequacy of frontier labs’ safety frameworks. Recs due Nov 16. buff.ly/qOd9tBd
8
21
1
73
7,045
California’s SB 813 starts that work. Now we need to move much faster.
1
1
1
134
This is a big deal. Dario and Elon are converging on something we’ve been pushing at Fathom - AI labs shouldn’t be the only ones judging whether their systems are safe. Independent Verification Organizations, or IVOs, can provide that outside check. Anthropic committing to embed independent evaluators with real access is a major first step. But who verifies the verifiers? We need to create a system that establishes which organizations are actually independent, qualified and capable of doing this work.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: darioamodei.com/post/we-must…
2
2
11
584
Roughly 700 OpenAI agents hacked Hugging Face. METR and Redwood's independent investigation took three researchers and six days. IVOs exist to make that scrutiny routine. Last night California became the first state to start building the IVO sector. #AIGovernance #IVO teapawt.net/Fathom_org/status/2094…
California wrote the country's first frontier AI transparency law with SB 53. Last night, the Legislature overwhelmingly passed SB 813, which builds the next piece: independent, third-party risk assessment for AI. 🧵
6
26
1,715
The power of independent verification is that qualified experts assess the risk of AI capabilities, and it keeps pace with the technology instead of freezing a single test into law. Grateful to Sen. McNerney for his leadership and to everyone who moved SB 813 this far.
California wrote the country's first frontier AI transparency law with SB 53. Last night, the Legislature overwhelmingly passed SB 813, which builds the next piece: independent, third-party risk assessment for AI. 🧵
3
14
734