Measurement science for trustworthy AI in high-trust domains. Benchmarks and open research on evidence, authority, human control, and accountability.
United States
Joined June 2026
- Tweets24
- Following25
- Followers37
- Likes12
Pinned Tweet
AI can pass the bar exam. Can it be trusted with grief, confession, or a crisis of faith?
Churches need real research to decide how and whether to use powerful AI systems. We're building the research lab to do that.
Introducing fideai.org
This is our story 🧵
Fide AI retweeted
🎤 As AI systems become more capable, who or what can reliably judge whether their behavior is correct, safe, and aligned?
In this episode of the @HumansofAIPod, I sit down with @shubadubadub and @joshnotjacob, co-founders of @SampuraResearch, an independent nonprofit exploring how humans and AI can work together to oversee increasingly powerful systems.
Rishub and Josh explain why better AI judges are central to the future of alignment. Many of the hardest behaviors to evaluate are subjective, ambiguous, or difficult to verify and neither humans nor AI systems are reliable enough to handle every case alone.
Sampura’s research on "human–AI complementarity" asks how the strengths of each can be combined to create more trustworthy evaluations, stronger benchmarks, and oversight methods that continue to work as models improve.
We discuss what meaningful benchmarks for AI judges should look like, where human judgment remains essential, and how scalable oversight connects to the larger challenge of robust alignment.
Rishub and Josh also share why they left Google DeepMind to start Sampura, how they are shaping the organization’s culture and research agenda, and the unexpected operational realities of building an independent research lab.
@FideAILabs is happy to sign this open letter calling for minimum requirements for truly independent evaluators to be taken seriously by AI orgs
We look forward to collaborating with a growing bench of amazing evaluators across multiple disciplines, worldviews, and perspectives!
Today, more than 100 leading AI experts endorsed a set of minimum requirements to take seriously AI companies' recent call to embed external evaluators.
These evaluators need to be genuinely independent, transparent, and represent a range of expertise areas. They also need to be guaranteed employee-level access and to be protected from retaliation for findings that make companies look bad.
We welcome model developers’ recent calls for independent oversight, but it’s what they do next that matters. The labs must be accountable for ensuring these requirements are met, so that the public can have faith in the process and the outcomes.
Over the past week, the AI community has debated the appropriate role of external evaluation, including who should do it and on what terms. We may not agree on everything, but there is a lot of common ground.
To make embedded evaluations credible, more than 100 experts with varying backgrounds and ideas about AI risk agree in today’s letter that frontier AI developers should:
1. Guarantee embedded evaluators full editorial independence and mitigate conflicts of interest
2. Rely on multiple evaluators with differing viewpoints and areas of expertise
3. Publicly document the terms under which evaluators operate, as well as facilitating permissive publication of methods and findings
4. Shield evaluators from retaliation
5. Grant access equivalent to that of highly privileged employees
There is a thriving and growing ecosystem of independent AI evaluators who are advancing this science every day – but we need aligned standards, guaranteed protections, and independent funding. That’s why we created the AI Evaluator Forum.
Today we are entering our next phase. We’re launching an open call for new members, collaborators, and independent funding sources to help evaluators meet this moment and demand accountability from developers. Join us in building the evaluator ecosystem.
See the public letter here: aievaluatorforum.org/initiat…
Learn more at aievaluatorforum.org/path-ah…
What should people of faith demand from AI?
Whose teachings does it represent? When should it involve a person? Who can correct it when it gets something wrong?
We offer six questions communities can bring to AI providers to make AI more trustworthy:
fideai.org/insights/what-fai…
Fide AI retweeted
Re: Embedded Evaluators, this is exactly what we're building with @FideAILabs and can collab with the frontier labs like @AnthropicAI to help pace the frontier!
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
Fide AI retweeted
🎤 How is AI reshaping not only what we do, but who we are?
In this episode of @HumansofAIPod, I chat with Andrew McLuhan (@amicusadastra), a poet, writer, researcher, and the founder and director of The McLuhan Institute (@McLinstitute), where he develops and teaches ways of analyzing the nature of technologies and their personal and social effects.
As the grandson of renowned media theorist Marshall McLuhan and the son of Eric McLuhan, Andrew brings a perspective shaped by three generations of thinking about how media and technology transform human life.
Andrew explains why artificial intelligence should be understood as more than just another tool. AI is becoming part of the environment around us, reshaping how we think, create, communicate, trust information, and understand ourselves.
We unpack Marshall McLuhan’s famous idea that “the medium is the message,” why focusing only on the content produced by AI can distract us from its deeper effects, and what earlier shifts—from oral culture and print to radio, television, and the internet—can teach us about the present moment.
We also explore how AI overviews and agents are changing search, discoverability, and the economics of publishing; why the risks of AI extend beyond automation to mental fatigue, weakened judgment, trust erosion, and value misalignment; and how every technology strengthens certain human abilities while allowing others to fade.
Andrew offers a practical way to evaluate new technologies against our personal, family, and community values and asks which parts of ourselves we should be unwilling to outsource.
Andrew also shares his unconventional path from punk rock, poetry, and furniture upholstery to carrying forward the work of Marshall and Eric McLuhan. The result is a wide-ranging conversation about media, culture, identity, consciousness, and the question underneath nearly every debate about AI: not only “What can this technology do?” but “What is it doing to us?”
🤖 Made with AI
Fide AI retweeted
The question we should be asking more towards AI is not just “What can this tool do?” but “What kind of person am I becoming while I use it?”
🎙️ What is the future of learning? And how do we balance using AI and human wisdom?
In this episode of the @HumansofAIPod, I chat with Elijah Rivera, an Assistant Professor of Computer Science at Brandeis University, whose research looks at how people learn computer science, and at how programming languages themselves are designed.
Elijah earned his Ph.D. at Brown University, and before that a bachelor's and master's at MIT. At Brandeis, he developed and teaches a course on automation in software development that gives students hands-on experience with the modern software toolchain, while also rooting those tools in their respective histories and in the related questions philosophers of technology have continually raised.
Overarching this is a broader conviction Elijah keeps pursuing: that computer science is best served by learning to communicate effectively with other disciplines such as cognitive science, philosophy, and theology, and, more importantly, by being willing to listen to their challenge and critique.
Whether you’re a student, an AI engineer, founder, researcher, or just someone curious about the future of education, I hope you learn as much as I did from my talk with Elijah.
Fide AI retweeted
Good to see more independent research studying how AI responds to mental health crises.
@TransluceAI was able to get anonymized data from @OpenAI and @AnthropicAI on how users talk with ChatGPT and Claude specifically on queries related to mental health support.
I appreciated their separation of studying the harness (the chatbot) vs the raw model (the API) which is very much the methodology we take in the "When AI is Your Pastor" paper (fideai.org/research/fmg-benc…).
Looking forward to reading through the results and seeing what lessons and future research can be done through @FideAILabs
Glad we could collaborate with @apologist_ai on this important work!
Generative AI models misquote Scripture. This is not a defect that better training or better access to Bible translations will resolve. The solution is not a better model. It is a better harness. Read our joint study with @FideAILabs: apologistproject.org/scriptu…
Fide AI retweeted
🎙️ How can AI be more humane?
In this episode of the @HumansofAIPod, I chat with Erika Anderson (@ErikaOnFire) the founder of Building Humane Technology and co-creator of HumaneBench — an open-source benchmark that asks the question most of the AI industry isn't: not whether AI can perform, but whether it treats you like a person.
As a former UN press correspondent and writing professor, Erika combines technical rigor and humanistic grounding to the field of AI accountability.
Find the podcast on:
YouTube: lnkd.in/gSwu_W46
Spotify: lnkd.in/gNrcWrvu
Fide AI's newsletter is live!
Serving as a companion to the main site, the newsletter breaks down research in plain language, features behind the scenes interviews with the researchers, and offers practical guidance for people navigating AI today.
fideai.substack.com/p/introd…
This is one of the core ideas behind @FideAILabs.
The issue is not that AI models know nothing about theology. Many of them are surprisingly competent in the abstract.
The deeper problem is that theological fluency is not the same as pastoral discernment.
We'd love to connect with people working in:
- AI evaluation and benchmark design
- theology, philosophy, ethics, or moral psychology
- human review and expert judging workflows
- ministry technology and faith-based AI
- institutional governance and trust
If you want to contribute, advise, review, sponsor, or help shape this direction, reach out at [email protected]
For God's Glory.
fideai.org