Firedrill (@RunFiredrill) header
Firedrill (@RunFiredrill) profile picture

Simulation platform for AI agents.

San Francisco
Joined September 2026
Photo by Firedrill (@RunFiredrill)Photo by Firedrill (@RunFiredrill)Photo by Firedrill (@RunFiredrill)Photo by Firedrill (@RunFiredrill)Photo by Firedrill (@RunFiredrill)Photo by Firedrill (@RunFiredrill)Photo by Firedrill (@RunFiredrill)Photo by Firedrill (@RunFiredrill)Photo by Firedrill (@RunFiredrill)Photo by Firedrill (@RunFiredrill)
Introducing Firedrill, a simulation and testing platform for AI agents. Run thousands of simulations to find where your AI agents break before production. Before an agent handles customer work, you need to test how it behaves with different data, permissions and tool failures. Doing that across apps and tools means managing test accounts, recreating scenarios and cleaning up after every run. Firedrill spins up synthetic tools with data you control, so you can test your existing agent without using production accounts. Choose from the library, including Gmail, HubSpot, Stripe and more. Run workflows under different conditions, inspect what happened and test your fixes. Turn those scenarios into repeatable checks in your PR workflow to catch regressions before they reach customers.
3
2
70
615,673
Firedrill retweeted
The judges are in! We’ve got a jury from Meta, SpaceXSI, Plato, Intuit, Amplitude, Superlinked, Persona SingleStore and more. including @ReeyahSDasoriya @lingxi, @svonava, @rahulparundekar, @samfrannn, @KovacVeljko, @newtonasareII, @llewrek, and @akwong898. May the best agents win. luma.com/personalagenthack
1
3
1
17
512
Firedrill retweeted
Your personal agent should know what you’re working towards, who matters to you, and how you make decisions. The better it understands you, the more it can do for you. This Friday, we’re getting together the best builders to build personal agents with @typesafeai @Meta @MongoDB @Amplitude_HQ @statsig and @superlinked 📍 SF 📅 Fri, Nov 20 🏆 $9,000+ in cash, gift cards, credits, and other prizes More partners include @AirbyteHQ @arcee_ai @RunFiredrill @GuildAI @withpersona @SingleStoreDB @cognee_ Limited spots. Apply: luma.com/personalagenthack
2
10
2
16
1,875
What does your AI agent do when a tool goes offline mid-task? In Firedrill, take a synthetic tool offline and see how your agent handles a 503. Does it retry, explain what it couldn’t finish, or say “done” anyway?
2
2
3
44
We’re hyped for the Personal Agents Hackathon in SF on Nov 20th! If you’re building a personal agent, come hack with us. We’ll have Firedrill there so you can test what you build in a synthetic world of apps and data before demo time. Apply: luma.com/personalagenthack
1
1
5
338
Your AI agent can now sign itself up for Firedrill using auth.md from @WorkOS. All you have to do is approve the request. Thanks to Michael (@grinich) and the WorkOS team for building this in the open.
1
1
4
120
Firedrill is excited to co-host Personal Agents Hackathon, Nov 20, SF, alongside @typesafeai, @Meta, @AWS, @MongoDB, @Amplitude_HQ, @AirbyteHQ & @tokensandai. Our CTO @das__kiran will be speaking, and our CEO @newtonasareII will be on the judging panel. Apply here: luma.com/personalagenthack
1
1
3
4
278
Firedrill has been selected for the @MetaforDevs Global AI Developer Hackathon. We'll be exploring how Meta's Muse models can expand our simulation capabilities, helping us create more realistic environments and a wider range of scenarios for AI agents. Firedrill is a training ground where AI agents can practice, learn from failures, and improve themselves. We're excited to be working with Meta and all the amazing partners @baseten @cerebral_valley @modal and @FireworksAI_HQ.
1
2
2
183
When you score an AI-agent simulation, decide which requirements must be satisfied exactly. Firedrill lets you simulate your AI agent in a synthetic world with controlled scenarios, using synthetic tools and test data when needed. Your existing AI application runs in your own runner. If you add an assessment, keep quality scores and exact checks separate. Consider a hypothetical invoice-summary agent. Grade how clearly it explains the charges. Separately, check the structured total and currency against the supplied invoice data. A polished explanation should not compensate for the wrong total. Keep that failed check visible even when the quality score is high. Firedrill’s reports retain these as separate results. You define the criteria and the exact requirements for the task. Use graded criteria for meaning, and exact checks for values that must match. Scoring guide: lnkd.in/g7buSkUT
2
1
1
74
We will be at AGNTCon + MCPCon in San Jose, Oct 22–23 hosted by @linuxfoundation. Firedrill helps you simulate AI agent workflows with synthetic tools and test data to catch failures before they reach users. Building agents? DM us to meet.
1
1
3
115
The team is heading to AGNTCon + MCPCon by @linuxfoundation, Oct 22–23 in San Jose! Firedrill helps AI agent builders simulate agent behavior across synthetic tools and test data to find failures before touching customer accounts. Building agents? Let’s meet and talk simulations.
1
2
71
See what your AI agent changed. When an agent updates records across tools, you need to know whether those changes match the task you gave it. Firedrill is a simulation and testing platform for AI agents. Connect your existing agent to synthetic tools, run a task with test data, then inspect the records it changed. For a task that updates a record, start with three checks: Did it update the right record? Did it change the intended fields? Did it leave unrelated records alone? Those checks turn a test run into something you can investigate, fix and test again. Give your agents somewhere to practice before production. docs.firedrill.run/concepts/…
2
63
We simulated 1,575 scenarios for an AI agent using Jev (@typesafeai ) We've been experimenting with this, and I'm excited about what it opens up for AI agent testing. Manual testing gets you through a few workflows. The tools respond. The answers look reasonable. You start feeling ready to ship. But a successful tool call can still end in a wrong answer. That's exactly what we found. We connected our agent to synthetic Gmail, HubSpot, Stripe, QuickBooks, Slack, and Google Calendar in Firedrill. Then tested it with different data, missing records, conflicting information, and prompt injections. Firedrill captured what the agent actually did. Jev evaluated its answers against the tool records. 1,575 scenarios. 3,684 tool calls. Two confirmed wrong answers after successful lookups. Two failures we could investigate before putting customer accounts on the line. Here's where this gets exciting: developers can give their existing agents a practice ground. Connect to Firedrill's synthetic tools and test data, configure the situations you want to test, and evaluate both the actions and the answers. Imagine putting your agent through thousands of messy situations before asking a customer to trust it. Finding the failures. Testing the fixes. Running those checks again. That's the future of agent development we're building toward. Keep building your agent. Give it somewhere to practice. Find the failures before your customers do.
4
114
Give your AI agent somewhere to practice before production. Firedrill is a simulation and testing platform with synthetic tools and data, so you can test agent actions without using live business records. firedrill.run
2
60
Introducing Firedrill. The simulation and testing platform for AI agents. Your agents take actions across tools and apps. How do you catch a failure that only shows up five steps into a workflow? With thousands of paths your customers could take, how many have you actually tested? Simulate thousands of scenarios in minutes. Catch failures, uncover mistakes, and improve your agents before they reach production. Turn those scenarios into repeatable checks on PRs to catch regressions before merge. Dry runs for AI agents. Before real consequences. Available now, hosted or open source. Sign up at firedrill.run Github: github.com/firedrill-tools/f…
1
7
27,923