Build your own AI agent harness in plain Python. Tools are functions, every run is replayable, no YAML or DSL. Open source, MIT.
Joined November 2026
- Tweets19
- Following18
- Followers15
- Likes3
Pinned Tweet
Your agent's model will change every few months. The harness around it shouldn't have to.
Railtracks is an open-source Python framework for building your own agent harness: the loop, tools, context, controls, and a replayable record of every run.
Plain Python. MIT.
1/ Your billing support bot will happily write someone's cover letter, and you pay for the tokens.
New in Railtracks: ScopeGuard 🧵
Describe what your agent covers in one sentence. A small judge model checks every request against it before your agent's model runs.
Every call the judge makes is in the trace, label and reason included.
This one cost $0.00006 (326 tokens in, 59 out), and the billing agent's own model was never called.
Three sensitivity levels:
LOW: only labels. Run it before launch to see what would get blocked
MEDIUM (default): blocks OUT_OF_SCOPE
HIGH: also blocks UNKNOWN
If the judge goes down, requests go through by default.
docs.railtracks.org/document…
Railtracks retweeted
If you're building agents in python, try out railtracks! we're using it on several agent builds!
GitHub: github.com/RailtownAI/railtr…
Docs: docs.railtracks.org
> pip install 'railtracks'
Not every step in an agent needs an LLM to write text. Sometimes you just need a decision.
New in Railtracks: rt.decision_node 🧵
Ask questions about some text and get a calibrated probability for each answer. One request, usually under a second.
In `railtracks viz --beta` it sits right next to your agents:
🤖 Agent: the full conversation, tool calls and every LLM call
🔧 Decision: text in, answers with probabilities out, plus tokens and cost.
It’s also a tool. Put it in an agent’s tool_nodes and the visualizer shows it under the agent, with what it saw and what it cost.
Need content → agent_node
Need a judgment → decision_node
docs.railtracks.org/document…
Agent observability shouldn't need Docker Compose and a cloud account.
Railtracks ships with a visualizer. Free, local, no signup, one command:
Every run: tool calls, LLM calls, tokens, cost, latency, and exactly what the model saw.
Thanks for the shout-out, Dan 🙏
That's exactly why we built Railtracks: when an agent misbehaves, you should be able to see the loop, the tools and the context, not dig through framework internals.
We're on 1.8 now. More coming.
Your agent framework shouldn’t hide the harness you need to debug.
Railtracks is a Python agent framework for developers building custom agents and harnesses.
It helps you assemble tool-calling loops, tools, context, controls, and a record of every run with ordinary Python objects instead of YAML, DSLs, or a black-box runtime.
Key features:
• Plain Python assembly – express execution order, branching, and loops with standard Python control flow
• Function-based tools – turn functions into tools and connect sub-agents or MCP servers
• Context and memory options – configure system messages, retrieval, todo lists, and key-value memory toolsets
• Built-in controls – add call limits, timeouts, retries, locks, guardrails, and human approval steps
• Local run inspection – use the visualizer to inspect agent runs and evaluations locally without signups
It’s open-source (MIT license).
Link in the reply 👇
GitHub: github.com/RailtownAI/railtr…
Docs: docs.railtracks.org
> pip install 'railtracks'