Production AI Agents & LLM Evals | Father of 3
Austin, TX
Joined February 2009
- Tweets4.1K
- Following469
- Followers664
- Likes26.7K
Price: $200K
Available for preorder
The full demo: teapawt.net/Tesla/status/211089724…
People of @pidotdev
You can do this with pi and this extension pi.dev/packages/pi-you-shoul…
@pidotdev with several extensions:
Model Providers
- **pi-xai-oauth** — xAI OAuth provider with an account-specific Grok model catalog
- **pi-cursor-provider** — Cursor models via OAuth and a local OpenAI-compatible proxy
- **pi-codex-fast** — Fast and Ultrafast service tiers for supported OpenAI requests
- **pi-claude-bridge** — Claude Code (Agent SDK) as a model provider, plus an AskClaude tool
Context and Compaction
- **pi-openai-server-compaction** — Codex-style server-side compaction for OpenAI models
- **@monotykamary/pi-vcc** — algorithmic conversation compactor with transcript-preserving summaries, no LLM calls
- **pi-context-view** — visualize context usage and inspect hidden parts: base prompt, tool defs, extensions
Delegation and Autonomy
- **pi-subagents** — single-agent delegation and scripted multi-agent workflows
- **pi-fusion** — architect, analyst, and builder workflows built on pi-subagents
- **pi-boomerang** — token-efficient autonomous task execution with context collapse
- **pi-goal** — autonomous goal DAG with dependency chains, steering, and queue management
Autonomous Research
- **pi-autoresearch** — autonomous experiment loop: run, measure, keep or discard
- **pi-autoresearch-studio** — dashboard, plan editor, and PR workflow for autoresearch sessions
Search and Web Access
- **pi-web-access** — web search, URL fetching, GitHub repo cloning, PDF extraction, YouTube understanding
- **@ff-labs/pi-fff** — fuzzy file and content search
UI and Interaction
- **pi-you-should-know** — on-demand private session briefing with follow-up questions via `/ysk`
- **pi-open-tui** — polished TUI: animated header, status footer, rounded editor
- **pi-side-chat** — fork the current conversation into a side chat
- **@narumitw/pi-btw** — `/btw` command for quick side questions
- **@howaboua/pi-codex-imagegen** — Codex image generation and editing
My AI Shed is a Beelink AI PC (128GB) running Omarchy 4.
It has Hermes as Code Factory with 3 bots (Chief, Developer, Reviewer) and Herdr + Collie + Herdr Projects running Pi sessions.
I can securely connect to it via Tailscale from any device and control it from my phone.
I can do stuff with my own harness that any commercial harness either blocks or limits.
And remember:
Not your keys, not your ...
Not your harness, not your agent.
Give your eyes a break while working on Omarchy.
I recommend taking a break every 45 min for at least 1 min.
plugins.omarchy.org/plugin.h…
github.com/dpaluy/omarchy-lo…
“Pencils down” at @37signals
DHH and his team joined the revolution of no human written code anymore.
Welcome to the club!
#railsworld
My Rails World keynote will be streaming in a few hours on YouTube. I dare say this one might ruffle a few feathers, but it also just needs to be said. youtube.com/live/V9SxpJpHuus…
People of @pidotdev
Jev Compact reduces the context sent to your Pi model.
I recommend set `jev-prune on` by default in your config.
github.com/dpaluy/pi-jev-com…
Congrats @paolino
RubricLLM now supports RubyLLM 2.0
github.com/dpaluy/rubric_llm
RubyLLM 2.0 is the biggest release of my career.
Don't believe me?
Check out the release notes.
github.com/crmne/ruby_llm/re…
Realtime control
Use case: Many decisions/sec over a structured state (mostly games)
- Doom bot typesafe.ai/blog/introducing…
- Rocket-dodge every ~330ms
teapawt.net/atomic_chat_hq/status/…
Jev v1.13 dodges rockets with probability calculation 🚀
@typesafeai's new non-LLM model returns decisions instead of text so we had it calculate a safe tile every 330 ms while rockets fell and it survived 25 of 26 for under a cent
Run Jev via API -> atomic.chat
Tutorial teapawt.net/moritzkremb/status/210…
The best explanation of Jev by @typesafeai software process is:
BEFORE AI
"Tell the computer exactly what to do."
It has a complex maze of rules.
AI AGENTS
"Tell AI the goal and let it figure out what to do."
It has autonomous AI navigation tools.
JEV
"Let software ask AI whenever it needs judgment."
You can keep your software deterministic and ask Jev when judgment is required
Production classifiers & safety gates
Use case: Allow/deny, route, score on every request.
- Vercel classifier eval teapawt.net/cramforce/status/21002…
- CLI Safety reviewer with gpt-5.6-luna
teapawt.net/rauchg/status/21003079…
We're seeing extraordinary results from @typesafeai. Default mode in 𝚏𝚡 is auto, with a safety reviewer analyzing every command.
That reviewer runs on GPT Luna today. Jev is up to 18x faster (p95) *and* more accurate. It's coming to @vercel AI Gateway and likely new default.
LLM-as-judge / grading / verification
Use case: Judging is choice + confidence, not writing.
- @danshipper share vibe check judge teapawt.net/danshipper/status/2099…
- Verification @goodstartlabs
teapawt.net/alxai_/status/20999331…
Awesome to see @EGafni & Diogo launch their first model Jev!
Verification is the bottleneck for building great AI learning environments. Jev is a welcome, unique, addition to our toolkit @goodstartlabs
Typed decisions,
probabilities for each option,
free output tokens!
On our sample:
• Jev agreed with Fable 5.1 nine times out of ten
• ~200× lower grading cost
• <0.5 seconds per grading call
That means more eyes on what agents are doing and what we’re teaching them.
More on how we’re using it to build game environments, experts to play them, and ensuring they teach the right things:
goodstartlabs.com/research/v…
