Know someone whose team could beat Proof of Quality? If you refer the team that wins, we’ll pay you $1,000. Ask them to put your name and email on their application.

Find the rules here: sapien.io/challenge

One question has shaped much of what we’ve built at Sapien: where can human judgment have the most impact on AI? When we started, people were doing much of the work of creating and labeling data. After two years and 195 million tasks, we learned how difficult it is to measure quality reliably at scale. As AI gets better at doing the work itself, the role of human expertise becomes even more important. People need to define what good means, decide which tradeoffs are acceptable, and retain responsibility for the decisions that deserve human judgment. Where we think human judgment goes next, and why this question is central to Sapien’s long term direction: sapien.io/blog/poq-where-hum…

A small update for our token holders: We’re simplifying how wallet exports work and winding down Sapien’s Privy integration, which means wallets.sapien.io is being retired. Privy provides the export flow directly at home.privy.io whenever you need it.

AI systems can generate thousands of outputs before a team has time to inspect a fraction of them. When outsourcing this to AI, one of the hardest parts is deciding how much confidence to place in a single judgment. Proof of Quality supports multiple AI reviewers in a single workflow, so teams can compare scores, reasoning, agreement, and disagreement, giving them a faster way to test evaluation workflows at scale while keeping expert attention focused on ambiguity and high consequence cases. Read more: sapien.io/blog/proof-of-qual…

We’ve been thinking about a modern problem in city intelligence: AI has made it possible for cities to easily collect and process huge amounts of sensor data, however the decisions attached to that data still depend on sparse human expert judgment. We wrote about that gap, and how Proof of Quality can make judgment measurable in city intelligence. sapien.io/blog/proof-of-qual…

A robotics pilot under Proof of Quality, Sapien's evaluation system for AI work, is shaped by one decision your team makes at the start: which failure classes go straight to a qualified expert. Experts apply that rubric to real episodes, and their decisions set the standard for difficult decisions. Agents then take a pass at qualifying the data against that standard so your experts can fully focus on the difficult decisions that need their attention.

Every evaluation pass in a robotics programme ends in a decision: train on this batch of demonstration episodes, or send part of it back. What a team can say about that decision a month later depends on what the pass wrote down. Each pass in Proof of Quality, Sapien's evaluation system for AI work, closes with a Proof Report recording what was measured, against which rubric, and what it scored.

Two qualified traffic engineers can read the same intersection data and recommend different signal timings. City intelligence work produces disagreements like this at every stage, and a single opinion usually settles them, which makes the recommendation depend on which engineer happened to review it. Contested work goes to independent scoring in Proof of Quality, Sapien's evaluation system for AI work, and consensus resolves it inside the same pass. When two of your reviewers disagree, what settles it?

Most episodes in a batch of robot demonstrations are uncontested. Any qualified reviewer would score them the same way, and they still hold the same place in the review schedule as the difficult ones. Volume sets the duration of the pass, so the episodes that need a specialist wait behind work that needed a check. Proof of Quality, Sapien's evaluation system for AI work, scores that volume with agents and puts qualified experts on the episodes where context, uncertainty or consequence makes their judgment worth the time. The queue in front of the panel shrinks to a fraction of the batch, and the pass finishes sooner. Which part of your review queue absorbs the most expert time?

Finding the cause of repeated failures is what lets a robotics team improve the system. A ten percent failure rate measures the problem, but it does not identify the cause. Grouping those failures can show whether they share a specific condition, such as gripper slippage or poor tracking under certain lighting. Proof of Quality, Sapien's evaluation system for AI work, reports where results fell below the defined standard and groups related failures. That gives the team specific problems to investigate. When a batch falls below standard, how long does it take your team to identify the cause?

City intelligence systems continuously analyze road sensors and camera feeds to recommend changes such as signal timing. When those recommendations enter a review queue, the traffic data behind them becomes less current. Proof of Quality, Sapien’s evaluation system for AI work, reduces review time by defining the rubric and panel size before evaluation begins. Agents perform the initial evaluations, while qualified experts review contested cases. The first Proof Report can then be produced in days, while the recommendation still reflects recent conditions.

The slowest stage sets the pace of a robotics programme. A team can say how long collection, processing and training take, since each has a measured throughput. Evaluation, the stage that decides which demonstration episodes are fit to train on, runs in months, and the count varies from one project to the next. Sapien built Proof of Quality, its evaluation system for AI work, to settle the rubric and the evaluation method before a pilot begins, with expected effort and cost known at the start. The pilot runs in days, and the first Proof Report arrives while the decision it informs is still open. How long does the first read on a new batch take for your team?