Tyler Xiao

all technical work is becoming evals. your primary job as an engineer is now building loops that make coding agents effective maintainers of the systems you own. models are immensely capable - given clear instructions, the right environment, and a way to verify their work, agents can complete open-ended, long-running tasks. if you still think coding agents are shipping “slop”, it simply means one of these three things is missing. if you build product, you might need to restructure your codebase and give your agents fast, high-fidelity ways to test changes. if you’re a researcher, you might spend most of your time building pipelines for data synthesis and environment construction - experiments are the bottleneck, not ideas. the art is translating the real-world problem into the right eval: will the agent understand your exact criteria for when a task is sufficiently complete? and when the frontier shifts, do these criteria change? every engineer is an AI engineer now. in the long run, the ones who build the machine that builds the machine will be most effective. of course, the best will adapt. but if you’re not actively thinking about how to make agents better at your work, or you’re relying on other people to do it for you, you’re probably ngmi.