I watched both the original talk and the QA. I listened very carefully and have a few thoughts What I liked: ⁃The Michelin kitchen analogy is great ⁃Every team should dial up every verification method available (tests, types, lint rules, etc) to 11, to the point where its hard to write inconsistent code and it’d be very tedious for a human to work in the code. We have a team of 3-4 people at jump.ai furiously working on this too ⁃Think about the system and environment, not tasks. Sample review PRs to look for ways to iterate on the system. We are building a “software factory” at jump.ai, and a lot of this resonates. I like the vision of the future she paints, and I think we’ll get there. My problem is I don’t think we’re there yet and here’s why: ⁃They talk about building trust through empirical verification, but then mention some of the verification is non-deterministic agents checking things ⁃They talk about how they are still iterating on this and about all the problems in the codebase, but they are still shipping 1000s of PRs a month not looking at most of them? ⁃IMO a truly 100x engineer is prescient. They talk to product and even customers, think long term, and plan way ahead. They save the company tons of time and money by predicting and identifying one way doors. Ive never seen agents do this. They work towards what you prompted, they don’t dig deeper to plan ahead beyond this prompt. Thats a huge part of code quality and it can’t be empirically verified. This is the #1 reason I want my best engineers deeply involved. ⁃What about a PR that migrates 10m records in production (because it didn’t think ahead, see previous item). How is your agent empirically verifying that will work correctly on all that production data before merge and deploy? ⁃They talk about 10 chief of staff agents directing hundreds of sub agents all through the night, but they are doing things like rewriting the app in a different language for fun. Talking to customers and knowing what to build is the hardest part of product development. Thats my biggest question: can any one person keep 100s of agents busy doing actually valuable work? ⁃Does anyone know how to empirically verify a PR has no serious security vulnerabilities? My company is financial services… People think agents are at least as reliable as people. I actually think there’s an uncanny valley, where agents often act close enough to give you the ick. Like how confident they are that there are 3 “e”s in “seventeen”. This is all very impressive and forward thinking, but I lead a team of 80 engineers earnestly trying to do this safely and I think this level is a tad early.