Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS

By ai.engineer

Categories: AI, Tools

Summary

An agent deleted a production database in 9 seconds, highlighting a critical trust problem in AI automation. WorkOS engineers teach agents to self-verify their work through browser testing and persistent state checks, enabling safe autonomous code generation without manual code review bottlenecks.

Key Takeaways

  1. Agents can generate code faster than humans review it, creating a signal-to-noise ratio problem that makes traditional code review unsustainable at scale. Shift verification from manual PR review to automated self-reporting mechanisms.
  2. Teach agents to verify their own work using browser automation tools like Playwright—checking for UI success messages, form submissions, and page state changes. This self-reporting creates an auditable trail without requiring human review.
  3. Distinguish between UI-level changes and persistent state verification. An agent may update the UI successfully, but if data doesn't persist after a page reload, the action failed—agents must check both layers.
  4. Trust agents gradually through cumulative verification reports rather than all-or-nothing human oversight. Each successful self-verified action builds confidence in the agent's reliability over time.
  5. Provide agents with a workbench repository containing interactive examples, step-by-step guides, glossaries, and AI-trained support to enable hands-on experimentation and knowledge transfer without constant human supervision.

Related topics

Transcript Excerpt

[music] >> Hey, welcome. Welcome to Beyond Done. Uh we're going to get started here with this workshop. We got a lot to pack in and uh we want to make sure that we have time for questions and um uh getting getting like everyone situated. So, we're going to go ahead and get started. >> [clears throat and cough] >> So, what we mean by Beyond Done, just to like jump straight into it, um you probably saw a tweet about this incident that happened a while back uh where an agent just started deleting production databases. And in 9 seconds, everything was gone. Um that is what we want to help you prevent here today uh while still trusting your agents and not having to look at every line, even look at every PR. Like this is where we think things are going and how do we get there? How do we make thi…

More from ai.engineer