How to Build an AI Code Review Agent - Daksh Gupta, Greptile

By ai.engineer

Categories: AI, Tools

Summary

About 1% of all GitHub PRs flow through Greptile's AI code review agent, which reveals a critical bottleneck: naive implementations miss bugs, lack codebase context, and cost $3-$25 per review. Daksh Gupta breaks down the three essential components that separate functional from great code review agents.

Key Takeaways

  1. The naive approach—webhook → agent → feedback—fails on four fronts: missed bugs (fractally infinite problem space), missing codebase context across microservices/repos, runaway costs ($3-$25 per PR), and nitpicky comments that erode signal-to-noise ratio.
  2. Codebase context is foundational. Greptile started as a code indexing company (2023+) teaching LLMs to understand large codebases before adding validation. Without this, agents misunderstand library behavior, architectural intent, and interdependencies in microservice ecosystems.
  3. Execution matters. Great code review agents spin up full dependency-rich sandboxes, not just static analysis—they run code, click through UIs, pressure-test APIs, and simulate real user interactions to catch runtime bugs LLMs miss.
  4. Model selection is an efficiency multiplier. Allocating the right model and inference budget per subtask (not one model for all tasks) drives dramatically better cost-per-performance. Go model-agnostic with harnesses like Open Code or Pixi.
  5. Infrastructure first: Start with a sandbox provider (Firecracker, E2B, etc.), wire GitHub webhooks to pull request events, connect via GitHub review APIs, and keep the harness decoupled from any single LLM to avoid vendor lock-in.

Related topics

Transcript Excerpt

[music] >> Um all right, my name is Dax. I am one of the co-founders of a company called Griptile and we build agents that review and test pull requests with the full context of the code base. And by reviewing and testing these pull requests, we reveal bugs, logical errors, code quality concerns, primarily from the AI-generated code. Now, to give you a sense of scale, many thousands of companies use Griptile every day. About 1% of all pull requests on GitHub go through us and a variety of many different types of code bases, whether it's the CUDA code base at Nvidia or the wallet code base at Coinbase and many, many others. And every now and then, we meet a company where we're actually not the right code review or code testing product for them. Their needs are so specialized and so esoteric…

More from ai.engineer