From fork() to Fleet: Designing an Agent Sandbox Cloud — Abhishek Bhardwaj, OpenAI

By ai.engineer

Categories: AI, Tools

Summary

OpenAI's agent infrastructure team reveals how code execution unlocks AI reasoning: models trained with tool-calling capabilities solve verifiable problems (math, code) that pure language models fail at. The future of AI agents runs persistent, long-running workloads in cloud sandboxes—not locally on laptops—requiring new security architectures to isolate untrusted code execution at scale.

Key Takeaways

  1. Models achieve >95% accuracy on math/code tasks when given code execution capability through tool calling, versus inconsistent performance without it. This verifiable reward signal allows hill-climbing optimization during training.
  2. Agent sandbox architecture requires two-layer security: isolate untrusted code from host OS (prevent root exploits, kernel vulnerabilities) and isolate multi-tenant workloads from each other (prevent data exfiltration between users).
  3. Training loop structure: task → model response → harness parses tool calls → code execution → grader validates reward → backprop. This teaches models both when to call tools and how to generate correct code.
  4. Current AI agents running locally on laptops represents regression from 20 years of cloud infrastructure investment. Future requires persistent, long-running cloud-based agents with robust sandbox isolation—creating entire new infrastructure category.
  5. Attacks on agent execution environments are both intentional (malicious prompts) and unintentional (overzealous model assistance). Sandbox must block both scenarios while preserving legitimate tool functionality and cross-user data isolation.

Related topics

Transcript Excerpt

[music] Welcome everyone. Can you guys hear me? Okay. >> Yeah. >> I've been standing here for 15 minutes without saying anything. So, we can start now. My name is Abishek. I'm on the RL and agent infrastructure team at OpenAI. Um what that means is we work on the infra for reinforcement learning specifically and on the product side we also develop infra that helps run untrusted code as part of chat GPT codeex web securely and reliably at scale. This talk is called from folk to fleet designing an agent sandbox cloud. Um I'll be very clear that there are a lot of words in the title that have OS and infra concepts but this is the first principles talk. Uh so we'll cover what sandboxes are and why they are needed from first principles. We will also try to cover design intuitions around designi…

More from ai.engineer