Computer use in Codex

By OpenAI

Categories: AI, Product

Summary

OpenAI's Codex now enables AI agents to control any desktop application through graphical interfaces, not just code execution—letting users run multiple concurrent background tasks while maintaining their own computer control. This shifts AI from tool-specific to truly general-purpose desktop automation.

Key Takeaways

  1. Non-blocking concurrent execution: Codex uses a separate cursor that doesn't interrupt user input, enabling parallel task automation across multiple apps simultaneously without taking over the entire computer.
  2. Intentional UX design matters: Curved, natural cursor motion with directional swimming animations improves user understanding of agent actions and creates a more intuitive experience.
  3. Two-step onboarding reduces friction: Simple animated permission flows guide users through setup (demonstrated with two drags) rather than complex configuration processes.
  4. Multi-application orchestration enables workflow automation: Agents can coordinate actions across Spotify, Reminders, UTM, and other GUI apps in sequence—moving beyond single-tool automation.
  5. Model flexibility strategy: Computer use works with faster models like Spark, enabling cost-effective task automation while maintaining multimodal and accessibility capabilities.

Topics

Transcript Excerpt

And we'll roll in both cameras. >> Great. Thank you. >> Yeah. >> Hi everyone, Roma here. Codex has quickly evolved from a coding agent into a real teammate. But not just a coding teammate anymore. You can literally use Codex for any tasks and computer use is a big part of that shift. It takes Codex beyond your tools and files and into the real work you do with your local apps. Today I'm joined by Ari who has spent a lot of time thinking about this problem. So Arie, why computer use? Tell me more about how this works. >> Yeah, I'm so excited about computer use. Codeex already had the ability to do so many things on your computer because it could run commands. It could write code. So it can solve all kinds of problems for you. What's new is that there's all this software on your computer tha…