Computer use for agents

Computer Use enables Warp’s cloud agents to interact with desktop environments in a secure, isolated sandbox. The agent can see what’s displayed on screen, click and drag, type text, use keyboard shortcuts, and perform other GUI interactions.

A key use case is testing UI changes with a self-contained feedback loop, where the agent can verify that your code changes produce the expected visual and behavioral results without requiring manual testing.

Overview

With Computer Use, agents can:

• Take screenshots - Capture and analyze the current display
• Interact with applications - Click buttons, fill forms, navigate interfaces
• Type and control keyboard - Enter text and use keyboard shortcuts
• Automate testing workflows - Test UI changes end-to-end without manual intervention
• Work with browser-based interfaces - Test web apps and navigate the web

Computer Use is only available in Warp’s sandboxed cloud environments, not in local interactive terminal sessions.

---

Configuring Computer Use

Computer Use is enabled by default for Warp-hosted Oz cloud runs. When starting a run from the Oz web app, the Computer Use checkbox is pre-selected. When calling the API without specifying computer_use_enabled, Computer Use is on for Warp-hosted runs; runs dispatched to self-hosted runners still require an explicit computer_use_enabled: true. You can disable or explicitly control it through the following entry points:

Oz web app

In the Oz web app, Computer Use is enabled by default when starting a run. To change it for a specific run or agent:

• New agent runs - Deselect Enable computer use when starting a new agent run
• Scheduled agent runs - Configure Enable computer use in the scheduled agent settings
• Integrations - Configure Enable computer use for Slack, Linear, and other integration-triggered agents

CLI

When running agents in the cloud via the Oz CLI, use flags to override the default per run:

oz agent run-cloud --computer-use --prompt "<task>"
oz agent run-cloud --no-computer-use --prompt "<task>"

API

When calling the Warp API to create agent runs, computer use is enabled by default for Warp-hosted runs when computer_use_enabled is omitted. The worker_host field determines this: when worker_host is unset or set to "warp", computer use defaults to enabled; when worker_host is set to any other value (a self-hosted runner endpoint), computer use defaults to disabled — pass computer_use_enabled: true to enable it. To disable computer use explicitly for Warp-hosted runs, set computer_use_enabled to false:

{
  "prompt": "Build a button component that matches this design, then test it in the browser",
  "computer_use_enabled": false
}

Explicit true is also accepted and behaves the same as omitting the field:

{
  "prompt": "Build a button component that matches this design, then test it in the browser",
  "computer_use_enabled": true,
  "environment_id": "optional-environment-id"
}

For full API documentation, see the Oz API & SDK reference.

Warp desktop app

For cloud agent conversations started from the Warp desktop app, Computer Use can be toggled independently of the web UI and API defaults. The toggle is off by default in released builds and must be enabled manually. In the Warp app, navigate to Settings > Agents > Warp Agent > Experimental > Computer use in Cloud Agents to enable or disable Computer Use for cloud agents initiated from the desktop.

Note

This toggle applies only to cloud runs started from the Warp desktop app. Cloud runs started via the Oz web app or API have Computer Use enabled by default regardless of this setting.

---

How Computer Use works

Setup and requirements

Computer Use runs in a containerized sandbox, allowing headless cloud environments to render and interact with graphical applications. The sandbox is fully isolated—it does not have access to your local machine, credentials, or sensitive data outside the sandbox environment.

Your cloud environment must include any applications you want the agent to control. For example, to test a web app in a browser, install Chrome or Firefox in your environment configuration.

Model selection

Computer Use supports multiple Anthropic Claude models, including Claude 4.5 Sonnet, Claude 4.5 Opus, Claude 4.5 Haiku, Claude 4.6 Sonnet, and Claude 4.6 Opus. Warp uses an auto model selector to choose the best-suited model for each Computer Use task.

---

Recording sessions

Agents can record their Computer Use sessions as annotated video artifacts — capturing clicks, key presses, and cursor movement alongside the interaction, with idle time trimmed out.

Recordings appear in the conversation, on the Oz web run page, and are attached to the pull request description when a cloud agent run produces them. They are captured entirely inside the sandbox and never include your local desktop or credentials.

For a full explanation of the recording pipeline, what the finished video contains, where to find recordings, and how to handle recording failures, see Testing with Computer Use and video recordings.

---

Security considerations

Computer Use operates inside a sandboxed cloud environment and has unique security considerations. These risks are heightened when interacting with the internet.

To minimize risks when using Computer Use:

1. Avoid sensitive data - Do not pass API keys, authentication tokens, or personal information to agents using Computer Use
2. Limit internet access - If your environment has internet access, consider restricting to an allowlist of known-safe domains
3. Require human confirmation - For tasks with real-world consequences (e.g., financial transactions, accepting legal terms), ask a human to confirm before the agent proceeds
4. Review agent actions - Regularly review what agents are doing on your behalf, especially when testing new workflows

---

Example workflows

Testing UI changes

Verify that code changes produce the expected visual results and behavior:

• Build from mockups - Receive a Figma design or mockup image, build the UI, and test it matches
• Visual regression testing - After code changes, verify UI renders correctly
• Form and interaction testing - Test form submissions, validation, error handling
• Responsive design validation - Test layout on different screen sizes

Example: Testing a React component

1. You ask the agent: “Build a React button component that matches this design, then test it”
2. Agent takes a screenshot to see the current state
3. Agent opens your dev server in a browser
4. Agent navigates to the component, verifies it renders correctly
5. Agent tests interactions (hover, click) and reports back

Example: Testing a web form

1. You provide a form design and ask the agent to build and test it
2. Agent renders your form in the browser
3. Agent fills fields with valid and invalid data
4. Agent verifies validation messages and submission behavior
5. Agent reports which fields worked correctly and which need adjustment

Example: Verifying UI responsiveness

1. You ask the agent to test your app on different screen sizes
2. Agent resizes the browser window to mobile, tablet, and desktop widths
3. Agent takes screenshots at each size and verifies layout is correct
4. Agent reports any responsive design issues

Browsing and web interaction

Computer Use can also help with general web tasks:

• Browse websites and interact with web interfaces
• Fill out and submit web forms
• Navigate multi-step workflows in web applications

---

Related capabilities

• Images as Context - Pass design mockups and screenshots as context
• Full Terminal Use - Let agents drive interactive terminal apps, see live output, and run commands