> ## Documentation Index
> Fetch the complete documentation index at: https://docs.runtools.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Computer Use

> Use desktop sandboxes for browser and GUI automation

When an agent needs eyes and a mouse — a browser, a GUI app, anything graphical — reach for the `desktop-ubuntu` template. It's a full Linux desktop you can watch and drive from the browser, with shell-driven automation available inside the image.

## Create a desktop sandbox

```typescript theme={null}
const sandbox = await rt.sandbox.create({
  template: 'desktop-ubuntu',
  name: 'browser-env',
});

await sandbox.waitForReady();

sandbox.on('status', (state) => {
  if (state.vncReady && sandbox.vncUrl) {
    console.log(sandbox.vncUrl);
  }
});
```

```bash theme={null}
runtools sandbox create --name browser-env --template desktop-ubuntu
runtools sandbox get browser-env
```

`waitForReady()` waits for SSH readiness. Desktop access has its own readiness
state, so wait for `vncReady` and `vncUrl` before starting visual automation.

## Link An Agent

```typescript agents/browser-agent.ts theme={null}
import { defineAgent } from '@runtools-ai/sdk';

export default defineAgent({
  slug: 'browser-agent',
  name: 'Browser Agent',
  model: 'claude-sonnet-4',
  executionMode: 'in_sandbox',
  sandbox: 'browser-env',
  tools: ['exec_command', 'write_stdin', 'apply_patch', 'view_image', 'web_search', 'get_dev_url'],
  systemPrompt: `Use the desktop tools available through shell commands when a task requires browser or GUI inspection.`,
});
```

```bash theme={null}
runtools deploy --agents-only
runtools agent run browser-agent "Open a browser, inspect the page, and summarize what you find"
```

## Browser Desktop URL

The SDK exposes a normalized `sandbox.vncUrl` when desktop access is ready:

```typescript theme={null}
sandbox.on('status', (state) => {
  if (state.vncReady) {
    console.log(sandbox.vncUrl);
  }
});
```

The REST `GET /v1/sandboxes/{id}` response can also include `vncReady` and
`vncUrl`. Desktop URLs are normalized to
`/vnc.html?autoconnect=true&resize=remote` before the SDK and CLI display them.

## Automation Pattern

Agents currently use shell-accessible tools for desktop automation. For example:

```typescript theme={null}
await sandbox.exec('xdotool getmouselocation');
await sandbox.exec('scrot /tmp/screen.png');
```

The `desktop-ubuntu` image includes XFCE, Firefox ESR, KasmVNC, `xdotool`,
`scrot`, and ImageMagick. Use `exec_command` or `sandbox.exec()` for GUI-driving
commands, then use a vision-capable model and `view_image` when your workflow
depends on screenshot interpretation.

## Best Practices

<AccordionGroup>
  <Accordion title="Prefer APIs for structured integrations">
    Use Tool Hub or direct APIs when a provider offers a reliable API. Desktop automation is best for web or GUI workflows that do not have a usable API.
  </Accordion>

  <Accordion title="Wait for readiness">
    Wait for both sandbox readiness and desktop readiness before starting browser tasks.
  </Accordion>

  <Accordion title="Keep prompts concrete">
    Give the agent specific page goals, selectors, or visual landmarks when possible.
  </Accordion>

  <Accordion title="Clean up long sessions">
    Pause or destroy desktop sandboxes when you are done.
  </Accordion>
</AccordionGroup>