---
title: Computer Use
description: Programmatic control of desktop environments within Daytona sandboxes.
---
Computer Use enables programmatic control of desktop environments within sandboxes. It provides mouse, keyboard, screenshot, screen recording, accessibility, and display operations for automating GUI interactions and testing desktop applications.
Computer Use and [VNC](/docs/v2/vnc-access) work together to enable both manual and automated desktop interactions. VNC provides the visual interface for users to manually interact with the desktop, while Computer Use provides the programmatic API for AI agents to automate operations.
The desktop stack (D-Bus, Xvfb, the window manager, and the x11vnc server) ships with the sandbox runtime as a self-contained add-on, so it works on any Linux sandbox image, including images with no desktop packages. The virtual display runs at 1280x720. All Computer Use calls go through the toolbox API and require an API key with the sandbox write permission.
## Use cases
Computer Use is used when a task needs a graphical desktop instead of an API or a shell.
- **Computer-use agents**: an agent reads screenshots and the accessibility tree, then moves the mouse and types to operate desktop applications.
- **Desktop application tests**: GUI tests click, type, and take screenshots to check the state of an application.
- **Browser automation**: a browser on the sandbox desktop is driven through mouse and keyboard input, for sites that have no API.
- **Element-level automation**: the accessibility tree finds a button or a text field by role, name, and state. An agent invokes the element or sets its text without screen coordinates.
- **Screenshots for models**: region and compressed screenshots capture part of the screen or a smaller image. An agent sends less image data to a model on each step.
- **Window-aware automation**: the window list reports the title, position, size, and active flag of each window. An agent checks which application is in front before it acts.
- **Agents on Windows and macOS**: Windows and macOS sandboxes provide screenshots, mouse and keyboard input, the accessibility tree, and screen recordings through the same API. On Windows, a desktop session must be open.
- **Recorded sessions**: screen recordings capture what an agent or a test did, for review and debugging. Recordings are MP4 files stored in the sandbox and downloaded through the API.
- **Human in the loop**: a person watches or takes over the same desktop through [VNC](/docs/v2/vnc-access) while an agent works.
## Start Computer Use
Start all computer use processes (D-Bus, Xvfb, the window manager, and the x11vnc server) in the sandbox. The call blocks until the X server and the VNC server accept connections and returns the status of each process. If either does not come up within 5 s, the desktop is marked failed and the call returns an error. Calling start on a desktop that is already ready returns the current status; calling it while the desktop is starting or failed returns an error.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
result = sandbox.computer_use.start()
print("Computer use processes started:", result.message)
for name, process in result.status.items():
print(f"{name}: running={process.running} pid={process.pid}")
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
const result = await sandbox.computerUse.start();
console.log('Computer use processes started:', result.message);
for (const [name, process] of Object.entries(result.status)) {
console.log(`${name}: running=${process.running} pid=${process.pid}`);
}
```
- `restart()` leaves the failed state.
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/start' \
--request POST \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Stop Computer Use
Stop all computer use processes in the sandbox. Stopping a ready desktop returns it to not started. Stopping a failed desktop kills any remaining processes but keeps the failed state and its cause; only [restart](#restart-computer-use) leaves the failed state.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
result = sandbox.computer_use.stop()
print("Computer use processes stopped:", result.message)
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const result = await sandbox.computerUse.stop();
console.log('Computer use processes stopped:', result.message);
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/stop' \
--request POST \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Restart Computer Use
Tear the desktop down and build a new one. Restart is valid from every state and is the only way out of the failed state. Concurrent restart calls join the same in-flight restart and receive the same result. Application windows do not survive a restart.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
result = sandbox.computer_use.restart()
print(result.message, result.desktop) # computer use restarted ready
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const result = await sandbox.computerUse.restart();
console.log(result.message, result.desktop); // computer use restarted ready
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/restart' \
--request POST \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Get status
Get the desktop status. The status field is active only while the desktop field is ready.
| **`desktop`** | **`status`** | **Meaning** |
| ----------------- | ------------ | --------------------------------------------------------------------------------------------------------- |
| `not_started` | `inactive` | No desktop exists. Call [start](#start-computer-use). |
| `starting` | `inactive` | Start is in progress. Poll status. |
| `ready` | `active` | Actions are accepted. |
| `failed` | `inactive` | The desktop is gone. `cause.code` and `cause.message` explain why. Call [restart](#restart-computer-use). |
The cause object is present only when the desktop is failed. Possible cause codes:
| **Code** | **Meaning** |
| ----------------------------- | --------------------------------------------------------------------------------------------- |
| **`desktop_exited`** | The X server (`xvfb`) exited. Windows in it are lost. |
| **`addon_crashed`** | The add-on that supervises the desktop died while a desktop existed. |
| **`start_failed`** | Creating the desktop or probing its readiness failed, for example a missing binary. |
| **`start_interrupted`** | The add-on died while the desktop was being created. |
| **`state_corrupt`** | Retained desktop state could not be read or validated. |
| **`state_persistence_failed`** | A state transition could not be persisted. |
| **`session_ended`** | Windows only. The interactive desktop session ended. |
| **`session_replaced`** | Windows only. Another interactive desktop session replaced the one the desktop was bound to. |
Failure evidence is boot-scoped: a sandbox reboot resets the desktop to not started.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
status = sandbox.computer_use.status()
print("Computer use status:", status.status, status.desktop)
if status.desktop == "failed":
print(f"Failure: {status.cause.code}: {status.cause.message}")
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
const status = await sandbox.computerUse.status();
console.log('Computer use status:', status.status, status.desktop);
if (status.desktop === 'failed') {
console.log(`Failure: ${status.cause.code}: ${status.cause.message}`);
}
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/status' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Get process status
Get the status of one desktop process: D-Bus, Xvfb, the window manager, or the x11vnc server. Without a name, it returns the [status of every process](#get-all-process-statuses).
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
xvfb_status = sandbox.computer_use.service.status("xvfb")
print(xvfb_status.process_name, xvfb_status.running)
```
- The process names are `dbus`, `xvfb`, `wm`, and `x11vnc`.
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const xvfbStatus = await sandbox.computerUse.service.status('xvfb');
console.log(xvfbStatus.processName, xvfbStatus.running);
```
- The process names are `dbus`, `xvfb`, `wm`, and `x11vnc`.
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process/{processName}/status' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
- The process names are `dbus`, `xvfb`, `wm`, and `x11vnc`.
## Get all process statuses
Get the status of every desktop process. The response is a map keyed by process name, with one entry each for D-Bus, Xvfb, the window manager, and the x11vnc server. Each entry reports whether the process is alive, its supervisor restart state, and its startup priority.
| **API field** | **Description** |
| ---------------------- | ------------------------------------------------------------------- |
| **`Running`** | Whether the process is alive. |
| **`Pid`** | Process ID while the process is running. |
| **`Status`** | `running`, `stopped`, or `failed`. |
| **`Failed`** | Whether the process entered the terminal failed state. |
| **`AutoRestart`** | Whether the supervisor restarts the process automatically. |
| **`RestartAttempts`** | Restart attempts consumed by the current generation of the process. |
| **`RestartBackoffMs`** | Current restart backoff in milliseconds. |
| **`Priority`** | Relative startup priority within the desktop stack. |
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
for name, process in sandbox.computer_use.service.status().items():
print(f"{name}: {process.status} pid={process.pid} restarts={process.restart_attempts}")
```
- The call returns a dictionary keyed by process name: `dbus`, `xvfb`, `wm`, and `x11vnc`.
- The API fields map to snake_case attributes, for example `RestartAttempts` to `restart_attempts` and `AutoRestart` to `auto_restart`.
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const statuses = await sandbox.computerUse.service.status();
for (const [name, process] of Object.entries(statuses)) {
console.log(`${name}: ${process.status} pid=${process.pid} restarts=${process.restartAttempts}`);
}
```
- The call returns a record keyed by process name: `dbus`, `xvfb`, `wm`, and `x11vnc`.
- The API fields map to camelCase properties, for example `RestartAttempts` to `restartAttempts` and `AutoRestart` to `autoRestart`.
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process-status' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Restart process
Restart a specific desktop process. Per-process restart works only while the desktop is ready. The Xvfb process cannot be restarted on its own because application windows live inside it; use [restart](#restart-computer-use) instead.
The D-Bus, window manager, and x11vnc processes are restarted automatically if they exit: up to 6 attempts with exponential backoff from 1 s to 30 s, reset after 30 s of uptime. When attempts run out, the process is marked failed in the status map while the desktop stays ready.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
result = sandbox.computer_use.service.restart("wm")
print(result.message, result.process_name)
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const result = await sandbox.computerUse.service.restart('wm');
console.log(result.message, result.processName);
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process/{processName}/restart' \
--request POST \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Get process logs
Get logs for a specific desktop process. Logs are retained up to 256 KiB per process and trimmed to the most recent 128 KiB while the process runs.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
logs = sandbox.computer_use.service.logs("x11vnc")
print("x11vnc logs:", logs.logs)
```
The response has the following fields:
| **Field** | **Description** |
| --------- | --------------- |
| `logs` | Holds the retained standard output of the process. |
- [Get process errors](#get-process-errors) returns standard error.
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const logs = await sandbox.computerUse.service.logs('x11vnc');
console.log('x11vnc logs:', logs.logs);
```
The response has the following fields:
| **Field** | **Description** |
| --------- | --------------- |
| `logs` | Holds the retained standard output of the process. |
- [Get process errors](#get-process-errors) returns standard error.
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process/{processName}/logs' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Get process errors
Get error logs for a specific desktop process.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
errors = sandbox.computer_use.service.errors("x11vnc")
print("x11vnc errors:", errors.errors)
```
The response has the following fields:
| **Field** | **Description** |
| --------- | --------------- |
| `errors` | Holds the retained standard error output of the process. The output is capped at 256 KiB, like the logs. The value is an empty string when the process has written nothing to standard error. |
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const errors = await sandbox.computerUse.service.errors('x11vnc');
console.log('x11vnc errors:', errors.errors);
```
The response has the following fields:
| **Field** | **Description** |
| --------- | --------------- |
| `errors` | Holds the retained standard error output of the process. The output is capped at 256 KiB, like the logs. The value is an empty string when the process has written nothing to standard error. |
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process/{processName}/errors' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Mouse operations
Every mouse, keyboard, screenshot, display, accessibility, and recording-start call requires the desktop to be ready; otherwise it returns 409 with a desktop not ready error. Coordinates are screen pixels in the range -32768 to 32767; values outside it return 400.
### Click
Click the mouse at the specified coordinates with the left, right, or middle button, as a single or a double click.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
sandbox.computer_use.mouse.click(x=100, y=200)
```
| **Parameter** | **Type** | **Description** |
| ------------- | ------------------------ | -------------------- |
| `button` | `ComputerUseMouseButton` | Sets the mouse button. The default is `"left"`. |
| `double` | `bool` | Makes the click a double click. The default is `False`. |
- The call returns the pointer position after the click.
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
await sandbox.computerUse.mouse.click({ x: 100, y: 200 });
```
| **Parameter** | **Type** | **Description** |
| ------------- | ------------------------ | -------------------- |
| `button` | `ComputerUseMouseButton` | Sets the mouse button. The default is `'left'`. |
| `double` | `boolean` | Makes the click a double click. The default is `false`. |
- The call returns the pointer position after the click.
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/click' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{
"x": 100,
"y": 200
}'
```
| **Field** | **Type** | **Description** |
| --------- | --------- | ------------------- |
| `button` | `string` | Sets the mouse button. The default is `left`. |
| `double` | `boolean` | Makes the click a double click. The default is `false`. |
The response has the following fields:
| **Field** | **Description** |
| --------- | --------------- |
| `x` | Holds the x coordinate of the pointer after the click. |
| `y` | Holds the y coordinate of the pointer after the click. |
### Move
Move the mouse cursor to the given coordinates. The response holds the final pointer position.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
result = sandbox.computer_use.mouse.move(x=100, y=200)
print(f"Mouse moved to: {result.x}, {result.y}")
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const result = await sandbox.computerUse.mouse.move({ x: 100, y: 200 });
console.log(`Mouse moved to: ${result.x}, ${result.y}`);
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/move' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{
"x": 100,
"y": 200
}'
```
### Drag
Drag the mouse from start to end coordinates while holding the selected button. The pointer moves in 10 interpolated steps so applications that ignore a teleporting drag still register it.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
result = sandbox.computer_use.mouse.drag(
start_x=50, start_y=50, end_x=150, end_y=150, button="left"
)
print(f"Drag ended at {result.x}, {result.y}")
```
| **Parameter** | **Type** | **Description** |
| ------------- | ------------------------ | ------------------ |
| `button` | `ComputerUseMouseButton` | Sets the mouse button to hold. The default is `"left"`. |
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const result = await sandbox.computerUse.mouse.drag({
startX: 50,
startY: 50,
endX: 150,
endY: 150,
button: 'left',
});
console.log(`Drag ended at ${result.x}, ${result.y}`);
```
| **Parameter** | **Type** | **Description** |
| ------------- | ------------------------ | --------------- |
| `button` | `ComputerUseMouseButton` | Sets the mouse button to hold. This parameter is required. |
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/drag' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{
"button": "left",
"endX": 150,
"endY": 150,
"startX": 50,
"startY": 50
}'
```
| **Field** | **Type** | **Description** |
| --------- | -------- | ---------------- |
| `button` | `string` | Sets the mouse button to hold. The default is `left`. |
### Scroll
Scroll the mouse wheel up or down at the specified coordinates by a number of wheel ticks. One tick is one notch of a physical mouse wheel.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
result = sandbox.computer_use.mouse.scroll(x=100, y=200, direction="down")
print(result.success)
```
- The default scroll amount is `1`.
- The scroll amount must not be negative.
- The maximum scroll amount is `100`.
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const result = await sandbox.computerUse.mouse.scroll({ x: 100, y: 200, direction: 'down' });
console.log(result.success);
```
- The default scroll amount is `1`.
- The scroll amount must not be negative.
- The maximum scroll amount is `100`.
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/scroll' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{
"direction": "down",
"x": 100,
"y": 200
}'
```
- The default scroll amount is `1`.
- The scroll amount must not be negative.
- The maximum scroll amount is `100`.
### Get position
Get the current mouse cursor position.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
position = sandbox.computer_use.mouse.position()
print(f"Mouse is at: {position.x}, {position.y}")
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const position = await sandbox.computerUse.mouse.position();
console.log(`Mouse is at: ${position.x}, ${position.y}`);
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/position' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Keyboard operations
Send keyboard input to the desktop. Type a text string, press one key with optional modifiers, or press a hotkey combination. Key names resolve against the keyboard map of the virtual display, and, like mouse calls, keyboard calls require the desktop to be ready.
### Type
Type arbitrary text, including uppercase letters, symbols, and non-ASCII characters. Newlines are sent as Enter and tabs as Tab. Characters with no key in the X11 keymap are typed through a temporary key mapping.
Set a delay between characters to slow typing down for applications that drop fast input.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
sandbox.computer_use.keyboard.type(text="Hello, World!")
```
- The call sends `\n` and `\r` as Enter.
- The delay between characters is in milliseconds.
- The default delay is `0`.
- The maximum delay is `1000` per character.
- The text is limited to 4096 characters.
- The number of characters multiplied by the delay must not exceed 25 s.
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
await sandbox.computerUse.keyboard.type({ text: 'Hello, World!' });
```
- The call sends `\n` and `\r` as Enter.
- The delay between characters is in milliseconds.
- The default delay is `0`.
- The maximum delay is `1000` per character.
- The text is limited to 4096 characters.
- The number of characters multiplied by the delay must not exceed 25 s.
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/keyboard/type' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{
"text": "Hello, World!"
}'
```
- The call sends `\n` and `\r` as Enter.
- The delay between characters is in milliseconds.
- The default delay is `0`.
- The maximum delay is `1000` per character.
- The text is limited to 4096 characters.
- The number of characters multiplied by the delay must not exceed 25 s.
### Press
Press a key with optional modifiers. Modifiers are held in order, the key is pressed and released, and the modifiers are released in reverse, even if the press fails. See [supported keys](#supported-keys).
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
sandbox.computer_use.keyboard.key(key="enter")
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
await sandbox.computerUse.keyboard.key({ key: 'enter' });
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/keyboard/key' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{
"key": "enter"
}'
```
### Hotkey
Press a hotkey combination. The combination is a plus-separated chord string: the last entry is the key, the others are modifiers.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
sandbox.computer_use.keyboard.hotkey(keys="ctrl+c")
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
await sandbox.computerUse.keyboard.hotkey({ keys: 'ctrl+c' });
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/keyboard/hotkey' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{
"keys": "ctrl+c"
}'
```
### Supported keys
Key names are case-insensitive. The following are supported for the key and modifiers parameters and for the parts of the keys parameter:
| Category | Keys |
| ------------- | ------------------------------------------------------------------------------------------------------------- |
| Modifiers | `ctrl` (alias `control`), `alt`, `shift`, `cmd` (aliases `super`, `meta`, `win`) |
| Editing | `enter` (alias `return`), `tab`, `escape` (alias `esc`), `backspace`, `delete` (alias `del`), `space` |
| Navigation | `home`, `end`, `pageup` (alias `page_up`), `pagedown` (alias `page_down`), `insert`, `up`, `down`, `left`, `right` |
| Function keys | `f1` through `f12` |
| Symbols | `minus`, `plus`, `equal` (alias `equals`) |
| Characters | Any single character, for example `a`, `A`, `5`, `/`, `-` |
The SDK modifiers type accepts ctrl, alt, shift, and cmd; the API also accepts the aliases.
An unknown key or modifier returns 400.
## Accessibility operations
Use accessibility operations to inspect the AT-SPI tree and interact with UI elements by node ID. Start Computer Use before calling accessibility methods.
Each node carries an ID, role, name, description, bounds (x, y, width, height), states, actions, and children. Node IDs have the form `:` and are valid only while the owning application keeps the object alive; an unknown or malformed ID returns 404 or 400.
The scope parameter selects the root of the search:
| **Scope** | **Root** |
| ------------- | -------------------------------------------------------------------------------------------------- |
| **`focused`** | The application whose state includes `active`, else the first registered application. Default. |
| **`pid`** | The application owned by process `pid`. Returns `404` if no accessible application has that PID. |
| **`all`** | The AT-SPI registry root, covering every application. |
Traversal is bounded: 2000 nodes and 2 s for the tree, 5000 nodes and 2 s for find. When a bound is reached, the response is marked as truncated.
Accessibility operations read the semantic UI information that applications expose over AT-SPI. Apps or custom widgets that do not expose accessibility objects may return sparse nodes, generic roles, or no actionable nodes; mouse, keyboard, and screenshot operations remain available for those cases.
### Get tree
Read an accessibility tree for the focused app, a specific process, or all apps. The maximum depth parameter limits the depth below the root: 0 returns only the root, and omitting it or passing -1 returns the whole subtree within the traversal budget.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
tree = sandbox.computer_use.a11y.tree(scope="focused")
print(tree.root.role, tree.root.name, tree.truncated)
```
| **Parameter** | **Type** | **Description** |
| ------------- | ---------------------- | --------------------------------------------------- |
| `scope` | `ComputerUseA11yScope` | Sets the root of the search: `"focused"`, `"pid"`, or `"all"`. With `scope="pid"`, also pass `pid`. This parameter is required. |
| `max_depth` | `int` | Limits the depth below the root. Omit the parameter to get the whole subtree. |
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const tree = await sandbox.computerUse.a11y.tree({ scope: 'focused' });
console.log(tree.root.role, tree.root.name, tree.truncated);
```
| **Parameter** | **Type** | **Description** |
| ------------- | ---------------------- | --------------------------------------------------- |
| `scope` | `ComputerUseA11yScope` | Sets the root of the search: `'focused'`, `'pid'`, or `'all'`. With `scope: 'pid'`, also pass `pid`. This parameter is required. |
| `maxDepth` | `number` | Limits the depth below the root. Omit the parameter to get the whole subtree. |
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/tree?scope=focused' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
| **Field** | **Type** | **Description** |
| ---------- | --------- | ------------------------------------------------------- |
| `scope` | `string` | Sets the root of the search. An unknown scope returns `400`. The default is `focused`. |
| `pid` | `integer` | Selects the application when `scope=pid`. |
| `maxDepth` | `integer` | Limits the depth below the root. The query parameter name is camelCase. |
### Find nodes
Search the accessibility tree by role, accessible name, state, and scope. All filters must match.
Node states: active, armed, busy, checked, collapsed, defunct, editable, enabled, expandable, expanded, focusable, focused, has-tooltip, horizontal, iconified, modal, multi-line, multiselectable, opaque, pressed, resizable, selectable, selected, sensitive, showing, single-line, stale, transient, vertical, visible, manages-descendants, indeterminate, required, truncated, and animated.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
buttons = sandbox.computer_use.a11y.find(scope="focused", role="button", name="Submit")
for node in buttons.matches:
print(node.id, node.name, node.bounds.x, node.bounds.y)
```
| **Parameter** | **Type** | **Description** |
| ------------- | -------------------------- | ---------------------------------------------------------------------------- |
| `scope` | `ComputerUseA11yScope` | Sets the root of the search. This parameter is required. |
| `role` | `str` | Matches the node role exactly, for example `button` or `entry`. |
| `name` | `str` | Matches the accessible name according to `name_match`. |
| `name_match` | `ComputerUseA11yNameMatch` | Sets how `name` is compared: `exact`, `substring`, or `regex`. The default is `exact`. |
| `states` | `Sequence[str]` | Requires the node to have every listed state, for example `["enabled", "focusable"]`. |
| `limit` | `int` | Caps the number of matches. With more matches, `truncated` is `True`. The default is `50`. |
- Filters other than `scope` are optional.
- An omitted filter matches every node.
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const buttons = await sandbox.computerUse.a11y.find({ scope: 'focused', role: 'button', name: 'Submit' });
for (const node of buttons.matches) {
console.log(node.id, node.name, node.bounds.x, node.bounds.y);
}
```
| **Parameter** | **Type** | **Description** |
| ------------- | -------------------------- | ---------------------------------------------------------------------------- |
| `scope` | `ComputerUseA11yScope` | Sets the root of the search. This parameter is required. |
| `role` | `string` | Matches the node role exactly, for example `'button'` or `'entry'`. |
| `name` | `string` | Matches the accessible name according to `nameMatch`. |
| `nameMatch` | `ComputerUseA11yNameMatch` | Sets how `name` is compared: `'exact'`, `'substring'`, or `'regex'`. The default is `'exact'`. |
| `states` | `string[]` | Requires the node to have every listed state, for example `['enabled', 'focusable']`. |
| `limit` | `number` | Caps the number of matches. With more matches, `truncated` is `true`. The default is `50`. |
- Filters other than `scope` are optional.
- An omitted filter matches every node.
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/find' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{
"scope": "focused",
"role": "button",
"name": "Submit"
}'
```
| **Field** | **Type** | **Description** |
| ----------- | ---------- | -------------------------------------------------------------------------------------------- |
| `scope` | `string` | Sets the root of the search. The default is `focused`. |
| `role` | `string` | Matches the node role exactly, for example `button` or `entry`. |
| `name` | `string` | Matches the accessible name according to `nameMatch`. |
| `nameMatch` | `string` | Sets how `name` is compared: `exact`, `substring`, or `regex`. The default is `exact`. |
| `states` | `string[]` | Requires the node to have every listed state, for example `["enabled", "focusable"]`. |
| `limit` | `integer` | Caps the number of matches. A value of `0` or less uses the default. With more matches, `truncated` is `true`. The default is `50`. |
### Focus node
Focus a node from [get tree](#get-tree) or [find nodes](#find-nodes).
```python
sandbox.computer_use.a11y.focus(id=node.id)
```
```typescript
await sandbox.computerUse.a11y.focus({ id: node.id });
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/node/focus' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{"id":"node-id"}'
```
### Invoke node
Run a node action, such as pressing a button. Without an action name, the node's click action is invoked, falling back to its first action. An action name must appear in the node's actions list.
```python
sandbox.computer_use.a11y.invoke(id=node.id)
```
```typescript
await sandbox.computerUse.a11y.invoke({ id: node.id });
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/node/invoke' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{"id":"node-id"}'
```
### Set node value
Write text content to nodes that implement editable text, such as entries and text areas. The value replaces the current content.
```python
sandbox.computer_use.a11y.set_value(id=node.id, value="hello")
```
```typescript
await sandbox.computerUse.a11y.setValue({ id: node.id, value: 'hello' });
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/node/value' \
--request POST \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--data '{"id":"node-id","value":"hello"}'
```
## Screenshot operations
Screenshots return the image as a base64 string, its encoded size in bytes, and, when the cursor is requested, the pointer coordinates. The cursor is never drawn. Full-screen and region screenshots are PNG; compressed variants accept format, quality, and scale parameters.
### Take full screen
Take a screenshot of the entire screen.
```python
import base64
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
screenshot = sandbox.computer_use.screen.capture()
print(f"Screenshot size: {screenshot.size_bytes} bytes")
with open("screen.png", "wb") as f:
f.write(base64.b64decode(screenshot.screenshot))
```
```typescript
import { writeFile } from 'node:fs/promises';
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const screenshot = await sandbox.computerUse.screen.capture();
console.log(`Screenshot size: ${screenshot.sizeBytes} bytes`);
await writeFile('screen.png', Buffer.from(screenshot.screenshot, 'base64'));
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/screenshot' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
### Take region
Take a screenshot of a specific region. The width and height must be positive, and the region must lie within the screen.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
screenshot = sandbox.computer_use.screen.region(x=100, y=100, width=300, height=200)
print(f"Captured region: {screenshot.size_bytes} bytes")
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const screenshot = await sandbox.computerUse.screen.region({ x: 100, y: 100, width: 300, height: 200 });
console.log(`Captured region: ${screenshot.sizeBytes} bytes`);
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/screenshot/region?x=100&y=100&width=300&height=200' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
### Take compressed
Take a compressed screenshot of the entire screen.
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
screenshot = sandbox.computer_use.screen.capture_compressed(format="jpeg", quality=80)
print(f"Compressed size: {screenshot.size_bytes} bytes")
```
| **Parameter** | **Type** | **Description** |
| ------------- | -------- | --------------------------------------------------------------- |
| `format` | `str` | Sets the image format, `png` or `jpeg`. The default is `png`. |
| `quality` | `int` | Sets the image quality, from `1` to `100`. The quality applies to `jpeg`. The default is `85`. |
| `scale` | `float` | Resizes the captured image by a factor from `0.1` to `1.0`. The default is `1.0`. |
| `show_cursor` | `bool` | Returns the pointer coordinates with the screenshot. The cursor is never drawn. The default is `False`. |
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const screenshot = await sandbox.computerUse.screen.captureCompressed({ format: 'jpeg', quality: 80 });
console.log(`Compressed size: ${screenshot.sizeBytes} bytes`);
```
| **Parameter** | **Type** | **Description** |
| ------------- | --------- | ----------------------------------------------------------- |
| `format` | `string` | Sets the image format, `'png'` or `'jpeg'`. The default is `'png'`. |
| `quality` | `number` | Sets the image quality, from `1` to `100`. The quality applies to `'jpeg'`. The default is `85`. |
| `scale` | `number` | Resizes the captured image by a factor from `0.1` to `1.0`. The default is `1.0`. |
| `showCursor` | `boolean` | Returns the pointer coordinates with the screenshot. The cursor is never drawn. The default is `false`. |
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/screenshot/compressed?format=jpeg&quality=80' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
| **Field** | **Type** | **Description** |
| ------------- | --------- | ----------------------------------------------------------- |
| `format` | `string` | Sets the image format, `png` or `jpeg`. The default is `png`. |
| `quality` | `integer` | Sets the image quality, from `1` to `100`. The quality applies to `jpeg`. The default is `85`. |
| `scale` | `number` | Resizes the captured image by a factor from `0.1` to `1.0`. The default is `1.0`. |
| `show_cursor` | `boolean` | Returns the pointer coordinates with the screenshot. The cursor is never drawn. The default is `false`. |
### Take compressed region
Take a compressed screenshot of a specific region. Accepts the region parameters of [take region](#take-region) and the compression parameters of [take compressed](#take-compressed).
```python
from daytona import Daytona
daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()
screenshot = sandbox.computer_use.screen.region_compressed(
x=0, y=0, width=800, height=600, format="jpeg", quality=80
)
print(f"Compressed size: {screenshot.size_bytes} bytes")
```
```typescript
import { Daytona } from '@daytona/sdk';
const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();
const screenshot = await sandbox.computerUse.screen.regionCompressed({
x: 0,
y: 0,
width: 800,
height: 600,
format: 'jpeg',
quality: 80,
});
console.log(`Compressed size: ${screenshot.sizeBytes} bytes`);
```
```bash
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/screenshot/region/compressed?x=0&y=0&width=800&height=600&format=jpeg&quality=80' \
--header 'Authorization: Bearer YOUR_API_KEY'
```
## Screen Recording
Computer Use records desktop sessions for debugging, documentation, or automation workflows. Recordings are MP4 files captured at 10 frames per second and stored inside the sandbox; the file path field in every recording response reports the location.
Each recording has the following fields:
| **Field** | **Description** |
| ---------------------- | --------------------------------------------------------------------------------------------------------- |
| `id` | Recording ID, prefixed `rec_`. Used for stop, get, download, and delete. |
| `file_name` | `