--- title: Computer Use description: Programmatic control of desktop environments within Daytona sandboxes. --- Computer Use enables programmatic control of desktop environments within sandboxes. It provides mouse, keyboard, screenshot, screen recording, accessibility, and display operations for automating GUI interactions and testing desktop applications. Computer Use and [VNC](/docs/v2/vnc-access) work together to enable both manual and automated desktop interactions. VNC provides the visual interface for users to manually interact with the desktop, while Computer Use provides the programmatic API for AI agents to automate operations. The desktop stack (D-Bus, Xvfb, the window manager, and the x11vnc server) ships with the sandbox runtime as a self-contained add-on, so it works on any Linux sandbox image, including images with no desktop packages. The virtual display runs at 1280x720. All Computer Use calls go through the toolbox API and require an API key with the sandbox write permission. ## Use cases Computer Use is used when a task needs a graphical desktop instead of an API or a shell. - **Computer-use agents**: an agent reads screenshots and the accessibility tree, then moves the mouse and types to operate desktop applications. - **Desktop application tests**: GUI tests click, type, and take screenshots to check the state of an application. - **Browser automation**: a browser on the sandbox desktop is driven through mouse and keyboard input, for sites that have no API. - **Element-level automation**: the accessibility tree finds a button or a text field by role, name, and state. An agent invokes the element or sets its text without screen coordinates. - **Screenshots for models**: region and compressed screenshots capture part of the screen or a smaller image. An agent sends less image data to a model on each step. - **Window-aware automation**: the window list reports the title, position, size, and active flag of each window. An agent checks which application is in front before it acts. - **Agents on Windows and macOS**: Windows and macOS sandboxes provide screenshots, mouse and keyboard input, the accessibility tree, and screen recordings through the same API. On Windows, a desktop session must be open. - **Recorded sessions**: screen recordings capture what an agent or a test did, for review and debugging. Recordings are MP4 files stored in the sandbox and downloaded through the API. - **Human in the loop**: a person watches or takes over the same desktop through [VNC](/docs/v2/vnc-access) while an agent works. ## Start Computer Use Start all computer use processes (D-Bus, Xvfb, the window manager, and the x11vnc server) in the sandbox. The call blocks until the X server and the VNC server accept connections and returns the status of each process. If either does not come up within 5 s, the desktop is marked failed and the call returns an error. Calling start on a desktop that is already ready returns the current status; calling it while the desktop is starting or failed returns an error. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() result = sandbox.computer_use.start() print("Computer use processes started:", result.message) for name, process in result.status.items(): print(f"{name}: running={process.running} pid={process.pid}") ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); const result = await sandbox.computerUse.start(); console.log('Computer use processes started:', result.message); for (const [name, process] of Object.entries(result.status)) { console.log(`${name}: running=${process.running} pid=${process.pid}`); } ``` - `restart()` leaves the failed state. ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/start' \ --request POST \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Stop Computer Use Stop all computer use processes in the sandbox. Stopping a ready desktop returns it to not started. Stopping a failed desktop kills any remaining processes but keeps the failed state and its cause; only [restart](#restart-computer-use) leaves the failed state. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() result = sandbox.computer_use.stop() print("Computer use processes stopped:", result.message) ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const result = await sandbox.computerUse.stop(); console.log('Computer use processes stopped:', result.message); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/stop' \ --request POST \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Restart Computer Use Tear the desktop down and build a new one. Restart is valid from every state and is the only way out of the failed state. Concurrent restart calls join the same in-flight restart and receive the same result. Application windows do not survive a restart. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() result = sandbox.computer_use.restart() print(result.message, result.desktop) # computer use restarted ready ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const result = await sandbox.computerUse.restart(); console.log(result.message, result.desktop); // computer use restarted ready ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/restart' \ --request POST \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Get status Get the desktop status. The status field is active only while the desktop field is ready. | **`desktop`** | **`status`** | **Meaning** | | ----------------- | ------------ | --------------------------------------------------------------------------------------------------------- | | `not_started` | `inactive` | No desktop exists. Call [start](#start-computer-use). | | `starting` | `inactive` | Start is in progress. Poll status. | | `ready` | `active` | Actions are accepted. | | `failed` | `inactive` | The desktop is gone. `cause.code` and `cause.message` explain why. Call [restart](#restart-computer-use). | The cause object is present only when the desktop is failed. Possible cause codes: | **Code** | **Meaning** | | ----------------------------- | --------------------------------------------------------------------------------------------- | | **`desktop_exited`** | The X server (`xvfb`) exited. Windows in it are lost. | | **`addon_crashed`** | The add-on that supervises the desktop died while a desktop existed. | | **`start_failed`** | Creating the desktop or probing its readiness failed, for example a missing binary. | | **`start_interrupted`** | The add-on died while the desktop was being created. | | **`state_corrupt`** | Retained desktop state could not be read or validated. | | **`state_persistence_failed`** | A state transition could not be persisted. | | **`session_ended`** | Windows only. The interactive desktop session ended. | | **`session_replaced`** | Windows only. Another interactive desktop session replaced the one the desktop was bound to. | Failure evidence is boot-scoped: a sandbox reboot resets the desktop to not started. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() status = sandbox.computer_use.status() print("Computer use status:", status.status, status.desktop) if status.desktop == "failed": print(f"Failure: {status.cause.code}: {status.cause.message}") ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); const status = await sandbox.computerUse.status(); console.log('Computer use status:', status.status, status.desktop); if (status.desktop === 'failed') { console.log(`Failure: ${status.cause.code}: ${status.cause.message}`); } ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/status' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Get process status Get the status of one desktop process: D-Bus, Xvfb, the window manager, or the x11vnc server. Without a name, it returns the [status of every process](#get-all-process-statuses). ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() xvfb_status = sandbox.computer_use.service.status("xvfb") print(xvfb_status.process_name, xvfb_status.running) ``` - The process names are `dbus`, `xvfb`, `wm`, and `x11vnc`. ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const xvfbStatus = await sandbox.computerUse.service.status('xvfb'); console.log(xvfbStatus.processName, xvfbStatus.running); ``` - The process names are `dbus`, `xvfb`, `wm`, and `x11vnc`. ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process/{processName}/status' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` - The process names are `dbus`, `xvfb`, `wm`, and `x11vnc`. ## Get all process statuses Get the status of every desktop process. The response is a map keyed by process name, with one entry each for D-Bus, Xvfb, the window manager, and the x11vnc server. Each entry reports whether the process is alive, its supervisor restart state, and its startup priority. | **API field** | **Description** | | ---------------------- | ------------------------------------------------------------------- | | **`Running`** | Whether the process is alive. | | **`Pid`** | Process ID while the process is running. | | **`Status`** | `running`, `stopped`, or `failed`. | | **`Failed`** | Whether the process entered the terminal failed state. | | **`AutoRestart`** | Whether the supervisor restarts the process automatically. | | **`RestartAttempts`** | Restart attempts consumed by the current generation of the process. | | **`RestartBackoffMs`** | Current restart backoff in milliseconds. | | **`Priority`** | Relative startup priority within the desktop stack. | ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() for name, process in sandbox.computer_use.service.status().items(): print(f"{name}: {process.status} pid={process.pid} restarts={process.restart_attempts}") ``` - The call returns a dictionary keyed by process name: `dbus`, `xvfb`, `wm`, and `x11vnc`. - The API fields map to snake_case attributes, for example `RestartAttempts` to `restart_attempts` and `AutoRestart` to `auto_restart`. ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const statuses = await sandbox.computerUse.service.status(); for (const [name, process] of Object.entries(statuses)) { console.log(`${name}: ${process.status} pid=${process.pid} restarts=${process.restartAttempts}`); } ``` - The call returns a record keyed by process name: `dbus`, `xvfb`, `wm`, and `x11vnc`. - The API fields map to camelCase properties, for example `RestartAttempts` to `restartAttempts` and `AutoRestart` to `autoRestart`. ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process-status' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Restart process Restart a specific desktop process. Per-process restart works only while the desktop is ready. The Xvfb process cannot be restarted on its own because application windows live inside it; use [restart](#restart-computer-use) instead. The D-Bus, window manager, and x11vnc processes are restarted automatically if they exit: up to 6 attempts with exponential backoff from 1 s to 30 s, reset after 30 s of uptime. When attempts run out, the process is marked failed in the status map while the desktop stays ready. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() result = sandbox.computer_use.service.restart("wm") print(result.message, result.process_name) ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const result = await sandbox.computerUse.service.restart('wm'); console.log(result.message, result.processName); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process/{processName}/restart' \ --request POST \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Get process logs Get logs for a specific desktop process. Logs are retained up to 256 KiB per process and trimmed to the most recent 128 KiB while the process runs. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() logs = sandbox.computer_use.service.logs("x11vnc") print("x11vnc logs:", logs.logs) ``` The response has the following fields: | **Field** | **Description** | | --------- | --------------- | | `logs` | Holds the retained standard output of the process. | - [Get process errors](#get-process-errors) returns standard error. ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const logs = await sandbox.computerUse.service.logs('x11vnc'); console.log('x11vnc logs:', logs.logs); ``` The response has the following fields: | **Field** | **Description** | | --------- | --------------- | | `logs` | Holds the retained standard output of the process. | - [Get process errors](#get-process-errors) returns standard error. ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process/{processName}/logs' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Get process errors Get error logs for a specific desktop process. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() errors = sandbox.computer_use.service.errors("x11vnc") print("x11vnc errors:", errors.errors) ``` The response has the following fields: | **Field** | **Description** | | --------- | --------------- | | `errors` | Holds the retained standard error output of the process. The output is capped at 256 KiB, like the logs. The value is an empty string when the process has written nothing to standard error. | ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const errors = await sandbox.computerUse.service.errors('x11vnc'); console.log('x11vnc errors:', errors.errors); ``` The response has the following fields: | **Field** | **Description** | | --------- | --------------- | | `errors` | Holds the retained standard error output of the process. The output is capped at 256 KiB, like the logs. The value is an empty string when the process has written nothing to standard error. | ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/process/{processName}/errors' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Mouse operations Every mouse, keyboard, screenshot, display, accessibility, and recording-start call requires the desktop to be ready; otherwise it returns 409 with a desktop not ready error. Coordinates are screen pixels in the range -32768 to 32767; values outside it return 400. ### Click Click the mouse at the specified coordinates with the left, right, or middle button, as a single or a double click. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() sandbox.computer_use.mouse.click(x=100, y=200) ``` | **Parameter** | **Type** | **Description** | | ------------- | ------------------------ | -------------------- | | `button` | `ComputerUseMouseButton` | Sets the mouse button. The default is `"left"`. | | `double` | `bool` | Makes the click a double click. The default is `False`. | - The call returns the pointer position after the click. ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); await sandbox.computerUse.mouse.click({ x: 100, y: 200 }); ``` | **Parameter** | **Type** | **Description** | | ------------- | ------------------------ | -------------------- | | `button` | `ComputerUseMouseButton` | Sets the mouse button. The default is `'left'`. | | `double` | `boolean` | Makes the click a double click. The default is `false`. | - The call returns the pointer position after the click. ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/click' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "x": 100, "y": 200 }' ``` | **Field** | **Type** | **Description** | | --------- | --------- | ------------------- | | `button` | `string` | Sets the mouse button. The default is `left`. | | `double` | `boolean` | Makes the click a double click. The default is `false`. | The response has the following fields: | **Field** | **Description** | | --------- | --------------- | | `x` | Holds the x coordinate of the pointer after the click. | | `y` | Holds the y coordinate of the pointer after the click. | ### Move Move the mouse cursor to the given coordinates. The response holds the final pointer position. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() result = sandbox.computer_use.mouse.move(x=100, y=200) print(f"Mouse moved to: {result.x}, {result.y}") ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const result = await sandbox.computerUse.mouse.move({ x: 100, y: 200 }); console.log(`Mouse moved to: ${result.x}, ${result.y}`); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/move' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "x": 100, "y": 200 }' ``` ### Drag Drag the mouse from start to end coordinates while holding the selected button. The pointer moves in 10 interpolated steps so applications that ignore a teleporting drag still register it. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() result = sandbox.computer_use.mouse.drag( start_x=50, start_y=50, end_x=150, end_y=150, button="left" ) print(f"Drag ended at {result.x}, {result.y}") ``` | **Parameter** | **Type** | **Description** | | ------------- | ------------------------ | ------------------ | | `button` | `ComputerUseMouseButton` | Sets the mouse button to hold. The default is `"left"`. | ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const result = await sandbox.computerUse.mouse.drag({ startX: 50, startY: 50, endX: 150, endY: 150, button: 'left', }); console.log(`Drag ended at ${result.x}, ${result.y}`); ``` | **Parameter** | **Type** | **Description** | | ------------- | ------------------------ | --------------- | | `button` | `ComputerUseMouseButton` | Sets the mouse button to hold. This parameter is required. | ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/drag' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "button": "left", "endX": 150, "endY": 150, "startX": 50, "startY": 50 }' ``` | **Field** | **Type** | **Description** | | --------- | -------- | ---------------- | | `button` | `string` | Sets the mouse button to hold. The default is `left`. | ### Scroll Scroll the mouse wheel up or down at the specified coordinates by a number of wheel ticks. One tick is one notch of a physical mouse wheel. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() result = sandbox.computer_use.mouse.scroll(x=100, y=200, direction="down") print(result.success) ``` - The default scroll amount is `1`. - The scroll amount must not be negative. - The maximum scroll amount is `100`. ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const result = await sandbox.computerUse.mouse.scroll({ x: 100, y: 200, direction: 'down' }); console.log(result.success); ``` - The default scroll amount is `1`. - The scroll amount must not be negative. - The maximum scroll amount is `100`. ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/scroll' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "direction": "down", "x": 100, "y": 200 }' ``` - The default scroll amount is `1`. - The scroll amount must not be negative. - The maximum scroll amount is `100`. ### Get position Get the current mouse cursor position. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() position = sandbox.computer_use.mouse.position() print(f"Mouse is at: {position.x}, {position.y}") ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const position = await sandbox.computerUse.mouse.position(); console.log(`Mouse is at: ${position.x}, ${position.y}`); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/mouse/position' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Keyboard operations Send keyboard input to the desktop. Type a text string, press one key with optional modifiers, or press a hotkey combination. Key names resolve against the keyboard map of the virtual display, and, like mouse calls, keyboard calls require the desktop to be ready. ### Type Type arbitrary text, including uppercase letters, symbols, and non-ASCII characters. Newlines are sent as Enter and tabs as Tab. Characters with no key in the X11 keymap are typed through a temporary key mapping. Set a delay between characters to slow typing down for applications that drop fast input. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() sandbox.computer_use.keyboard.type(text="Hello, World!") ``` - The call sends `\n` and `\r` as Enter. - The delay between characters is in milliseconds. - The default delay is `0`. - The maximum delay is `1000` per character. - The text is limited to 4096 characters. - The number of characters multiplied by the delay must not exceed 25 s. ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); await sandbox.computerUse.keyboard.type({ text: 'Hello, World!' }); ``` - The call sends `\n` and `\r` as Enter. - The delay between characters is in milliseconds. - The default delay is `0`. - The maximum delay is `1000` per character. - The text is limited to 4096 characters. - The number of characters multiplied by the delay must not exceed 25 s. ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/keyboard/type' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "text": "Hello, World!" }' ``` - The call sends `\n` and `\r` as Enter. - The delay between characters is in milliseconds. - The default delay is `0`. - The maximum delay is `1000` per character. - The text is limited to 4096 characters. - The number of characters multiplied by the delay must not exceed 25 s. ### Press Press a key with optional modifiers. Modifiers are held in order, the key is pressed and released, and the modifiers are released in reverse, even if the press fails. See [supported keys](#supported-keys). ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() sandbox.computer_use.keyboard.key(key="enter") ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); await sandbox.computerUse.keyboard.key({ key: 'enter' }); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/keyboard/key' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "key": "enter" }' ``` ### Hotkey Press a hotkey combination. The combination is a plus-separated chord string: the last entry is the key, the others are modifiers. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() sandbox.computer_use.keyboard.hotkey(keys="ctrl+c") ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); await sandbox.computerUse.keyboard.hotkey({ keys: 'ctrl+c' }); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/keyboard/hotkey' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "keys": "ctrl+c" }' ``` ### Supported keys Key names are case-insensitive. The following are supported for the key and modifiers parameters and for the parts of the keys parameter: | Category | Keys | | ------------- | ------------------------------------------------------------------------------------------------------------- | | Modifiers | `ctrl` (alias `control`), `alt`, `shift`, `cmd` (aliases `super`, `meta`, `win`) | | Editing | `enter` (alias `return`), `tab`, `escape` (alias `esc`), `backspace`, `delete` (alias `del`), `space` | | Navigation | `home`, `end`, `pageup` (alias `page_up`), `pagedown` (alias `page_down`), `insert`, `up`, `down`, `left`, `right` | | Function keys | `f1` through `f12` | | Symbols | `minus`, `plus`, `equal` (alias `equals`) | | Characters | Any single character, for example `a`, `A`, `5`, `/`, `-` | The SDK modifiers type accepts ctrl, alt, shift, and cmd; the API also accepts the aliases. An unknown key or modifier returns 400. ## Accessibility operations Use accessibility operations to inspect the AT-SPI tree and interact with UI elements by node ID. Start Computer Use before calling accessibility methods. Each node carries an ID, role, name, description, bounds (x, y, width, height), states, actions, and children. Node IDs have the form `:` and are valid only while the owning application keeps the object alive; an unknown or malformed ID returns 404 or 400. The scope parameter selects the root of the search: | **Scope** | **Root** | | ------------- | -------------------------------------------------------------------------------------------------- | | **`focused`** | The application whose state includes `active`, else the first registered application. Default. | | **`pid`** | The application owned by process `pid`. Returns `404` if no accessible application has that PID. | | **`all`** | The AT-SPI registry root, covering every application. | Traversal is bounded: 2000 nodes and 2 s for the tree, 5000 nodes and 2 s for find. When a bound is reached, the response is marked as truncated. Accessibility operations read the semantic UI information that applications expose over AT-SPI. Apps or custom widgets that do not expose accessibility objects may return sparse nodes, generic roles, or no actionable nodes; mouse, keyboard, and screenshot operations remain available for those cases. ### Get tree Read an accessibility tree for the focused app, a specific process, or all apps. The maximum depth parameter limits the depth below the root: 0 returns only the root, and omitting it or passing -1 returns the whole subtree within the traversal budget. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() tree = sandbox.computer_use.a11y.tree(scope="focused") print(tree.root.role, tree.root.name, tree.truncated) ``` | **Parameter** | **Type** | **Description** | | ------------- | ---------------------- | --------------------------------------------------- | | `scope` | `ComputerUseA11yScope` | Sets the root of the search: `"focused"`, `"pid"`, or `"all"`. With `scope="pid"`, also pass `pid`. This parameter is required. | | `max_depth` | `int` | Limits the depth below the root. Omit the parameter to get the whole subtree. | ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const tree = await sandbox.computerUse.a11y.tree({ scope: 'focused' }); console.log(tree.root.role, tree.root.name, tree.truncated); ``` | **Parameter** | **Type** | **Description** | | ------------- | ---------------------- | --------------------------------------------------- | | `scope` | `ComputerUseA11yScope` | Sets the root of the search: `'focused'`, `'pid'`, or `'all'`. With `scope: 'pid'`, also pass `pid`. This parameter is required. | | `maxDepth` | `number` | Limits the depth below the root. Omit the parameter to get the whole subtree. | ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/tree?scope=focused' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` | **Field** | **Type** | **Description** | | ---------- | --------- | ------------------------------------------------------- | | `scope` | `string` | Sets the root of the search. An unknown scope returns `400`. The default is `focused`. | | `pid` | `integer` | Selects the application when `scope=pid`. | | `maxDepth` | `integer` | Limits the depth below the root. The query parameter name is camelCase. | ### Find nodes Search the accessibility tree by role, accessible name, state, and scope. All filters must match. Node states: active, armed, busy, checked, collapsed, defunct, editable, enabled, expandable, expanded, focusable, focused, has-tooltip, horizontal, iconified, modal, multi-line, multiselectable, opaque, pressed, resizable, selectable, selected, sensitive, showing, single-line, stale, transient, vertical, visible, manages-descendants, indeterminate, required, truncated, and animated. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() buttons = sandbox.computer_use.a11y.find(scope="focused", role="button", name="Submit") for node in buttons.matches: print(node.id, node.name, node.bounds.x, node.bounds.y) ``` | **Parameter** | **Type** | **Description** | | ------------- | -------------------------- | ---------------------------------------------------------------------------- | | `scope` | `ComputerUseA11yScope` | Sets the root of the search. This parameter is required. | | `role` | `str` | Matches the node role exactly, for example `button` or `entry`. | | `name` | `str` | Matches the accessible name according to `name_match`. | | `name_match` | `ComputerUseA11yNameMatch` | Sets how `name` is compared: `exact`, `substring`, or `regex`. The default is `exact`. | | `states` | `Sequence[str]` | Requires the node to have every listed state, for example `["enabled", "focusable"]`. | | `limit` | `int` | Caps the number of matches. With more matches, `truncated` is `True`. The default is `50`. | - Filters other than `scope` are optional. - An omitted filter matches every node. ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const buttons = await sandbox.computerUse.a11y.find({ scope: 'focused', role: 'button', name: 'Submit' }); for (const node of buttons.matches) { console.log(node.id, node.name, node.bounds.x, node.bounds.y); } ``` | **Parameter** | **Type** | **Description** | | ------------- | -------------------------- | ---------------------------------------------------------------------------- | | `scope` | `ComputerUseA11yScope` | Sets the root of the search. This parameter is required. | | `role` | `string` | Matches the node role exactly, for example `'button'` or `'entry'`. | | `name` | `string` | Matches the accessible name according to `nameMatch`. | | `nameMatch` | `ComputerUseA11yNameMatch` | Sets how `name` is compared: `'exact'`, `'substring'`, or `'regex'`. The default is `'exact'`. | | `states` | `string[]` | Requires the node to have every listed state, for example `['enabled', 'focusable']`. | | `limit` | `number` | Caps the number of matches. With more matches, `truncated` is `true`. The default is `50`. | - Filters other than `scope` are optional. - An omitted filter matches every node. ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/find' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{ "scope": "focused", "role": "button", "name": "Submit" }' ``` | **Field** | **Type** | **Description** | | ----------- | ---------- | -------------------------------------------------------------------------------------------- | | `scope` | `string` | Sets the root of the search. The default is `focused`. | | `role` | `string` | Matches the node role exactly, for example `button` or `entry`. | | `name` | `string` | Matches the accessible name according to `nameMatch`. | | `nameMatch` | `string` | Sets how `name` is compared: `exact`, `substring`, or `regex`. The default is `exact`. | | `states` | `string[]` | Requires the node to have every listed state, for example `["enabled", "focusable"]`. | | `limit` | `integer` | Caps the number of matches. A value of `0` or less uses the default. With more matches, `truncated` is `true`. The default is `50`. | ### Focus node Focus a node from [get tree](#get-tree) or [find nodes](#find-nodes). ```python sandbox.computer_use.a11y.focus(id=node.id) ``` ```typescript await sandbox.computerUse.a11y.focus({ id: node.id }); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/node/focus' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{"id":"node-id"}' ``` ### Invoke node Run a node action, such as pressing a button. Without an action name, the node's click action is invoked, falling back to its first action. An action name must appear in the node's actions list. ```python sandbox.computer_use.a11y.invoke(id=node.id) ``` ```typescript await sandbox.computerUse.a11y.invoke({ id: node.id }); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/node/invoke' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{"id":"node-id"}' ``` ### Set node value Write text content to nodes that implement editable text, such as entries and text areas. The value replaces the current content. ```python sandbox.computer_use.a11y.set_value(id=node.id, value="hello") ``` ```typescript await sandbox.computerUse.a11y.setValue({ id: node.id, value: 'hello' }); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/a11y/node/value' \ --request POST \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer YOUR_API_KEY' \ --data '{"id":"node-id","value":"hello"}' ``` ## Screenshot operations Screenshots return the image as a base64 string, its encoded size in bytes, and, when the cursor is requested, the pointer coordinates. The cursor is never drawn. Full-screen and region screenshots are PNG; compressed variants accept format, quality, and scale parameters. ### Take full screen Take a screenshot of the entire screen. ```python import base64 from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() screenshot = sandbox.computer_use.screen.capture() print(f"Screenshot size: {screenshot.size_bytes} bytes") with open("screen.png", "wb") as f: f.write(base64.b64decode(screenshot.screenshot)) ``` ```typescript import { writeFile } from 'node:fs/promises'; import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const screenshot = await sandbox.computerUse.screen.capture(); console.log(`Screenshot size: ${screenshot.sizeBytes} bytes`); await writeFile('screen.png', Buffer.from(screenshot.screenshot, 'base64')); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/screenshot' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ### Take region Take a screenshot of a specific region. The width and height must be positive, and the region must lie within the screen. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() screenshot = sandbox.computer_use.screen.region(x=100, y=100, width=300, height=200) print(f"Captured region: {screenshot.size_bytes} bytes") ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const screenshot = await sandbox.computerUse.screen.region({ x: 100, y: 100, width: 300, height: 200 }); console.log(`Captured region: ${screenshot.sizeBytes} bytes`); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/screenshot/region?x=100&y=100&width=300&height=200' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ### Take compressed Take a compressed screenshot of the entire screen. ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() screenshot = sandbox.computer_use.screen.capture_compressed(format="jpeg", quality=80) print(f"Compressed size: {screenshot.size_bytes} bytes") ``` | **Parameter** | **Type** | **Description** | | ------------- | -------- | --------------------------------------------------------------- | | `format` | `str` | Sets the image format, `png` or `jpeg`. The default is `png`. | | `quality` | `int` | Sets the image quality, from `1` to `100`. The quality applies to `jpeg`. The default is `85`. | | `scale` | `float` | Resizes the captured image by a factor from `0.1` to `1.0`. The default is `1.0`. | | `show_cursor` | `bool` | Returns the pointer coordinates with the screenshot. The cursor is never drawn. The default is `False`. | ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const screenshot = await sandbox.computerUse.screen.captureCompressed({ format: 'jpeg', quality: 80 }); console.log(`Compressed size: ${screenshot.sizeBytes} bytes`); ``` | **Parameter** | **Type** | **Description** | | ------------- | --------- | ----------------------------------------------------------- | | `format` | `string` | Sets the image format, `'png'` or `'jpeg'`. The default is `'png'`. | | `quality` | `number` | Sets the image quality, from `1` to `100`. The quality applies to `'jpeg'`. The default is `85`. | | `scale` | `number` | Resizes the captured image by a factor from `0.1` to `1.0`. The default is `1.0`. | | `showCursor` | `boolean` | Returns the pointer coordinates with the screenshot. The cursor is never drawn. The default is `false`. | ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/screenshot/compressed?format=jpeg&quality=80' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` | **Field** | **Type** | **Description** | | ------------- | --------- | ----------------------------------------------------------- | | `format` | `string` | Sets the image format, `png` or `jpeg`. The default is `png`. | | `quality` | `integer` | Sets the image quality, from `1` to `100`. The quality applies to `jpeg`. The default is `85`. | | `scale` | `number` | Resizes the captured image by a factor from `0.1` to `1.0`. The default is `1.0`. | | `show_cursor` | `boolean` | Returns the pointer coordinates with the screenshot. The cursor is never drawn. The default is `false`. | ### Take compressed region Take a compressed screenshot of a specific region. Accepts the region parameters of [take region](#take-region) and the compression parameters of [take compressed](#take-compressed). ```python from daytona import Daytona daytona = Daytona() sandbox = daytona.v2.create() sandbox.computer_use.start() screenshot = sandbox.computer_use.screen.region_compressed( x=0, y=0, width=800, height=600, format="jpeg", quality=80 ) print(f"Compressed size: {screenshot.size_bytes} bytes") ``` ```typescript import { Daytona } from '@daytona/sdk'; const daytona = new Daytona(); const sandbox = await daytona.v2.create(); await sandbox.computerUse.start(); const screenshot = await sandbox.computerUse.screen.regionCompressed({ x: 0, y: 0, width: 800, height: 600, format: 'jpeg', quality: 80, }); console.log(`Compressed size: ${screenshot.sizeBytes} bytes`); ``` ```bash curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/screenshot/region/compressed?x=0&y=0&width=800&height=600&format=jpeg&quality=80' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Screen Recording Computer Use records desktop sessions for debugging, documentation, or automation workflows. Recordings are MP4 files captured at 10 frames per second and stored inside the sandbox; the file path field in every recording response reports the location. Each recording has the following fields: | **Field** | **Description** | | ---------------------- | --------------------------------------------------------------------------------------------------------- | | `id` | Recording ID, prefixed `rec_`. Used for stop, get, download, and delete. | | `file_name` | `