Skip to content
tarsk

Computer Use

The computer tool gives the agent a real desktop: it lists windows, reads accessibility trees, clicks, types, scrolls, and captures screenshots of native apps. Use it for work that has no CLI or API, such as changing a system setting or operating a GUI-only app.

Enable it

Section titled “Enable it”

The bundled tarsk-computer Tool skill publishes computer as a deferred tool, and new projects start with it off.

  1. Open Skills → Tools in Project Settings.
  2. Turn on Computer.
  3. Ask for the desktop task. The agent searches for the tool before it uses one.

Computer use runs on macOS, Windows, and Linux on arm64 and x64. On unsupported platforms the tool reports that computer use needs one of those systems.

First use installs the driver

Section titled “First use installs the driver”

Tarsk pins one CUA Driver release and checks its SHA-256 before use. On the first computer call it downloads the matching archive into the tool cache, starts the daemon, and runs the requested action.

Platform Cache location
macOS ~/Library/Caches/Tarsk/tools
Windows %LOCALAPPDATA%\Tarsk\Cache\tools
Linux $XDG_CACHE_HOME/tarsk/tools (~/.cache fallback)

On macOS the driver starts as the signed CuaDriver.app bundle so that Accessibility and Screen Recording grants stick. macOS may ask for both permissions on first use. If the daemon fails to start, run cua-driver permissions grant and retry.

You can override the driver setup with environment variables:

Variable Effect
TARSK_TOOL_CACHE_DIR Move the tool cache, including the driver binary
TARSK_CUA_DRIVER_PATH Use an existing CUA Driver binary instead of the managed one
TARSK_CUA_DRIVER_SOCKET Point at a daemon socket you manage
TARSK_CUA_DRIVER_AUTO_INSTALL=0 Skip the managed download; fail with instructions instead

The daemon is shared. It keeps running after Tarsk exits, and other CUA clients can use it. Cancelling a run revokes that conversation’s session; it does not stop the daemon.

Approval

Section titled “Approval”

Discovery calls such as list_apps, list_windows, get_window_state, describe, and check_permissions run without a prompt. Every other action asks first:

Allow CUA Driver action click?

The prompt shows the complete action input, including values that arrive from a script. Tarsk never lets input text hide part of the action. Choose Allow once or Cancel.

What the agent can call

Section titled “What the agent can call”

The tool takes a CUA Driver tool name and its arguments.

Group Actions
Observe list_apps, list_windows, get_accessibility_tree, get_window_state, get_screen_size, get_guide, describe, check_permissions
Act click, double_click, right_click, type_text, press_key, hotkey, scroll, drag, set_value, move_cursor, invoke_menu, launch_app, bring_to_front, set_window_frame, kill_app
Batch run_steps, run_script
Clipboard clipboard_read, clipboard_write

Prefer get_window_state element tokens over screen coordinates. The agent passes a single named tool per call, not a shell command.

Screenshots

Section titled “Screenshots”

Each screenshot lands in .tarsk/tmp-images/ in the workspace and returns the saved path. The chat renders the image inline. Screenshots stay on disk until you remove them.

Privacy and limits

Section titled “Privacy and limits”

CUA Driver sends content-free product telemetry. Run cua-driver telemetry disable to stop it.

Tarsk does not install or bundle cua-perception. A call that misses the 60-second timeout is treated as hung, not slow.

Section titled “Related”