# Automate Desktop Apps With Accessibility Controls

Desktop workflows often require repeated navigation, data entry, and checks across application windows. This skill guides agents through scoped accessibility actions and verification using agent-desktop.

## Install

```bash
npx skillstore add vercel-labs/agent-desktop
```

## Metadata

- Status: approved
- Slug: vercel-labs-agent-desktop
- Skillstore revision: r1
- Version status: missing
- Tree hash: b328acda88308da904e5ff151f450ccdab5ce12259d31b5d02ec217e2a42fca8
- Author: vercel-labs
- GitHub username: vercel-labs
- License: MIT
- Repository: https://github.com/vercel-labs/agent-desktop/tree/5261b5287b5208e7c4bf3713df6f66faaaf2c4cc/skills/agent-desktop
- Ref: 443e0d99cb0af7754aabaecc5bf8b6dcf2dae44d
- Supported tools: Claude, Codex, Claude Code
- Audit status: complete
- Agent install advisory: allowed
- Manual install advisory: allowed
- Artifact signature: available
- Audit attestation: unavailable
- Human verification: not\_verified
- Risk factors: external\_commands, network, filesystem
- Quality score: 67
- Public page: https://skillstore.pages.dev/skills/vercel-labs-agent-desktop
- Manifest: https://skillstore.pages.dev/api/skills/vercel-labs-agent-desktop/manifest

## Capabilities

- Inspect accessibility trees, find controls by role or name, and read live element properties.
- Click controls, enter text, select options, and set checkbox states with action-result verification.
- Navigate native menus, dialogs, and overlays using snapshot-qualified element references.
- Launch and close apps, manage windows, and wait for text, element, or lifecycle changes.
- Capture screenshots, manage clipboard representations, and inspect supported notification surfaces.
- Organize runs into sessions, inspect traces, and hand off Chromium web contents to an optional CDP client.

## Use Cases

- Complete Repeated Desktop Forms: Enter approved data into application fields, select options, and verify the completed form before submission.
- Check Desktop UI Workflows: Inspect controls, exercise menus and dialogs, and distinguish verified changes from uncertain input delivery.
- Review Application Notifications: List notifications from a selected app and act on specific entries only after checking their current fingerprints.

## Prompt Templates

### Inspect an Application

```
Check agent-desktop status and permissions, then inspect [app]. List its visible controls without changing anything.
```

### Fill a Form Without Submitting

```
In [app], fill [fields] with [approved values]. Use current references and verify each field. Stop before submitting or enabling physical input.
```

### Navigate a Save Dialog

```
In [app], save the current document to [approved destination]. Inspect each menu or dialog surface. Ask before overwriting files or using physical input.
```

### Run a Scoped Multi-Step Workflow

```
Run [approved workflow] in [app] within a dedicated session. Use event waits and stop batches on errors. Verify uncertain deliveries before any repeat. Keep screenshot tracing off.
```

## Limitations

- Requires the agent-desktop binary, an active desktop session, and the permissions required by each operating system.
- Accessibility coverage and headless action support vary by app and platform; physical input requires explicit user permission.
- A delivered action does not always prove application acceptance; batches are not transactions and uncertain actions need independent verification.
- Clipboard data, notification bodies, and optional screenshot artifacts can contain sensitive information; loopback CDP allows same-user local control.

## Best Practices

- Check status and permissions, then scope reads and actions to the intended app, window, or overlay.
- Use fresh qualified references, event waits, and independent state checks before repeating an uncertain action.
- Require explicit permission for physical input and minimize sensitive artifacts, clipboard reads, and CDP exposure.

## Anti Patterns

- Retrying a click, submission, or toggle merely because a wait timed out.
- Switching to physical input after a headless refusal without explicit user permission.
- Sharing unredacted screenshots, notifications, or clipboard contents, or running parallel writers against one editor.

## Security Audit

- Audited at: 2026-10-31T14:11:33.437\+00:00
- Summary: All 67 static findings are benign documentation matches, including Markdown backticks, keystroke delivery, local session storage, and loopback CDP endpoints. One semantic finding concerns pointer dispatch after inconclusive hit tests, which can misdirect input despite visibility checks. No evidence found of credential theft, covert keylogging, external exfiltration, or audit-directed prompt injection in the five reviewed files.

## Stats

- Views: 0
- Downloads: 0
- Favorites: 0
- Popularity score: 0