agent-desktop

agent-desktop is a command-line tool and companion skill from Vercel Labs that reads and interacts with desktop applications via the operating system's accessibility tree: clicking buttons, filling forms, reading UI states, navigating menus, checking boxes, scrolling, dragging, typing, using keyboard shortcuts, taking screenshots, managing windows, accessing the clipboard, launching and closing applications, and reading notifications. Each command returns a JSON; it is merely a tool, and the "observe-operate-verify" loop is performed by the agent invoking it.

agent-desktop is a command-line tool and companion skill from Vercel Labs that reads and interacts with desktop applications via the operating system's accessibility tree: clicking buttons, filling forms, reading UI states, navigating menus, checking boxes, scrolling, dragging, typing, using keyboard shortcuts, taking screenshots, managing windows, accessing the clipboard, launching and closing applications, and reading notifications. Each command returns a JSON; it is merely a tool, and the "observe-operate-verify" loop is performed by the agent invoking it.

Why it matters Instead of guessing coordinates from screenshots, it clicks buttons, fills forms, and reads states based on UI elements, making desktop automation more stable.

Security Audit: Not audited

  • Not yet scanned

Reasons

  • skills.sh 24 hours installed 433 (indicates only attention level)

Usage

  • First install the agent-desktop command line, run agent-desktop status to check permissions, then use snapshot to get element references before operating

Risks

  • Not yet audited, installation commands are not displayed
  • Requires granting accessibility permissions to the terminal; the agent can operate any application on the computer, and sensitive operations require manual confirmation

Open · Repository

Best Skills 每日数据(LinklyAI / @BlueeonY,CC BY 4.0)

Explore the latest AI Radar updates