Original title: vercel-labs/agent-desktop
Vercel Labs' "computer operation" tool: instead of relying on screenshots and pixel matching, it reads the accessibility tree of macOS, Windows, and Linux desktop applications, assigns a reference to each operable element, and the agent clicks, types, and then re-reads the interface to confirm the result. 11 5 released 1.2.0 added agent-desktop mcp, providing each command as an MCP tool via stdio; it also supports reading text to be typed from a pipe, selecting list rows on macOS that can only be changed via containers, and can be installed directly from the default Homebrew repository.
Why it matters What's new compared to previous issues: no longer need to teach the agent to call command lines in prompts; MCP-supported clients can directly operate desktop applications.
Stars 16868 (+2173 / 24h)
Reason
- 2173 new stars in 24 hours, total stars 16868 (only indicates attention level).
Risks
- Snapshots, clipboard, and screenshots may contain private content; the README warns against putting them in public logs
- Requires granting system accessibility permissions