Release day
Mac Computer Use is a small Mac app that lets Claude Code, Codex and other MCP clients see and use your apps. It works in the background, leaves your pointer alone, and stops the moment you press esc.
Below is a sketch of an agent at work, drawn in your browser with the app's own cursor. Your pointer is yours the whole time.
iamngoni / mac-computer-use
v0.10.1 Draft
Resolve windows through focused AX references.
Publish v0.10.1 now?
Mac Computer Use gives an agent eyes and hands on your Mac, through the same Accessibility interface that screen readers use. It works beside you, not instead of you: clicks go to the right window in the background, and your pointer stays where you left it. Every window is checked before anything happens, so an agent can't click into the wrong app. And you're always in charge: risky buttons wait two seconds for you, and Esc stops every agent at once.
Clicks use the app's own Accessibility actions where possible. Typing, scrolling and dragging go to one checked window. You keep working in another.
You grant Accessibility and Screen Recording to Mac Computer Use, not to every agent. The first time a new client tries to act, the app asks you, and it remembers that client by its code signature.
Its cursor flies to each target before acting. It can point at things, draw on screen, ask you a question, or hand you a password step and wait.
esc pauses every agent. Send, Delete and Buy get a two-second countdown you can cancel. Agents wait while you're using the same app.
Open the DMG and drag Mac Computer Use into Applications. It lives in the menu bar and updates itself between tool calls.
The setup window shows whether Accessibility and Screen Recording are on, and opens the right System Settings pane for each.
Setup registers Claude Code or Codex with one click. Or run this yourself:
claude mcp add --scope user mac-computer-use -- \
"/Applications/MacComputerUse.app/Contents/MacOS/mac-computer-use" mcpWhat an agent can do once it's connected.
get_app_state A window's screenshot plus a numbered list of its buttons, fields and text.list_apps list_windows What's running, and each window's exact ID.get_desktop_state The whole screen, on macOS 14 and later.verify_state Wait until something appears or goes away.click type_text press_key scroll drag In the background, by element or by position.set_value select_text invoke_menu Direct, without hunting for pixels.open_app navigate set_window_frame Open apps, go to URLs, move windows.point_at annotate show_visual Point, draw, or put a live chart next to the thing it explains.ask_user suggest pick_element Questions only your click can answer.guide say magnify Walkthroughs, spoken words with captions, a zoom lens.wait_for_user Give you the 2FA or password step and wait.watch_user Watch your "now you try" step; never records what you type.job A card with progress, Pause and Stop for long tasks.health_report Permissions and status as JSON.No. Clicks, scrolling and dragging go to a checked app and window without moving your pointer. The agent's cursor is drawn separately. Agents can control the whole desktop only when they ask for it explicitly with allow_global_input.
Accessibility and Screen Recording, granted once to Mac Computer Use. Claude Code, Codex and other clients never need them. The app asks you once per client before it may act.
Press esc. Every agent pauses until you resume from the menu bar, and the interrupted action reports that you stopped it. Quitting the app turns automation off.
Yes. It's open source, needs no account, and signed releases update themselves.
macOS 13 Ventura or later, on Apple silicon and Intel. Whole-desktop capture needs macOS 14.
It's free, works offline, and updates itself.