Mac Computer Use

Let your agent use your Mac, without taking it from you.

Mac Computer Use is a small Mac app that lets Claude Code, Codex and other MCP clients see and use your apps. It works in the background, leaves your pointer alone, and stops the moment you press esc.

Download for macOS

Latest version · Apple silicon and Intel · macOS 13 or later · ZIP

Below is a sketch of an agent at work, drawn in your browser with the app's own cursor. Your pointer is yours the whole time.

Notes

Release day

github.com

iamngoni / mac-computer-use

Releases

v0.10.1 Draft

Resolve windows through focused AX references.

Publish v0.10.1 now?

PublishNot yet
A sketch of what an agent does with Mac Computer Use. Your own pointer, top right, never moves.

Mac Computer Use gives an agent eyes and hands on your Mac, through the same Accessibility interface that screen readers use. It works beside you, not instead of you: clicks go to the right window in the background, and your pointer stays where you left it. Every window is checked before anything happens, so an agent can't click into the wrong app. And you're always in charge: risky buttons wait two seconds for you, and Esc stops every agent at once.

Background, not takeover

Clicks use the app's own Accessibility actions where possible. Typing, scrolling and dragging go to one checked window. You keep working in another.

Permissions, once

You grant Accessibility and Screen Recording to Mac Computer Use, not to every agent. The first time a new client tries to act, the app asks you, and it remembers that client by its code signature.

An agent you can see

Its cursor flies to each target before acting. It can point at things, draw on screen, ask you a question, or hand you a password step and wait.

Stops when you say so

esc pauses every agent. Send, Delete and Buy get a two-second countdown you can cancel. Agents wait while you're using the same app.

Set up in a few minutes.

  1. 1

    Install

    Open the DMG and drag Mac Computer Use into Applications. It lives in the menu bar and updates itself between tool calls.

  2. 2

    Grant two permissions

    The setup window shows whether Accessibility and Screen Recording are on, and opens the right System Settings pane for each.

  3. 3

    Connect your agent

    Setup registers Claude Code or Codex with one click. Or run this yourself:

    claude mcp add --scope user mac-computer-use -- \
      "/Applications/MacComputerUse.app/Contents/MacOS/mac-computer-use" mcp

37 tools.

What an agent can do once it's connected.

See
get_app_state A window's screenshot plus a numbered list of its buttons, fields and text.
list_apps list_windows What's running, and each window's exact ID.
get_desktop_state The whole screen, on macOS 14 and later.
verify_state Wait until something appears or goes away.
Act
click type_text press_key scroll drag In the background, by element or by position.
set_value select_text invoke_menu Direct, without hunting for pixels.
open_app navigate set_window_frame Open apps, go to URLs, move windows.
Show and ask
point_at annotate show_visual Point, draw, or put a live chart next to the thing it explains.
ask_user suggest pick_element Questions only your click can answer.
guide say magnify Walkthroughs, spoken words with captions, a zoom lens.
Hand over
wait_for_user Give you the 2FA or password step and wait.
watch_user Watch your "now you try" step; never records what you type.
job A card with progress, Pause and Stop for long tasks.
health_report Permissions and status as JSON.

The full list, with every option

Questions

Does it move my mouse?

No. Clicks, scrolling and dragging go to a checked app and window without moving your pointer. The agent's cursor is drawn separately. Agents can control the whole desktop only when they ask for it explicitly with allow_global_input.

Which permissions does it need?

Accessibility and Screen Recording, granted once to Mac Computer Use. Claude Code, Codex and other clients never need them. The app asks you once per client before it may act.

How do I stop an agent?

Press esc. Every agent pauses until you resume from the menu bar, and the interrupted action reports that you stopped it. Quitting the app turns automation off.

Is it free?

Yes. It's open source, needs no account, and signed releases update themselves.

Which Macs does it run on?

macOS 13 Ventura or later, on Apple silicon and Intel. Whole-desktop capture needs macOS 14.

Hand your agent the Mac.

It's free, works offline, and updates itself.