iPhone controlfor your agent

Features

• Any iOS app. Input goes through AssistiveTouch, like a real mouse. Nothing runs on the iPhone, no app and no test runner.
• Tap, double and triple tap, hold, swipe and hold-drag with the mouse pointer, at screenshot coordinates or on OCR text (tap --text, hold --text).
• Scroll with the mouse wheel, also in lists that a drag cannot scroll.
• Type text fast. type sends text as key presses. key sends Home, Enter, Esc, arrows and volume keys.
• Screenshots over USB, PNG or JPEG, about 0.3 s each. The capture stops 1 s after the last frame, so an idle daemon uses no CPU for it.
• Record the screen with sound. record start and record stop write H.264 and AAC. Actions and screenshots keep working during the recording.
• Screen points by default. Screenshots, OCR boxes and taps share one small coordinate space, sized for LLM vision.
• Read screen text with on-device OCR (Vision). The boxes are in screenshot coordinates.
• Unlock a locked iPhone with its passcode, with no touch. list shows the lock state.
• Lock with the Bluetooth keyboard. The daemon checks the result over USB.
• Mute and unmute all iPhone sound. The daemon confirms the change from the iPhone log.
• Files, photos and apps over USB. Pull screenshots and photos, push files, list and uninstall apps, and read battery, storage and the system log. No Developer Mode.
• Several iPhones at once, each controlled on its own.
• CLI, remote MCP server and skill for agents, with the same help text.
• Leases give one agent one iPhone at a time. Other agents wait or get a "leased by" error.
• HTTP API described by OpenAPI, for any language.
• Every Mac of your account through one MCP URL, or from your own scripts with the HTTP API and an API token.

Install

123npx ghosthand setup       # run without installing
bunx ghosthand setup      # same, with Bun
npm install -g ghosthand  # install the `ghosthand` command

npx ghosthand setup ──> Ghosthand app + daemon ──> sign in, allow access, plug in, pair
                                                        │
ghosthand list / screenshot / tap / type <──────────────┘ iPhone ready

123ghosthand account   # who is signed in, the plan, the subscribe link
ghosthand account login     # for people without the setup window; agents use setup
ghosthand account logout

1ghosthand setup  # open the app, then let the user click its permission buttons

Setup (once per iPhone)

12ghosthand setup status                     # JSON, works before permissions or USB trust
ghosthand setup status --device "iPhone XR" # select the intended phone when several exist

1. Plug in and trust

1ghosthand list

2. Pair the Mac as a mouse and keyboard

1ghosthand key home

• The Mac is discoverable only while a command waits for pairing.
• The Mac is not listed? Turn Bluetooth off and on again on the iPhone.
• After you tap Pair, the iPhone opens the connection, and the command continues by itself.
• Later runs reconnect the paired iPhone from the Mac in about 2 s.
• The link lasts only while the iPhone is on USB. While the link is up, iOS hides its onscreen keyboard. To type on the iPhone yourself, unplug the cable. Plug it back in and the link comes back within a few seconds, without pairing.
• After a report map change, forget the Mac on the iPhone and pair again. The iPhone keeps the map it read at pairing.
• Pairing fails, or the Mac is gone from the list? Forget on both sides, then pair again. See Pairing no longer works.

3. AssistiveTouch (automatic)

123ghosthand acquire --reason "reply to Anna"  # AssistiveTouch on
ghosthand release -l 42                     # lease released; AssistiveTouch turned off
ghosthand assistivetouch -l 42 on           # on for good: release leaves it on

• release turns off only what ghosthand turned on. If AssistiveTouch was on before acquire, for example for a broken Home button, release leaves it on.
• assistivetouch on|off sets the iPhone's own setting, so release leaves it as set.
• Expired leases count too. When a lease ends by its idle timeout, the daemon turns AssistiveTouch off a few seconds later.
• Tap, swipe and wake check it again, in case a person turned it off during the task.

Always Show Menu ON   ──> round button on screen ──> in screenshots and videos
Always Show Menu OFF  ──> hidden while the Mac mouse is connected; taps still work

• Set it on the iPhone, once. USB lockdown has a key to turn AssistiveTouch on and off (AssistiveTouchEnabledByiTunes), but no key for this switch. So ghosthand cannot change it over USB. An agent can change it with taps in Settings (tested on an XR, iOS 18.7).
• It stays off when acquire and release turn AssistiveTouch on and off.
• Apple Pay needs it on. Paying without Face ID uses the AssistiveTouch menu, so it needs Always Show Menu on and two more settings. See Pay with Apple Pay.

1ghosthand key home  # Home Screen

4. Keep awake (automatic)

daemon, every 2 s ──> unlocked and idle for 10 s? ──yes──> Left Control press (Bluetooth)
                            │                                    │
                            └─ locked or command running: skip   └─> iOS resets the Auto-Lock timer

• No visible effect. A Control key press alone types nothing and shows no pointer, so screenshots stay clean. It works with AssistiveTouch off.
• Never in the middle of input. A tap or typed text holds the Bluetooth link, and the key waits for it to finish.
• Locked iPhones are skipped. The lock state comes from USB lockdown. A key press would light the Lock Screen (seen on iOS 18), so a locked iPhone gets no key and its display turns off normally.
• To let the iPhone lock by itself, unplug USB or stop the daemon (ghosthand daemon stop). Keep-awake stops on the next check, within 2 seconds.
• To turn keep-awake off, run defaults write com.remorses.ghosthand KeepAwake -bool false and set Auto-Lock to Never. Do this, for example, if you use Switch Control with a keyboard switch, because the key counts as a switch press. The change applies on the next check, with no restart.

Use it with an agent

1npx -y skills add https://ghosthand.dev -g -y

1Use ghosthand to open Messages on my iPhone and reply "on my way" to Anna.

123ghosthand screentext
ghosthand tap --text "Anna" --screenshot s.png --screentext
ghosthand type "on my way" --enter --screentext

12ghosthand double-tap -l 42 120 300 --screenshot after.png
ghosthand double-tap -l 42 --text "example" --screentext

123ghosthand hold -l 42 144 324 --screentext          # Notes icon: its menu with Remove App
ghosthand hold -l 42 --text "Safari" --duration 1500
ghosthand key -l 42 esc                             # close the menu

12ghosthand hold-drag -l 42 144 324 57 412 --hold 2500   # move a Home Screen icon
ghosthand key -l 42 home                                # leave edit mode

Control plane: watch your agents

• The sidebar lists each iPhone with a status dot: blue in use, green free, orange locked, yellow connecting, red needs setup. A badge counts the agents that wait for it.
• Lease details show the agent label, the task, the lease age, the last command, the idle timeout, and the agents that run ghosthand wait.
• The screen is small and live at up to 30 fps while the window is visible. It gets no frames when the window is hidden or covered. It needs no lease and does not reset the agent's idle timer.
• Connection details show USB, Bluetooth, pointer mode, AssistiveTouch, and the device ID to copy for acquire -d.
• Take Control starts when you hover the screen and click it. Then a click on the screen taps the iPhone there, and a drag swipes with the same start, end and duration. The Mac keyboard types into the iPhone, shortcuts like ⌘A included, and ⌘V types the Mac clipboard as text. To stop, press Esc or click Release Control under the screen. It needs no lease, so it also works while an agent holds the iPhone, and your input goes between the agent's commands. In Bluetooth mode a tap or swipe turns AssistiveTouch on. Release Control turns it off again if no lease holds the iPhone. If the app crashes, it turns off 60 s after your last input. ⌘Space and ⌘Tab stay on the Mac unless you allow Ghosthand in System Settings > Privacy & Security > Accessibility.
• Agent chat is the right column. Type a task, and an agent on ghosthand.dev does it on the selected iPhone. It is the same agent as ghosthand task. Each prompt takes a lease and releases it when the agent is done. A new prompt stops the running one.

Agent feedback loop

      ┌─────────────────────────────────────────────────────────────────┐
      v                                                                 │
 1. see the screen           screentext (text + boxes, no image)        │
                             or screenshot -o s.png (~500 tokens)       │
 2. act, and see the result  tap / swipe / key / type                   │
                             + --screenshot s.png --screentext          │
 3. read the printed text, and s.png if the text is not enough          │
 4. decide the next action toward the user's task  ─────────────────────┘

123456export GHOSTHAND_LANGUAGE=it-IT                        # phone language for OCR, same as --language
ghosthand list
ghosthand screentext
ghosthand tap --text "Tommaso De Rossi" --screenshot s.png --screentext
ghosthand tap 200 610 --screenshot s.png --screentext  # x,y read from s.png
ghosthand type "ciao" --enter --screentext

• Text first, image when needed. --screentext output is cheap and often enough to decide, for example a new title or a sent message. Read s.png for icons, layout, or text that OCR misses.
• Both flags use one frame. The daemon runs them after the input, under the same lease, and waits for a still screen (up to 2 s after frames start). OCR reads the native frame, where small text reads better, and the image is resized from that same frame. A blank or unchanged image is not proof that input failed.
• Check focus before you type. type sends keys to the field that has focus, and the command does not show which field that is. Read a screenshot first and check that the cursor is in the field you want. In Messages, Enter in the To: field adds the recipient but keeps focus there, so type "ciao" --enter put "ciao" into To: (tested on iOS 18.7). Tap the message field, check the screenshot, then type.

  123ghosthand type "5551234567" --enter         # To: field, adds the recipient
  ghosthand tap 200 562 --screenshot s.png    # check focus in the image
  ghosthand type "ciao" --enter --screentext  # Enter sends

• Tap text, not pixels. tap --text "..." saves a screenshot read and a coordinate guess. On the Home Screen, app labels do not open apps (tested on iOS 15). Tap the icon, about 40 points above the label.
• Do not script wait loops. Do not write loops that wait for a text you guess, like "Open" or "Done". The next screen is hard to predict, because iOS shows sheets and alerts at any time. For example, the App Store asks to confirm Install, then can say "requires iOS 16 or later". A loop that waits for one word then waits forever, as happened while installing Reddit on the 6s. Use a loop only when a skill or document you follow asks for it. Otherwise use the plain feedback loop: do one action, read the whole screen, and decide the next action.
• Capture failed after an action? The error says Input sent. Do not repeat the action blindly. Take a screenshot first.
• No frames? screenshot and screentext reject a locked iPhone at once. If list reports the iPhone unlocked but capture still gets no frame, see USB screen capture stops.

1ghosthand tap 207 100 --screenshot s.png --screentext   # empty top area: full screen returns

123ghosthand screenshot -l 42 --native -o real.png            # read real.png: the dialog is there
ghosthand tap -l 42 120 401 --screenshot s.png              # tap the button you read
ghosthand hold -l 42 120 401 --duration 400                 # button ignored the tap? press it once more

123ghosthand key -l 42 cmd+shift+3                             # iOS saves a real screenshot
ghosthand photo pull -l 42 -o .                             # newest Camera Roll file, e.g. ./IMG_0004.PNG
ghosthand photo delete -l 42 IMG_0004.PNG                   # remove it from Photos again

• The native PNG is in native pixels (750x1334 on an SE2). screenshot --native scales it for you. With the manual steps, pass its numbers with tap --scale 1 (native pixels). See Coordinates.
• Deleting needs the optional Photos access on the Mac. Run ghosthand setup --permissions once and click Allow Photos with the iPhone plugged in and unlocked (macOS asks about removable volumes). Without this access, the screenshot stays in Photos and the output says which file to delete.
• A dialog that just opened can ignore the first press. On an SE2 (iOS 18.6, Perform Touch Gestures on), the Remove "Notes"? alert ignored the first press on Cancel in 8 of 8 trials and took the second. A tap, hold --duration 400 and a stationary swipe all behaved the same. Earlier reports said that a 400 ms press works where a tap fails (location prompt, Photos Delete Photo sheet), but those were second presses too. Read the screen, then press once more.
• Still blocked? Try a 5-second hold at verified coordinates. On an SE2 (iOS 26.5.2), Apple Account's Do Not Sign Out accepted ghosthand hold -l 42 --text "Do Not Sign Out" --duration 5000 after taps and 1500...2500 ms holds failed. A native screenshot then showed Sign-In & Security. This is one success, not proof that the duration caused it. Check the result before you repeat input.

12ghosthand lock -l 42
secret-provider | ghosthand unlock -l 42 --stdin --screenshot s.png --screentext

Example: turn off the SIM PIN

1Use ghosthand to turn off the SIM PIN on my iPhone. The PIN is SIM_PIN in sigillo.

123456789ghosthand acquire --reason "turn off SIM PIN"                 # prints 42
ghosthand tap -l 42 --text "Settings" --screentext             # or tap its Home Screen icon
ghosthand tap -l 42 --text "Cellular" --screentext             # "Mobile Service" in some regions
ghosthand tap -l 42 --text "SIM PIN" --screenshot s.png        # dual SIM: tap the line first
ghosthand tap -l 42 340 180 --screenshot s.png                 # the switch, x,y read from s.png
ghosthand tap -l 42 60 520 --screenshot s.png                  # one keypad digit; check one new dot
# ... one tap per digit, then check: exactly 4 dots
ghosthand tap -l 42 --text "Done" --screentext                 # once; the switch is now off
ghosthand release -l 42

• Tap the keypad, do not type. The SIM PIN sheet ignores the Bluetooth keyboard. type says "typed" and the field stays empty (tested on an SE2 with iOS 18.6). Tap each digit on the on-screen keypad and check that one dot appears after each tap.
• Tap the switch, not the label. Read its position from the screenshot.
• Wrong digit? Clear it with the keypad delete key before you tap Done. Done is the one try.

CLI

Secret text from stdin

1secret-provider | ghosthand type --stdin

Keyboard shortcuts

12ghosthand key -l 42 cmd+a
ghosthand key -l 42 backspace

Full Keyboard Access (optional)

• The focus box shows in screenshots. It is blue and can frame the whole screen. Turn on Auto-Hide to hide it after idle time.
• A command can do nothing. The first tab+f and the first tab+b did nothing, and the second press worked. Check the screen after each key.
• Dangerous chords. With it on, ctrl+alt+shift+cmd+s calls Emergency SOS and ctrl+alt+shift+cmd+r restarts the iPhone. Never send them.

Mute

12ghosthand mute  # muted iPhone XR Tommy
ghosthand unmute

Mac ──BT Mute key──> SpringBoard ──FullMute──> audiomxd
 ^                       │
 └──USB syslog_relay─────┘  "Updated fullyMuted to true"

Record the screen

1234ghosthand record start demo.mov --fps 30   # returns at once
ghosthand tap --text "Settings"
ghosthand record stop
# /Users/me/demo.mov  12.4s  2.1 MB  828x1792  30 fps  sound  stopped by request

123iPhone ──USB──> capture session ──┬──> latest frame ──> screenshot, screentext, tap --text
                                  └──> fps cap ──> AVAssetWriter (H.264 + AAC) ──> demo.mov
                 sound output ────────────────────────────┘  (added only while recording)

Bounded text verification

1ghosthand tap --text "General" --wait-text "About" --timeout-ms 5000

Files, photos and apps over USB

12345ghosthand photo pull -l 42 --last 3 -o ./photos      # newest Camera Roll files to this Mac
ghosthand file push -l 42 report.pdf /Downloads/report.pdf
ghosthand app list -l 42                             # bundle ids, versions, Home Screen page
ghosthand info -l 42                                 # battery, storage, iOS build, time zone
ghosthand logs -l 42 --process appstored --seconds 10

1234567                         ┌── com.apple.afc ─────────────> /var/mobile/Media (DCIM, Downloads)
ghosthand ──USB lockdown─┼── house_arrest ──────────────> an app's Documents (file sharing apps)
 (MobileDevice.framework)├── installation_proxy ────────> app list, install, uninstall
                         ├── springboardservices ───────> Home Screen page of each app
                         ├── diagnostics_relay ─────────> restart, shutdown
                         └── syslog_relay ──────────────> system log
ImageCaptureCore (like Image Capture.app) ──────────────> delete photos so Photos forgets them

Get photos and files from the iPhone

1234ghosthand photo list -l 42 --limit 5
# 845 KB    2026-10-10 10:52  /DCIM/100APPLE/IMG_0008.PNG
ghosthand file pull -l 42 /DCIM/100APPLE/IMG_0008.PNG shot.png
ghosthand file pull -l 42 /DCIM/100APPLE ./camera-roll   # whole folder

Send files to the iPhone

123ghosthand file push -l 42 report.pdf /Downloads/report.pdf
ghosthand file push -l 42 movie.mp4 /Documents/movie.mp4 --app org.videolan.vlc-ios
ghosthand file remove -l 42 /Downloads/report.pdf

Send media for uploads in apps

1234ghosthand upload clip.mp4          # https://ghosthand.dev/u/3fK9xQ2mZr7T, expires in 1 day
ghosthand key -l 42 cmd+space      # open Safari, then cmd+l, type the link, enter
ghosthand tap -l 42 --text Save    # wait until the button says Save, not Loading...
ghosthand tap -l 42 --text "Save Video"

• It needs a signed-in account, on any plan. It needs no lease, because the upload does not touch the iPhone.
• Files can be at most 2 GB. Anyone with the link can download the file for 1 day.
• Offline, or a bigger file? Use file push into VLC, then Files > Share > Save Video.

Delete photos

1ghosthand photo delete -l 42 IMG_0008.PNG

Apps

12345ghosthand app list -l 42
# com.burbn.instagram  Instagram 450.1.0  (page 2)
# com.example.beta  Beta 1.2  [installing]  (App Library only)
ghosthand app install -l 42 MyApp.ipa         # signed for this iPhone: ad hoc, enterprise, development
ghosthand app uninstall -l 42 com.example.app

• The Home Screen place is dock, page N, or page N, folder <name>. Swipe to that page, then tap the icon.
• [installing] marks an App Store download that is not done. Trust it over the App Store button label, which can show Resume during the download.
• Opening an app over USB needs Developer Mode, so ghosthand taps its icon instead.
• App Store apps install from the App Store app. app install takes only IPAs signed for this iPhone.

Device info, logs, restart

1234567ghosthand info -l 42
# battery: 100%, on power
# storage: 51.0 GB free of 57.4 GB
# sim tray: SIM card in
# sim line 2: Vodafone_Ho_it, physical SIM, MCC 222 MNC 10
ghosthand logs -l 42 --process SpringBoard --grep fullyMuted --seconds 5
ghosthand restart -l 42      # ask the user first

Leases: one agent per iPhone

123ghosthand acquire -d 7EF219AD-69B4-423E-B8C4-4F4A6140822F --label ses_123 --reason "reply to Anna"  # prints studio:42 (42 below)
ghosthand tap -l 42 207 448    # runs on the lease's iPhone; no -d
ghosthand release -l 42

agent A: acquire ──> lease 42 ──> tap -l 42, type -l 42 ────────> release -l 42
agent B:       acquire ──> error: in use by "ses_123" for "reply to Anna":
                           acquired 2m3s ago, last command 4s ago, ends after 9m56s idle
agent B:       decides: give up / tell the user, or
               wait --reason "..." --timeout 600 ─────── waits ──────> lease 43

• Always pass --reason with the task. If the phone is busy, acquire fails at once. The error shows the label, its reason, when it was acquired and its last command, so the next agent can decide to give up instead of waiting.
• wait is a separate command on purpose. An agent must read who holds the phone before it waits. wait retries every second, with no queue order. On timeout it shows the current label and reason.
• A lease ends 10 minutes after its last command (--idle-timeout), so a crashed agent never blocks the phone.
• --force takes the phone from a lease whose holder is gone. That holder's next command fails with lease-invalid.
• After wait or --force, take a screenshot first. Someone else used the phone.
• Pass the ID explicitly with -l. Tool-call environments may not keep variables between calls, so do not rely on an exported GHOSTHAND_LEASE.
• IDs only increase. SQLite stores the counter, so the daemon never reuses an ID after expiry, force, release or a daemon restart. Do not delete or restore state.db.
• --label is only display text, for example your agent session ID, so others can find who holds the phone. The daemon does not check it, it need not be unique, and an acquire with the same label does not resume an active lease.
• Lease numbers are coordination IDs, not authentication. The tunnel needs an API token. Anyone with daemon access can acquire or force leases.
• Lease IDs always carry their Mac. An iPhone on Mac studio gets studio:42, also on that Mac itself, so the ID works from every Mac of the account (see Several Macs). Only a signed-out Mac returns a plain 42. A plain 42 also works on its own Mac.
• Leases live in the lease table of state.db, so they survive a daemon restart, for example after a rebuild.

Coordinates: screen points

12ghosthand screenshot -o full.png --scale 1  # native pixels, 828x1792
ghosthand tap 414 896 --scale 1             # native pixel coordinates

Does Ghosthand need iPhone Mirroring?

Why this instead of Appium?

1. USB. Connect the iPhone, unlock it, and accept Trust This Computer.
2. Bluetooth. Pair the Mac as a mouse and keyboard from the iPhone's Settings.

No developer account or weekly signing renewal

Reliability

Optional: XCTest mode

12ghosthand control -l 42 xctest      # builds WebDriverAgent once (1-3 min), then starts it
ghosthand control -l 42 bluetooth   # back to the default

Alternatives

More iPhones

1234567891011ghosthand list
# mac studio:
#   7EF219AD-...  Morse's iPhone  iPhone XR (iPhone11,8) iOS 18.7  (bluetooth connected, absolute pointer)
#   42B874EC-...  Test iPhone     iPhone 6s (iPhone8,1) iOS 15.8.4  (bluetooth connected, relative pointer)

ghosthand acquire -d 7EF219AD-69B4-423E-B8C4-4F4A6140822F  # prints e.g. studio:42 (42 below)
ghosthand acquire -d "Test iPhone"                         # prints e.g. studio:43
ghosthand screenshot -l 42 -o a.png
ghosthand tap -l 43 206 332
ghosthand release -l 42
ghosthand release -l 43

capture id 7EF219AD-... ── saved once ──> UDID 00008020-... ──> lock state, BT address
   first match: the only new iPhone on USB, or else the only one with that name

1234ghosthand acquire -d 7EF219AD-69B4-423E-B8C4-4F4A6140822F --reason "rename"  # prints a lease id, e.g. 42
ghosthand rename -l 42 "iPhone XR"
ghosthand release -l 42
# renamed "iPhone" to "iPhone XR". The id stays 7EF219AD-69B4-423E-B8C4-4F4A6140822F

• list and acquire -d "iPhone XR" use the new name at once. The Bluetooth link stays connected (tested on iOS 18.7).
• macOS screen capture (QuickTime, daemon logs) keeps the old name until you unplug and plug in the cable.

How many iPhones per Mac

Several Macs, one list

1234567ghosthand list
# mac studio:
#   7EF219AD-...  Tommy's iPhone  iPhone XR (iPhone11,8) iOS 18.7  (bluetooth connected)
# mac laptop:
#   0A1B2C3D-...  Test iPhone  iPhone SE (iPhone12,8) iOS 17.6  (bluetooth connected)
ghosthand acquire -d 7EF219AD-... --reason "reply to Anna"   # studio:42
ghosthand tap -l studio:42 --text General --screenshot s.png

CLI (laptop) ──unix socket──> daemon (laptop) ──/macs/studio/v1/..., Mac session──> hub
hub (ghosthand.dev) ──WebSocket──> daemon (studio) ──USB + Bluetooth──> iPhone

• There is no token to copy. The local daemon calls the other Mac through ghosthand.dev with its own sign-in.
• acquire finds the Mac. If only another Mac has the -d ID or name, acquire takes the iPhone there. A Mac with no iPhone takes the account's only iPhone without -d.
• Every Mac needs this version. list skips a Mac with an older ghosthand and shows a warning. It also warns about a Mac that does not answer within 10 s.
• Screenshots and input go through ghosthand.dev, so each command is slower than on the Mac with the cable.

Several Macs: give each a unique name

123# System Settings > General > About > Name, or:
sudo scutil --set ComputerName "Lab Mac 1"
sudo scutil --set LocalHostName "lab-mac-1"   # optional: network name, no spaces

• The Bluetooth radio uses the computer name. There is one name per Mac, so ghosthand cannot show a name of its own next to it.
• The pairing steps print the name to tap: tap "Lab Mac 1", then Pair.
• An iPhone that paired before the rename can keep showing the old name. Forget the Mac on the iPhone (Settings > Bluetooth > (i) > Forget This Device) and pair again.
• Apple's guide explains how to change your Mac's name.

Devices in list

123mac studio:                                the Mac with the cable (`this Mac:` while signed out)
  7EF219AD-69B4-423E-B8C4-4F4A6140822F  Morse’s iPhone (2)  iPhone XR (iPhone11,8) iOS 18.7.10  (bluetooth connected, absolute pointer)
     USB capture ID                        name                model and iOS (USB)                   Bluetooth link, pointer mode

12error: No iPhone in macOS screen capture. USB check:
- iPhone di Carla (2): on USB but not trusted (AMDeviceStartSession 0xe8000025). Unlock it and tap Trust on the iPhone; ...

Unlock

enter ──> screen on ──1 s──> enter ──> passcode screen ──1 s──> type passcode once
                                                                  └──> check USB lock state

• The keyboard must be the first HID collection. iOS takes the device type from it. With the mouse first, SpringBoard tagged keys source:mouse and the lock screen dropped them. After a report map change, forget and pair the iPhone again, because iOS keeps the old map from pairing.
• The lock state comes over USB from lockdown (PasswordProtected) through the private MobileDevice.framework. list shows [locked], and unlock checks the state before and after.
• unlock never retries a wrong passcode. Failed tries count toward the iPhone lockout. Ghosthand does not store the passcode.
• Screenshots of a locked phone fail at once. USB capture returns no frames from a locked phone, so screenshot and screentext check the lock state first.
• OCR is for the agent loop after unlock, not for the locked-screen sequence. For example, screentext can find a button label and its pixel box, and the agent picks the next tap from that result.

HTTP API

1234567891011S=~/.ghosthand/daemon.sock
curl --unix-socket $S http://localhost/v1/devices
curl --unix-socket $S http://localhost/v1/phones   # setup step, lease, last command, waiting agents
curl --unix-socket $S -H 'content-type: application/json' \
  -d '{"device":"default","label":"curl","reason":"reply to Anna"}' http://localhost/v1/leases   # {"id":"42",…}
L=http://localhost/v1/leases/42
curl --unix-socket $S $L/screenshot -o s.png
curl --unix-socket $S -H 'content-type: application/json' \
  -d '{"x":207,"y":448}' $L/tap                    # screen points, like the screenshot
curl --unix-socket $S $L/assistive-touch            # {"enabled":true}
curl --unix-socket $S -X DELETE $L                  # release

• Screenshots and input are under /v1/leases/{lease}/ and run on the lease's iPhone. A lease that is unknown, expired, released or taken with force gets lease-invalid (403). An acquire while another lease holds the phone gets device-leased (409). Its detail shows the label, reason, acquire time and last call. To wait, retry.
• Input routes return JSON. Send Accept: text/event-stream to get progress as server-sent events.
• Do one call per step. Input routes take waitText, screenshot=png|jpeg and screentext=true. The daemon runs them after the input on the same lease. The result has screenshot (base64) and screen (OCR lines). tap also takes {"text":"General","index":1} instead of x and y.

  12curl --unix-socket $S -H 'content-type: application/json' \
    -d '{"text":"General"}' "$L/tap?waitText=About&screentext=true"

• Errors are application/problem+json with a stable code. In event-stream mode the status is 200 and the problem arrives as the final error event.

Scripts and other computers

123456# on the Mac with the iPhones
ghosthand tunnel url                            # https://ghosthand.dev/macs/studio
ghosthand tunnel url --new-token                # prints export GHOSTHAND_URL=... and export GHOSTHAND_TOKEN=...

# anywhere else: paste those two export lines, then
curl -H "Authorization: Bearer $GHOSTHAND_TOKEN" $GHOSTHAND_URL/v1/devices

script ──HTTPS /macs/studio/v1/..., Bearer API token──> ghosthand.dev hub ──WebSocket──> daemon

• One token works for the whole account. tunnel url --new-token creates it and shows it once. ghosthand tunnel revoke <id> ends it. Only the Mac can create or revoke tokens.
• Leases through the tunnel carry the Mac. POST /v1/leases returns studio:42. The plain 42 also works on its own Mac's URL.
• GET https://ghosthand.dev/macs lists your Macs and whether they are online. Request bodies through the tunnel are at most 20 MB.

MCP server

1claude mcp add --transport http ghosthand https://ghosthand.dev/mcp

1234// opencode.json
{ "mcp": { "ghosthand": { "type": "remote", "url": "https://ghosthand.dev/mcp" } } }
// .cursor/mcp.json
{ "mcpServers": { "ghosthand": { "url": "https://ghosthand.dev/mcp" } } }

MCP client ──POST /mcp, OAuth──> ghosthand.dev hub (one per account)
                                   ├─ lease "studio:42" ──> Mac studio
                                   ├─ mac: "laptop" ──────> Mac laptop
                                   └─ list ───────────────> every Mac, joined
                                          ▲  WebSocket the daemon opens (session token)
                                          └── daemon ──> tunnel.sock, /mcp and /v1 ──> Controller

• Leases name their Mac. acquire returns studio:42. Pass it as is, and it picks the Mac.
• With several Macs, list shows each iPhone under its Mac. Pass mac to acquire, wait, setup and setup status. With one Mac online, mac is optional.
• CLI flags are tool arguments, for example lease, text and index on tap, waitText, timeoutMs, screenshot: true and screentext: true. Screenshots come back as image content, not files.
• The server uses stateless HTTP. Modern clients (MCP 2026-07-28) and legacy clients (initialize) both work. When the client closes a request, the tool stops.
• A new account can connect before any Mac is set up. The hub then answers initialize and tools/list itself, and every tool call returns the next step for the user. That step is the download link, or which Mac is offline and since when. A Mac that never had an iPhone answers with the plug-in and Trust steps.
• There is no MCP library. A few JSON-RPC methods in the daemon's mcp.swift handle MCP. The hub in website/src/tunnel.ts only routes. While no Mac is online, it answers from the generated website/mcp-tools.json.
• ghosthand daemon status shows this Mac's name in the account.

Security

who can reach the phone              how
local user who owns ~/.ghosthand ──> unix socket, file mode 0600
MCP client of your account ────────> ghosthand.dev/mcp: OAuth, then the Mac's tunnel
script of your account ────────────> ghosthand.dev/macs/<mac>: API token, then the Mac's tunnel

1. USB cable plugged in, and the phone trusts this Mac (Trust prompt and passcode).
2. Bluetooth pairing accepted on the phone in Settings > Bluetooth.
3. The daemon running.

• Ghosthand never stores the passcode. unlock gets it for each call. Pipe it to unlock --stdin instead of passing it as an argument.

Limits

• AssistiveTouch must be on for tap and swipe. Ghosthand turns it on over USB, and you can hide its floating button (see AssistiveTouch).
• The keyboard without AssistiveTouch is not tested.
• cmd+h did nothing in tests. cmd+space opens Spotlight, so key cmd+space, type Safari and key enter open an app.
• There is no verified global Back command. key home goes Home, and key esc sends Escape. Neither one is sure to go back in Settings or Safari. Ghosthand does not expose AC Back (Consumer usage 0x0224), because its behavior on iOS is not verified. The descriptors do not change.
• A screen that is off gives no frames. unlock tries to wake a locked iPhone, but USB capture may still return no frame. An unlocked iPhone with the screen off needs a tap from you.
• Several new iPhones with one name, plugged in together, cannot be matched to their USB data. Plug them in one at a time, once (see More iPhones).
• Many fast daemon restarts can hide the iPhones for about a minute, because macOS throttles the screen capture helper (see The startup timer).
• About 7 iPhones per Mac is the Bluetooth limit, not a ghosthand limit. We tested 2. See How many iPhones per Mac.
• The cable must stay in. Screenshots come only over USB, and list does not show a phone on Bluetooth or Wi-Fi alone.
• Typing is US-ASCII only, so no accents or emoji. There is no length limit.

Something went wrong: read the daemon log

123tail -n 100 ~/.ghosthand/daemon.log                   # last lines
tail -f ~/.ghosthand/daemon.log                       # follow live while you run a command
grep -iE 'error|fail' ~/.ghosthand/daemon.log | tail  # recent errors only

122026-10-08T13:02:40Z error no-device: No iPhone found on USB. A charge-only cable does not work.
2026-10-08T16:43:31Z no iPhone visible yet, restarting the screen capture helper

• ghosthand daemon status prints the log path, the pid, and the Bluetooth state.
• The daemon never rotates the log. Delete or truncate it when it gets large: : > ~/.ghosthand/daemon.log. Never delete state.db.
• The live progress of one command (pairing, retries) also goes to that command's stderr, so you see it without the log.

Pairing no longer works: forget on both sides

1. On the iPhone, open Settings > Bluetooth > the Mac > (i) and tap Forget This Device.
2. On the Mac, run ghosthand forget with a lease on that iPhone. It removes the Mac's bond, like System Settings > Bluetooth > the iPhone > (i) > Forget This Device.
3. Run any input command, for example ghosthand key home. It makes the Mac discoverable and waits.
4. On the iPhone, open Settings > Bluetooth, tap the Mac under Other Devices, then tap Pair.

USB screen capture stops (iOS bug): restart the daemon, then the iPhone

1error: No frame from Morse’s iPhone (2) after 8s (session running, 0 frames since open). ...

• First try ghosthand daemon restart. It fixed a stuck capture that failed with Recording Stopped (AVFoundationErrorDomain -11808) (iOS 18.7).
• If frames still do not arrive, restart the iPhone. Replugging the cable does not help, and in some cases the daemon restart did not help either (tested on iOS 18.7).
• This is an iOS bug, not a ghosthand bug. A plain AVFoundation program gets no frames either, and QuickTime Player crashed when it opened the same iPhone screen.
• Two Apple reports have the same symptom: FB21608780 (after "Stop Mirroring" in Control Center, fixed only by a reboot) and FB22281473 (screen capture trust revoked).
• After the restart, unlock the iPhone by hand once. Before the first unlock, iOS ignores the Bluetooth keyboard, so ghosthand unlock fails.

1. ghosthand daemon stop
2. QuickTime Player > File > New Movie Recording
3. Click the ⌄ arrow next to the record button. Under Camera:

• If the screen entry shows the live screen, capture works, and the problem is on the Mac side.
• If the screen entry is black or missing, or QuickTime crashes, iOS stopped serving the screen. Restart the iPhone.
• Two iPhones with the same name both show up as ... Camera. Only the one on USB has a screen entry.

How the Mac becomes a mouse

Mac (daemon)                                          iPhone
┌──────────────────────────────────┐                 ┌─────────────────────────────┐
│ SDP record: HID, report map      │ <──SDP query─── │ Settings > Bluetooth > Pair │
│ L2CAP 0x11 control  (handshakes) │ <──opens─────── │                             │
│ L2CAP 0x13 interrupt             │ ──reports─────> │ AssistiveTouch              │
│  report 1: mouse (absolute)      │                 │  pointer + clicks = touches │
│  report 7: mouse (rel.), wheels  │                 │                             │
│  report 2: keyboard, 6: media    │                 │                             │
└──────────────────────────────────┘                 └─────────────────────────────┘

check:  corner ──> move (80,120) pt ──> screenshot P1
        corner ──> move (160,120) pt ──> screenshot P2
        pointer found in P1 and P2 ──> gain = measured / computed (saved in state.db)
tap:    computed reports from the last known spot ──> 100 ms pause ──> click

How the USB screen is captured

daemon ──sets CMIO flag──> launchd ──starts──> iOSScreenCaptureAssistant
                                                 │ sets up each USB iPhone once, on attach
                                                 v
daemon <──────────── one AVCaptureDevice per iPhone (frames) ──── AVFoundation

The startup timer

• No iPhone is listed. The new helper started while launchd still throttled the old one.
• Only some iPhones are listed. The old helper's USB teardown disconnected some iPhones while the new helper set them up (kAMDDeviceDisconnectedError in its log). We saw this with 4 iPhones. The control plane then says "Screen capture does not show this iPhone yet".

tick ──> every trusted, unlocked USB iPhone listed? ──yes──> stop the timer (usually 1st tick)
          │
          ├── none listed ──────────────> flag off (helper exits), next tick on (fresh helper)
          │
          └── some missing on 2 ticks ──> same restart, at most 3 times,
              in a row (11 s)              then log which iPhones are still missing

• Nothing waits for the timer. The API answers at once, and listed iPhones work at once.
• A restart costs about 11 s and stops capture of every iPhone during that time. That is why a partial list gets only 3 restarts.
• After the timer stops, nothing restarts the helper. An iPhone that you plug in locked or untrusted stays missing. list warns, the log says screen capture does not list <name>, and a replug fixes it.

123screen capture does not list iPhone, restarting the screen capture helper
screen capture devices on again
screen capture lists iPhone, Naomi’s iPhone, Morse’s iPhone, iPhone XR Tommy

How a phone is found

USB lockdown  BluetoothAddress ──── EC:CE:D7:A4:9B:C1 ────> Bluetooth HID link

State database

phone (USB)                                  bluetooth_pointer (Bluetooth)
┌───────────────────────────────────┐        ┌──────────────────────────────────────┐
│ usb_udid           PK             │<───────│ usb_udid   PK, FK, ON DELETE CASCADE │
│ screen_capture_id  UNIQUE         │        │ mode       absolute | relative       │
│ capture_matched_by / _at          │        │ gain, screen_width_pt, _height_pt    │
└───────────────────────────────────┘        │ report_map, found_at                 │
                                             └──────────────────────────────────────┘

12345sqlite3 -header -column ~/.ghosthand/state.db 'SELECT * FROM phone; SELECT * FROM bluetooth_pointer'
# usb_udid                   screen_capture_id                     capture_matched_by  capture_matched_at
# 00008020-000405C214BB002E  7EF219AD-69B4-423E-B8C4-4F4A6140822F  only-new-phone      1790773434
# usb_udid                   mode      gain  screen_width_pt  screen_height_pt  report_map                found_at
# 00008020-000405C214BB002E  absolute                                           classic-a85efc997e025d8f  1790773436

• Some fields are not stored, because USB lockdown gives them on every command. These are the name, the Bluetooth address, the lock state and the AssistiveTouch setting.
• Bluetooth pairing is not in state.db. macOS keeps the bond. A phone paired while the daemon was stopped, or with no state.db, reconnects on the next command (tested).
• The daemon never changes a saved match by itself, even while its phone is missing from one list during a plug-in. If a match is wrong, delete its phone row (below).
• Schema migrations keep the state. The migration from version 5 to 6 adds AssistiveTouch ownership without deleting matches, pointer settings, leases or the lease counter. The daemon refuses newer or unsupported schemas without deleting data. Never delete state.db to get past that error.

123DB=~/.ghosthand/state.db
sqlite3 $DB "PRAGMA foreign_keys = ON; DELETE FROM phone WHERE usb_udid = '00008020-000405C214BB002E'"   # one iPhone
sqlite3 $DB 'DELETE FROM bluetooth_pointer'                                                             # all pointer modes

• Screen from one phone, input to another? Delete its phone row. Plug in new phones with the same name one at a time.
• Pointer lands in the wrong spot? Delete its bluetooth_pointer row. ghosthand forget also deletes it, but it removes the Bluetooth bond too, so you must pair again.
• Never delete or restore state.db. Resetting its counter can give a new lease an old ID.

iOS versions

• The daemon picks the mode by itself on the first tap. iOS 17 and later use absolute. On older versions, the daemon moves the pointer to check. list shows the mode, for example (bluetooth connected, relative pointer).
• Apple does not document absolute pointers for AssistiveTouch, and no changelog mentions them. We found the iOS 15 behavior by testing over BLE. The phone read report 1, but the pointer stayed still.
• In relative mode, the daemon knows exactly how far each report moves the pointer, from Apple's own acceleration table. See Relative pointer.

Which iPhones get which pointer

• The "absolute from iOS 17" claim comes from a vendor of ESP32 BLE HID firmware, not from Apple. Bluetooth Classic HID may behave differently, so the daemon checks older iOS versions instead of assuming.

Relative pointer (iOS 15)

tap 300,500 with the pointer at 40,120 (move +260,+380 pt) on a 6s:
  (+169,+248) (+170,+247) ── speed 300, past the flat end of the table ──> 214.9 pt each
  (+16,+24)               ── speed 28 ─────────────────────────────────> 30.0 pt
  (0,-1)                  ── speed 1: 0.2 pt, lands within 0.2 pt
  100 ms pause, then click                                   ~0.45 s in total

• We measured this on an iPhone 6s (iOS 15.8.4). 15 report sizes, diagonals, and bursts compared with 100 ms gaps all matched the table to 0.5 pt. 40 taps in a row landed 0.2 pt from the target on average, with no drift. See docs/relative-pointer.md for details.
• The daemon remembers where the pointer is. Only when it does not know (daemon start, Bluetooth reconnect, rotation, 60 s idle) does it first push the pointer into the top-left corner, which adds 0.3 s.
• Tracking Speed changes the distances. The pointer check measures a gain (1.000 at the default speed) and scales the table by it. Other speeds are not tested.
• Swipes follow --duration. The daemon spreads the reports over the duration and sizes them so the drag ends exactly at the target.

CLI / curl / any language
   │ HTTP over ~/.ghosthand/daemon.sock (mode 0600, no token)
   v
Hummingbird ──> generated API (openapi.yaml) ──> own thread ──> Controller
     ^
     │ /mcp and /v1, via tunnel.sock
MCP clients ──> ghosthand.dev/mcp (OAuth) ────┐
scripts ──> ghosthand.dev/macs/<mac>/v1 (API token) ──> WebSocket the daemon opens while signed in
                                                                  │
      ┌───────────────────────────────────────┬───────────────────┤
      v                                       v                   v
  FrameStream per iPhone (AVFoundation)   Lockdown lock state     HIDDevice: BT Classic
  OCR (Vision)                            (MobileDevice)          HID, CBClassicManager

Daemon source code