screentext

screentextTOOL

Read the text on the iPhone screen with on-device OCR. One line per text: x,y wxh text in screenshot coordinates (same space as tap)

Output: one line per text, x,y wxh text, top-left corner and size in screen points. The center of a line is x+w/2, y+h/2. To tap text, use ghosthand tap --text "...", which does this for you. Add --language for non-English UIs (it-IT, de-DE, ...) or accents are misread. Body text and labels read at about 99% of words. Not read: red badge counts (Mail "6"), icons (read as stray letters like "Q", "A", "4"), text cut by an edge. Text inside photos comes out garbled. Use a screenshot for these.

CLI flags in this text are arguments: --language = language, --settle = settle, --scale = scale

Examples (arguments): {"lease":"studio:42"} {"language":"it-IT","lease":"studio:42"}

Parameters

language?string

OCR language, e.g. it-IT; en-US is always included

lease *string

Lease id exactly as acquire printed it, e.g. "studio:42"

mac?string

Mac slug from list, e.g. "studio". Needed when several Macs are online

scale?number

Image and coordinate size as a fraction of native pixels (default: screen points; 1 = native)

settle?boolean

Wait (up to 2s) until the screen stops changing

Request example

{
  "method": "tools/call",
  "params": {
    "name": "screentext",
    "arguments": {
      "language": "string",
      "lease": "string",
      "mac": "string",
      "scale": 0,
      "settle": true
    }
  }
}