feat: Phase 3 recipes file, spotify_play, and locked-screen refusal

Phase 3 of AGENTS.md section 11.

Recipes file (FR-5)
- recipes/ holds one hand-editable file per task; recipes/spotify_play.recipe
  is the first one.
- internal/recipe parses directives, guards, parameters and steps, and runs
  them with a per-step and a whole-run timeout (FR-11).
- Steps name text ("tap the element whose text is Search"), never coordinates.
- guards name screens Artemis must never tap on, so a signed-out or
  first-run Spotify fails with its own reason (FR-8).
- Every run appends task, params, steps, result and duration to
  logs/runs.jsonl (FR-10).

spotify_play
- launch, search, play, then wait for the transport button to say "Pause".
- internal/ui/rows.go adds tap_row: a result row is a clickable container with
  a title and a subtitle, so a one-line search suggestion is not a result, and
  the tap lands on the row's centre rather than on its "More options" button.
  This fixes a real wrong tap: a plain text tap hit a 126x12 pixel
  "More options for song Honky Tonk Women - Live" button.

Locked screen (AGENTS.md section 13 Q2, decided)
- Artemis fails loudly on a secure keyguard and never types a PIN. No secret is
  stored. A non-secure keyguard is still dismissed.
- internal/ui/keyguard.go reads the KeyguardServiceDelegate block plus
  cmd lock_settings is-user-secure.
- Every acting command wakes the screen before it checks the keyguard: a dozing
  phone reports showing=false, because the lock screen is not drawn yet.
- AGENTS.md records the decision and that storing a PIN for auto-unlock is
  rejected for now.

Verified live: the phone locked itself mid-session, and keyguard, tap and
run spotify_play all refused with "phone is locked; unlock it and retry".
The live end-to-end playback run still needs an unlocked phone.

.gitignore

@@ -34,3 +34,6 @@ local.properties
 
 # Pi agent generated memory (local state, not source)
 .pi/memory-vault/
+
+# Artemis run log (local state, not source)
+logs/

AGENTS.md

@@ -176,12 +176,23 @@ Resolved:
    was USB-only and is removed in this NixOS release anyway. Go is likewise run
    from a one-off `nix shell nixpkgs#go`, not installed system-wide. Never run
    `nixos-rebuild` on `.13`: it runs the Hub, MQTT and every service.
+6. **Locked screen — Artemis fails loudly. It never unlocks the phone.** It
+   detects a secure keyguard (PIN, pattern or password) before acting and
+   refuses with `phone is locked; unlock it and retry`. It does not dismiss a
+   secure keyguard and it does not type anything into one. A *non-secure*
+   keyguard (the phone has no PIN at all) is still dismissed, because that
+   screen guards nothing. **No PIN is stored, read or transmitted**, and there
+   is no secret for one. **Storing a PIN for auto-unlock is rejected for now** —
+   the phone is a pocket device, and Artemis is a convenience, not a reason to
+   keep Sam's unlock credential in a service on `.13`. This is revisit-able if
+   the daily friction turns out to be worse than the risk.
 
 Remaining:
 1. Which tasks after Spotify and maps? (WhatsApp message? Other music apps?)
-2. Locked screen: must Artemis unlock it, or does Sam leave it awake?
+2. Action API shape: one generic `/act`, or one endpoint per task?
-3. Action API shape: one generic `/act`, or one endpoint per task?
+3. **Service packaging** for Artemis on `.13` — container or systemd — is still
-4. **Service packaging** for Artemis on `.13` — container or systemd — is still
    open. ADB can be awkward in a container, so this is decided in Phase 6. The
    tooling-placement half of the old Q4 is resolved above.
-5. Speed: what is an acceptable time for a 6-step recipe?
+4. Speed: what is an acceptable time for a 6-step recipe?
+5. The recipes file has grown `guard`, `tap_if` and `wait_gone` verbs. If it
+   keeps growing, decide whether the format stays bespoke or moves to YAML.

README.md

@@ -12,7 +12,7 @@ order.
 |---|---|
 | 1 — Prove the ADB link | ✅ proven — see [`docs/phase-1-adb-link.md`](docs/phase-1-adb-link.md) |
 | 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch |
-| 3 — Spotify recipe | not started |
+| 3 — Spotify recipe | 🟡 built — recipes file, guards, `spotify_play`; the live end-to-end run is still to do, see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) |
 | 4 — Song radio | not started |
 | 5 — Maps recipe | not started |
 | 6 — Action API | not started |
@@ -25,6 +25,9 @@ order.
 cmd/artemis/        command line entry point
 internal/adb/       timeout-bounded wrapper around the adb binary
 internal/ui/        read the screen (uiautomator dump) and act on it (input)
+internal/recipe/    the recipes file format and its runner
+recipes/            one hand-editable file per task
+logs/               run log (JSON lines, not committed)
 docs/               phase evidence and notes
 ```
 
@@ -42,13 +45,15 @@ nix shell nixpkgs#go -c go test ./...
 ## Use
 
 The phone address defaults to `100.101.49.17`. Override with `ARTEMIS_PHONE`,
-pin a serial with `ARTEMIS_ADB_SERIAL`, or point at a different adb with
+pin a serial with `ARTEMIS_ADB_SERIAL`, point at a different adb with
-`ARTEMIS_ADB`.
+`ARTEMIS_ADB`, point at another recipes directory with `ARTEMIS_RECIPES`, and
+move the run log with `ARTEMIS_LOG`.
 
 ```sh
 ./bin/artemis connect          # scan for the wireless-debugging port, then connect
 ./bin/artemis devices
 ./bin/artemis dump             # text nodes and bounds
+./bin/artemis keyguard         # lock-screen state; exits 1 when locked
 ./bin/artemis launch com.spotify.music
 ./bin/artemis tap "Search"     # find by text, tap its centre (FR-4)
 ./bin/artemis type "Rolling Stones Honky Tonk"
@@ -58,11 +63,31 @@ pin a serial with `ARTEMIS_ADB_SERIAL`, or point at a different adb with
 ./bin/artemis demo             # Phase 2 DoD: launch an app, tap a named element
 ```
 
+## Recipes
+
+A recipe is a plain-text file in `recipes/` that Sam edits by hand. Steps name
+what to find on screen; there are no coordinates anywhere.
+
+```sh
+./bin/artemis recipes                                     # list them
+./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
+```
+
+Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the
+file format, the verbs, and why list rows need their own step.
+
 ## Rules that the code enforces
 
 - **Find by text, never fixed coordinates.** `tap` resolves a text match to
-  bounds and taps the centre.
+  bounds and taps the centre; `tap_row` scores result rows against the query
+  and taps the winning row's centre.
 - **Never tap blindly.** A missing or disabled element is a loud failure, not a
-  guessed coordinate.
+  guessed coordinate. A named bad screen (`guard`) fails with its own reason.
-- **Wake first.** Every acting command wakes the screen (FR-7).
+- **Never touch a locked phone.** A secure keyguard stops Artemis with
-- **Bound every command.** No adb call runs without a timeout (FR-11).
+  `phone is locked; unlock it and retry`. No PIN is stored, read or typed
+  (AGENTS.md section 13 Q2).
+- **Wake first.** Every acting command wakes the screen before it acts (FR-7),
+  and it checks the keyguard only after the screen is on.
+- **Bound every command.** No adb call runs without a timeout, and a recipe has
+  a per-step and a whole-run timeout (FR-11).
+- **Log every run.** Each run appends a JSON line to `logs/runs.jsonl` (FR-10).

cmd/artemis/main.go

@@ -1,7 +1,8 @@
 // Command artemis drives an Android phone over ADB wireless.
 //
-// This is the Phase 2 surface: act primitives on the command line. The Hub's
+// This is the command line surface: connection, act primitives, and the named
-// HTTP action API arrives in Phase 6 and reuses these packages.
+// recipes in recipes/. The Hub's HTTP action API arrives in Phase 6 and reuses
+// these packages.
 package main
 
 import (
@@ -22,6 +23,7 @@ connection
   connect [host]        connect to the phone (scans ports if host given)
   devices               list adb devices
   wake                  wake the screen
+  keyguard              report the lock-screen state
 
 reading
   dump                  print visible text nodes with bounds
@@ -37,6 +39,10 @@ acting
   back                  press back
   home                  press home
 
+recipes
+  recipes               list the recipes in the recipes directory
+  run <task> [k=v ...]  run a named recipe, e.g. run spotify_play query="..."
+
 proof
   demo                  Phase 2 DoD: launch an app, tap a named element
 
@@ -44,6 +50,8 @@ environment
   ARTEMIS_PHONE         phone tailscale IP (default 100.101.49.17)
   ARTEMIS_ADB_SERIAL    pin a specific adb serial
   ARTEMIS_ADB           path to the adb binary (default: PATH)
+  ARTEMIS_RECIPES       recipes directory (default: recipes)
+  ARTEMIS_LOG           run log file (default: logs/runs.jsonl)
 `
 
 func main() {
@@ -64,6 +72,8 @@ func main() {
 		err = cmdDevices(r)
 	case "wake":
 		err = cmdWake(r)
+	case "keyguard":
+		err = cmdKeyguard(r)
 	case "dump":
 		err = cmdDump(r)
 	case "find":
@@ -81,9 +91,23 @@ func main() {
 	case "scroll":
 		err = cmdScroll(r, args)
 	case "back":
-		err = ensurePhone(r, func() error { return ui.Back(r) })
+		err = ensurePhone(r, func() error {
+			if err := wakeAndGuard(r); err != nil {
+				return err
+			}
+			return ui.Back(r)
+		})
 	case "home":
-		err = ensurePhone(r, func() error { return ui.Home(r) })
+		err = ensurePhone(r, func() error {
+			if err := wakeAndGuard(r); err != nil {
+				return err
+			}
+			return ui.Home(r)
+		})
+	case "recipes":
+		err = cmdRecipes(args)
+	case "run":
+		err = cmdRun(r, args)
 	case "demo":
 		err = cmdDemo(r)
 	case "-h", "--help", "help":
@@ -119,6 +143,19 @@ func ensurePhone(r *adb.Runner, fn func() error) error {
 	return fn()
 }
 
+// wakeAndGuard wakes the screen and refuses to act while a secure keyguard is
+// showing. Artemis has no PIN and never types one (AGENTS.md section 13 Q2).
+//
+// Waking first matters. A dozing phone reports `showing=false` because the lock
+// screen is not drawn yet, so the keyguard state is only trustworthy once the
+// screen is on.
+func wakeAndGuard(r *adb.Runner) error {
+	if err := ui.Wake(r); err != nil {
+		return err
+	}
+	return ui.RequireUnlocked(r)
+}
+
 func cmdConnect(r *adb.Runner, args []string) error {
 	if len(args) > 0 {
 		msg, err := r.Connect(args[0])
@@ -156,6 +193,30 @@ func cmdWake(r *adb.Runner) error {
 			return err
 		}
 		fmt.Println("screen awake")
+		showing, secure, err := ui.KeyguardState(r)
+		if err != nil {
+			return err
+		}
+		if showing && secure {
+			fmt.Println("phone is locked (secure keyguard); Artemis will not act until it is unlocked")
+			return nil
+		}
+		fmt.Println("phone is unlocked")
+		return nil
+	})
+}
+
+// cmdKeyguard reports the lock-screen state without touching it.
+func cmdKeyguard(r *adb.Runner) error {
+	return ensurePhone(r, func() error {
+		showing, secure, err := ui.KeyguardState(r)
+		if err != nil {
+			return err
+		}
+		fmt.Printf("keyguard showing=%t secure=%t\n", showing, secure)
+		if showing && secure {
+			return ui.ErrLocked
+		}
 		return nil
 	})
 }
@@ -219,7 +280,7 @@ func cmdLaunch(r *adb.Runner, args []string) error {
 		return fmt.Errorf("usage: artemis launch <package>")
 	}
 	return ensurePhone(r, func() error {
-		if err := ui.Wake(r); err != nil {
+		if err := wakeAndGuard(r); err != nil {
 			return err
 		}
 		start := time.Now()
@@ -236,7 +297,7 @@ func cmdTap(r *adb.Runner, args []string) error {
 		return fmt.Errorf("usage: artemis tap <text>")
 	}
 	return ensurePhone(r, func() error {
-		if err := ui.Wake(r); err != nil {
+		if err := wakeAndGuard(r); err != nil {
 			return err
 		}
 		node, err := ui.TapText(r, args[0], false)
@@ -253,7 +314,12 @@ func cmdType(r *adb.Runner, args []string) error {
 	if len(args) != 1 {
 		return fmt.Errorf("usage: artemis type <text>")
 	}
-	return ensurePhone(r, func() error { return ui.Type(r, args[0]) })
+	return ensurePhone(r, func() error {
+		if err := wakeAndGuard(r); err != nil {
+			return err
+		}
+		return ui.Type(r, args[0])
+	})
 }
 
 func cmdSwipe(r *adb.Runner, args []string) error {
@@ -276,21 +342,31 @@ func cmdSwipe(r *adb.Runner, args []string) error {
 		}
 		ms = v
 	}
-	return ensurePhone(r, func() error { return ui.Swipe(r, nums[0], nums[1], nums[2], nums[3], ms) })
+	return ensurePhone(r, func() error {
+		if err := wakeAndGuard(r); err != nil {
+			return err
+		}
+		return ui.Swipe(r, nums[0], nums[1], nums[2], nums[3], ms)
+	})
 }
 
 func cmdScroll(r *adb.Runner, args []string) error {
 	if len(args) != 1 {
 		return fmt.Errorf("usage: artemis scroll <up|down>")
 	}
-	return ensurePhone(r, func() error { return ui.Scroll(r, args[0]) })
+	return ensurePhone(r, func() error {
+		if err := wakeAndGuard(r); err != nil {
+			return err
+		}
+		return ui.Scroll(r, args[0])
+	})
 }
 
 // cmdDemo is the Phase 2 Definition of Done, runnable on demand:
 // Artemis opens a named app and taps a named element on it.
 func cmdDemo(r *adb.Runner) error {
 	return ensurePhone(r, func() error {
-		if err := ui.Wake(r); err != nil {
+		if err := wakeAndGuard(r); err != nil {
 			return err
 		}
 

cmd/artemis/run.go

@@ -0,0 +1,122 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"gitea.lab.audasmedia.com.au/sam/artemis_control/internal/adb"
+	"gitea.lab.audasmedia.com.au/sam/artemis_control/internal/recipe"
+)
+
+// recipesDir is where hand-editable recipes live.
+func recipesDir() string {
+	if d := os.Getenv("ARTEMIS_RECIPES"); d != "" {
+		return d
+	}
+	return "recipes"
+}
+
+// runLogPath is the JSON-lines action log (FR-10).
+func runLogPath() string {
+	if p := os.Getenv("ARTEMIS_LOG"); p != "" {
+		return p
+	}
+	return filepath.Join("logs", "runs.jsonl")
+}
+
+func cmdRecipes(args []string) error {
+	recipes, err := recipe.List(recipesDir())
+	if err != nil {
+		return err
+	}
+	for _, r := range recipes {
+		fmt.Printf("%-16s %s\n", r.Task, r.Description)
+	}
+	return nil
+}
+
+// cmdRun runs a named recipe: artemis run spotify_play query="Rolling Stones".
+func cmdRun(r *adb.Runner, args []string) error {
+	if len(args) < 1 {
+		return fmt.Errorf("usage: artemis run <task> [name=value ...]")
+	}
+	task := args[0]
+
+	params := map[string]string{}
+	for _, kv := range args[1:] {
+		name, value, ok := strings.Cut(kv, "=")
+		if !ok {
+			return fmt.Errorf("parameter %q must be name=value", kv)
+		}
+		params[strings.TrimSpace(name)] = value
+	}
+
+	rec, err := recipe.Find(recipesDir(), task)
+	if err != nil {
+		return err
+	}
+	if missing := rec.MissingParams(params); len(missing) > 0 {
+		return fmt.Errorf("recipe %s needs %s", task, strings.Join(missing, ", "))
+	}
+
+	var res *recipe.RunResult
+	err = ensurePhone(r, func() error {
+		res = recipe.Run(r, rec, recipe.Options{Params: params, Log: os.Stdout})
+		if logErr := appendRunLog(runLogPath(), res); logErr != nil {
+			fmt.Fprintf(os.Stderr, "artemis: could not write the run log: %v\n", logErr)
+		}
+		if !res.OK {
+			return fmt.Errorf("%s", res.Error)
+		}
+		return nil
+	})
+
+	fmt.Printf("\n%s\n", summaryLine(task, res))
+	return err
+}
+
+func summaryLine(task string, res *recipe.RunResult) string {
+	if res == nil {
+		return "task " + task + " did not run"
+	}
+	steps := len(res.Steps)
+	if res.OK {
+		return fmt.Sprintf("ok: %s ran %d steps in %s", task, steps, res.Duration.Round(time.Millisecond))
+	}
+	return fmt.Sprintf("failed: %s stopped at step %d in %s: %s",
+		task, steps, res.Duration.Round(time.Millisecond), res.Error)
+}
+
+// appendRunLog adds one JSON line per run: task, params, steps, result and
+// duration (FR-10).
+func appendRunLog(path string, res *recipe.RunResult) error {
+	if res == nil {
+		return nil
+	}
+	if dir := filepath.Dir(path); dir != "" && dir != "." {
+		if err := os.MkdirAll(dir, 0o755); err != nil {
+			return err
+		}
+	}
+	f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
+	if err != nil {
+		return err
+	}
+	defer f.Close()
+
+	entry := struct {
+		Time string `json:"time"`
+		*recipe.RunResult
+	}{Time: time.Now().Format(time.RFC3339), RunResult: res}
+
+	raw, err := json.Marshal(entry)
+	if err != nil {
+		return err
+	}
+	_, err = f.Write(append(raw, '\n'))
+	return err
+}

docs/phase-3-spotify-recipe.md

@@ -0,0 +1,205 @@
+# Phase 3 — the recipes file and `spotify_play`
+
+Phase 3 of `AGENTS.md` section 11: the first named recipe, plus the resolved
+answer to section 13 Q2 (the locked screen).
+
+Date: 2026-10-16. Phone: `Pixel_8a (akita)` over Tailscale at
+`100.101.49.17:46689` (the port moves on every reboot; nothing hardcodes it).
+
+## What was built
+
+| File | What it holds |
+|---|---|
+| `recipes/spotify_play.recipe` | The recipe, written to be edited by hand. |
+| `internal/recipe/recipe.go` | The file format: directives, guards, steps, parameters. |
+| `internal/recipe/engine.go` | Runs a recipe step by step with timeouts and a run log. |
+| `internal/ui/rows.go` | Picks a list row by text and taps its centre. |
+| `internal/ui/keyguard.go` | Reads the lock state and refuses to act when locked. |
+| `cmd/artemis/run.go` | `artemis recipes`, `artemis run <task> k=v`, JSON run log. |
+
+## The recipes file
+
+One file per task in `recipes/`, plain text, no build step. It names the app,
+the parameters the caller must supply, the screens Artemis must never tap on,
+and the ordered steps.
+
+```
+task: spotify_play
+app: com.spotify.music
+param: query
+row_id: com.spotify.music:id/row_root
+
+guard "Log in" => Spotify is signed out; sign in on the phone and retry
+
+steps:
+  - launch
+  - wait "Search" 30s
+  - tap "Search"
+  - tap_if "Clear search query"
+  - wait "What do you want to listen to?" 30s
+  - tap "What do you want to listen to?"
+  - type "{query}"
+  - wait "Clear search query" 30s
+  - wait_gone "Recents" 30s
+  - tap_row "{query}" 40s
+  - wait exact "Pause" 40s
+```
+
+Steps express intent. There is no coordinate anywhere in the file (FR-4).
+
+Run it with:
+
+```sh
+./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
+```
+
+The verbs are `launch`, `wait`, `wait_gone`, `tap`, `tap_if`, `tap_row`, `type`,
+`key`, `back`, `home` and `sleep`. `tap exact "Pause"` matches the whole text,
+not a substring. `tap_if` taps only when the text is already on screen, which
+lets one recipe cope with an app that remembers its last screen.
+
+## Why `tap_row` exists
+
+A plain text tap picked the wrong element on the real Spotify screen. On the
+results list, `artemis tap "Honky Tonk Women"` tapped
+
+```
+ImageButton[context_menu_button] text="More options for song Honky Tonk Women - Live"
+  bounds=[828,2388][954,2400]
+```
+
+That is the smallest clickable match, and it sits at the bottom edge of the
+screen. It was the wrong target: a "More options" button, not the song.
+
+`tap_row` scores every result row against the query and taps the row itself:
+
+* A candidate row is a clickable container that carries a title **and** a
+  subtitle. A one-line search suggestion (Spotify's "rolling stones honky tonk
+  women single") is therefore not a result row.
+* Text inside a clickable child is skipped, because that is an action button
+  ("More options", "Add to library", "Add suggestion"), not the row's own text.
+* A row clipped by the bottom of the list is dropped. It is much shorter than a
+  row that is fully drawn, and its centre sits under the navigation area.
+* Rows are ranked by F1 score over the query words, so an exact title beats a
+  noisy deluxe-edition title. Equal scores go to the highest row, because search
+  results are ranked by relevance.
+
+On the real screen for `Rolling Stones Honky Tonk`, the top hit wins:
+
+```
+Honky Tonk Women / Song • The Rolling Stones   bounds=[0,794][1080,962]  F1=0.727
+```
+
+and the tap lands at the row's centre, `(540,878)` — on the title, clear of the
+action buttons. `internal/ui/rows_test.go` runs this against a trimmed copy of
+the real dump.
+
+## Failing cleanly (FR-8)
+
+Every failure names its cause, and they are distinguishable:
+
+| Situation | Message |
+|---|---|
+| App missing | `app is not installed: com.spotify.music` |
+| Signed out | `Spotify is signed out; sign in on the phone and retry (phone is showing "Log in")` |
+| Update nag | `Spotify wants an update; update it on the phone and retry` |
+| Element missing | `text "Search" did not appear within 30s; foreground=com.spotify.music; screen shows: "Log in", "Sign up free"` |
+| Phone locked | `phone is locked; unlock it and retry` |
+
+A guard is checked when a step cannot find its target, so a bad screen produces
+its own reason instead of a generic "element not found".
+
+## Timeouts (FR-11)
+
+`step_timeout: 20s` is the default per step, `total_timeout: 150s` bounds the
+whole run. A step may name its own timeout (`wait "Search" 30s`). The engine
+checks the remaining budget before each step and stops when it runs out. Every
+`adb` call is already bounded by `adb.DefaultTimeout`.
+
+## Logging (FR-10)
+
+Every run appends one JSON line to `logs/runs.jsonl` (override with
+`ARTEMIS_LOG`): task, params, each step with its duration, the result and the
+total duration. Two real entries:
+
+```json
+{"time":"2026-10-16T13:23:31+11:00","task":"spotify_play","params":{"query":"Rolling Stones Honky Tonk"},"ok":false,"error":"phone is locked; unlock it and retry","steps":null,"duration_ms":832}
+```
+
+The same steps print to the terminal as they run.
+
+## The locked screen (section 13 Q2)
+
+**Decision: Artemis fails loudly. No PIN is stored and none is ever typed.**
+
+`ui.KeyguardState` reads two things: whether the lock screen is showing
+(`dumpsys window`, the `KeyguardServiceDelegate` block) and whether the device
+has a secure lock at all (`cmd lock_settings is-user-secure`). Acting commands
+wake the screen first and then refuse if the keyguard is showing and secure. A
+non-secure keyguard is still dismissed, as before. An unreadable lock state is a
+failure, not a guess.
+
+### Evidence
+
+The prescribed test (put the phone to sleep, wake it, Artemis must refuse):
+
+```sh
+adb -s 100.101.49.17:46689 shell input keyevent 26    # POWER = sleep
+adb -s 100.101.49.17:46689 shell input keyevent 224   # WAKEUP
+```
+
+```
+$ adb shell dumpsys window policy | grep -E 'showing=|screenState'
+      showing=true
+      screenState=SCREEN_STATE_ON
+$ adb shell cmd lock_settings is-user-secure
+true
+
+$ ./bin/artemis keyguard
+keyguard showing=true secure=true
+artemis: phone is locked; unlock it and retry
+exit=1
+
+$ ./bin/artemis tap "Search"
+artemis: phone is locked; unlock it and retry
+exit=1
+
+$ ./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
+task spotify_play (Search Spotify for a song and play it.)
+FAILED spotify_play after 833ms: phone is locked; unlock it and retry
+exit=1
+```
+
+No PIN was entered, nothing was read from the phone, and the lock screen was
+still on screen afterwards — no blind tap happened. The screen dump after the
+refusals still showed the lock screen clock and date.
+
+A dozing phone reports `showing=false`, because the lock screen is not drawn
+yet. That is why every acting command wakes the screen **before** it checks the
+keyguard; reading the state first would say "unlocked" about a locked phone.
+
+## Verification status
+
+| Item | State |
+|---|---|
+| Recipes file format, guards, parameters, timeouts | unit tested (`internal/recipe`) |
+| Row selection against a real Spotify dump | unit tested (`internal/ui`) |
+| Keyguard parsing and secure-lock detection | unit tested (`internal/ui`) |
+| All acting commands refuse on a locked phone | **live, verified** (above) |
+| `artemis run spotify_play` refuses on a locked phone | **live, verified** (above) |
+| `spotify_play` plays the right song | **not yet run end to end** — see below |
+| Spotify search reaches the result list | live (Home → Search → type → results) |
+
+The live end-to-end run of `spotify_play` (Phase 3 DoD) has not been completed.
+The phone locked itself part-way through the session (5-minute screen timeout,
+secure keyguard) and Artemis will not unlock it by design. It needs a human to
+unlock the phone, and then:
+
+```sh
+./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
+./bin/artemis fg
+./bin/artemis dump | grep -i pause
+tail -1 logs/runs.jsonl
+```
+
+Phase 4 (song radio) and Phase 5 (maps) have not been started, on instruction.

internal/recipe/engine.go

@@ -0,0 +1,376 @@
+package recipe
+
+import (
+	"fmt"
+	"io"
+	"strings"
+	"time"
+
+	"gitea.lab.audasmedia.com.au/sam/artemis_control/internal/adb"
+	"gitea.lab.audasmedia.com.au/sam/artemis_control/internal/ui"
+)
+
+// StepResult is the record of one executed step (FR-10).
+type StepResult struct {
+	Index      int           `json:"step"`
+	Verb       string        `json:"verb"`
+	Args       []string      `json:"args,omitempty"`
+	Detail     string        `json:"detail,omitempty"`
+	OK         bool          `json:"ok"`
+	Error      string        `json:"error,omitempty"`
+	DurationMS int64         `json:"duration_ms"`
+	Duration   time.Duration `json:"-"`
+}
+
+// RunResult is the record of one whole recipe run (FR-10).
+type RunResult struct {
+	Task       string            `json:"task"`
+	Params     map[string]string `json:"params,omitempty"`
+	OK         bool              `json:"ok"`
+	Error      string            `json:"error,omitempty"`
+	Steps      []StepResult      `json:"steps"`
+	DurationMS int64             `json:"duration_ms"`
+	Duration   time.Duration     `json:"-"`
+}
+
+// Options carry the caller's parameters and where to write the step log.
+type Options struct {
+	Params map[string]string
+	Log    io.Writer
+}
+
+// Run executes a recipe against the phone.
+//
+// It refuses to start on a locked phone or a missing app, bounds every step
+// (FR-11), and reports the reason for the first failing step. A run never
+// returns an error silently: the returned RunResult always says what happened.
+func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult {
+	started := time.Now()
+	params := opts.Params
+	if params == nil {
+		params = map[string]string{}
+	}
+	res := &RunResult{Task: rec.Task, Params: params}
+	defer func() {
+		res.Duration = time.Since(started)
+		res.DurationMS = res.Duration.Milliseconds()
+	}()
+
+	logf := func(format string, args ...any) {
+		if opts.Log != nil {
+			fmt.Fprintf(opts.Log, format+"\n", args...)
+		}
+	}
+
+	fail := func(err error) *RunResult {
+		res.OK = false
+		res.Error = err.Error()
+		logf("FAILED %s after %s: %s", rec.Task, time.Since(started).Round(time.Millisecond), res.Error)
+		return res
+	}
+
+	logf("task %s (%s)", rec.Task, rec.Description)
+
+	if missing := rec.MissingParams(params); len(missing) > 0 {
+		return fail(fmt.Errorf("missing required parameter(s): %s", strings.Join(missing, ", ")))
+	}
+	if rec.App != "" {
+		installed, err := ui.AppInstalled(r, rec.App)
+		if err != nil {
+			return fail(fmt.Errorf("could not check whether %s is installed: %w", rec.App, err))
+		}
+		if !installed {
+			return fail(fmt.Errorf("app is not installed: %s", rec.App))
+		}
+	}
+
+	if err := ui.Wake(r); err != nil {
+		return fail(err)
+	}
+	// A secure keyguard is a hard stop: Artemis has no credential and must
+	// never type one (AGENTS.md section 13 Q2).
+	if err := ui.RequireUnlocked(r); err != nil {
+		return fail(err)
+	}
+
+	w, h, _ := ui.ScreenSize(r) // best effort: bounds checking degrades gracefully
+
+	deadline := started.Add(rec.TotalTimeout)
+	for i, step := range rec.Steps {
+		if remaining := time.Until(deadline); remaining <= 0 {
+			return fail(fmt.Errorf("recipe timed out after %s at step %d (%s)", rec.TotalTimeout, i+1, step.Verb))
+		}
+		sr := executeStep(r, rec, step, i, params, w, h, deadline, logf)
+		res.Steps = append(res.Steps, sr)
+		if !sr.OK {
+			return fail(fmt.Errorf("step %d (%s %s) failed: %s", i+1, step.Verb, strings.Join(step.Args, " "), sr.Error))
+		}
+	}
+
+	res.OK = true
+	logf("OK %s in %s", rec.Task, time.Since(started).Round(time.Millisecond))
+	return res
+}
+
+func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[string]string,
+	w, h int, deadline time.Time, logf func(string, ...any)) StepResult {
+
+	started := time.Now()
+	sr := StepResult{Index: index + 1, Verb: step.Verb, Args: step.Args}
+	defer func() {
+		sr.Duration = time.Since(started)
+		sr.DurationMS = sr.Duration.Milliseconds()
+	}()
+
+	timeout := step.Timeout
+	if timeout <= 0 {
+		timeout = rec.StepTimeout
+	}
+	if remaining := time.Until(deadline); timeout > remaining {
+		timeout = remaining
+	}
+	if timeout <= 0 {
+		sr.Error = "no time left in the recipe budget"
+		return sr
+	}
+
+	args := make([]string, len(step.Args))
+	for i, a := range step.Args {
+		v, err := Substitute(a, params)
+		if err != nil {
+			sr.Error = err.Error()
+			return sr
+		}
+		args[i] = v
+	}
+
+	stepErr := func() error {
+		switch step.Verb {
+		case "launch":
+			pkg := rec.App
+			if len(args) == 1 {
+				pkg = args[0]
+			}
+			if pkg == "" {
+				return fmt.Errorf("no app named: add an \"app:\" setting or pass a package")
+			}
+			if err := ui.Launch(r, pkg); err != nil {
+				return err
+			}
+			sr.Detail = "launched " + pkg
+			return nil
+
+		case "wait":
+			node, err := ui.WaitForText(r, args[0], step.Exact, timeout)
+			if err != nil {
+				return describeFailure(r, rec, err)
+			}
+			sr.Detail = "found " + node.Label()
+			return nil
+
+		case "wait_gone":
+			if err := ui.WaitForNoText(r, args[0], step.Exact, timeout); err != nil {
+				return describeFailure(r, rec, err)
+			}
+			sr.Detail = fmt.Sprintf("%q is gone", args[0])
+			return nil
+
+		case "tap":
+			return tapByText(r, rec, args[0], step.Exact, timeout, &sr)
+
+		case "tap_if":
+			// Tap only if the text is already on screen. This makes a recipe
+			// tolerant of a screen that remembers state, without ever tapping
+			// something that was not matched by text.
+			h, err := ui.Dump(r)
+			if err != nil {
+				return err
+			}
+			matches := h.FindText(args[0], step.Exact)
+			if len(matches) == 0 {
+				sr.Detail = fmt.Sprintf("%q not on screen, skipped", args[0])
+				return nil
+			}
+			node, err := ui.TapNode(r, matches)
+			if err != nil {
+				return err
+			}
+			cx, cy, _ := node.Center()
+			sr.Detail = fmt.Sprintf("tapped %q at (%d,%d)", args[0], cx, cy)
+			return nil
+
+		case "tap_row":
+			return tapRow(r, rec, args[0], timeout, w, h, &sr)
+
+		case "type":
+			if err := ui.Type(r, args[0]); err != nil {
+				return err
+			}
+			sr.Detail = fmt.Sprintf("typed %d characters", len(args[0]))
+			return nil
+
+		case "key":
+			return sendKey(r, args[0], &sr)
+
+		case "back":
+			if err := ui.Back(r); err != nil {
+				return err
+			}
+			sr.Detail = "pressed back"
+			return nil
+
+		case "home":
+			if err := ui.Home(r); err != nil {
+				return err
+			}
+			sr.Detail = "pressed home"
+			return nil
+
+		case "sleep":
+			time.Sleep(timeout)
+			sr.Detail = "slept " + timeout.String()
+			return nil
+		}
+		return fmt.Errorf("unknown step verb %q", step.Verb)
+	}()
+
+	if stepErr != nil {
+		sr.OK = false
+		sr.Error = stepErr.Error()
+		logf("  step %d %s %v: %s", sr.Index, step.Verb, args, sr.Error)
+		return sr
+	}
+
+	sr.OK = true
+	logf("  step %d %s %v: ok (%s)", sr.Index, step.Verb, args, sr.Duration.Round(time.Millisecond))
+	return sr
+}
+
+// tapByText finds an element by text and taps its centre (FR-4).
+//
+// The element is looked up twice: once to wait for it, and again immediately
+// before the tap, so the coordinates come from the newest dump. On a busy
+// phone the first dump can be seconds old by the time the tap is sent.
+func tapByText(r *adb.Runner, rec *Recipe, query string, exact bool, timeout time.Duration, sr *StepResult) error {
+	if _, err := ui.WaitForText(r, query, exact, timeout); err != nil {
+		return describeFailure(r, rec, err)
+	}
+	h, err := ui.Dump(r)
+	if err != nil {
+		return err
+	}
+	matches := h.FindText(query, exact)
+	if len(matches) == 0 {
+		return describeFailure(r, rec, fmt.Errorf("element with text %q disappeared before the tap", query))
+	}
+	node, err := ui.TapNode(r, matches)
+	if err != nil {
+		return err
+	}
+	cx, cy, _ := node.Center()
+	sr.Detail = fmt.Sprintf("tapped %q at (%d,%d) %s", query, cx, cy, node.Label())
+	return nil
+}
+
+// tapRow taps the list row that best matches the query. It retries the dump
+// until a matching row is on screen, then taps a point derived from that row's
+// own bounds.
+func tapRow(r *adb.Runner, rec *Recipe, query string, timeout time.Duration, w, h int, sr *StepResult) error {
+	deadline := time.Now().Add(timeout)
+	var lastErr error
+	for time.Now().Before(deadline) {
+		hier, err := ui.Dump(r)
+		if err == nil {
+			row, rerr := hier.BestRow(query, rec.RowID, w, h)
+			if rerr == nil {
+				if err := ui.Tap(r, row.Centre.X, row.Centre.Y); err != nil {
+					return err
+				}
+				sr.Detail = fmt.Sprintf("tapped row %q at (%d,%d) from %s", row.Title, row.Centre.X, row.Centre.Y,
+					strings.Join(row.Texts, " / "))
+				return nil
+			}
+			lastErr = rerr
+		} else {
+			lastErr = err
+		}
+		time.Sleep(500 * time.Millisecond)
+	}
+	if lastErr == nil {
+		lastErr = fmt.Errorf("no result row matches %q", query)
+	}
+	return describeFailure(r, rec, lastErr)
+}
+
+func sendKey(r *adb.Runner, name string, sr *StepResult) error {
+	codes := map[string]int{
+		"enter":  ui.KeyEnter,
+		"back":   ui.KeyBack,
+		"home":   ui.KeyHome,
+		"power":  26,
+		"wakeup": ui.KeyWakeup,
+		"escape": 111,
+	}
+	code, ok := codes[strings.ToLower(name)]
+	if !ok {
+		return fmt.Errorf("unknown key %q", name)
+	}
+	if err := ui.KeyEvent(r, code); err != nil {
+		return err
+	}
+	sr.Detail = "pressed " + strings.ToLower(name)
+	return nil
+}
+
+// describeFailure turns a step failure into a clean, named failure (FR-8).
+//
+// A guard match wins: "Spotify is signed out" is far more useful than "text
+// Search did not appear". Otherwise the message names the screen that was on
+// the phone when the step gave up.
+func describeFailure(r *adb.Runner, rec *Recipe, err error) error {
+	if err == nil {
+		return nil
+	}
+	h, dumpErr := ui.Dump(r)
+	if dumpErr == nil {
+		for _, g := range rec.Guards {
+			if len(h.FindText(g.Text, false)) > 0 {
+				return fmt.Errorf("%s (phone is showing %q): %w", g.Reason, g.Text, err)
+			}
+		}
+	}
+	screen := describeScreen(r, h, dumpErr)
+	if screen == "" {
+		return err
+	}
+	return fmt.Errorf("%w; %s", err, screen)
+}
+
+// describeScreen names the screen Artemis gave up on: the foreground package
+// plus the first visible lines of text.
+func describeScreen(r *adb.Runner, h *ui.Hierarchy, dumpErr error) string {
+	var parts []string
+	if fg, err := ui.ForegroundPackage(r); err == nil {
+		parts = append(parts, "foreground="+fg)
+	}
+	if dumpErr == nil && h != nil {
+		var labels []string
+		for _, n := range h.TextNodes() {
+			label := strings.TrimSpace(n.Text)
+			if label == "" {
+				label = strings.TrimSpace(n.ContentDesc)
+			}
+			if label == "" || len(label) > 60 {
+				continue
+			}
+			labels = append(labels, fmt.Sprintf("%q", label))
+			if len(labels) == 6 {
+				break
+			}
+		}
+		if len(labels) > 0 {
+			parts = append(parts, "screen shows: "+strings.Join(labels, ", "))
+		}
+	}
+	return strings.Join(parts, "; ")
+}

internal/recipe/recipe.go

@@ -0,0 +1,437 @@
+// Package recipe loads and runs named, deterministic, hand-editable tasks.
+//
+// A recipe is one plain-text file in recipes/, written so that Sam can change
+// it by hand without a build step. It names the app, the parameters it needs,
+// the screens it must never tap on, and the ordered steps that do the work.
+// Steps talk about text ("tap the element whose text is Search"), never about
+// coordinates, so a recipe never taps blindly (FR-4, FR-5, FR-8).
+//
+// # File format
+//
+// A recipe file is a list of lines. Blank lines and lines starting with "#"
+// are ignored.
+//
+// Directives are "key: value" lines placed before the steps:
+//
+//	task: spotify_play            unique name, must match the file name
+//	description: ...              one line, for humans
+//	app: com.spotify.music        the package the recipe drives
+//	param: query                  a parameter the caller must supply (repeatable)
+//	row_id: com.spotify:id/row_root   container id for list rows (optional)
+//	step_timeout: 20s             default per-step timeout (FR-11)
+//	total_timeout: 120s           whole-recipe timeout (FR-11)
+//
+// A guard names a screen Artemis must never act on. Guards are checked when a
+// step cannot find its target, so the failure names the real problem:
+//
+//	guard "Log in" => Spotify is signed out; sign in on the phone and retry
+//
+// Steps are "- " lines, run in order:
+//
+//   - launch                  start the recipe's app
+//   - wait   "Search" 30s     wait for an element whose text matches
+//   - wait_gone "Recents" 30s wait until no element matches the text
+//   - tap    "Search"         tap the element whose text matches (FR-4)
+//   - tap    exact "Pause"    tap an element whose text matches exactly
+//   - tap_if "Clear search query"  tap it when it is on screen, else skip
+//   - tap_row "{query}" 30s   tap the list row that best matches the text
+//   - type   "{query}"        type into the focused field
+//   - key    enter            send a named key event
+//   - back / home             press a system key
+//   - sleep  1s               wait a fixed time
+//
+// {name} placeholders are replaced with the value of parameter "name". The
+// trailing "30s" on wait/tap/tap_row steps is that step's timeout.
+package recipe
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+	"time"
+)
+
+// Recipe is one parsed task.
+type Recipe struct {
+	Task         string
+	Description  string
+	App          string
+	Params       []string
+	RowID        string
+	StepTimeout  time.Duration
+	TotalTimeout time.Duration
+	Guards       []Guard
+	Steps        []Step
+	Source       string
+}
+
+// Guard is a screen Artemis must never act on, with the reason to report.
+type Guard struct {
+	Text   string
+	Reason string
+}
+
+// Step is one ordered action.
+type Step struct {
+	Verb    string
+	Args    []string
+	Exact   bool
+	Timeout time.Duration
+	Line    int
+}
+
+// Defaults used when a recipe does not set its own timeouts.
+const (
+	DefaultStepTimeout  = 20 * time.Second
+	DefaultTotalTimeout = 120 * time.Second
+)
+
+// validKeys are the directive names a recipe file may use.
+var validKeys = map[string]bool{
+	"task":          true,
+	"description":   true,
+	"app":           true,
+	"param":         true,
+	"row_id":        true,
+	"step_timeout":  true,
+	"total_timeout": true,
+}
+
+// verbs lists the step verbs and how many arguments each takes.
+var verbs = map[string]struct {
+	min, max   int
+	durArg     bool // the single argument is a duration
+	allowExact bool
+}{
+	"launch":    {0, 1, false, false},
+	"wait":      {1, 2, false, true},
+	"wait_gone": {1, 2, false, true},
+	"tap":       {1, 2, false, true},
+	"tap_if":    {1, 1, false, true},
+	"tap_row":   {1, 2, false, true},
+	"type":      {1, 1, false, false},
+	"key":       {1, 1, false, false},
+	"back":      {0, 0, false, false},
+	"home":      {0, 0, false, false},
+	"sleep":     {1, 1, true, false},
+}
+
+// keyNames are the key event names a recipe may use.
+var keyNames = map[string]bool{
+	"enter":  true,
+	"back":   true,
+	"home":   true,
+	"power":  true,
+	"wakeup": true,
+	"escape": true,
+}
+
+// Load reads and parses one recipe file.
+func Load(path string) (*Recipe, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return nil, err
+	}
+	rec, err := Parse(string(raw))
+	if err != nil {
+		return nil, fmt.Errorf("%s: %w", path, err)
+	}
+	rec.Source = path
+	if rec.Task == "" {
+		rec.Task = strings.TrimSuffix(filepath.Base(path), filepath.Ext(path))
+	}
+	return rec, nil
+}
+
+// List loads every recipe in dir, sorted by task name.
+func List(dir string) ([]*Recipe, error) {
+	entries, err := os.ReadDir(dir)
+	if err != nil {
+		return nil, fmt.Errorf("read recipes directory %s: %w", dir, err)
+	}
+	var out []*Recipe
+	for _, e := range entries {
+		if e.IsDir() || strings.HasPrefix(e.Name(), ".") {
+			continue
+		}
+		rec, err := Load(filepath.Join(dir, e.Name()))
+		if err != nil {
+			return nil, err
+		}
+		out = append(out, rec)
+	}
+	sort.Slice(out, func(i, j int) bool { return out[i].Task < out[j].Task })
+	return out, nil
+}
+
+// Find loads the recipe for one task name from dir.
+func Find(dir, task string) (*Recipe, error) {
+	recipes, err := List(dir)
+	if err != nil {
+		return nil, err
+	}
+	for _, r := range recipes {
+		if r.Task == task {
+			return r, nil
+		}
+	}
+	names := make([]string, 0, len(recipes))
+	for _, r := range recipes {
+		names = append(names, r.Task)
+	}
+	if len(names) == 0 {
+		return nil, fmt.Errorf("no recipes in %s", dir)
+	}
+	return nil, fmt.Errorf("no recipe named %q in %s (have: %s)", task, dir, strings.Join(names, ", "))
+}
+
+// Parse turns the body of a recipe file into a Recipe.
+func Parse(src string) (*Recipe, error) {
+	rec := &Recipe{
+		StepTimeout:  DefaultStepTimeout,
+		TotalTimeout: DefaultTotalTimeout,
+	}
+
+	for i, raw := range strings.Split(src, "\n") {
+		lineNo := i + 1
+		line := strings.TrimSpace(raw)
+		switch {
+		case line == "", strings.HasPrefix(line, "#"):
+			continue
+		case line == "steps:":
+			continue
+		case strings.HasPrefix(line, "-"):
+			step, err := parseStep(strings.TrimSpace(strings.TrimPrefix(line, "-")), lineNo)
+			if err != nil {
+				return nil, err
+			}
+			rec.Steps = append(rec.Steps, *step)
+		case strings.HasPrefix(line, "guard "):
+			guard, err := parseGuard(line, lineNo)
+			if err != nil {
+				return nil, err
+			}
+			rec.Guards = append(rec.Guards, *guard)
+		default:
+			if err := rec.parseDirective(line, lineNo); err != nil {
+				return nil, err
+			}
+		}
+	}
+
+	if len(rec.Steps) == 0 {
+		return nil, fmt.Errorf("recipe has no steps")
+	}
+	return rec, nil
+}
+
+func (rec *Recipe) parseDirective(line string, lineNo int) error {
+	key, value, ok := strings.Cut(line, ":")
+	if !ok {
+		return fmt.Errorf("line %d: expected \"key: value\", a \"guard ...\" line or a \"- step\" line, got %q", lineNo, line)
+	}
+	key = strings.ToLower(strings.TrimSpace(key))
+	value = strings.TrimSpace(value)
+	if !validKeys[key] {
+		return fmt.Errorf("line %d: unknown setting %q", lineNo, key)
+	}
+
+	switch key {
+	case "task":
+		rec.Task = value
+	case "description":
+		rec.Description = value
+	case "app":
+		rec.App = value
+	case "param":
+		if value == "" {
+			return fmt.Errorf("line %d: param needs a name", lineNo)
+		}
+		rec.Params = append(rec.Params, value)
+	case "row_id":
+		rec.RowID = value
+	case "step_timeout":
+		d, err := parseDuration(value)
+		if err != nil {
+			return fmt.Errorf("line %d: step_timeout: %w", lineNo, err)
+		}
+		rec.StepTimeout = d
+	case "total_timeout":
+		d, err := parseDuration(value)
+		if err != nil {
+			return fmt.Errorf("line %d: total_timeout: %w", lineNo, err)
+		}
+		rec.TotalTimeout = d
+	}
+	return nil
+}
+
+func parseGuard(line string, lineNo int) (*Guard, error) {
+	body := strings.TrimSpace(strings.TrimPrefix(line, "guard"))
+	text, reason, ok := strings.Cut(body, "=>")
+	if !ok {
+		return nil, fmt.Errorf("line %d: a guard line needs \"guard \\\"text\\\" => reason\"", lineNo)
+	}
+	tokens, err := tokenize(text, lineNo)
+	if err != nil {
+		return nil, err
+	}
+	if len(tokens) != 1 {
+		return nil, fmt.Errorf("line %d: a guard needs exactly one quoted text, got %d", lineNo, len(tokens))
+	}
+	reason = strings.TrimSpace(reason)
+	if reason == "" {
+		return nil, fmt.Errorf("line %d: a guard needs a reason after =>", lineNo)
+	}
+	return &Guard{Text: tokens[0], Reason: reason}, nil
+}
+
+func parseStep(body string, lineNo int) (*Step, error) {
+	tokens, err := tokenize(body, lineNo)
+	if err != nil {
+		return nil, err
+	}
+	if len(tokens) == 0 {
+		return nil, fmt.Errorf("line %d: empty step", lineNo)
+	}
+
+	verb := strings.ToLower(tokens[0])
+	spec, ok := verbs[verb]
+	if !ok {
+		return nil, fmt.Errorf("line %d: unknown step verb %q", lineNo, tokens[0])
+	}
+	step := &Step{Verb: verb, Line: lineNo}
+	args := tokens[1:]
+
+	if spec.allowExact && len(args) > 0 && strings.EqualFold(args[0], "exact") {
+		step.Exact = true
+		args = args[1:]
+	}
+
+	// A trailing duration is the step timeout, except for sleep where the
+	// duration is the step itself.
+	if !spec.durArg && len(args) > spec.min {
+		if d, derr := parseDuration(args[len(args)-1]); derr == nil {
+			step.Timeout = d
+			args = args[:len(args)-1]
+		}
+	}
+
+	if len(args) < spec.min || len(args) > spec.max {
+		return nil, fmt.Errorf("line %d: %s takes %s, got %d", lineNo, verb, argRange(spec.min, spec.max), len(args))
+	}
+	step.Args = args
+
+	if spec.durArg {
+		d, derr := parseDuration(args[0])
+		if derr != nil {
+			return nil, fmt.Errorf("line %d: sleep: %w", lineNo, derr)
+		}
+		step.Timeout = d
+	}
+	if verb == "key" && !keyNames[strings.ToLower(args[0])] {
+		return nil, fmt.Errorf("line %d: unknown key %q", lineNo, args[0])
+	}
+	return step, nil
+}
+
+func argRange(min, max int) string {
+	switch {
+	case min == max:
+		return fmt.Sprintf("exactly %d argument(s)", min)
+	case min == 0:
+		return fmt.Sprintf("up to %d argument(s)", max)
+	default:
+		return fmt.Sprintf("%d or %d arguments", min, max)
+	}
+}
+
+func parseDuration(s string) (time.Duration, error) {
+	d, err := time.ParseDuration(strings.TrimSpace(s))
+	if err != nil {
+		return 0, fmt.Errorf("%q is not a duration (use 20s, 1500ms, 2m)", s)
+	}
+	if d <= 0 {
+		return 0, fmt.Errorf("duration %q must be positive", s)
+	}
+	return d, nil
+}
+
+// tokenize splits a recipe line into words, treating a double-quoted run as one
+// word. Backslash escapes the next character inside quotes.
+func tokenize(s string, lineNo int) ([]string, error) {
+	var out []string
+	for i := 0; i < len(s); {
+		switch {
+		case s[i] == ' ' || s[i] == '\t':
+			i++
+		case s[i] == '"':
+			var b strings.Builder
+			j := i + 1
+			for j < len(s) && s[j] != '"' {
+				if s[j] == '\\' && j+1 < len(s) {
+					j++
+				}
+				b.WriteByte(s[j])
+				j++
+			}
+			if j >= len(s) {
+				return nil, fmt.Errorf("line %d: unterminated quoted string", lineNo)
+			}
+			out = append(out, b.String())
+			i = j + 1
+		default:
+			j := i
+			for j < len(s) && s[j] != ' ' && s[j] != '\t' {
+				j++
+			}
+			out = append(out, s[i:j])
+			i = j
+		}
+	}
+	return out, nil
+}
+
+// Substitute replaces {name} placeholders with parameter values.
+//
+// Every declared parameter must be present, and every placeholder must be
+// declared: an unset value would otherwise be typed into the phone as an empty
+// string.
+func Substitute(text string, params map[string]string) (string, error) {
+	var out strings.Builder
+	for i := 0; i < len(text); {
+		if text[i] != '{' {
+			out.WriteByte(text[i])
+			i++
+			continue
+		}
+		end := strings.IndexByte(text[i:], '}')
+		if end < 0 {
+			return "", fmt.Errorf("unclosed { in %q", text)
+		}
+		name := text[i+1 : i+end]
+		value, ok := params[name]
+		if !ok {
+			return "", fmt.Errorf("no value for parameter %q in %q", name, text)
+		}
+		if strings.TrimSpace(value) == "" {
+			return "", fmt.Errorf("parameter %q is empty", name)
+		}
+		out.WriteString(value)
+		i += end + 1
+	}
+	return out.String(), nil
+}
+
+// MissingParams returns the declared parameters that the caller did not supply.
+func (rec *Recipe) MissingParams(params map[string]string) []string {
+	var missing []string
+	for _, name := range rec.Params {
+		if strings.TrimSpace(params[name]) == "" {
+			missing = append(missing, name)
+		}
+	}
+	return missing
+}

internal/recipe/recipe_test.go

@@ -0,0 +1,194 @@
+package recipe
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+)
+
+const sample = `# a comment
+task: spotify_play
+description: Search Spotify for a song and play it.
+app: com.spotify.music
+param: query
+row_id: com.spotify.music:id/row_root
+step_timeout: 20s
+total_timeout: 150s
+
+guard "Log in" => Spotify is signed out
+guard "Update Spotify" => Spotify wants an update
+
+steps:
+  - launch
+  - wait "Search" 30s
+  - tap "Search"
+  - tap_if "Clear search query"
+  - wait_gone "Recents" 30s
+  - type "{query}"
+  - tap_row "{query}" 40s
+  - wait exact "Pause" 40s
+  - key enter
+  - sleep 1s
+`
+
+func TestParseSampleRecipe(t *testing.T) {
+	rec, err := Parse(sample)
+	if err != nil {
+		t.Fatalf("Parse: %v", err)
+	}
+	if rec.Task != "spotify_play" || rec.App != "com.spotify.music" {
+		t.Fatalf("unexpected header: %+v", rec)
+	}
+	if len(rec.Params) != 1 || rec.Params[0] != "query" {
+		t.Fatalf("params = %v", rec.Params)
+	}
+	if rec.StepTimeout != 20*time.Second || rec.TotalTimeout != 150*time.Second {
+		t.Fatalf("timeouts = %s / %s", rec.StepTimeout, rec.TotalTimeout)
+	}
+	if len(rec.Guards) != 2 || rec.Guards[0].Text != "Log in" || rec.Guards[0].Reason != "Spotify is signed out" {
+		t.Fatalf("guards = %+v", rec.Guards)
+	}
+	if len(rec.Steps) != 10 {
+		t.Fatalf("steps = %d, want 10", len(rec.Steps))
+	}
+
+	wait := rec.Steps[1]
+	if wait.Verb != "wait" || wait.Args[0] != "Search" || wait.Timeout != 30*time.Second {
+		t.Fatalf("wait step = %+v", wait)
+	}
+	exact := rec.Steps[7]
+	if exact.Verb != "wait" || !exact.Exact || exact.Args[0] != "Pause" || exact.Timeout != 40*time.Second {
+		t.Fatalf("exact wait step = %+v", exact)
+	}
+	sleep := rec.Steps[9]
+	if sleep.Verb != "sleep" || sleep.Timeout != time.Second {
+		t.Fatalf("sleep step = %+v", sleep)
+	}
+}
+
+func TestParseRejectsBadInput(t *testing.T) {
+	cases := map[string]string{
+		"unknown verb":     "steps:\n  - teleport \"Home\"\n",
+		"missing settings": "steps:\n  - wait \"x\"\n  - frobnicate: 3\n",
+		"bad duration":     "step_timeout: soon\nsteps:\n  - wait \"x\"\n",
+		"no steps":         "task: nothing\n",
+		"guard needs text": "guard only a reason => nope\nsteps:\n  - home\n",
+		"guard needs why":  "guard \"Log in\" =>\nsteps:\n  - home\n",
+		"unknown key name": "steps:\n  - key volup\n",
+		"unclosed quote":   "steps:\n  - wait \"Search\n",
+		"too many args":    "steps:\n  - type \"a\" \"b\"\n",
+		"unknown setting":  "colour: blue\nsteps:\n  - home\n",
+	}
+	for name, src := range cases {
+		if _, err := Parse(src); err == nil {
+			t.Errorf("%s: expected a parse error", name)
+		}
+	}
+}
+
+func TestSubstitute(t *testing.T) {
+	got, err := Substitute("play {query} now", map[string]string{"query": "Honky Tonk Women"})
+	if err != nil {
+		t.Fatalf("Substitute: %v", err)
+	}
+	if got != "play Honky Tonk Women now" {
+		t.Fatalf("Substitute = %q", got)
+	}
+	for _, bad := range []map[string]string{{}, {"query": "  "}} {
+		if _, err := Substitute("{query}", bad); err == nil {
+			t.Fatalf("expected an error for params %v", bad)
+		}
+	}
+	if _, err := Substitute("{other}", map[string]string{"query": "x"}); err == nil {
+		t.Fatal("expected an error for an undeclared placeholder")
+	}
+}
+
+func TestMissingParams(t *testing.T) {
+	rec, err := Parse(sample)
+	if err != nil {
+		t.Fatalf("Parse: %v", err)
+	}
+	if got := rec.MissingParams(map[string]string{"query": "x"}); len(got) != 0 {
+		t.Fatalf("MissingParams = %v", got)
+	}
+	if got := rec.MissingParams(nil); len(got) != 1 || got[0] != "query" {
+		t.Fatalf("MissingParams = %v", got)
+	}
+}
+
+func TestParseRecipeWithoutTimeoutsUsesDefaults(t *testing.T) {
+	rec, err := Parse("steps:\n  - home\n")
+	if err != nil {
+		t.Fatalf("Parse: %v", err)
+	}
+	if rec.StepTimeout != DefaultStepTimeout || rec.TotalTimeout != DefaultTotalTimeout {
+		t.Fatalf("defaults = %s / %s", rec.StepTimeout, rec.TotalTimeout)
+	}
+}
+
+// TestSpotifyRecipeIsValid keeps the shipped recipe honest: a typo in
+// recipes/spotify_play.recipe fails the test suite, not the phone.
+func TestSpotifyRecipeIsValid(t *testing.T) {
+	dir := filepath.Join("..", "..", "recipes")
+	rec, err := Load(filepath.Join(dir, "spotify_play.recipe"))
+	if err != nil {
+		t.Fatalf("Load: %v", err)
+	}
+	if rec.Task != "spotify_play" {
+		t.Fatalf("task = %q", rec.Task)
+	}
+	if rec.App != "com.spotify.music" {
+		t.Fatalf("app = %q", rec.App)
+	}
+	if len(rec.MissingParams(map[string]string{"query": "x"})) != 0 {
+		t.Fatal("spotify_play must accept a query parameter")
+	}
+	if len(rec.Guards) < 3 {
+		t.Fatalf("spotify_play needs guards for its bad screens, got %d", len(rec.Guards))
+	}
+	// Every step must be runnable: the parsed steps are what the engine gets.
+	for i, s := range rec.Steps {
+		if _, ok := verbs[s.Verb]; !ok {
+			t.Fatalf("step %d uses unknown verb %q", i+1, s.Verb)
+		}
+	}
+	// The recipe must wait for playback, not assume it.
+	last := rec.Steps[len(rec.Steps)-1]
+	if last.Verb != "wait" || last.Args[0] != "Pause" {
+		t.Fatalf("the last step must confirm playback, got %+v", last)
+	}
+}
+
+func TestLoadAndList(t *testing.T) {
+	dir := t.TempDir()
+	write := func(name, body string) {
+		t.Helper()
+		if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o644); err != nil {
+			t.Fatal(err)
+		}
+	}
+	write("b.recipe", "task: beta\nsteps:\n  - home\n")
+	write("a.recipe", "task: alpha\nsteps:\n  - home\n")
+
+	list, err := List(dir)
+	if err != nil {
+		t.Fatalf("List: %v", err)
+	}
+	if len(list) != 2 || list[0].Task != "alpha" || list[1].Task != "beta" {
+		t.Fatalf("List = %+v", list)
+	}
+
+	rec, err := Find(dir, "beta")
+	if err != nil {
+		t.Fatalf("Find: %v", err)
+	}
+	if rec.Task != "beta" {
+		t.Fatalf("Find = %+v", rec)
+	}
+	if _, err := Find(dir, "gamma"); err == nil || !strings.Contains(err.Error(), "alpha") {
+		t.Fatalf("Find unknown task: %v", err)
+	}
+}

internal/ui/actions.go

@@ -128,6 +128,17 @@ func TapText(r *adb.Runner, query string, exact bool) (*Node, error) {
 	if len(matches) == 0 {
 		return nil, fmt.Errorf("element with text %q not found on screen; refusing to tap blindly", query)
 	}
+	return TapNode(r, matches)
+}
+
+// TapNode taps the centre of the most tappable node among matches.
+//
+// Callers that already hold a hierarchy use this instead of TapText, so the
+// coordinates come from the dump they just read rather than an older one.
+func TapNode(r *adb.Runner, matches []*Node) (*Node, error) {
+	if len(matches) == 0 {
+		return nil, fmt.Errorf("no element was matched, refusing to tap blindly")
+	}
 	node, err := pickBestMatch(matches)
 	if err != nil {
 		return nil, err
@@ -170,6 +181,24 @@ func WaitForText(r *adb.Runner, query string, exact bool, timeout time.Duration)
 	return nil, fmt.Errorf("text %q did not appear within %s", query, timeout)
 }
 
+// WaitForNoText polls the screen until no element matches query. Recipes use it
+// to wait for one list to be replaced by another, for example a search result
+// list replacing the "Recents" list.
+func WaitForNoText(r *adb.Runner, query string, exact bool, timeout time.Duration) error {
+	if timeout <= 0 {
+		timeout = 10 * time.Second
+	}
+	deadline := time.Now().Add(timeout)
+	for time.Now().Before(deadline) {
+		h, err := Dump(r)
+		if err == nil && len(h.FindText(query, exact)) == 0 {
+			return nil
+		}
+		time.Sleep(400 * time.Millisecond)
+	}
+	return fmt.Errorf("text %q was still on screen after %s", query, timeout)
+}
+
 // ScreenSize reads the physical screen size.
 func ScreenSize(r *adb.Runner) (width, height int, err error) {
 	out, err := r.Shell("wm size")
@@ -247,6 +276,10 @@ func Scroll(r *adb.Runner, direction string) error {
 }
 
 // Wake makes sure the screen is on before any action (FR-7).
+//
+// It dismisses the keyguard only when that keyguard is not secure. A secure
+// keyguard (PIN, pattern or password) is left exactly as it is: Artemis never
+// tries to get past it (AGENTS.md section 13 Q2).
 func Wake(r *adb.Runner) error {
 	if _, err := r.Shell("input keyevent " + strconv.Itoa(KeyWakeup)); err != nil {
 		return fmt.Errorf("wake keyevent: %w", err)
@@ -255,8 +288,11 @@ func Wake(r *adb.Runner) error {
 	for time.Now().Before(deadline) {
 		out, err := r.Shell("dumpsys power | grep mWakefulness=")
 		if err == nil && strings.Contains(out, "mWakefulness=Awake") {
-			// Dismiss a non-secure keyguard; harmless when the phone is unlocked.
+			// Only a non-secure keyguard may be dismissed. If the state cannot
-			_, _ = r.Shell("wm dismiss-keyguard")
+			// be read we leave the keyguard alone rather than guess.
+			if showing, secure, kerr := KeyguardState(r); kerr == nil && (!showing || !secure) {
+				_, _ = r.Shell("wm dismiss-keyguard")
+			}
 			return nil
 		}
 		time.Sleep(300 * time.Millisecond)

internal/ui/keyguard.go

@@ -0,0 +1,114 @@
+package ui
+
+import (
+	"errors"
+	"fmt"
+	"strings"
+
+	"gitea.lab.audasmedia.com.au/sam/artemis_control/internal/adb"
+)
+
+// ErrLocked is returned when a secure keyguard is showing. Artemis never tries
+// to get past it: no PIN, pattern or password is stored, read or typed
+// (AGENTS.md section 13 Q2).
+var ErrLocked = errors.New("phone is locked; unlock it and retry")
+
+// KeyguardState reports whether the lock screen is showing and, separately,
+// whether the device has a secure lock (PIN, pattern or password) at all.
+//
+// The two are independent: a phone with no PIN still shows a keyguard when it
+// wakes, but that one may be dismissed. A secure one may not.
+func KeyguardState(r *adb.Runner) (showing bool, secure bool, err error) {
+	dump, err := r.Shell("dumpsys window")
+	if err != nil {
+		return false, false, fmt.Errorf("read window state: %w", err)
+	}
+	showing, err = keyguardShowing(dump)
+	if err != nil {
+		return false, false, err
+	}
+	secure, err = userHasSecureLock(r)
+	if err != nil {
+		return false, false, err
+	}
+	return showing, secure, nil
+}
+
+// keyguardShowing parses `dumpsys window` for the KeyguardServiceDelegate
+// block. An unparsable dump is an error: guessing here would mean acting on a
+// phone whose lock state is unknown.
+func keyguardShowing(dump string) (bool, error) {
+	showing := false
+	sawDelegate := false
+	sawShowing := false
+	inDelegate := false
+
+	for _, raw := range strings.Split(dump, "\n") {
+		line := strings.TrimSpace(raw)
+		switch {
+		case strings.HasPrefix(line, "KeyguardServiceDelegate"):
+			inDelegate, sawDelegate = true, true
+			continue
+		case strings.HasPrefix(line, "isKeyguardShowing="):
+			if strings.TrimPrefix(line, "isKeyguardShowing=") == "true" {
+				showing = true
+			}
+			continue
+		}
+		if !inDelegate {
+			continue
+		}
+		// The delegate block is a flat list of key=value lines.
+		if !strings.Contains(line, "=") {
+			inDelegate = false
+			continue
+		}
+		if strings.HasPrefix(line, "showing=") {
+			showing = strings.TrimPrefix(line, "showing=") == "true"
+			sawShowing = true
+		}
+	}
+
+	if !sawDelegate || !sawShowing {
+		return false, fmt.Errorf("could not read the keyguard state from dumpsys window")
+	}
+	return showing, nil
+}
+
+// userHasSecureLock asks the lock settings service whether the current user has
+// a secure lock screen. It never asks for or handles the credential itself.
+func userHasSecureLock(r *adb.Runner) (bool, error) {
+	out, err := r.Shell("cmd lock_settings is-user-secure")
+	if err != nil {
+		return false, fmt.Errorf("read lock state: %w", err)
+	}
+	switch strings.TrimSpace(out) {
+	case "true":
+		return true, nil
+	case "false":
+		return false, nil
+	}
+	return false, fmt.Errorf("unexpected lock state %q from cmd lock_settings is-user-secure", strings.TrimSpace(out))
+}
+
+// RequireUnlocked refuses to act while a secure keyguard is showing.
+//
+// A non-secure keyguard (the phone has no PIN at all) is dismissed, because
+// that screen guards nothing. A secure one is left alone: Artemis stops and
+// says so.
+func RequireUnlocked(r *adb.Runner) error {
+	showing, secure, err := KeyguardState(r)
+	if err != nil {
+		return fmt.Errorf("cannot confirm the phone is unlocked: %w", err)
+	}
+	if !showing {
+		return nil
+	}
+	if secure {
+		return ErrLocked
+	}
+	if _, err := r.Shell("wm dismiss-keyguard"); err != nil {
+		return fmt.Errorf("dismiss the non-secure keyguard: %w", err)
+	}
+	return nil
+}

internal/ui/keyguard_test.go

@@ -0,0 +1,81 @@
+package ui
+
+import "testing"
+
+// A real (trimmed) KeyguardServiceDelegate block from `dumpsys window`.
+const keyguardLockedDump = `WINDOW MANAGER POLICY STATE (dumpsys window policy)
+    mSafeMode=false mSystemReady=true
+    mKeyguardOccluded=false mPendingKeyguardOccluded=false
+    KeyguardServiceDelegate
+      showing=true
+      inputRestricted=true
+      occluded=false
+      trusted=false
+      simSecure=false
+      dreaming=false
+      systemReady=true
+      deviceHasKeyguard=true
+      enabled=true
+      userId=0
+      bootCompleted=true
+      screenState=SCREEN_STATE_ON
+      interactiveState=INTERACTIVE_STATE_AWAKE
+    Looper state:
+      Looper (android.ui, tid 22) {4c5c23e}
+`
+
+const keyguardUnlockedDump = `WINDOW MANAGER POLICY STATE (dumpsys window policy)
+    mKeyguardOccluded=false mPendingKeyguardOccluded=false
+    KeyguardServiceDelegate
+      showing=false
+      inputRestricted=false
+      occluded=false
+      trusted=false
+      simSecure=false
+      dreaming=false
+      systemReady=true
+      deviceHasKeyguard=true
+      enabled=true
+      userId=0
+      bootCompleted=true
+      screenState=SCREEN_STATE_ON
+      interactiveState=INTERACTIVE_STATE_AWAKE
+    Looper state:
+      Looper (android.ui, tid 22) {4c5c23e}
+`
+
+func TestKeyguardShowingParsesTheDelegateBlock(t *testing.T) {
+	showing, err := keyguardShowing(keyguardLockedDump)
+	if err != nil {
+		t.Fatalf("keyguardShowing: %v", err)
+	}
+	if !showing {
+		t.Fatal("want showing=true")
+	}
+
+	showing, err = keyguardShowing(keyguardUnlockedDump)
+	if err != nil {
+		t.Fatalf("keyguardShowing: %v", err)
+	}
+	if showing {
+		t.Fatal("want showing=false")
+	}
+}
+
+func TestKeyguardShowingRejectsAnUnparsableDump(t *testing.T) {
+	// Acting on a phone whose lock state is unknown is not allowed.
+	if _, err := keyguardShowing("dumpsys window: service not found\n"); err == nil {
+		t.Fatal("expected an error when the delegate block is missing")
+	}
+}
+
+func TestKeyguardShowingReadsIsKeyguardShowing(t *testing.T) {
+	dump := "  KeyguardServiceDelegate\n      showing=false\n  mDisableSecureWindows=false\n  isKeyguardShowing=true\n"
+	showing, err := keyguardShowing(dump)
+	if err != nil {
+		t.Fatalf("keyguardShowing: %v", err)
+	}
+	if !showing {
+		t.Fatal("want showing=true from isKeyguardShowing")
+	}
+}

internal/ui/rows.go

@@ -0,0 +1,252 @@
+package ui
+
+import (
+	"fmt"
+	"math"
+	"strings"
+	"unicode"
+)
+
+// Rows are the only safe way to press a list result.
+//
+// A search result row is a clickable container that holds a title and a
+// subtitle ("Honky Tonk Women" / "Song • The Rolling Stones"). The container is
+// the text-matched element, and its centre is the tap point: that lands on the
+// title, never on the row's "More options" or "Add to library" buttons, which
+// sit in a small action bar at the row's right edge.
+//
+// Selecting the row instead of the smallest clickable child matters. On the
+// real Spotify screen a plain text match resolved to a 126x12 pixel "More
+// options for song ..." button, because that was the smallest clickable node.
+
+// Row is one pressable list row.
+type Row struct {
+	Node   *Node
+	Texts  []string
+	Score  float64
+	Title  string
+	Top    int
+	Centre Point
+}
+
+// Point is a screen coordinate.
+type Point struct {
+	X, Y int
+}
+
+// BestRow chooses the row that best matches query.
+//
+// rowID, when set, restricts candidates to elements with that resource id
+// (either the full "package:id/name" form or a suffix of it).
+func (h *Hierarchy) BestRow(query, rowID string, screenW, screenH int) (*Row, error) {
+	query = strings.TrimSpace(query)
+	if query == "" {
+		return nil, fmt.Errorf("row match needs a non-empty query")
+	}
+	qTokens := tokenize(query)
+	if len(qTokens) == 0 {
+		return nil, fmt.Errorf("query %q has no matchable words", query)
+	}
+
+	candidates := h.rowCandidates(rowID, screenW, screenH)
+	if len(candidates) == 0 {
+		return nil, fmt.Errorf("no result row is fully drawn on screen; refusing to tap blindly")
+	}
+
+	var best *Row
+	for _, node := range candidates {
+		rect, err := node.BoundsRect()
+		if err != nil {
+			continue
+		}
+		texts := rowTexts(node)
+		score, matched := rowScore(qTokens, texts)
+		// More than half of the query words must be on the row, and the row
+		// must not be buried under unrelated words.
+		if matched*2 < len(qTokens) || score < minRowScore {
+			continue
+		}
+		cx, cy, err := node.Center()
+		if err != nil {
+			continue
+		}
+		candidate := &Row{
+			Node:   node,
+			Texts:  texts,
+			Score:  score,
+			Title:  texts[0],
+			Top:    rect.Y1,
+			Centre: Point{X: cx, Y: cy},
+		}
+		if best == nil || betterRow(candidate, best) {
+			best = candidate
+		}
+	}
+
+	if best == nil {
+		return nil, fmt.Errorf("no result row matches %q; refusing to tap blindly", query)
+	}
+	return best, nil
+}
+
+// minRowScore is the F1 floor for an acceptable row match.
+const minRowScore = 0.5
+
+// rowCandidates returns the fully drawn rows on screen: clickable containers
+// that carry at least a title and a subtitle.
+//
+// Two shapes are rejected on purpose. A one-line search *suggestion* is not a
+// result. And a row clipped by the bottom of the list is much shorter than a
+// row that is fully scrolled into view, so its centre is not a point a human
+// could press; rows shorter than half the tallest candidate are dropped.
+func (h *Hierarchy) rowCandidates(rowID string, screenW, screenH int) []*Node {
+	var found []*Node
+	tallest := 0
+	h.Walk(func(n *Node) {
+		if !n.Clickable || !n.Enabled || !rowMatchesID(n, rowID) {
+			return
+		}
+		rect, err := n.BoundsRect()
+		if err != nil || !onScreen(rect, screenW, screenH) {
+			return
+		}
+		if len(rowTexts(n)) < 2 {
+			return
+		}
+		if h := rect.Y2 - rect.Y1; h > tallest {
+			tallest = h
+		}
+		found = append(found, n)
+	})
+
+	var out []*Node
+	for _, n := range found {
+		rect, err := n.BoundsRect()
+		if err != nil {
+			continue
+		}
+		if (rect.Y2-rect.Y1)*2 < tallest {
+			continue
+		}
+		out = append(out, n)
+	}
+	return out
+}
+
+// betterRow ranks rows: the best text match wins, and equal matches go to the
+// row nearest the top, because search results are ranked by relevance.
+func betterRow(a, b *Row) bool {
+	if math.Abs(a.Score-b.Score) > 1e-9 {
+		return a.Score > b.Score
+	}
+	return a.Top < b.Top
+}
+
+func rowMatchesID(n *Node, rowID string) bool {
+	if rowID == "" {
+		return true
+	}
+	return n.ResourceID == rowID || strings.HasSuffix(n.ResourceID, rowID)
+}
+
+// textDescendants returns the visible text lines inside a node, in reading
+// order, without duplicates. It is used for dumps and diagnostics.
+func textDescendants(n *Node) []string {
+	var out []string
+	seen := map[string]bool{}
+	n.Walk(func(c *Node) {
+		for _, t := range []string{c.Text, c.ContentDesc} {
+			t = strings.TrimSpace(t)
+			if t == "" || seen[t] {
+				continue
+			}
+			seen[t] = true
+			out = append(out, t)
+		}
+	})
+	return out
+}
+
+// rowTexts returns the text that belongs to the row itself: its title and
+// subtitle.
+//
+// Anything inside a clickable child is an action button ("More options", "Add
+// to library", "Add suggestion"), not the row's own text, so that subtree is
+// skipped. This is what keeps a one-line search suggestion from looking like a
+// two-line result.
+func rowTexts(n *Node) []string {
+	var out []string
+	seen := map[string]bool{}
+	add := func(s string) {
+		s = strings.TrimSpace(s)
+		if s == "" || seen[s] {
+			return
+		}
+		seen[s] = true
+		out = append(out, s)
+	}
+
+	var visit func(c *Node, root bool)
+	visit = func(c *Node, root bool) {
+		if !root && c.Clickable {
+			return
+		}
+		add(c.Text)
+		if !c.Clickable {
+			add(c.ContentDesc)
+		}
+		for i := range c.Children {
+			visit(&c.Children[i], false)
+		}
+	}
+	visit(n, true)
+	return out
+}
+
+// onScreen reports whether the whole rectangle is inside the display. A row
+// clipped by the screen edge would be tapped at a point the user cannot see,
+// so it is not a candidate.
+func onScreen(r Rect, w, h int) bool {
+	if w <= 0 || h <= 0 {
+		return true
+	}
+	return r.X1 >= 0 && r.Y1 >= 0 && r.X2 <= w && r.Y2 <= h
+}
+
+// rowScore returns the F1 score of the query words against the row's words,
+// plus how many distinct query words were found.
+func rowScore(query []string, texts []string) (score float64, matched int) {
+	row := map[string]bool{}
+	for _, t := range texts {
+		for _, w := range tokenize(t) {
+			row[w] = true
+		}
+	}
+	hit := map[string]bool{}
+	for _, w := range query {
+		if row[w] {
+			hit[w] = true
+		}
+	}
+	if len(hit) == 0 || len(row) == 0 {
+		return 0, 0
+	}
+	precision := float64(len(hit)) / float64(len(row))
+	recall := float64(len(hit)) / float64(len(query))
+	if precision+recall == 0 {
+		return 0, len(hit)
+	}
+	return 2 * precision * recall / (precision + recall), len(hit)
+}
+
+// tokenize splits text into lowercase words.
+func tokenize(s string) []string {
+	fields := strings.FieldsFunc(s, func(r rune) bool {
+		return !unicode.IsLetter(r) && !unicode.IsDigit(r)
+	})
+	out := make([]string, 0, len(fields))
+	for _, f := range fields {
+		out = append(out, strings.ToLower(f))
+	}
+	return out
+}

internal/ui/rows_test.go

@@ -0,0 +1,174 @@
+package ui
+
+import (
+	"strings"
+	"testing"
+)
+
+// spotifyRowDump is the shape of a real Spotify search-results screen, trimmed
+// to the rows that matter. The first row is a search *suggestion* (one line of
+// text); the rest are result rows (title + subtitle).
+const spotifyRowDump = `<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>
+<hierarchy rotation="0">
+  <node index="0" class="androidx.recyclerview.widget.RecyclerView" resource-id="com.spotify.music:id/search_content_recyclerview" clickable="false" enabled="true" bounds="[0,251][1080,2400]">
+    <node index="0" class="android.view.ViewGroup" resource-id="com.spotify.music:id/row_root" clickable="true" enabled="true" bounds="[0,251][1080,425]">
+      <node index="0" class="android.widget.ImageView" resource-id="com.spotify.music:id/icon_view" clickable="false" enabled="true" bounds="[42,296][126,380]"/>
+      <node index="1" class="android.widget.TextView" resource-id="com.spotify.music:id/query" text="rolling stones honky tonk women single" clickable="false" enabled="true" bounds="[158,272][954,404]"/>
+      <node index="2" class="android.widget.ImageView" resource-id="com.spotify.music:id/tap_ahead_button" content-desc="Add suggestion “rolling stones honky tonk women single”" clickable="true" enabled="true" bounds="[954,296][1080,380]"/>
+    </node>
+    <node index="1" class="android.view.ViewGroup" resource-id="com.spotify.music:id/row_root" clickable="true" enabled="true" bounds="[0,794][1080,962]">
+      <node index="0" class="android.widget.ImageView" resource-id="com.spotify.music:id/artwork" clickable="false" enabled="true" bounds="[42,815][168,941]"/>
+      <node index="1" class="android.widget.TextView" resource-id="com.spotify.music:id/title" text="Honky Tonk Women" clickable="false" enabled="true" bounds="[200,815][828,881]"/>
+      <node index="2" class="android.widget.TextView" resource-id="com.spotify.music:id/subtitle" text="Song • The Rolling Stones" clickable="false" enabled="true" bounds="[200,886][828,941]"/>
+      <node index="3" class="android.view.ViewGroup" resource-id="com.spotify.music:id/action_bar" clickable="false" enabled="true" bounds="[828,815][1080,941]">
+        <node index="0" class="android.widget.ImageButton" resource-id="com.spotify.music:id/context_menu_button" content-desc="More options for song Honky Tonk Women" clickable="true" enabled="true" bounds="[828,815][954,941]"/>
+        <node index="1" class="android.widget.Button" resource-id="com.spotify.music:id/add_button" content-desc="Add Honky Tonk Women to My Library" clickable="true" enabled="true" bounds="[954,815][1080,941]"/>
+      </node>
+    </node>
+    <node index="2" class="android.view.ViewGroup" resource-id="com.spotify.music:id/row_root" clickable="true" enabled="true" bounds="[0,962][1080,1130]">
+      <node index="0" class="android.widget.TextView" resource-id="com.spotify.music:id/title" text="Honky Tonk Women - Mono Version" clickable="false" enabled="true" bounds="[200,983][828,1049]"/>
+      <node index="1" class="android.widget.TextView" resource-id="com.spotify.music:id/subtitle" text="Song • The Rolling Stones" clickable="false" enabled="true" bounds="[200,1054][828,1109]"/>
+    </node>
+    <node index="3" class="android.view.ViewGroup" resource-id="com.spotify.music:id/row_root" clickable="true" enabled="true" bounds="[0,1130][1080,1359]">
+      <node index="0" class="android.widget.TextView" resource-id="com.spotify.music:id/title" text="Honky Tonk Woman (The Rolling Stones) Covers" clickable="false" enabled="true" bounds="[200,1151][954,1283]"/>
+      <node index="1" class="android.widget.TextView" resource-id="com.spotify.music:id/subtitle" text="Playlist" clickable="false" enabled="true" bounds="[200,1283][954,1338]"/>
+    </node>
+    <node index="4" class="android.view.ViewGroup" resource-id="com.spotify.music:id/row_root" clickable="true" enabled="true" bounds="[0,2367][1080,2400]">
+      <node index="0" class="android.widget.TextView" resource-id="com.spotify.music:id/title" text="Honky Tonk Women - Live" clickable="false" enabled="true" bounds="[200,2388][828,2400]"/>
+      <node index="1" class="android.widget.TextView" resource-id="com.spotify.music:id/subtitle" text="Song • The Rolling Stones" clickable="false" enabled="true" bounds="[200,2388][828,2400]"/>
+    </node>
+  </node>
+</hierarchy>`
+
+const spotifyRowID = "com.spotify.music:id/row_root"
+
+func spotifyRows(t *testing.T) *Hierarchy {
+	t.Helper()
+	h, err := parseHierarchy([]byte(spotifyRowDump))
+	if err != nil {
+		t.Fatalf("parseHierarchy: %v", err)
+	}
+	return h
+}
+
+func TestBestRowPicksTheTopResultOfARealScreen(t *testing.T) {
+	h := spotifyRows(t)
+	row, err := h.BestRow("Rolling Stones Honky Tonk", spotifyRowID, 1080, 2400)
+	if err != nil {
+		t.Fatalf("BestRow: %v", err)
+	}
+	if row.Title != "Honky Tonk Women" {
+		t.Fatalf("chose row %q, want the canonical top hit; rows: %v", row.Title, row.Texts)
+	}
+	// The tap must land on the row, on the title, not on a row button.
+	if row.Centre.X < 200 || row.Centre.X > 828 {
+		t.Fatalf("tap x=%d is outside the title", row.Centre.X)
+	}
+	if row.Centre.Y < 794 || row.Centre.Y > 962 {
+		t.Fatalf("tap y=%d is outside the row", row.Centre.Y)
+	}
+}
+
+func TestBestRowIgnoresSearchSuggestions(t *testing.T) {
+	h := spotifyRows(t)
+	// The suggestion row carries every query word, but it is a single line of
+	// text, so it is not a result row.
+	query := "rolling stones honky tonk women single"
+	row, err := h.BestRow(query, spotifyRowID, 1080, 2400)
+	if err != nil {
+		t.Fatalf("BestRow: %v", err)
+	}
+	if row.Title == "rolling stones honky tonk women single" {
+		t.Fatal("chose the search suggestion; a suggestion has no subtitle and is not a result")
+	}
+}
+
+func TestBestRowIgnoresRowsClippedByTheScreenEdge(t *testing.T) {
+	// The last row is cut off at the bottom of the list: only 33 pixels tall,
+	// with its centre underneath the navigation area. It matches the query
+	// better than any other row, and it must still lose, because tapping it
+	// would press something the user cannot see.
+	h := spotifyRows(t)
+	row, err := h.BestRow("Honky Tonk Women Live", spotifyRowID, 1080, 2400)
+	if err != nil {
+		t.Fatalf("BestRow: %v", err)
+	}
+	if row.Title == "Honky Tonk Women - Live" {
+		t.Fatal("chose a row clipped by the bottom of the list")
+	}
+}
+
+func TestBestRowRefusesAnUnknownQuery(t *testing.T) {
+	h := spotifyRows(t)
+	if _, err := h.BestRow("Chopin Nocturne", spotifyRowID, 1080, 2400); err == nil {
+		t.Fatal("expected an error when no row matches the query")
+	}
+}
+
+func TestBestRowWorksWithoutARowID(t *testing.T) {
+	h := spotifyRows(t)
+	row, err := h.BestRow("The Rolling Stones Honky Tonk Women", "", 1080, 2400)
+	if err != nil {
+		t.Fatalf("BestRow without a row id: %v", err)
+	}
+	if row.Title != "Honky Tonk Women" {
+		t.Fatalf("chose %q", row.Title)
+	}
+}
+
+func TestRowScorePrefersPrecision(t *testing.T) {
+	exact, _ := rowScore(tokenize("honky tonk women"), []string{"Honky Tonk Women", "Song • The Rolling Stones"})
+	noisy, _ := rowScore(tokenize("honky tonk women"), []string{"Honky Tonk Women Live Licks Tour 2009 Remastered Deluxe Edition", "Song • The Rolling Stones, Sheryl Crow"})
+	if exact <= noisy {
+		t.Fatalf("an exact title (%f) must beat a noisy one (%f)", exact, noisy)
+	}
+}
+
+func TestRowTextsSkipActionButtons(t *testing.T) {
+	h := spotifyRows(t)
+
+	// The suggestion row: one line of text plus an "Add suggestion" button.
+	var suggestion *Node
+	h.Walk(func(n *Node) {
+		if n.ResourceID == spotifyRowID && n.Bounds == "[0,251][1080,425]" {
+			suggestion = n
+		}
+	})
+	if suggestion == nil {
+		t.Fatal("suggestion row not found in the sample dump")
+	}
+	texts := rowTexts(suggestion)
+	if len(texts) != 1 {
+		t.Fatalf("a suggestion row's own text = %v, want 1 line", texts)
+	}
+
+	// A result row: title + subtitle, without the row's action buttons.
+	var result *Node
+	h.Walk(func(n *Node) {
+		if n.ResourceID == spotifyRowID && n.Bounds == "[0,794][1080,962]" {
+			result = n
+		}
+	})
+	texts = rowTexts(result)
+	if len(texts) != 2 || texts[0] != "Honky Tonk Women" || texts[1] != "Song • The Rolling Stones" {
+		t.Fatalf("a result row's own text = %v", texts)
+	}
+	for _, bad := range texts {
+		if strings.Contains(bad, "More options") || strings.Contains(bad, "Add ") {
+			t.Fatalf("row text %q came from an action button", bad)
+		}
+	}
+}
+
+func TestTokenize(t *testing.T) {
+	got := tokenize("Honky Tonk Women - Mono Version (Remastered)")
+	want := []string{"honky", "tonk", "women", "mono", "version", "remastered"}
+	if len(got) != len(want) {
+		t.Fatalf("tokenize = %v, want %v", got, want)
+	}
+	for i := range want {
+		if got[i] != want[i] {
+			t.Fatalf("tokenize = %v, want %v", got, want)
+		}
+	}
+}

recipes/spotify_play.recipe

@@ -0,0 +1,63 @@
+# spotify_play — search Spotify for a song and play it.
+#
+# Edit this file by hand. Steps name the text Artemis must find on screen; it
+# taps the centre of the element it matched. There are no coordinates here on
+# purpose, so this recipe survives a Spotify redesign (FR-4).
+#
+# Run it with:
+#   artemis run spotify_play query="Rolling Stones Honky Tonk"
+
+task: spotify_play
+description: Search Spotify for a song and play it.
+app: com.spotify.music
+
+# The caller must supply these.
+param: query
+
+# Spotify's list rows: title + subtitle inside one clickable container. Naming
+# the container keeps the tap on the row itself and off the row's buttons.
+row_id: com.spotify.music:id/row_root
+
+step_timeout: 20s
+total_timeout: 150s
+
+# Screens Artemis must never tap on. Each one fails the run with the reason, so
+# a wrong tap is impossible (FR-8).
+guard "Log in"        => Spotify is signed out; sign in on the phone and retry
+guard "Sign up free"  => Spotify is signed out; sign in on the phone and retry
+guard "Log in to Spotify" => Spotify is signed out; sign in on the phone and retry
+guard "Continue with Facebook" => Spotify is showing its welcome screen; sign in on the phone and retry
+guard "Continue with Google"   => Spotify is showing its welcome screen; sign in on the phone and retry
+guard "Update Spotify" => Spotify wants an update; update it on the phone and retry
+guard "What's new"     => Spotify is showing an announcement screen; dismiss it on the phone and retry
+
+steps:
+  # 1. Make sure Spotify is up. Launching an app that is already open is a
+  #    no-op, so this is safe to repeat.
+  - launch
+  - wait "Search" 30s
+
+  # 2. Open the search tab. Spotify remembers the last tab, so a search screen
+  #    with the previous query can come back: clear that box only when the
+  #    clear button is actually on screen.
+  - tap "Search"
+  - tap_if "Clear search query"
+  - wait "What do you want to listen to?" 30s
+  - tap "What do you want to listen to?"
+
+  # 3. Type the query. The clear button proves the text landed.
+  - type "{query}"
+  - wait "Clear search query" 30s
+
+  # 4. Wait for the results to replace the "Recents" list, so the next step can
+  #    never match something from the history instead of a search result.
+  - wait_gone "Recents" 30s
+
+  # 5. Play the result that matches the query. tap_row scores every visible
+  #    result row against the query words and taps the best one; equal matches
+  #    go to the highest row, which is how Spotify ranks them.
+  - tap_row "{query}" 40s
+
+  # 6. Playback started when the transport button says "Pause". Exact match, so
+  #    Spotify's "Paused" labels do not count.
+  - wait exact "Pause" 40s