feat: Phase 3 recipes file, spotify_play, and locked-screen refusal

Phase 3 of AGENTS.md section 11.

Recipes file (FR-5)
- recipes/ holds one hand-editable file per task; recipes/spotify_play.recipe
  is the first one.
- internal/recipe parses directives, guards, parameters and steps, and runs
  them with a per-step and a whole-run timeout (FR-11).
- Steps name text ("tap the element whose text is Search"), never coordinates.
- guards name screens Artemis must never tap on, so a signed-out or
  first-run Spotify fails with its own reason (FR-8).
- Every run appends task, params, steps, result and duration to
  logs/runs.jsonl (FR-10).

spotify_play
- launch, search, play, then wait for the transport button to say "Pause".
- internal/ui/rows.go adds tap_row: a result row is a clickable container with
  a title and a subtitle, so a one-line search suggestion is not a result, and
  the tap lands on the row's centre rather than on its "More options" button.
  This fixes a real wrong tap: a plain text tap hit a 126x12 pixel
  "More options for song Honky Tonk Women - Live" button.

Locked screen (AGENTS.md section 13 Q2, decided)
- Artemis fails loudly on a secure keyguard and never types a PIN. No secret is
  stored. A non-secure keyguard is still dismissed.
- internal/ui/keyguard.go reads the KeyguardServiceDelegate block plus
  cmd lock_settings is-user-secure.
- Every acting command wakes the screen before it checks the keyguard: a dozing
  phone reports showing=false, because the lock screen is not drawn yet.
- AGENTS.md records the decision and that storing a PIN for auto-unlock is
  rejected for now.

Verified live: the phone locked itself mid-session, and keyguard, tap and
run spotify_play all refused with "phone is locked; unlock it and retry".
The live end-to-end playback run still needs an unlocked phone.
This commit is contained in:
2026-10-16 13:24:22 +11:00
parent 7767651c11
commit 2604fea483
15 changed files with 2192 additions and 23 deletions

View File

@@ -12,7 +12,7 @@ order.
|---|---|
| 1 — Prove the ADB link | ✅ proven — see [`docs/phase-1-adb-link.md`](docs/phase-1-adb-link.md) |
| 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch |
| 3 — Spotify recipe | not started |
| 3 — Spotify recipe | 🟡 built — recipes file, guards, `spotify_play`; the live end-to-end run is still to do, see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) |
| 4 — Song radio | not started |
| 5 — Maps recipe | not started |
| 6 — Action API | not started |
@@ -25,6 +25,9 @@ order.
cmd/artemis/ command line entry point
internal/adb/ timeout-bounded wrapper around the adb binary
internal/ui/ read the screen (uiautomator dump) and act on it (input)
internal/recipe/ the recipes file format and its runner
recipes/ one hand-editable file per task
logs/ run log (JSON lines, not committed)
docs/ phase evidence and notes
```
@@ -42,13 +45,15 @@ nix shell nixpkgs#go -c go test ./...
## Use
The phone address defaults to `100.101.49.17`. Override with `ARTEMIS_PHONE`,
pin a serial with `ARTEMIS_ADB_SERIAL`, or point at a different adb with
`ARTEMIS_ADB`.
pin a serial with `ARTEMIS_ADB_SERIAL`, point at a different adb with
`ARTEMIS_ADB`, point at another recipes directory with `ARTEMIS_RECIPES`, and
move the run log with `ARTEMIS_LOG`.
```sh
./bin/artemis connect # scan for the wireless-debugging port, then connect
./bin/artemis devices
./bin/artemis dump # text nodes and bounds
./bin/artemis keyguard # lock-screen state; exits 1 when locked
./bin/artemis launch com.spotify.music
./bin/artemis tap "Search" # find by text, tap its centre (FR-4)
./bin/artemis type "Rolling Stones Honky Tonk"
@@ -58,11 +63,31 @@ pin a serial with `ARTEMIS_ADB_SERIAL`, or point at a different adb with
./bin/artemis demo # Phase 2 DoD: launch an app, tap a named element
```
## Recipes
A recipe is a plain-text file in `recipes/` that Sam edits by hand. Steps name
what to find on screen; there are no coordinates anywhere.
```sh
./bin/artemis recipes # list them
./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
```
Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the
file format, the verbs, and why list rows need their own step.
## Rules that the code enforces
- **Find by text, never fixed coordinates.** `tap` resolves a text match to
bounds and taps the centre.
bounds and taps the centre; `tap_row` scores result rows against the query
and taps the winning row's centre.
- **Never tap blindly.** A missing or disabled element is a loud failure, not a
guessed coordinate.
- **Wake first.** Every acting command wakes the screen (FR-7).
- **Bound every command.** No adb call runs without a timeout (FR-11).
guessed coordinate. A named bad screen (`guard`) fails with its own reason.
- **Never touch a locked phone.** A secure keyguard stops Artemis with
`phone is locked; unlock it and retry`. No PIN is stored, read or typed
(AGENTS.md section 13 Q2).
- **Wake first.** Every acting command wakes the screen before it acts (FR-7),
and it checks the keyguard only after the screen is on.
- **Bound every command.** No adb call runs without a timeout, and a recipe has
a per-step and a whole-run timeout (FR-11).
- **Log every run.** Each run appends a JSON line to `logs/runs.jsonl` (FR-10).