artemis_control/docs/phase-3-spotify-recipe.md

Phase 3 — the recipes file and spotify_play

Phase 3 of AGENTS.md section 11: the first named recipe, plus the resolved answer to section 13 Q2 (the locked screen).

Date: 2026-10-16. Phone: Pixel_8a (akita) over Tailscale at 100.101.49.17:46689 (the port moves on every reboot; nothing hardcodes it).

What was built

File	What it holds
recipes/spotify_play.recipe	The recipe, written to be edited by hand.
internal/recipe/recipe.go	The file format: directives, guards, steps, parameters.
internal/recipe/engine.go	Runs a recipe step by step with timeouts and a run log.
internal/ui/rows.go	Picks a list row by text and taps its centre.
internal/ui/keyguard.go	Reads the lock state and refuses to act when locked.
cmd/artemis/run.go	artemis recipes, artemis run <task> k=v, JSON run log.

The recipes file

One file per task in recipes/, plain text, no build step. It names the app, the parameters the caller must supply, the screens Artemis must never tap on, and the ordered steps.

task: spotify_play
app: com.spotify.music
param: query
row_id: com.spotify.music:id/row_root

guard "Log in" => Spotify is signed out; sign in on the phone and retry

steps:
  - launch
  - wait "Search" 30s
  - tap "Search"
  - tap_if "Clear search query"
  - wait "What do you want to listen to?" 30s
  - tap "What do you want to listen to?"
  - type "{query}"
  - wait "Clear search query" 30s
  - wait_gone "Recents" 30s
  - tap_row "{query}" 40s
  - wait exact "Pause" 40s

Steps express intent. There is no coordinate anywhere in the file (FR-4).

Run it with:

./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"

The verbs are launch, wait, wait_gone, tap, tap_if, tap_row, type, key, back, home and sleep. tap exact "Pause" matches the whole text, not a substring. tap_if taps only when the text is already on screen, which lets one recipe cope with an app that remembers its last screen.

Why tap_row exists

A plain text tap picked the wrong element on the real Spotify screen. On the results list, artemis tap "Honky Tonk Women" tapped

ImageButton[context_menu_button] text="More options for song Honky Tonk Women - Live"
  bounds=[828,2388][954,2400]

That is the smallest clickable match, and it sits at the bottom edge of the screen. It was the wrong target: a "More options" button, not the song.

tap_row scores every result row against the query and taps the row itself:

• A candidate row is a clickable container that carries a title and a subtitle. A one-line search suggestion (Spotify's "rolling stones honky tonk women single") is therefore not a result row.
• Text inside a clickable child is skipped, because that is an action button ("More options", "Add to library", "Add suggestion"), not the row's own text.
• A row clipped by the bottom of the list is dropped. It is much shorter than a row that is fully drawn, and its centre sits under the navigation area.
• Rows are ranked by F1 score over the query words, so an exact title beats a noisy deluxe-edition title. Equal scores go to the highest row, because search results are ranked by relevance.

On the real screen for Rolling Stones Honky Tonk, the top hit wins:

Honky Tonk Women / Song • The Rolling Stones   bounds=[0,794][1080,962]  F1=0.727

and the tap lands at the row's centre, (540,878) — on the title, clear of the action buttons. internal/ui/rows_test.go runs this against a trimmed copy of the real dump.

Failing cleanly (FR-8)

Every failure names its cause, and they are distinguishable:

Situation	Message
App missing	app is not installed: com.spotify.music
Signed out	Spotify is signed out; sign in on the phone and retry (phone is showing "Log in")
Update nag	Spotify wants an update; update it on the phone and retry
Element missing	text "Search" did not appear within 30s; foreground=com.spotify.music; screen shows: "Log in", "Sign up free"
Phone locked	phone is locked; unlock it and retry

A guard is checked when a step cannot find its target, so a bad screen produces its own reason instead of a generic "element not found".

Timeouts (FR-11)

step_timeout: 20s is the default per step, total_timeout: 150s bounds the whole run. A step may name its own timeout (wait "Search" 30s). The engine checks the remaining budget before each step and stops when it runs out. Every adb call is already bounded by adb.DefaultTimeout.

Logging (FR-10)

Every run appends one JSON line to logs/runs.jsonl (override with ARTEMIS_LOG): task, params, each step with its duration, the result and the total duration. Two real entries:

{"time":"2026-10-16T13:23:31+11:00","task":"spotify_play","params":{"query":"Rolling Stones Honky Tonk"},"ok":false,"error":"phone is locked; unlock it and retry","steps":null,"duration_ms":832}

The same steps print to the terminal as they run.

The locked screen (section 13 Q2)

Decision: Artemis fails loudly. No PIN is stored and none is ever typed.

ui.KeyguardState reads two things: whether the lock screen is showing (dumpsys window, the KeyguardServiceDelegate block) and whether the device has a secure lock at all (cmd lock_settings is-user-secure). Acting commands wake the screen first and then refuse if the keyguard is showing and secure. A non-secure keyguard is still dismissed, as before. An unreadable lock state is a failure, not a guess.

Evidence

The prescribed test (put the phone to sleep, wake it, Artemis must refuse):

adb -s 100.101.49.17:46689 shell input keyevent 26    # POWER = sleep
adb -s 100.101.49.17:46689 shell input keyevent 224   # WAKEUP

$ adb shell dumpsys window policy | grep -E 'showing=|screenState'
      showing=true
      screenState=SCREEN_STATE_ON
$ adb shell cmd lock_settings is-user-secure
true

$ ./bin/artemis keyguard
keyguard showing=true secure=true
artemis: phone is locked; unlock it and retry
exit=1

$ ./bin/artemis tap "Search"
artemis: phone is locked; unlock it and retry
exit=1

$ ./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
task spotify_play (Search Spotify for a song and play it.)
FAILED spotify_play after 833ms: phone is locked; unlock it and retry
exit=1

No PIN was entered, nothing was read from the phone, and the lock screen was still on screen afterwards — no blind tap happened. The screen dump after the refusals still showed the lock screen clock and date.

A dozing phone reports showing=false, because the lock screen is not drawn yet. That is why every acting command wakes the screen before it checks the keyguard; reading the state first would say "unlocked" about a locked phone.

Verification status

Item	State
Recipes file format, guards, parameters, timeouts	unit tested (internal/recipe)
Row selection against a real Spotify dump	unit tested (internal/ui)
Keyguard parsing and secure-lock detection	unit tested (internal/ui)
All acting commands refuse on a locked phone	live, verified (above)
artemis run spotify_play refuses on a locked phone	live, verified (above)
spotify_play plays the right song	not yet run end to end — see below
Spotify search reaches the result list	live (Home → Search → type → results)

The live end-to-end run of spotify_play (Phase 3 DoD) has not been completed. The phone locked itself part-way through the session (5-minute screen timeout, secure keyguard) and Artemis will not unlock it by design. It needs a human to unlock the phone, and then:

./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
./bin/artemis fg
./bin/artemis dump | grep -i pause
tail -1 logs/runs.jsonl

Phase 4 (song radio) and Phase 5 (maps) have not been started, on instruction.