Files

Sam Rolfe 2604fea483 feat: Phase 3 recipes file, spotify_play, and locked-screen refusal

Phase 3 of AGENTS.md section 11.

Recipes file (FR-5)
- recipes/ holds one hand-editable file per task; recipes/spotify_play.recipe
  is the first one.
- internal/recipe parses directives, guards, parameters and steps, and runs
  them with a per-step and a whole-run timeout (FR-11).
- Steps name text ("tap the element whose text is Search"), never coordinates.
- guards name screens Artemis must never tap on, so a signed-out or
  first-run Spotify fails with its own reason (FR-8).
- Every run appends task, params, steps, result and duration to
  logs/runs.jsonl (FR-10).

spotify_play
- launch, search, play, then wait for the transport button to say "Pause".
- internal/ui/rows.go adds tap_row: a result row is a clickable container with
  a title and a subtitle, so a one-line search suggestion is not a result, and
  the tap lands on the row's centre rather than on its "More options" button.
  This fixes a real wrong tap: a plain text tap hit a 126x12 pixel
  "More options for song Honky Tonk Women - Live" button.

Locked screen (AGENTS.md section 13 Q2, decided)
- Artemis fails loudly on a secure keyguard and never types a PIN. No secret is
  stored. A non-secure keyguard is still dismissed.
- internal/ui/keyguard.go reads the KeyguardServiceDelegate block plus
  cmd lock_settings is-user-secure.
- Every acting command wakes the screen before it checks the keyguard: a dozing
  phone reports showing=false, because the lock screen is not drawn yet.
- AGENTS.md records the decision and that storing a PIN for auto-unlock is
  rejected for now.

Verified live: the phone locked itself mid-session, and keyguard, tap and
run spotify_play all refused with "phone is locked; unlock it and retry".
The live end-to-end playback run still needs an unlocked phone.

2026-10-16 13:24:22 +11:00

3.7 KiB

Raw Blame History

Artemis Control

Artemis is the phone's hands. It drives the Pixel 8a screen from .13 over Tailscale using ADB wireless, so the Hub can do things no deep link allows.

The build spec is AGENTS.md. Follow the phases in section 11 in order.

Status

Phase	State
1 — Prove the ADB link	✅ proven — see `docs/phase-1-adb-link.md`
2 — Act primitives	✅ done — tap by text, type, scroll, back, home, launch
3 — Spotify recipe	🟡 built — recipes file, guards, `spotify_play`; the live end-to-end run is still to do, see `docs/phase-3-spotify-recipe.md`
4 — Song radio	not started
5 — Maps recipe	not started
6 — Action API	not started
7 — Resilience	not started
8 — Documentation	not started

Layout

cmd/artemis/        command line entry point
internal/adb/       timeout-bounded wrapper around the adb binary
internal/ui/        read the screen (uiautomator dump) and act on it (input)
internal/recipe/    the recipes file format and its runner
recipes/            one hand-editable file per task
logs/               run log (JSON lines, not committed)
docs/               phase evidence and notes

Build and test

Go is not installed system-wide on .13; use a one-off nix shell. Nothing is added to nixos-rebuild and no system module changes.

nix shell nixpkgs#go -c go build -o bin/artemis ./cmd/artemis
nix shell nixpkgs#go -c go vet ./...
nix shell nixpkgs#go -c go test ./...

Use

The phone address defaults to 100.101.49.17. Override with ARTEMIS_PHONE, pin a serial with ARTEMIS_ADB_SERIAL, point at a different adb with ARTEMIS_ADB, point at another recipes directory with ARTEMIS_RECIPES, and move the run log with ARTEMIS_LOG.

./bin/artemis connect          # scan for the wireless-debugging port, then connect
./bin/artemis devices
./bin/artemis dump             # text nodes and bounds
./bin/artemis keyguard         # lock-screen state; exits 1 when locked
./bin/artemis launch com.spotify.music
./bin/artemis tap "Search"     # find by text, tap its centre (FR-4)
./bin/artemis type "Rolling Stones Honky Tonk"
./bin/artemis scroll down
./bin/artemis back
./bin/artemis home
./bin/artemis demo             # Phase 2 DoD: launch an app, tap a named element

Recipes

A recipe is a plain-text file in recipes/ that Sam edits by hand. Steps name what to find on screen; there are no coordinates anywhere.

./bin/artemis recipes                                     # list them
./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"

Read docs/phase-3-spotify-recipe.md for the file format, the verbs, and why list rows need their own step.

Rules that the code enforces

Find by text, never fixed coordinates. tap resolves a text match to bounds and taps the centre; tap_row scores result rows against the query and taps the winning row's centre.
Never tap blindly. A missing or disabled element is a loud failure, not a guessed coordinate. A named bad screen (guard) fails with its own reason.
Never touch a locked phone. A secure keyguard stops Artemis with phone is locked; unlock it and retry. No PIN is stored, read or typed (AGENTS.md section 13 Q2).
Wake first. Every acting command wakes the screen before it acts (FR-7), and it checks the keyguard only after the screen is on.
Bound every command. No adb call runs without a timeout, and a recipe has a per-step and a whole-run timeout (FR-11).
Log every run. Each run appends a JSON line to logs/runs.jsonl (FR-10).

3.7 KiB Raw Blame History