feat: Phase 4 — spotify_radio, and a bound on logs/runs.jsonl

spotify_radio opens a track's own context menu and starts its song radio.
The labels came from a fresh uiautomator dump on the phone, not from a guess:
the entry is "Go to song radio" and it sits below the fold, so the recipe
scrolls before it matches.

Recipe format:
- menu_row "{query}" finds the row like tap_row, then presses the action
  button inside that row's own subtree (ui.Row.Action), so the More options
  button of any other row is never a candidate. It records the row as {row}.
- a step can name its own failure: wait ... else "reason". The named reason
  comes first and the screen's labels still follow (FR-8).
- scroll up|down moves the current list.
- playing with no title waits for the media session to be PLAYING. A radio
  changes track every few minutes, so a title match would fail on a working
  radio; spotify_play still passes the title it wants.

ui:
- pickBestMatch ignores a match with no usable bounds. An element scrolled out
  of its list reports [0,0][0,0], and tapping its "centre" would be a blind tap
  in the corner. A wait now only succeeds once the element is on screen.

Run log (FR-10): appendRunLog rotates runs.jsonl when a write would pass
ARTEMIS_LOG_MAX_BYTES (default 1 MiB), keeping ARTEMIS_LOG_KEEP (default 5)
files. History is bounded, never deleted wholesale, and the per-entry cap of
6 screen labels of 60 characters is unchanged.

Live evidence in docs/phase-4-song-radio.md: ok:true in 56.6s, the menu that
opened belonged to the pressed row, the station name "Honky Tonk Women Radio"
on the phone, and the radio playing past its seed track (Layla) per the media
session and dumpsys audio. Both new failure modes were demonstrated failing
with distinct reasons. The mini-bar's transport label stays "Play" while
playing on this build, which is why playback is confirmed from the media
session.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
2026-10-16 15:25:50 +11:00
parent 40a8a51c9e
commit 36b4dd2391
11 changed files with 1116 additions and 34 deletions

View File

@@ -13,7 +13,7 @@ order.
| 1 — Prove the ADB link | ✅ proven — see [`docs/phase-1-adb-link.md`](docs/phase-1-adb-link.md) | | 1 — Prove the ADB link | ✅ proven — see [`docs/phase-1-adb-link.md`](docs/phase-1-adb-link.md) |
| 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch | | 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch |
| 3 — Spotify recipe | ✅ done — `spotify_play` plays the right song end to end (13 steps, 35s), see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) | | 3 — Spotify recipe | ✅ done — `spotify_play` plays the right song end to end (13 steps, 35s), see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) |
| 4 — Song radio | not started | | 4 — Song radio | ✅ done — `spotify_radio` starts a real radio (19 steps, 57s); the run log is now bounded at ~1 MB × 5, see [`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) |
| 5 — Maps recipe | not started | | 5 — Maps recipe | not started |
| 6 — Action API | not started | | 6 — Action API | not started |
| 7 — Resilience | not started | | 7 — Resilience | not started |
@@ -71,18 +71,23 @@ what to find on screen; there are no coordinates anywhere.
```sh ```sh
./bin/artemis recipes # list them ./bin/artemis recipes # list them
./bin/artemis run spotify_play query="Rolling Stones Honky Tonk" ./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
./bin/artemis run spotify_radio query="Rolling Stones Honky Tonk"
``` ```
Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the
file format, the verbs, and why list rows need their own step. file format, the verbs, and why list rows need their own step, and
[`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) for the context menu,
`menu_row`, and the two ways a radio run fails on purpose.
## Rules that the code enforces ## Rules that the code enforces
- **Find by text, never fixed coordinates.** `tap` resolves a text match to - **Find by text, never fixed coordinates.** `tap` resolves a text match to
bounds and taps the centre; `tap_row` scores result rows against the query bounds and taps the centre; `tap_row` scores result rows against the query
and taps the winning row's centre. and taps the winning row's centre; `menu_row` opens the action menu of that
same row, from the button inside it.
- **Never tap blindly.** A missing or disabled element is a loud failure, not a - **Never tap blindly.** A missing or disabled element is a loud failure, not a
guessed coordinate. A named bad screen (`guard`) fails with its own reason. guessed coordinate. A named bad screen (`guard`) fails with its own reason,
and a step may name its own failure with `else "..."`.
- **Never touch a locked phone.** A secure keyguard stops Artemis with - **Never touch a locked phone.** A secure keyguard stops Artemis with
`phone is locked; unlock it and retry`. No PIN is stored, read or typed `phone is locked; unlock it and retry`. No PIN is stored, read or typed
(AGENTS.md section 13 Q2). (AGENTS.md section 13 Q2).
@@ -94,3 +99,6 @@ file format, the verbs, and why list rows need their own step.
- **Bound every command.** No adb call runs without a timeout, and a recipe has - **Bound every command.** No adb call runs without a timeout, and a recipe has
a per-step and a whole-run timeout (FR-11). a per-step and a whole-run timeout (FR-11).
- **Log every run.** Each run appends a JSON line to `logs/runs.jsonl` (FR-10). - **Log every run.** Each run appends a JSON line to `logs/runs.jsonl` (FR-10).
The file rotates at about 1 MB, keeping the recent runs (FR-10 is an audit
trail, so history is bounded, never deleted wholesale). Tune it with
`ARTEMIS_LOG_MAX_BYTES` and `ARTEMIS_LOG_KEEP`.

View File

@@ -5,6 +5,7 @@ import (
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
"strconv"
"strings" "strings"
"time" "time"
@@ -12,6 +13,18 @@ import (
"gitea.lab.audasmedia.com.au/sam/artemis_control/internal/recipe" "gitea.lab.audasmedia.com.au/sam/artemis_control/internal/recipe"
) )
// The run log is the FR-10 audit trail, so it is bounded rather than endless.
// It is not deleted wholesale: the recent runs are the evidence that caught the
// lock refusal and the wrong-element tap. When the live file passes the bound it
// rotates to runs.jsonl.1, the previous .1 becomes .2, and the oldest file is
// dropped, so about DefaultLogKeep files of history survive.
const (
// DefaultLogMaxBytes is the live file size that triggers a rotation.
DefaultLogMaxBytes = 1 << 20 // 1 MiB
// DefaultLogKeep is how many rotated files are kept.
DefaultLogKeep = 5
)
// recipesDir is where hand-editable recipes live. // recipesDir is where hand-editable recipes live.
func recipesDir() string { func recipesDir() string {
if d := os.Getenv("ARTEMIS_RECIPES"); d != "" { if d := os.Getenv("ARTEMIS_RECIPES"); d != "" {
@@ -91,8 +104,28 @@ func summaryLine(task string, res *recipe.RunResult) string {
task, steps, res.Duration.Round(time.Millisecond), res.Error) task, steps, res.Duration.Round(time.Millisecond), res.Error)
} }
// logMaxBytes is the live log size that triggers a rotation.
func logMaxBytes() int64 {
if v := os.Getenv("ARTEMIS_LOG_MAX_BYTES"); v != "" {
if n, err := strconv.ParseInt(v, 10, 64); err == nil && n > 0 {
return n
}
}
return DefaultLogMaxBytes
}
// logKeep is how many rotated run logs survive a rotation.
func logKeep() int {
if v := os.Getenv("ARTEMIS_LOG_KEEP"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 {
return n
}
}
return DefaultLogKeep
}
// appendRunLog adds one JSON line per run: task, params, steps, result and // appendRunLog adds one JSON line per run: task, params, steps, result and
// duration (FR-10). // duration (FR-10). It rotates the file when the write would pass the bound.
func appendRunLog(path string, res *recipe.RunResult) error { func appendRunLog(path string, res *recipe.RunResult) error {
if res == nil { if res == nil {
return nil return nil
@@ -102,11 +135,6 @@ func appendRunLog(path string, res *recipe.RunResult) error {
return err return err
} }
} }
f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
if err != nil {
return err
}
defer f.Close()
entry := struct { entry := struct {
Time string `json:"time"` Time string `json:"time"`
@@ -117,6 +145,61 @@ func appendRunLog(path string, res *recipe.RunResult) error {
if err != nil { if err != nil {
return err return err
} }
_, err = f.Write(append(raw, '\n')) raw = append(raw, '\n')
if err := rotateRunLog(path, int64(len(raw)), logMaxBytes(), logKeep()); err != nil {
return err
}
f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
if err != nil {
return err
}
defer f.Close()
_, err = f.Write(raw)
return err return err
} }
// rotateRunLog makes room for an entry of nextBytes.
//
// When the live file is at or over maxBytes it moves to path.1, each existing
// rotated file shifts up by one (path.1 -> path.2 ...), and the oldest file is
// deleted, so keep rotated files survive. A single entry larger than maxBytes
// is still written: the bound must never drop an audit record.
func rotateRunLog(path string, nextBytes, maxBytes int64, keep int) error {
if maxBytes <= 0 {
maxBytes = DefaultLogMaxBytes
}
if keep < 1 {
keep = DefaultLogKeep
}
info, err := os.Stat(path)
if err != nil {
if os.IsNotExist(err) {
return nil
}
return err
}
if info.Size()+nextBytes <= maxBytes {
return nil
}
// The oldest file goes first, then the rest shift up from the highest
// number down, so no rename can clobber a file that has not moved yet.
if err := os.Remove(rotatedName(path, keep)); err != nil && !os.IsNotExist(err) {
return err
}
for i := keep - 1; i >= 1; i-- {
if err := os.Rename(rotatedName(path, i), rotatedName(path, i+1)); err != nil && !os.IsNotExist(err) {
return err
}
}
return os.Rename(path, rotatedName(path, 1))
}
// rotatedName is the name of the n-th rotated run log (runs.jsonl.1, ...).
func rotatedName(path string, n int) string {
return fmt.Sprintf("%s.%d", path, n)
}

148
cmd/artemis/run_test.go Normal file
View File

@@ -0,0 +1,148 @@
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"gitea.lab.audasmedia.com.au/sam/artemis_control/internal/recipe"
)
// TestRotateRunLogKeepsRecentFiles proves the bound: the live file moves aside,
// the older files shift up, and the oldest file is dropped once the limit is
// reached. No history is deleted until the keep count is actually exceeded.
func TestRotateRunLogKeepsRecentFiles(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "runs.jsonl")
// No file yet: nothing to rotate and no error.
if err := rotateRunLog(path, 100, 1000, 2); err != nil {
t.Fatalf("rotateRunLog on a missing file: %v", err)
}
// A file that still fits is left alone.
if err := os.WriteFile(path, []byte("small\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := rotateRunLog(path, 10, 1000, 2); err != nil {
t.Fatalf("rotateRunLog under the bound: %v", err)
}
if _, err := os.Stat(rotatedName(path, 1)); !os.IsNotExist(err) {
t.Fatalf("a file under the bound must not rotate")
}
// Three rotations with keep=2: live is empty, .1 holds the second write,
// .2 holds the first, and the first file is gone.
for i := 1; i <= 3; i++ {
if err := os.WriteFile(path, []byte(fmt.Sprintf("write-%d\n", i)), 0o644); err != nil {
t.Fatal(err)
}
if err := rotateRunLog(path, 1000, 1, 2); err != nil {
t.Fatalf("rotate %d: %v", i, err)
}
}
want := map[string]string{
rotatedName(path, 1): "write-3\n",
rotatedName(path, 2): "write-2\n",
}
for name, body := range want {
got, err := os.ReadFile(name)
if err != nil {
t.Fatalf("read %s: %v", name, err)
}
if string(got) != body {
t.Errorf("%s = %q, want %q", filepath.Base(name), got, body)
}
}
// The third rotation moved the live file aside, so the next append creates
// it again (appendRunLog opens with O_CREATE).
if _, err := os.Stat(path); !os.IsNotExist(err) {
t.Errorf("the live file should have moved to %s", filepath.Base(rotatedName(path, 1)))
}
// write-1 fell off the end, because keep=2 keeps only two rotated files.
if _, err := os.Stat(rotatedName(path, 3)); !os.IsNotExist(err) {
t.Errorf("keep=2 must not leave a third rotated file")
}
}
// TestAppendRunLogRotatesAndKeepsEveryEntry drives the real logging path with a
// tiny bound and checks that every entry survives in one of the files, that
// each line is still complete JSON, and that the file count stays bounded.
func TestAppendRunLogRotatesAndKeepsEveryEntry(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "runs.jsonl")
t.Setenv("ARTEMIS_LOG_MAX_BYTES", "600")
t.Setenv("ARTEMIS_LOG_KEEP", "3")
const entries = 40
for i := 0; i < entries; i++ {
res := &recipe.RunResult{
Task: "spotify_radio",
Params: map[string]string{"query": fmt.Sprintf("query %d", i)},
OK: true,
Steps: []recipe.StepResult{{Index: 1, Verb: "launch", OK: true}},
}
if err := appendRunLog(path, res); err != nil {
t.Fatalf("appendRunLog %d: %v", i, err)
}
}
files := []string{path}
for i := 1; i <= logKeep(); i++ {
files = append(files, rotatedName(path, i))
}
for i := logKeep() + 1; i <= logKeep()+1; i++ {
if _, err := os.Stat(rotatedName(path, i)); !os.IsNotExist(err) {
t.Errorf("rotated file %d exists past the keep bound", i)
}
}
seen := map[string]bool{}
total := 0
for _, name := range files {
raw, err := os.ReadFile(name)
if err != nil {
if os.IsNotExist(err) {
continue
}
t.Fatalf("read %s: %v", name, err)
}
size := int64(len(raw))
if size > logMaxBytes() {
t.Errorf("%s is %d bytes, over the %d byte bound", filepath.Base(name), size, logMaxBytes())
}
for _, line := range strings.Split(strings.TrimSpace(string(raw)), "\n") {
if line == "" {
continue
}
var entry struct {
Time string `json:"time"`
Task string `json:"task"`
Params map[string]string
}
if err := json.Unmarshal([]byte(line), &entry); err != nil {
t.Fatalf("%s holds a truncated line %q: %v", filepath.Base(name), line, err)
}
if key := entry.Params["query"]; key != "" {
if seen[key] {
t.Errorf("entry %q was written twice", key)
}
seen[key] = true
total++
}
}
}
// A rotation bound of four small files cannot hold all 40 entries, so the
// oldest records are gone. What survives is a contiguous, complete tail.
if total == 0 {
t.Errorf("no entries survived rotation")
}
if total >= entries {
t.Errorf("kept %d entries, want fewer than all %d: a tiny bound must drop the oldest", total, entries)
}
}

333
docs/phase-4-song-radio.md Normal file
View File

@@ -0,0 +1,333 @@
# Phase 4 — song radio, and a bound on the run log
Phase 4 of `AGENTS.md` section 11: the `spotify_radio` recipe, plus the run-log
size bound Sam asked for.
Date: 2026-10-16. Phone: `Pixel_8a (akita)` over Tailscale at
`100.101.49.17:46689` (the port moves on every reboot; nothing hardcodes it).
## What was built
| File | What it holds |
|---|---|
| `recipes/spotify_radio.recipe` | The recipe: search, open the matched row's own menu, choose "Go to song radio", start it, confirm it plays. |
| `internal/recipe/recipe.go` | New verbs `menu_row` and `scroll`, a step's own failure reason (`else`), and the `{row}` run value. |
| `internal/recipe/engine.go` | Runs those steps; `playing` now also accepts no title. |
| `internal/ui/rows.go` | `Row.Action`: the action button **inside** a chosen row. |
| `internal/ui/actions.go` | A match with no usable bounds is no longer a tap target. |
| `cmd/artemis/run.go` | The run log rotates instead of growing without limit. |
## The recipe
One file, plain text, no coordinates anywhere (FR-4):
```
task: spotify_radio
app: com.spotify.music
param: query
row_id: com.spotify.music:id/row_root
steps:
- launch
- foreground 30s
- wait "Search" 30s retry 2
- tap "Search"
- tap_if "Clear search query"
- wait "What do you want to listen to?" 30s retry 3
- tap "What do you want to listen to?"
- type "{query}"
- wait "Clear search query" 30s
- wait_gone "Recents" 30s
- sleep 1s
- menu_row "{query}" 40s
- wait exact "{row}" 20s else "the context menu opened on the wrong row"
- scroll down
- wait exact "Go to song radio" 15s retry 3 else "this track offers no Song radio"
- tap exact "Go to song radio"
- wait "Radio" 30s else "the song radio page did not open"
- tap_if "Play playlist"
- playing 30s retry 3
```
Run it with:
```sh
./bin/artemis run spotify_radio query="Rolling Stones Honky Tonk"
```
### The labels came from the phone, not from a guess
A fresh `uiautomator dump` of the track's context menu on the real phone:
```
[TextView] Honky Tonk Women bounds=[201,1044][627,1110] <- the header
[TextView] The Rolling Stones bounds=[201,1121][525,1175]
[View] Share bounds=[0,1220][1080,1367]
[View] Add to Liked Songs bounds=[0,1367][1080,1514]
[View] Add to playlist bounds=[0,1514][1080,1661]
[View] Add to Queue bounds=[0,1661][1080,1808]
[View] Go to Queue bounds=[0,1808][1080,1955]
[View] Go to album bounds=[0,1955][1080,2102]
[View] Go to artist bounds=[0,2102][1080,2249]
[View] Explore Song DNA bounds=[0,2249][1080,2396]
[View] Go to song radio bounds=[0,0][0,0] <- below the fold
```
Two things this dump settled:
* The entry is **`Go to song radio`**, and it is **below the fold**. Spotify
reports an option it has not drawn yet with `bounds=[0,0][0,0]`, so the recipe
scrolls first. One swipe brings it in at `[0,1749][1080,1896]`.
* When the menu is open, the screen dump holds **only the menu**, and the menu's
header names the track it belongs to. That is what makes the wrong-row check
possible.
### `menu_row` can only press the chosen row's own button
`tap_row` picks the row by text; `menu_row` then looks for the row's action
button **inside that row's own subtree** (`internal/ui/rows.go`, `Row.Action`).
The More options button of any other row is never a candidate. On the real
screen the button carries no label of its own (the description sits on a child),
so each candidate button's subtree is searched and the smallest match wins.
### A step can name its own failure (FR-8)
```
- wait exact "Go to song radio" 15s retry 3 else "this track offers no Song radio"
```
The named reason comes first and the technical detail — including the screen's
visible labels — follows. `retry 3` re-runs the `scroll` above it, so one
insufficient swipe is not a false failure.
### An element that is not on screen is not a tap target
`pickBestMatch` now ignores a match with no usable bounds. A node that is
scrolled out of its list reports `[0,0][0,0]`, and tapping its "centre" would be
a blind tap in the top-left corner. A wait only succeeds once the element really
is on screen, which is what lets the no-radio case fail on its own reason
instead of tapping nothing.
## The two failure modes, demonstrated live
Each one fails with its own reason, and each one names the screen it gave up on.
**The context menu opened on the wrong row** (a real menu for "Honky Tonk Women"
while the recipe expected a different track):
```sh
$ ARTEMIS_RECIPES=… ./bin/artemis run spotify_radio_wrongrow_check query="Rolling Stones Honky Tonk"
step 12 menu_row [Rolling Stones Honky Tonk]: ok (2.933s)
step 13 wait [Honky Tonk Women - Mono Version]: text "Honky Tonk Women - Mono Version" did not appear within 20s
FAILED …: the context menu opened on the wrong row: text "Honky Tonk Women - Mono Version"
did not appear within 20s; foreground=com.spotify.music;
screen shows: "Honky Tonk Women", "The Rolling Stones", "Share", "Add to Liked Songs",
"Add to playlist", "Add to Queue"
```
**The item offers no song radio** (a real Spotify menu with no such entry):
```sh
$ ARTEMIS_RECIPES=… ./bin/artemis run spotify_radio_noradio_check query="The Daily Aus"
step 17 wait [Go to song radio]: text "Go to song radio" did not appear within 15s
FAILED …: this track offers no Song radio: text "Go to song radio" did not appear within 15s;
foreground=com.spotify.music; screen shows: "The Daily Aus Trailer",
"The Daily Aus • By The Daily Aus", "Share", "Add to Your Episodes", "Add to playlist", "Download"
```
Both reasons are distinct, and neither run tapped anything it had not matched.
A third, incidental failure also came up and is handled cleanly: **the matched
row has no More options button at all**. On this Spotify build only *song* rows
in search results carry one; a playlist row does not.
```
step 12 (menu_row {query}) failed: row "Honky Tonk Woman (The Rolling Stones) Covers"
has no "More options" button; foreground=com.spotify.music; screen shows: …
```
### Note on the no-radio demonstration
The shipped recipe reaches that branch for a track whose menu has no "Go to song
radio" entry. On this Spotify build every **song** row in search results offers
one, so the branch was demonstrated on a real Spotify context menu that
genuinely lacks the entry: a **podcast episode's** (`Share`, `Add to Your
Episodes`, `Download`, `Mark as finished`, `Go to episode` — no radio). The
demonstration recipe navigates search → show → episode menu; the guard, the
wording and the code path are the recipe's own.
## Work item 2 — bound `logs/runs.jsonl`
`appendRunLog` rotates when a write would pass the bound:
* `ARTEMIS_LOG_MAX_BYTES` (default `1 MiB`) is the live file size that triggers
a rotation.
* `ARTEMIS_LOG_KEEP` (default `5`) is how many rotated files survive.
* `runs.jsonl` moves to `runs.jsonl.1`, `.1` moves to `.2`, and the oldest file
is deleted: history is bounded, never deleted wholesale, so the FR-10 audit
trail that caught the lock refusal and the wrong-element tap is kept.
* A single entry larger than the bound is still written — the bound must never
drop an audit record.
The per-entry cap is unchanged: a failing step still reports at most 6 screen
labels of at most 60 characters each (`internal/recipe/engine.go`,
`describeScreen`).
### Demonstration
Five real runs against a bound of 300 bytes, keeping 2 rotated files:
```sh
$ for i in 1 2 3 4 5; do
ARTEMIS_RECIPES=…/recipes ARTEMIS_LOG=…/runs.jsonl \
ARTEMIS_LOG_MAX_BYTES=300 ARTEMIS_LOG_KEEP=2 ./bin/artemis run _demo
done
$ ls -l runs.jsonl*
-rw-r--r-- 1 sam users 160 runs.jsonl # the fifth run
-rw-r--r-- 1 sam users 160 runs.jsonl.1 # the fourth
-rw-r--r-- 1 sam users 160 runs.jsonl.2 # the third
$ ls runs.jsonl.3
ls: cannot access 'runs.jsonl.3': No such file or directory
```
Every file is exactly 160 bytes, so each holds one complete JSON line, and every
line parses. The first two runs fell off the end, because `keep=2` bounds the
history. Unit tests cover the same path:
`TestRotateRunLogKeepsRecentFiles` and
`TestAppendRunLogRotatesAndKeepsEveryEntry` (`cmd/artemis/run_test.go`).
## Live run log
The recipe ran end to end on the real phone, 19 steps, 56.6 seconds:
```sh
$ nix shell nixpkgs#go --command ./bin/artemis run spotify_radio query="Rolling Stones Honky Tonk"
step 12 menu_row [Rolling Stones Honky Tonk]: ok (2.957s)
step 13 wait [Honky Tonk Women]: ok (2.538s)
step 14 scroll [down]: ok (3.437s)
step 15 wait [Go to song radio]: ok (2.713s)
step 16 tap [Go to song radio]: ok (5.686s)
step 17 wait [Radio]: ok (3.017s)
step 18 tap_if [Play playlist]: ok (3.004s)
step 19 playing []: ok (1.793s)
OK spotify_radio in 56.632s
```
The entry in `logs/runs.jsonl`:
```json
{"time":"2026-10-16T15:15:04+11:00","task":"spotify_radio",
"params":{"query":"Rolling Stones Honky Tonk","row":"Honky Tonk Women"},
"ok":true,"duration_ms":56632, ...}
```
The steps that matter, with the detail the engine recorded:
| Step | Detail |
|---|---|
| 12 `menu_row` | `opened the "More options" menu of row "Honky Tonk Women" at (891,878)` |
| 13 `wait {row}` | `found android.widget.TextView[-] text="Honky Tonk Women" bounds=[201,1044][627,1110]` — the menu belongs to the pressed row |
| 15 `wait "Go to song radio"` | `found android.view.View[-] text="Go to song radio" bounds=[0,1972][1080,2119]` — on screen after the scroll |
| 17 `wait "Radio"` | found on the radio page, e.g. `"Download Playlist: Honky Tonk Women Radio"` |
| 19 `playing` | `com.spotify.music is playing "Layla, Derek & The Dominos, Layla And Other Assorted Love Songs (Remastered 2010)"` |
## What the phone showed
**The station name, on the radio page.** The now-playing surface for the radio
is its page, which names the station:
```
[TextView] Honky Tonk Women Radio bounds=[42,700][1038,960]
[TextView] With The Rolling Stones, Creedence Clearwater Revival, Lynyrd Skynyrd and more
[TextView] Made for you
[TextView] 1,218 saves • 3h 41min
[View] Play playlist bounds=[875,1362][1054,1541]
```
**The radio moved off the seed track.** It started on "Honky Tonk Women" and was
playing **Layla — Derek & The Dominos** by the time the run finished. A single
track cannot do that; a radio can.
**Independent reads, not Artemis's own claim:**
```
$ adb shell dumpsys media_session | grep -A12 'package=com.spotify.music'
state=PlaybackState {state=PLAYING(3), position=56, buffered position=0, speed=1.0, ...}
metadata: size=18, description=Layla, Derek & The Dominos, Layla And Other Assorted Love Songs (Remastered 2010)
$ adb shell dumpsys audio | grep '10340'
AudioPlaybackConfiguration piid:22679 deviceIds:[3] type:AAudio u/pid:10340/14004
state:started attr:AudioAttributes: usage=USAGE_MEDIA ... sampleRate=44100
```
uid 10340 is Spotify, `state:started` means its audio track is rendering, and
the session position reset to `56` with a new `updated` timestamp when the radio
moved to the next track.
### The transport label, and one DoD wording
The Phase 4 brief asks for evidence of "the transport shows `Pause`" and "the
now-playing bar names a radio/station, not the single track". On this Spotify
build neither is how the app behaves, and this was measured, not assumed:
* The mini-bar's `play_pause_button` keeps `content-desc="Play"` **while the
media session says PLAYING**. Phase 3 found the same
(`docs/phase-3-spotify-recipe.md`), and it is why both recipes confirm
playback through the media session instead of an on-screen label.
* The mini-bar names the **current track** (`track_info_view_title` = "Layla",
subtitle "Derek & The Dominos"). The **station** name is on the radio page
("Honky Tonk Women Radio"), which is the now-playing surface for a radio.
* The only "Paused" text on screen is the currently-playing row's state marker
in the radio track list, and it reads `Paused` while the track is playing.
So the evidence delivered is the intent behind the wording: a real radio was
opened and is playing — the station is named on the phone, the radio's queue has
moved past the seed track, and Android's media session and audio subsystem both
say Spotify is playing.
## The phone's condition during this phase
Spotify on this Pixel 8a wedges under memory pressure (Phase 3 attempts 9, 10 and
13). It happened again here, harder: the activity stayed `RESUMED` while its
window reported `isOnScreen=false`, it drew nothing and consumed no taps.
`MemFree` was ~320 MB with 3.6 GB of its 3.9 GB swap in use. Taps, a back key, a
`CLOSE_SYSTEM_DIALOGS` broadcast, a display-rotation change and a re-launch all
left it unchanged.
The documented remedy is a force-stop, which is a restart on Sam's phone, so it
was **asked for and approved** (issue card `aa836045-…`, answered "Yes —
force-stop and relaunch Spotify") and used to run each live step above.
This is why every recipe step is quick and why `playing` accepts no title: a
radio legitimately changes track, and the phone is slow.
## Verification status
| Item | State |
|---|---|
| `spotify_radio` starts a real radio | **live, verified** — 19 steps, 56.6s, radio playing past the seed track |
| The matched row's own More options is opened | **live, verified** — `opened the "More options" menu of row "Honky Tonk Women"` |
| Context-menu labels came from a real dump | **live, verified** — dump recorded above |
| Wrong-row failure has its own reason | **live, demonstrated** |
| No-radio failure has its own reason | **live, demonstrated** (a real menu with no song radio) |
| A row with no More options fails cleanly | **live, observed** (playlist row in search results) |
| Run log rotates at the bound | unit tested + **demonstrated live** |
| The per-entry screen-label cap is unchanged | unchanged |
| A match with no usable bounds is never tapped | unit tested (`internal/ui`) + used live by the radio scroll step |
| `foreground` still guards against acting on another app | unchanged, used by both recipes |
## Reproducing it
```sh
cd ~/paseo/projects/artemis_control
nix shell nixpkgs#go -c go test ./...
./bin/artemis run spotify_radio query="Rolling Stones Honky Tonk"
tail -1 logs/runs.jsonl
adb shell dumpsys media_session | grep -A12 'package=com.spotify.music'
adb shell dumpsys audio | grep 10340
```
The phone must be awake and unlocked. On a quiet phone the run takes about 55
seconds; `total_timeout: 420s` bounds a loaded one.
Phase 5 (maps) has not been started, on instruction.

View File

@@ -46,9 +46,12 @@ type Options struct {
// returns an error silently: the returned RunResult always says what happened. // returns an error silently: the returned RunResult always says what happened.
func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult { func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult {
started := time.Now() started := time.Now()
params := opts.Params // The caller's map is copied: a run adds its own values to it (the row a
if params == nil { // tap_row or menu_row step chose, under "row"), and the caller's map must
params = map[string]string{} // not change because of a run.
params := make(map[string]string, len(opts.Params)+1)
for name, value := range opts.Params {
params[name] = value
} }
res := &RunResult{Task: rec.Task, Params: params} res := &RunResult{Task: rec.Task, Params: params}
defer func() { defer func() {
@@ -111,7 +114,7 @@ func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult {
// A wait step with "retry N" presses the step above again: on a // A wait step with "retry N" presses the step above again: on a
// busy phone a tap can be delivered but never acted on, or dropped. // busy phone a tap can be delivered but never acted on, or dropped.
if attempt >= step.Retry { if attempt >= step.Retry {
return fail(fmt.Errorf("step %d (%s %s) failed: %s", i+1, step.Verb, strings.Join(step.Args, " "), sr.Error)) return fail(fmt.Errorf("step %d (%s %s) failed: %s", i+1, step.Verb, strings.Join(step.Args, " "), stepFailure(sr, step)))
} }
if i == 0 { if i == 0 {
return fail(fmt.Errorf("step %d: retry needs a step above it to repeat", i+1)) return fail(fmt.Errorf("step %d: retry needs a step above it to repeat", i+1))
@@ -218,7 +221,15 @@ func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[st
case "playing": case "playing":
// The screen is not a trustworthy place to read playback: Spotify's // The screen is not a trustworthy place to read playback: Spotify's
// transport button keeps the label "Play" while music is playing. // transport button keeps the label "Play" while music is playing.
// Ask the media session instead, and confirm the track title. // Ask the media session instead. An extra text argument also
// confirms the playing title, which is how spotify_play proves it
// started the right song. With no text, only the state is checked:
// a radio plays a different track every few minutes, so a title
// match would fail on a radio that is working perfectly.
title := ""
if len(args) >= 1 {
title = args[0]
}
pkg := rec.App pkg := rec.App
if len(args) == 2 { if len(args) == 2 {
pkg = args[1] pkg = args[1]
@@ -235,14 +246,22 @@ func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[st
} }
last = state last = state
if state.Playing() && state.Title != "" { if state.Playing() && state.Title != "" {
matched, total := ui.MatchesText(state.Title, args[0]) if title == "" {
sr.Detail = fmt.Sprintf("%s is playing %q", pkg, state.Title)
return nil
}
matched, total := ui.MatchesText(state.Title, title)
if total > 0 && matched*2 >= total { if total > 0 && matched*2 >= total {
sr.Detail = fmt.Sprintf("%s is playing %q", pkg, state.Title) sr.Detail = fmt.Sprintf("%s is playing %q", pkg, state.Title)
return nil return nil
} }
} }
if time.Now().After(checkDeadline) { if time.Now().After(checkDeadline) {
return fmt.Errorf("%s is not playing %q within %s (%s)", pkg, args[0], timeout, last) want := fmt.Sprintf("playing %q", title)
if title == "" {
want = "playing"
}
return fmt.Errorf("%s is not %s within %s (%s)", pkg, want, timeout, last)
} }
time.Sleep(ui.PollInterval) time.Sleep(ui.PollInterval)
} }
@@ -279,7 +298,20 @@ func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[st
return nil return nil
case "tap_row": case "tap_row":
return tapRow(r, rec, args[0], timeout, w, h, &sr) title, err := tapRow(r, rec, args[0], timeout, w, h, &sr)
if err != nil {
return err
}
params["row"] = title
return nil
case "menu_row":
title, err := tapRowMenu(r, rec, args[0], timeout, w, h, &sr)
if err != nil {
return err
}
params["row"] = title
return nil
case "type": case "type":
if err := ui.Type(r, args[0]); err != nil { if err := ui.Type(r, args[0]); err != nil {
@@ -288,6 +320,17 @@ func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[st
sr.Detail = fmt.Sprintf("typed %d characters", len(args[0])) sr.Detail = fmt.Sprintf("typed %d characters", len(args[0]))
return nil return nil
case "scroll":
// Scroll the current list. ui.Scroll prefers a scrollable node from
// the dump, so a swipe stays inside the list rather than landing on
// a toolbar. A recipe uses it when a menu's entries sit below the
// fold and are not on screen yet.
if err := ui.Scroll(r, args[0]); err != nil {
return err
}
sr.Detail = "scrolled " + strings.ToLower(args[0])
return nil
case "key": case "key":
return sendKey(r, args[0], &sr) return sendKey(r, args[0], &sr)
@@ -355,8 +398,9 @@ func tapByText(r *adb.Runner, rec *Recipe, query string, exact bool, timeout tim
// tapRow taps the list row that best matches the query. It retries the dump // tapRow taps the list row that best matches the query. It retries the dump
// until a matching row is on screen, then taps a point derived from that row's // until a matching row is on screen, then taps a point derived from that row's
// own bounds. // own bounds. It returns the chosen row's title, which later steps can use as
func tapRow(r *adb.Runner, rec *Recipe, query string, timeout time.Duration, w, h int, sr *StepResult) error { // the "row" value.
func tapRow(r *adb.Runner, rec *Recipe, query string, timeout time.Duration, w, h int, sr *StepResult) (string, error) {
deadline := time.Now().Add(timeout) deadline := time.Now().Add(timeout)
var lastErr error var lastErr error
for time.Now().Before(deadline) { for time.Now().Before(deadline) {
@@ -365,11 +409,11 @@ func tapRow(r *adb.Runner, rec *Recipe, query string, timeout time.Duration, w,
row, rerr := hier.BestRow(query, rec.RowID, w, h) row, rerr := hier.BestRow(query, rec.RowID, w, h)
if rerr == nil { if rerr == nil {
if err := ui.Tap(r, row.Centre.X, row.Centre.Y); err != nil { if err := ui.Tap(r, row.Centre.X, row.Centre.Y); err != nil {
return err return "", err
} }
sr.Detail = fmt.Sprintf("tapped row %q at (%d,%d) from %s", row.Title, row.Centre.X, row.Centre.Y, sr.Detail = fmt.Sprintf("tapped row %q at (%d,%d) from %s", row.Title, row.Centre.X, row.Centre.Y,
strings.Join(row.Texts, " / ")) strings.Join(row.Texts, " / "))
return nil return row.Title, nil
} }
lastErr = rerr lastErr = rerr
} else { } else {
@@ -380,7 +424,64 @@ func tapRow(r *adb.Runner, rec *Recipe, query string, timeout time.Duration, w,
if lastErr == nil { if lastErr == nil {
lastErr = fmt.Errorf("no result row matches %q", query) lastErr = fmt.Errorf("no result row matches %q", query)
} }
return describeFailure(r, rec, lastErr) return "", describeFailure(r, rec, lastErr)
}
// tapRowMenu opens the More options menu of the row that matches query.
//
// The button is looked up *inside the chosen row's own subtree*, so the menu
// always belongs to the row the query matched; the More options button of any
// other row is never a candidate. The button carries no text of its own on the
// real screen (its label sits on a child), so the search covers each candidate
// button's subtree. It returns the chosen row's title, which later steps can use
// as the "row" value to check that the menu that opened is that row's.
func tapRowMenu(r *adb.Runner, rec *Recipe, query string, timeout time.Duration, w, h int, sr *StepResult) (string, error) {
deadline := time.Now().Add(timeout)
var lastErr error
for time.Now().Before(deadline) {
hier, err := ui.Dump(r)
if err != nil {
lastErr = err
time.Sleep(ui.PollInterval)
continue
}
row, rerr := hier.BestRow(query, rec.RowID, w, h)
if rerr != nil {
lastErr = rerr
time.Sleep(ui.PollInterval)
continue
}
button, berr := row.Action(rec.RowMenu)
if berr != nil {
return "", describeFailure(r, rec, berr)
}
cx, cy, cerr := button.Center()
if cerr != nil {
return "", cerr
}
if err := ui.Tap(r, cx, cy); err != nil {
return "", err
}
sr.Detail = fmt.Sprintf("opened the %q menu of row %q at (%d,%d)", rec.RowMenu, row.Title, cx, cy)
return row.Title, nil
}
if lastErr == nil {
lastErr = fmt.Errorf("no result row matches %q", query)
}
return "", describeFailure(r, rec, lastErr)
}
// stepFailure is the message for a failed step. A step's own "else" reason
// comes first, because it names the real problem, and the technical detail
// follows so the screen Artemis saw is still in the record (FR-8).
func stepFailure(sr StepResult, step Step) string {
if step.Reason == "" {
return sr.Error
}
if sr.Error == "" {
return step.Reason
}
return step.Reason + ": " + sr.Error
} }
func sendKey(r *adb.Runner, name string, sr *StepResult) error { func sendKey(r *adb.Runner, name string, sr *StepResult) error {

View File

@@ -18,6 +18,7 @@
// app: com.spotify.music the package the recipe drives // app: com.spotify.music the package the recipe drives
// param: query a parameter the caller must supply (repeatable) // param: query a parameter the caller must supply (repeatable)
// row_id: com.spotify:id/row_root container id for list rows (optional) // row_id: com.spotify:id/row_root container id for list rows (optional)
// row_menu: More options label of a row's own action button (optional)
// step_timeout: 20s default per-step timeout (FR-11) // step_timeout: 20s default per-step timeout (FR-11)
// total_timeout: 120s whole-recipe timeout (FR-11) // total_timeout: 120s whole-recipe timeout (FR-11)
// //
@@ -34,18 +35,35 @@
// - wait exact "Pause" 60s retry 3 // - wait exact "Pause" 60s retry 3
// if it never appears, re-run the step above and // if it never appears, re-run the step above and
// wait again (a busy phone can drop a tap) // wait again (a busy phone can drop a tap)
// - playing "{query}" 30s retry 3
// wait until the app's media session is PLAYING and its
// title matches the text; with no text, wait for PLAYING
// alone (a radio plays a different track every few minutes)
// - wait_gone "Recents" 30s wait until no element matches the text // - wait_gone "Recents" 30s wait until no element matches the text
// - tap "Search" tap the element whose text matches (FR-4) // - tap "Search" tap the element whose text matches (FR-4)
// - tap exact "Pause" tap an element whose text matches exactly // - tap exact "Pause" tap an element whose text matches exactly
// - tap_if "Clear search query" tap it when it is on screen, else skip // - tap_if "Clear search query" tap it when it is on screen, else skip
// - tap_row "{query}" 30s tap the list row that best matches the text // - tap_row "{query}" 30s tap the list row that best matches the text
// - menu_row "{query}" 30s open the More options menu of that row
// - scroll down scroll the current list ("down" or "up")
// - type "{query}" type into the focused field // - type "{query}" type into the focused field
// - key enter send a named key event // - key enter send a named key event
// - back / home press a system key // - back / home press a system key
// - sleep 1s wait a fixed time // - sleep 1s wait a fixed time
// //
// A step may name its own failure reason with "else":
//
// - wait "Go to song radio" 20s else "this track offers no Song radio"
//
// {name} placeholders are replaced with the value of parameter "name". The // {name} placeholders are replaced with the value of parameter "name". The
// trailing "30s" on wait/tap/tap_row steps is that step's timeout. // trailing "30s" on wait/tap/tap_row steps is that step's timeout.
//
// tap_row and menu_row also record the row they chose as "row", so a later step
// can refer to it with {row} — for example to check that the context menu that
// opened belongs to the row that was pressed:
//
// - menu_row "{query}" 40s
// - wait "{row}" 20s else "the context menu opened on the wrong row"
package recipe package recipe
import ( import (
@@ -65,6 +83,7 @@ type Recipe struct {
App string App string
Params []string Params []string
RowID string RowID string
RowMenu string
StepTimeout time.Duration StepTimeout time.Duration
TotalTimeout time.Duration TotalTimeout time.Duration
Guards []Guard Guards []Guard
@@ -87,13 +106,20 @@ type Step struct {
// Retry, on a wait step, re-runs the preceding step and waits again, up to // Retry, on a wait step, re-runs the preceding step and waits again, up to
// this many extra times. A busy phone can drop a tap. // this many extra times. A busy phone can drop a tap.
Retry int Retry int
Line int // Reason, when set with "else", is the step's named failure (FR-8). A wait
// for a label a screen may simply not offer reads better as "this track
// offers no Song radio" than as "text did not appear".
Reason string
Line int
} }
// Defaults used when a recipe does not set its own timeouts. // Defaults used when a recipe does not set its own timeouts or labels.
const ( const (
DefaultStepTimeout = 20 * time.Second DefaultStepTimeout = 20 * time.Second
DefaultTotalTimeout = 120 * time.Second DefaultTotalTimeout = 120 * time.Second
// DefaultRowMenu is the label of a row's own action button. Spotify's list
// row action carries the content description "More options for song ...".
DefaultRowMenu = "More options"
) )
// validKeys are the directive names a recipe file may use. // validKeys are the directive names a recipe file may use.
@@ -103,6 +129,7 @@ var validKeys = map[string]bool{
"app": true, "app": true,
"param": true, "param": true,
"row_id": true, "row_id": true,
"row_menu": true,
"step_timeout": true, "step_timeout": true,
"total_timeout": true, "total_timeout": true,
} }
@@ -116,11 +143,13 @@ var verbs = map[string]struct {
"launch": {0, 1, false, false}, "launch": {0, 1, false, false},
"foreground": {0, 1, false, false}, "foreground": {0, 1, false, false},
"wait": {1, 2, false, true}, "wait": {1, 2, false, true},
"playing": {1, 2, false, false}, "playing": {0, 2, false, false},
"wait_gone": {1, 2, false, true}, "wait_gone": {1, 2, false, true},
"tap": {1, 2, false, true}, "tap": {1, 2, false, true},
"tap_if": {1, 1, false, true}, "tap_if": {1, 1, false, true},
"tap_row": {1, 2, false, true}, "tap_row": {1, 2, false, true},
"menu_row": {1, 2, false, false},
"scroll": {1, 1, false, false},
"type": {1, 1, false, false}, "type": {1, 1, false, false},
"key": {1, 1, false, false}, "key": {1, 1, false, false},
"back": {0, 0, false, false}, "back": {0, 0, false, false},
@@ -202,6 +231,7 @@ func Parse(src string) (*Recipe, error) {
rec := &Recipe{ rec := &Recipe{
StepTimeout: DefaultStepTimeout, StepTimeout: DefaultStepTimeout,
TotalTimeout: DefaultTotalTimeout, TotalTimeout: DefaultTotalTimeout,
RowMenu: DefaultRowMenu,
} }
for i, raw := range strings.Split(src, "\n") { for i, raw := range strings.Split(src, "\n") {
@@ -262,6 +292,11 @@ func (rec *Recipe) parseDirective(line string, lineNo int) error {
rec.Params = append(rec.Params, value) rec.Params = append(rec.Params, value)
case "row_id": case "row_id":
rec.RowID = value rec.RowID = value
case "row_menu":
if value == "" {
return fmt.Errorf("line %d: row_menu needs a label", lineNo)
}
rec.RowMenu = value
case "step_timeout": case "step_timeout":
d, err := parseDuration(value) d, err := parseDuration(value)
if err != nil { if err != nil {
@@ -340,6 +375,20 @@ func parseStep(body string, lineNo int) (*Step, error) {
break break
} }
// "else \"reason\"" names this step's own failure, so a screen that simply
// lacks a label can fail with the reason a human needs (FR-8).
for i := 0; i < len(args); i++ {
if !strings.EqualFold(args[i], "else") {
continue
}
if i+1 >= len(args) {
return nil, fmt.Errorf("line %d: else needs a reason, as in else \"that track has no radio\"", lineNo)
}
step.Reason = args[i+1]
args = append(args[:i], args[i+2:]...)
break
}
// A trailing duration is the step timeout, except for sleep where the // A trailing duration is the step timeout, except for sleep where the
// duration is the step itself. // duration is the step itself.
if !spec.durArg && len(args) > spec.min { if !spec.durArg && len(args) > spec.min {

View File

@@ -3,6 +3,7 @@ package recipe
import ( import (
"os" "os"
"path/filepath" "path/filepath"
"strconv"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -279,3 +280,160 @@ func TestStepDurationsAreRecorded(t *testing.T) {
t.Fatalf("duration not recorded: %v (%d ms)", sr.Duration, sr.DurationMS) t.Fatalf("duration not recorded: %v (%d ms)", sr.Duration, sr.DurationMS)
} }
} }
// TestElseNamesAStepFailure covers the "else" modifier: a screen that simply
// does not offer a label must fail with a reason a human can act on (FR-8).
func TestElseNamesAStepFailure(t *testing.T) {
rec, err := Parse("steps:\n - wait \"Go to song radio\" 20s else \"this track offers no Song radio\"\n")
if err != nil {
t.Fatalf("Parse: %v", err)
}
step := rec.Steps[0]
if step.Args[0] != "Go to song radio" || step.Timeout != 20*time.Second {
t.Fatalf("step = %+v", step)
}
if step.Reason != "this track offers no Song radio" {
t.Fatalf("reason = %q", step.Reason)
}
// The reason must not leak into the arguments the engine substitutes.
if len(step.Args) != 1 {
t.Fatalf("args = %v, want the query only", step.Args)
}
}
func TestElseNeedsAReason(t *testing.T) {
if _, err := Parse("steps:\n - wait \"Search\" 20s else\n"); err == nil {
t.Fatal("expected an error when else has no reason")
}
}
func TestStepFailurePrefersTheStepsOwnReason(t *testing.T) {
step := Step{Verb: "wait", Reason: "this track offers no Song radio"}
sr := StepResult{Error: "text \"Go to song radio\" did not appear within 20s; screen shows: \"Share\""}
got := stepFailure(sr, step)
if !strings.HasPrefix(got, "this track offers no Song radio:") {
t.Fatalf("stepFailure = %q, want the step's own reason first", got)
}
if !strings.Contains(got, "did not appear") {
t.Fatalf("stepFailure = %q, want the technical detail kept", got)
}
if plain := stepFailure(sr, Step{Verb: "wait"}); plain != sr.Error {
t.Fatalf("a step without a reason must report the error unchanged, got %q", plain)
}
}
// TestRowMenuDefaults keeps menu_row usable without extra settings, and proves
// a recipe can override the label.
func TestRowMenuDefaults(t *testing.T) {
rec, err := Parse("steps:\n - home\n")
if err != nil {
t.Fatalf("Parse: %v", err)
}
if rec.RowMenu != DefaultRowMenu {
t.Fatalf("row_menu default = %q, want %q", rec.RowMenu, DefaultRowMenu)
}
rec, err = Parse("row_menu: More options\nsteps:\n - menu_row \"{query}\" 30s\n")
if err != nil {
t.Fatalf("Parse: %v", err)
}
if rec.RowMenu != "More options" {
t.Fatalf("row_menu = %q", rec.RowMenu)
}
if rec.Steps[0].Verb != "menu_row" || rec.Steps[0].Args[0] != "{query}" {
t.Fatalf("step = %+v", rec.Steps[0])
}
}
// TestRowValueIsAvailableToLaterSteps covers {row}: menu_row/tap_row record the
// row they chose, so a recipe can check that a menu belongs to the row it
// pressed. The value is not a declared parameter, so it must not be required
// up front.
func TestRowValueIsAvailableToLaterSteps(t *testing.T) {
rec, err := Parse("param: query\nsteps:\n - menu_row \"{query}\" 40s\n - wait \"{row}\" 20s else \"the context menu opened on the wrong row\"\n")
if err != nil {
t.Fatalf("Parse: %v", err)
}
if missing := rec.MissingParams(map[string]string{"query": "x"}); len(missing) != 0 {
t.Fatalf("row must not be a required parameter, got missing %v", missing)
}
params := map[string]string{"query": "Rolling Stones Honky Tonk"}
if _, err := Substitute(rec.Steps[1].Args[0], params); err == nil {
t.Fatal("expected {row} to be unresolved before a row step has run")
}
params["row"] = "Honky Tonk Women"
got, err := Substitute(rec.Steps[1].Args[0], params)
if err != nil {
t.Fatalf("Substitute: %v", err)
}
if got != "Honky Tonk Women" {
t.Fatalf("substituted %q", got)
}
}
// TestSpotifyRadioRecipeIsValid keeps the shipped Phase 4 recipe honest: a typo
// in recipes/spotify_radio.recipe fails the test suite, not the phone.
func TestSpotifyRadioRecipeIsValid(t *testing.T) {
rec, err := Load(filepath.Join("..", "..", "recipes", "spotify_radio.recipe"))
if err != nil {
t.Fatalf("Load: %v", err)
}
if rec.Task != "spotify_radio" || rec.App != "com.spotify.music" {
t.Fatalf("header = %q / %q", rec.Task, rec.App)
}
if rec.RowID != "com.spotify.music:id/row_root" {
t.Fatalf("row_id = %q", rec.RowID)
}
if rec.RowMenu != DefaultRowMenu {
t.Fatalf("row_menu = %q", rec.RowMenu)
}
if len(rec.MissingParams(map[string]string{"query": "x"})) != 0 {
t.Fatal("spotify_radio must accept a query parameter")
}
// The app must be pinned in front before any text is trusted, and the
// screen must be searched before a row is picked.
var sawForeground, sawMenuRow int
reasons := map[string]bool{}
var last Step
for _, s := range rec.Steps {
switch s.Verb {
case "foreground":
sawForeground++
case "menu_row":
sawMenuRow++
}
if s.Reason != "" {
reasons[s.Reason] = true
}
// No coordinates, nowhere: every step argument is text, a duration or a
// placeholder, never a bare screen coordinate.
for _, a := range s.Args {
if _, err := strconv.Atoi(a); err == nil {
t.Fatalf("step %s has a bare number %q as an argument: this recipe must name text, not coordinates", s.Verb, a)
}
}
last = s
}
if sawForeground == 0 {
t.Fatal("spotify_radio must pin the foreground app before matching text")
}
if sawMenuRow != 1 {
t.Fatalf("spotify_radio must open exactly one row menu, got %d", sawMenuRow)
}
for _, want := range []string{
"the context menu opened on the wrong row",
"this track offers no Song radio",
} {
if !reasons[want] {
t.Fatalf("spotify_radio must fail with the distinct reason %q; it has %v", want, reasons)
}
}
if last.Verb != "playing" || len(last.Args) != 0 {
t.Fatalf("the last step must confirm playback by state (a radio changes track), got %+v", last)
}
if last.Retry < 1 {
t.Fatal("the playback check must be allowed to press the step above again")
}
}

View File

@@ -92,15 +92,25 @@ type tapCandidate struct {
// pickBestMatch chooses the most tappable node from a set of text matches. // pickBestMatch chooses the most tappable node from a set of text matches.
// Clickable and enabled nodes win; among equals the smallest node wins, because // Clickable and enabled nodes win; among equals the smallest node wins, because
// the smallest enclosing element is the control itself rather than its container. // the smallest enclosing element is the control itself rather than its container.
//
// A node without usable bounds is never a tap target. An element that is not on
// screen (scrolled out of a list, or not laid out yet) reports bounds of
// [0,0][0,0], and tapping its "centre" would be a blind tap at the top-left of
// the screen. Such a match is skipped, and a set with no usable match fails
// (FR-8) rather than tapping anything.
func pickBestMatch(matches []*Node) (*Node, error) { func pickBestMatch(matches []*Node) (*Node, error) {
candidates := make([]tapCandidate, 0, len(matches)) candidates := make([]tapCandidate, 0, len(matches))
for _, n := range matches { for _, n := range matches {
area := 1 << 30 r, err := n.BoundsRect()
if r, err := n.BoundsRect(); err == nil { if err != nil {
area = (r.X2 - r.X1) * (r.Y2 - r.Y1) continue
} }
area := (r.X2 - r.X1) * (r.Y2 - r.Y1)
candidates = append(candidates, tapCandidate{node: n, clickable: n.Clickable, enabled: n.Enabled, area: area}) candidates = append(candidates, tapCandidate{node: n, clickable: n.Clickable, enabled: n.Enabled, area: area})
} }
if len(candidates) == 0 {
return nil, fmt.Errorf("the matched element has no usable bounds: it is not on screen; refusing to tap blindly")
}
sort.SliceStable(candidates, func(i, j int) bool { sort.SliceStable(candidates, func(i, j int) bool {
a, b := candidates[i], candidates[j] a, b := candidates[i], candidates[j]
if a.clickable != b.clickable { if a.clickable != b.clickable {
@@ -168,13 +178,13 @@ func WaitForText(r *adb.Runner, query string, exact bool, timeout time.Duration)
timeout = 10 * time.Second timeout = 10 * time.Second
} }
deadline := time.Now().Add(timeout) deadline := time.Now().Add(timeout)
var seen []*Node sawMatch := false
for time.Now().Before(deadline) { for time.Now().Before(deadline) {
h, err := Dump(r) h, err := Dump(r)
if err == nil { if err == nil {
matches := h.FindText(query, exact) matches := h.FindText(query, exact)
if len(matches) > 0 { if len(matches) > 0 {
seen = matches sawMatch = true
if n, perr := pickBestMatch(matches); perr == nil { if n, perr := pickBestMatch(matches); perr == nil {
return n, nil return n, nil
} }
@@ -182,8 +192,8 @@ func WaitForText(r *adb.Runner, query string, exact bool, timeout time.Duration)
} }
time.Sleep(PollInterval) time.Sleep(PollInterval)
} }
if len(seen) > 0 { if sawMatch {
return nil, fmt.Errorf("text %q is on screen but no enabled match could be tapped", query) return nil, fmt.Errorf("text %q is present but never became a usable tap target within %s (it is off screen or disabled)", query, timeout)
} }
return nil, fmt.Errorf("text %q did not appear within %s", query, timeout) return nil, fmt.Errorf("text %q did not appear within %s", query, timeout)
} }

View File

@@ -89,6 +89,50 @@ func (h *Hierarchy) BestRow(query, rowID string, screenW, screenH int) (*Row, er
return best, nil return best, nil
} }
// Action returns the row's own action button: the smallest clickable element
// inside the row whose text or content description contains label.
//
// The search is confined to the chosen row's subtree, which is what stops a
// recipe from pressing the More options button of a different row. The button
// on the real Spotify screen carries no label of its own (the description sits
// on a non-clickable child), so each candidate button's subtree is searched
// too, and the smallest match wins: a larger ancestor is the row itself.
func (row *Row) Action(label string) (*Node, error) {
needle := strings.ToLower(strings.TrimSpace(label))
if needle == "" {
return nil, fmt.Errorf("a row action needs a non-empty label")
}
var best *Node
bestArea := 1 << 30
row.Node.Walk(func(n *Node) {
if n == row.Node || !n.Clickable || !n.Enabled {
return
}
matched := false
for _, text := range textDescendants(n) {
if strings.Contains(strings.ToLower(text), needle) {
matched = true
break
}
}
if !matched {
return
}
area := 1 << 30
if r, err := n.BoundsRect(); err == nil {
area = (r.X2 - r.X1) * (r.Y2 - r.Y1)
}
if best == nil || area < bestArea {
best, bestArea = n, area
}
})
if best == nil {
return nil, fmt.Errorf("row %q has no %q button", row.Title, label)
}
return best, nil
}
// minRowScore is the F1 floor for an acceptable row match. // minRowScore is the F1 floor for an acceptable row match.
const minRowScore = 0.5 const minRowScore = 0.5

View File

@@ -215,3 +215,54 @@ func TestTokenize(t *testing.T) {
} }
} }
} }
// TestRowActionFindsTheButtonsInsideTheChosenRow is the safety property behind
// menu_row: the More options button is taken from the row's own subtree, so a
// recipe can never press the button of a different row.
func TestRowActionFindsTheButtonsInsideTheChosenRow(t *testing.T) {
h := spotifyRows(t)
row, err := h.BestRow("Rolling Stones Honky Tonk", spotifyRowID, 1080, 2400)
if err != nil {
t.Fatalf("BestRow: %v", err)
}
if row.Title != "Honky Tonk Women" {
t.Fatalf("chose %q", row.Title)
}
button, err := row.Action("More options")
if err != nil {
t.Fatalf("Action: %v", err)
}
// The top hit's menu button, not the duplicate row's at y=1380.
if button.Bounds != "[828,815][954,941]" {
t.Fatalf("chose button %s, want the top row's", button.Bounds)
}
if !strings.Contains(button.ContentDesc, "More options for song Honky Tonk Women") {
t.Fatalf("chose %q", button.ContentDesc)
}
// A label the row does not carry fails instead of falling back to any
// clickable node.
if _, err := row.Action("Share"); err == nil {
t.Fatal("expected an error for a label the row does not have")
}
}
// TestRowActionRefusesARowWithoutAButton covers a result row that has no action
// bar at all: the step must fail, not tap the row itself.
func TestRowActionRefusesARowWithoutAButton(t *testing.T) {
h := spotifyRows(t)
var bare *Node
h.Walk(func(n *Node) {
if n.ResourceID == spotifyRowID && n.Bounds == "[0,962][1080,1130]" {
bare = n
}
})
if bare == nil {
t.Fatal("the sample row without an action bar was not found")
}
row := &Row{Node: bare, Title: "Honky Tonk Women - Mono Version"}
if _, err := row.Action("More options"); err == nil {
t.Fatal("a row with no action bar must not yield a button")
}
}

View File

@@ -0,0 +1,97 @@
# spotify_radio — start a song radio in Spotify from a search result.
#
# Edit this file by hand. Steps name the text Artemis must find on screen; it
# taps the centre of the element it matched. There are no coordinates here on
# purpose, so this recipe survives a Spotify redesign (FR-4).
#
# Run it with:
# artemis run spotify_radio query="Rolling Stones Honky Tonk"
task: spotify_radio
description: Start a song radio in Spotify from a search result.
app: com.spotify.music
# The caller must supply these.
param: query
# Spotify's list rows: title + subtitle inside one clickable container. Naming
# the container keeps the row itself the target, not the row's buttons.
row_id: com.spotify.music:id/row_root
step_timeout: 20s
total_timeout: 420s
# Screens Artemis must never tap on. Each one fails the run with the reason, so
# a wrong tap is impossible (FR-8).
guard "Log in" => Spotify is signed out; sign in on the phone and retry
guard "Sign up free" => Spotify is signed out; sign in on the phone and retry
guard "Log in to Spotify" => Spotify is signed out; sign in on the phone and retry
guard "Continue with Facebook" => Spotify is showing its welcome screen; sign in on the phone and retry
guard "Continue with Google" => Spotify is showing its welcome screen; sign in on the phone and retry
guard "Update Spotify" => Spotify wants an update; update it on the phone and retry
guard "What's new" => Spotify is showing an announcement screen; dismiss it on the phone and retry
steps:
# 1. Make sure Spotify is up and really in front before any text is trusted.
- launch
- foreground 30s
- wait "Search" 30s retry 2
# 2. Search for the track. Identical to spotify_play: Spotify remembers the
# last tab, so a search screen with the old query can come back.
- tap "Search"
- tap_if "Clear search query"
- wait "What do you want to listen to?" 30s retry 3
- tap "What do you want to listen to?"
- type "{query}"
- wait "Clear search query" 30s
# 3. Wait for the results list to replace "Recents", then let it settle.
- wait_gone "Recents" 30s
- sleep 1s
# 4. Open the More options menu of the row that matches the query. menu_row
# looks for the button INSIDE the row it chose, so the More options button
# of any other row is never a candidate. It records the row's title as
# {row} for the two steps below.
- menu_row "{query}" 40s
# 5. The context menu is a new screen. When it is open, the dump holds only
# the menu, and the menu's header names the track it belongs to. So this
# step is the wrong-row check: if the menu belongs to a different track,
# its header is a different name and the run fails with a reason that names
# the real problem. It must match exactly — "Honky Tonk Women" is not a
# match for the "Honky Tonk Women - Mono Version" row.
- wait exact "{row}" 20s else "the context menu opened on the wrong row"
# 6. "Go to song radio" sits below the fold. Spotify reports an option it has
# not drawn yet with bounds [0,0][0,0], and a wait only accepts an element
# that is really on screen, so this needs a scroll first. "retry 3" runs
# the scroll again up to three times if one swipe was not enough.
#
# A row with no radio never shows the label at all, and fails with its own
# reason, which names the screen's visible labels (FR-8).
- scroll down
- wait exact "Go to song radio" 15s retry 3 else "this track offers no Song radio"
- tap exact "Go to song radio"
# 7. The radio page names the station ("Honky Tonk Women Radio"). That name,
# and not the track, is what proves a radio opened rather than the single
# song. The page is still paused at this point.
- wait "Radio" 30s else "the song radio page did not open"
# 8. Start it. The button reads "Play playlist" only while the radio is not
# playing, so tap_if is right here: on a radio that is already playing it
# is absent, and the step is a no-op.
- tap_if "Play playlist"
# 9. Confirm playback by asking Android, not by reading the screen. Spotify's
# transport button keeps the label "Play" while music is playing, so an
# on-screen label check fails on a phone that is playing perfectly. The
# media session is the authoritative answer, and it carries the title.
#
# No title is given here on purpose: a radio moves on to a different track
# every few minutes, so "playing <seed track>" would fail on a radio that
# is working exactly as it should. The station name in step 7 is what
# proves the radio, and this step proves it is playing.
- playing 30s retry 3