spotify_radio opens a track's own context menu and starts its song radio.
The labels came from a fresh uiautomator dump on the phone, not from a guess:
the entry is "Go to song radio" and it sits below the fold, so the recipe
scrolls before it matches.
Recipe format:
- menu_row "{query}" finds the row like tap_row, then presses the action
button inside that row's own subtree (ui.Row.Action), so the More options
button of any other row is never a candidate. It records the row as {row}.
- a step can name its own failure: wait ... else "reason". The named reason
comes first and the screen's labels still follow (FR-8).
- scroll up|down moves the current list.
- playing with no title waits for the media session to be PLAYING. A radio
changes track every few minutes, so a title match would fail on a working
radio; spotify_play still passes the title it wants.
ui:
- pickBestMatch ignores a match with no usable bounds. An element scrolled out
of its list reports [0,0][0,0], and tapping its "centre" would be a blind tap
in the corner. A wait now only succeeds once the element is on screen.
Run log (FR-10): appendRunLog rotates runs.jsonl when a write would pass
ARTEMIS_LOG_MAX_BYTES (default 1 MiB), keeping ARTEMIS_LOG_KEEP (default 5)
files. History is bounded, never deleted wholesale, and the per-entry cap of
6 screen labels of 60 characters is unchanged.
Live evidence in docs/phase-4-song-radio.md: ok:true in 56.6s, the menu that
opened belonged to the pressed row, the station name "Honky Tonk Women Radio"
on the phone, and the radio playing past its seed track (Layla) per the media
session and dumpsys audio. Both new failure modes were demonstrated failing
with distinct reasons. The mini-bar's transport label stays "Play" while
playing on this build, which is why playback is confirmed from the media
session.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
517 lines
15 KiB
Go
517 lines
15 KiB
Go
// Package recipe loads and runs named, deterministic, hand-editable tasks.
|
|
//
|
|
// A recipe is one plain-text file in recipes/, written so that Sam can change
|
|
// it by hand without a build step. It names the app, the parameters it needs,
|
|
// the screens it must never tap on, and the ordered steps that do the work.
|
|
// Steps talk about text ("tap the element whose text is Search"), never about
|
|
// coordinates, so a recipe never taps blindly (FR-4, FR-5, FR-8).
|
|
//
|
|
// # File format
|
|
//
|
|
// A recipe file is a list of lines. Blank lines and lines starting with "#"
|
|
// are ignored.
|
|
//
|
|
// Directives are "key: value" lines placed before the steps:
|
|
//
|
|
// task: spotify_play unique name, must match the file name
|
|
// description: ... one line, for humans
|
|
// app: com.spotify.music the package the recipe drives
|
|
// param: query a parameter the caller must supply (repeatable)
|
|
// row_id: com.spotify:id/row_root container id for list rows (optional)
|
|
// row_menu: More options label of a row's own action button (optional)
|
|
// step_timeout: 20s default per-step timeout (FR-11)
|
|
// total_timeout: 120s whole-recipe timeout (FR-11)
|
|
//
|
|
// A guard names a screen Artemis must never act on. Guards are checked when a
|
|
// step cannot find its target, so the failure names the real problem:
|
|
//
|
|
// guard "Log in" => Spotify is signed out; sign in on the phone and retry
|
|
//
|
|
// Steps are "- " lines, run in order:
|
|
//
|
|
// - launch start the recipe's app
|
|
// - foreground 30s wait until the recipe's app is in front
|
|
// - wait "Search" 30s wait for an element whose text matches
|
|
// - wait exact "Pause" 60s retry 3
|
|
// if it never appears, re-run the step above and
|
|
// wait again (a busy phone can drop a tap)
|
|
// - playing "{query}" 30s retry 3
|
|
// wait until the app's media session is PLAYING and its
|
|
// title matches the text; with no text, wait for PLAYING
|
|
// alone (a radio plays a different track every few minutes)
|
|
// - wait_gone "Recents" 30s wait until no element matches the text
|
|
// - tap "Search" tap the element whose text matches (FR-4)
|
|
// - tap exact "Pause" tap an element whose text matches exactly
|
|
// - tap_if "Clear search query" tap it when it is on screen, else skip
|
|
// - tap_row "{query}" 30s tap the list row that best matches the text
|
|
// - menu_row "{query}" 30s open the More options menu of that row
|
|
// - scroll down scroll the current list ("down" or "up")
|
|
// - type "{query}" type into the focused field
|
|
// - key enter send a named key event
|
|
// - back / home press a system key
|
|
// - sleep 1s wait a fixed time
|
|
//
|
|
// A step may name its own failure reason with "else":
|
|
//
|
|
// - wait "Go to song radio" 20s else "this track offers no Song radio"
|
|
//
|
|
// {name} placeholders are replaced with the value of parameter "name". The
|
|
// trailing "30s" on wait/tap/tap_row steps is that step's timeout.
|
|
//
|
|
// tap_row and menu_row also record the row they chose as "row", so a later step
|
|
// can refer to it with {row} — for example to check that the context menu that
|
|
// opened belongs to the row that was pressed:
|
|
//
|
|
// - menu_row "{query}" 40s
|
|
// - wait "{row}" 20s else "the context menu opened on the wrong row"
|
|
package recipe
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"path/filepath"
|
|
"sort"
|
|
"strconv"
|
|
"strings"
|
|
"time"
|
|
)
|
|
|
|
// Recipe is one parsed task.
|
|
type Recipe struct {
|
|
Task string
|
|
Description string
|
|
App string
|
|
Params []string
|
|
RowID string
|
|
RowMenu string
|
|
StepTimeout time.Duration
|
|
TotalTimeout time.Duration
|
|
Guards []Guard
|
|
Steps []Step
|
|
Source string
|
|
}
|
|
|
|
// Guard is a screen Artemis must never act on, with the reason to report.
|
|
type Guard struct {
|
|
Text string
|
|
Reason string
|
|
}
|
|
|
|
// Step is one ordered action.
|
|
type Step struct {
|
|
Verb string
|
|
Args []string
|
|
Exact bool
|
|
Timeout time.Duration
|
|
// Retry, on a wait step, re-runs the preceding step and waits again, up to
|
|
// this many extra times. A busy phone can drop a tap.
|
|
Retry int
|
|
// Reason, when set with "else", is the step's named failure (FR-8). A wait
|
|
// for a label a screen may simply not offer reads better as "this track
|
|
// offers no Song radio" than as "text did not appear".
|
|
Reason string
|
|
Line int
|
|
}
|
|
|
|
// Defaults used when a recipe does not set its own timeouts or labels.
|
|
const (
|
|
DefaultStepTimeout = 20 * time.Second
|
|
DefaultTotalTimeout = 120 * time.Second
|
|
// DefaultRowMenu is the label of a row's own action button. Spotify's list
|
|
// row action carries the content description "More options for song ...".
|
|
DefaultRowMenu = "More options"
|
|
)
|
|
|
|
// validKeys are the directive names a recipe file may use.
|
|
var validKeys = map[string]bool{
|
|
"task": true,
|
|
"description": true,
|
|
"app": true,
|
|
"param": true,
|
|
"row_id": true,
|
|
"row_menu": true,
|
|
"step_timeout": true,
|
|
"total_timeout": true,
|
|
}
|
|
|
|
// verbs lists the step verbs and how many arguments each takes.
|
|
var verbs = map[string]struct {
|
|
min, max int
|
|
durArg bool // the single argument is a duration
|
|
allowExact bool
|
|
}{
|
|
"launch": {0, 1, false, false},
|
|
"foreground": {0, 1, false, false},
|
|
"wait": {1, 2, false, true},
|
|
"playing": {0, 2, false, false},
|
|
"wait_gone": {1, 2, false, true},
|
|
"tap": {1, 2, false, true},
|
|
"tap_if": {1, 1, false, true},
|
|
"tap_row": {1, 2, false, true},
|
|
"menu_row": {1, 2, false, false},
|
|
"scroll": {1, 1, false, false},
|
|
"type": {1, 1, false, false},
|
|
"key": {1, 1, false, false},
|
|
"back": {0, 0, false, false},
|
|
"home": {0, 0, false, false},
|
|
"sleep": {1, 1, true, false},
|
|
}
|
|
|
|
// keyNames are the key event names a recipe may use.
|
|
var keyNames = map[string]bool{
|
|
"enter": true,
|
|
"back": true,
|
|
"home": true,
|
|
"power": true,
|
|
"wakeup": true,
|
|
"escape": true,
|
|
}
|
|
|
|
// Load reads and parses one recipe file.
|
|
func Load(path string) (*Recipe, error) {
|
|
raw, err := os.ReadFile(path)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
rec, err := Parse(string(raw))
|
|
if err != nil {
|
|
return nil, fmt.Errorf("%s: %w", path, err)
|
|
}
|
|
rec.Source = path
|
|
if rec.Task == "" {
|
|
rec.Task = strings.TrimSuffix(filepath.Base(path), filepath.Ext(path))
|
|
}
|
|
return rec, nil
|
|
}
|
|
|
|
// List loads every recipe in dir, sorted by task name.
|
|
func List(dir string) ([]*Recipe, error) {
|
|
entries, err := os.ReadDir(dir)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("read recipes directory %s: %w", dir, err)
|
|
}
|
|
var out []*Recipe
|
|
for _, e := range entries {
|
|
if e.IsDir() || strings.HasPrefix(e.Name(), ".") {
|
|
continue
|
|
}
|
|
rec, err := Load(filepath.Join(dir, e.Name()))
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
out = append(out, rec)
|
|
}
|
|
sort.Slice(out, func(i, j int) bool { return out[i].Task < out[j].Task })
|
|
return out, nil
|
|
}
|
|
|
|
// Find loads the recipe for one task name from dir.
|
|
func Find(dir, task string) (*Recipe, error) {
|
|
recipes, err := List(dir)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
for _, r := range recipes {
|
|
if r.Task == task {
|
|
return r, nil
|
|
}
|
|
}
|
|
names := make([]string, 0, len(recipes))
|
|
for _, r := range recipes {
|
|
names = append(names, r.Task)
|
|
}
|
|
if len(names) == 0 {
|
|
return nil, fmt.Errorf("no recipes in %s", dir)
|
|
}
|
|
return nil, fmt.Errorf("no recipe named %q in %s (have: %s)", task, dir, strings.Join(names, ", "))
|
|
}
|
|
|
|
// Parse turns the body of a recipe file into a Recipe.
|
|
func Parse(src string) (*Recipe, error) {
|
|
rec := &Recipe{
|
|
StepTimeout: DefaultStepTimeout,
|
|
TotalTimeout: DefaultTotalTimeout,
|
|
RowMenu: DefaultRowMenu,
|
|
}
|
|
|
|
for i, raw := range strings.Split(src, "\n") {
|
|
lineNo := i + 1
|
|
line := strings.TrimSpace(raw)
|
|
switch {
|
|
case line == "", strings.HasPrefix(line, "#"):
|
|
continue
|
|
case line == "steps:":
|
|
continue
|
|
case strings.HasPrefix(line, "-"):
|
|
step, err := parseStep(strings.TrimSpace(strings.TrimPrefix(line, "-")), lineNo)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
rec.Steps = append(rec.Steps, *step)
|
|
case strings.HasPrefix(line, "guard "):
|
|
guard, err := parseGuard(line, lineNo)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
rec.Guards = append(rec.Guards, *guard)
|
|
default:
|
|
if err := rec.parseDirective(line, lineNo); err != nil {
|
|
return nil, err
|
|
}
|
|
}
|
|
}
|
|
|
|
if len(rec.Steps) == 0 {
|
|
return nil, fmt.Errorf("recipe has no steps")
|
|
}
|
|
return rec, nil
|
|
}
|
|
|
|
func (rec *Recipe) parseDirective(line string, lineNo int) error {
|
|
key, value, ok := strings.Cut(line, ":")
|
|
if !ok {
|
|
return fmt.Errorf("line %d: expected \"key: value\", a \"guard ...\" line or a \"- step\" line, got %q", lineNo, line)
|
|
}
|
|
key = strings.ToLower(strings.TrimSpace(key))
|
|
value = strings.TrimSpace(value)
|
|
if !validKeys[key] {
|
|
return fmt.Errorf("line %d: unknown setting %q", lineNo, key)
|
|
}
|
|
|
|
switch key {
|
|
case "task":
|
|
rec.Task = value
|
|
case "description":
|
|
rec.Description = value
|
|
case "app":
|
|
rec.App = value
|
|
case "param":
|
|
if value == "" {
|
|
return fmt.Errorf("line %d: param needs a name", lineNo)
|
|
}
|
|
rec.Params = append(rec.Params, value)
|
|
case "row_id":
|
|
rec.RowID = value
|
|
case "row_menu":
|
|
if value == "" {
|
|
return fmt.Errorf("line %d: row_menu needs a label", lineNo)
|
|
}
|
|
rec.RowMenu = value
|
|
case "step_timeout":
|
|
d, err := parseDuration(value)
|
|
if err != nil {
|
|
return fmt.Errorf("line %d: step_timeout: %w", lineNo, err)
|
|
}
|
|
rec.StepTimeout = d
|
|
case "total_timeout":
|
|
d, err := parseDuration(value)
|
|
if err != nil {
|
|
return fmt.Errorf("line %d: total_timeout: %w", lineNo, err)
|
|
}
|
|
rec.TotalTimeout = d
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func parseGuard(line string, lineNo int) (*Guard, error) {
|
|
body := strings.TrimSpace(strings.TrimPrefix(line, "guard"))
|
|
text, reason, ok := strings.Cut(body, "=>")
|
|
if !ok {
|
|
return nil, fmt.Errorf("line %d: a guard line needs \"guard \\\"text\\\" => reason\"", lineNo)
|
|
}
|
|
tokens, err := tokenize(text, lineNo)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if len(tokens) != 1 {
|
|
return nil, fmt.Errorf("line %d: a guard needs exactly one quoted text, got %d", lineNo, len(tokens))
|
|
}
|
|
reason = strings.TrimSpace(reason)
|
|
if reason == "" {
|
|
return nil, fmt.Errorf("line %d: a guard needs a reason after =>", lineNo)
|
|
}
|
|
return &Guard{Text: tokens[0], Reason: reason}, nil
|
|
}
|
|
|
|
func parseStep(body string, lineNo int) (*Step, error) {
|
|
tokens, err := tokenize(body, lineNo)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if len(tokens) == 0 {
|
|
return nil, fmt.Errorf("line %d: empty step", lineNo)
|
|
}
|
|
|
|
verb := strings.ToLower(tokens[0])
|
|
spec, ok := verbs[verb]
|
|
if !ok {
|
|
return nil, fmt.Errorf("line %d: unknown step verb %q", lineNo, tokens[0])
|
|
}
|
|
step := &Step{Verb: verb, Line: lineNo}
|
|
args := tokens[1:]
|
|
|
|
if spec.allowExact && len(args) > 0 && strings.EqualFold(args[0], "exact") {
|
|
step.Exact = true
|
|
args = args[1:]
|
|
}
|
|
|
|
// "retry N" is pulled out first, so the timeout can still be the last word.
|
|
for i := 0; i < len(args); i++ {
|
|
if !strings.EqualFold(args[i], "retry") {
|
|
continue
|
|
}
|
|
if i+1 >= len(args) {
|
|
return nil, fmt.Errorf("line %d: retry needs a number, as in retry 3", lineNo)
|
|
}
|
|
n, err := strconv.Atoi(args[i+1])
|
|
if err != nil || n < 1 {
|
|
return nil, fmt.Errorf("line %d: retry needs a positive number, got %q", lineNo, args[i+1])
|
|
}
|
|
if verb != "wait" && verb != "playing" {
|
|
return nil, fmt.Errorf("line %d: retry is only allowed on a wait or playing step", lineNo)
|
|
}
|
|
step.Retry = n
|
|
args = append(args[:i], args[i+2:]...)
|
|
break
|
|
}
|
|
|
|
// "else \"reason\"" names this step's own failure, so a screen that simply
|
|
// lacks a label can fail with the reason a human needs (FR-8).
|
|
for i := 0; i < len(args); i++ {
|
|
if !strings.EqualFold(args[i], "else") {
|
|
continue
|
|
}
|
|
if i+1 >= len(args) {
|
|
return nil, fmt.Errorf("line %d: else needs a reason, as in else \"that track has no radio\"", lineNo)
|
|
}
|
|
step.Reason = args[i+1]
|
|
args = append(args[:i], args[i+2:]...)
|
|
break
|
|
}
|
|
|
|
// A trailing duration is the step timeout, except for sleep where the
|
|
// duration is the step itself.
|
|
if !spec.durArg && len(args) > spec.min {
|
|
if d, derr := parseDuration(args[len(args)-1]); derr == nil {
|
|
step.Timeout = d
|
|
args = args[:len(args)-1]
|
|
}
|
|
}
|
|
|
|
if len(args) < spec.min || len(args) > spec.max {
|
|
return nil, fmt.Errorf("line %d: %s takes %s, got %d", lineNo, verb, argRange(spec.min, spec.max), len(args))
|
|
}
|
|
step.Args = args
|
|
|
|
if spec.durArg {
|
|
d, derr := parseDuration(args[0])
|
|
if derr != nil {
|
|
return nil, fmt.Errorf("line %d: sleep: %w", lineNo, derr)
|
|
}
|
|
step.Timeout = d
|
|
}
|
|
if verb == "key" && !keyNames[strings.ToLower(args[0])] {
|
|
return nil, fmt.Errorf("line %d: unknown key %q", lineNo, args[0])
|
|
}
|
|
return step, nil
|
|
}
|
|
|
|
func argRange(min, max int) string {
|
|
switch {
|
|
case min == max:
|
|
return fmt.Sprintf("exactly %d argument(s)", min)
|
|
case min == 0:
|
|
return fmt.Sprintf("up to %d argument(s)", max)
|
|
default:
|
|
return fmt.Sprintf("%d or %d arguments", min, max)
|
|
}
|
|
}
|
|
|
|
func parseDuration(s string) (time.Duration, error) {
|
|
d, err := time.ParseDuration(strings.TrimSpace(s))
|
|
if err != nil {
|
|
return 0, fmt.Errorf("%q is not a duration (use 20s, 1500ms, 2m)", s)
|
|
}
|
|
if d <= 0 {
|
|
return 0, fmt.Errorf("duration %q must be positive", s)
|
|
}
|
|
return d, nil
|
|
}
|
|
|
|
// tokenize splits a recipe line into words, treating a double-quoted run as one
|
|
// word. Backslash escapes the next character inside quotes.
|
|
func tokenize(s string, lineNo int) ([]string, error) {
|
|
var out []string
|
|
for i := 0; i < len(s); {
|
|
switch {
|
|
case s[i] == ' ' || s[i] == '\t':
|
|
i++
|
|
case s[i] == '"':
|
|
var b strings.Builder
|
|
j := i + 1
|
|
for j < len(s) && s[j] != '"' {
|
|
if s[j] == '\\' && j+1 < len(s) {
|
|
j++
|
|
}
|
|
b.WriteByte(s[j])
|
|
j++
|
|
}
|
|
if j >= len(s) {
|
|
return nil, fmt.Errorf("line %d: unterminated quoted string", lineNo)
|
|
}
|
|
out = append(out, b.String())
|
|
i = j + 1
|
|
default:
|
|
j := i
|
|
for j < len(s) && s[j] != ' ' && s[j] != '\t' {
|
|
j++
|
|
}
|
|
out = append(out, s[i:j])
|
|
i = j
|
|
}
|
|
}
|
|
return out, nil
|
|
}
|
|
|
|
// Substitute replaces {name} placeholders with parameter values.
|
|
//
|
|
// Every declared parameter must be present, and every placeholder must be
|
|
// declared: an unset value would otherwise be typed into the phone as an empty
|
|
// string.
|
|
func Substitute(text string, params map[string]string) (string, error) {
|
|
var out strings.Builder
|
|
for i := 0; i < len(text); {
|
|
if text[i] != '{' {
|
|
out.WriteByte(text[i])
|
|
i++
|
|
continue
|
|
}
|
|
end := strings.IndexByte(text[i:], '}')
|
|
if end < 0 {
|
|
return "", fmt.Errorf("unclosed { in %q", text)
|
|
}
|
|
name := text[i+1 : i+end]
|
|
value, ok := params[name]
|
|
if !ok {
|
|
return "", fmt.Errorf("no value for parameter %q in %q", name, text)
|
|
}
|
|
if strings.TrimSpace(value) == "" {
|
|
return "", fmt.Errorf("parameter %q is empty", name)
|
|
}
|
|
out.WriteString(value)
|
|
i += end + 1
|
|
}
|
|
return out.String(), nil
|
|
}
|
|
|
|
// MissingParams returns the declared parameters that the caller did not supply.
|
|
func (rec *Recipe) MissingParams(params map[string]string) []string {
|
|
var missing []string
|
|
for _, name := range rec.Params {
|
|
if strings.TrimSpace(params[name]) == "" {
|
|
missing = append(missing, name)
|
|
}
|
|
}
|
|
return missing
|
|
}
|