feat: bounded retry for a dropped tap, and poll the screen more gently

Running spotify_play against the real phone showed a tap can be delivered
to the app and then never acted on (the dispatcher lists the Spotify window
as the touch target, and the screen does not move). The run log also showed
step durations were printed as 0s while the recorded ones were correct.

- wait steps accept "retry N": on timeout, re-run the step above and wait
  again, up to N extra times. Only a wait may use it, it needs a step above
  it, and it can only press something already matched by text. spotify_play
  uses it on the search box and on the playback check.
- The success log line printed the duration before the deferred assignment
  ran. It now records the duration first.
- uiautomator dump starts a JVM on the phone, and the wait helpers polled
  every 400ms. ui.PollInterval is now 1500ms, which cuts that load on a
  phone that is already short of memory.

docs/phase-3-spotify-recipe.md logs attempts 7-10: the recipe now reaches
tap_row and picks "Honky Tonk Women" at (540,878), but the app stops acting
on input on this loaded phone, so the end-to-end run is still outstanding.
This commit is contained in:
2026-10-16 14:12:01 +11:00
parent 767844cacc
commit ab69f37aa4
6 changed files with 175 additions and 23 deletions

View File

@@ -54,7 +54,8 @@ Run it with:
```
The verbs are `launch`, `foreground`, `wait`, `wait_gone`, `tap`, `tap_if`,
`tap_row`, `type`, `key`, `back`, `home` and `sleep`. `tap exact "Pause"`
`tap_row`, `type`, `key`, `back`, `home` and `sleep`. A `wait` step may add
`retry N` (see below). `tap exact "Pause"`
matches the whole text, not a substring. `tap_if` taps only when the text is
already on screen, which lets one recipe cope with an app that remembers its
last screen. `foreground` waits until the recipe's app is really in front, so a
@@ -114,7 +115,7 @@ its own reason instead of a generic "element not found".
## Timeouts (FR-11)
`step_timeout: 20s` is the default per step, `total_timeout: 150s` bounds the
`step_timeout: 20s` is the default per step, `total_timeout: 420s` bounds the
whole run. A step may name its own timeout (`wait "Search" 30s`). The engine
checks the remaining budget before each step and stops when it runs out. Every
`adb` call is already bounded by `adb.DefaultTimeout`.
@@ -192,8 +193,23 @@ keyguard; reading the state first would say "unlocked" about a locked phone.
| All acting commands refuse on a locked phone | **live, verified** (above) |
| `artemis run spotify_play` refuses on a locked phone | **live, verified** (above) |
| `spotify_play` plays the right song | **not yet run end to end** — see below |
| `spotify_play` picks the right row on the real results list | **live, verified** — `tapped row "Honky Tonk Women" at (540,878)` |
| Spotify search reaches the result list | live (Home → Search → type → results) |
## Retrying a dropped tap
A step can ask for the step above it to be pressed again:
```
- tap_row "{query}" 40s
- wait exact "Pause" 45s retry 3
```
`retry N` is only allowed on a `wait` step, needs a step above it to repeat, and
never taps anything that was not matched by text. It exists because a busy phone
can deliver a tap and then not act on it. The retry only re-runs the step above,
so nothing else can be pressed by accident.
## Live run log
Six attempts, in order. Each failure was clean, named its screen and tapped
@@ -220,12 +236,46 @@ nothing random — which is the point of the phase.
screen shows: "Cancel", "App logo", "Settings", "Verify that it's you" ...`.
Artemis did not touch the prompt, and it must not: it needs Sam's fingerprint
or PIN, exactly like the lock screen.
6. **Still blocked** by the same prompt on the next attempt.
6. **Same prompt again**, until Sam cleared it.
7. **Through to the row, but no playback.** After the prompt was cleared the
recipe reached step 11 (`tap_row`) with every earlier step ok, and the run
log shows it chose exactly the right element:
`tapped row "Honky Tonk Women" at (540,878) from Honky Tonk Women /
Song • The Rolling Stones`. Only the last step failed:
`text "Pause" did not appear within 40s`, with the search results still on
screen. The tap was delivered (the dispatcher lists the Spotify window as the
touch target) and Spotify did nothing with it.
8. **Duration logging fixed, retry added.** `wait ... retry N` now re-runs the
step above and waits again. Steps 9–12 made four attempts each.
9. **A dropped tap is not the whole story.** With retries, `tap_if "Clear
search query"` reported ok four times and never cleared the box. A probe
showed typing no longer reached the app either (`input text "zz"` left the
field unchanged) and neither did taps on the tab bar, while `keyevent 3`
(HOME) still worked. The app was deaf to input while still drawing.
10. **It recovers on its own, then wedges again.** After a 75-second rest with
no dumps and no input, the screen had moved to Home — queued input had been
processed — so a fresh run was started immediately. It failed at
`tap "Search"`: the tap reported ok, the screen never left Home.
### Diagnosis
This is the phone, not the recipe. Spotify's UI thread is starved: the phone is
at `MemFree ~300MB` with 3.2GB of swap in use, load average ~4, and Spotify
sustains ~70% CPU. When it is in that state it accepts no input at all — not
injected taps, not injected keys — while `keyevent 3` (HOME, handled by the
system) still works, and it recovers only after minutes of quiet. A force-stop
would clear it, and that is a restart on Sam's phone, so it is not something
Artemis does on its own.
Artemis made this worse than it needed to be: `uiautomator dump` starts a JVM on
the phone, and the wait helpers were polling every 400ms. `ui.PollInterval` is
now 1500ms, which cuts that load by roughly two thirds.
### What is left
The live end-to-end run of `spotify_play` (Phase 3 DoD) has not completed. It
needs the phone unlocked and free of the biometric prompt, and then:
needs the phone's Spotify restarted — swipe it away from Recents, or force-stop
it — and the phone left awake and unlocked. Then:
```sh
./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"