sam/artemis_control
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Phase 5 — maps_navigate, and read the screen before the input

Browse Source 
The Maps recipe: open the search box, type the place, and confirm the
result list names it. Every label comes from a live uiautomator dump
(search_omnibox_text_box, typed_suggest_container, the account sheet),
never from a guess. Text only, one step_timeout/total_timeout, guards for
the account sheet.

The first live run exposed an ordering bug in expect_change. A dump is not
instant, so a baseline taken after the input already showed the app's
response and the check could never see a wedge; it failed against a
healthy Maps. The engine now reads the screen before the step above runs
and hands that signature to the check. Each retry re-reads it.

Live: maps_navigate ok in 27.6s, result list names "Melbourne Museum".
The wedge reason and its bounded retry are demonstrated with a probe
recipe whose input changes nothing; a real wedge could not be forced, and
docs/phase-5-maps-recipe.md says so plainly.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Sam Rolfe
2026-10-16 16:18:06 +11:00
parent 43cd4282d0
commit a7c07c645e
7 changed files with 420 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
README.md
View File

@@ -14,7 +14,7 @@ order.
| 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch | | 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch |
| 3 — Spotify recipe | ✅ done — `spotify_play` plays the right song end to end (13 steps, 35s), see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) | | 3 — Spotify recipe | ✅ done — `spotify_play` plays the right song end to end (13 steps, 35s), see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) |
| 4 — Song radio | ✅ done — `spotify_radio` starts a real radio (19 steps, 57s); the run log is now bounded at ~1 MB × 5, see [`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) | | 4 — Song radio | ✅ done — `spotify_radio` starts a real radio (19 steps, 57s); the run log is now bounded at ~1 MB × 5, see [`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) |
| 5 — Maps recipe | not started | | 5 — Maps recipe | ✅ done — `maps_navigate` opens a real search and the result list names the place (8 steps, 28s), see [`docs/phase-5-maps-recipe.md`](docs/phase-5-maps-recipe.md) |
| 6 — Action API | not started | | 6 — Action API | not started |
| 7 — Resilience | not started | | 7 — Resilience | not started |
| 8 — Documentation | not started | | 8 — Documentation | not started |
@@ -72,12 +72,15 @@ what to find on screen; there are no coordinates anywhere.
./bin/artemis recipes # list them ./bin/artemis recipes # list them
./bin/artemis run spotify_play query="Rolling Stones Honky Tonk" ./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
./bin/artemis run spotify_radio query="Rolling Stones Honky Tonk" ./bin/artemis run spotify_radio query="Rolling Stones Honky Tonk"
./bin/artemis run maps_navigate query="Melbourne Museum"
``` ```
Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the
file format, the verbs, and why list rows need their own step, and file format, the verbs, and why list rows need their own step,
[`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) for the context menu, [`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) for the context menu,
`menu_row`, and the two ways a radio run fails on purpose. `menu_row`, and the two ways a radio run fails on purpose, and
[`docs/phase-5-maps-recipe.md`](docs/phase-5-maps-recipe.md) for the Maps
labels, the `expect_change` check, and the one ordering rule it depends on.
## Rules that the code enforces ## Rules that the code enforces

247
docs/phase-5-maps-recipe.md Normal file
View File

@@ -0,0 +1,247 @@
# Phase 5 — the Maps recipe, and the wedged-app check
Phase 5 of `AGENTS.md` section 11: the `maps_navigate` recipe, plus the wedged-app
detection the task asked for.
Date: 2026-10-16. Phone: `Pixel_8a (akita)` over Tailscale at
`100.101.49.17:46689` (the port moves on every reboot; nothing hardcodes it).
## What was built
| File | What it holds |
|---|---|
| `recipes/maps_navigate.recipe` | The recipe: open the search box, type the place, confirm the result list names it. |
| `internal/ui/change.go` | `ScreenSignature` and `WaitForChange`: the screen fingerprint and the bounded change check. |
| `internal/recipe/recipe.go` | The new `expect_change <duration>` verb. |
| `internal/recipe/engine.go` | Runs the check, and reads the screen **before** the step above it. |
| `internal/ui/change_test.go`, `internal/recipe/recipe_test.go` | Unit tests for the fingerprint and the verb. |
## The recipe
One file, plain text, no coordinates anywhere (FR-4):
```
task: maps_navigate
app: com.google.android.apps.maps
param: query
guard "Manage your Google Account" => Maps is showing its account sheet; press back on the phone and retry
guard "Turn on Incognito mode" => Maps is showing its account sheet; press back on the phone and retry
steps:
- launch
- foreground 30s
- wait "Search here" 30s retry 2
- tap "Search here"
- expect_change 15s retry 2
- type "{query}"
- expect_change 15s retry 2
- wait "{query}" 30s else "the Maps result list did not name the place"
```
Run it with:
```sh
./bin/artemis run maps_navigate query="Melbourne Museum"
```
## The labels came from the phone, not from a guess
A fresh `uiautomator dump` of the Maps home and the search screen on the real
phone:
```
# Map home
[EditText] Search here id=search_omnibox_text_box bounds=[31,142][663,268]
[View] Voice search bounds=[663,142][789,268]
[View] Lens in Maps bounds=[789,142][915,268]
[FrameLayout] Account and settings. bounds=[915,142][1049,268]
[FrameLayout] Explore / You / Contribute / Business bounds=[0,2168][1080,2337]
# Search screen, after tapping "Search here"
[Button] Navigate up bounds=[31,142][157,268]
[EditText] Search here id=search_omnibox_text_box bounds=[157,142][921,268]
[EditText] <query> id=search_omnibox_edit_text bounds=[157,142][921,268]
[Button] Voice search bounds=[923,142][1049,268]
[ImageView] Clear bounds=[954,173][1017,236]
[RecyclerView] <results> id=typed_suggest_container bounds=[0,284][1080,1577]
# One suggestion row in that list
[TextView] 7.6 km bounds=[0,378][210,431]
[TextView] Melbourne Museum bounds=[210,296][668,366]
[TextView] Nicholson Street, Carlton VIC bounds=[210,366][823,429]
[FrameLayout] Activate to enter suggestion Melbourne Museum into search bar
bounds=[933,300][1059,426]
```
Three things this dump settled:
* The search box is the `EditText` labelled **`Search here`** (resource id
`search_omnibox_text_box`). Tapping it opens the search screen with the box
focused.
* Typing shows the results list immediately; there is no separate "press enter"
step. The list is the `RecyclerView` with id `typed_suggest_container`.
* Each suggestion row carries the place as its title **and** as the label of its
own row button: `Activate to enter suggestion <place> into search bar`. So the
final `wait "{query}"` matches a real row, not the empty box. Because that row
button is clickable and small, it is the match the run log reports.
The account sheet was opened on purpose to take its labels for the guards
(`Manage your Google Account`, `Turn on Incognito mode`). Maps on this phone is
signed in as `samuelrolfe@gmail.com`, so there is no sign-in screen in the
search path and no guard was invented for one.
## The wedged-app check (`expect_change`)
An app can stop consuming input while it keeps drawing: Android delivers the
tap, the app never acts on it (`docs/phase-3-spotify-recipe.md`, attempts 9, 10
and 13). Before Phase 5 the recipe reported that as the label it was waiting
for, which names the wrong problem.
`expect_change <duration>` reads the screen, waits a bounded time for it to
change, and fails with the distinct reason:
```
the app stopped consuming input: the screen did not change within 6s;
foreground=com.google.android.apps.maps; screen shows: "Layers", ...
```
The screen fingerprint is the foreground package plus every visible label and
its bounds, sorted so that the order uiautomator happens to report never looks
like a change (`internal/ui/change.go`).
### The baseline must be read *before* the input
The first live run of the recipe failed the check against a perfectly healthy
Maps:
```
step 4 tap [Search here]: ok (5.463s)
step 5 expect_change [15s]: the app stopped consuming input: the screen did not change within 15s
```
The tap had worked and the search screen had opened. The bug was ordering: a
`uiautomator dump` is not instant, so a baseline taken *after* the tap already
showed the app's response, and the "after" screen could never differ.
The engine now reads the screen immediately **before** the step above runs, and
hands that signature to the `expect_change` step that follows it
(`internal/recipe/engine.go`). A dump runs a JVM on the phone, so this read
happens only when a recipe asks for the check. Each `retry` re-reads the screen
before it re-presses the step above.
## The live run
```sh
$ ./bin/artemis run maps_navigate query="Melbourne Museum"
task maps_navigate (Search Google Maps for a place and show the matching result list.)
step 1 launch []: ok (616ms)
step 2 foreground []: ok (157ms)
step 3 wait [Search here]: ok (2.672s)
step 4 tap [Search here]: ok (5.647s)
step 5 expect_change [15s]: ok (3.156s)
step 6 type [Melbourne Museum]: ok (447ms)
step 7 expect_change [15s]: ok (3.077s)
step 8 wait [Melbourne Museum]: ok (2.692s)
OK maps_navigate in 27.606s
```
The run log (`logs/runs.jsonl`) records `"ok":true`, and the step that proves the
DoD reports:
```json
{"step":8,"verb":"wait","args":["{query}"],
"detail":"found android.widget.FrameLayout[-] text=\"Activate to enter suggestion Melbourne Museum into search bar\" bounds=[933,300][1059,426]",
"ok":true,"duration_ms":2692}
```
And the screen after the run, read back with a fresh dump — the result list
names the searched place:
```
$ ./bin/artemis dump
38 text nodes
[TextView] 7.6 km bounds=[0,378][210,431]
[TextView] Melbourne Museum bounds=[210,296][668,366] <- the place, first row
[TextView] Nicholson Street, Carlton VIC bounds=[210,366][823,429]
[FrameLayout] Activate to enter suggestion Melbourne Museum into search bar
[TextView] 7.6 km bounds=[0,539][210,592]
[TextView] Melbourne Museum - Museum Market Cafe bounds=[210,457][923,527]
[TextView] Nicholson Street, Carlton VIC bounds=[210,527][823,590]
[TextView] 7.5 km bounds=[0,700][210,753]
[TextView] Melbourne Museum - Museum Shop bounds=[210,618][923,688]
[TextView] Nicholson Street, Carlton VIC bounds=[210,688][823,751]
[TextView] 8.2 km bounds=[0,861][210,914]
[TextView] Museum of Chinese Australian History bounds=[210,779][923,849]
[TextView] Cohen Place, Melbourne VIC bounds=[210,849][803,912]
[TextView] 7.6 km bounds=[0,1022][210,1075]
[TextView] Children's Gallery (Melbourne Museum) bounds=[210,940][923,1010]
[TextView] Carlton VIC bounds=[210,1010][451,1073]
```
## The wedge reason, demonstrated live
A real wedged app could **not** be reproduced in Phase 5: the Phase 3 wedge came
from a memory-starved phone under load, and it cannot be forced on demand. What
was verified instead is the detection path itself, with a probe recipe (not
shipped) whose step above is a real input that Maps answers with no screen
change — an ENTER key event on the map home, where nothing has focus:
```sh
$ ARTEMIS_RECIPES=<probe dir> ./bin/artemis run maps_wedge_probe
step 4 key [enter]: ok (150ms)
step 5 expect_change [6s]: the app stopped consuming input: the screen did not change within 6s; foreground=com.google.android.apps.maps; screen shows: "Layers", "Re-centre map to your location", "Directions", "Carlton", "Mostly cloudy, 23°, Good, AQC", "23°"
FAILED maps_wedge_probe after 17.857s: ...
```
The bounded retry was verified in the same way (`expect_change 6s retry 2`):
the engine re-pressed the step above twice more and then stopped, three attempts
in total, so a wedged app ends the run instead of looping (FR-11):
```sh
step 5 expect_change [6s]: the app stopped consuming input: ...
step 5 expect_change: retrying, pressing key again
step 4 key [enter]: ok (223ms)
step 5 expect_change [6s]: the app stopped consuming input: ...
step 5 expect_change: retrying, pressing key again
step 4 key [enter]: ok (150ms)
step 5 expect_change [6s]: the app stopped consuming input: ...
FAILED maps_wedge_probe after 46.099s: ...
```
**Limit, stated plainly.** The check cannot tell a wedged app from an input that
legitimately changes nothing: it keys on "the screen did not change". A recipe
must place `expect_change` only after an input that must change the screen. The
`maps_navigate` recipe does that twice — the tap that opens the search screen,
and the typing that draws the result list.
## Verification status
| Item | State |
|---|---|
| Screen fingerprint, stability and sensitivity | unit tested (`internal/ui`) |
| `expect_change` parsing, `retry`, `else` | unit tested (`internal/recipe`) |
| All shipped recipes parse | unit tested (`internal/recipe`) |
| `maps_navigate` opens a real search and names the place | **live, verified** — 8 steps, 27.6s, result list above |
| The wedge reason fires on an input that changes nothing | **live, verified** — probe above |
| A wedged app was reproduced | **no** — could not be forced; see the limit above |
| The wedge check is bounded by `retry` | **live, verified** — 3 attempts, then stop |
## Reproducing it
```sh
cd ~/paseo/projects/artemis_control
./bin/artemis run maps_navigate query="Melbourne Museum"
tail -1 logs/runs.jsonl
./bin/artemis dump
```
The phone must be awake and unlocked (a secure keyguard is a loud refusal,
`AGENTS.md` section 13 Q2). From the map home this takes about 28 seconds.
## Not done
* The recipe stops at the result list, which is what the Phase 5 DoD asks for.
It does not tap a result to open the place, so it does not start navigation.
* Phase 6 (the action API) and Phase 7 (resilience) have not been started, on
instruction.

46
internal/recipe/engine.go
View File

@@ -99,14 +99,30 @@ func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult {
w, h, _ := ui.ScreenSize(r) // best effort: bounds checking degrades gracefully w, h, _ := ui.ScreenSize(r) // best effort: bounds checking degrades gracefully
deadline := started.Add(rec.TotalTimeout) deadline := started.Add(rec.TotalTimeout)
// beforeChange is the screen signature captured immediately before the step
// above, for a following expect_change step. The capture happens before that
// step's input runs, because an app that never acts on the input leaves the
// screen exactly as it was — and a dump taken afterwards would already show
// the app's response (AGENTS.md section 11 Phase 5).
var beforeChange string
for i := 0; i < len(rec.Steps); i++ { for i := 0; i < len(rec.Steps); i++ {
step := rec.Steps[i] step := rec.Steps[i]
if step.Verb != "expect_change" && i+1 < len(rec.Steps) && rec.Steps[i+1].Verb == "expect_change" {
// A dump runs a JVM on the phone, so this happens only when a recipe
// actually asks for the wedge check.
if sig, err := ui.ScreenSignature(r); err == nil {
beforeChange = sig
} else {
beforeChange = ""
logf(" step %d %s: could not read the screen before the wedge check: %v", i+1, step.Verb, err)
}
}
var sr StepResult var sr StepResult
for attempt := 0; ; attempt++ { for attempt := 0; ; attempt++ {
if remaining := time.Until(deadline); remaining <= 0 { if remaining := time.Until(deadline); remaining <= 0 {
return fail(fmt.Errorf("recipe timed out after %s at step %d (%s)", rec.TotalTimeout, i+1, step.Verb)) return fail(fmt.Errorf("recipe timed out after %s at step %d (%s)", rec.TotalTimeout, i+1, step.Verb))
} }
sr = executeStep(r, rec, step, i, params, w, h, deadline, logf) sr = executeStep(r, rec, step, i, params, w, h, deadline, beforeChange, logf)
res.Steps = append(res.Steps, sr) res.Steps = append(res.Steps, sr)
if sr.OK { if sr.OK {
break break
@@ -120,7 +136,16 @@ func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult {
return fail(fmt.Errorf("step %d: retry needs a step above it to repeat", i+1)) return fail(fmt.Errorf("step %d: retry needs a step above it to repeat", i+1))
} }
logf(" step %d %s: retrying, pressing %s again", i+1, step.Verb, rec.Steps[i-1].Verb) logf(" step %d %s: retrying, pressing %s again", i+1, step.Verb, rec.Steps[i-1].Verb)
again := executeStep(r, rec, rec.Steps[i-1], i-1, params, w, h, deadline, logf) if step.Verb == "expect_change" {
// Judge the re-press on its own: capture the screen again before
// the step above runs a second time.
if sig, err := ui.ScreenSignature(r); err == nil {
beforeChange = sig
} else {
beforeChange = ""
}
}
again := executeStep(r, rec, rec.Steps[i-1], i-1, params, w, h, deadline, "", logf)
res.Steps = append(res.Steps, again) res.Steps = append(res.Steps, again)
if !again.OK { if !again.OK {
// The wait is the real failure; the repeat just could not run. // The wait is the real failure; the repeat just could not run.
@@ -136,7 +161,7 @@ func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult {
} }
func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[string]string, func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[string]string,
w, h int, deadline time.Time, logf func(string, ...any)) (sr StepResult) { w, h int, deadline time.Time, beforeChange string, logf func(string, ...any)) (sr StepResult) {
started := time.Now() started := time.Now()
sr = StepResult{Index: index + 1, Verb: step.Verb, Args: step.Args} sr = StepResult{Index: index + 1, Verb: step.Verb, Args: step.Args}
@@ -274,12 +299,15 @@ func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[st
return nil return nil
case "expect_change": case "expect_change":
// The wedged-app check. An app that has stopped consuming input // The wedged-app check. beforeChange was captured before the step
// still draws its old screen, so a delivered tap or key changes // above ran. An app that has stopped consuming input still draws
// nothing. Failing here names the real problem instead of blaming // its old screen, so the signature does not move; failing here
// the label the next step would have waited for (Phase 5). // names the real problem instead of blaming the label the next
_, _, err := ui.ExpectChange(r, timeout) // step would have waited for (Phase 5).
if err != nil { if beforeChange == "" {
return fmt.Errorf("the screen was not read before the step above, so a change cannot be judged")
}
if _, err := ui.WaitForChange(r, beforeChange, timeout); err != nil {
return describeFailure(r, rec, fmt.Errorf("the app stopped consuming input: %w", err)) return describeFailure(r, rec, fmt.Errorf("the app stopped consuming input: %w", err))
} }
sr.Detail = "screen changed" sr.Detail = "screen changed"

5
internal/recipe/recipe.go
View File

@@ -40,8 +40,9 @@
// title matches the text; with no text, wait for PLAYING // title matches the text; with no text, wait for PLAYING
// alone (a radio plays a different track every few minutes) // alone (a radio plays a different track every few minutes)
// - wait_gone "Recents" 30s wait until no element matches the text // - wait_gone "Recents" 30s wait until no element matches the text
// - expect_change 10s fail unless the screen changes within the time // - expect_change 10s the screen is read before the step above, so this
// (the wedged-app check: a delivered input that the app never acts on) // fails with "the app stopped consuming input" when
// that step's input left the screen unchanged (a wedged app)
// - tap "Search" tap the element whose text matches (FR-4) // - tap "Search" tap the element whose text matches (FR-4)
// - tap exact "Pause" tap an element whose text matches exactly // - tap exact "Pause" tap an element whose text matches exactly
// - tap_if "Clear search query" tap it when it is on screen, else skip // - tap_if "Clear search query" tap it when it is on screen, else skip

52
internal/recipe/recipe_test.go
View File

@@ -178,6 +178,56 @@ func TestSpotifyRecipeIsValid(t *testing.T) {
} }
} }
// TestMapsRecipeIsValid keeps the shipped Phase 5 recipe honest: a typo in
// recipes/maps_navigate.recipe fails the test suite, not the phone.
func TestMapsRecipeIsValid(t *testing.T) {
rec, err := Load(filepath.Join("..", "..", "recipes", "maps_navigate.recipe"))
if err != nil {
t.Fatalf("Load: %v", err)
}
if rec.Task != "maps_navigate" || rec.App != "com.google.android.apps.maps" {
t.Fatalf("header = %q / %q", rec.Task, rec.App)
}
if len(rec.MissingParams(map[string]string{"query": "x"})) != 0 {
t.Fatal("maps_navigate must accept a query parameter")
}
if len(rec.Guards) == 0 {
t.Fatal("maps_navigate needs guards for screens it must not tap")
}
var sawForeground, sawChange int
for i, s := range rec.Steps {
switch s.Verb {
case "foreground":
sawForeground++
case "expect_change":
sawChange++
// The engine reads the screen before the step above, so the check
// needs a step above it.
if i == 0 {
t.Fatal("expect_change must have a step above it")
}
}
// No coordinates, nowhere: every argument is text, a duration or a
// placeholder, never a bare screen coordinate.
for _, a := range s.Args {
if _, err := strconv.Atoi(a); err == nil {
t.Fatalf("step %s has a bare number %q as an argument: this recipe must name text, not coordinates", s.Verb, a)
}
}
}
if sawForeground == 0 {
t.Fatal("maps_navigate must pin the foreground app before matching text")
}
if sawChange == 0 {
t.Fatal("maps_navigate must check that its inputs changed the screen (the wedged-app check)")
}
last := rec.Steps[len(rec.Steps)-1]
if last.Verb != "wait" || last.Args[0] != "{query}" {
t.Fatalf("the last step must confirm the result list names the place, got %+v", last)
}
}
func TestLoadAndList(t *testing.T) { func TestLoadAndList(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
write := func(name, body string) { write := func(name, body string) {
@@ -290,7 +340,7 @@ func TestForegroundStep(t *testing.T) {
func TestStepDurationsAreRecorded(t *testing.T) { func TestStepDurationsAreRecorded(t *testing.T) {
rec := &Recipe{StepTimeout: time.Second} rec := &Recipe{StepTimeout: time.Second}
step := Step{Verb: "sleep", Timeout: 50 * time.Millisecond} step := Step{Verb: "sleep", Timeout: 50 * time.Millisecond}
sr := executeStep(adb.New(""), rec, step, 2, nil, 0, 0, time.Now().Add(time.Minute), func(string, ...any) {}) sr := executeStep(adb.New(""), rec, step, 2, nil, 0, 0, time.Now().Add(time.Minute), "", func(string, ...any) {})
if !sr.OK { if !sr.OK {
t.Fatalf("sleep step failed: %s", sr.Error) t.Fatalf("sleep step failed: %s", sr.Error)
} }

37
internal/ui/change.go
View File

@@ -50,34 +50,33 @@ func (h *Hierarchy) Signature(foreground string) string {
return foreground + "\n" + strings.Join(parts, "\n") return foreground + "\n" + strings.Join(parts, "\n")
} }
// ExpectChange waits until the screen no longer looks like it did when the // WaitForChange waits until the screen no longer matches before, and returns
// function was called, and returns the signatures before and after. // the new signature.
// //
// This is the wedged-app check. An app that has stopped consuming input still // The caller captures before with ScreenSignature *before* the input it wants to
// draws the screen it had, so a delivered tap changes nothing. A timeout here // judge. That ordering is the whole check: an app that has stopped consuming
// means the last input was accepted by Android but never acted on by the app, // input still draws the screen it had, so the signature after the input is
// which the recipe reports as "the app stopped consuming input" instead of // identical to the one before it. A dump is not instant, so a baseline taken
// blaming the label it was waiting for (AGENTS.md section 11 Phase 5). // after the input would already include the app's response and could never see
func ExpectChange(r *adb.Runner, timeout time.Duration) (before, after string, err error) { // the wedge.
func WaitForChange(r *adb.Runner, before string, timeout time.Duration) (string, error) {
if strings.TrimSpace(before) == "" {
return "", fmt.Errorf("no screen signature was captured before the input")
}
if timeout <= 0 { if timeout <= 0 {
timeout = 10 * time.Second timeout = 10 * time.Second
} }
before, err = ScreenSignature(r)
if err != nil {
return "", "", err
}
deadline := time.Now().Add(timeout) deadline := time.Now().Add(timeout)
for { for {
sig, derr := ScreenSignature(r) sig, err := ScreenSignature(r)
if derr != nil { if err != nil {
return before, "", derr return "", err
} }
if sig != before { if sig != before {
return before, sig, nil return sig, nil
} }
if !time.Now().Before(deadline) { if time.Now().After(deadline) {
return before, before, fmt.Errorf("the screen did not change within %s", timeout) return before, fmt.Errorf("the screen did not change within %s", timeout)
} }
time.Sleep(PollInterval) time.Sleep(PollInterval)
} }

58
recipes/maps_navigate.recipe Normal file
View File

@@ -0,0 +1,58 @@
# maps_navigate — search Google Maps for a place.
#
# Edit this file by hand. Steps name the text Artemis must find on screen; it
# taps the centre of the element it matched. There are no coordinates here on
# purpose, so this recipe survives a Maps redesign (FR-4).
#
# Run it with:
# artemis run maps_navigate query="Melbourne Museum"
task: maps_navigate
description: Search Google Maps for a place and show the matching result list.
app: com.google.android.apps.maps
# The caller must supply these.
param: query
step_timeout: 20s
total_timeout: 180s
# Screens Artemis must never tap on. Each one fails the run with the reason, so
# a wrong tap is impossible (FR-8). Maps on this phone is signed in, so the only
# screen that can get in the way of the search box is the account and settings
# sheet (labels taken from a live dump of that sheet).
guard "Manage your Google Account" => Maps is showing its account sheet; press back on the phone and retry
guard "Turn on Incognito mode" => Maps is showing its account sheet; press back on the phone and retry
steps:
# 1. Make sure Maps is up. Launching an app that is already open is a no-op,
# so this is safe to repeat.
- launch
- foreground 30s
# 2. Open the search box. On the map, the box is the EditText labelled
# "Search here" (resource id search_omnibox_text_box); tapping it opens the
# search screen with the box focused and a suggestions list below it.
- wait "Search here" 30s retry 2
- tap "Search here"
# 3. The search screen is a new screen, and the wedge check sits here. If the
# tap was delivered but Maps never acted on it, the screen does not change:
# expect_change waits a bounded time, then fails with the distinct reason
# "the app stopped consuming input". "retry 2" presses the tap again first,
# because a dropped tap is more common than a wedged app.
- expect_change 15s retry 2
# 4. Type the place name into the focused box. The suggestions list redraws
# underneath it.
- type "{query}"
# 5. Same wedge check for the typing: a wedged app draws its old screen, so
# the box keeps the old text and this fails on its own reason.
- expect_change 15s retry 2
# 6. The result list must name the place the caller asked for. The first
# suggestion row for the query carries the place as its title and as the
# label of its own row button ("Activate to enter suggestion <place> into
# search bar"), so a match here is a real suggestion row, not an empty box.
- wait "{query}" 30s else "the Maps result list did not name the place"