feat: Phase 5 — maps_navigate, and read the screen before the input
The Maps recipe: open the search box, type the place, and confirm the result list names it. Every label comes from a live uiautomator dump (search_omnibox_text_box, typed_suggest_container, the account sheet), never from a guess. Text only, one step_timeout/total_timeout, guards for the account sheet. The first live run exposed an ordering bug in expect_change. A dump is not instant, so a baseline taken after the input already showed the app's response and the check could never see a wedge; it failed against a healthy Maps. The engine now reads the screen before the step above runs and hands that signature to the check. Each retry re-reads it. Live: maps_navigate ok in 27.6s, result list names "Melbourne Museum". The wedge reason and its bounded retry are demonstrated with a probe recipe whose input changes nothing; a real wedge could not be forced, and docs/phase-5-maps-recipe.md says so plainly. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
@@ -14,7 +14,7 @@ order.
|
||||
| 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch |
|
||||
| 3 — Spotify recipe | ✅ done — `spotify_play` plays the right song end to end (13 steps, 35s), see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) |
|
||||
| 4 — Song radio | ✅ done — `spotify_radio` starts a real radio (19 steps, 57s); the run log is now bounded at ~1 MB × 5, see [`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) |
|
||||
| 5 — Maps recipe | not started |
|
||||
| 5 — Maps recipe | ✅ done — `maps_navigate` opens a real search and the result list names the place (8 steps, 28s), see [`docs/phase-5-maps-recipe.md`](docs/phase-5-maps-recipe.md) |
|
||||
| 6 — Action API | not started |
|
||||
| 7 — Resilience | not started |
|
||||
| 8 — Documentation | not started |
|
||||
@@ -72,12 +72,15 @@ what to find on screen; there are no coordinates anywhere.
|
||||
./bin/artemis recipes # list them
|
||||
./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
|
||||
./bin/artemis run spotify_radio query="Rolling Stones Honky Tonk"
|
||||
./bin/artemis run maps_navigate query="Melbourne Museum"
|
||||
```
|
||||
|
||||
Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the
|
||||
file format, the verbs, and why list rows need their own step, and
|
||||
file format, the verbs, and why list rows need their own step,
|
||||
[`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) for the context menu,
|
||||
`menu_row`, and the two ways a radio run fails on purpose.
|
||||
`menu_row`, and the two ways a radio run fails on purpose, and
|
||||
[`docs/phase-5-maps-recipe.md`](docs/phase-5-maps-recipe.md) for the Maps
|
||||
labels, the `expect_change` check, and the one ordering rule it depends on.
|
||||
|
||||
## Rules that the code enforces
|
||||
|
||||
|
||||
247
docs/phase-5-maps-recipe.md
Normal file
247
docs/phase-5-maps-recipe.md
Normal file
@@ -0,0 +1,247 @@
|
||||
# Phase 5 — the Maps recipe, and the wedged-app check
|
||||
|
||||
Phase 5 of `AGENTS.md` section 11: the `maps_navigate` recipe, plus the wedged-app
|
||||
detection the task asked for.
|
||||
|
||||
Date: 2026-10-16. Phone: `Pixel_8a (akita)` over Tailscale at
|
||||
`100.101.49.17:46689` (the port moves on every reboot; nothing hardcodes it).
|
||||
|
||||
## What was built
|
||||
|
||||
| File | What it holds |
|
||||
|---|---|
|
||||
| `recipes/maps_navigate.recipe` | The recipe: open the search box, type the place, confirm the result list names it. |
|
||||
| `internal/ui/change.go` | `ScreenSignature` and `WaitForChange`: the screen fingerprint and the bounded change check. |
|
||||
| `internal/recipe/recipe.go` | The new `expect_change <duration>` verb. |
|
||||
| `internal/recipe/engine.go` | Runs the check, and reads the screen **before** the step above it. |
|
||||
| `internal/ui/change_test.go`, `internal/recipe/recipe_test.go` | Unit tests for the fingerprint and the verb. |
|
||||
|
||||
## The recipe
|
||||
|
||||
One file, plain text, no coordinates anywhere (FR-4):
|
||||
|
||||
```
|
||||
task: maps_navigate
|
||||
app: com.google.android.apps.maps
|
||||
param: query
|
||||
|
||||
guard "Manage your Google Account" => Maps is showing its account sheet; press back on the phone and retry
|
||||
guard "Turn on Incognito mode" => Maps is showing its account sheet; press back on the phone and retry
|
||||
|
||||
steps:
|
||||
- launch
|
||||
- foreground 30s
|
||||
- wait "Search here" 30s retry 2
|
||||
- tap "Search here"
|
||||
- expect_change 15s retry 2
|
||||
- type "{query}"
|
||||
- expect_change 15s retry 2
|
||||
- wait "{query}" 30s else "the Maps result list did not name the place"
|
||||
```
|
||||
|
||||
Run it with:
|
||||
|
||||
```sh
|
||||
./bin/artemis run maps_navigate query="Melbourne Museum"
|
||||
```
|
||||
|
||||
## The labels came from the phone, not from a guess
|
||||
|
||||
A fresh `uiautomator dump` of the Maps home and the search screen on the real
|
||||
phone:
|
||||
|
||||
```
|
||||
# Map home
|
||||
[EditText] Search here id=search_omnibox_text_box bounds=[31,142][663,268]
|
||||
[View] Voice search bounds=[663,142][789,268]
|
||||
[View] Lens in Maps bounds=[789,142][915,268]
|
||||
[FrameLayout] Account and settings. bounds=[915,142][1049,268]
|
||||
[FrameLayout] Explore / You / Contribute / Business bounds=[0,2168][1080,2337]
|
||||
|
||||
# Search screen, after tapping "Search here"
|
||||
[Button] Navigate up bounds=[31,142][157,268]
|
||||
[EditText] Search here id=search_omnibox_text_box bounds=[157,142][921,268]
|
||||
[EditText] <query> id=search_omnibox_edit_text bounds=[157,142][921,268]
|
||||
[Button] Voice search bounds=[923,142][1049,268]
|
||||
[ImageView] Clear bounds=[954,173][1017,236]
|
||||
[RecyclerView] <results> id=typed_suggest_container bounds=[0,284][1080,1577]
|
||||
|
||||
# One suggestion row in that list
|
||||
[TextView] 7.6 km bounds=[0,378][210,431]
|
||||
[TextView] Melbourne Museum bounds=[210,296][668,366]
|
||||
[TextView] Nicholson Street, Carlton VIC bounds=[210,366][823,429]
|
||||
[FrameLayout] Activate to enter suggestion Melbourne Museum into search bar
|
||||
bounds=[933,300][1059,426]
|
||||
```
|
||||
|
||||
Three things this dump settled:
|
||||
|
||||
* The search box is the `EditText` labelled **`Search here`** (resource id
|
||||
`search_omnibox_text_box`). Tapping it opens the search screen with the box
|
||||
focused.
|
||||
* Typing shows the results list immediately; there is no separate "press enter"
|
||||
step. The list is the `RecyclerView` with id `typed_suggest_container`.
|
||||
* Each suggestion row carries the place as its title **and** as the label of its
|
||||
own row button: `Activate to enter suggestion <place> into search bar`. So the
|
||||
final `wait "{query}"` matches a real row, not the empty box. Because that row
|
||||
button is clickable and small, it is the match the run log reports.
|
||||
|
||||
The account sheet was opened on purpose to take its labels for the guards
|
||||
(`Manage your Google Account`, `Turn on Incognito mode`). Maps on this phone is
|
||||
signed in as `samuelrolfe@gmail.com`, so there is no sign-in screen in the
|
||||
search path and no guard was invented for one.
|
||||
|
||||
## The wedged-app check (`expect_change`)
|
||||
|
||||
An app can stop consuming input while it keeps drawing: Android delivers the
|
||||
tap, the app never acts on it (`docs/phase-3-spotify-recipe.md`, attempts 9, 10
|
||||
and 13). Before Phase 5 the recipe reported that as the label it was waiting
|
||||
for, which names the wrong problem.
|
||||
|
||||
`expect_change <duration>` reads the screen, waits a bounded time for it to
|
||||
change, and fails with the distinct reason:
|
||||
|
||||
```
|
||||
the app stopped consuming input: the screen did not change within 6s;
|
||||
foreground=com.google.android.apps.maps; screen shows: "Layers", ...
|
||||
```
|
||||
|
||||
The screen fingerprint is the foreground package plus every visible label and
|
||||
its bounds, sorted so that the order uiautomator happens to report never looks
|
||||
like a change (`internal/ui/change.go`).
|
||||
|
||||
### The baseline must be read *before* the input
|
||||
|
||||
The first live run of the recipe failed the check against a perfectly healthy
|
||||
Maps:
|
||||
|
||||
```
|
||||
step 4 tap [Search here]: ok (5.463s)
|
||||
step 5 expect_change [15s]: the app stopped consuming input: the screen did not change within 15s
|
||||
```
|
||||
|
||||
The tap had worked and the search screen had opened. The bug was ordering: a
|
||||
`uiautomator dump` is not instant, so a baseline taken *after* the tap already
|
||||
showed the app's response, and the "after" screen could never differ.
|
||||
|
||||
The engine now reads the screen immediately **before** the step above runs, and
|
||||
hands that signature to the `expect_change` step that follows it
|
||||
(`internal/recipe/engine.go`). A dump runs a JVM on the phone, so this read
|
||||
happens only when a recipe asks for the check. Each `retry` re-reads the screen
|
||||
before it re-presses the step above.
|
||||
|
||||
## The live run
|
||||
|
||||
```sh
|
||||
$ ./bin/artemis run maps_navigate query="Melbourne Museum"
|
||||
task maps_navigate (Search Google Maps for a place and show the matching result list.)
|
||||
step 1 launch []: ok (616ms)
|
||||
step 2 foreground []: ok (157ms)
|
||||
step 3 wait [Search here]: ok (2.672s)
|
||||
step 4 tap [Search here]: ok (5.647s)
|
||||
step 5 expect_change [15s]: ok (3.156s)
|
||||
step 6 type [Melbourne Museum]: ok (447ms)
|
||||
step 7 expect_change [15s]: ok (3.077s)
|
||||
step 8 wait [Melbourne Museum]: ok (2.692s)
|
||||
OK maps_navigate in 27.606s
|
||||
```
|
||||
|
||||
The run log (`logs/runs.jsonl`) records `"ok":true`, and the step that proves the
|
||||
DoD reports:
|
||||
|
||||
```json
|
||||
{"step":8,"verb":"wait","args":["{query}"],
|
||||
"detail":"found android.widget.FrameLayout[-] text=\"Activate to enter suggestion Melbourne Museum into search bar\" bounds=[933,300][1059,426]",
|
||||
"ok":true,"duration_ms":2692}
|
||||
```
|
||||
|
||||
And the screen after the run, read back with a fresh dump — the result list
|
||||
names the searched place:
|
||||
|
||||
```
|
||||
$ ./bin/artemis dump
|
||||
38 text nodes
|
||||
[TextView] 7.6 km bounds=[0,378][210,431]
|
||||
[TextView] Melbourne Museum bounds=[210,296][668,366] <- the place, first row
|
||||
[TextView] Nicholson Street, Carlton VIC bounds=[210,366][823,429]
|
||||
[FrameLayout] Activate to enter suggestion Melbourne Museum into search bar
|
||||
[TextView] 7.6 km bounds=[0,539][210,592]
|
||||
[TextView] Melbourne Museum - Museum Market Cafe bounds=[210,457][923,527]
|
||||
[TextView] Nicholson Street, Carlton VIC bounds=[210,527][823,590]
|
||||
[TextView] 7.5 km bounds=[0,700][210,753]
|
||||
[TextView] Melbourne Museum - Museum Shop bounds=[210,618][923,688]
|
||||
[TextView] Nicholson Street, Carlton VIC bounds=[210,688][823,751]
|
||||
[TextView] 8.2 km bounds=[0,861][210,914]
|
||||
[TextView] Museum of Chinese Australian History bounds=[210,779][923,849]
|
||||
[TextView] Cohen Place, Melbourne VIC bounds=[210,849][803,912]
|
||||
[TextView] 7.6 km bounds=[0,1022][210,1075]
|
||||
[TextView] Children's Gallery (Melbourne Museum) bounds=[210,940][923,1010]
|
||||
[TextView] Carlton VIC bounds=[210,1010][451,1073]
|
||||
```
|
||||
|
||||
## The wedge reason, demonstrated live
|
||||
|
||||
A real wedged app could **not** be reproduced in Phase 5: the Phase 3 wedge came
|
||||
from a memory-starved phone under load, and it cannot be forced on demand. What
|
||||
was verified instead is the detection path itself, with a probe recipe (not
|
||||
shipped) whose step above is a real input that Maps answers with no screen
|
||||
change — an ENTER key event on the map home, where nothing has focus:
|
||||
|
||||
```sh
|
||||
$ ARTEMIS_RECIPES=<probe dir> ./bin/artemis run maps_wedge_probe
|
||||
step 4 key [enter]: ok (150ms)
|
||||
step 5 expect_change [6s]: the app stopped consuming input: the screen did not change within 6s; foreground=com.google.android.apps.maps; screen shows: "Layers", "Re-centre map to your location", "Directions", "Carlton", "Mostly cloudy, 23°, Good, AQC", "23°"
|
||||
FAILED maps_wedge_probe after 17.857s: ...
|
||||
```
|
||||
|
||||
The bounded retry was verified in the same way (`expect_change 6s retry 2`):
|
||||
the engine re-pressed the step above twice more and then stopped, three attempts
|
||||
in total, so a wedged app ends the run instead of looping (FR-11):
|
||||
|
||||
```sh
|
||||
step 5 expect_change [6s]: the app stopped consuming input: ...
|
||||
step 5 expect_change: retrying, pressing key again
|
||||
step 4 key [enter]: ok (223ms)
|
||||
step 5 expect_change [6s]: the app stopped consuming input: ...
|
||||
step 5 expect_change: retrying, pressing key again
|
||||
step 4 key [enter]: ok (150ms)
|
||||
step 5 expect_change [6s]: the app stopped consuming input: ...
|
||||
FAILED maps_wedge_probe after 46.099s: ...
|
||||
```
|
||||
|
||||
**Limit, stated plainly.** The check cannot tell a wedged app from an input that
|
||||
legitimately changes nothing: it keys on "the screen did not change". A recipe
|
||||
must place `expect_change` only after an input that must change the screen. The
|
||||
`maps_navigate` recipe does that twice — the tap that opens the search screen,
|
||||
and the typing that draws the result list.
|
||||
|
||||
## Verification status
|
||||
|
||||
| Item | State |
|
||||
|---|---|
|
||||
| Screen fingerprint, stability and sensitivity | unit tested (`internal/ui`) |
|
||||
| `expect_change` parsing, `retry`, `else` | unit tested (`internal/recipe`) |
|
||||
| All shipped recipes parse | unit tested (`internal/recipe`) |
|
||||
| `maps_navigate` opens a real search and names the place | **live, verified** — 8 steps, 27.6s, result list above |
|
||||
| The wedge reason fires on an input that changes nothing | **live, verified** — probe above |
|
||||
| A wedged app was reproduced | **no** — could not be forced; see the limit above |
|
||||
| The wedge check is bounded by `retry` | **live, verified** — 3 attempts, then stop |
|
||||
|
||||
## Reproducing it
|
||||
|
||||
```sh
|
||||
cd ~/paseo/projects/artemis_control
|
||||
./bin/artemis run maps_navigate query="Melbourne Museum"
|
||||
tail -1 logs/runs.jsonl
|
||||
./bin/artemis dump
|
||||
```
|
||||
|
||||
The phone must be awake and unlocked (a secure keyguard is a loud refusal,
|
||||
`AGENTS.md` section 13 Q2). From the map home this takes about 28 seconds.
|
||||
|
||||
## Not done
|
||||
|
||||
* The recipe stops at the result list, which is what the Phase 5 DoD asks for.
|
||||
It does not tap a result to open the place, so it does not start navigation.
|
||||
* Phase 6 (the action API) and Phase 7 (resilience) have not been started, on
|
||||
instruction.
|
||||
@@ -99,14 +99,30 @@ func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult {
|
||||
w, h, _ := ui.ScreenSize(r) // best effort: bounds checking degrades gracefully
|
||||
|
||||
deadline := started.Add(rec.TotalTimeout)
|
||||
// beforeChange is the screen signature captured immediately before the step
|
||||
// above, for a following expect_change step. The capture happens before that
|
||||
// step's input runs, because an app that never acts on the input leaves the
|
||||
// screen exactly as it was — and a dump taken afterwards would already show
|
||||
// the app's response (AGENTS.md section 11 Phase 5).
|
||||
var beforeChange string
|
||||
for i := 0; i < len(rec.Steps); i++ {
|
||||
step := rec.Steps[i]
|
||||
if step.Verb != "expect_change" && i+1 < len(rec.Steps) && rec.Steps[i+1].Verb == "expect_change" {
|
||||
// A dump runs a JVM on the phone, so this happens only when a recipe
|
||||
// actually asks for the wedge check.
|
||||
if sig, err := ui.ScreenSignature(r); err == nil {
|
||||
beforeChange = sig
|
||||
} else {
|
||||
beforeChange = ""
|
||||
logf(" step %d %s: could not read the screen before the wedge check: %v", i+1, step.Verb, err)
|
||||
}
|
||||
}
|
||||
var sr StepResult
|
||||
for attempt := 0; ; attempt++ {
|
||||
if remaining := time.Until(deadline); remaining <= 0 {
|
||||
return fail(fmt.Errorf("recipe timed out after %s at step %d (%s)", rec.TotalTimeout, i+1, step.Verb))
|
||||
}
|
||||
sr = executeStep(r, rec, step, i, params, w, h, deadline, logf)
|
||||
sr = executeStep(r, rec, step, i, params, w, h, deadline, beforeChange, logf)
|
||||
res.Steps = append(res.Steps, sr)
|
||||
if sr.OK {
|
||||
break
|
||||
@@ -120,7 +136,16 @@ func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult {
|
||||
return fail(fmt.Errorf("step %d: retry needs a step above it to repeat", i+1))
|
||||
}
|
||||
logf(" step %d %s: retrying, pressing %s again", i+1, step.Verb, rec.Steps[i-1].Verb)
|
||||
again := executeStep(r, rec, rec.Steps[i-1], i-1, params, w, h, deadline, logf)
|
||||
if step.Verb == "expect_change" {
|
||||
// Judge the re-press on its own: capture the screen again before
|
||||
// the step above runs a second time.
|
||||
if sig, err := ui.ScreenSignature(r); err == nil {
|
||||
beforeChange = sig
|
||||
} else {
|
||||
beforeChange = ""
|
||||
}
|
||||
}
|
||||
again := executeStep(r, rec, rec.Steps[i-1], i-1, params, w, h, deadline, "", logf)
|
||||
res.Steps = append(res.Steps, again)
|
||||
if !again.OK {
|
||||
// The wait is the real failure; the repeat just could not run.
|
||||
@@ -136,7 +161,7 @@ func Run(r *adb.Runner, rec *Recipe, opts Options) *RunResult {
|
||||
}
|
||||
|
||||
func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[string]string,
|
||||
w, h int, deadline time.Time, logf func(string, ...any)) (sr StepResult) {
|
||||
w, h int, deadline time.Time, beforeChange string, logf func(string, ...any)) (sr StepResult) {
|
||||
|
||||
started := time.Now()
|
||||
sr = StepResult{Index: index + 1, Verb: step.Verb, Args: step.Args}
|
||||
@@ -274,12 +299,15 @@ func executeStep(r *adb.Runner, rec *Recipe, step Step, index int, params map[st
|
||||
return nil
|
||||
|
||||
case "expect_change":
|
||||
// The wedged-app check. An app that has stopped consuming input
|
||||
// still draws its old screen, so a delivered tap or key changes
|
||||
// nothing. Failing here names the real problem instead of blaming
|
||||
// the label the next step would have waited for (Phase 5).
|
||||
_, _, err := ui.ExpectChange(r, timeout)
|
||||
if err != nil {
|
||||
// The wedged-app check. beforeChange was captured before the step
|
||||
// above ran. An app that has stopped consuming input still draws
|
||||
// its old screen, so the signature does not move; failing here
|
||||
// names the real problem instead of blaming the label the next
|
||||
// step would have waited for (Phase 5).
|
||||
if beforeChange == "" {
|
||||
return fmt.Errorf("the screen was not read before the step above, so a change cannot be judged")
|
||||
}
|
||||
if _, err := ui.WaitForChange(r, beforeChange, timeout); err != nil {
|
||||
return describeFailure(r, rec, fmt.Errorf("the app stopped consuming input: %w", err))
|
||||
}
|
||||
sr.Detail = "screen changed"
|
||||
|
||||
@@ -40,8 +40,9 @@
|
||||
// title matches the text; with no text, wait for PLAYING
|
||||
// alone (a radio plays a different track every few minutes)
|
||||
// - wait_gone "Recents" 30s wait until no element matches the text
|
||||
// - expect_change 10s fail unless the screen changes within the time
|
||||
// (the wedged-app check: a delivered input that the app never acts on)
|
||||
// - expect_change 10s the screen is read before the step above, so this
|
||||
// fails with "the app stopped consuming input" when
|
||||
// that step's input left the screen unchanged (a wedged app)
|
||||
// - tap "Search" tap the element whose text matches (FR-4)
|
||||
// - tap exact "Pause" tap an element whose text matches exactly
|
||||
// - tap_if "Clear search query" tap it when it is on screen, else skip
|
||||
|
||||
@@ -178,6 +178,56 @@ func TestSpotifyRecipeIsValid(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestMapsRecipeIsValid keeps the shipped Phase 5 recipe honest: a typo in
|
||||
// recipes/maps_navigate.recipe fails the test suite, not the phone.
|
||||
func TestMapsRecipeIsValid(t *testing.T) {
|
||||
rec, err := Load(filepath.Join("..", "..", "recipes", "maps_navigate.recipe"))
|
||||
if err != nil {
|
||||
t.Fatalf("Load: %v", err)
|
||||
}
|
||||
if rec.Task != "maps_navigate" || rec.App != "com.google.android.apps.maps" {
|
||||
t.Fatalf("header = %q / %q", rec.Task, rec.App)
|
||||
}
|
||||
if len(rec.MissingParams(map[string]string{"query": "x"})) != 0 {
|
||||
t.Fatal("maps_navigate must accept a query parameter")
|
||||
}
|
||||
if len(rec.Guards) == 0 {
|
||||
t.Fatal("maps_navigate needs guards for screens it must not tap")
|
||||
}
|
||||
|
||||
var sawForeground, sawChange int
|
||||
for i, s := range rec.Steps {
|
||||
switch s.Verb {
|
||||
case "foreground":
|
||||
sawForeground++
|
||||
case "expect_change":
|
||||
sawChange++
|
||||
// The engine reads the screen before the step above, so the check
|
||||
// needs a step above it.
|
||||
if i == 0 {
|
||||
t.Fatal("expect_change must have a step above it")
|
||||
}
|
||||
}
|
||||
// No coordinates, nowhere: every argument is text, a duration or a
|
||||
// placeholder, never a bare screen coordinate.
|
||||
for _, a := range s.Args {
|
||||
if _, err := strconv.Atoi(a); err == nil {
|
||||
t.Fatalf("step %s has a bare number %q as an argument: this recipe must name text, not coordinates", s.Verb, a)
|
||||
}
|
||||
}
|
||||
}
|
||||
if sawForeground == 0 {
|
||||
t.Fatal("maps_navigate must pin the foreground app before matching text")
|
||||
}
|
||||
if sawChange == 0 {
|
||||
t.Fatal("maps_navigate must check that its inputs changed the screen (the wedged-app check)")
|
||||
}
|
||||
last := rec.Steps[len(rec.Steps)-1]
|
||||
if last.Verb != "wait" || last.Args[0] != "{query}" {
|
||||
t.Fatalf("the last step must confirm the result list names the place, got %+v", last)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadAndList(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
write := func(name, body string) {
|
||||
@@ -290,7 +340,7 @@ func TestForegroundStep(t *testing.T) {
|
||||
func TestStepDurationsAreRecorded(t *testing.T) {
|
||||
rec := &Recipe{StepTimeout: time.Second}
|
||||
step := Step{Verb: "sleep", Timeout: 50 * time.Millisecond}
|
||||
sr := executeStep(adb.New(""), rec, step, 2, nil, 0, 0, time.Now().Add(time.Minute), func(string, ...any) {})
|
||||
sr := executeStep(adb.New(""), rec, step, 2, nil, 0, 0, time.Now().Add(time.Minute), "", func(string, ...any) {})
|
||||
if !sr.OK {
|
||||
t.Fatalf("sleep step failed: %s", sr.Error)
|
||||
}
|
||||
|
||||
@@ -50,34 +50,33 @@ func (h *Hierarchy) Signature(foreground string) string {
|
||||
return foreground + "\n" + strings.Join(parts, "\n")
|
||||
}
|
||||
|
||||
// ExpectChange waits until the screen no longer looks like it did when the
|
||||
// function was called, and returns the signatures before and after.
|
||||
// WaitForChange waits until the screen no longer matches before, and returns
|
||||
// the new signature.
|
||||
//
|
||||
// This is the wedged-app check. An app that has stopped consuming input still
|
||||
// draws the screen it had, so a delivered tap changes nothing. A timeout here
|
||||
// means the last input was accepted by Android but never acted on by the app,
|
||||
// which the recipe reports as "the app stopped consuming input" instead of
|
||||
// blaming the label it was waiting for (AGENTS.md section 11 Phase 5).
|
||||
func ExpectChange(r *adb.Runner, timeout time.Duration) (before, after string, err error) {
|
||||
// The caller captures before with ScreenSignature *before* the input it wants to
|
||||
// judge. That ordering is the whole check: an app that has stopped consuming
|
||||
// input still draws the screen it had, so the signature after the input is
|
||||
// identical to the one before it. A dump is not instant, so a baseline taken
|
||||
// after the input would already include the app's response and could never see
|
||||
// the wedge.
|
||||
func WaitForChange(r *adb.Runner, before string, timeout time.Duration) (string, error) {
|
||||
if strings.TrimSpace(before) == "" {
|
||||
return "", fmt.Errorf("no screen signature was captured before the input")
|
||||
}
|
||||
if timeout <= 0 {
|
||||
timeout = 10 * time.Second
|
||||
}
|
||||
before, err = ScreenSignature(r)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
|
||||
deadline := time.Now().Add(timeout)
|
||||
for {
|
||||
sig, derr := ScreenSignature(r)
|
||||
if derr != nil {
|
||||
return before, "", derr
|
||||
sig, err := ScreenSignature(r)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if sig != before {
|
||||
return before, sig, nil
|
||||
return sig, nil
|
||||
}
|
||||
if !time.Now().Before(deadline) {
|
||||
return before, before, fmt.Errorf("the screen did not change within %s", timeout)
|
||||
if time.Now().After(deadline) {
|
||||
return before, fmt.Errorf("the screen did not change within %s", timeout)
|
||||
}
|
||||
time.Sleep(PollInterval)
|
||||
}
|
||||
|
||||
58
recipes/maps_navigate.recipe
Normal file
58
recipes/maps_navigate.recipe
Normal file
@@ -0,0 +1,58 @@
|
||||
# maps_navigate — search Google Maps for a place.
|
||||
#
|
||||
# Edit this file by hand. Steps name the text Artemis must find on screen; it
|
||||
# taps the centre of the element it matched. There are no coordinates here on
|
||||
# purpose, so this recipe survives a Maps redesign (FR-4).
|
||||
#
|
||||
# Run it with:
|
||||
# artemis run maps_navigate query="Melbourne Museum"
|
||||
|
||||
task: maps_navigate
|
||||
description: Search Google Maps for a place and show the matching result list.
|
||||
app: com.google.android.apps.maps
|
||||
|
||||
# The caller must supply these.
|
||||
param: query
|
||||
|
||||
step_timeout: 20s
|
||||
total_timeout: 180s
|
||||
|
||||
# Screens Artemis must never tap on. Each one fails the run with the reason, so
|
||||
# a wrong tap is impossible (FR-8). Maps on this phone is signed in, so the only
|
||||
# screen that can get in the way of the search box is the account and settings
|
||||
# sheet (labels taken from a live dump of that sheet).
|
||||
guard "Manage your Google Account" => Maps is showing its account sheet; press back on the phone and retry
|
||||
guard "Turn on Incognito mode" => Maps is showing its account sheet; press back on the phone and retry
|
||||
|
||||
steps:
|
||||
# 1. Make sure Maps is up. Launching an app that is already open is a no-op,
|
||||
# so this is safe to repeat.
|
||||
- launch
|
||||
- foreground 30s
|
||||
|
||||
# 2. Open the search box. On the map, the box is the EditText labelled
|
||||
# "Search here" (resource id search_omnibox_text_box); tapping it opens the
|
||||
# search screen with the box focused and a suggestions list below it.
|
||||
- wait "Search here" 30s retry 2
|
||||
- tap "Search here"
|
||||
|
||||
# 3. The search screen is a new screen, and the wedge check sits here. If the
|
||||
# tap was delivered but Maps never acted on it, the screen does not change:
|
||||
# expect_change waits a bounded time, then fails with the distinct reason
|
||||
# "the app stopped consuming input". "retry 2" presses the tap again first,
|
||||
# because a dropped tap is more common than a wedged app.
|
||||
- expect_change 15s retry 2
|
||||
|
||||
# 4. Type the place name into the focused box. The suggestions list redraws
|
||||
# underneath it.
|
||||
- type "{query}"
|
||||
|
||||
# 5. Same wedge check for the typing: a wedged app draws its old screen, so
|
||||
# the box keeps the old text and this fails on its own reason.
|
||||
- expect_change 15s retry 2
|
||||
|
||||
# 6. The result list must name the place the caller asked for. The first
|
||||
# suggestion row for the query carries the place as its title and as the
|
||||
# label of its own row button ("Activate to enter suggestion <place> into
|
||||
# search bar"), so a match here is a real suggestion row, not an empty box.
|
||||
- wait "{query}" 30s else "the Maps result list did not name the place"
|
||||
Reference in New Issue
Block a user