artemis_control/docs/phase-5-maps-recipe.md

# Phase 5 — the Maps recipe, and the wedged-app check

Phase 5 of `AGENTS.md` section 11: the `maps_navigate` recipe, plus the wedged-app
detection the task asked for.

Date: 2026-10-16. Phone: `Pixel_8a (akita)` over Tailscale at
`100.101.49.17:46689` (the port moves on every reboot; nothing hardcodes it).

## What was built

| File | What it holds |
|---|---|
| `recipes/maps_navigate.recipe` | The recipe: open the search box, type the place, confirm the result list names it. |
| `internal/ui/change.go` | `ScreenSignature` and `WaitForChange`: the screen fingerprint and the bounded change check. |
| `internal/recipe/recipe.go` | The new `expect_change <duration>` verb. |
| `internal/recipe/engine.go` | Runs the check, and reads the screen **before** the step above it. |
| `internal/ui/change_test.go`, `internal/recipe/recipe_test.go` | Unit tests for the fingerprint and the verb. |

## The recipe

One file, plain text, no coordinates anywhere (FR-4):

```
task: maps_navigate
app: com.google.android.apps.maps
param: query

guard "Manage your Google Account" => Maps is showing its account sheet; press back on the phone and retry
guard "Turn on Incognito mode"     => Maps is showing its account sheet; press back on the phone and retry

steps:
  - launch
  - foreground 30s
  - wait "Search here" 30s retry 2
  - tap "Search here"
  - expect_change 15s retry 2
  - type "{query}"
  - expect_change 15s retry 2
  - wait "{query}" 30s else "the Maps result list did not name the place"
```

Run it with:

```sh
./bin/artemis run maps_navigate query="Melbourne Museum"
```

## The labels came from the phone, not from a guess

A fresh `uiautomator dump` of the Maps home and the search screen on the real
phone:

```
# Map home
[EditText]  Search here          id=search_omnibox_text_box   bounds=[31,142][663,268]
[View]      Voice search                                       bounds=[663,142][789,268]
[View]      Lens in Maps                                       bounds=[789,142][915,268]
[FrameLayout] Account and settings.                            bounds=[915,142][1049,268]
[FrameLayout] Explore / You / Contribute / Business            bounds=[0,2168][1080,2337]

# Search screen, after tapping "Search here"
[Button]      Navigate up                                      bounds=[31,142][157,268]
[EditText]    Search here        id=search_omnibox_text_box   bounds=[157,142][921,268]
[EditText]    <query>            id=search_omnibox_edit_text  bounds=[157,142][921,268]
[Button]      Voice search                                     bounds=[923,142][1049,268]
[ImageView]   Clear                                            bounds=[954,173][1017,236]
[RecyclerView] <results>         id=typed_suggest_container    bounds=[0,284][1080,1577]

# One suggestion row in that list
[TextView]    7.6 km                                           bounds=[0,378][210,431]
[TextView]    Melbourne Museum                                 bounds=[210,296][668,366]
[TextView]    Nicholson Street, Carlton VIC                    bounds=[210,366][823,429]
[FrameLayout] Activate to enter suggestion Melbourne Museum into search bar
                                                               bounds=[933,300][1059,426]
```

Three things this dump settled:

* The search box is the `EditText` labelled **`Search here`** (resource id
  `search_omnibox_text_box`). Tapping it opens the search screen with the box
  focused.
* Typing shows the results list immediately; there is no separate "press enter"
  step. The list is the `RecyclerView` with id `typed_suggest_container`.
* Each suggestion row carries the place as its title **and** as the label of its
  own row button: `Activate to enter suggestion <place> into search bar`. So the
  final `wait "{query}"` matches a real row, not the empty box. Because that row
  button is clickable and small, it is the match the run log reports.

The account sheet was opened on purpose to take its labels for the guards
(`Manage your Google Account`, `Turn on Incognito mode`). Maps on this phone is
signed in as `samuelrolfe@gmail.com`, so there is no sign-in screen in the
search path and no guard was invented for one.

## The wedged-app check (`expect_change`)

An app can stop consuming input while it keeps drawing: Android delivers the
tap, the app never acts on it (`docs/phase-3-spotify-recipe.md`, attempts 9, 10
and 13). Before Phase 5 the recipe reported that as the label it was waiting
for, which names the wrong problem.

`expect_change <duration>` reads the screen, waits a bounded time for it to
change, and fails with the distinct reason:

```
the app stopped consuming input: the screen did not change within 6s;
foreground=com.google.android.apps.maps; screen shows: "Layers", ...
```

The screen fingerprint is the foreground package plus every visible label and
its bounds, sorted so that the order uiautomator happens to report never looks
like a change (`internal/ui/change.go`).

### The baseline must be read *before* the input

The first live run of the recipe failed the check against a perfectly healthy
Maps:

```
step 4 tap [Search here]: ok (5.463s)
step 5 expect_change [15s]: the app stopped consuming input: the screen did not change within 15s
```

The tap had worked and the search screen had opened. The bug was ordering: a
`uiautomator dump` is not instant, so a baseline taken *after* the tap already
showed the app's response, and the "after" screen could never differ.

The engine now reads the screen immediately **before** the step above runs, and
hands that signature to the `expect_change` step that follows it
(`internal/recipe/engine.go`). A dump runs a JVM on the phone, so this read
happens only when a recipe asks for the check. Each `retry` re-reads the screen
before it re-presses the step above.

## The live run

```sh
$ ./bin/artemis run maps_navigate query="Melbourne Museum"
task maps_navigate (Search Google Maps for a place and show the matching result list.)
  step 1 launch []: ok (616ms)
  step 2 foreground []: ok (157ms)
  step 3 wait [Search here]: ok (2.672s)
  step 4 tap [Search here]: ok (5.647s)
  step 5 expect_change [15s]: ok (3.156s)
  step 6 type [Melbourne Museum]: ok (447ms)
  step 7 expect_change [15s]: ok (3.077s)
  step 8 wait [Melbourne Museum]: ok (2.692s)
OK maps_navigate in 27.606s
```

The run log (`logs/runs.jsonl`) records `"ok":true`, and the step that proves the
DoD reports:

```json
{"step":8,"verb":"wait","args":["{query}"],
 "detail":"found android.widget.FrameLayout[-] text=\"Activate to enter suggestion Melbourne Museum into search bar\" bounds=[933,300][1059,426]",
 "ok":true,"duration_ms":2692}
```

And the screen after the run, read back with a fresh dump — the result list
names the searched place:

```
$ ./bin/artemis dump
38 text nodes
[TextView]     7.6 km                                          bounds=[0,378][210,431]
[TextView]     Melbourne Museum                                bounds=[210,296][668,366]   <- the place, first row
[TextView]     Nicholson Street, Carlton VIC                    bounds=[210,366][823,429]
[FrameLayout]  Activate to enter suggestion Melbourne Museum into search bar
[TextView]     7.6 km                                          bounds=[0,539][210,592]
[TextView]     Melbourne Museum - Museum Market Cafe           bounds=[210,457][923,527]
[TextView]     Nicholson Street, Carlton VIC                    bounds=[210,527][823,590]
[TextView]     7.5 km                                          bounds=[0,700][210,753]
[TextView]     Melbourne Museum - Museum Shop                  bounds=[210,618][923,688]
[TextView]     Nicholson Street, Carlton VIC                    bounds=[210,688][823,751]
[TextView]     8.2 km                                          bounds=[0,861][210,914]
[TextView]     Museum of Chinese Australian History            bounds=[210,779][923,849]
[TextView]     Cohen Place, Melbourne VIC                       bounds=[210,849][803,912]
[TextView]     7.6 km                                          bounds=[0,1022][210,1075]
[TextView]     Children's Gallery (Melbourne Museum)           bounds=[210,940][923,1010]
[TextView]     Carlton VIC                                      bounds=[210,1010][451,1073]
```

## The wedge reason, demonstrated live

A real wedged app could **not** be reproduced in Phase 5: the Phase 3 wedge came
from a memory-starved phone under load, and it cannot be forced on demand. What
was verified instead is the detection path itself, with a probe recipe (not
shipped) whose step above is a real input that Maps answers with no screen
change — an ENTER key event on the map home, where nothing has focus:

```sh
$ ARTEMIS_RECIPES=<probe dir> ./bin/artemis run maps_wedge_probe
  step 4 key [enter]: ok (150ms)
  step 5 expect_change [6s]: the app stopped consuming input: the screen did not change within 6s; foreground=com.google.android.apps.maps; screen shows: "Layers", "Re-centre map to your location", "Directions", "Carlton", "Mostly cloudy, 23°, Good, AQC", "23°"
FAILED maps_wedge_probe after 17.857s: ...
```

The bounded retry was verified in the same way (`expect_change 6s retry 2`):
the engine re-pressed the step above twice more and then stopped, three attempts
in total, so a wedged app ends the run instead of looping (FR-11):

```sh
  step 5 expect_change [6s]: the app stopped consuming input: ...
  step 5 expect_change: retrying, pressing key again
  step 4 key [enter]: ok (223ms)
  step 5 expect_change [6s]: the app stopped consuming input: ...
  step 5 expect_change: retrying, pressing key again
  step 4 key [enter]: ok (150ms)
  step 5 expect_change [6s]: the app stopped consuming input: ...
FAILED maps_wedge_probe after 46.099s: ...
```

**Limit, stated plainly.** The check cannot tell a wedged app from an input that
legitimately changes nothing: it keys on "the screen did not change". A recipe
must place `expect_change` only after an input that must change the screen. The
`maps_navigate` recipe does that twice — the tap that opens the search screen,
and the typing that draws the result list.

## Verification status

| Item | State |
|---|---|
| Screen fingerprint, stability and sensitivity | unit tested (`internal/ui`) |
| `expect_change` parsing, `retry`, `else` | unit tested (`internal/recipe`) |
| All shipped recipes parse | unit tested (`internal/recipe`) |
| `maps_navigate` opens a real search and names the place | **live, verified** — 8 steps, 27.6s, result list above |
| The wedge reason fires on an input that changes nothing | **live, verified** — probe above |
| A wedged app was reproduced | **no** — could not be forced; see the limit above |
| The wedge check is bounded by `retry` | **live, verified** — 3 attempts, then stop |

## Reproducing it

```sh
cd ~/paseo/projects/artemis_control
./bin/artemis run maps_navigate query="Melbourne Museum"
tail -1 logs/runs.jsonl
./bin/artemis dump
```

The phone must be awake and unlocked (a secure keyguard is a loud refusal,
`AGENTS.md` section 13 Q2). From the map home this takes about 28 seconds.

## Not done

* The recipe stops at the result list, which is what the Phase 5 DoD asks for.
  It does not tap a result to open the place, so it does not start navigation.
* Phase 6 (the action API) and Phase 7 (resilience) have not been started, on
  instruction.