fix: address the Maps search box by id, and re-prove Phase 5

The Maps omnibox hint rotates: it read "Search here" in the Phase 5 run and
"Try petrol stations, ATMs" in the AUD-128 re-verification, so the recipe's
text match on it failed. The resource id does not rotate.

- internal/ui: add Find/FindID and the "id:" selector; Find dispatches an
  id: selector to the resource id and everything else to text as before.
- internal/recipe: wait/tap/wait_gone/tap_if and the tap helper use Find, so a
  recipe may name a resource id. Guards stay text-only.
- recipes/maps_navigate.recipe: wait/tap id:search_omnibox_text_box.
- tests for the id selector; docs and README updated.
This commit is contained in:
2026-10-17 09:12:01 +11:00
parent a7c07c645e
commit be22eba386
9 changed files with 206 additions and 26 deletions

View File

@@ -11,6 +11,7 @@ Date: 2026-10-16. Phone: `Pixel_8a (akita)` over Tailscale at
| File | What it holds |
|---|---|
| `recipes/maps_navigate.recipe` | The recipe: open the search box, type the place, confirm the result list names it. |
| `internal/ui/dump.go` | `Find` / `FindID`: the `id:` selector, for a label that rotates. |
| `internal/ui/change.go` | `ScreenSignature` and `WaitForChange`: the screen fingerprint and the bounded change check. |
| `internal/recipe/recipe.go` | The new `expect_change <duration>` verb. |
| `internal/recipe/engine.go` | Runs the check, and reads the screen **before** the step above it. |
@@ -31,8 +32,8 @@ guard "Turn on Incognito mode" => Maps is showing its account sheet; press b
steps:
- launch
- foreground 30s
- wait "Search here" 30s retry 2
- tap "Search here"
- wait id:search_omnibox_text_box 30s retry 2
- tap id:search_omnibox_text_box
- expect_change 15s retry 2
- type "{query}"
- expect_change 15s retry 2
@@ -45,6 +46,41 @@ Run it with:
./bin/artemis run maps_navigate query="Melbourne Museum"
```
## The search box is addressed by id, not by its label
The first version of the recipe waited for the text **`Search here`**. On
2026-10-17 a re-verification run failed at that step: the box was on screen,
but its label had rotated to **`Try petrol stations, ATMs`**:
```
step 3 wait [Search here]: element matching "Search here" did not appear within 30s;
foreground=com.google.android.apps.maps;
screen shows: "Layers", "Enter compass mode", "Directions", "Ivanhoe", ...
```
The Maps omnibox carries its hint as a **rotating content description**, so a
text match on it is not stable. A fresh dump of the live node shows the stable
half and the rotating half side by side:
```
<node resource-id="com.google.android.apps.maps:id/search_omnibox_text_box"
class="android.widget.EditText" content-desc="Try petrol stations, ATMs"
clickable="true" enabled="true" bounds="[31,142][663,268]"/>
```
The recipe now names the resource id with the **`id:` selector**:
`wait id:search_omnibox_text_box` and `tap id:search_omnibox_text_box`. This is
still a named match with the tap coordinate derived from the matched element's
bounds — not a fixed coordinate, so FR-4 holds. `internal/ui/dump.go` dispatches
`Find`: a selector that starts with `id:` is matched against the resource id
(full or short name), and every other selector against text and content
description as before. `guards` remain text-only, because a guard names visible
screen text.
The rule to keep: **use a text selector where a stable label exists, and an
`id:` selector where the label rotates.** The final `wait "{query}"` stays a
text match on purpose — the result row must name the place the caller asked for.
## The labels came from the phone, not from a guess
A fresh `uiautomator dump` of the Maps home and the search screen on the real
@@ -223,10 +259,47 @@ and the typing that draws the result list.
| `expect_change` parsing, `retry`, `else` | unit tested (`internal/recipe`) |
| All shipped recipes parse | unit tested (`internal/recipe`) |
| `maps_navigate` opens a real search and names the place | **live, verified** — 8 steps, 27.6s, result list above |
| `maps_navigate` re-verified after the label rotated | **live, verified** — 8 steps, 27.2s, id selector; see “Re-verification 2026-10-17” below |
| The wedge reason fires on an input that changes nothing | **live, verified** — probe above |
| A wedged app was reproduced | **no** — could not be forced; see the limit above |
| The wedge check is bounded by `retry` | **live, verified** — 3 attempts, then stop |
## Re-verification 2026-10-17
AUD-128 re-ran the recipe on the current tree and the current phone session.
The text selector failed (the label had rotated, above), so the recipe moved to
the `id:` selector and the run went green:
```sh
$ ./bin/artemis run maps_navigate query="Melbourne Museum"
task maps_navigate (Search Google Maps for a place and show the matching result list.)
step 1 launch []: ok (676ms)
step 2 foreground []: ok (166ms)
step 3 wait [id:search_omnibox_text_box]: ok (2.795s)
step 4 tap [id:search_omnibox_text_box]: ok (5.768s)
step 5 expect_change [15s]: ok (3.21s)
step 6 type [Melbourne Museum]: ok (420ms)
step 7 expect_change [15s]: ok (3.479s)
step 8 wait [Melbourne Museum]: ok (2.992s)
OK maps_navigate in 27.177s
```
The run log records `"ok":true`, and the wedge probe fired on a live no-op input
(`key escape` on the map home), three attempts in total, then stopped:
```
step 3 key [escape]: ok (151ms)
step 4 expect_change [6s]: the app stopped consuming input: the screen did not change within 6s;
foreground=com.google.android.apps.maps; screen shows: "Layers", "Enter compass mode", ...
step 4 expect_change: retrying, pressing key again
step 3 key [escape]: ok (207ms)
step 4 expect_change [6s]: the app stopped consuming input: ...
step 4 expect_change: retrying, pressing key again
step 3 key [escape]: ok (180ms)
step 4 expect_change [6s]: the app stopped consuming input: ...
FAILED maps_wedge_probe after 44.346s: ...
```
## Reproducing it
```sh