Sam Rolfe
a7c07c645e
The Maps recipe: open the search box, type the place, and confirm the result list names it. Every label comes from a live uiautomator dump (search_omnibox_text_box, typed_suggest_container, the account sheet), never from a guess. Text only, one step_timeout/total_timeout, guards for the account sheet. The first live run exposed an ordering bug in expect_change. A dump is not instant, so a baseline taken after the input already showed the app's response and the check could never see a wedge; it failed against a healthy Maps. The engine now reads the screen before the step above runs and hands that signature to the check. Each retry re-reads it. Live: maps_navigate ok in 27.6s, result list names "Melbourne Museum". The wedge reason and its bounded retry are demonstrated with a probe recipe whose input changes nothing; a real wedge could not be forced, and docs/phase-5-maps-recipe.md says so plainly. Co-Authored-By: Paperclip <noreply@paperclip.ing>
112 lines
5.2 KiB
Markdown
112 lines
5.2 KiB
Markdown
# Artemis Control
|
||
|
||
Artemis is the phone's **hands**. It drives the Pixel 8a screen from `.13` over
|
||
Tailscale using **ADB wireless**, so the Hub can do things no deep link allows.
|
||
|
||
The build spec is [`AGENTS.md`](AGENTS.md). Follow the phases in section 11 in
|
||
order.
|
||
|
||
## Status
|
||
|
||
| Phase | State |
|
||
|---|---|
|
||
| 1 — Prove the ADB link | ✅ proven — see [`docs/phase-1-adb-link.md`](docs/phase-1-adb-link.md) |
|
||
| 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch |
|
||
| 3 — Spotify recipe | ✅ done — `spotify_play` plays the right song end to end (13 steps, 35s), see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) |
|
||
| 4 — Song radio | ✅ done — `spotify_radio` starts a real radio (19 steps, 57s); the run log is now bounded at ~1 MB × 5, see [`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) |
|
||
| 5 — Maps recipe | ✅ done — `maps_navigate` opens a real search and the result list names the place (8 steps, 28s), see [`docs/phase-5-maps-recipe.md`](docs/phase-5-maps-recipe.md) |
|
||
| 6 — Action API | not started |
|
||
| 7 — Resilience | not started |
|
||
| 8 — Documentation | not started |
|
||
|
||
## Layout
|
||
|
||
```
|
||
cmd/artemis/ command line entry point
|
||
internal/adb/ timeout-bounded wrapper around the adb binary
|
||
internal/ui/ read the screen (uiautomator dump) and act on it (input)
|
||
internal/recipe/ the recipes file format and its runner
|
||
recipes/ one hand-editable file per task
|
||
logs/ run log (JSON lines, not committed)
|
||
docs/ phase evidence and notes
|
||
```
|
||
|
||
## Build and test
|
||
|
||
Go is not installed system-wide on `.13`; use a one-off nix shell. Nothing is
|
||
added to `nixos-rebuild` and no system module changes.
|
||
|
||
```sh
|
||
nix shell nixpkgs#go -c go build -o bin/artemis ./cmd/artemis
|
||
nix shell nixpkgs#go -c go vet ./...
|
||
nix shell nixpkgs#go -c go test ./...
|
||
```
|
||
|
||
## Use
|
||
|
||
The phone address defaults to `100.101.49.17`. Override with `ARTEMIS_PHONE`,
|
||
pin a serial with `ARTEMIS_ADB_SERIAL`, point at a different adb with
|
||
`ARTEMIS_ADB`, point at another recipes directory with `ARTEMIS_RECIPES`, and
|
||
move the run log with `ARTEMIS_LOG`.
|
||
|
||
```sh
|
||
./bin/artemis connect # scan for the wireless-debugging port, then connect
|
||
./bin/artemis devices
|
||
./bin/artemis dump # text nodes and bounds
|
||
./bin/artemis keyguard # lock-screen state; exits 1 when locked
|
||
./bin/artemis launch com.spotify.music
|
||
./bin/artemis tap "Search" # find by text, tap its centre (FR-4)
|
||
./bin/artemis type "Rolling Stones Honky Tonk"
|
||
./bin/artemis scroll down
|
||
./bin/artemis back
|
||
./bin/artemis home
|
||
./bin/artemis demo # Phase 2 DoD: launch an app, tap a named element
|
||
```
|
||
|
||
## Recipes
|
||
|
||
A recipe is a plain-text file in `recipes/` that Sam edits by hand. Steps name
|
||
what to find on screen; there are no coordinates anywhere.
|
||
|
||
```sh
|
||
./bin/artemis recipes # list them
|
||
./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
|
||
./bin/artemis run spotify_radio query="Rolling Stones Honky Tonk"
|
||
./bin/artemis run maps_navigate query="Melbourne Museum"
|
||
```
|
||
|
||
Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the
|
||
file format, the verbs, and why list rows need their own step,
|
||
[`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) for the context menu,
|
||
`menu_row`, and the two ways a radio run fails on purpose, and
|
||
[`docs/phase-5-maps-recipe.md`](docs/phase-5-maps-recipe.md) for the Maps
|
||
labels, the `expect_change` check, and the one ordering rule it depends on.
|
||
|
||
## Rules that the code enforces
|
||
|
||
- **Find by text, never fixed coordinates.** `tap` resolves a text match to
|
||
bounds and taps the centre; `tap_row` scores result rows against the query
|
||
and taps the winning row's centre; `menu_row` opens the action menu of that
|
||
same row, from the button inside it.
|
||
- **Never tap blindly.** A missing or disabled element is a loud failure, not a
|
||
guessed coordinate. A named bad screen (`guard`) fails with its own reason,
|
||
and a step may name its own failure with `else "..."`.
|
||
- **Detect a wedged app.** An app can stop consuming input while it keeps
|
||
drawing: Android delivers the tap, the app never acts on it. `expect_change`
|
||
fails with `the app stopped consuming input` when the screen does not change
|
||
after the previous step, instead of blaming the label the next step waited for.
|
||
- **Never touch a locked phone.** A secure keyguard stops Artemis with
|
||
`phone is locked; unlock it and retry`. No PIN is stored, read or typed
|
||
(AGENTS.md section 13 Q2).
|
||
- **Confirm playback from the media session, not the screen.** Spotify's
|
||
transport button keeps the label "Play" while music plays; the `playing` step
|
||
asks Android what is actually playing and checks the track title.
|
||
- **Wake first.** Every acting command wakes the screen before it acts (FR-7),
|
||
and it checks the keyguard only after the screen is on.
|
||
- **Bound every command.** No adb call runs without a timeout, and a recipe has
|
||
a per-step and a whole-run timeout (FR-11).
|
||
- **Log every run.** Each run appends a JSON line to `logs/runs.jsonl` (FR-10).
|
||
The file rotates at about 1 MB, keeping the recent runs (FR-10 is an audit
|
||
trail, so history is bounded, never deleted wholesale). Tune it with
|
||
`ARTEMIS_LOG_MAX_BYTES` and `ARTEMIS_LOG_KEEP`.
|