Sam Rolfe
36b4dd2391
spotify_radio opens a track's own context menu and starts its song radio.
The labels came from a fresh uiautomator dump on the phone, not from a guess:
the entry is "Go to song radio" and it sits below the fold, so the recipe
scrolls before it matches.
Recipe format:
- menu_row "{query}" finds the row like tap_row, then presses the action
button inside that row's own subtree (ui.Row.Action), so the More options
button of any other row is never a candidate. It records the row as {row}.
- a step can name its own failure: wait ... else "reason". The named reason
comes first and the screen's labels still follow (FR-8).
- scroll up|down moves the current list.
- playing with no title waits for the media session to be PLAYING. A radio
changes track every few minutes, so a title match would fail on a working
radio; spotify_play still passes the title it wants.
ui:
- pickBestMatch ignores a match with no usable bounds. An element scrolled out
of its list reports [0,0][0,0], and tapping its "centre" would be a blind tap
in the corner. A wait now only succeeds once the element is on screen.
Run log (FR-10): appendRunLog rotates runs.jsonl when a write would pass
ARTEMIS_LOG_MAX_BYTES (default 1 MiB), keeping ARTEMIS_LOG_KEEP (default 5)
files. History is bounded, never deleted wholesale, and the per-entry cap of
6 screen labels of 60 characters is unchanged.
Live evidence in docs/phase-4-song-radio.md: ok:true in 56.6s, the menu that
opened belonged to the pressed row, the station name "Honky Tonk Women Radio"
on the phone, and the radio playing past its seed track (Layla) per the media
session and dumpsys audio. Both new failure modes were demonstrated failing
with distinct reasons. The mini-bar's transport label stays "Play" while
playing on this build, which is why playback is confirmed from the media
session.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
105 lines
4.5 KiB
Markdown
105 lines
4.5 KiB
Markdown
# Artemis Control
|
||
|
||
Artemis is the phone's **hands**. It drives the Pixel 8a screen from `.13` over
|
||
Tailscale using **ADB wireless**, so the Hub can do things no deep link allows.
|
||
|
||
The build spec is [`AGENTS.md`](AGENTS.md). Follow the phases in section 11 in
|
||
order.
|
||
|
||
## Status
|
||
|
||
| Phase | State |
|
||
|---|---|
|
||
| 1 — Prove the ADB link | ✅ proven — see [`docs/phase-1-adb-link.md`](docs/phase-1-adb-link.md) |
|
||
| 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch |
|
||
| 3 — Spotify recipe | ✅ done — `spotify_play` plays the right song end to end (13 steps, 35s), see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) |
|
||
| 4 — Song radio | ✅ done — `spotify_radio` starts a real radio (19 steps, 57s); the run log is now bounded at ~1 MB × 5, see [`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) |
|
||
| 5 — Maps recipe | not started |
|
||
| 6 — Action API | not started |
|
||
| 7 — Resilience | not started |
|
||
| 8 — Documentation | not started |
|
||
|
||
## Layout
|
||
|
||
```
|
||
cmd/artemis/ command line entry point
|
||
internal/adb/ timeout-bounded wrapper around the adb binary
|
||
internal/ui/ read the screen (uiautomator dump) and act on it (input)
|
||
internal/recipe/ the recipes file format and its runner
|
||
recipes/ one hand-editable file per task
|
||
logs/ run log (JSON lines, not committed)
|
||
docs/ phase evidence and notes
|
||
```
|
||
|
||
## Build and test
|
||
|
||
Go is not installed system-wide on `.13`; use a one-off nix shell. Nothing is
|
||
added to `nixos-rebuild` and no system module changes.
|
||
|
||
```sh
|
||
nix shell nixpkgs#go -c go build -o bin/artemis ./cmd/artemis
|
||
nix shell nixpkgs#go -c go vet ./...
|
||
nix shell nixpkgs#go -c go test ./...
|
||
```
|
||
|
||
## Use
|
||
|
||
The phone address defaults to `100.101.49.17`. Override with `ARTEMIS_PHONE`,
|
||
pin a serial with `ARTEMIS_ADB_SERIAL`, point at a different adb with
|
||
`ARTEMIS_ADB`, point at another recipes directory with `ARTEMIS_RECIPES`, and
|
||
move the run log with `ARTEMIS_LOG`.
|
||
|
||
```sh
|
||
./bin/artemis connect # scan for the wireless-debugging port, then connect
|
||
./bin/artemis devices
|
||
./bin/artemis dump # text nodes and bounds
|
||
./bin/artemis keyguard # lock-screen state; exits 1 when locked
|
||
./bin/artemis launch com.spotify.music
|
||
./bin/artemis tap "Search" # find by text, tap its centre (FR-4)
|
||
./bin/artemis type "Rolling Stones Honky Tonk"
|
||
./bin/artemis scroll down
|
||
./bin/artemis back
|
||
./bin/artemis home
|
||
./bin/artemis demo # Phase 2 DoD: launch an app, tap a named element
|
||
```
|
||
|
||
## Recipes
|
||
|
||
A recipe is a plain-text file in `recipes/` that Sam edits by hand. Steps name
|
||
what to find on screen; there are no coordinates anywhere.
|
||
|
||
```sh
|
||
./bin/artemis recipes # list them
|
||
./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
|
||
./bin/artemis run spotify_radio query="Rolling Stones Honky Tonk"
|
||
```
|
||
|
||
Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the
|
||
file format, the verbs, and why list rows need their own step, and
|
||
[`docs/phase-4-song-radio.md`](docs/phase-4-song-radio.md) for the context menu,
|
||
`menu_row`, and the two ways a radio run fails on purpose.
|
||
|
||
## Rules that the code enforces
|
||
|
||
- **Find by text, never fixed coordinates.** `tap` resolves a text match to
|
||
bounds and taps the centre; `tap_row` scores result rows against the query
|
||
and taps the winning row's centre; `menu_row` opens the action menu of that
|
||
same row, from the button inside it.
|
||
- **Never tap blindly.** A missing or disabled element is a loud failure, not a
|
||
guessed coordinate. A named bad screen (`guard`) fails with its own reason,
|
||
and a step may name its own failure with `else "..."`.
|
||
- **Never touch a locked phone.** A secure keyguard stops Artemis with
|
||
`phone is locked; unlock it and retry`. No PIN is stored, read or typed
|
||
(AGENTS.md section 13 Q2).
|
||
- **Confirm playback from the media session, not the screen.** Spotify's
|
||
transport button keeps the label "Play" while music plays; the `playing` step
|
||
asks Android what is actually playing and checks the track title.
|
||
- **Wake first.** Every acting command wakes the screen before it acts (FR-7),
|
||
and it checks the keyguard only after the screen is on.
|
||
- **Bound every command.** No adb call runs without a timeout, and a recipe has
|
||
a per-step and a whole-run timeout (FR-11).
|
||
- **Log every run.** Each run appends a JSON line to `logs/runs.jsonl` (FR-10).
|
||
The file rotates at about 1 MB, keeping the recent runs (FR-10 is an audit
|
||
trail, so history is bounded, never deleted wholesale). Tune it with
|
||
`ARTEMIS_LOG_MAX_BYTES` and `ARTEMIS_LOG_KEEP`.
|