Sam Rolfe
c178f92851
Phase 3 DoD reached end to end: `artemis run spotify_play query="Rolling Stones Honky Tonk"` played the right song in 13 steps / 35.3s, with playback paused first so the pass proves the row tap started it. Two things stopped it before that, both found on the real phone: - The playback check asserted on an on-screen "Pause" label that Spotify never shows: its transport button keeps the label "Play" while music is playing. The recipe now ends with a `playing` step that reads the app's media session (state PLAYING) and confirms the track title contains the query words. A failure reports the state and the track it found. - tap_row pressed the wrong row. Once a track is loaded its row carries a `Paused` state indicator, and counting that content description as row text dropped the top hit below an identical duplicate further down, so the tap went to y=1443 instead of y=878. Rows are now scored on their title and subtitle only (state lives in content descriptions), and scores within rowScoreTolerance are treated as equal so the higher row wins, because Spotify already ranks by relevance. internal/ui/rows_test.go reproduces that results screen, including the state indicator and the duplicate row. docs/phase-3-spotify-recipe.md records the successful run, the independent media-session and audio-output reads, and the failure that preceded it.
97 lines
3.9 KiB
Markdown
97 lines
3.9 KiB
Markdown
# Artemis Control
|
|
|
|
Artemis is the phone's **hands**. It drives the Pixel 8a screen from `.13` over
|
|
Tailscale using **ADB wireless**, so the Hub can do things no deep link allows.
|
|
|
|
The build spec is [`AGENTS.md`](AGENTS.md). Follow the phases in section 11 in
|
|
order.
|
|
|
|
## Status
|
|
|
|
| Phase | State |
|
|
|---|---|
|
|
| 1 — Prove the ADB link | ✅ proven — see [`docs/phase-1-adb-link.md`](docs/phase-1-adb-link.md) |
|
|
| 2 — Act primitives | ✅ done — tap by text, type, scroll, back, home, launch |
|
|
| 3 — Spotify recipe | ✅ done — `spotify_play` plays the right song end to end (13 steps, 35s), see [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) |
|
|
| 4 — Song radio | not started |
|
|
| 5 — Maps recipe | not started |
|
|
| 6 — Action API | not started |
|
|
| 7 — Resilience | not started |
|
|
| 8 — Documentation | not started |
|
|
|
|
## Layout
|
|
|
|
```
|
|
cmd/artemis/ command line entry point
|
|
internal/adb/ timeout-bounded wrapper around the adb binary
|
|
internal/ui/ read the screen (uiautomator dump) and act on it (input)
|
|
internal/recipe/ the recipes file format and its runner
|
|
recipes/ one hand-editable file per task
|
|
logs/ run log (JSON lines, not committed)
|
|
docs/ phase evidence and notes
|
|
```
|
|
|
|
## Build and test
|
|
|
|
Go is not installed system-wide on `.13`; use a one-off nix shell. Nothing is
|
|
added to `nixos-rebuild` and no system module changes.
|
|
|
|
```sh
|
|
nix shell nixpkgs#go -c go build -o bin/artemis ./cmd/artemis
|
|
nix shell nixpkgs#go -c go vet ./...
|
|
nix shell nixpkgs#go -c go test ./...
|
|
```
|
|
|
|
## Use
|
|
|
|
The phone address defaults to `100.101.49.17`. Override with `ARTEMIS_PHONE`,
|
|
pin a serial with `ARTEMIS_ADB_SERIAL`, point at a different adb with
|
|
`ARTEMIS_ADB`, point at another recipes directory with `ARTEMIS_RECIPES`, and
|
|
move the run log with `ARTEMIS_LOG`.
|
|
|
|
```sh
|
|
./bin/artemis connect # scan for the wireless-debugging port, then connect
|
|
./bin/artemis devices
|
|
./bin/artemis dump # text nodes and bounds
|
|
./bin/artemis keyguard # lock-screen state; exits 1 when locked
|
|
./bin/artemis launch com.spotify.music
|
|
./bin/artemis tap "Search" # find by text, tap its centre (FR-4)
|
|
./bin/artemis type "Rolling Stones Honky Tonk"
|
|
./bin/artemis scroll down
|
|
./bin/artemis back
|
|
./bin/artemis home
|
|
./bin/artemis demo # Phase 2 DoD: launch an app, tap a named element
|
|
```
|
|
|
|
## Recipes
|
|
|
|
A recipe is a plain-text file in `recipes/` that Sam edits by hand. Steps name
|
|
what to find on screen; there are no coordinates anywhere.
|
|
|
|
```sh
|
|
./bin/artemis recipes # list them
|
|
./bin/artemis run spotify_play query="Rolling Stones Honky Tonk"
|
|
```
|
|
|
|
Read [`docs/phase-3-spotify-recipe.md`](docs/phase-3-spotify-recipe.md) for the
|
|
file format, the verbs, and why list rows need their own step.
|
|
|
|
## Rules that the code enforces
|
|
|
|
- **Find by text, never fixed coordinates.** `tap` resolves a text match to
|
|
bounds and taps the centre; `tap_row` scores result rows against the query
|
|
and taps the winning row's centre.
|
|
- **Never tap blindly.** A missing or disabled element is a loud failure, not a
|
|
guessed coordinate. A named bad screen (`guard`) fails with its own reason.
|
|
- **Never touch a locked phone.** A secure keyguard stops Artemis with
|
|
`phone is locked; unlock it and retry`. No PIN is stored, read or typed
|
|
(AGENTS.md section 13 Q2).
|
|
- **Confirm playback from the media session, not the screen.** Spotify's
|
|
transport button keeps the label "Play" while music plays; the `playing` step
|
|
asks Android what is actually playing and checks the track title.
|
|
- **Wake first.** Every acting command wakes the screen before it acts (FR-7),
|
|
and it checks the keyguard only after the screen is on.
|
|
- **Bound every command.** No adb call runs without a timeout, and a recipe has
|
|
a per-step and a whole-run timeout (FR-11).
|
|
- **Log every run.** Each run appends a JSON line to `logs/runs.jsonl` (FR-10).
|