1115531684
Add the Artemis Go CLI and its packages: - internal/adb: timeout-bounded adb wrapper, device list, wireless-debugging port scan and auto-connect (never hardcodes the moving port). - internal/ui: uiautomator dump parsing, find-by-text, tap the centre of a text match (FR-4), type, swipe, scroll, back, home, launch, wake. - cmd/artemis: CLI for the primitives plus 'demo', the Phase 2 DoD run. Failures are loud: a missing or disabled element errors instead of tapping a guessed coordinate. The screen is woken before acting, and every adb call has a deadline. Also record the section 13 Q4 decision (adb lives in the user profile on .13, wireless ADB needs no udev rules; service packaging still open until Phase 6), add unit tests, and document the phase evidence.
104 lines
4.0 KiB
Markdown
104 lines
4.0 KiB
Markdown
# Phase 2 — Act primitives
|
|
|
|
Status: **done**. Verified live against the Pixel 8a on 2026-10-16.
|
|
|
|
Definition of Done: *Artemis opens a named app and taps a named element on it.*
|
|
|
|
---
|
|
|
|
## 1. Primitives
|
|
|
|
All primitives live in `internal/ui`. Every acting command wakes the screen
|
|
first (FR-7) and every adb call is bounded by a timeout (FR-11).
|
|
|
|
| Primitive | Command | How it works |
|
|
|---|---|---|
|
|
| Tap by text | `artemis tap <text>` | dump → match text/content-desc → tap element centre (FR-4) |
|
|
| Type | `artemis type <text>` | `input text`, with `%s` space encoding and shell-safe quoting |
|
|
| Swipe | `artemis swipe x1 y1 x2 y2 [ms]` | `input swipe` |
|
|
| Scroll | `artemis scroll up\|down` | picks the tallest vertical scrollable node, swipes inside it |
|
|
| Back | `artemis back` | key event 4 |
|
|
| Home | `artemis home` | key event 3 |
|
|
| Launch app | `artemis launch <package>` | `monkey -p <pkg> ... 1`, then waits for foreground |
|
|
| Wake | `artemis wake` | key event 224, polls `dumpsys power` until `Awake` |
|
|
| Read screen | `artemis dump` | `uiautomator dump`, parsed as XML, text nodes with bounds |
|
|
| Find (no tap) | `artemis find <text>` | dump → matching elements |
|
|
| Foreground app | `artemis fg` | `dumpsys window` → `mCurrentFocus` |
|
|
|
|
## 2. Definition of Done run
|
|
|
|
`artemis demo` launches Settings, backs out to the Settings home, taps
|
|
**Network and internet**, then proves the tap navigated by waiting for
|
|
**Aeroplane mode**, which only exists on the target page.
|
|
|
|
```
|
|
$ ./bin/artemis demo
|
|
connected to 100.101.49.17:46689
|
|
step 1: launch com.android.settings
|
|
step 2: tap "Network and internet"
|
|
tapped android.widget.TextView[title] text="Network and internet" bounds=[221,663][748,727] at (484,695)
|
|
step 3: wait for "Aeroplane mode"
|
|
step 4: foreground package is com.android.settings
|
|
demo OK
|
|
```
|
|
|
|
## 3. Other live checks
|
|
|
|
```
|
|
$ ./bin/artemis home ; ./bin/artemis fg
|
|
com.google.android.apps.nexuslauncher
|
|
|
|
$ ./bin/artemis launch com.spotify.music
|
|
launched com.spotify.music in 698ms
|
|
|
|
$ ./bin/artemis scroll down # Spotify home: tabs stay, list content changes
|
|
before: [TextView] Julius Caesar Part 1: The Roman Dictator
|
|
after: [TextView] It's New Music Friday!
|
|
```
|
|
|
|
## 4. Failure is loud (FR-8)
|
|
|
|
The first demo run used the wrong string, `"Network & internet"`, while the
|
|
Pixel spells it `"Network and internet"`. Artemis refused to act:
|
|
|
|
```
|
|
artemis: element with text "Network & internet" not found on screen; refusing to tap blindly
|
|
```
|
|
|
|
No coordinate was guessed. A test in `internal/ui/dump_test.go` pins the same
|
|
behaviour: a set of matches whose only member is disabled yields an error, not a
|
|
tap.
|
|
|
|
## 5. Design notes
|
|
|
|
- **Text first, coordinates derived.** `TapText` finds matches, ranks them
|
|
(clickable first, then enabled, then smallest area), reads the winner's
|
|
bounds, and taps the centre. Coordinates are never written down by hand.
|
|
- **Scroll targets vertical lists only.** An earlier version picked the
|
|
smallest scrollable node, which on Spotify was a short horizontal carousel; a
|
|
vertical swipe there became a mis-gesture. The current version picks the
|
|
tallest scrollable node that is taller than it is wide and covers at least a
|
|
third of the screen, else it falls back to the screen rectangle.
|
|
- **Launch waits for a result.** `Launch` does not return success until the
|
|
package is actually the foreground app, or it fails after 10s.
|
|
- **Timeout everywhere.** `adb.Runner` runs each command through
|
|
`context.WithTimeout`; a recipe can never hang.
|
|
|
|
## 6. Tests
|
|
|
|
```sh
|
|
nix shell nixpkgs#go -c go vet ./...
|
|
nix shell nixpkgs#go -c go test ./...
|
|
```
|
|
|
|
`internal/ui` unit tests cover bounds parsing, centre maths, exact vs substring
|
|
text matching, content descriptions, disabled matches, and shell-safe input
|
|
escaping. The live checks above cover the phone.
|
|
|
|
## 7. Not done / next
|
|
|
|
- No recipes file yet; that is Phase 3.
|
|
- No HTTP action API yet; that is Phase 6.
|
|
- Locked-screen handling is unsettled (§13 Q2): `wake` dismisses a non-secure
|
|
keyguard but cannot unlock a PIN-locked phone.
|