# Phase 2 — Act primitives Status: **done**. Verified live against the Pixel 8a on 2026-10-16. Definition of Done: *Artemis opens a named app and taps a named element on it.* --- ## 1. Primitives All primitives live in `internal/ui`. Every acting command wakes the screen first (FR-7) and every adb call is bounded by a timeout (FR-11). | Primitive | Command | How it works | |---|---|---| | Tap by text | `artemis tap ` | dump → match text/content-desc → tap element centre (FR-4) | | Type | `artemis type ` | `input text`, with `%s` space encoding and shell-safe quoting | | Swipe | `artemis swipe x1 y1 x2 y2 [ms]` | `input swipe` | | Scroll | `artemis scroll up\|down` | picks the tallest vertical scrollable node, swipes inside it | | Back | `artemis back` | key event 4 | | Home | `artemis home` | key event 3 | | Launch app | `artemis launch ` | `monkey -p ... 1`, then waits for foreground | | Wake | `artemis wake` | key event 224, polls `dumpsys power` until `Awake` | | Read screen | `artemis dump` | `uiautomator dump`, parsed as XML, text nodes with bounds | | Find (no tap) | `artemis find ` | dump → matching elements | | Foreground app | `artemis fg` | `dumpsys window` → `mCurrentFocus` | ## 2. Definition of Done run `artemis demo` launches Settings, backs out to the Settings home, taps **Network and internet**, then proves the tap navigated by waiting for **Aeroplane mode**, which only exists on the target page. ``` $ ./bin/artemis demo connected to 100.101.49.17:46689 step 1: launch com.android.settings step 2: tap "Network and internet" tapped android.widget.TextView[title] text="Network and internet" bounds=[221,663][748,727] at (484,695) step 3: wait for "Aeroplane mode" step 4: foreground package is com.android.settings demo OK ``` ## 3. Other live checks ``` $ ./bin/artemis home ; ./bin/artemis fg com.google.android.apps.nexuslauncher $ ./bin/artemis launch com.spotify.music launched com.spotify.music in 698ms $ ./bin/artemis scroll down # Spotify home: tabs stay, list content changes before: [TextView] Julius Caesar Part 1: The Roman Dictator after: [TextView] It's New Music Friday! ``` ## 4. Failure is loud (FR-8) The first demo run used the wrong string, `"Network & internet"`, while the Pixel spells it `"Network and internet"`. Artemis refused to act: ``` artemis: element with text "Network & internet" not found on screen; refusing to tap blindly ``` No coordinate was guessed. A test in `internal/ui/dump_test.go` pins the same behaviour: a set of matches whose only member is disabled yields an error, not a tap. ## 5. Design notes - **Text first, coordinates derived.** `TapText` finds matches, ranks them (clickable first, then enabled, then smallest area), reads the winner's bounds, and taps the centre. Coordinates are never written down by hand. - **Scroll targets vertical lists only.** An earlier version picked the smallest scrollable node, which on Spotify was a short horizontal carousel; a vertical swipe there became a mis-gesture. The current version picks the tallest scrollable node that is taller than it is wide and covers at least a third of the screen, else it falls back to the screen rectangle. - **Launch waits for a result.** `Launch` does not return success until the package is actually the foreground app, or it fails after 10s. - **Timeout everywhere.** `adb.Runner` runs each command through `context.WithTimeout`; a recipe can never hang. ## 6. Tests ```sh nix shell nixpkgs#go -c go vet ./... nix shell nixpkgs#go -c go test ./... ``` `internal/ui` unit tests cover bounds parsing, centre maths, exact vs substring text matching, content descriptions, disabled matches, and shell-safe input escaping. The live checks above cover the phone. ## 7. Not done / next - No recipes file yet; that is Phase 3. - No HTTP action API yet; that is Phase 6. - Locked-screen handling is unsettled (§13 Q2): `wake` dismisses a non-secure keyguard but cannot unlock a PIN-locked phone.