artemis_control/AGENTS.md

# Artemis Control

> Build spec for Paperclip. Follow the phases in section 11 in order. Do not skip a
> phase's Definition of Done. When something is unclear, ask Sam — do not guess.

---

## 1. Purpose and outcome

Artemis is the phone's **hands**. It drives the phone screen from `.13` over
Tailscale using **ADB wireless**, so the Hub can do things no deep link allows.

**Outcome (the single success statement):**
> Sam says *"Open Spotify, search Rolling Stones Honky Tonk, play it, make a song
> radio"* and it happens on the phone, reliably.

Google Assistant cannot do this, because it cannot search inside an app. Artemis
can, because it uses the app's own UI.

## 2. Scope

**In scope**
- Hold an ADB session to the phone over Tailscale.
- Read the screen (element text and positions).
- Act: launch, tap, type, swipe, scroll, back, home.
- Named **recipes** for common tasks (for example, play a song in Spotify).
- A small action API for the Hub.

**Out of scope**
- Understanding speech. The Hub does that.
- Native phone actions (calls, contacts, SMS). Those belong to the Kotlin app.
- The phone app itself. That is `kotlin_phone_app_admin`.
- Voice and outbound conversations. That is `voice_outbound`.

## 3. Users and roles

- **No direct user.** Artemis is driven by the **Hub**.
- Sam may edit the **recipes file** by hand.

## 4. Functional requirements

| ID | Requirement |
|---|---|
| FR-1 | Hold an ADB session to the phone over Tailscale. Reconnect after a drop. |
| FR-2 | Read the screen with `adb shell uiautomator dump` — element text and bounds. |
| FR-3 | Act with `adb shell input`: tap, type, swipe, scroll, back, home, key events. |
| FR-4 | **Find an element by text and tap its centre.** Do not use fixed coordinates where text exists. |
| FR-5 | Named recipes for known tasks (`spotify_play`, `spotify_radio`, `maps_navigate`). |
| FR-6 | Action API: `POST /act {"task": "...", "params": {...}}` returns `ok` or `failed` with a reason. |
| FR-7 | Wake the screen before acting. |
| FR-8 | Fail cleanly: element not found, app not installed, app logged out, timeout. Never tap blindly. |
| FR-9 | Health: report ADB connected and phone reachable. |
| FR-10 | Log every action: task, steps, result, duration. |
| FR-11 | Per-step timeout. A recipe never runs forever. |

## 5. Screens and UX flow

**No UI in v1.** Artemis is headless. It exposes an API and reads a recipes file.

- **Recipes file** — a plain file (YAML or JSON) that Sam can edit to add or change
  a task. This replaces a UI for now.
- **Optional later** — a small desktop page to view recipes and recent runs. Build
  it only if the recipes file becomes painful.

**Example flow**
```
Hub: POST /act {"task":"spotify_play","params":{"query":"Rolling Stones Honky Tonk"}}
Artemis: wake screen → launch Spotify → tap "Search" → type query
         → wait → tap first result → tap Play
Artemis: {"status":"ok","duration_ms": 4200}
```

## 6. Look and feel

Not applicable in v1 (no UI). If a desktop page is added later, reuse the Family
Home Lab design system:
`/home/sam/Docker/Containers/family-home-lab/portal/static/tokens.css`.

## 7. Data model

Files and SQLite inside the service. Suggested:

| Store | Holds |
|---|---|
| `recipes/` | One file per task. Steps and selectors. Editable. |
| `action_log` (SQLite) | task, params, steps, result, duration, created-at |
| `session` | ADB connection state |

## 8. Integrations

| System | Use |
|---|---|
| **ADB** | The control channel. Wireless, over Tailscale. |
| **Tailscale** | Reaches the phone from `.13` when away from home. |
| **Hub** | Calls `/act`. Nothing comes back to the Hub. |
| **Kotlin app** | The partition: native actions go there, in-app navigation comes here. |
| **OmniRoute** | Only if a future step needs a vision fallback. Not in v1. |

## 9. Technology (locked)

| Item | Value |
|---|---|
| Language | **Go** |
| Transport | **ADB wireless over Tailscale** |
| Host | `.13`, Docker (or a systemd service — decide in Phase 1) |
| Container | **This project's own.** |
| Port | Allocate one (suggest **8793**) for the action API. |
| URL | none (no UI). A status page is optional later. |
| Auth | A bearer token for the Hub, from `environment.d`. |

## 10. Rules and constraints

1. **No LLM in the action path.** Recipes are deterministic.
2. **Find by text**, not fixed coordinates, where text exists.
3. **Never tap blindly.** If a step cannot find its target, fail.
4. Screen must be awake. Wake it first.
5. Secrets come from `environment.d`.
6. Log every action.
7. Own folder, own container, own port.
8. Do not put the Hub in the loop for the phone call itself. Calls are the Kotlin app.

## 11. Build order

**Phase 1 — Prove the ADB link**
Install ADB on `.13`. Pair the phone. Connect over Tailscale. Screenshot and dump the
UI tree.
*DoD:* the view tree of a real phone screen is fetched. **This is the risk phase —
prove it first.**

**Phase 2 — Act primitives**
Tap by text, type, scroll, back, home, launch an app.
*DoD:* Artemis opens a named app and taps a named element on it.

**Phase 3 — Spotify recipe**
Launch, search, play.
*DoD:* `spotify_play` with a real query plays the right song.

**Phase 4 — Song radio**
Open the menu and choose "song radio".
*DoD:* `spotify_radio` starts a real radio.

**Phase 5 — Maps recipe**
Open maps and search a place.
*DoD:* `maps_navigate` opens a real search.

**Phase 6 — Action API**
`POST /act`, bearer token, health, timeouts.
*DoD:* the Hub calls `/act` and the phone responds.

**Phase 7 — Resilience**
Wake the screen, reconnect after a drop, fail cleanly on a missing element, and
handle an app that stops consuming input (a memory-starved phone can deliver a
tap that the app never acts on — see `docs/phase-3-spotify-recipe.md`, attempts
9, 10 and 13).
*DoD:* a dropped ADB session recovers without a restart.

**Phase 8 — Documentation**
*DoD:* project-docs run; docs updated.

## 12. Acceptance criteria

- `spotify_play` plays the correct song from a spoken query, end to end.
- `spotify_radio` starts a radio.
- A missing element produces a clean failure, not a wrong tap.
- A dropped ADB session recovers automatically.
- The Hub can call `/act` and get a result.
- Every action is logged.

## 13. Decisions and remaining questions

Resolved:
1. **ADB over Tailscale** is the mechanism.
2. **No UI in v1.** A recipes file replaces a UI.
3. **Find by text** from the UI dump, not screenshots or vision.
4. **Partition:** native actions → Kotlin app; in-app navigation → Artemis.
5. **ADB tooling lives in the user profile on `.13`**, installed with
   `nix profile install nixpkgs#android-tools` — **not** a `nixos-rebuild`
   system module. Wireless ADB over TCP needs **no udev rules**; `programs.adb`
   was USB-only and is removed in this NixOS release anyway. Go is likewise run
   from a one-off `nix shell nixpkgs#go`, not installed system-wide. Never run
   `nixos-rebuild` on `.13`: it runs the Hub, MQTT and every service.
6. **Locked screen — Artemis fails loudly. It never unlocks the phone.** It
   detects a secure keyguard (PIN, pattern or password) before acting and
   refuses with `phone is locked; unlock it and retry`. It does not dismiss a
   secure keyguard and it does not type anything into one. A *non-secure*
   keyguard (the phone has no PIN at all) is still dismissed, because that
   screen guards nothing. **No PIN is stored, read or transmitted**, and there
   is no secret for one. **Storing a PIN for auto-unlock is rejected for now** —
   the phone is a pocket device, and Artemis is a convenience, not a reason to
   keep Sam's unlock credential in a service on `.13`. This is revisit-able if
   the daily friction turns out to be worse than the risk.

Remaining:
1. Which tasks after Spotify and maps? (WhatsApp message? Other music apps?)
2. Action API shape: one generic `/act`, or one endpoint per task?
3. **Service packaging** for Artemis on `.13` — container or systemd — is still
   open. ADB can be awkward in a container, so this is decided in Phase 6. The
   tooling-placement half of the old Q4 is resolved above.
4. Speed: what is an acceptable time for a 6-step recipe?
5. The recipes file has grown `guard`, `tap_if` and `wait_gone` verbs. If it
   keeps growing, decide whether the format stays bespoke or moves to YAML.