diff --git a/docs/phase2-verification.md b/docs/phase2-verification.md new file mode 100644 index 0000000..2ff7dcd --- /dev/null +++ b/docs/phase2-verification.md @@ -0,0 +1,87 @@ +# Phase 2 verification — read mail (IMAP) on `.13` + +Status: **done and deployed.** [AUD-67](/AUD/issues/AUD-67), commit `df3b6c9`. + +## What Phase 2 added + +- `internal/mailfetch`: a dependency-free, read-only IMAP client. It speaks + `LOGIN`, `SELECT`, `UID SEARCH SINCE`, and `UID FETCH (UID X-GM-MSGID + X-GM-THRID BODY.PEEK[])`, and it **dials `tcp4` only** — IPv6 to Gmail is + broken from `.13` and previously produced a socket EOF (AGENTS.md §9). +- MIME → text extraction (`text/plain` preferred, HTML stripped as a fallback, + quoted-printable/base64 decoded) and RFC2047 header decoding. +- FR-2 source-address resolution: + `X-Pm-Original-To` → `To` → `Received … for ` → `Return-Path`. + `Delivered-To` is never consulted. The matched header is recorded in the fetch + action log for audit. +- SQLite storage via `UpsertMessage` (idempotent on Gmail's `X-GM-MSGID`), plus + schema migration v2 adding `imap_uid` and `imap_uidvalidity`. +- Dashboard and `/partials/inbox` render real mail, grouped by category, with a + recommended action per row. Phase 3 (JEV) supplies real categories; until then + everything is grouped under **Unclassified** with the action **Review**. +- `POST /actions/fetch` (CSRF-checked) backs the dashboard **Check mail** + button. `email-agent -fetch` runs one scan and exits — the n8n/CLI entry point + (FR-1). + +## Configuration + +Secrets stay in `~/.config/environment.d/10-secrets.conf` and are passed into the +container by compose. New names (all in [`.env.example`](../.env.example)): + +| Variable | Default | +|---|---| +| `GMAIL_USER` | _(required for sync)_ | +| `GMAIL_APP_PASSWORD` | _(required for sync)_ | +| `EMAIL_AGENT_IMAP_HOST` | `imap.gmail.com` | +| `EMAIL_AGENT_IMAP_PORT` | `993` | +| `EMAIL_AGENT_IMAP_LOOKBACK_DAYS` | `14` | +| `EMAIL_AGENT_IMAP_MAX_MESSAGES` | `100` | + +## Deployment (executed 2026-10-11, AUD-67) + +On `.13`, from the repo, with the host secrets in the environment: + +```sh +set -a; . ~/.config/environment.d/10-secrets.conf; set +a +docker compose build +docker compose up -d +docker compose ps +``` + +The Phase 1 volume `email_agent_data` was kept; migration v2 added the IMAP +columns in place. The container was recreated so it received `GMAIL_USER` and +`GMAIL_APP_PASSWORD`. + +## Verification results + +| Check | Result | +|---|---| +| `docker compose ps` | `email-agent` up, healthy, `0.0.0.0:8790->8790/tcp` | +| `GET /health` (public HTTPS) | `200 {"status":"ok","env":"production","database":"ok"}` | +| `email-agent -fetch` (in container) | `fetched=100 stored=100 uidvalidity=3 since=2026-09-27` | +| `GET /` after login (public HTTPS) | `200`; nav shows All mail / categories, **Check mail** button | +| `GET /partials/inbox` | `200`, **100** message rows, grouped under `Unclassified` | +| FR-2 on the real sample | `favrolfe@gmail.com → Hello@where-woof.com` stored with `source_address = Hello@where-woof.com` from `X-Pm-Original-To` | +| Source address coverage | 100/100 messages have a non-empty `source_address`; 4 distinct addresses | +| Fetch action log | `actions_log`: `fetch` … `uidvalidity=3 since=2026-09-27 fetched=100 stored=100 source_headers=To,X-Pm-Original-To` | +| `POST /actions/fetch` without CSRF | `403` | +| `POST /actions/fetch` with CSRF | `200`, note `Scanned 100 message(s); 100 stored or refreshed.` | +| Idempotency | a second scan reported 100 stored and left 100 rows | + +IPv4 was verified indirectly by success: the client only ever calls +`DialContext(ctx, "tcp4", …)`, and Gmail over IPv6 from `.13` fails. + +## Phase 2 Definition of Done (AGENTS.md §11) + +- [x] The inbox shows real mail in the UI. +- [x] FR-2 source address is read from `X-Pm-Original-To` (fallbacks `To`, + `Received … for `, `Return-Path`; `Delivered-To` never used). + +## Rollback + +1. `docker compose down` on `.13` (keeps `email_agent_data`). +2. `git revert df3b6c9` (or check out the previous image) and + `docker compose up -d --build`. +3. To roll back only the credential change, remove `GMAIL_*` from the host + environment and recreate; the service still starts and the UI shows the empty + state.