docs: Phase 2 verification and deploy log (AUD-67)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
87
docs/phase2-verification.md
Normal file
87
docs/phase2-verification.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# Phase 2 verification — read mail (IMAP) on `.13`
|
||||
|
||||
Status: **done and deployed.** [AUD-67](/AUD/issues/AUD-67), commit `df3b6c9`.
|
||||
|
||||
## What Phase 2 added
|
||||
|
||||
- `internal/mailfetch`: a dependency-free, read-only IMAP client. It speaks
|
||||
`LOGIN`, `SELECT`, `UID SEARCH SINCE`, and `UID FETCH (UID X-GM-MSGID
|
||||
X-GM-THRID BODY.PEEK[])`, and it **dials `tcp4` only** — IPv6 to Gmail is
|
||||
broken from `.13` and previously produced a socket EOF (AGENTS.md §9).
|
||||
- MIME → text extraction (`text/plain` preferred, HTML stripped as a fallback,
|
||||
quoted-printable/base64 decoded) and RFC2047 header decoding.
|
||||
- FR-2 source-address resolution:
|
||||
`X-Pm-Original-To` → `To` → `Received … for <addr>` → `Return-Path`.
|
||||
`Delivered-To` is never consulted. The matched header is recorded in the fetch
|
||||
action log for audit.
|
||||
- SQLite storage via `UpsertMessage` (idempotent on Gmail's `X-GM-MSGID`), plus
|
||||
schema migration v2 adding `imap_uid` and `imap_uidvalidity`.
|
||||
- Dashboard and `/partials/inbox` render real mail, grouped by category, with a
|
||||
recommended action per row. Phase 3 (JEV) supplies real categories; until then
|
||||
everything is grouped under **Unclassified** with the action **Review**.
|
||||
- `POST /actions/fetch` (CSRF-checked) backs the dashboard **Check mail**
|
||||
button. `email-agent -fetch` runs one scan and exits — the n8n/CLI entry point
|
||||
(FR-1).
|
||||
|
||||
## Configuration
|
||||
|
||||
Secrets stay in `~/.config/environment.d/10-secrets.conf` and are passed into the
|
||||
container by compose. New names (all in [`.env.example`](../.env.example)):
|
||||
|
||||
| Variable | Default |
|
||||
|---|---|
|
||||
| `GMAIL_USER` | _(required for sync)_ |
|
||||
| `GMAIL_APP_PASSWORD` | _(required for sync)_ |
|
||||
| `EMAIL_AGENT_IMAP_HOST` | `imap.gmail.com` |
|
||||
| `EMAIL_AGENT_IMAP_PORT` | `993` |
|
||||
| `EMAIL_AGENT_IMAP_LOOKBACK_DAYS` | `14` |
|
||||
| `EMAIL_AGENT_IMAP_MAX_MESSAGES` | `100` |
|
||||
|
||||
## Deployment (executed 2026-10-11, AUD-67)
|
||||
|
||||
On `.13`, from the repo, with the host secrets in the environment:
|
||||
|
||||
```sh
|
||||
set -a; . ~/.config/environment.d/10-secrets.conf; set +a
|
||||
docker compose build
|
||||
docker compose up -d
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
The Phase 1 volume `email_agent_data` was kept; migration v2 added the IMAP
|
||||
columns in place. The container was recreated so it received `GMAIL_USER` and
|
||||
`GMAIL_APP_PASSWORD`.
|
||||
|
||||
## Verification results
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `docker compose ps` | `email-agent` up, healthy, `0.0.0.0:8790->8790/tcp` |
|
||||
| `GET /health` (public HTTPS) | `200 {"status":"ok","env":"production","database":"ok"}` |
|
||||
| `email-agent -fetch` (in container) | `fetched=100 stored=100 uidvalidity=3 since=2026-09-27` |
|
||||
| `GET /` after login (public HTTPS) | `200`; nav shows All mail / categories, **Check mail** button |
|
||||
| `GET /partials/inbox` | `200`, **100** message rows, grouped under `Unclassified` |
|
||||
| FR-2 on the real sample | `favrolfe@gmail.com → Hello@where-woof.com` stored with `source_address = Hello@where-woof.com` from `X-Pm-Original-To` |
|
||||
| Source address coverage | 100/100 messages have a non-empty `source_address`; 4 distinct addresses |
|
||||
| Fetch action log | `actions_log`: `fetch` … `uidvalidity=3 since=2026-09-27 fetched=100 stored=100 source_headers=To,X-Pm-Original-To` |
|
||||
| `POST /actions/fetch` without CSRF | `403` |
|
||||
| `POST /actions/fetch` with CSRF | `200`, note `Scanned 100 message(s); 100 stored or refreshed.` |
|
||||
| Idempotency | a second scan reported 100 stored and left 100 rows |
|
||||
|
||||
IPv4 was verified indirectly by success: the client only ever calls
|
||||
`DialContext(ctx, "tcp4", …)`, and Gmail over IPv6 from `.13` fails.
|
||||
|
||||
## Phase 2 Definition of Done (AGENTS.md §11)
|
||||
|
||||
- [x] The inbox shows real mail in the UI.
|
||||
- [x] FR-2 source address is read from `X-Pm-Original-To` (fallbacks `To`,
|
||||
`Received … for <addr>`, `Return-Path`; `Delivered-To` never used).
|
||||
|
||||
## Rollback
|
||||
|
||||
1. `docker compose down` on `.13` (keeps `email_agent_data`).
|
||||
2. `git revert df3b6c9` (or check out the previous image) and
|
||||
`docker compose up -d --build`.
|
||||
3. To roll back only the credential change, remove `GMAIL_*` from the host
|
||||
environment and recreate; the service still starts and the UI shows the empty
|
||||
state.
|
||||
Reference in New Issue
Block a user