Phase 2: Gmail IMAP fetch (IPv4), FR-2 source address, real mail in dashboard (AUD-67)
- internal/mailfetch: stdlib-only read-only IMAP client (LOGIN, SELECT, UID SEARCH, UID FETCH) that always dials tcp4; IPv6 to Gmail is broken from .13 and caused a socket EOF. - MIME parsing to text, RFC2047 header decoding, FR-2 source address: X-Pm-Original-To -> To -> Received 'for <addr>' -> Return-Path, never Delivered-To. The matched header is recorded for audit. - store: Message upsert/list/count + LogAction; schema migration v2 adds imap_uid and imap_uidvalidity. - server: dashboard and /partials/inbox render real mail grouped by category with a recommended action; POST /actions/fetch runs a scan (CSRF-checked); nav counts; 'Check mail' button. - config: GMAIL_USER, GMAIL_APP_PASSWORD, IMAP host/port/lookback/max. - cmd: -fetch runs one scan and exits (n8n/CLI entry point, FR-1). - compose/.env.example expose the Gmail + IMAP settings. - tests: FR-2 precedence, MIME bodies, IMAP literal/parser, store upsert/migration, inbox rendering. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
43
README.md
43
README.md
@@ -9,16 +9,20 @@ pre-planning material and is not authoritative.
|
||||
|
||||
## Status
|
||||
|
||||
**Phase 1.2 — Session login + empty dashboard.** The Go skeleton (Phase 1.1)
|
||||
plus a single-user session login, a copied Family Home Lab `tokens.css`, and an
|
||||
HTMX dashboard shell (left nav, centre list, right detail pane) with dark mode.
|
||||
Mail fetch, classification, and the rest come in later phases.
|
||||
**Phase 2 — Read mail (IMAP).** The Go skeleton (Phase 1.1) plus session login
|
||||
and the HTMX dashboard shell (Phase 1.2), now fetching real mail from Gmail over
|
||||
**IMAP with an app password, forced to IPv4** (AGENTS.md §9). Messages are stored
|
||||
in SQLite and listed in the dashboard, grouped by category with a recommended
|
||||
action per row. The FR-2 source address is resolved from `X-Pm-Original-To`, with
|
||||
`To`, the PurelyMail `Received … for` clause, and `Return-Path` as fallbacks;
|
||||
`Delivered-To` is never used. Classification and triage actions come in Phase 3+.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
cmd/email-agent/ service entry point
|
||||
cmd/email-agent/ service entry point (`-fetch` runs one scan and exits)
|
||||
internal/config/ environment-driven configuration
|
||||
internal/mailfetch/ stdlib-only IMAP client, MIME parsing, FR-2 source address
|
||||
internal/session/ server-side session store + signed cookie values
|
||||
internal/store/ SQLite open + schema (internal/store/schema.sql)
|
||||
internal/server/ HTTP routes, auth middleware, HTMX handlers
|
||||
@@ -47,6 +51,29 @@ Key variables:
|
||||
| `EMAIL_AGENT_LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
|
||||
| `EMAIL_AGENT_SESSION_SECRET` | _(empty)_ | Session cookie signing key. If unset, an ephemeral key is used (sessions reset on restart) |
|
||||
| `EMAIL_AGENT_PASSWORD` | _(empty)_ | Login password for Sam. If unset, `EMAIL_AGENT_SESSION_SECRET` doubles as the password |
|
||||
| `GMAIL_USER` | _(empty)_ | Gmail account that receives the forwarded mail |
|
||||
| `GMAIL_APP_PASSWORD` | _(empty)_ | Gmail app password for IMAP/SMTP (no OAuth) |
|
||||
| `EMAIL_AGENT_IMAP_HOST` | `imap.gmail.com` | IMAP host |
|
||||
| `EMAIL_AGENT_IMAP_PORT` | `993` | IMAP port (implicit TLS) |
|
||||
| `EMAIL_AGENT_IMAP_LOOKBACK_DAYS` | `14` | First-scan window |
|
||||
| `EMAIL_AGENT_IMAP_MAX_MESSAGES` | `100` | Per-scan message cap |
|
||||
|
||||
## Mail sync (Phase 2)
|
||||
|
||||
`internal/mailfetch` speaks the minimum IMAP needed: `LOGIN`, `SELECT INBOX`,
|
||||
`UID SEARCH SINCE`, and `UID FETCH` of the full RFC822 source plus Gmail's
|
||||
`X-GM-MSGID`/`X-GM-THRID`. The dialer uses `tcp4` so Gmail is only ever reached
|
||||
over IPv4 — IPv6 to Gmail is broken from `.13` and previously caused a socket EOF.
|
||||
|
||||
Trigger a scan either from the dashboard (**Check mail**) or from the CLI:
|
||||
|
||||
```sh
|
||||
docker exec email-agent email-agent -fetch
|
||||
```
|
||||
|
||||
Each scan is bounded by the lookback and max-message settings. Gmail's
|
||||
`X-GM-MSGID` is the stable message key; the IMAP `UID` and `UIDVALIDITY` are
|
||||
stored alongside it for later label operations.
|
||||
|
||||
## Run locally
|
||||
|
||||
@@ -76,8 +103,9 @@ Caddy on `.35` proxies `mail.lab.audasmedia.com.au` to this port.
|
||||
| `GET` | `/login` | public | Sign-in form (deep-indigo band) |
|
||||
| `POST` | `/login` | public | CSRF-checked password sign-in, sets the session cookie |
|
||||
| `POST` | `/logout` | session | Destroys the session, returns to `/login` |
|
||||
| `GET` | `/` | session | Empty dashboard shell |
|
||||
| `GET` | `/` | session | Dashboard: nav counts, real message list |
|
||||
| `GET` | `/partials/inbox` | session | HTMX fragment for the centre list |
|
||||
| `POST` | `/actions/fetch` | session + CSRF | Runs one IMAP scan, returns the refreshed list |
|
||||
| `GET` | `/static/…` | public | Embedded `tokens.css`, `app.css`, Inter fonts, `htmx.min.js` |
|
||||
|
||||
Unauthenticated requests to `/` and `/partials/…` redirect to `/login?next=…`.
|
||||
@@ -90,4 +118,5 @@ SQLite is opened with the pure-Go `modernc.org/sqlite` driver (no CGO). The
|
||||
schema in `internal/store/schema.sql` creates the AGENTS.md §7 tables:
|
||||
`messages`, `classifications`, `rules`, `actions_log`, `unsubscribe`, `drafts`,
|
||||
`summaries`, plus `schema_migrations`. Schema init is idempotent and runs on
|
||||
every start.
|
||||
every start. Migration v2 adds `imap_uid` and `imap_uidvalidity` to `messages`
|
||||
for databases created by Phase 1.
|
||||
|
||||
Reference in New Issue
Block a user