email_agent/README.md

# Email Agent

Triage the family inbox from one page: all mail lands in a single Gmail account
and this service classifies it, recommends actions, performs bulk actions, drafts
replies, and answers the Hub.

The build spec is [`AGENTS.md`](AGENTS.md). `ORIGINAL_NOTES.md` is archived
pre-planning material and is not authoritative.

## Status

**Phase 4 — Triage actions.** The Go skeleton, session login, and HTMX
dashboard (Phase 1), real mail over **IMAP with an app password, forced to IPv4**
(Phase 2), **JEV classification** (Phase 3), and now **bulk triage actions**
(Phase 4). The inbox has a select-all toolbar for delete, move, and mark (FR-6),
and a folder view for the recoverable Delete/Saved/Replied-to Gmail labels (FR-9).
A delete moves the message to a Gmail label (never a hard delete) and a row in
the Delete folder can be **restored** to the inbox. Bulk delete needs a UI
confirmation and a message classified Spam needs a second confirmation — spam is
never auto-deleted (§10). Every action is appended to `actions_log` (FR-13). See
[`docs/phase4-triage.md`](docs/phase4-triage.md) for the phase evidence.

## Layout

```
cmd/email-agent/        service entry point (`-fetch`, `-classify`, `-classify-all`)
internal/classify/      JEV (TypeSafe System One) client, rule engine, orchestration
internal/config/        environment-driven configuration
internal/mailfetch/      stdlib-only IMAP client, MIME parsing, FR-2 source address
internal/mailops/       Gmail label moves and flag changes for triage actions
internal/session/       server-side session store + signed cookie values
internal/store/         SQLite open + schema (internal/store/schema.sql)
internal/server/        HTTP routes, auth middleware, HTMX handlers
internal/version/       build metadata (set via -ldflags)
web/static/             embedded design tokens, Inter fonts, htmx, app.css
web/templates/          embedded server-rendered HTML
Dockerfile              multi-stage, CGO-free build
docker-compose.yml      this project's own container and image
```

## Configuration

All configuration is read from environment variables. Secrets live in
`~/.config/environment.d/10-secrets.conf` on the host and are passed into the
container by `docker-compose`; no secret value is ever committed. See
[`.env.example`](.env.example) for the full list of names.

Key variables:

| Variable | Default | Purpose |
|---|---|---|
| `EMAIL_AGENT_ENV` | `development` | `development`, `production`, or `test` |
| `EMAIL_AGENT_ADDR` | `:8790` | Listen address |
| `EMAIL_AGENT_PORT` | `8790` | Listen port (used when `EMAIL_AGENT_ADDR` is unset) |
| `EMAIL_AGENT_DB_PATH` | `data/email_agent.db` | SQLite file |
| `EMAIL_AGENT_LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
| `EMAIL_AGENT_SESSION_SECRET` | _(empty)_ | Session cookie signing key. If unset, an ephemeral key is used (sessions reset on restart) |
| `EMAIL_AGENT_PASSWORD` | _(empty)_ | Login password for Sam. If unset, `EMAIL_AGENT_SESSION_SECRET` doubles as the password |
| `GMAIL_USER` | _(empty)_ | Gmail account that receives the forwarded mail |
| `GMAIL_APP_PASSWORD` | _(empty)_ | Gmail app password for IMAP/SMTP (no OAuth) |
| `EMAIL_AGENT_IMAP_HOST` | `imap.gmail.com` | IMAP host |
| `EMAIL_AGENT_IMAP_PORT` | `993` | IMAP port (implicit TLS) |
| `EMAIL_AGENT_IMAP_LOOKBACK_DAYS` | `14` | First-scan window |
| `EMAIL_AGENT_IMAP_MAX_MESSAGES` | `100` | Per-scan message cap |
| `JEV_BASE_URL` | `https://api.typesafe.ai` | TypeSafe System One base URL |
| `JEV_API_KEY` | _(empty)_ | JEV API key (required to classify) |
| `JEV_MODEL` | `jev-latest` | System One model alias |
| `EMAIL_AGENT_CLASSIFY_CONCURRENCY` | `4` | In-flight JEV requests per run |
| `EMAIL_AGENT_CLASSIFY_MAX_MESSAGES` | `200` | Messages per classify run |

## Mail sync (Phase 2)

`internal/mailfetch` speaks the minimum IMAP needed: `LOGIN`, `SELECT INBOX`,
`UID SEARCH SINCE`, and `UID FETCH` of the full RFC822 source plus Gmail's
`X-GM-MSGID`/`X-GM-THRID`. The dialer uses `tcp4` so Gmail is only ever reached
over IPv4 — IPv6 to Gmail is broken from `.13` and previously caused a socket EOF.

Trigger a scan either from the dashboard (**Check mail**) or from the CLI:

```sh
docker exec email-agent email-agent -fetch
```

Each scan is bounded by the lookback and max-message settings. Gmail's
`X-GM-MSGID` is the stable message key; the IMAP `UID` and `UIDVALIDITY` are
stored alongside it for later label operations.

## Classification (Phase 3)

`internal/classify` sends each message to JEV (`POST /v1/systemone`) as three
Choice questions — category, importance, suggested action — with the **live**
category list as the category options, and stores the answer plus JEV's own
confidence. One request per message, bounded concurrency, retry with backoff on
`429`/`529`/`5xx`.

Trigger classification from the dashboard (**Classify** for new mail,
**Re-classify all** after a category or rules change) or from the CLI:

```sh
docker exec email-agent email-agent -classify      # pending messages
docker exec email-agent email-agent -classify-all  # every stored message
```

### Rules (FR-4)

Rules are plain text, stored versioned in `rules`. One rule per line:

```
<condition> => <effects>

from contains "amazon.com" => category=Newsletter, action=unsubscribe
* => category=Info
```

Conditions join with ` and ` over `from`, `to`, `subject`, `source`, `body`
using `contains`, `equals`, `starts_with`, `ends_with`. The first matching rule
wins and overrides only the fields it names. The rules editor UI is Phase 6; the
classifier already reads the stored text.

### Categories (FR-14)

`GET /categories` edits the category list. A category's name is the key stored
in `classifications.category` and the Choice option sent to JEV, so the model
always sees the current list. Renaming cascades to existing classifications;
removing a category reassigns its messages to the first remaining category.

## Triage actions (Phase 4)

`internal/mailops` turns an app folder into a Gmail label and performs the move
over IMAP, forcing IPv4 exactly like the read path:

| App folder | Gmail label |
|---|---|
| `inbox` | `INBOX` |
| `delete` | `Delete` (recoverable; never a hard delete) |
| `saved` | `Saved` |
| `replied` | `Replied-to/<address the mail arrived at>` |

A move uses the IMAP `MOVE` extension (with a `COPY` + `\Deleted` + `EXPUNGE`
fallback). Because Gmail gives each label its own `UID`, a message is located by
its stable `X-GM-MSGID`, not by a stored `UID`. A message in the Delete, Saved,
or Replied-to folder has a **Restore** button that moves it back to `INBOX`.

Every action appends to `actions_log`: one `move` or `mark` row per message,
plus one `bulk_<action>` summary row.

## Run locally

```sh
go run ./cmd/email-agent
curl -s localhost:8790/health
```

## Docker

```sh
docker compose up -d --build
curl -s localhost:8790/health
```

The container is `email-agent`, image `email-agent:latest`, named volume
`email_agent_data` mounted at `/data`, listening on **8790**. It runs as an
unprivileged user and has a built-in healthcheck (`email-agent -healthcheck`).
Caddy on `.35` proxies `mail.lab.audasmedia.com.au` to this port.

## Endpoints

| Method | Path | Auth | Purpose |
|---|---|---|---|
| `GET` | `/health` | public | Liveness + database check, JSON |
| `GET` | `/readyz` | public | Readiness, JSON |
| `GET` | `/login` | public | Sign-in form (deep-indigo band) |
| `POST` | `/login` | public | CSRF-checked password sign-in, sets the session cookie |
| `POST` | `/logout` | session | Destroys the session, returns to `/login` |
| `GET` | `/` | session | Dashboard: nav counts, real message list |
| `GET` | `/partials/inbox` | session | HTMX fragment for the centre list |
| `POST` | `/actions/fetch` | session + CSRF | Runs one IMAP scan, returns the refreshed list |
| `POST` | `/actions/classify` | session + CSRF | Classifies pending messages, returns the refreshed list |
| `POST` | `/actions/reclassify` | session + CSRF | Re-classifies every message with the current categories and rules |
| `POST` | `/actions/bulk` | session + CSRF | Bulk delete/move/mark and restore over selected message ids (FR-6/FR-9) |
| `GET` | `/categories` | session | Category editor (FR-14) |
| `POST` | `/categories/add` | session + CSRF | Adds a category |
| `POST` | `/categories/rename` | session + CSRF | Renames a category and updates existing classifications |
| `POST` | `/categories/remove` | session + CSRF | Removes a category and reassigns its messages |
| `GET` | `/static/…` | public | Embedded `tokens.css`, `app.css`, Inter fonts, `htmx.min.js` |

Unauthenticated requests to `/` and `/partials/…` redirect to `/login?next=…`.
Form posts use a double-submit CSRF token (`email_agent_csrf` cookie plus a
hidden `csrf` field).

## Database

SQLite is opened with the pure-Go `modernc.org/sqlite` driver (no CGO). The
schema in `internal/store/schema.sql` creates the AGENTS.md §7 tables:
`messages`, `classifications`, `rules`, `actions_log`, `unsubscribe`, `drafts`,
`summaries`, plus `categories`, `schema_migrations`. Schema init is idempotent
and runs on every start. Migration v2 adds `imap_uid` and `imap_uidvalidity` to
`messages`; migration v3 seeds the first category list and a starter rules
version once, so a category removed by the user is not silently re-added;
migration v4 adds `folder` and `mailbox` to `messages` for the Phase 4 triage
folders.