email_agent/AGENTS.md

# Email Agent

> Build spec for Paperclip. Follow the phases in section 11 in order. Do not skip
> a phase's Definition of Done. When something is unclear, ask Sam — do not guess.

---

## Critical build callouts

Read these facts before you start a build phase. They already caused a
confusing failure once.

1. **Force IPv4 for every Gmail connection.** This applies to IMAP and SMTP.
   See §9 and the infrastructure notes. The first IMAP attempt failed with a
   socket EOF over IPv6 from `.13`.
2. **The source address header is `X-Pm-Original-To`** (added by PurelyMail).
   See FR-2 and §13.7. **Never use `Delivered-To`** — Gmail overwrites it.
3. **Every `docker run` in a heartbeat must be bounded and shell-safe.** The image has
   `ENTRYPOINT ["/usr/local/bin/email-agent"]`, so run shells with
   `timeout 10 docker run --rm --entrypoint sh email-agent:dev -c '<cmd>'`. Never issue an
   unbounded `docker run`; a bare `sh -c` becomes ARGUMENTS to the binary, starts the
   server, and hangs the run.

---

## 1. Purpose and outcome

A web app that triages the family inbox. All mail lands in one Gmail account
(`samuelrolfe@gmail.com`); PurelyMail forwards every domain address there.

The app sorts mail, recommends actions, performs them in bulk, drafts replies, and
answers questions from the Hub.

**Outcome (the single success statement):**
> Sam opens one page, clears the inbox in a few minutes, and knows nothing
> important was missed.

## 2. Scope

**In scope**
- Read mail from Gmail.
- Classify each message: category, importance, suggested action.
- Recommend: delete, unsubscribe, reply, file, keep.
- Bulk actions: delete, move, mark.
- AI reply drafts from templates and rules.
- A rules editor (text) that the classifier uses.
- A daily summary.
- Answer the Hub.

**Out of scope**
- Sending marketing or bulk mail.
- DNS, MX, SPF, DKIM. That is infrastructure, not this app.
- The other domains' websites.
- Any auto-send without review unless a rule explicitly allows it.

## 3. Users and roles

- **One user: Sam.** No multi-user. A session login protects the app.
- Adding a second user is out of scope.

## 4. Functional requirements

Each item is testable. Numbering is stable; do not renumber.

| ID | Requirement |
|---|---|
| FR-1 | Fetch new mail from Gmail with the Gmail API, on an n8n schedule. Store metadata and body. |
| FR-2 | Record which address the mail came through. Read **`X-Pm-Original-To`** (added by PurelyMail). Fallbacks, in order: `To`, the PurelyMail `Received ... for <address>` clause, `Return-Path`. |
| FR-3 | Classify each message with JEV: category, importance, suggested action, and a confidence. |
| FR-14 | Edit the category list in the UI: add, rename, remove. JEV uses the current list. |
| FR-4 | Apply user-written rules from the rules editor to override or refine the classification. |
| FR-5 | Show a triage list grouped by category, with the recommended action on each row. |
| FR-6 | Bulk actions: select all in a group, then delete, move, or mark. |
| FR-7 | Detect unsubscribe links. Keep an "ignore unsubscribe" list. Record what was unsubscribed. |
| FR-8 | Draft a reply from a template or rule with the LLM. Let Sam edit, then send. |
| FR-9 | Move messages to folders (Gmail labels): replied-to by address, a delete folder, and a saved folder. Deletes are recoverable. |
| FR-10 | Produce a daily summary and deliver it **by email**. |
| FR-11 | Rules editor: a plain-text area, saved, versioned. JEV and the LLM read it. |
| FR-12 | Expose the Hub API: `GET /capabilities`, `POST /ask` (read-only), `POST /notify`. |
| FR-13 | Log every action: what, when, which message, which rule. |
| FR-15 | Reply identity: send as the address the mail arrived at. Otherwise send as `Sam Rolfe`. |

## 5. Screens and UX flow

**Screens**
1. **Login** — session form. One dark indigo band (the design system's login moment).
2. **Inbox (main)** — three areas: left nav (categories and folders), centre list
   (grouped, with select-all and bulk actions), right detail pane.
3. **Message detail** — body, recommended action, reply draft, and history.
4. **Rules editor** — plain text, save, and a version list.
5. **Reports** — the daily summary, with delete-report.

**Main flow**
```
Login → Inbox → scan categories → bulk-delete junk → review "important"
      → approve or edit reply drafts → send → close
```

**Rules**
- Works on a phone browser. Touch targets ≥44px.
- No spinners on decorative elements. Subtle opacity pulse only.

## 6. Look and feel

Reuse the Family Home Lab design system. Do not invent a new one.

- Source: `/home/sam/Docker/Containers/family-home-lab/DESIGN.md`
- Tokens: `/home/sam/Docker/Containers/family-home-lab/portal/static/tokens.css`
- Copy `tokens.css` into this project. Do not import across projects.

Key rules: canvas `#f6f5f4`, one action blue `#0075de`, Inter, 8px grid, pill CTAs,
4px inputs, hairline borders, layered micro-shadows. Include dark mode.

## 7. Data model

SQLite, inside the service. Suggested tables:

| Table | Holds |
|---|---|
| `messages` | Gmail id, thread id, from, to, subject, date, body text, source address, received-at |
| `classifications` | message id, category, importance, action, confidence, model, created-at |
| `rules` | id, text, version, created-at |
| `actions_log` | message id, action, actor, rule id, created-at |
| `unsubscribe` | domain, status (`done`, `ignored`), created-at |
| `drafts` | message id, body, template id, status |
| `summaries` | date, text, delivered-to |

Retention: keep message bodies for **one month**. Logs are append-only and
size-capped.

## 8. Integrations

| System | Use |
|---|---|
| **Gmail IMAP + SMTP** | Read mail, move labels, send replies. App password (`GMAIL_USER`, `GMAIL_APP_PASSWORD`). **Force IPv4** — see section 9. |
| **PurelyMail** | Source of the forwarding. Read-only reference. Not required for v1. |
| **Amazon SES** | App/website mail only. Not for personal replies in v1. |
| **JEV** | Classification (typed category + confidence). HTTP API. |
| **OmniRoute** | All other models. |
| **n8n** | Triggers the scan on a schedule. |
| **Apprise** | Delivery of the daily summary. |
| **Hub** | `POST /ask` (read-only), `POST /notify`, `GET /capabilities`. |

The Hub interface is frozen in `central_ai_llm_master/docs/ask-contract.md`.
Build to it. `/ask` never changes state.

## 9. Technology (locked)

| Item | Value |
|---|---|
| Backend | **Go** |
| UI | **HTMX** + server-rendered HTML |
| Database | **SQLite** |
| Mail read | **IMAP** `imap.gmail.com:993` (SSL), **IPv4 only** |
| Mail send | **SMTP** `smtp.gmail.com:587` (STARTTLS), **IPv4 only** |
| Mail auth | **Gmail app password** from `environment.d`. No OAuth, no console. |
| Design | Family Home Lab `tokens.css` |
| Host | `.13`, Docker |
| Container | **This project's own** image and compose file. Not shared with any other project. |
| Folder | `/home/sam/paseo/projects/email_agent/` |
| Port | **8790** (chosen; container listens on 8790, published as `${EMAIL_AGENT_PORT:-8790}`). Confirm against infra-facts [AUD-60](/AUD/issues/AUD-60). |
| URL | `mail.lab.audasmedia.com.au` via Caddy on `.35` |
| Auth | **Separate session login.** Not the console SSO. |

## 10. Rules and constraints

1. All models go through **OmniRoute**, except JEV.
2. Secrets come from `~/.config/environment.d/10-secrets.conf`. Never commit them.
3. `/ask` is read-only. It never changes state.
4. Sending a reply or deleting in bulk needs Sam's action in the UI. No auto-send
   unless a rule explicitly allows it.
5. **Spam is recommended, never auto-deleted.** Important mail sometimes looks like
   spam. Confirm every spam action.
6. Deletes are recoverable. Move to a delete folder or trash, never hard-delete.
6. Replies return to the source and stay short.
7. Every action is logged.
8. Do not merge this service into another. Own folder, own container, own port.

## 11. Build order

**STATUS — ALL 8 PHASES DONE (checked 2026-10-14). This project is COMPLETE.**

- [x] Phase 1 — Skeleton
- [x] Phase 2 — Read mail
- [x] Phase 3 — Classify
- [x] Phase 4 — Triage actions
- [x] Phase 5 — Reply drafts
- [x] Phase 6 — Unsubscribe and rules editor
- [x] Phase 7 — Summary and Hub
- [x] Phase 8 — Documentation

Re-open this list if new work is added. An empty Paperclip board must mean
"finished", never "the agent stopped".

Each phase ends with a Definition of Done (DoD). Do not start the next phase until
the DoD passes.

**Phase 1 — Skeleton**
Go service, SQLite, config, session login, Dockerfile, compose, Caddy route.
*DoD:* the app runs on `.13`, login works, an empty dashboard renders, `/health` returns ok.

**Phase 2 — Read mail**
Connect to Gmail over **IMAP** with the app password. **Force IPv4.** Fetch, store,
and list real messages.
*DoD:* the inbox shows real mail. Confirm the source address from FR-2 against a
real forwarded message (see section 13).

**Phase 3 — Classify**
JEV classification plus the rules table.
*DoD:* every message has a category, importance, and suggested action.

**Phase 4 — Triage actions**
Delete, move, mark; bulk select.
*DoD:* select-all then delete works, and the message is recoverable.

**Phase 5 — Reply drafts**
Templates, LLM draft, edit, send.
*DoD:* a reply is drafted, edited, and sent, and the thread updates.

**Phase 6 — Unsubscribe and rules editor**
*DoD:* a rule written in the editor changes the classification of a real message.

**Phase 7 — Summary and Hub**
Daily summary; `/capabilities`, `/ask`, `/notify`.
*DoD:* the Hub asks "anything important in my inbox?" and gets a correct short answer.

**Phase 8 — Documentation**
*DoD:* project-docs run; Gitea, Obsidian, Outline, Vikunja, and the map are updated.

## 12. Acceptance criteria

The project is done when:

- Sam can log in from a phone browser.
- Real mail appears, grouped, with the source address shown.
- Bulk delete works and is recoverable.
- A reply can be drafted, edited, and sent without leaving the app.
- The rules editor changes real classifications.
- The Hub gets a correct short answer to "anything important in my inbox?".
- Every action is in the log.
- The app runs in its own container on its own port, behind Caddy.

## 13. Decisions and remaining questions

Resolved:

1. **Reply identity:** the address the mail arrived at. Otherwise `Sam Rolfe`.
2. **Categories:** `action`, `important`, `info`, `spam`, `newsletter`,
   `auto-reply`. **Editable in the UI** (FR-14).
3. **Spam:** recommend and confirm. Never auto-delete.
4. **Daily summary:** by email.
5. **Retention:** one month of message bodies.
6. **Mail access is IMAP + app password** — no console, no OAuth. Working on `.13`
   and `.51`.
7. **Source address is `X-Pm-Original-To`.** Confirmed on a real forwarded message
   (`X-Pm-Original-To: Hello@where-woof.com`). PurelyMail adds this header, so
   Gmail's overwritten `Delivered-To` does not matter. See FR-2.
8. **PurelyMail forwarding is live**, confirmed end to end.

Remaining:

1. **Send-as:** reported as already set up in Gmail. Not verifiable from the sent
   folder — no message has yet been sent from a domain address. Confirm at Phase 5.

### Infrastructure notes (found during setup)

- **IPv6 to Gmail is broken from `.13`.** The first IMAP attempt failed with a
  socket EOF over IPv6. Forcing IPv4 works. **The agent must force IPv4** for IMAP
  and SMTP.
- Credentials live in `~/.config/environment.d/10-secrets.conf` on `.13` and `.51`,
  as `GMAIL_USER` and `GMAIL_APP_PASSWORD`.
- **Header fields seen on a real forwarded message** (id 619, from `favrolfe@gmail.com`):
  ```
  X-Pm-Original-To: Hello@where-woof.com   <- use this
  To: Hello@where-woof.com
  Return-Path: <Hello@where-woof.com>
  Delivered-To: samuelrolfe@gmail.com       <- Gmail overwrites this; ignore
  ```