email_agent/README.md

Email Agent

Triage the family inbox from one page: all mail lands in a single Gmail account and this service classifies it, recommends actions, performs bulk actions, drafts replies, and answers the Hub.

The build spec is AGENTS.md. ORIGINAL_NOTES.md is archived pre-planning material and is not authoritative.

Status

Phase 4 — Triage actions. The Go skeleton, session login, and HTMX dashboard (Phase 1), real mail over IMAP with an app password, forced to IPv4 (Phase 2), JEV classification (Phase 3), and now bulk triage actions (Phase 4). The inbox has a select-all toolbar for delete, move, and mark (FR-6), and a folder view for the recoverable Delete/Saved/Replied-to Gmail labels (FR-9). A delete moves the message to a Gmail label (never a hard delete) and a row in the Delete folder can be restored to the inbox. Bulk delete needs a UI confirmation and a message classified Spam needs a second confirmation — spam is never auto-deleted (§10). Every action is appended to actions_log (FR-13). See docs/phase4-triage.md for the phase evidence.

Layout

cmd/email-agent/        service entry point (`-fetch`, `-classify`, `-classify-all`)
internal/classify/      JEV (TypeSafe System One) client, rule engine, orchestration
internal/config/        environment-driven configuration
internal/mailfetch/      stdlib-only IMAP client, MIME parsing, FR-2 source address
internal/mailops/       Gmail label moves and flag changes for triage actions
internal/session/       server-side session store + signed cookie values
internal/store/         SQLite open + schema (internal/store/schema.sql)
internal/server/        HTTP routes, auth middleware, HTMX handlers
internal/version/       build metadata (set via -ldflags)
web/static/             embedded design tokens, Inter fonts, htmx, app.css
web/templates/          embedded server-rendered HTML
Dockerfile              multi-stage, CGO-free build
docker-compose.yml      this project's own container and image

Configuration

All configuration is read from environment variables. Secrets live in ~/.config/environment.d/10-secrets.conf on the host and are passed into the container by docker-compose; no secret value is ever committed. See .env.example for the full list of names.

Key variables:

Variable	Default	Purpose
EMAIL_AGENT_ENV	development	development, production, or test
EMAIL_AGENT_ADDR	:8790	Listen address
EMAIL_AGENT_PORT	8790	Listen port (used when EMAIL_AGENT_ADDR is unset)
EMAIL_AGENT_DB_PATH	data/email_agent.db	SQLite file
EMAIL_AGENT_LOG_LEVEL	info	debug, info, warn, error
EMAIL_AGENT_SESSION_SECRET	(empty)	Session cookie signing key. If unset, an ephemeral key is used (sessions reset on restart)
EMAIL_AGENT_PASSWORD	(empty)	Login password for Sam. If unset, EMAIL_AGENT_SESSION_SECRET doubles as the password
GMAIL_USER	(empty)	Gmail account that receives the forwarded mail
GMAIL_APP_PASSWORD	(empty)	Gmail app password for IMAP/SMTP (no OAuth)
EMAIL_AGENT_IMAP_HOST	imap.gmail.com	IMAP host
EMAIL_AGENT_IMAP_PORT	993	IMAP port (implicit TLS)
EMAIL_AGENT_IMAP_LOOKBACK_DAYS	14	First-scan window
EMAIL_AGENT_IMAP_MAX_MESSAGES	100	Per-scan message cap
JEV_BASE_URL	https://api.typesafe.ai	TypeSafe System One base URL
JEV_API_KEY	(empty)	JEV API key (required to classify)
JEV_MODEL	jev-latest	System One model alias
EMAIL_AGENT_CLASSIFY_CONCURRENCY	4	In-flight JEV requests per run
EMAIL_AGENT_CLASSIFY_MAX_MESSAGES	200	Messages per classify run

Mail sync (Phase 2)

internal/mailfetch speaks the minimum IMAP needed: LOGIN, SELECT INBOX, UID SEARCH SINCE, and UID FETCH of the full RFC822 source plus Gmail's X-GM-MSGID/X-GM-THRID. The dialer uses tcp4 so Gmail is only ever reached over IPv4 — IPv6 to Gmail is broken from .13 and previously caused a socket EOF.

Trigger a scan either from the dashboard (Check mail) or from the CLI:

docker exec email-agent email-agent -fetch

Each scan is bounded by the lookback and max-message settings. Gmail's X-GM-MSGID is the stable message key; the IMAP UID and UIDVALIDITY are stored alongside it for later label operations.

Classification (Phase 3)

internal/classify sends each message to JEV (POST /v1/systemone) as three Choice questions — category, importance, suggested action — with the live category list as the category options, and stores the answer plus JEV's own confidence. One request per message, bounded concurrency, retry with backoff on 429/529/5xx.

Trigger classification from the dashboard (Classify for new mail, Re-classify all after a category or rules change) or from the CLI:

docker exec email-agent email-agent -classify      # pending messages
docker exec email-agent email-agent -classify-all  # every stored message

Rules (FR-4)

Rules are plain text, stored versioned in rules. One rule per line:

<condition> => <effects>

from contains "amazon.com" => category=Newsletter, action=unsubscribe
* => category=Info

Conditions join with and over from, to, subject, source, body using contains, equals, starts_with, ends_with. The first matching rule wins and overrides only the fields it names. The rules editor UI is Phase 6; the classifier already reads the stored text.

Categories (FR-14)

GET /categories edits the category list. A category's name is the key stored in classifications.category and the Choice option sent to JEV, so the model always sees the current list. Renaming cascades to existing classifications; removing a category reassigns its messages to the first remaining category.

Triage actions (Phase 4)

internal/mailops turns an app folder into a Gmail label and performs the move over IMAP, forcing IPv4 exactly like the read path:

App folder	Gmail label
inbox	INBOX
delete	Delete (recoverable; never a hard delete)
saved	Saved
replied	Replied-to/<address the mail arrived at>

A move uses the IMAP MOVE extension (with a COPY + \Deleted + EXPUNGE fallback). Because Gmail gives each label its own UID, a message is located by its stable X-GM-MSGID, not by a stored UID. A message in the Delete, Saved, or Replied-to folder has a Restore button that moves it back to INBOX.

Every action appends to actions_log: one move or mark row per message, plus one bulk_<action> summary row.

Run locally

go run ./cmd/email-agent
curl -s localhost:8790/health

Docker

docker compose up -d --build
curl -s localhost:8790/health

The container is email-agent, image email-agent:latest, named volume email_agent_data mounted at /data, listening on 8790. It runs as an unprivileged user and has a built-in healthcheck (email-agent -healthcheck). Caddy on .35 proxies mail.lab.audasmedia.com.au to this port.

Endpoints

Method	Path	Auth	Purpose
GET	/health	public	Liveness + database check, JSON
GET	/readyz	public	Readiness, JSON
GET	/login	public	Sign-in form (deep-indigo band)
POST	/login	public	CSRF-checked password sign-in, sets the session cookie
POST	/logout	session	Destroys the session, returns to /login
GET	/	session	Dashboard: nav counts, real message list
GET	/partials/inbox	session	HTMX fragment for the centre list
POST	/actions/fetch	session + CSRF	Runs one IMAP scan, returns the refreshed list
POST	/actions/classify	session + CSRF	Classifies pending messages, returns the refreshed list
POST	/actions/reclassify	session + CSRF	Re-classifies every message with the current categories and rules
POST	/actions/bulk	session + CSRF	Bulk delete/move/mark and restore over selected message ids (FR-6/FR-9)
GET	/categories	session	Category editor (FR-14)
POST	/categories/add	session + CSRF	Adds a category
POST	/categories/rename	session + CSRF	Renames a category and updates existing classifications
POST	/categories/remove	session + CSRF	Removes a category and reassigns its messages
GET	/static/…	public	Embedded tokens.css, app.css, Inter fonts, htmx.min.js

Unauthenticated requests to / and /partials/… redirect to /login?next=…. Form posts use a double-submit CSRF token (email_agent_csrf cookie plus a hidden csrf field).

Database

SQLite is opened with the pure-Go modernc.org/sqlite driver (no CGO). The schema in internal/store/schema.sql creates the AGENTS.md §7 tables: messages, classifications, rules, actions_log, unsubscribe, drafts, summaries, plus categories, schema_migrations. Schema init is idempotent and runs on every start. Migration v2 adds imap_uid and imap_uidvalidity to messages; migration v3 seeds the first category list and a starter rules version once, so a category removed by the user is not silently re-added; migration v4 adds folder and mailbox to messages for the Phase 4 triage folders.