Email Agent

Triage the family inbox from one page: all mail lands in a single Gmail account and this service classifies it, recommends actions, performs bulk actions, drafts replies, and answers the Hub.

The build spec is AGENTS.md. ORIGINAL_NOTES.md is archived pre-planning material and is not authoritative.

Architecture

flowchart LR
    SAM["Sam<br/>phone or desktop browser"] --> CADDY["Caddy on .35<br/>mail.lab.audasmedia.com.au"]
    CADDY --> UI["HTMX dashboard<br/>login, inbox, rules, reports"]
    UI --> DB[("SQLite<br/>email_agent_data")]
    UI --> GMAIL["Gmail<br/>IMAP :993 / SMTP :587 (IPv4)"]
    UI --> JEV["JEV / TypeSafe<br/>classify"]
    UI --> OMNI["OmniRoute :20129<br/>draft reply"]
    UI --> APPRISE["Apprise :8210<br/>summary email"]
    HUB["Central AI Hub :8788"] --> API["Hub API<br/>/capabilities /ask /notify"]
    API --> DB
    N8N["n8n schedule"] --> JOBS["CLI jobs<br/>-fetch -classify -summary-send"]
    JOBS --> DB

Sources: docs/email-agent-architecture.mmd and docs/email-agent-map.html. Interactive map: https://maps.lab.audasmedia.com.au/email_agent/docs/.

Status

Complete and deployed. All eight AGENTS.md §11 phases are done. The Go skeleton, session login, and HTMX dashboard (Phase 1), real mail over IMAP with an app password, forced to IPv4 (Phase 2), JEV classification (Phase 3), bulk triage actions (Phase 4), reply drafts (Phase 5), unsubscribe detection and the rules editor (Phase 6), and the daily summary by email plus the read-only Hub API (Phase 7) run on .13 behind Caddy. Phase 8 documents the project across Gitea, Obsidian, Outline, Vikunja, and the architecture map.

Open a message to read it, draft a reply with the LLM through OmniRoute from a template, edit it, and send it over Gmail SMTP (IPv4 + STARTTLS). A reply is sent as the address the mail arrived at (X-Pm-Original-To, FR-15) and returns to the original sender; the reply sets In-Reply-To/References so Gmail keeps the thread, and the original is filed under its Replied-to/<address> label. Sending is a deliberate UI action with a confirmation — there is no auto-send (§10). The rules editor (/rules) saves and versions the plain-text rules that JEV and the reply model read. Unsubscribe detection (/unsubscribe) finds RFC 2369/8058 links and body links, keeps a domain ignore list, and records what Sam marks — it never follows a link. Every action, including drafts, sends, rules edits, and unsubscribe marks, is appended to actions_log (FR-13). Phase 7 adds the daily summary (-summary, -summary-send; delivered by email through Apprise) and the read-only Hub API (/capabilities, /ask, /notify). Retention (AGENTS.md §7) is enforced by the bounded -retain job: it clears message bodies older than one month, keeps metadata, and caps the append-only actions_log; see docs/retention.md. Phase evidence: docs/phase7-summary-hub.md, docs/phase6-unsubscribe.md, docs/phase5-replies.md, docs/phase4-triage.md, and docs/phase8-documentation.md.

Layout

cmd/email-agent/        service entry point (`-fetch`, `-classify`, `-classify-all`, `-retain`)
internal/classify/      JEV (TypeSafe System One) client, rule engine, orchestration
internal/config/        environment-driven configuration
internal/mailfetch/      stdlib-only IMAP client, MIME parsing, FR-2 source address
internal/mailops/       Gmail label moves and flag changes, and SMTP reply sending
internal/reply/          LLM reply drafts through OmniRoute, reply templates
internal/session/       server-side session store + signed cookie values
internal/store/         SQLite open + schema (internal/store/schema.sql), retention
internal/server/        HTTP routes, auth middleware, HTMX handlers
internal/unsubscribe/   RFC 2369/8058 + body unsubscribe-link detection (no network)
internal/version/       build metadata (set via -ldflags)
web/static/             embedded design tokens, Inter fonts, htmx, app.css
web/templates/          embedded server-rendered HTML
docs/                   phase evidence, retention, AI context, map, diagrams
docs/outline/           AI-context documents (system, config, code, runbook)
docs/archify/           Archify JSON spec for the architecture map
docs/diagrams/          rendered Mermaid PNG
Dockerfile              multi-stage, CGO-free build
docker-compose.yml      this project's own container and image

Configuration

All configuration is read from environment variables. Secrets live in ~/.config/environment.d/10-secrets.conf on the host and are passed into the container by docker-compose; no secret value is ever committed. See .env.example for the full list of names.

Key variables:

Variable	Default	Purpose
EMAIL_AGENT_ENV	development	development, production, or test
EMAIL_AGENT_ADDR	:8790	Listen address
EMAIL_AGENT_PORT	8790	Listen port (used when EMAIL_AGENT_ADDR is unset)
EMAIL_AGENT_DB_PATH	data/email_agent.db	SQLite file
EMAIL_AGENT_LOG_LEVEL	info	debug, info, warn, error
EMAIL_AGENT_SESSION_SECRET	(empty)	Session cookie signing key. If unset, an ephemeral key is used (sessions reset on restart)
EMAIL_AGENT_PASSWORD	(empty)	Login password for Sam. If unset, EMAIL_AGENT_SESSION_SECRET doubles as the password
GMAIL_USER	(empty)	Gmail account that receives the forwarded mail
GMAIL_APP_PASSWORD	(empty)	Gmail app password for IMAP/SMTP (no OAuth)
EMAIL_AGENT_SMTP_HOST	smtp.gmail.com	SMTP submission host
EMAIL_AGENT_SMTP_PORT	587	SMTP submission port (STARTTLS, IPv4)
EMAIL_AGENT_IMAP_HOST	imap.gmail.com	IMAP host
EMAIL_AGENT_IMAP_PORT	993	IMAP port (implicit TLS)
EMAIL_AGENT_IMAP_LOOKBACK_DAYS	14	First-scan window
EMAIL_AGENT_IMAP_MAX_MESSAGES	100	Per-scan message cap
JEV_BASE_URL	https://api.typesafe.ai	TypeSafe System One base URL
JEV_API_KEY	(empty)	JEV API key (required to classify)
JEV_MODEL	jev-latest	System One model alias
EMAIL_AGENT_CLASSIFY_CONCURRENCY	4	In-flight JEV requests per run
EMAIL_AGENT_CLASSIFY_MAX_MESSAGES	200	Messages per classify run
OMNIROUTE_BASE_URL	http://192.168.20.13:20129/v1	OmniRoute gateway (all non-JEV models)
OMNIROUTE_API_KEY	(empty)	OmniRoute API key (required to draft replies)
OMNIROUTE_MODEL	auto/best-chat	Model alias for reply drafts
EMAIL_AGENT_RETENTION_DAYS	30	Message-body retention window in days (§7)
EMAIL_AGENT_ACTIONS_LOG_MAX_ROWS	50000	Newest actions_log rows kept

Mail sync (Phase 2)

internal/mailfetch speaks the minimum IMAP needed: LOGIN, SELECT INBOX, UID SEARCH SINCE, and UID FETCH of the full RFC822 source plus Gmail's X-GM-MSGID/X-GM-THRID. The dialer uses tcp4 so Gmail is only ever reached over IPv4 — IPv6 to Gmail is broken from .13 and previously caused a socket EOF.

Trigger a scan either from the dashboard (Check mail) or from the CLI:

docker exec email-agent email-agent -fetch

Each scan is bounded by the lookback and max-message settings. Gmail's X-GM-MSGID is the stable message key; the IMAP UID and UIDVALIDITY are stored alongside it for later label operations.

Classification (Phase 3)

internal/classify sends each message to JEV (POST /v1/systemone) as three Choice questions — category, importance, suggested action — with the live category list as the category options, and stores the answer plus JEV's own confidence. One request per message, bounded concurrency, retry with backoff on 429/529/5xx.

Trigger classification from the dashboard (Classify for new mail, Re-classify all after a category or rules change) or from the CLI:

docker exec email-agent email-agent -classify      # pending messages
docker exec email-agent email-agent -classify-all  # every stored message

Rules (FR-4)

Rules are plain text, stored versioned in rules. One rule per line:

<condition> => <effects>

from contains "amazon.com" => category=Newsletter, action=unsubscribe
* => category=Info

Conditions join with and over from, to, subject, source, body using contains, equals, starts_with, ends_with. The first matching rule wins and overrides only the fields it names. Edit and version the rules at /rules (Phase 6); the classifier and the reply model both read the newest version.

Categories (FR-14)

GET /categories edits the category list. A category's name is the key stored in classifications.category and the Choice option sent to JEV, so the model always sees the current list. Renaming cascades to existing classifications; removing a category reassigns its messages to the first remaining category.

Triage actions (Phase 4)

internal/mailops turns an app folder into a Gmail label and performs the move over IMAP, forcing IPv4 exactly like the read path:

App folder	Gmail label
inbox	INBOX
delete	Delete (recoverable; never a hard delete)
saved	Saved
replied	Replied-to/<address the mail arrived at>

A move uses the IMAP MOVE extension (with a COPY + \Deleted + EXPUNGE fallback). Because Gmail gives each label its own UID, a message is located by its stable X-GM-MSGID, not by a stored UID. A message in the Delete, Saved, or Replied-to folder has a Restore button that moves it back to INBOX.

Every action appends to actions_log: one move or mark row per message, plus one bulk_<action> summary row.

Reply drafts (Phase 5)

Opening a message in the inbox loads the detail pane: the body, the recommended action, the reply identity, a template picker, and the draft. The draft is generated by an LLM through OmniRoute (the only non-JEV model path, §10) using a built-in template (quick-ack, answer, confirm, decline) plus any note Sam types. Sam edits the draft, then sends it.

The send path uses Gmail SMTP (smtp.gmail.com:587, STARTTLS) and, like the IMAP read path, forces IPv4 (AGENTS.md §9).

• Reply identity (FR-15): the From header is Sam Rolfe <address the mail arrived at>; when the arrived-at address is unknown it falls back to Sam Rolfe <GMAIL_USER>. The SMTP envelope sender is always the authenticated Gmail account.
• Threading: the reply sets In-Reply-To and References from the original Message-ID (stored by Phase 5), so Gmail keeps the thread. After a send the service checks the [Gmail]/Sent Mail mailbox for the thread and reports whether it updated.
• No auto-send (§10): sending needs a CSRF token, an explicit UI confirmation, and the guard field confirm=yes. The handler refuses a request without them and never runs from a schedule. A draft is sent once; a second attempt is refused.
• Filing: after a successful send the original moves to its Replied-to/<address> label (FR-9).

Verify the SMTP path without sending mail:

# inside the container
email-agent -check-smtp   # IPv4 + STARTTLS + app-password login, no message sent

A real send is a UI action by Sam; it is never triggered by an agent test without explicit approval (§10, AGENTS.md §13).

Retention (AGENTS.md §7)

AGENTS.md §7 says: keep message bodies for one month. Logs are append-only and size-capped. The bounded -retain job enforces both rules:

• Message bodies. Every message whose received_at is older than EMAIL_AGENT_RETENTION_DAYS (default 30) has body_text set to NULL. All metadata (sender, subject, source address, classification, folder, labels) stays. A second run is a no-op.
• Action log. Only the newest EMAIL_AGENT_ACTIONS_LOG_MAX_ROWS (default 50000) rows of the append-only actions_log are kept; older rows are deleted.

The job runs in one SQLite transaction, writes one retain row to actions_log for the audit trail, and never touches a message inside the window. It is the CLI -retain flag, so it can be scheduled exactly like -fetch and -classify and is never run by an HTTP request:

# inside the container
docker exec email-agent email-agent -retain

The decision is recorded in docs/retention.md.

Run locally

go run ./cmd/email-agent
curl -s localhost:8790/health

Docker

docker compose up -d --build
curl -s localhost:8790/health

The container is email-agent, image email-agent:latest, named volume email_agent_data mounted at /data, listening on 8790. It runs as an unprivileged user and has a built-in healthcheck (email-agent -healthcheck). Caddy on .35 proxies mail.lab.audasmedia.com.au to this port.

Endpoints

Method	Path	Auth	Purpose
GET	/health	public	Liveness + database check, JSON
GET	/readyz	public	Readiness, JSON
GET	/login	public	Sign-in form (deep-indigo band)
POST	/login	public	CSRF-checked password sign-in, sets the session cookie
POST	/logout	session	Destroys the session, returns to /login
GET	/	session	Dashboard: nav counts, real message list
GET	/partials/inbox	session	HTMX fragment for the centre list
POST	/actions/fetch	session + CSRF	Runs one IMAP scan, returns the refreshed list
POST	/actions/classify	session + CSRF	Classifies pending messages, returns the refreshed list
POST	/actions/reclassify	session + CSRF	Re-classifies every message with the current categories and rules
POST	/actions/bulk	session + CSRF	Bulk delete/move/mark and restore over selected message ids (FR-6/FR-9)
GET	/partials/message	session	HTMX fragment for the message detail, reply draft, and history
POST	/actions/draft	session + CSRF	Generates and stores a reply draft with the LLM (FR-8)
POST	/actions/draft/save	session + CSRF	Saves Sam's edits to a draft
POST	/actions/send	session + CSRF + confirm	Sends a reply over SMTP (FR-8/FR-15); never automatic
GET	/categories	session	Category editor (FR-14)
POST	/categories/add	session + CSRF	Adds a category
POST	/categories/rename	session + CSRF	Renames a category and updates existing classifications
POST	/categories/remove	session + CSRF	Removes a category and reassigns its messages
GET	/rules	session	Rules editor: current text, versions, parse errors (FR-11)
POST	/rules/save	session + CSRF	Saves the rules text as a new version
POST	/rules/restore	session + CSRF	Restores an older version as a new version
POST	/rules/reclassify	session + CSRF	Re-classifies every message with the current rules
GET	/unsubscribe	session	Detected unsubscribe links, ignore list, and records (FR-7)
POST	/unsubscribe/ignore	session + CSRF	Adds a domain to the ignore list
POST	/unsubscribe/unignore	session + CSRF	Removes a domain record
POST	/unsubscribe/mark	session + CSRF + confirm	Records a domain for unsubscribe; never contacts the link
GET	/reports	session	Reports screen: build, send, and delete a daily summary (FR-10)
POST	/reports/generate	session + CSRF	Builds and stores today's summary; sends nothing
POST	/reports/send	session + CSRF + confirm	Delivers today's summary by email via Apprise
POST	/reports/delete	session + CSRF	Deletes a stored summary
GET	/capabilities	Hub bearer	Hub capability list (FR-12)
POST	/ask	Hub bearer	Read-only short answer (FR-12); never writes
GET	/static/…	public	Embedded tokens.css, app.css, Inter fonts, htmx.min.js

Unauthenticated requests to / and /partials/… redirect to /login?next=…. Form posts use a double-submit CSRF token (email_agent_csrf cookie plus a hidden csrf field).

Database

SQLite is opened with the pure-Go modernc.org/sqlite driver (no CGO). The schema in internal/store/schema.sql creates the AGENTS.md §7 tables: messages, classifications, rules, actions_log, unsubscribe, drafts, summaries, plus categories, schema_migrations. Schema init is idempotent and runs on every start. Migration v2 adds imap_uid and imap_uidvalidity to messages; migration v3 seeds the first category list and a starter rules version once, so a category removed by the user is not silently re-added; migration v4 adds folder and mailbox to messages for the Phase 4 triage folders; migration v5 adds messages.message_id_header (reply threading) and drafts.sent_at; migration v6 adds messages.list_unsubscribe and messages.list_unsubscribe_post and extends unsubscribe with url and updated_at.

Documentation

Phase 8 recorded the project in five places.

Surface	Location
Gitea	This README, docs/phaseN-*.md, docs/outline/, docs/email-agent-architecture.mmd, docs/email-agent-map.html
Obsidian	Note Email Agent — Family Inbox Triage in 010 inbox on .27 (the sorter files it)
Outline	Collection Email Agent (00-index, system overview, config, code map, runbook)
Vikunja	Project Email Agent (human milestones)
Map	central_ai_llm_master/docs/hub-architecture-map.html and this repo's map, published to https://maps.lab.audasmedia.com.au/email_agent/docs/

Rebuild the map:

mmdc -i docs/email-agent-architecture.mmd -o docs/diagrams/email-agent-architecture.png --scale 2 -b white
~/.agents/skills/project-docs/scripts/publish-maps.sh /home/sam/paseo/projects/email_agent

See docs/phase8-documentation.md for the evidence.