email_agent/docs/phase6-unsubscribe.md

Phase 6 — Unsubscribe and rules editor

Status: complete. Built, tested, deployed on .13, and the DoD was demonstrated against real mail. Parent AUD-68. Phase 6 of AGENTS.md §11. Commit 00bb0e0.

DoD: a rule written in the editor changes the classification of a real message. — met (§2.1).

1. What was built

1.1 Rules editor (FR-4, FR-11)

• /rules is a plain-text editor over the rules table. Saving appends a new numbered version (store.SaveRules); nothing is overwritten. The page shows the current version, the active rule count, and a version list.
• Version history lists the newest revisions with timestamp, line count, and a first-rule preview. Restore re-saves an older version as a new version, so history stays append-only.
• Parse errors are surfaced on the page. classify.ParseRules collects a bad line in Errors and skips it; the editor shows exactly which lines were ignored and why.
• Apply re-runs the classifier over every message (POST /rules/reclassify) and reports considered/classified/failed back on the same page.
• JEV and the LLM read it. The classifier already read the rules and applied the first match as an override. The reply model now reads the same document: reply.Request.Rules is added to the OmniRoute prompt as a bounded USER RULES block (internal/reply/draft.go), and handleDraft passes the current rules text.

1.2 Unsubscribe detection and tracking (FR-7)

• Capture. List-Unsubscribe (RFC 2369) and List-Unsubscribe-Post (RFC 8058) are parsed by internal/mailfetch and stored raw on the message (messages.list_unsubscribe, messages.list_unsubscribe_post). They are kept raw so detection can be re-run without a refetch.
• Detect. internal/unsubscribe parses angle-bracketed and bare URIs into http/mailto links, recognises the one-click header, de-duplicates, and falls back to a body scan when there is no header (a URL on a line that mentions "unsubscribe"). The ignore-list key is the sender's brand domain when it is a real domain, with the link host as the fallback for free-mail senders (so a mailto endpoint at list-manage.com does not group unrelated senders).
• View and act. /unsubscribe lists the messages with a detected link, the domain-level ignore list, and the domains recorded for unsubscribe. Sam can ignore a domain, un-ignore it, or record it for unsubscribe. Every action is a CSRF-protected POST and is written to actions_log (FR-13).
• No link is ever followed. The app makes no network call to an unsubscribe URL and sends no mail. "Mark for unsubscribe" records the decision only; the page states this plainly, and following a real link stays a manual step that needs Sam's explicit approval. A pending RFC 8058 one-click POST is detected and shown, but never sent.

1.3 Schema (migration v6, additive)

Object	Change
messages	list_unsubscribe, list_unsubscribe_post (TEXT)
unsubscribe	url (TEXT NOT NULL DEFAULT ''), updated_at (TEXT)
schema_migrations	records version 6

The unsubscribe table keeps its status values done (recorded) and ignored (ignore list). Migration v6 is additive; a Phase 5 binary ignores the new columns.

2. DoD evidence

2.1 A rule written in the editor changed a real message

Real message 87, from Heritage Fruit Trees <support@heritagefruittrees.com.au>.

Before. Classified by JEV with no rule (rules version 1):

category	importance	action	confidence	rules_version
Newsletter	low	unsubscribe	0.98	1

Rule written in the editor. POST /rules/save with:

# Phase 6 DoD rule (AUD-72)
from contains "heritagefruittrees.com.au" => category=Action, importance=high, action=file

The page then showed version 2 · 1 active rule.

After. POST /rules/reclassify returned Re-classified 103 of 103 message(s); 0 failed. The current rules are now applied. Message 87's newest classification row is now:

category	importance	action	confidence	model	rules_version
Action	high	file	1.00	jev-1.13.0	2

The per-message action log records the override and the matching rule line:

classify  category=Action importance=high action=file confidence=1.00
          model=jev-1.13.0 rules_version=2 rule_line=2 rule="from contains ..."

Message 87's history therefore shows the change: three Newsletter/low/ unsubscribe rows under rules_version=1, then Action/high/file under rules_version=2.

2.2 Unsubscribe detection

docker exec email-agent email-agent -fetch refreshed 100 messages, filling the new header columns. 31 messages carried a List-Unsubscribe header. The live /unsubscribe page reported 33 messages with an unsubscribe link — the 31 header matches plus 2 body-fallback matches (for example allakiyas.com).

Example detected domains in the live page: google.com, spotify.com, subaru.com.au, linkedin.com, email.centrecom.com.au, r.repco.com.au, ozmobiles.com.au, heritagefruittrees.com.au, edm.booktopia.com.au, netflix.com, urbnsurf.com.

A mailto: example (Google Alerts) and a web example (message 87, https://drdzlr.clicks.mlsend.com/...) both render in the list and in the message detail pane.

2.3 Ignore list and record

Demonstrated on the live UI with heritagefruittrees.com.au, then reverted (the domain-level row is deleted; the append-only log keeps the history):

Step	Result
POST /unsubscribe/ignore	303; page shows Ignore list · 1 domain(s) with an ignored pill and the recorded URL
POST /unsubscribe/mark confirm=yes	303; page shows Recorded for unsubscribe · 1 domain(s) with a recorded pill, the URL, and a timestamp, and the note "The app has not contacted any link."
POST /unsubscribe/unignore	303; domain removed, unsubscribe table back to 0 rows

Durable actions_log rows (FR-13):

rules_save          version=2 rules=1 parse_errors=0
reclassify          considered=103 classified=103 failed=0 rules_version=2
unsubscribe_ignore  domain=heritagefruittrees.com.au
unsubscribe_marked  domain=heritagefruittrees.com.au url=https://drdzlr.clicks.mlsend.com/...
unsubscribe_unignore domain=heritagefruittrees.com.au

No email was sent, and no unsubscribe URL was contacted.

2.4 Live deployment

Check	Result
GET /health	200 {"status":"ok","version":"phase6","commit":"00bb0e0","database":"ok"}
Container log	database ready … schema_version: 6 (migration v6 ran in place on the Phase 1–5 volume)
Login over public HTTPS	https://mail.lab.audasmedia.com.au/health → 200; /rules → 303 to /login (auth enforced)
Dashboard	shows Rules and Unsubscribe links
Message detail (87)	shows action-pill">file and the detected unsubscribe link

2.5 Automated tests

gofmt -l clean, go vet ./... clean, go test ./... green (in golang:1.25-alpine). New coverage:

Package	Tests
internal/unsubscribe	header parsing (mailto + http, bare, one-click, de-dup), body fallback, unrelated-link rejection, free-mail domain fallback, Domain()
internal/mailfetch	ParseMessage captures both List-Unsubscribe headers and the source address
internal/reply	the rules document is included in the draft prompt
internal/store	migration v6 on a hand-built v5 database; unsubscribe record upsert/list/clear and status validation; candidate selection (header + body); rules version list + restore lookup
internal/server	rules page renders; save creates a version; CSRF required; parse errors shown; restore creates a new version; re-classify redirects without JEV; unsubscribe detection renders; ignore/un-ignore; mark requires confirmation and records the URL; CSRF on every mutation; bad domain rejected; message detail shows the link

3. Configuration

No new environment variables. The rules editor, unsubscribe detection, and tracking work without any new credentials. Re-classify needs JEV_API_KEY (already configured); reply drafting that reads the rules needs OMNIROUTE_API_KEY (already configured). No secret value is committed.

4. Deployment

On .13, from the repo, with the host secrets in scope:

set -a; . ~/.config/environment.d/10-secrets.conf; set +a
docker compose build
docker compose up -d          # migration v6 runs on start; data volume kept
docker exec email-agent email-agent -fetch   # fills the List-Unsubscribe columns

5. Rollback

1. git revert 00bb0e0 && docker compose up -d --build.
2. Migration v6 is additive; the Phase 5 binary ignores the new columns and the new tables. The unsubscribe table and actions_log rows are left in place.
3. Reverting the code does not un-save a rules version. Restore the intended version from /rules, or re-save it.

6. Not done / deferred

• Following an unsubscribe link is not implemented. This is deliberate (AUD-72): the app only marks and records. Acting on a real link — an RFC 8058 one-click POST, a mailto, or a web form — needs Sam's explicit approval and is a separate, manually approved step.
• The ignore-list key is the sender's domain as seen in From:. Different subdomains of one brand are tracked separately; registrable-domain grouping (a public suffix list) is not implemented.
• 1-month body retention (§7) is still not implemented.