Phase 7 follow-up: 1-month body retention and actions_log cap (AUD-75)

Implements the AGENTS.md §7 data rule: keep message bodies for one month, and
size-cap the append-only action log.

- internal/store/retention.go: Store.Retain clears messages.body_text older
  than the window (metadata stays) and deletes actions_log rows beyond the
  newest N, in one transaction.
- internal/config: EMAIL_AGENT_RETENTION_DAYS (30) and
  EMAIL_AGENT_ACTIONS_LOG_MAX_ROWS (50000).
- cmd/email-agent: bounded -retain flag for the n8n schedule.
- Tests for body clearing, metadata retention, log cap, and disabled policy.
This commit is contained in:
Builder
2026-10-13 10:40:52 +11:00
parent 55a08524b5
commit e5cce1c93a
9 changed files with 380 additions and 5 deletions

View File

@@ -9,7 +9,7 @@ pre-planning material and is not authoritative.
## Status
**Phase 6 — Unsubscribe and rules editor.** The Go skeleton, session login,
**Phase 7 — Summary and Hub; retention.** The Go skeleton, session login,
and HTMX dashboard (Phase 1), real mail over **IMAP with an app password,
forced to IPv4** (Phase 2), **JEV classification** (Phase 3), **bulk triage
actions** (Phase 4), **reply drafts** (Phase 5), and now **unsubscribe
@@ -24,7 +24,14 @@ deliberate UI action with a confirmation — there is no auto-send (§10). The
the reply model read. **Unsubscribe detection** (`/unsubscribe`) finds RFC
2369/8058 links and body links, keeps a domain ignore list, and records what Sam
marks — it **never follows a link**. Every action, including drafts, sends,
rules edits, and unsubscribe marks, is appended to `actions_log` (FR-13). See
rules edits, and unsubscribe marks, is appended to `actions_log` (FR-13).
**Phase 7** adds the daily **summary** (`-summary`, `-summary-send`; delivered
by email through Apprise) and the read-only **Hub API** (`/capabilities`,
`/ask`, `/notify`). **Retention** (AGENTS.md §7) is enforced by the bounded
`-retain` job: it clears message bodies older than one month, keeps metadata,
and caps the append-only `actions_log`; see
[`docs/retention.md`](docs/retention.md). See
[`docs/phase7-summary-hub.md`](docs/phase7-summary-hub.md) for Phase 7,
[`docs/phase6-unsubscribe.md`](docs/phase6-unsubscribe.md) for the phase evidence,
[`docs/phase5-replies.md`](docs/phase5-replies.md) for Phase 5, and
[`docs/phase4-triage.md`](docs/phase4-triage.md) for Phase 4.
@@ -32,14 +39,14 @@ rules edits, and unsubscribe marks, is appended to `actions_log` (FR-13). See
## Layout
```
cmd/email-agent/ service entry point (`-fetch`, `-classify`, `-classify-all`)
cmd/email-agent/ service entry point (`-fetch`, `-classify`, `-classify-all`, `-retain`)
internal/classify/ JEV (TypeSafe System One) client, rule engine, orchestration
internal/config/ environment-driven configuration
internal/mailfetch/ stdlib-only IMAP client, MIME parsing, FR-2 source address
internal/mailops/ Gmail label moves and flag changes, and SMTP reply sending
internal/reply/ LLM reply drafts through OmniRoute, reply templates
internal/session/ server-side session store + signed cookie values
internal/store/ SQLite open + schema (internal/store/schema.sql)
internal/store/ SQLite open + schema (internal/store/schema.sql), retention
internal/server/ HTTP routes, auth middleware, HTMX handlers
internal/unsubscribe/ RFC 2369/8058 + body unsubscribe-link detection (no network)
internal/version/ build metadata (set via -ldflags)
@@ -83,6 +90,8 @@ Key variables:
| `OMNIROUTE_BASE_URL` | `http://192.168.20.13:20129/v1` | OmniRoute gateway (all non-JEV models) |
| `OMNIROUTE_API_KEY` | _(empty)_ | OmniRoute API key (required to draft replies) |
| `OMNIROUTE_MODEL` | `auto/best-chat` | Model alias for reply drafts |
| `EMAIL_AGENT_RETENTION_DAYS` | `30` | Message-body retention window in days (§7) |
| `EMAIL_AGENT_ACTIONS_LOG_MAX_ROWS` | `50000` | Newest `actions_log` rows kept |
## Mail sync (Phase 2)
@@ -197,6 +206,30 @@ email-agent -check-smtp # IPv4 + STARTTLS + app-password login, no message sen
A real send is a UI action by Sam; it is never triggered by an agent test without
explicit approval (§10, AGENTS.md §13).
## Retention (AGENTS.md §7)
AGENTS.md §7 says: *keep message bodies for one month. Logs are append-only and
size-capped.* The bounded `-retain` job enforces both rules:
- **Message bodies.** Every message whose `received_at` is older than
`EMAIL_AGENT_RETENTION_DAYS` (default 30) has `body_text` set to `NULL`. All
metadata (sender, subject, source address, classification, folder, labels)
stays. A second run is a no-op.
- **Action log.** Only the newest `EMAIL_AGENT_ACTIONS_LOG_MAX_ROWS` (default
50000) rows of the append-only `actions_log` are kept; older rows are deleted.
The job runs in one SQLite transaction, writes one `retain` row to
`actions_log` for the audit trail, and never touches a message inside the
window. It is the CLI `-retain` flag, so it can be scheduled exactly like
`-fetch` and `-classify` and is never run by an HTTP request:
```sh
# inside the container
docker exec email-agent email-agent -retain
```
The decision is recorded in [`docs/retention.md`](docs/retention.md).
## Run locally
```sh