Phase 7 follow-up: 1-month body retention and actions_log cap (AUD-75)
Implements the AGENTS.md §7 data rule: keep message bodies for one month, and size-cap the append-only action log. - internal/store/retention.go: Store.Retain clears messages.body_text older than the window (metadata stays) and deletes actions_log rows beyond the newest N, in one transaction. - internal/config: EMAIL_AGENT_RETENTION_DAYS (30) and EMAIL_AGENT_ACTIONS_LOG_MAX_ROWS (50000). - cmd/email-agent: bounded -retain flag for the n8n schedule. - Tests for body clearing, metadata retention, log cap, and disabled policy. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
41
README.md
41
README.md
@@ -9,7 +9,7 @@ pre-planning material and is not authoritative.
|
||||
|
||||
## Status
|
||||
|
||||
**Phase 6 — Unsubscribe and rules editor.** The Go skeleton, session login,
|
||||
**Phase 7 — Summary and Hub; retention.** The Go skeleton, session login,
|
||||
and HTMX dashboard (Phase 1), real mail over **IMAP with an app password,
|
||||
forced to IPv4** (Phase 2), **JEV classification** (Phase 3), **bulk triage
|
||||
actions** (Phase 4), **reply drafts** (Phase 5), and now **unsubscribe
|
||||
@@ -24,7 +24,14 @@ deliberate UI action with a confirmation — there is no auto-send (§10). The
|
||||
the reply model read. **Unsubscribe detection** (`/unsubscribe`) finds RFC
|
||||
2369/8058 links and body links, keeps a domain ignore list, and records what Sam
|
||||
marks — it **never follows a link**. Every action, including drafts, sends,
|
||||
rules edits, and unsubscribe marks, is appended to `actions_log` (FR-13). See
|
||||
rules edits, and unsubscribe marks, is appended to `actions_log` (FR-13).
|
||||
**Phase 7** adds the daily **summary** (`-summary`, `-summary-send`; delivered
|
||||
by email through Apprise) and the read-only **Hub API** (`/capabilities`,
|
||||
`/ask`, `/notify`). **Retention** (AGENTS.md §7) is enforced by the bounded
|
||||
`-retain` job: it clears message bodies older than one month, keeps metadata,
|
||||
and caps the append-only `actions_log`; see
|
||||
[`docs/retention.md`](docs/retention.md). See
|
||||
[`docs/phase7-summary-hub.md`](docs/phase7-summary-hub.md) for Phase 7,
|
||||
[`docs/phase6-unsubscribe.md`](docs/phase6-unsubscribe.md) for the phase evidence,
|
||||
[`docs/phase5-replies.md`](docs/phase5-replies.md) for Phase 5, and
|
||||
[`docs/phase4-triage.md`](docs/phase4-triage.md) for Phase 4.
|
||||
@@ -32,14 +39,14 @@ rules edits, and unsubscribe marks, is appended to `actions_log` (FR-13). See
|
||||
## Layout
|
||||
|
||||
```
|
||||
cmd/email-agent/ service entry point (`-fetch`, `-classify`, `-classify-all`)
|
||||
cmd/email-agent/ service entry point (`-fetch`, `-classify`, `-classify-all`, `-retain`)
|
||||
internal/classify/ JEV (TypeSafe System One) client, rule engine, orchestration
|
||||
internal/config/ environment-driven configuration
|
||||
internal/mailfetch/ stdlib-only IMAP client, MIME parsing, FR-2 source address
|
||||
internal/mailops/ Gmail label moves and flag changes, and SMTP reply sending
|
||||
internal/reply/ LLM reply drafts through OmniRoute, reply templates
|
||||
internal/session/ server-side session store + signed cookie values
|
||||
internal/store/ SQLite open + schema (internal/store/schema.sql)
|
||||
internal/store/ SQLite open + schema (internal/store/schema.sql), retention
|
||||
internal/server/ HTTP routes, auth middleware, HTMX handlers
|
||||
internal/unsubscribe/ RFC 2369/8058 + body unsubscribe-link detection (no network)
|
||||
internal/version/ build metadata (set via -ldflags)
|
||||
@@ -83,6 +90,8 @@ Key variables:
|
||||
| `OMNIROUTE_BASE_URL` | `http://192.168.20.13:20129/v1` | OmniRoute gateway (all non-JEV models) |
|
||||
| `OMNIROUTE_API_KEY` | _(empty)_ | OmniRoute API key (required to draft replies) |
|
||||
| `OMNIROUTE_MODEL` | `auto/best-chat` | Model alias for reply drafts |
|
||||
| `EMAIL_AGENT_RETENTION_DAYS` | `30` | Message-body retention window in days (§7) |
|
||||
| `EMAIL_AGENT_ACTIONS_LOG_MAX_ROWS` | `50000` | Newest `actions_log` rows kept |
|
||||
|
||||
## Mail sync (Phase 2)
|
||||
|
||||
@@ -197,6 +206,30 @@ email-agent -check-smtp # IPv4 + STARTTLS + app-password login, no message sen
|
||||
A real send is a UI action by Sam; it is never triggered by an agent test without
|
||||
explicit approval (§10, AGENTS.md §13).
|
||||
|
||||
## Retention (AGENTS.md §7)
|
||||
|
||||
AGENTS.md §7 says: *keep message bodies for one month. Logs are append-only and
|
||||
size-capped.* The bounded `-retain` job enforces both rules:
|
||||
|
||||
- **Message bodies.** Every message whose `received_at` is older than
|
||||
`EMAIL_AGENT_RETENTION_DAYS` (default 30) has `body_text` set to `NULL`. All
|
||||
metadata (sender, subject, source address, classification, folder, labels)
|
||||
stays. A second run is a no-op.
|
||||
- **Action log.** Only the newest `EMAIL_AGENT_ACTIONS_LOG_MAX_ROWS` (default
|
||||
50000) rows of the append-only `actions_log` are kept; older rows are deleted.
|
||||
|
||||
The job runs in one SQLite transaction, writes one `retain` row to
|
||||
`actions_log` for the audit trail, and never touches a message inside the
|
||||
window. It is the CLI `-retain` flag, so it can be scheduled exactly like
|
||||
`-fetch` and `-classify` and is never run by an HTTP request:
|
||||
|
||||
```sh
|
||||
# inside the container
|
||||
docker exec email-agent email-agent -retain
|
||||
```
|
||||
|
||||
The decision is recorded in [`docs/retention.md`](docs/retention.md).
|
||||
|
||||
## Run locally
|
||||
|
||||
```sh
|
||||
|
||||
Reference in New Issue
Block a user