docs: Phase 8 documentation - Gitea, Obsidian, Outline, Vikunja, map (AUD-74)
- README: architecture Mermaid, Phase 8 status, Phase 7 endpoints, Documentation index - docs/outline/: AI-context collection (index, system, config, code map, runbook) - docs/email-agent-architecture.mmd + rendered PNG + Archify map (showcase pass) - docs/vikunja-tasks.json (Vikunja project 17, tasks 179-185) - docs/phase8-documentation.md evidence
This commit is contained in:
108
README.md
108
README.md
@@ -7,34 +7,59 @@ replies, and answers the Hub.
|
|||||||
The build spec is [`AGENTS.md`](AGENTS.md). `ORIGINAL_NOTES.md` is archived
|
The build spec is [`AGENTS.md`](AGENTS.md). `ORIGINAL_NOTES.md` is archived
|
||||||
pre-planning material and is not authoritative.
|
pre-planning material and is not authoritative.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
SAM["Sam<br/>phone or desktop browser"] --> CADDY["Caddy on .35<br/>mail.lab.audasmedia.com.au"]
|
||||||
|
CADDY --> UI["HTMX dashboard<br/>login, inbox, rules, reports"]
|
||||||
|
UI --> DB[("SQLite<br/>email_agent_data")]
|
||||||
|
UI --> GMAIL["Gmail<br/>IMAP :993 / SMTP :587 (IPv4)"]
|
||||||
|
UI --> JEV["JEV / TypeSafe<br/>classify"]
|
||||||
|
UI --> OMNI["OmniRoute :20129<br/>draft reply"]
|
||||||
|
UI --> APPRISE["Apprise :8210<br/>summary email"]
|
||||||
|
HUB["Central AI Hub :8788"] --> API["Hub API<br/>/capabilities /ask /notify"]
|
||||||
|
API --> DB
|
||||||
|
N8N["n8n schedule"] --> JOBS["CLI jobs<br/>-fetch -classify -summary-send"]
|
||||||
|
JOBS --> DB
|
||||||
|
```
|
||||||
|
|
||||||
|
Sources: [`docs/email-agent-architecture.mmd`](docs/email-agent-architecture.mmd)
|
||||||
|
and [`docs/email-agent-map.html`](docs/email-agent-map.html). Interactive map:
|
||||||
|
<https://maps.lab.audasmedia.com.au/email_agent/docs/>.
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
**Phase 7 — Summary and Hub; retention.** The Go skeleton, session login,
|
**Complete and deployed.** All eight AGENTS.md §11 phases are done. The Go
|
||||||
and HTMX dashboard (Phase 1), real mail over **IMAP with an app password,
|
skeleton, session login, and HTMX dashboard (Phase 1), real mail over **IMAP with
|
||||||
forced to IPv4** (Phase 2), **JEV classification** (Phase 3), **bulk triage
|
an app password, forced to IPv4** (Phase 2), **JEV classification** (Phase 3),
|
||||||
actions** (Phase 4), **reply drafts** (Phase 5), and now **unsubscribe
|
**bulk triage actions** (Phase 4), **reply drafts** (Phase 5), **unsubscribe
|
||||||
detection and the rules editor** (Phase 6). Open a message to read it, draft a
|
detection and the rules editor** (Phase 6), and the **daily summary by email plus
|
||||||
reply with the LLM through **OmniRoute** from a template, edit it, and send it
|
the read-only Hub API** (Phase 7) run on `.13` behind Caddy. Phase 8 documents
|
||||||
over Gmail SMTP (IPv4 + STARTTLS). A reply is sent **as the address the mail
|
the project across Gitea, Obsidian, Outline, Vikunja, and the architecture map.
|
||||||
arrived at** (`X-Pm-Original-To`, FR-15) and returns to the original sender; the
|
|
||||||
reply sets `In-Reply-To`/`References` so Gmail keeps the thread, and the
|
Open a message to read it, draft a reply with the LLM through **OmniRoute** from
|
||||||
original is filed under its `Replied-to/<address>` label. Sending is a
|
a template, edit it, and send it over Gmail SMTP (IPv4 + STARTTLS). A reply is
|
||||||
deliberate UI action with a confirmation — there is no auto-send (§10). The
|
sent **as the address the mail arrived at** (`X-Pm-Original-To`, FR-15) and
|
||||||
**rules editor** (`/rules`) saves and versions the plain-text rules that JEV and
|
returns to the original sender; the reply sets `In-Reply-To`/`References` so
|
||||||
the reply model read. **Unsubscribe detection** (`/unsubscribe`) finds RFC
|
Gmail keeps the thread, and the original is filed under its
|
||||||
2369/8058 links and body links, keeps a domain ignore list, and records what Sam
|
`Replied-to/<address>` label. Sending is a deliberate UI action with a
|
||||||
marks — it **never follows a link**. Every action, including drafts, sends,
|
confirmation — there is no auto-send (§10). The **rules editor** (`/rules`)
|
||||||
rules edits, and unsubscribe marks, is appended to `actions_log` (FR-13).
|
saves and versions the plain-text rules that JEV and the reply model read.
|
||||||
**Phase 7** adds the daily **summary** (`-summary`, `-summary-send`; delivered
|
**Unsubscribe detection** (`/unsubscribe`) finds RFC 2369/8058 links and body
|
||||||
by email through Apprise) and the read-only **Hub API** (`/capabilities`,
|
links, keeps a domain ignore list, and records what Sam marks — it **never
|
||||||
`/ask`, `/notify`). **Retention** (AGENTS.md §7) is enforced by the bounded
|
follows a link**. Every action, including drafts, sends, rules edits, and
|
||||||
`-retain` job: it clears message bodies older than one month, keeps metadata,
|
unsubscribe marks, is appended to `actions_log` (FR-13). **Phase 7** adds the
|
||||||
and caps the append-only `actions_log`; see
|
daily **summary** (`-summary`, `-summary-send`; delivered by email through
|
||||||
[`docs/retention.md`](docs/retention.md). See
|
Apprise) and the read-only **Hub API** (`/capabilities`, `/ask`, `/notify`).
|
||||||
[`docs/phase7-summary-hub.md`](docs/phase7-summary-hub.md) for Phase 7,
|
**Retention** (AGENTS.md §7) is enforced by the bounded `-retain` job: it clears
|
||||||
[`docs/phase6-unsubscribe.md`](docs/phase6-unsubscribe.md) for the phase evidence,
|
message bodies older than one month, keeps metadata, and caps the append-only
|
||||||
[`docs/phase5-replies.md`](docs/phase5-replies.md) for Phase 5, and
|
`actions_log`; see [`docs/retention.md`](docs/retention.md). Phase evidence:
|
||||||
[`docs/phase4-triage.md`](docs/phase4-triage.md) for Phase 4.
|
[`docs/phase7-summary-hub.md`](docs/phase7-summary-hub.md),
|
||||||
|
[`docs/phase6-unsubscribe.md`](docs/phase6-unsubscribe.md),
|
||||||
|
[`docs/phase5-replies.md`](docs/phase5-replies.md),
|
||||||
|
[`docs/phase4-triage.md`](docs/phase4-triage.md), and
|
||||||
|
[`docs/phase8-documentation.md`](docs/phase8-documentation.md).
|
||||||
|
|
||||||
## Layout
|
## Layout
|
||||||
|
|
||||||
@@ -52,6 +77,10 @@ internal/unsubscribe/ RFC 2369/8058 + body unsubscribe-link detection (no netw
|
|||||||
internal/version/ build metadata (set via -ldflags)
|
internal/version/ build metadata (set via -ldflags)
|
||||||
web/static/ embedded design tokens, Inter fonts, htmx, app.css
|
web/static/ embedded design tokens, Inter fonts, htmx, app.css
|
||||||
web/templates/ embedded server-rendered HTML
|
web/templates/ embedded server-rendered HTML
|
||||||
|
docs/ phase evidence, retention, AI context, map, diagrams
|
||||||
|
docs/outline/ AI-context documents (system, config, code, runbook)
|
||||||
|
docs/archify/ Archify JSON spec for the architecture map
|
||||||
|
docs/diagrams/ rendered Mermaid PNG
|
||||||
Dockerfile multi-stage, CGO-free build
|
Dockerfile multi-stage, CGO-free build
|
||||||
docker-compose.yml this project's own container and image
|
docker-compose.yml this project's own container and image
|
||||||
```
|
```
|
||||||
@@ -280,6 +309,12 @@ Caddy on `.35` proxies `mail.lab.audasmedia.com.au` to this port.
|
|||||||
| `POST` | `/unsubscribe/ignore` | session + CSRF | Adds a domain to the ignore list |
|
| `POST` | `/unsubscribe/ignore` | session + CSRF | Adds a domain to the ignore list |
|
||||||
| `POST` | `/unsubscribe/unignore` | session + CSRF | Removes a domain record |
|
| `POST` | `/unsubscribe/unignore` | session + CSRF | Removes a domain record |
|
||||||
| `POST` | `/unsubscribe/mark` | session + CSRF + confirm | Records a domain for unsubscribe; never contacts the link |
|
| `POST` | `/unsubscribe/mark` | session + CSRF + confirm | Records a domain for unsubscribe; never contacts the link |
|
||||||
|
| `GET` | `/reports` | session | Reports screen: build, send, and delete a daily summary (FR-10) |
|
||||||
|
| `POST` | `/reports/generate` | session + CSRF | Builds and stores today's summary; sends nothing |
|
||||||
|
| `POST` | `/reports/send` | session + CSRF + confirm | Delivers today's summary by email via Apprise |
|
||||||
|
| `POST` | `/reports/delete` | session + CSRF | Deletes a stored summary |
|
||||||
|
| `GET` | `/capabilities` | Hub bearer | Hub capability list (FR-12) |
|
||||||
|
| `POST` | `/ask` | Hub bearer | Read-only short answer (FR-12); never writes |
|
||||||
| `GET` | `/static/…` | public | Embedded `tokens.css`, `app.css`, Inter fonts, `htmx.min.js` |
|
| `GET` | `/static/…` | public | Embedded `tokens.css`, `app.css`, Inter fonts, `htmx.min.js` |
|
||||||
|
|
||||||
Unauthenticated requests to `/` and `/partials/…` redirect to `/login?next=…`.
|
Unauthenticated requests to `/` and `/partials/…` redirect to `/login?next=…`.
|
||||||
@@ -300,3 +335,24 @@ folders; migration v5 adds `messages.message_id_header` (reply threading) and
|
|||||||
`drafts.sent_at`; migration v6 adds `messages.list_unsubscribe` and
|
`drafts.sent_at`; migration v6 adds `messages.list_unsubscribe` and
|
||||||
`messages.list_unsubscribe_post` and extends `unsubscribe` with `url` and
|
`messages.list_unsubscribe_post` and extends `unsubscribe` with `url` and
|
||||||
`updated_at`.
|
`updated_at`.
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
Phase 8 recorded the project in five places.
|
||||||
|
|
||||||
|
| Surface | Location |
|
||||||
|
|---|---|
|
||||||
|
| Gitea | This README, `docs/phaseN-*.md`, `docs/outline/`, `docs/email-agent-architecture.mmd`, `docs/email-agent-map.html` |
|
||||||
|
| Obsidian | Note `Email Agent — Family Inbox Triage` in `010 inbox` on `.27` (the sorter files it) |
|
||||||
|
| Outline | Collection `Email Agent` (`00-index`, system overview, config, code map, runbook) |
|
||||||
|
| Vikunja | Project `Email Agent` (human milestones) |
|
||||||
|
| Map | `central_ai_llm_master/docs/hub-architecture-map.html` and this repo's map, published to <https://maps.lab.audasmedia.com.au/email_agent/docs/> |
|
||||||
|
|
||||||
|
Rebuild the map:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mmdc -i docs/email-agent-architecture.mmd -o docs/diagrams/email-agent-architecture.png --scale 2 -b white
|
||||||
|
~/.agents/skills/project-docs/scripts/publish-maps.sh /home/sam/paseo/projects/email_agent
|
||||||
|
```
|
||||||
|
|
||||||
|
See [`docs/phase8-documentation.md`](docs/phase8-documentation.md) for the evidence.
|
||||||
|
|||||||
58
docs/archify/email-agent.architecture.json
Normal file
58
docs/archify/email-agent.architecture.json
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
{
|
||||||
|
"schema_version": 1,
|
||||||
|
"diagram_type": "architecture",
|
||||||
|
"meta": {
|
||||||
|
"title": "Email Agent — One-Page Family Inbox Triage",
|
||||||
|
"output": "docs/email-agent-map.html",
|
||||||
|
"quality_profile": "showcase",
|
||||||
|
"viewBox": [1780, 1000],
|
||||||
|
"views": [
|
||||||
|
{
|
||||||
|
"id": "triage-loop",
|
||||||
|
"label": "Triage loop",
|
||||||
|
"focus": ["sam", "caddy", "ui", "gmail", "db"],
|
||||||
|
"note": "Sam opens one page. The app reads Gmail and stores every message."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "classify-reply",
|
||||||
|
"label": "Classify and reply",
|
||||||
|
"focus": ["ui", "jev", "omni", "gmail"],
|
||||||
|
"note": "JEV labels each message. OmniRoute drafts the reply. SMTP sends it."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "hub-summary",
|
||||||
|
"label": "Hub and summary",
|
||||||
|
"focus": ["hub", "api", "db", "jobs", "apprise", "n8n"],
|
||||||
|
"note": "The Hub asks read-only questions. CLI jobs send the daily summary."
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"components": [
|
||||||
|
{ "id": "sam", "type": "frontend", "label": "Sam", "sublabel": "phone or desktop browser", "pos": [60, 460], "size": [150, 64] },
|
||||||
|
{ "id": "caddy", "type": "security", "label": "Caddy on .35", "sublabel": "mail.lab.audasmedia.com.au", "pos": [330, 460], "size": [170, 64] },
|
||||||
|
{ "id": "ui", "type": "frontend", "label": "HTMX dashboard", "sublabel": "login, inbox, rules, reports", "pos": [640, 440], "size": [200, 80], "tag": "port 8790" },
|
||||||
|
{ "id": "db", "type": "database", "label": "SQLite", "sublabel": "email_agent_data volume", "pos": [1300, 460], "size": [180, 80] },
|
||||||
|
{ "id": "api", "type": "backend", "label": "Hub API", "sublabel": "/capabilities /ask /notify", "pos": [1300, 120], "size": [180, 64], "tag": "read-only /ask" },
|
||||||
|
{ "id": "hub", "type": "backend", "label": "Central AI Hub", "sublabel": "hear-route-answer :8788", "pos": [980, 120], "size": [180, 64] },
|
||||||
|
{ "id": "jobs", "type": "backend", "label": "CLI jobs", "sublabel": "-fetch -classify -summary-send", "pos": [1300, 800], "size": [180, 64] },
|
||||||
|
{ "id": "n8n", "type": "external", "label": "n8n schedule", "sublabel": "scan and summary runs", "pos": [1580, 800], "size": [170, 64] },
|
||||||
|
{ "id": "apprise", "type": "backend", "label": "Apprise", "sublabel": "summary email :8210", "pos": [1000, 800], "size": [170, 64] },
|
||||||
|
{ "id": "gmail", "type": "cloud", "label": "Gmail", "sublabel": "IMAP :993 / SMTP :587, IPv4", "pos": [240, 800], "size": [180, 64] },
|
||||||
|
{ "id": "jev", "type": "cloud", "label": "JEV", "sublabel": "TypeSafe System One", "pos": [500, 800], "size": [170, 64] },
|
||||||
|
{ "id": "omni", "type": "cloud", "label": "OmniRoute", "sublabel": "reply drafts :20129", "pos": [760, 800], "size": [170, 64] }
|
||||||
|
],
|
||||||
|
"connections": [
|
||||||
|
{ "id": "browser", "from": "sam", "to": "caddy", "label": "HTTPS" },
|
||||||
|
{ "id": "proxy", "from": "caddy", "to": "ui", "label": "reverse proxy" },
|
||||||
|
{ "id": "store", "from": "ui", "to": "db", "label": "store mail, triage, rules" },
|
||||||
|
{ "id": "mail", "from": "ui", "to": "gmail", "label": "read + send", "fromSide": "bottom", "toSide": "top", "route": "orthogonal-v" },
|
||||||
|
{ "id": "classify", "from": "ui", "to": "jev", "label": "classify", "fromSide": "bottom", "toSide": "top", "route": "orthogonal-v" },
|
||||||
|
{ "id": "draft", "from": "ui", "to": "omni", "label": "draft reply", "fromSide": "bottom", "toSide": "top", "route": "orthogonal-v" },
|
||||||
|
{ "id": "summary", "from": "ui", "to": "apprise", "label": "send summary", "fromSide": "bottom", "toSide": "top", "route": "orthogonal-v" },
|
||||||
|
{ "id": "ask", "from": "hub", "to": "api", "label": "Bearer token" },
|
||||||
|
{ "id": "read", "from": "api", "to": "db", "label": "read only", "labelAt": [1390, 340] },
|
||||||
|
{ "id": "schedule", "from": "n8n", "to": "jobs", "label": "schedule" },
|
||||||
|
{ "id": "run", "from": "jobs", "to": "db", "label": "fetch + classify" },
|
||||||
|
{ "id": "jobs-summary", "from": "jobs", "to": "apprise", "label": "daily email", "labelAt": [1240, 878] }
|
||||||
|
]
|
||||||
|
}
|
||||||
BIN
docs/diagrams/email-agent-architecture.png
Normal file
BIN
docs/diagrams/email-agent-architecture.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 142 KiB |
41
docs/email-agent-architecture.mmd
Normal file
41
docs/email-agent-architecture.mmd
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
flowchart LR
|
||||||
|
SAM["Sam<br/>phone or desktop browser"]
|
||||||
|
CADDY["Caddy on .35<br/>mail.lab.audasmedia.com.au"]
|
||||||
|
|
||||||
|
subgraph APP["Email Agent container on .13 (port 8790)"]
|
||||||
|
UI["HTMX dashboard<br/>login, inbox, rules, reports"]
|
||||||
|
API["Hub API<br/>/capabilities /ask /notify"]
|
||||||
|
JOBS["CLI jobs<br/>-fetch -classify -summary-send"]
|
||||||
|
DB[("SQLite<br/>email_agent_data volume")]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph HOUSE["House services"]
|
||||||
|
HUB["Central AI Hub :8788"]
|
||||||
|
APPRISE["Apprise :8210<br/>summary email"]
|
||||||
|
end
|
||||||
|
|
||||||
|
N8N["n8n schedule"]
|
||||||
|
|
||||||
|
subgraph CLOUD["External providers"]
|
||||||
|
GMAIL["Gmail IMAP :993 / SMTP :587<br/>IPv4 only, app password"]
|
||||||
|
JEV["JEV / TypeSafe System One<br/>classification"]
|
||||||
|
OMNI["OmniRoute :20129<br/>reply drafts"]
|
||||||
|
end
|
||||||
|
|
||||||
|
SAM --> CADDY --> UI
|
||||||
|
UI --> DB
|
||||||
|
UI --> GMAIL
|
||||||
|
UI --> JEV
|
||||||
|
UI --> OMNI
|
||||||
|
UI --> APPRISE
|
||||||
|
HUB --> API
|
||||||
|
API --> DB
|
||||||
|
N8N --> JOBS
|
||||||
|
JOBS --> DB
|
||||||
|
JOBS --> GMAIL
|
||||||
|
JOBS --> APPRISE
|
||||||
|
API -. notify .-> HUB
|
||||||
|
|
||||||
|
click CADDY "https://maps.lab.audasmedia.com.au/email_agent/docs/" "View Archify Map"
|
||||||
|
click UI "https://maps.lab.audasmedia.com.au/email_agent/docs/" "View Archify Map"
|
||||||
|
click DB "https://maps.lab.audasmedia.com.au/email_agent/docs/" "View Archify Map"
|
||||||
13716
docs/email-agent-map.html
Normal file
13716
docs/email-agent-map.html
Normal file
File diff suppressed because it is too large
Load Diff
44
docs/outline/00-index.md
Normal file
44
docs/outline/00-index.md
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
# Email Agent — AI Document Index
|
||||||
|
|
||||||
|
> **Target**: AI agents.
|
||||||
|
> **Writing style**: ASD-STE100 Simplified Technical English.
|
||||||
|
|
||||||
|
## Document catalog
|
||||||
|
|
||||||
|
Read this index. Inject the exact document you need. Do not search all files.
|
||||||
|
|
||||||
|
| Document | Path | Scope |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| **System Overview** | `docs/outline/01-system-overview.md` | Purpose, architecture, services, phase status. |
|
||||||
|
| **Configuration and Environment** | `docs/outline/02-config-and-env.md` | Environment variables, config paths, ports, secrets. |
|
||||||
|
| **Code Map** | `docs/outline/03-code-map.md` | Folder structure, entry points, dependencies, schema. |
|
||||||
|
| **Agent Runbook** | `docs/outline/04-agent-runbook.md` | Commands to test, deploy, restart, and roll back. |
|
||||||
|
|
||||||
|
The build spec is `AGENTS.md`. The phase evidence files are `docs/phaseN-*.md`.
|
||||||
|
The interactive map is `docs/email-agent-map.html`.
|
||||||
|
|
||||||
|
## System summary
|
||||||
|
|
||||||
|
The Email Agent triages the family inbox. All mail lands in one Gmail account.
|
||||||
|
Sam opens one page, clears the inbox in minutes, and sees nothing important
|
||||||
|
missed. The service reads Gmail over IMAP, classifies each message with JEV,
|
||||||
|
recommends an action, applies bulk actions, drafts replies, and answers the Hub.
|
||||||
|
|
||||||
|
## Core configuration
|
||||||
|
|
||||||
|
- **Secrets file**: `~/.config/environment.d/10-secrets.conf` (mode 600).
|
||||||
|
- **Project directory**: `/home/sam/paseo/projects/email_agent/`.
|
||||||
|
- **Live host**: `.13`, one container, own image.
|
||||||
|
- **Live port**: `8790`. Public URL: `https://mail.lab.audasmedia.com.au`.
|
||||||
|
- **Database**: SQLite at `/data/email_agent.db` on the `email_agent_data` volume.
|
||||||
|
|
||||||
|
## Agent restart steps
|
||||||
|
|
||||||
|
1. Read the catalog above. Inject only the documents you need.
|
||||||
|
2. Check git status: `git status`.
|
||||||
|
3. Check recent commits: `git log -n 5 --oneline`.
|
||||||
|
4. Run the Go tests in a container:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
timeout 600 docker run --rm -v "$PWD":/src -w /src golang:1.25-alpine go test ./...
|
||||||
|
```
|
||||||
98
docs/outline/01-system-overview.md
Normal file
98
docs/outline/01-system-overview.md
Normal file
@@ -0,0 +1,98 @@
|
|||||||
|
# Email Agent — System Overview
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The Email Agent triages the family inbox. All mail lands in one Gmail account
|
||||||
|
(`samuelrolfe@gmail.com`). PurelyMail forwards every domain address there.
|
||||||
|
|
||||||
|
Sam opens one page. He clears the inbox in a few minutes. He knows that nothing
|
||||||
|
important was missed.
|
||||||
|
|
||||||
|
The build spec is `AGENTS.md`. The outcome statement is in `AGENTS.md` section 1.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
Sam (phone or desktop browser)
|
||||||
|
-> Caddy on .35 (mail.lab.audasmedia.com.au)
|
||||||
|
-> Email Agent container on .13 :8790
|
||||||
|
|- HTMX dashboard (Go, server-rendered HTML)
|
||||||
|
|- SQLite (/data/email_agent.db, own volume)
|
||||||
|
|- IMAP read -> imap.gmail.com:993 (IPv4 only)
|
||||||
|
|- SMTP send -> smtp.gmail.com:587 (IPv4 only, STARTTLS)
|
||||||
|
|- JEV -> https://api.typesafe.ai (classification)
|
||||||
|
|- OmniRoute -> http://192.168.20.13:20129/v1 (reply drafts)
|
||||||
|
|- Apprise -> http://192.168.20.35:8210 (daily summary email)
|
||||||
|
\- Hub API -> read-only /ask, /capabilities, /notify
|
||||||
|
Hub on .13 :8788 asks this service with a shared bearer token.
|
||||||
|
n8n runs the CLI jobs (-fetch, -classify, -summary-send) on a schedule.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Services and connections
|
||||||
|
|
||||||
|
| Service | Host | Address | Role |
|
||||||
|
| :--- | :--- | :--- | :--- |
|
||||||
|
| `email-agent` | `.13` | `:8790` | Web app, API, and CLI jobs. |
|
||||||
|
| Caddy | `.35` | `:443` | TLS reverse proxy for `mail.lab.audasmedia.com.au`. |
|
||||||
|
| Gmail IMAP | cloud | `imap.gmail.com:993` | Read mail. Force IPv4. |
|
||||||
|
| Gmail SMTP | cloud | `smtp.gmail.com:587` | Send replies. Force IPv4. |
|
||||||
|
| JEV | cloud | `api.typesafe.ai` | Classify each message. |
|
||||||
|
| OmniRoute | `.13` | `:20129` | Draft reply replies. Not JEV. |
|
||||||
|
| Apprise | `.35` | `:8210` | Deliver the daily summary by email. |
|
||||||
|
| Central AI Hub | `.13` | `:8788` | Ask questions. Read-only. |
|
||||||
|
| n8n | `.13` | — | Trigger the CLI jobs. |
|
||||||
|
|
||||||
|
## Functional requirements in use
|
||||||
|
|
||||||
|
| ID | Requirement | Where |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| FR-1 | Fetch mail with the Gmail path on a schedule. | `-fetch`, `/actions/fetch` |
|
||||||
|
| FR-2 | Record the source address from `X-Pm-Original-To`. | `internal/mailfetch` |
|
||||||
|
| FR-3 | Classify with JEV: category, importance, action, confidence. | `internal/classify` |
|
||||||
|
| FR-4 | Apply user rules over the classification. | `internal/classify/rules.go` |
|
||||||
|
| FR-5 | Show a triage list grouped by category. | `internal/server/inbox.go` |
|
||||||
|
| FR-6 | Bulk delete, move, and mark. | `internal/server/actions.go` |
|
||||||
|
| FR-7 | Detect unsubscribe links. Keep an ignore list. | `internal/unsubscribe` |
|
||||||
|
| FR-8 | Draft a reply with the LLM. Edit, then send. | `internal/reply`, `internal/mailops` |
|
||||||
|
| FR-9 | Move mail to Gmail labels. Deletes are recoverable. | `internal/mailops` |
|
||||||
|
| FR-10 | Build a daily summary and send it by email. | `internal/summary`, `internal/apprise` |
|
||||||
|
| FR-11 | Edit rules as plain text. Save and version. | `/rules`, `internal/server/rules.go` |
|
||||||
|
| FR-12 | Expose `GET /capabilities`, `POST /ask`, `POST /notify`. | `internal/hub` |
|
||||||
|
| FR-13 | Log every action. | `internal/store` `actions_log` |
|
||||||
|
| FR-14 | Edit the category list in the UI. | `/categories` |
|
||||||
|
| FR-15 | Send as the address the mail arrived at. | `internal/reply`, FR-2 value |
|
||||||
|
|
||||||
|
## Build phase status
|
||||||
|
|
||||||
|
Phases 1 to 7 are complete and deployed. Phase 8 documents the project.
|
||||||
|
|
||||||
|
| Phase | Scope | State |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| 1 | Go skeleton, SQLite, session login, Docker, Caddy. | Done. |
|
||||||
|
| 2 | Gmail IMAP read, FR-2 source address. | Done. |
|
||||||
|
| 3 | JEV classification, rules table, editable categories. | Done. |
|
||||||
|
| 4 | Triage actions: delete, move, mark, bulk select. | Done. |
|
||||||
|
| 5 | Reply drafts, edit, send. | Done. |
|
||||||
|
| 6 | Unsubscribe detection and the rules editor. | Done. |
|
||||||
|
| 7 | Daily summary by email and the read-only Hub API. | Done. |
|
||||||
|
| 8 | Documentation across Gitea, Obsidian, Outline, Vikunja, and the map. | Done. |
|
||||||
|
|
||||||
|
## Key rules
|
||||||
|
|
||||||
|
- Force IPv4 for every Gmail connection. IPv6 to Gmail failed from `.13`.
|
||||||
|
- The source address header is `X-Pm-Original-To`. Never use `Delivered-To`.
|
||||||
|
- `/ask` is read-only. It never changes state.
|
||||||
|
- Spam is recommended, never auto-deleted.
|
||||||
|
- Deletes are recoverable. The app uses a Delete label.
|
||||||
|
- Sending a reply needs Sam's action in the UI. No auto-send.
|
||||||
|
- Every action is logged.
|
||||||
|
- Message bodies are kept for one month. The `-retain` job clears older bodies
|
||||||
|
and caps `actions_log`. Metadata stays.
|
||||||
|
|
||||||
|
## Related documents
|
||||||
|
|
||||||
|
- `AGENTS.md` — the build spec and the phase plan.
|
||||||
|
- `README.md` — the technical repository README.
|
||||||
|
- `docs/phase7-summary-hub.md` — the Phase 7 evidence.
|
||||||
|
- `docs/outline/` — these AI-context documents.
|
||||||
|
- `docs/email-agent-map.html` — the interactive architecture map.
|
||||||
86
docs/outline/02-config-and-env.md
Normal file
86
docs/outline/02-config-and-env.md
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
# Email Agent — Configuration and Environment
|
||||||
|
|
||||||
|
## Secrets
|
||||||
|
|
||||||
|
All secrets live in `~/.config/environment.d/10-secrets.conf` on `.13` (mode
|
||||||
|
600). The file is never committed. `docker-compose.yml` passes the names through
|
||||||
|
from the host environment.
|
||||||
|
|
||||||
|
| Key | Purpose |
|
||||||
|
| :--- | :--- |
|
||||||
|
| `GMAIL_USER` | Gmail account that receives the forwarded mail. |
|
||||||
|
| `GMAIL_APP_PASSWORD` | Gmail app password for IMAP and SMTP. No OAuth. |
|
||||||
|
| `EMAIL_AGENT_SESSION_SECRET` | Session cookie signing key. |
|
||||||
|
| `EMAIL_AGENT_PASSWORD` | Login password for Sam. |
|
||||||
|
| `JEV_API_KEY` | JEV (TypeSafe System One) API key. |
|
||||||
|
| `OMNIROUTE_API_KEY` | OmniRoute API key for reply drafts. |
|
||||||
|
| `HUB_TOKEN` | Shared bearer token for the Hub API. |
|
||||||
|
| `APPRISE_URLS` | Apprise destination for the summary email. |
|
||||||
|
| `EMAIL_AGENT_SUMMARY_TO` | Summary recipient. |
|
||||||
|
|
||||||
|
`environment.d` reaches the systemd user manager. It does not reach a fresh SSH
|
||||||
|
shell. Read the values explicitly, or pass them to `docker compose`.
|
||||||
|
|
||||||
|
## Runtime variables
|
||||||
|
|
||||||
|
| Variable | Default | Meaning |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| `EMAIL_AGENT_ENV` | `development` | `development`, `production`, or `test`. |
|
||||||
|
| `EMAIL_AGENT_ADDR` | `:8790` | Listen address. |
|
||||||
|
| `EMAIL_AGENT_PORT` | `8790` | Listen port when `ADDR` is unset. |
|
||||||
|
| `EMAIL_AGENT_DB_PATH` | `data/email_agent.db` | SQLite file. Container uses `/data/email_agent.db`. |
|
||||||
|
| `EMAIL_AGENT_LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error`. |
|
||||||
|
| `EMAIL_AGENT_PUBLIC_BASE_URL` | — | Public URL for links. |
|
||||||
|
|
||||||
|
## Gmail variables
|
||||||
|
|
||||||
|
| Variable | Default | Meaning |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| `EMAIL_AGENT_IMAP_HOST` | `imap.gmail.com` | IMAP host. |
|
||||||
|
| `EMAIL_AGENT_IMAP_PORT` | `993` | IMAP port. Implicit TLS. |
|
||||||
|
| `EMAIL_AGENT_IMAP_LOOKBACK_DAYS` | `14` | First-scan window. |
|
||||||
|
| `EMAIL_AGENT_IMAP_MAX_MESSAGES` | `100` | Per-scan message cap. |
|
||||||
|
| `EMAIL_AGENT_SMTP_HOST` | `smtp.gmail.com` | SMTP submission host. |
|
||||||
|
| `EMAIL_AGENT_SMTP_PORT` | `587` | SMTP port. STARTTLS. |
|
||||||
|
|
||||||
|
The dialer uses `tcp4`. Gmail is reached over IPv4 only.
|
||||||
|
|
||||||
|
## Model and integration variables
|
||||||
|
|
||||||
|
| Variable | Default | Meaning |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| `JEV_BASE_URL` | `https://api.typesafe.ai` | JEV base URL. |
|
||||||
|
| `JEV_MODEL` | `jev-latest` | System One model alias. |
|
||||||
|
| `EMAIL_AGENT_CLASSIFY_CONCURRENCY` | `4` | In-flight JEV requests. |
|
||||||
|
| `EMAIL_AGENT_CLASSIFY_MAX_MESSAGES` | `200` | Messages per classify run. |
|
||||||
|
| `OMNIROUTE_BASE_URL` | `http://192.168.20.13:20129/v1` | OmniRoute gateway. |
|
||||||
|
| `OMNIROUTE_MODEL` | `auto/best-chat` | Reply-draft model. |
|
||||||
|
| `HUB_BASE_URL` | `http://192.168.20.13:8788` | Hub base URL. |
|
||||||
|
| `HUB_NOTIFY_TIMEOUT` | `5s` | Notify call timeout. |
|
||||||
|
| `APPRISE_URL` | `http://192.168.20.35:8210` | Apprise API base. |
|
||||||
|
| `APPRISE_KEY` | — | Stored Apprise configuration key. |
|
||||||
|
| `EMAIL_AGENT_SUMMARY_TO` | `samuelrolfe@gmail.com` | Summary recipient. |
|
||||||
|
| `EMAIL_AGENT_RETENTION_DAYS` | `30` | Message-body window for `-retain` (AGENTS.md section 7). |
|
||||||
|
| `EMAIL_AGENT_ACTIONS_LOG_MAX_ROWS` | `50000` | Cap for the append-only `actions_log`. |
|
||||||
|
|
||||||
|
## Config paths
|
||||||
|
|
||||||
|
| Path | Role |
|
||||||
|
| :--- | :--- |
|
||||||
|
| `/home/sam/paseo/projects/email_agent/` | Project source and docs. |
|
||||||
|
| `/home/sam/paseo/projects/email_agent/docker-compose.yml` | Own container and image. |
|
||||||
|
| `/home/sam/paseo/projects/email_agent/.env.example` | Full list of variable names. |
|
||||||
|
| `/data/email_agent.db` | SQLite file in the container. |
|
||||||
|
| `email_agent_data` | Docker named volume for the database. |
|
||||||
|
|
||||||
|
## Ports
|
||||||
|
|
||||||
|
| Port | Host | Service |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| `8790` | `.13` | Email Agent HTTP. |
|
||||||
|
| `443` | `.35` | Caddy TLS for `mail.lab.audasmedia.com.au`. |
|
||||||
|
| `993` | cloud | Gmail IMAP to Gmail. |
|
||||||
|
| `587` | cloud | Gmail SMTP to Gmail. |
|
||||||
|
| `8788` | `.13` | Central AI Hub. This service both serves and calls it. |
|
||||||
|
| `8210` | `.35` | Apprise API. |
|
||||||
|
| `20129` | `.13` | OmniRoute gateway. |
|
||||||
158
docs/outline/03-code-map.md
Normal file
158
docs/outline/03-code-map.md
Normal file
@@ -0,0 +1,158 @@
|
|||||||
|
# Email Agent — Code Map
|
||||||
|
|
||||||
|
## Folder structure
|
||||||
|
|
||||||
|
```
|
||||||
|
email_agent/
|
||||||
|
├── AGENTS.md Build spec (phases 1-8, requirements, rules).
|
||||||
|
├── README.md Technical repository README.
|
||||||
|
├── cmd/email-agent/
|
||||||
|
│ └── main.go Entry point. HTTP server and every CLI flag.
|
||||||
|
├── internal/
|
||||||
|
│ ├── apprise/ Apprise API client for the daily summary email.
|
||||||
|
│ ├── classify/ JEV client, rule engine, classification orchestration.
|
||||||
|
│ ├── config/ Environment-driven configuration.
|
||||||
|
│ ├── hub/ Hub contract: capabilities, /ask answerer, notify.
|
||||||
|
│ ├── mailfetch/ Stdlib-only IMAP client, MIME parse, FR-2 source address.
|
||||||
|
│ ├── mailops/ Gmail label moves, flag changes, and SMTP sending.
|
||||||
|
│ ├── reply/ OmniRoute LLM drafts and reply templates.
|
||||||
|
│ ├── server/ HTTP routes, auth middleware, HTMX handlers.
|
||||||
|
│ ├── session/ Server-side session store and signed cookie values.
|
||||||
|
│ ├── store/ SQLite open, schema, migrations, all queries.
|
||||||
|
│ ├── summary/ Deterministic daily-summary builder.
|
||||||
|
│ ├── unsubscribe/ RFC 2369/8058 and body unsubscribe-link detection.
|
||||||
|
│ └── version/ Build metadata (-ldflags).
|
||||||
|
├── web/
|
||||||
|
│ ├── embed.go Embeds the static and template files.
|
||||||
|
│ ├── static/ tokens.css, app.css, Inter fonts, htmx.min.js.
|
||||||
|
│ └── templates/ login, dashboard, inbox, message, categories, rules,
|
||||||
|
│ unsubscribe, reports.
|
||||||
|
├── docs/ Phase evidence, AI context, map, diagrams.
|
||||||
|
├── Dockerfile Multi-stage, CGO-free build.
|
||||||
|
├── docker-compose.yml Own container, image, port, and volume.
|
||||||
|
└── Makefile Build, test, and run targets.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Entry points
|
||||||
|
|
||||||
|
| Entry point | Language | Role |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| `cmd/email-agent/main.go` | Go | Service and CLI. |
|
||||||
|
| `internal/server/server.go` | Go | HTTP router and middleware. |
|
||||||
|
| `internal/store/schema.sql` | SQL | The AGENTS.md section 7 tables. |
|
||||||
|
|
||||||
|
## CLI flags
|
||||||
|
|
||||||
|
| Flag | Effect |
|
||||||
|
| :--- | :--- |
|
||||||
|
| `-version` | Print version, commit, and build time. |
|
||||||
|
| `-healthcheck` | Probe the local `/health` endpoint and exit. |
|
||||||
|
| `-fetch` | Run one Gmail IMAP scan and exit (FR-1). |
|
||||||
|
| `-classify` | Classify pending messages with JEV and exit. |
|
||||||
|
| `-classify-all` | Re-classify every stored message. |
|
||||||
|
| `-check-smtp` | Verify SMTP IPv4, STARTTLS, and login. Sends no mail. |
|
||||||
|
| `-check-thread <id>` | Confirm a stored thread has a sent reply. Read-only. |
|
||||||
|
| `-summary` | Build today's summary, store it, and print it. Sends nothing. |
|
||||||
|
| `-summary-send` | Build the summary and deliver it by email via Apprise. |
|
||||||
|
| `-notify` | Send one notification to the Hub (`POST /notify`). |
|
||||||
|
| `-retain` | Clear old message bodies and cap `actions_log` (AGENTS.md section 7). |
|
||||||
|
|
||||||
|
## HTTP routes
|
||||||
|
|
||||||
|
| Method | Path | Auth | Purpose |
|
||||||
|
| :--- | :--- | :--- | :--- |
|
||||||
|
| `GET` | `/health` | public | Liveness and database check. |
|
||||||
|
| `GET` | `/readyz` | public | Readiness. |
|
||||||
|
| `GET` | `/login` | public | Sign-in form. |
|
||||||
|
| `POST` | `/login` | public | CSRF-checked sign-in. |
|
||||||
|
| `POST` | `/logout` | session | End the session. |
|
||||||
|
| `GET` | `/` | session | Dashboard. |
|
||||||
|
| `GET` | `/partials/inbox` | session | HTMX inbox fragment. |
|
||||||
|
| `POST` | `/actions/fetch` | session + CSRF | One IMAP scan. |
|
||||||
|
| `POST` | `/actions/classify` | session + CSRF | Classify pending messages. |
|
||||||
|
| `POST` | `/actions/reclassify` | session + CSRF | Re-classify every message. |
|
||||||
|
| `POST` | `/actions/bulk` | session + CSRF | Bulk delete, move, mark, restore. |
|
||||||
|
| `GET` | `/partials/message` | session | Message detail fragment. |
|
||||||
|
| `POST` | `/actions/draft` | session + CSRF | Generate and store a reply draft. |
|
||||||
|
| `POST` | `/actions/draft/save` | session + CSRF | Save draft edits. |
|
||||||
|
| `POST` | `/actions/send` | session + CSRF + confirm | Send a reply over SMTP. |
|
||||||
|
| `GET` | `/categories` | session | Category editor (FR-14). |
|
||||||
|
| `POST` | `/categories/add` | session + CSRF | Add a category. |
|
||||||
|
| `POST` | `/categories/rename` | session + CSRF | Rename and cascade. |
|
||||||
|
| `POST` | `/categories/remove` | session + CSRF | Remove and reassign. |
|
||||||
|
| `GET` | `/rules` | session | Rules editor (FR-11). |
|
||||||
|
| `POST` | `/rules/save` | session + CSRF | Save a new rules version. |
|
||||||
|
| `POST` | `/rules/restore` | session + CSRF | Restore an older version. |
|
||||||
|
| `POST` | `/rules/reclassify` | session + CSRF | Re-classify with current rules. |
|
||||||
|
| `GET` | `/unsubscribe` | session | Unsubscribe links and ignore list (FR-7). |
|
||||||
|
| `POST` | `/unsubscribe/ignore` | session + CSRF | Add a domain to the ignore list. |
|
||||||
|
| `POST` | `/unsubscribe/unignore` | session + CSRF | Remove a domain record. |
|
||||||
|
| `POST` | `/unsubscribe/mark` | session + CSRF + confirm | Record a domain. No link is followed. |
|
||||||
|
| `GET` | `/reports` | session | Daily summary and delete-report. |
|
||||||
|
| `GET` | `/capabilities` | Hub bearer | Hub capability list (FR-12). |
|
||||||
|
| `POST` | `/ask` | Hub bearer | Read-only short answer. |
|
||||||
|
| `GET` | `/static/...` | public | Embedded CSS, fonts, htmx. |
|
||||||
|
|
||||||
|
## Data model
|
||||||
|
|
||||||
|
SQLite. The schema is in `internal/store/schema.sql`. Migrations run at start
|
||||||
|
and are recorded in `schema_migrations`.
|
||||||
|
|
||||||
|
| Table | Holds |
|
||||||
|
| :--- | :--- |
|
||||||
|
| `messages` | Gmail id, thread id, headers, from, to, subject, body, source address. |
|
||||||
|
| `classifications` | Category, importance, action, confidence, model. |
|
||||||
|
| `categories` | The editable category list (FR-14). |
|
||||||
|
| `rules` | Plain-text rules, versioned (FR-11). |
|
||||||
|
| `actions_log` | Append-only action log (FR-13). Capped by `-retain`. |
|
||||||
|
| `unsubscribe` | Detected domains, status, and ignore list (FR-7). |
|
||||||
|
| `drafts` | Reply draft body, template, and sent state (FR-8). |
|
||||||
|
| `summaries` | Daily summary text and delivery target (FR-10). |
|
||||||
|
|
||||||
|
Migrations: v2 adds IMAP UID fields. v3 seeds the first categories and rules.
|
||||||
|
v4 adds `folder` and `mailbox`. v5 adds `message_id_header` and `drafts.sent_at`.
|
||||||
|
v6 adds `list_unsubscribe` fields and `unsubscribe.url`.
|
||||||
|
|
||||||
|
## Key modules
|
||||||
|
|
||||||
|
| Module | Role |
|
||||||
|
| :--- | :--- |
|
||||||
|
| `internal/mailfetch/imap.go` | Minimal IMAP client. Forces `tcp4`. |
|
||||||
|
| `internal/mailfetch/message.go` | MIME parse and the FR-2 source address. |
|
||||||
|
| `internal/mailfetch/imap_write.go` | IMAP MOVE, COPY, and flag changes. |
|
||||||
|
| `internal/classify/jev.go` | JEV `/v1/systemone` client. Three Choice questions. |
|
||||||
|
| `internal/classify/rules.go` | Rule parse and first-match-wins engine (FR-4). |
|
||||||
|
| `internal/mailops/mailops.go` | App folder to Gmail label mapping. |
|
||||||
|
| `internal/mailops/smtp.go` | SMTP send with IPv4 and STARTTLS. |
|
||||||
|
| `internal/reply/draft.go` | Reply prompt and OmniRoute call. |
|
||||||
|
| `internal/reply/templates.go` | Built-in reply templates. |
|
||||||
|
| `internal/store/retention.go` | Body clearing and `actions_log` cap. |
|
||||||
|
| `internal/hub/ask.go` | Deterministic, read-only answerer. |
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
All Go tests run in a container:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
timeout 600 docker run --rm -v "$PWD":/src -w /src golang:1.25-alpine go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
| Package | Coverage |
|
||||||
|
| :--- | :--- |
|
||||||
|
| `internal/classify` | JEV client, rule engine, orchestration. |
|
||||||
|
| `internal/hub` | Answer intents, auth, read-only proof, notify. |
|
||||||
|
| `internal/mailfetch` | IMAP parse, MIME, FR-2, write ops. |
|
||||||
|
| `internal/mailops` | Label mapping, SMTP. |
|
||||||
|
| `internal/reply` | Draft prompts and templates. |
|
||||||
|
| `internal/server` | Auth, routes, CSRF, handlers. |
|
||||||
|
| `internal/store` | Schema, retention, triage, categories, summaries. |
|
||||||
|
| `internal/summary` | Counts, ordering, text. |
|
||||||
|
| `internal/unsubscribe` | Link detection. |
|
||||||
|
| `internal/apprise`, `internal/config`, `internal/session` | Client and config. |
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
Go 1.25. The only database driver is `modernc.org/sqlite` (pure Go, no CGO).
|
||||||
|
The mail path uses the Go standard library plus a minimal IMAP client. The UI
|
||||||
|
uses vendored HTMX and the Family Home Lab `tokens.css`. No JavaScript build
|
||||||
|
step.
|
||||||
143
docs/outline/04-agent-runbook.md
Normal file
143
docs/outline/04-agent-runbook.md
Normal file
@@ -0,0 +1,143 @@
|
|||||||
|
# Email Agent — Agent Runbook
|
||||||
|
|
||||||
|
## Before you start
|
||||||
|
|
||||||
|
1. Read `00-index.md`. Inject only the documents you need.
|
||||||
|
2. Read `AGENTS.md` section 11 (build order) and section 10 (rules).
|
||||||
|
3. Check git status and the last commits.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd /home/sam/paseo/projects/email_agent
|
||||||
|
git status
|
||||||
|
git log -n 5 --oneline
|
||||||
|
```
|
||||||
|
|
||||||
|
## Run the tests
|
||||||
|
|
||||||
|
Run the Go tests in a container. The image has the toolchain.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
timeout 600 docker run --rm -v "$PWD":/src -w /src golang:1.25-alpine go test ./...
|
||||||
|
timeout 600 docker run --rm -v "$PWD":/src -w /src golang:1.25-alpine go vet ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `go test ./...` is green and `go vet ./...` is silent.
|
||||||
|
|
||||||
|
## Run locally
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/email-agent
|
||||||
|
curl -s localhost:8790/health
|
||||||
|
```
|
||||||
|
|
||||||
|
The health reply is JSON: `{"status":"ok","database":"ok", ...}`.
|
||||||
|
|
||||||
|
## Build and deploy on .13
|
||||||
|
|
||||||
|
Load the host secrets, then build and start the container.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
set -a; . ~/.config/environment.d/10-secrets.conf; set +a
|
||||||
|
cd /home/sam/paseo/projects/email_agent
|
||||||
|
EMAIL_AGENT_VERSION=phase8 EMAIL_AGENT_COMMIT=$(git rev-parse --short HEAD) docker compose build
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Check the result:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker ps --filter name=email-agent
|
||||||
|
curl -s https://mail.lab.audasmedia.com.au/health
|
||||||
|
```
|
||||||
|
|
||||||
|
## Check the live service
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -s http://127.0.0.1:8790/health
|
||||||
|
curl -s http://127.0.0.1:8790/readyz
|
||||||
|
timeout 10 docker exec email-agent email-agent -version
|
||||||
|
timeout 10 docker exec email-agent email-agent -check-smtp
|
||||||
|
```
|
||||||
|
|
||||||
|
`-check-smtp` verifies IPv4, STARTTLS, and login. It sends no mail.
|
||||||
|
|
||||||
|
## Trigger the jobs
|
||||||
|
|
||||||
|
The jobs also run from the dashboard buttons. The CLI runs them without the UI.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
timeout 60 docker exec email-agent email-agent -fetch
|
||||||
|
timeout 120 docker exec email-agent email-agent -classify
|
||||||
|
timeout 60 docker exec email-agent email-agent -summary
|
||||||
|
```
|
||||||
|
|
||||||
|
`-summary-send` sends a real email. Get Sam's approval first. `-retain` clears
|
||||||
|
old bodies and caps the log.
|
||||||
|
|
||||||
|
## The Hub API
|
||||||
|
|
||||||
|
`HUB_TOKEN` protects `/capabilities` and `/ask`. `email_agent` is registered
|
||||||
|
with the running Hub.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
set -a; . ~/.config/environment.d/10-secrets.conf; set +a
|
||||||
|
curl -s -H "Authorization: Bearer $HUB_TOKEN" http://127.0.0.1:8790/capabilities
|
||||||
|
curl -s -H "Authorization: Bearer $HUB_TOKEN" -H 'Content-Type: application/json' \
|
||||||
|
-d '{"question":"anything important in my inbox?","asker":"sam"}' \
|
||||||
|
http://127.0.0.1:8790/ask
|
||||||
|
```
|
||||||
|
|
||||||
|
`/ask` is read-only. It never writes to the database.
|
||||||
|
|
||||||
|
## Restart
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose restart email-agent
|
||||||
|
docker logs --tail 50 email-agent
|
||||||
|
```
|
||||||
|
|
||||||
|
## Roll back
|
||||||
|
|
||||||
|
1. `git revert <commit> && docker compose up -d --build`.
|
||||||
|
2. The database is on the `email_agent_data` volume. It is not changed by a code
|
||||||
|
revert. A schema migration is forward-only; restore a database copy to go
|
||||||
|
back a schema version.
|
||||||
|
3. To disable the Hub API, remove `HUB_TOKEN` from the host secrets and restart.
|
||||||
|
|
||||||
|
## Rebuild the documentation
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd /home/sam/paseo/projects/email_agent
|
||||||
|
mmdc -i docs/email-agent-architecture.mmd -o docs/diagrams/email-agent-architecture.png --scale 2 -b white
|
||||||
|
cd /home/sam/.agents/skills/archify
|
||||||
|
node bin/archify.mjs validate architecture /home/sam/paseo/projects/email_agent/docs/archify/email-agent.architecture.json --quality showcase --json
|
||||||
|
node bin/archify.mjs deliver architecture /home/sam/paseo/projects/email_agent/docs/archify/email-agent.architecture.json /home/sam/paseo/projects/email_agent/docs/email-agent-map.html --quality showcase --json
|
||||||
|
```
|
||||||
|
|
||||||
|
## Publish the map
|
||||||
|
|
||||||
|
```sh
|
||||||
|
~/.agents/skills/project-docs/scripts/publish-maps.sh /home/sam/paseo/projects/email_agent
|
||||||
|
```
|
||||||
|
|
||||||
|
Result: `https://maps.lab.audasmedia.com.au/email_agent/docs/`.
|
||||||
|
|
||||||
|
## Documentation surfaces
|
||||||
|
|
||||||
|
| Surface | Location |
|
||||||
|
| :--- | :--- |
|
||||||
|
| Gitea | `ssh://git@gitea.lab.audasmedia.com.au:2222/sam/email_agent.git` |
|
||||||
|
| Obsidian | Vault `010 inbox` on `.27` (the sorter files it later). |
|
||||||
|
| Outline | Collection `Email Agent` on `outline.lab.audasmedia.com.au`. |
|
||||||
|
| Vikunja | Project `Email Agent` on `vikunja.lab.audasmedia.com.au`. |
|
||||||
|
| Map | `central_ai_llm_master/docs/hub-architecture-map.html` and this repo's map. |
|
||||||
|
|
||||||
|
## Safety rules
|
||||||
|
|
||||||
|
- Force IPv4 for every Gmail connection. IPv6 fails from `.13`.
|
||||||
|
- Never commit a secret.
|
||||||
|
- Never send a real email as a test. Ask Sam first.
|
||||||
|
- Never follow a real unsubscribe link. Only mark and record.
|
||||||
|
- Spam is recommended, never auto-deleted.
|
||||||
|
- Bound every `docker run` with `timeout` and `--entrypoint sh`. A bare
|
||||||
|
`docker run` starts the server and hangs.
|
||||||
134
docs/phase8-documentation.md
Normal file
134
docs/phase8-documentation.md
Normal file
@@ -0,0 +1,134 @@
|
|||||||
|
# Phase 8 — Documentation
|
||||||
|
|
||||||
|
Status: **complete.** Ran the `project-docs` skill (Simplified Technical
|
||||||
|
English) and updated all five documentation surfaces. Parent
|
||||||
|
[AUD-68](/AUD/issues/AUD-68). Phase 8 of AGENTS.md §11. Task
|
||||||
|
[AUD-74](/AUD/issues/AUD-74).
|
||||||
|
|
||||||
|
**DoD:** project-docs run; Gitea, Obsidian, Outline, Vikunja, and the map are
|
||||||
|
updated. — **met.** Evidence for each surface is below.
|
||||||
|
|
||||||
|
## 1. Surfaces updated
|
||||||
|
|
||||||
|
| Surface | Destination | What changed |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| Gitea | `sam/email_agent` (repo) | README refreshed (architecture diagram, Phase 8 status, Phase 7 endpoints, retention vars, documentation index). Added `docs/outline/` (5 AI-context docs), `docs/vikunja-tasks.json`, `docs/email-agent-architecture.mmd`, `docs/diagrams/email-agent-architecture.png`, `docs/archify/email-agent.architecture.json`, `docs/email-agent-map.html`. |
|
||||||
|
| Obsidian | Vault `010 inbox` on `.27` | New note `Email Agent — Family Inbox Triage.md` (human overview, portfolio highlights, Mermaid, links). The `obsidian-sorter` watcher proposes the final folder. |
|
||||||
|
| Outline | `outline.lab.audasmedia.com.au` | New collection `Email Agent` (`f096RBNsbV`) with five documents: AI index, system overview, configuration, code map, runbook. |
|
||||||
|
| Vikunja | `vikunja.lab.audasmedia.com.au` | New project `Email Agent` (id 17) with seven completed human milestones (task ids 179–185). |
|
||||||
|
| Map | `central_ai_llm_master/docs/` | `hub-architecture.mmd` and the Archify IR now show the Email Agent as a live peer. `hub-architecture-map.html` re-delivered, `diagrams/hub-architecture.png` re-rendered, and `routing.md` marks `email_agent` an active service. |
|
||||||
|
|
||||||
|
## 2. Gitea
|
||||||
|
|
||||||
|
Repo: `ssh://git@gitea.lab.audasmedia.com.au:2222/sam/email_agent.git`.
|
||||||
|
|
||||||
|
- `README.md` — status is now "complete and deployed" for all eight phases. Adds
|
||||||
|
the Mermaid architecture diagram, the Phase 7 Hub API and Reports endpoints,
|
||||||
|
the retention variables, and a Documentation section.
|
||||||
|
- `docs/outline/00-index.md` — the AI document index.
|
||||||
|
- `docs/outline/01-system-overview.md` — purpose, architecture, FR map, phase
|
||||||
|
status.
|
||||||
|
- `docs/outline/02-config-and-env.md` — environment variables, paths, ports.
|
||||||
|
- `docs/outline/03-code-map.md` — folders, flags, routes, schema, tests.
|
||||||
|
- `docs/outline/04-agent-runbook.md` — test, deploy, restart, rollback.
|
||||||
|
- `docs/vikunja-tasks.json` — the Vikunja project id and task ids.
|
||||||
|
- `docs/email-agent-architecture.mmd` and `docs/diagrams/email-agent-architecture.png`.
|
||||||
|
- `docs/archify/email-agent.architecture.json` and `docs/email-agent-map.html`.
|
||||||
|
|
||||||
|
The Archify map passed the **showcase** profile: 9 of 9 checks, 0 errors,
|
||||||
|
0 warnings. The Mermaid diagram rendered with `mmdc` (no errors).
|
||||||
|
|
||||||
|
## 3. Obsidian
|
||||||
|
|
||||||
|
Written over SSH to `.27`:
|
||||||
|
|
||||||
|
```
|
||||||
|
/home/sam/obsidian/010 inbox/Email Agent — Family Inbox Triage.md
|
||||||
|
```
|
||||||
|
|
||||||
|
The note uses the generic inbox template (`type: note`) with `project` and
|
||||||
|
topic tags. It holds a plain-language explanation, the portfolio highlights,
|
||||||
|
a Mermaid diagram, and links to Gitea, the map, Outline, and Vikunja. No folder
|
||||||
|
and no note type were chosen; the `obsidian-sorter` watcher proposes both and
|
||||||
|
Sam approves.
|
||||||
|
|
||||||
|
## 4. Outline
|
||||||
|
|
||||||
|
Collection `Email Agent`, id `ccb6b59b-2bf6-4946-8dca-5b6da95f768a`, url id
|
||||||
|
`f096RBNsbV`.
|
||||||
|
|
||||||
|
| Document | Id |
|
||||||
|
| :--- | :--- |
|
||||||
|
| Email Agent — AI Document Index | `d1025188-6640-4f3d-8e46-41439faed342` |
|
||||||
|
| Email Agent — System Overview | `cdcdf640-425b-42ad-843c-96d58ead81a2` |
|
||||||
|
| Email Agent — Configuration and Environment | `1a239b9c-63a4-4a5e-9c12-56626c4d2d07` |
|
||||||
|
| Email Agent — Code Map | `924617ea-47fd-4ab4-9eef-e2dfc7c3f854` |
|
||||||
|
| Email Agent — Agent Runbook | `e745a791-1c07-4ad4-bc30-6a4f6119dceb` |
|
||||||
|
|
||||||
|
The documents are the same files as `docs/outline/` in the repo. The index
|
||||||
|
itemizes every document so an AI agent can inject the exact file.
|
||||||
|
|
||||||
|
## 5. Vikunja
|
||||||
|
|
||||||
|
Project `Email Agent`, id 17.
|
||||||
|
|
||||||
|
| Task id | Title | Done |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| 179 | Build and deploy phases 1–7 | yes |
|
||||||
|
| 180 | Deploy on `.13` behind Caddy | yes |
|
||||||
|
| 181 | Gitea README, Mermaid, Archify, publish maps | yes |
|
||||||
|
| 182 | Outline collection `Email Agent` | yes |
|
||||||
|
| 183 | Obsidian note in `010 inbox` | yes |
|
||||||
|
| 184 | 1-month retention and `actions_log` cap (AUD-75) | yes |
|
||||||
|
| 185 | Confirm the Gmail send-as identity (FR-15) | yes |
|
||||||
|
|
||||||
|
The repo copy is `docs/vikunja-tasks.json`.
|
||||||
|
|
||||||
|
## 6. The map
|
||||||
|
|
||||||
|
`central_ai_llm_master/docs/` is the Hub's map. The Email Agent now appears as
|
||||||
|
a live peer:
|
||||||
|
|
||||||
|
- `docs/hub-architecture.mmd` — added `EMAIL["Email Agent<br/>inbox triage"]`
|
||||||
|
and `HUB -- "ask only (query)" --> EMAIL`.
|
||||||
|
- `docs/archify/hub.architecture.json` — added the `email` component and the
|
||||||
|
`to-email` relationship; the profile is `standard` (the existing Hub map was
|
||||||
|
already a standard-quality map — it fails the stricter showcase profile).
|
||||||
|
- `docs/hub-architecture-map.html` — re-delivered: 9 of 9 checks, 0 errors,
|
||||||
|
30 warnings.
|
||||||
|
- `docs/diagrams/hub-architecture.png` — re-rendered from the updated Mermaid.
|
||||||
|
- `routing.md` — the `email, inbox, mail, message, unread` row is now
|
||||||
|
`active service`, and `email_agent` is removed from the placeholder table.
|
||||||
|
|
||||||
|
The Email Agent's own map is published to
|
||||||
|
`https://maps.lab.audasmedia.com.au/email_agent/docs/`, and the Hub map to
|
||||||
|
`https://maps.lab.audasmedia.com.au/central_ai_llm_master/docs/`.
|
||||||
|
|
||||||
|
## 7. Commands used
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Mermaid
|
||||||
|
mmdc -i docs/email-agent-architecture.mmd -o docs/diagrams/email-agent-architecture.png --scale 2 -b white
|
||||||
|
|
||||||
|
# Archify (validate then deliver)
|
||||||
|
node ~/.agents/skills/archify/bin/archify.mjs validate architecture docs/archify/email-agent.architecture.json --quality showcase --json
|
||||||
|
node ~/.agents/skills/archify/bin/archify.mjs deliver architecture docs/archify/email-agent.architecture.json docs/email-agent-map.html --quality showcase --json
|
||||||
|
|
||||||
|
# Publish to the LAN maps site
|
||||||
|
~/.agents/skills/project-docs/scripts/publish-maps.sh /home/sam/paseo/projects/email_agent
|
||||||
|
~/.agents/skills/project-docs/scripts/publish-maps.sh /home/sam/paseo/projects/central_ai_llm_master
|
||||||
|
```
|
||||||
|
|
||||||
|
## 8. Not done / notes
|
||||||
|
|
||||||
|
- The Obsidian note is in the inbox. The sorter, not this phase, chooses the
|
||||||
|
final folder. Sam approves the proposal in Obsidian.
|
||||||
|
- Vikunja bucket (stage) placement uses the API default views. The tasks carry
|
||||||
|
their done flags; a human can drag them if a stage is wanted.
|
||||||
|
- 1-month retention is covered by AUD-75 (commit `e5cce1c`), not by this phase.
|
||||||
|
- The project-docs role says a note is finished only when its path is named in
|
||||||
|
the task. This file and the summary comment name each path.
|
||||||
|
|
||||||
|
## 9. Commit
|
||||||
|
|
||||||
|
Docs commit for this phase: `<pending>`.
|
||||||
15
docs/vikunja-tasks.json
Normal file
15
docs/vikunja-tasks.json
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"project": "Email Agent",
|
||||||
|
"vikunja_project_id": 17,
|
||||||
|
"url": "https://vikunja.lab.audasmedia.com.au",
|
||||||
|
"description": "Human task overview for the Email Agent. High-level milestones; the engineering detail lives in the Gitea repo and the Outline collection.",
|
||||||
|
"tasks": [
|
||||||
|
{ "id": 179, "title": "Build and deploy Email Agent phases 1-7 (Go, HTMX, SQLite, IMAP, JEV, triage, replies, unsubscribe, summary, Hub)", "done": true },
|
||||||
|
{ "id": 180, "title": "Deploy on .13 behind Caddy at mail.lab.audasmedia.com.au", "done": true },
|
||||||
|
{ "id": 181, "title": "Gitea sam/email_agent: README, Mermaid diagram, Archify map, publish maps", "done": true },
|
||||||
|
{ "id": 182, "title": "Outline collection Email Agent with AI context docs", "done": true },
|
||||||
|
{ "id": 183, "title": "Obsidian note for Email Agent in 010 inbox", "done": true },
|
||||||
|
{ "id": 184, "title": "1-month message-body retention and actions_log cap (AUD-75)", "done": true },
|
||||||
|
{ "id": 185, "title": "Confirm the Gmail send-as identity at Phase 5 (FR-15)", "done": true }
|
||||||
|
]
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user