docs: Phase 7 summary + Hub evidence (AUD-73)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
251
docs/phase7-summary-hub.md
Normal file
251
docs/phase7-summary-hub.md
Normal file
@@ -0,0 +1,251 @@
|
||||
# Phase 7 — Summary and Hub
|
||||
|
||||
Status: **built, tested, deployed on `.13`, and the project side of the DoD is
|
||||
demonstrated against real mail.** Two items wait on Sam's explicit approval: the
|
||||
real summary email (Apprise key) and registering this project with the running
|
||||
Hub. Parent [AUD-68](/AUD/issues/AUD-68). Phase 7 of AGENTS.md §11. Code commit
|
||||
`a349b42`.
|
||||
|
||||
**DoD:** the Hub asks "anything important in my inbox?" and gets a correct short
|
||||
answer. — the Hub contract round-trip works (§2.1). Wiring the live Hub requires
|
||||
a shared token and a Hub restart, which is outside the "do not touch other
|
||||
projects" rule for this pass (see §7).
|
||||
|
||||
## 1. What was built
|
||||
|
||||
### 1.1 Hub API (FR-12)
|
||||
|
||||
The project side of the frozen contract
|
||||
(`central_ai_llm_master/docs/ask-contract.md`) lives in `internal/hub` and
|
||||
`internal/server/hub.go`.
|
||||
|
||||
| Endpoint | Auth | Behaviour |
|
||||
|---|---|---|
|
||||
| `GET /capabilities` | `Authorization: Bearer $HUB_TOKEN` | `{"project":"email_agent","topics":["email","inbox","mail","message"],"people":["sam"]}` |
|
||||
| `POST /ask` | `Authorization: Bearer $HUB_TOKEN` | A short, speakable answer or `{"answer":null,"reason":…}` |
|
||||
|
||||
- **Read-only.** `/ask` calls only `store.CountByCategory` and
|
||||
`store.MessagesByCategory`. It never writes, including no `actions_log` row
|
||||
(FR-13 is for actions, not reads). Proven live in §2.3 and by the unit test
|
||||
`TestAskWritesNothing`.
|
||||
- **No LLM, fast.** The answerer is deterministic and answers from SQLite in
|
||||
milliseconds. The 3-second contract budget is never at risk.
|
||||
- **Never guesses.** An unanswerable question returns `answer: null` plus a
|
||||
reason. The service does not track read/unread, so it says so rather than
|
||||
inventing a number.
|
||||
- **Token required.** With no `HUB_TOKEN` configured the endpoints return `503`;
|
||||
a wrong token returns `401`. No project trusts an unsigned request.
|
||||
- Topics deliberately omit `unread` (see `routing.md`), because the service has
|
||||
no read-state store.
|
||||
|
||||
Intents answered: **important / urgent / action**, **spam**, **newsletter /
|
||||
unsubscribe**, **inbox count**, and a one-sentence **overview / summary**.
|
||||
|
||||
### 1.2 Outbound notify (ask-contract "Notify (project to Hub)")
|
||||
|
||||
`internal/hub/notify.go` posts to `{HUB_BASE_URL}/notify` with
|
||||
`{"from","level","message","person"}` and the project bearer token. It is the
|
||||
one Hub call this service makes. The Hub's `/notify` receiver is not part of its
|
||||
frozen v1 routes (`docs/hub-api.md` §12), so the call is best-effort and a
|
||||
failure is logged, never blocking local work. The CLI `-notify` exercises it.
|
||||
|
||||
### 1.3 Daily summary (FR-10)
|
||||
|
||||
- `internal/summary` builds a deterministic plain-text summary: inbox total,
|
||||
per-category counts, the "needs attention" list, newsletter count, and spam
|
||||
count (with the "recommend, never auto-delete" wording from AGENTS.md §10.5).
|
||||
- `internal/store/summary.go` adds `CountByCategory`, `MessagesByCategory`, and
|
||||
summary CRUD over the existing `summaries` table. **No schema migration** was
|
||||
needed; the table exists since v1.
|
||||
- `internal/apprise` delivers the summary by email through the house Apprise API
|
||||
(`POST {APPRISE_URL}/notify/{APPRISE_KEY}`, or the stateless `/notify` form
|
||||
with `APPRISE_URLS`). Set `APPRISE_TOKEN` if the API is protected.
|
||||
- Delivery happens in exactly two explicit paths: the CLI `-summary-send` (for
|
||||
the daily n8n schedule) and the Reports screen's **Send now** button. Building
|
||||
a summary never sends mail.
|
||||
- `GET /reports` is the Reports screen (AGENTS.md §5): build today's summary,
|
||||
send it, and delete a stored report. All mutations are CSRF-protected POSTs.
|
||||
|
||||
### 1.4 CLI
|
||||
|
||||
| Flag | Effect |
|
||||
|---|---|
|
||||
| `-summary` | Build today's summary, store it, print it. Sends nothing. |
|
||||
| `-summary-send` | Build, store, and deliver by email via Apprise. |
|
||||
| `-notify` | Send one notification to the Hub (`POST /notify`). |
|
||||
|
||||
The n8n schedule (FR-1) runs `-summary-send` once a day; the summary job is a
|
||||
separate bounded process, like `-fetch` and `-classify`.
|
||||
|
||||
## 2. DoD evidence
|
||||
|
||||
### 2.1 Real `/ask` round-trip
|
||||
|
||||
Against the deployed container on `.13` (commit `a349b42`), sending the exact
|
||||
`ProjectClient` request shape the Hub uses (`content-type: application/json`,
|
||||
`Authorization: Bearer …`):
|
||||
|
||||
```
|
||||
POST /ask
|
||||
{"question":"anything important in my inbox?","asker":"sam"}
|
||||
|
||||
200 {"answer":"20 emails need your attention, including \"Security alert\" from Google.",
|
||||
"data":{"count":20,"messages":[ … ]},"expires":"2026-10-11T11:48:01Z"}
|
||||
```
|
||||
|
||||
The count is correct: the live inbox held **102** messages, of which **19 Action
|
||||
+ 1 Important = 20** needed attention. An overview question returns
|
||||
`"Your inbox has 102 emails: 20 need attention and 40 are newsletters."`
|
||||
|
||||
`GET /capabilities`:
|
||||
|
||||
```
|
||||
{"project":"email_agent","topics":["email","inbox","mail","message"],"people":["sam"]}
|
||||
```
|
||||
|
||||
Auth checks: no token → `401`; wrong token → `401`; correct token → `200`.
|
||||
An unanswerable question returns `{"answer":null,"reason":"I do not track read
|
||||
or unread mail"}`.
|
||||
|
||||
### 2.2 `/notify` path
|
||||
|
||||
`email-agent -notify` against a local stub implementing the contract captured:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/notify",
|
||||
"auth": "Bearer <project token>",
|
||||
"body": {
|
||||
"from": "email_agent",
|
||||
"level": "info",
|
||||
"message": "Email Agent notify path check.",
|
||||
"person": "sam"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This matches the `NotifyRequest` shape in `ask-contract.md` exactly.
|
||||
|
||||
### 2.3 `/ask` is read-only
|
||||
|
||||
Five `/ask` calls (important, count, spam, unknown, overview) were issued against
|
||||
the deployed service. The `/data` SQLite database was copied before and after and
|
||||
compared row-by-row:
|
||||
|
||||
| table | before | after | delta |
|
||||
|---|---|---|---|
|
||||
| messages | 103 | 103 | +0 |
|
||||
| classifications | 403 | 403 | +0 |
|
||||
| actions_log | 428 | 428 | +0 |
|
||||
| summaries | 0 | 0 | +0 |
|
||||
| rules | 2 | 2 | +0 |
|
||||
| unsubscribe | 0 | 0 | +0 |
|
||||
| drafts | 2 | 2 | +0 |
|
||||
|
||||
No table changed. The unit test `TestAskWritesNothing` asserts the same property
|
||||
for the message, classification, action-log, and summary counts.
|
||||
|
||||
### 2.4 Daily-summary-by-email path
|
||||
|
||||
`docker exec email-agent email-agent -summary` produced and stored the summary
|
||||
(row `id=1`, date `2026-10-11`, 1110 bytes, `delivered_to` empty):
|
||||
|
||||
```
|
||||
Email Agent — daily summary for 2026-10-11
|
||||
|
||||
Inbox: 102 message(s)
|
||||
Newsletter 40
|
||||
Auto-reply 34
|
||||
Action 19
|
||||
Info 7
|
||||
Important 1
|
||||
Spam 1
|
||||
|
||||
Needs attention: 20
|
||||
- Security alert — Google [Action/high]
|
||||
- …
|
||||
|
||||
Newsletters: 40
|
||||
Spam (recommend, never auto-delete): 1
|
||||
```
|
||||
|
||||
`-summary-send` with no Apprise key stops before sending:
|
||||
`summary failed: apprise is not configured: set APPRISE_URL and APPRISE_KEY (or
|
||||
APPRISE_URLS)`. **No summary email has been sent.** A real send waits on Sam's
|
||||
approval and an Apprise key (§7).
|
||||
|
||||
### 2.5 Automated tests
|
||||
|
||||
`gofmt -l` clean, `go vet ./...` clean, `go test ./...` green (in
|
||||
`golang:1.25-alpine`). New coverage:
|
||||
|
||||
| Package | Tests |
|
||||
|---|---|
|
||||
| `internal/hub` | important/empty/spam/newsletter/count/summary/unknown intents, person and asker checks, unread refusal, read errors, sender-name extraction, default capabilities, notify payload/auth/error/no-op |
|
||||
| `internal/apprise` | key mode, stateless URL mode, auth header, error status, `Configured` |
|
||||
| `internal/summary` | build counts and ordering, text rendering, empty inbox |
|
||||
| `internal/store` | category counts, multi-category importance ordering, summary upsert/get/list/delete/delivery |
|
||||
| `internal/config` | Hub/Apprise defaults, `HubConfigured`, `AppriseConfigured`, URL-list parsing |
|
||||
| `internal/server` | capabilities auth + shape, ask auth, ask answer, **ask writes nothing**, reports page auth, generate stores without sending, send delivers through Apprise, CSRF |
|
||||
|
||||
## 3. Configuration
|
||||
|
||||
New environment variables (names only; values live in
|
||||
`~/.config/environment.d/10-secrets.conf` on `.13` and are never committed):
|
||||
|
||||
| Variable | Purpose |
|
||||
|---|---|
|
||||
| `HUB_TOKEN` | Shared bearer token for `/capabilities` and `/ask`. Also the token sent to the Hub's `/notify`. |
|
||||
| `HUB_BASE_URL` | Hub base URL for outbound notify (default `http://192.168.20.13:8788`). |
|
||||
| `HUB_NOTIFY_TIMEOUT` | Notify call timeout (default `5s`). |
|
||||
| `APPRISE_URL` | Apprise API base (default `http://192.168.20.35:8210`). |
|
||||
| `APPRISE_KEY` | Stored Apprise configuration key. |
|
||||
| `APPRISE_URLS` | Stateless Apprise destinations (used when no key). |
|
||||
| `APPRISE_TOKEN` | Optional bearer token for a protected Apprise API. |
|
||||
| `EMAIL_AGENT_SUMMARY_TO` | Summary recipient (default `samuelrolfe@gmail.com`). |
|
||||
|
||||
A `HUB_TOKEN` was generated and written to the secrets file on `.13` (mode 600,
|
||||
backup `10-secrets.conf.bak.aud73-*`). The value is not in this repo.
|
||||
|
||||
## 4. Deployment
|
||||
|
||||
On `.13`, from the repo, with the host secrets in scope:
|
||||
|
||||
```sh
|
||||
set -a; . ~/.config/environment.d/10-secrets.conf; set +a
|
||||
EMAIL_AGENT_VERSION=phase7 EMAIL_AGENT_COMMIT=$(git rev-parse --short HEAD) \
|
||||
docker compose build
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
The migration-free change started in place on the Phase 1–6 volume
|
||||
(`schema_version: 6`). `https://mail.lab.audasmedia.com.au/health` → `200`.
|
||||
|
||||
## 5. Rollback
|
||||
|
||||
1. `git revert a349b42 && docker compose up -d --build`.
|
||||
2. No schema change to undo. The `summaries` table rows and the `HUB_TOKEN`
|
||||
secret are left in place; remove the token from the secrets file if the Hub
|
||||
API must be disabled.
|
||||
3. The Reports screen, `/capabilities`, and `/ask` disappear with the revert;
|
||||
the running Hub is unaffected because it is not yet registered.
|
||||
|
||||
## 6. Not done / deferred
|
||||
|
||||
- **1-month body retention** (§7) is still not implemented.
|
||||
- The Hub's `/notify` receiver does not exist yet in the Hub v1, so the outbound
|
||||
path is verified against a stub, not the live Hub.
|
||||
- The Hub routing decision is LLM-based (`understand.OmniRouter`). This pass did
|
||||
not run the Hub pipeline, to respect "the Hub endpoints are the only external
|
||||
calls" — no OmniRoute routing call was made.
|
||||
|
||||
## 7. Waiting on Sam (two approvals)
|
||||
|
||||
1. **Real summary email.** Provide the Apprise key (or URLs) and approve one
|
||||
real send for evidence. Until then no summary email is sent.
|
||||
2. **Register this project with the Hub.** Add
|
||||
`HUB_PROJECT_EMAIL_AGENT_TOKEN=<HUB_TOKEN>` and
|
||||
`HUB_PROJECTS=email_agent=http://192.168.20.13:8790` to the Hub's environment
|
||||
and restart the `hub` container. This touches a running family service, so it
|
||||
needs Sam's yes. After that the Hub can route "anything important in my
|
||||
inbox?" through `routing.md` to this project.
|
||||
Reference in New Issue
Block a user