Phase 3: JEV classification, rules table, editable categories (AUD-69)
Implements FR-3, FR-4, FR-5 and FR-14. - internal/classify: TypeSafe System One (JEV) HTTP client with the confirmed request/response shape, retry with backoff on 429/529/5xx, a plain-text rule engine (first match wins, overrides only the named fields), and bounded orchestration with a per-run cap. - internal/store: migration v3 adds an editable `categories` table (name = JEV Choice option and classifications key; rename cascades, remove reassigns) and seeds a starter rules version once. Classification persistence, coverage, and pending/all message queries. - internal/server: POST /actions/classify and /actions/reclassify, plus the FR-14 category editor (GET /categories and add/rename/remove). The triage list now groups by the live category list and shows action + confidence. - CLI: -classify and -classify-all for n8n. - Docs: docs/phase3-classify.md with the confirmed JEV shape, a live read-only classification, and the DoD evidence. DoD: all 100 stored messages have a non-null category, importance, and suggested action (0 nulls); JEV picked a newly added/renamed category for 6 real messages over HTTP. One live JEV call recorded in the phase doc. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
@@ -31,8 +31,14 @@ EMAIL_AGENT_IMAP_LOOKBACK_DAYS=14
|
||||
EMAIL_AGENT_IMAP_MAX_MESSAGES=100
|
||||
|
||||
# --- Integrations (used from later phases) ---------------------------------
|
||||
# JEV is the TypeSafe System One HTTP API. JEV_BASE_URL defaults to the
|
||||
# documented host https://api.typesafe.ai when unset.
|
||||
JEV_BASE_URL=
|
||||
JEV_API_KEY=
|
||||
JEV_MODEL=jev-latest
|
||||
# Classifier bounds (FR-3): in-flight JEV requests, and messages per run.
|
||||
EMAIL_AGENT_CLASSIFY_CONCURRENCY=4
|
||||
EMAIL_AGENT_CLASSIFY_MAX_MESSAGES=200
|
||||
OMNIROUTE_BASE_URL=
|
||||
OMNIROUTE_API_KEY=
|
||||
HUB_TOKEN=
|
||||
|
||||
80
README.md
80
README.md
@@ -9,18 +9,23 @@ pre-planning material and is not authoritative.
|
||||
|
||||
## Status
|
||||
|
||||
**Phase 2 — Read mail (IMAP).** The Go skeleton (Phase 1.1) plus session login
|
||||
and the HTMX dashboard shell (Phase 1.2), now fetching real mail from Gmail over
|
||||
**IMAP with an app password, forced to IPv4** (AGENTS.md §9). Messages are stored
|
||||
in SQLite and listed in the dashboard, grouped by category with a recommended
|
||||
action per row. The FR-2 source address is resolved from `X-Pm-Original-To`, with
|
||||
`To`, the PurelyMail `Received … for` clause, and `Return-Path` as fallbacks;
|
||||
`Delivered-To` is never used. Classification and triage actions come in Phase 3+.
|
||||
**Phase 3 — Classify.** The Go skeleton, session login, and HTMX dashboard
|
||||
(Phase 1), real mail over **IMAP with an app password, forced to IPv4**
|
||||
(Phase 2), and now **JEV classification** (Phase 3). Every message gets a
|
||||
category, an importance, and a suggested action, grouped in the triage list with
|
||||
the recommended action and confidence per row. The category list is editable in
|
||||
the UI (add, rename, remove) and JEV uses the current list; a plain-text rules
|
||||
table can override the classifier (FR-4). The FR-2 source address is resolved
|
||||
from `X-Pm-Original-To`, with `To`, the PurelyMail `Received … for` clause, and
|
||||
`Return-Path` as fallbacks; `Delivered-To` is never used. See
|
||||
[`docs/phase3-classify.md`](docs/phase3-classify.md) for the confirmed JEV shape
|
||||
and the phase evidence. Triage actions come in Phase 4.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
cmd/email-agent/ service entry point (`-fetch` runs one scan and exits)
|
||||
cmd/email-agent/ service entry point (`-fetch`, `-classify`, `-classify-all`)
|
||||
internal/classify/ JEV (TypeSafe System One) client, rule engine, orchestration
|
||||
internal/config/ environment-driven configuration
|
||||
internal/mailfetch/ stdlib-only IMAP client, MIME parsing, FR-2 source address
|
||||
internal/session/ server-side session store + signed cookie values
|
||||
@@ -57,6 +62,11 @@ Key variables:
|
||||
| `EMAIL_AGENT_IMAP_PORT` | `993` | IMAP port (implicit TLS) |
|
||||
| `EMAIL_AGENT_IMAP_LOOKBACK_DAYS` | `14` | First-scan window |
|
||||
| `EMAIL_AGENT_IMAP_MAX_MESSAGES` | `100` | Per-scan message cap |
|
||||
| `JEV_BASE_URL` | `https://api.typesafe.ai` | TypeSafe System One base URL |
|
||||
| `JEV_API_KEY` | _(empty)_ | JEV API key (required to classify) |
|
||||
| `JEV_MODEL` | `jev-latest` | System One model alias |
|
||||
| `EMAIL_AGENT_CLASSIFY_CONCURRENCY` | `4` | In-flight JEV requests per run |
|
||||
| `EMAIL_AGENT_CLASSIFY_MAX_MESSAGES` | `200` | Messages per classify run |
|
||||
|
||||
## Mail sync (Phase 2)
|
||||
|
||||
@@ -75,6 +85,45 @@ Each scan is bounded by the lookback and max-message settings. Gmail's
|
||||
`X-GM-MSGID` is the stable message key; the IMAP `UID` and `UIDVALIDITY` are
|
||||
stored alongside it for later label operations.
|
||||
|
||||
## Classification (Phase 3)
|
||||
|
||||
`internal/classify` sends each message to JEV (`POST /v1/systemone`) as three
|
||||
Choice questions — category, importance, suggested action — with the **live**
|
||||
category list as the category options, and stores the answer plus JEV's own
|
||||
confidence. One request per message, bounded concurrency, retry with backoff on
|
||||
`429`/`529`/`5xx`.
|
||||
|
||||
Trigger classification from the dashboard (**Classify** for new mail,
|
||||
**Re-classify all** after a category or rules change) or from the CLI:
|
||||
|
||||
```sh
|
||||
docker exec email-agent email-agent -classify # pending messages
|
||||
docker exec email-agent email-agent -classify-all # every stored message
|
||||
```
|
||||
|
||||
### Rules (FR-4)
|
||||
|
||||
Rules are plain text, stored versioned in `rules`. One rule per line:
|
||||
|
||||
```
|
||||
<condition> => <effects>
|
||||
|
||||
from contains "amazon.com" => category=Newsletter, action=unsubscribe
|
||||
* => category=Info
|
||||
```
|
||||
|
||||
Conditions join with ` and ` over `from`, `to`, `subject`, `source`, `body`
|
||||
using `contains`, `equals`, `starts_with`, `ends_with`. The first matching rule
|
||||
wins and overrides only the fields it names. The rules editor UI is Phase 6; the
|
||||
classifier already reads the stored text.
|
||||
|
||||
### Categories (FR-14)
|
||||
|
||||
`GET /categories` edits the category list. A category's name is the key stored
|
||||
in `classifications.category` and the Choice option sent to JEV, so the model
|
||||
always sees the current list. Renaming cascades to existing classifications;
|
||||
removing a category reassigns its messages to the first remaining category.
|
||||
|
||||
## Run locally
|
||||
|
||||
```sh
|
||||
@@ -94,7 +143,7 @@ The container is `email-agent`, image `email-agent:latest`, named volume
|
||||
unprivileged user and has a built-in healthcheck (`email-agent -healthcheck`).
|
||||
Caddy on `.35` proxies `mail.lab.audasmedia.com.au` to this port.
|
||||
|
||||
## Endpoints (Phase 1)
|
||||
## Endpoints
|
||||
|
||||
| Method | Path | Auth | Purpose |
|
||||
|---|---|---|---|
|
||||
@@ -106,6 +155,12 @@ Caddy on `.35` proxies `mail.lab.audasmedia.com.au` to this port.
|
||||
| `GET` | `/` | session | Dashboard: nav counts, real message list |
|
||||
| `GET` | `/partials/inbox` | session | HTMX fragment for the centre list |
|
||||
| `POST` | `/actions/fetch` | session + CSRF | Runs one IMAP scan, returns the refreshed list |
|
||||
| `POST` | `/actions/classify` | session + CSRF | Classifies pending messages, returns the refreshed list |
|
||||
| `POST` | `/actions/reclassify` | session + CSRF | Re-classifies every message with the current categories and rules |
|
||||
| `GET` | `/categories` | session | Category editor (FR-14) |
|
||||
| `POST` | `/categories/add` | session + CSRF | Adds a category |
|
||||
| `POST` | `/categories/rename` | session + CSRF | Renames a category and updates existing classifications |
|
||||
| `POST` | `/categories/remove` | session + CSRF | Removes a category and reassigns its messages |
|
||||
| `GET` | `/static/…` | public | Embedded `tokens.css`, `app.css`, Inter fonts, `htmx.min.js` |
|
||||
|
||||
Unauthenticated requests to `/` and `/partials/…` redirect to `/login?next=…`.
|
||||
@@ -117,6 +172,7 @@ hidden `csrf` field).
|
||||
SQLite is opened with the pure-Go `modernc.org/sqlite` driver (no CGO). The
|
||||
schema in `internal/store/schema.sql` creates the AGENTS.md §7 tables:
|
||||
`messages`, `classifications`, `rules`, `actions_log`, `unsubscribe`, `drafts`,
|
||||
`summaries`, plus `schema_migrations`. Schema init is idempotent and runs on
|
||||
every start. Migration v2 adds `imap_uid` and `imap_uidvalidity` to `messages`
|
||||
for databases created by Phase 1.
|
||||
`summaries`, plus `categories`, `schema_migrations`. Schema init is idempotent
|
||||
and runs on every start. Migration v2 adds `imap_uid` and `imap_uidvalidity` to
|
||||
`messages`; migration v3 seeds the first category list and a starter rules
|
||||
version once, so a category removed by the user is not silently re-added.
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/classify"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/config"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/mailfetch"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/server"
|
||||
@@ -24,6 +25,8 @@ func main() {
|
||||
showVersion = flag.Bool("version", false, "print version information and exit")
|
||||
healthcheck = flag.Bool("healthcheck", false, "probe the local /health endpoint and exit")
|
||||
fetchNow = flag.Bool("fetch", false, "run one Gmail IMAP scan and exit")
|
||||
classifyNow = flag.Bool("classify", false, "classify pending messages with JEV and exit")
|
||||
classifyAll = flag.Bool("classify-all", false, "re-classify every stored message with JEV and exit")
|
||||
)
|
||||
flag.Parse()
|
||||
|
||||
@@ -57,6 +60,14 @@ func main() {
|
||||
return
|
||||
}
|
||||
|
||||
if *classifyNow || *classifyAll {
|
||||
if err := runClassify(cfg, logger, *classifyAll); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "classify failed: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if err := run(cfg, logger); err != nil {
|
||||
logger.Error("fatal", "error", err)
|
||||
os.Exit(1)
|
||||
@@ -125,6 +136,41 @@ func runFetch(cfg *config.Config, logger *slog.Logger) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// runClassify classifies messages and exits. It is the n8n/CLI entry point for
|
||||
// FR-3, separate from the HTTP service so a scheduled scan can be bounded and
|
||||
// observed. all=true re-classifies every message, for use after the category
|
||||
// list or rules change.
|
||||
func runClassify(cfg *config.Config, logger *slog.Logger, all bool) error {
|
||||
if !cfg.JEVConfigured() {
|
||||
return classify.ErrNotConfigured
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
st, err := store.Open(ctx, cfg.DBPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("open store: %w", err)
|
||||
}
|
||||
defer func() { _ = st.Close() }()
|
||||
|
||||
classifier := classify.NewClassifier(
|
||||
classify.NewClient(cfg.JEVBaseURL, cfg.JEVAPIKey, cfg.JEVModel, logger),
|
||||
st, logger, cfg.ClassifyConcurrency, cfg.ClassifyMaxMessages,
|
||||
)
|
||||
var sum classify.Summary
|
||||
if all {
|
||||
sum, err = classifier.RunAll(ctx)
|
||||
} else {
|
||||
sum, err = classifier.Run(ctx)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("considered=%d classified=%d failed=%d remaining=%d model=%s rules_version=%d\n",
|
||||
sum.Considered, sum.Classified, sum.Failed, sum.Remaining, sum.Model, sum.RulesVersion)
|
||||
return nil
|
||||
}
|
||||
|
||||
// probeHealth requests the local health endpoint. It exists so the container
|
||||
// healthcheck does not need an external HTTP client in the image.
|
||||
func probeHealth(port int) error {
|
||||
|
||||
@@ -40,6 +40,9 @@ services:
|
||||
EMAIL_AGENT_IMAP_MAX_MESSAGES: ${EMAIL_AGENT_IMAP_MAX_MESSAGES:-100}
|
||||
JEV_BASE_URL: ${JEV_BASE_URL:-}
|
||||
JEV_API_KEY: ${JEV_API_KEY:-}
|
||||
JEV_MODEL: ${JEV_MODEL:-jev-latest}
|
||||
EMAIL_AGENT_CLASSIFY_CONCURRENCY: ${EMAIL_AGENT_CLASSIFY_CONCURRENCY:-4}
|
||||
EMAIL_AGENT_CLASSIFY_MAX_MESSAGES: ${EMAIL_AGENT_CLASSIFY_MAX_MESSAGES:-200}
|
||||
OMNIROUTE_BASE_URL: ${OMNIROUTE_BASE_URL:-}
|
||||
OMNIROUTE_API_KEY: ${OMNIROUTE_API_KEY:-}
|
||||
HUB_TOKEN: ${HUB_TOKEN:-}
|
||||
|
||||
225
docs/phase3-classify.md
Normal file
225
docs/phase3-classify.md
Normal file
@@ -0,0 +1,225 @@
|
||||
# Phase 3 — Classify (JEV, rules, editable categories)
|
||||
|
||||
Status: **done and deployed on `.13`.** [AUD-69](/AUD/issues/AUD-69), parent
|
||||
[AUD-68](/AUD/issues/AUD-68). Phase 3 of AGENTS.md §11.
|
||||
|
||||
**DoD:** every message has a category, an importance, and a suggested action.
|
||||
|
||||
## 1. Confirmed JEV HTTP shape (before wiring)
|
||||
|
||||
Read live from the TypeSafe docs (`https://docs.typesafe.ai/llms.txt` →
|
||||
`api.md`, `primitives/choice.md`, `confidence.md`, `models.md`) and confirmed with
|
||||
a real call. This is the shape `internal/classify/jev.go` implements.
|
||||
|
||||
**Endpoint** — `POST https://api.typesafe.ai/v1/systemone`, bearer auth:
|
||||
|
||||
```http
|
||||
POST /v1/systemone HTTP/1.1
|
||||
Authorization: Bearer <JEV_API_KEY>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
**Request**
|
||||
|
||||
```json
|
||||
{
|
||||
"state": { "email": { "from": "…", "to": "…", "source_address": "…",
|
||||
"source_domain": "…", "subject": "…", "received": "…", "body": "…" } },
|
||||
"model": "jev-latest",
|
||||
"questions": {
|
||||
"category": { "type": "choice", "instructions": "…", "criteria": { "‹option›": "‹rubric›", … } },
|
||||
"importance": { "type": "choice", "instructions": "…", "criteria": { "low": "…", "medium": "…", "high": "…" } },
|
||||
"action": { "type": "choice", "instructions": "…", "criteria": { "delete": "…", "unsubscribe": "…", "reply": "…", "file": "…", "keep": "…" } }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response** — one answer per question id; Choice answers carry the selected
|
||||
option, the full probability distribution, and a confidence in `[0,1]`:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "jev-1.13.0",
|
||||
"answers": {
|
||||
"category": { "type": "choice", "choice": "Action", "confidence": 0.94, "probabilities": { … } },
|
||||
"importance": { "type": "choice", "choice": "high", "confidence": 0.89, "probabilities": { … } },
|
||||
"action": { "type": "choice", "choice": "file", "confidence": 0.49, "probabilities": { … } }
|
||||
},
|
||||
"usage": { "input_tokens": 1047, "output_tokens": 146 }
|
||||
}
|
||||
```
|
||||
|
||||
Errors: `401` invalid key, `422` validation, `429`/`529` rate-limit/overloaded.
|
||||
The client retries `429`, `529`, and `5xx` up to four times with exponential
|
||||
backoff and jitter, and surfaces any other status with the response body.
|
||||
|
||||
### One real, read-only classification (live, message id 10)
|
||||
|
||||
Message 10 (`InMotion Hosting Billing · "InMotion Hosting - Payment Due"`,
|
||||
arrived at `samuelrolfe@gmail.com`) was sent to JEV with the six live categories.
|
||||
Live response, verbatim:
|
||||
|
||||
```json
|
||||
{"model":"jev-1.13.0","answers":{
|
||||
"category":{"type":"choice","choice":"Action","confidence":0.94,
|
||||
"probabilities":{"Action":0.95,"Important":0.0,"Info":0.0,"Spam":0.03,"Newsletter":0.0,"Auto-reply":0.02}},
|
||||
"importance":{"type":"choice","choice":"high","confidence":0.89,
|
||||
"probabilities":{"low":0.03,"medium":0.04,"high":0.93}},
|
||||
"action":{"type":"choice","choice":"file","confidence":0.49,
|
||||
"probabilities":{"delete":0.24,"unsubscribe":0.01,"reply":0.01,"file":0.59,"keep":0.15}}},
|
||||
"usage":{"input_tokens":1047,"output_tokens":146}}
|
||||
```
|
||||
|
||||
The stored row matches: `Action / high / file / 0.94 / jev-1.13.0`.
|
||||
|
||||
**`JEV_BASE_URL` was not set on `.13`** (only `JEV_API_KEY` is in
|
||||
`~/.config/environment.d/10-secrets.conf`). The TypeSafe docs fix the host, so
|
||||
the code defaults an unset/empty `JEV_BASE_URL` to the documented
|
||||
`https://api.typesafe.ai`, and `JEV_MODEL` defaults to `jev-latest`. This is
|
||||
recorded here rather than treated as a contradiction. A different base URL can
|
||||
still be supplied without a code change.
|
||||
|
||||
## 2. What was built
|
||||
|
||||
- **`internal/classify`** — JEV client (`jev.go`), rule engine (`rules.go`),
|
||||
and orchestration (`classify.go`). One JEV request per message, bounded
|
||||
concurrency (default 4), a per-run cap (default 200), and a worker pool.
|
||||
- **`internal/store`** — migration **v3**: a `categories` table seeded once with
|
||||
the six AGENTS.md §13.2 categories (name + JEV rubric + order) and a
|
||||
comment-only starter `rules` version. Category CRUD, classification
|
||||
persistence, `CurrentRules`/`SaveRules`, and pending/coverage queries.
|
||||
- **`internal/server`** — `POST /actions/classify` (pending) and
|
||||
`POST /actions/reclassify` (all), plus the **FR-14 categories UI**
|
||||
(`GET /categories`, `POST /categories/add|rename|remove`).
|
||||
- **FR-5 triage list** — messages grouped by the live category list, newest
|
||||
first, with the recommended action and the classifier confidence per row.
|
||||
- **CLI** — `email-agent -classify` (pending) and `-classify-all` (re-classify
|
||||
everything), the n8n/CLI entry points for FR-3.
|
||||
- **FR-13** — every classification is logged in `actions_log` with the message,
|
||||
the category/importance/action/confidence/model, the rules version, and the
|
||||
matching rule line when a rule applied. Category add/rename/remove are logged.
|
||||
|
||||
### Rules (FR-4)
|
||||
|
||||
Plain text, one rule per line; the classifier reads the newest `rules` row. The
|
||||
editor that writes it lands in Phase 6, but the engine is live now.
|
||||
|
||||
```
|
||||
<condition> => <effects>
|
||||
|
||||
from contains "amazon.com" => category=Newsletter, action=unsubscribe
|
||||
subject contains "invoice" and from ends_with "@stripe.com" => category=Action, importance=high
|
||||
* => category=Info
|
||||
```
|
||||
|
||||
Conditions join with ` and ` over `from`, `to`, `subject`, `source`, `body`
|
||||
using `contains`, `equals`, `starts_with`, `ends_with`. Effects are
|
||||
`category=<live name>`, `importance=<low|medium|high>`, `action=<delete|unsubscribe|reply|file|keep>`.
|
||||
The **first matching rule wins** and overrides only the fields it names. A rule
|
||||
naming a category that no longer exists is ignored and logged; malformed lines
|
||||
are skipped and reported. Evidence: `internal/classify/rules_test.go` and
|
||||
`internal/classify/classify_test.go` (override, unknown-category fallback,
|
||||
rule-in-log).
|
||||
|
||||
### Categories (FR-14)
|
||||
|
||||
`categories.name` is the key stored in `classifications.category` **and** sent to
|
||||
JEV as the Choice option name, so JEV always sees the current list. Renaming
|
||||
cascades to existing classifications; removing a category reassigns its messages
|
||||
to the first remaining category; the last category cannot be removed.
|
||||
|
||||
## 3. DoD evidence
|
||||
|
||||
Deployed: image rebuilt and `email-agent` recreated on `.13`; the Phase 1/2
|
||||
volume `email_agent_data` was kept and migration v3 ran in place
|
||||
(`schema_version: 3`).
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `docker compose ps` | `email-agent` up, healthy, `0.0.0.0:8790->8790/tcp` |
|
||||
| `GET /health` (public HTTPS) | `200 {"status":"ok","env":"production","database":"ok"}` |
|
||||
| `email-agent -classify` (container) | `considered=100 classified=100 failed=0 remaining=0 model=jev-latest rules_version=1` |
|
||||
| **Every message has a classification** | `100` messages, **`0`** null/empty category, **`0`** importance, **`0`** action; `0` messages without a classification row |
|
||||
| Model | all `100` rows from `jev-1.13.0` |
|
||||
| Distribution | Newsletter 43, Auto-reply 31, Action 20, Info 4, Spam 1, Important 1 |
|
||||
| `POST /actions/reclassify` (HTTP, session + CSRF) | `200`, note `Classified 100 of 100 message(s); 0 failed; 0 still unclassified.` |
|
||||
| FR-13 per-message log | `100` `classify` actions with category/importance/action/confidence/model; `classify_run` summary rows |
|
||||
| Public HTTPS | `https://mail.lab.audasmedia.com.au/health` → `200 ok` |
|
||||
|
||||
### FR-14 HTTP/UI proof (live, over HTTP on `.13`)
|
||||
|
||||
Login (session + CSRF cookie), then:
|
||||
|
||||
1. **Add** — `POST /categories/add` `name=Finance`,
|
||||
`description=Bills, invoices, payments, renewals, and financial account alerts`
|
||||
→ `303`. `GET /categories` then lists `… Newsletter, Auto-reply, Finance`.
|
||||
2. **Rename** — `POST /categories/rename` `id=7 name=Money & Bills` → `303`.
|
||||
`GET /categories` lists `Money & Bills`.
|
||||
3. **JEV uses them** — `POST /actions/reclassify` → `200`
|
||||
`Classified 100 of 100 message(s)`. JEV then answered with the new name for
|
||||
**6 real messages** that were not in that category before:
|
||||
|
||||
| id | from | subject | result |
|
||||
|---|---|---|---|
|
||||
| 1 | Crazy Domains | ALERT: Your auto-renewal… | `Money & Bills / high / delete / 0.71` |
|
||||
| 10 | InMotion Hosting Billing | Payment Due | `Money & Bills / high / file / 0.99` |
|
||||
| 23 | api@minimaxi.com | Pay-as-you-go Balance Disconnect | `Money & Bills / high / file / 0.70` |
|
||||
| 30 | Crazy Domains | cmorganics.com.au has Expired | `Money & Bills / high / delete / 0.45` |
|
||||
| 36 | api@minimaxi.com | Re: Pay-as-you-go… | `Money & Bills / high / file / 0.59` |
|
||||
| 82 | Crazy Domains | cmorganics.com.au has Expired | `Money & Bills / high / delete / 0.46` |
|
||||
|
||||
4. **Remove** — `POST /categories/remove` `id=7` → `200` with note
|
||||
`Removed category; its messages moved to the first remaining category.`;
|
||||
`GET /categories` no longer lists it. `actions_log` records
|
||||
`category_add name=Finance`, `category_rename id=7 name=Money & Bills`,
|
||||
`category_remove id=7`. A second `reclassify` restored the six default
|
||||
categories; final state is the table above.
|
||||
|
||||
### Tests
|
||||
|
||||
`gofmt -l` clean, `go vet ./...` clean, `go test ./...` green (in
|
||||
`golang:1.25-alpine`). New tests cover the rule parser and precedence, the JEV
|
||||
request/response shape and 429 retry, the classifier (every pending message,
|
||||
rule override, unknown-category fallback, rerun-only-pending, re-classify-all,
|
||||
rule-in-action-log), category CRUD and cascade, and the category HTTP handlers.
|
||||
|
||||
## 4. Configuration
|
||||
|
||||
Added to `.env.example` and `docker-compose.yml` (values stay in the host
|
||||
environment / secret file):
|
||||
|
||||
| Variable | Default |
|
||||
|---|---|
|
||||
| `JEV_BASE_URL` | `https://api.typesafe.ai` |
|
||||
| `JEV_API_KEY` | _(required for classification)_ |
|
||||
| `JEV_MODEL` | `jev-latest` |
|
||||
| `EMAIL_AGENT_CLASSIFY_CONCURRENCY` | `4` |
|
||||
| `EMAIL_AGENT_CLASSIFY_MAX_MESSAGES` | `200` |
|
||||
|
||||
## 5. Deployment
|
||||
|
||||
On `.13`, from the repo, with the host secrets in scope:
|
||||
|
||||
```sh
|
||||
set -a; . ~/.config/environment.d/10-secrets.conf; set +a
|
||||
docker compose build
|
||||
docker compose up -d # migration v3 runs on start; data volume kept
|
||||
docker compose exec -T email-agent email-agent -classify
|
||||
```
|
||||
|
||||
## 6. Rollback
|
||||
|
||||
1. `docker compose down` on `.13` (keeps `email_agent_data`).
|
||||
2. `git revert <this phase commit>` and `docker compose up -d --build`.
|
||||
3. Migration v3 is additive (new table + seeded rows) and is not undone by the
|
||||
revert; the Phase 2 binary ignores `categories` and shows every message as
|
||||
`Unclassified`.
|
||||
|
||||
## 7. Not done / deferred
|
||||
|
||||
- The rules **editor UI** is Phase 6 (FR-11). The rules table, versioning, and
|
||||
the classifier reading them are live.
|
||||
- `POST /actions/reclassify` and `-classify-all` were added so a category or
|
||||
rules change can be applied to existing mail; Phase 6 will use the same path.
|
||||
- Unsubscribe detection (FR-7) and the reply drafts (FR-8) remain later phases.
|
||||
- The 1-month body retention job (AGENTS.md §7) is not implemented yet.
|
||||
321
internal/classify/classify.go
Normal file
321
internal/classify/classify.go
Normal file
@@ -0,0 +1,321 @@
|
||||
// Classifier orchestration: read the live category list and rules, ask JEV about
|
||||
// every pending message, apply the user's rules as overrides, and store the
|
||||
// result (FR-3, FR-4, FR-5).
|
||||
package classify
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/store"
|
||||
)
|
||||
|
||||
// ActorAgent is the actor recorded in the action log for classifier writes.
|
||||
const ActorAgent = "agent"
|
||||
|
||||
// Summary describes one classification run.
|
||||
type Summary struct {
|
||||
Considered int
|
||||
Classified int
|
||||
Failed int
|
||||
Model string
|
||||
RulesVersion int
|
||||
Remaining int
|
||||
Duration time.Duration
|
||||
}
|
||||
|
||||
// Classifier ties the JEV client to the store.
|
||||
type Classifier struct {
|
||||
client *Client
|
||||
store *store.Store
|
||||
logger *slog.Logger
|
||||
concurrency int
|
||||
maxPerRun int
|
||||
}
|
||||
|
||||
// NewClassifier builds a Classifier. concurrency and maxPerRun fall back to safe
|
||||
// defaults when they are not positive.
|
||||
func NewClassifier(client *Client, st *store.Store, logger *slog.Logger, concurrency, maxPerRun int) *Classifier {
|
||||
if concurrency < 1 {
|
||||
concurrency = 4
|
||||
}
|
||||
if concurrency > 16 {
|
||||
concurrency = 16
|
||||
}
|
||||
if maxPerRun < 1 {
|
||||
maxPerRun = 200
|
||||
}
|
||||
if logger == nil {
|
||||
logger = slog.New(slog.DiscardHandler)
|
||||
}
|
||||
return &Classifier{
|
||||
client: client,
|
||||
store: st,
|
||||
logger: logger,
|
||||
concurrency: concurrency,
|
||||
maxPerRun: maxPerRun,
|
||||
}
|
||||
}
|
||||
|
||||
// Model returns the configured JEV model alias.
|
||||
func (c *Classifier) Model() string { return c.client.Model() }
|
||||
|
||||
// Configured reports whether JEV_API_KEY is present.
|
||||
func (c *Classifier) Configured() bool { return c.client.Configured() }
|
||||
|
||||
// Run classifies the pending messages, up to maxPerRun. It returns a Summary
|
||||
// even when individual messages fail; only setup failures return an error.
|
||||
func (c *Classifier) Run(ctx context.Context) (Summary, error) {
|
||||
return c.run(ctx, false)
|
||||
}
|
||||
|
||||
// RunAll re-classifies every stored message. It is used after the category list
|
||||
// or the rules change, so the new settings take effect on existing mail.
|
||||
func (c *Classifier) RunAll(ctx context.Context) (Summary, error) {
|
||||
return c.run(ctx, true)
|
||||
}
|
||||
|
||||
func (c *Classifier) run(ctx context.Context, all bool) (Summary, error) {
|
||||
if !c.client.Configured() {
|
||||
return Summary{}, ErrNotConfigured
|
||||
}
|
||||
|
||||
start := time.Now()
|
||||
mode := "pending"
|
||||
if all {
|
||||
mode = "all"
|
||||
}
|
||||
categories, err := c.store.ListCategories(ctx)
|
||||
if err != nil {
|
||||
return Summary{}, err
|
||||
}
|
||||
if len(categories) == 0 {
|
||||
return Summary{}, fmt.Errorf("no categories configured")
|
||||
}
|
||||
options := make([]CategoryOption, 0, len(categories))
|
||||
for _, cat := range categories {
|
||||
options = append(options, CategoryOption{Name: cat.Name, Description: cat.Description})
|
||||
}
|
||||
valid := make(map[string]string, len(categories)) // lower(name) -> name
|
||||
for _, cat := range categories {
|
||||
valid[strings.ToLower(cat.Name)] = cat.Name
|
||||
}
|
||||
|
||||
rulesText, rulesVersion, err := c.store.CurrentRules(ctx)
|
||||
if err != nil {
|
||||
return Summary{}, err
|
||||
}
|
||||
rules := ParseRules(rulesText)
|
||||
for _, perr := range rules.Errors {
|
||||
c.logger.Warn("rules line skipped", "error", perr.Error())
|
||||
}
|
||||
|
||||
pending, err := c.pending(ctx, all)
|
||||
if err != nil {
|
||||
return Summary{}, err
|
||||
}
|
||||
|
||||
sum := Summary{
|
||||
Considered: len(pending),
|
||||
Model: c.client.Model(),
|
||||
RulesVersion: rulesVersion,
|
||||
}
|
||||
if len(pending) == 0 {
|
||||
sum.Remaining = 0
|
||||
sum.Duration = time.Since(start)
|
||||
return sum, nil
|
||||
}
|
||||
|
||||
type outcome struct {
|
||||
msg store.Message
|
||||
class store.Classification
|
||||
rule *Rule
|
||||
failed bool
|
||||
}
|
||||
|
||||
work := make(chan store.Message)
|
||||
results := make(chan outcome)
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < c.concurrency; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for msg := range work {
|
||||
class, rule, err := c.classifyOne(ctx, msg, options, valid, rules)
|
||||
if err != nil {
|
||||
c.logger.Warn("classify message failed",
|
||||
"message_id", msg.ID, "subject", msg.Subject, "error", err)
|
||||
results <- outcome{msg: msg, failed: true}
|
||||
continue
|
||||
}
|
||||
results <- outcome{msg: msg, class: class, rule: rule}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
go func() {
|
||||
defer close(work)
|
||||
for _, msg := range pending {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case work <- msg:
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
go func() {
|
||||
wg.Wait()
|
||||
close(results)
|
||||
}()
|
||||
|
||||
for res := range results {
|
||||
if res.failed {
|
||||
sum.Failed++
|
||||
continue
|
||||
}
|
||||
if _, err := c.store.SaveClassification(ctx, res.class); err != nil {
|
||||
c.logger.Error("save classification failed", "message_id", res.msg.ID, "error", err)
|
||||
sum.Failed++
|
||||
continue
|
||||
}
|
||||
detail := fmt.Sprintf("category=%s importance=%s action=%s confidence=%.2f model=%s rules_version=%d",
|
||||
res.class.Category, res.class.Importance, res.class.Action,
|
||||
res.class.Confidence, res.class.Model, rulesVersion)
|
||||
if res.rule != nil {
|
||||
detail += fmt.Sprintf(" rule_line=%d rule=%q", res.rule.Line, res.rule.Raw)
|
||||
}
|
||||
if err := c.store.LogMessageAction(ctx, res.msg.ID, "classify", ActorAgent, detail); err != nil {
|
||||
c.logger.Warn("log classify action failed", "message_id", res.msg.ID, "error", err)
|
||||
}
|
||||
sum.Classified++
|
||||
}
|
||||
|
||||
if cov, err := c.store.ClassificationCoverage(ctx); err == nil {
|
||||
sum.Remaining = cov.Missing()
|
||||
}
|
||||
sum.Duration = time.Since(start)
|
||||
|
||||
if err := c.store.LogAction(ctx, "classify_run", ActorAgent,
|
||||
fmt.Sprintf("mode=%s considered=%d classified=%d failed=%d remaining=%d model=%s rules_version=%d",
|
||||
mode, sum.Considered, sum.Classified, sum.Failed, sum.Remaining, sum.Model, rulesVersion)); err != nil {
|
||||
c.logger.Warn("log classify run failed", "error", err)
|
||||
}
|
||||
|
||||
c.logger.Info("classification run complete",
|
||||
"considered", sum.Considered,
|
||||
"classified", sum.Classified,
|
||||
"failed", sum.Failed,
|
||||
"remaining", sum.Remaining,
|
||||
"model", sum.Model,
|
||||
"duration_ms", sum.Duration.Milliseconds(),
|
||||
)
|
||||
return sum, nil
|
||||
}
|
||||
|
||||
// classifyOne asks JEV about one message and overlays the matching user rule.
|
||||
func (c *Classifier) classifyOne(ctx context.Context, msg store.Message, options []CategoryOption, valid map[string]string, rules RuleSet) (store.Classification, *Rule, error) {
|
||||
result, err := c.client.Classify(ctx, MessageInput{
|
||||
From: msg.FromAddr,
|
||||
To: msg.ToAddr,
|
||||
Subject: msg.Subject,
|
||||
Source: msg.SourceAddress,
|
||||
Body: msg.BodyText,
|
||||
Received: receivedString(msg),
|
||||
}, options)
|
||||
if err != nil {
|
||||
return store.Classification{}, nil, err
|
||||
}
|
||||
|
||||
rule := rules.Match(map[string]string{
|
||||
"from": msg.FromAddr,
|
||||
"to": msg.ToAddr,
|
||||
"subject": msg.Subject,
|
||||
"source": msg.SourceAddress,
|
||||
"body": msg.BodyText,
|
||||
})
|
||||
|
||||
category := result.Category
|
||||
importance := result.Importance
|
||||
action := result.Action
|
||||
if rule != nil {
|
||||
if v, ok := rule.Effects["category"]; ok {
|
||||
if name, found := valid[strings.ToLower(v)]; found {
|
||||
category = name
|
||||
} else {
|
||||
c.logger.Warn("rule names an unknown category; ignoring effect",
|
||||
"message_id", msg.ID, "rule_line", rule.Line, "category", v)
|
||||
}
|
||||
}
|
||||
if v, ok := rule.Effects["importance"]; ok {
|
||||
importance = v
|
||||
}
|
||||
if v, ok := rule.Effects["action"]; ok {
|
||||
action = v
|
||||
}
|
||||
}
|
||||
|
||||
// The classifier must always store a usable category, importance, and
|
||||
// action (Phase 3 DoD). JEV can only pick from the supplied options, but a
|
||||
// broken response or a missing category list must not leave NULLs.
|
||||
if name, found := valid[strings.ToLower(category)]; found {
|
||||
category = name
|
||||
} else {
|
||||
if category != "" {
|
||||
c.logger.Warn("classifier returned an unknown category; using fallback",
|
||||
"message_id", msg.ID, "category", category)
|
||||
}
|
||||
category = options[0].Name
|
||||
}
|
||||
if !allowedImportance[strings.ToLower(importance)] {
|
||||
importance = "medium"
|
||||
} else {
|
||||
importance = strings.ToLower(importance)
|
||||
}
|
||||
if !allowedAction[strings.ToLower(action)] {
|
||||
action = "keep"
|
||||
} else {
|
||||
action = strings.ToLower(action)
|
||||
}
|
||||
|
||||
// A rule override invalidates the JEV confidence: report the confidence of
|
||||
// the answer the category actually came from.
|
||||
confidence := result.CategoryConfidence
|
||||
if rule != nil {
|
||||
if _, ok := rule.Effects["category"]; ok {
|
||||
confidence = 1
|
||||
}
|
||||
}
|
||||
|
||||
return store.Classification{
|
||||
MessageID: msg.ID,
|
||||
Category: category,
|
||||
Importance: importance,
|
||||
Action: action,
|
||||
Confidence: confidence,
|
||||
Model: result.Model,
|
||||
}, rule, nil
|
||||
}
|
||||
|
||||
// pending returns the messages to classify for this run.
|
||||
func (c *Classifier) pending(ctx context.Context, all bool) ([]store.Message, error) {
|
||||
if all {
|
||||
return c.store.AllMessagesForClassification(ctx, c.maxPerRun)
|
||||
}
|
||||
return c.store.PendingClassifications(ctx, c.maxPerRun)
|
||||
}
|
||||
|
||||
func receivedString(msg store.Message) string {
|
||||
t := msg.DateSent
|
||||
if t.IsZero() {
|
||||
t = msg.ReceivedAt
|
||||
}
|
||||
if t.IsZero() {
|
||||
return ""
|
||||
}
|
||||
return t.UTC().Format(time.RFC3339)
|
||||
}
|
||||
288
internal/classify/classify_test.go
Normal file
288
internal/classify/classify_test.go
Normal file
@@ -0,0 +1,288 @@
|
||||
package classify
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/store"
|
||||
)
|
||||
|
||||
// stubJEV returns a TypeSafe-shaped response with a fixed category/action while
|
||||
// recording how many requests it handled.
|
||||
func stubJEV(t *testing.T, category, importance, action string, calls *int32) *httptest.Server {
|
||||
t.Helper()
|
||||
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
*calls++
|
||||
var body struct {
|
||||
Questions map[string]any `json:"questions"`
|
||||
}
|
||||
_ = json.NewDecoder(r.Body).Decode(&body)
|
||||
if len(body.Questions) != 3 {
|
||||
t.Errorf("JEV stub saw %d questions, want 3", len(body.Questions))
|
||||
}
|
||||
resp := map[string]any{
|
||||
"model": "jev-1.13.0",
|
||||
"answers": map[string]any{
|
||||
"category": map[string]any{"type": "choice", "choice": category, "confidence": 0.8},
|
||||
"importance": map[string]any{"type": "choice", "choice": importance, "confidence": 0.7},
|
||||
"action": map[string]any{"type": "choice", "choice": action, "confidence": 0.6},
|
||||
},
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_ = json.NewEncoder(w).Encode(resp)
|
||||
}))
|
||||
}
|
||||
|
||||
func newTestStore(t *testing.T) *store.Store {
|
||||
t.Helper()
|
||||
st, err := store.Open(context.Background(), filepath.Join(t.TempDir(), "email_agent.db"))
|
||||
if err != nil {
|
||||
t.Fatalf("store.Open() error = %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = st.Close() })
|
||||
return st
|
||||
}
|
||||
|
||||
func seedMessages(t *testing.T, st *store.Store, from string, n int) {
|
||||
t.Helper()
|
||||
for i := 0; i < n; i++ {
|
||||
if _, err := st.UpsertMessage(context.Background(), &store.Message{
|
||||
GmailID: fmt.Sprintf("%s-%d@test", from, i),
|
||||
FromAddr: from,
|
||||
ToAddr: "hello@where-woof.com",
|
||||
Subject: fmt.Sprintf("message %d", i),
|
||||
BodyText: "body",
|
||||
SourceAddress: "hello@where-woof.com",
|
||||
DateSent: time.Now().UTC(),
|
||||
}); err != nil {
|
||||
t.Fatalf("seed message: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func newTestClassifier(t *testing.T, st *store.Store, baseURL string) *Classifier {
|
||||
t.Helper()
|
||||
client := NewClient(baseURL, "test-key", "jev-latest", slog.New(slog.DiscardHandler))
|
||||
return NewClassifier(client, st, slog.New(slog.DiscardHandler), 2, 50)
|
||||
}
|
||||
|
||||
func TestClassifierClassifiesEveryPendingMessage(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
st := newTestStore(t)
|
||||
seedMessages(t, st, "someone@example.com", 3)
|
||||
|
||||
var calls int32
|
||||
srv := stubJEV(t, "Info", "low", "keep", &calls)
|
||||
defer srv.Close()
|
||||
|
||||
sum, err := newTestClassifier(t, st, srv.URL).Run(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
if sum.Considered != 3 || sum.Classified != 3 || sum.Failed != 0 {
|
||||
t.Fatalf("summary = %+v, want 3 considered/classified, 0 failed", sum)
|
||||
}
|
||||
if calls != 3 {
|
||||
t.Errorf("JEV calls = %d, want 3", calls)
|
||||
}
|
||||
|
||||
coverage, err := st.ClassificationCoverage(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ClassificationCoverage() error = %v", err)
|
||||
}
|
||||
if coverage.Total != 3 || coverage.Classified != 3 || coverage.Missing() != 0 {
|
||||
t.Fatalf("coverage = %+v, want 3/3 classified", coverage)
|
||||
}
|
||||
|
||||
items, err := st.ListMessages(ctx, 10)
|
||||
if err != nil {
|
||||
t.Fatalf("ListMessages() error = %v", err)
|
||||
}
|
||||
for _, it := range items {
|
||||
if it.Category == "" || it.Importance == "" || it.Action == "" {
|
||||
t.Errorf("message %d has a null classification: %+v", it.ID, it)
|
||||
}
|
||||
if !it.HasClassification {
|
||||
t.Errorf("message %d HasClassification = false", it.ID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifierRuleOverridesJEV(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
st := newTestStore(t)
|
||||
seedMessages(t, st, "billing@stripe.com", 1)
|
||||
if _, err := st.SaveRules(ctx, `from contains "stripe" => category=Action, importance=high, action=reply`); err != nil {
|
||||
t.Fatalf("SaveRules() error = %v", err)
|
||||
}
|
||||
|
||||
var calls int32
|
||||
srv := stubJEV(t, "Newsletter", "low", "keep", &calls)
|
||||
defer srv.Close()
|
||||
|
||||
sum, err := newTestClassifier(t, st, srv.URL).Run(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
if sum.Classified != 1 {
|
||||
t.Fatalf("summary = %+v", sum)
|
||||
}
|
||||
|
||||
items, err := st.ListMessages(ctx, 10)
|
||||
if err != nil {
|
||||
t.Fatalf("ListMessages() error = %v", err)
|
||||
}
|
||||
got := items[0]
|
||||
if got.Category != "Action" || got.Importance != "high" || got.Action != "reply" {
|
||||
t.Errorf("rule override not applied: %+v", got)
|
||||
}
|
||||
if got.Confidence != 1 {
|
||||
t.Errorf("Confidence = %v, want 1 for a rule override", got.Confidence)
|
||||
}
|
||||
if calls != 1 {
|
||||
t.Errorf("JEV calls = %d, want 1 (JEV still consulted)", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifierIgnoresRuleWithUnknownCategory(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
st := newTestStore(t)
|
||||
seedMessages(t, st, "billing@stripe.com", 1)
|
||||
if _, err := st.SaveRules(ctx, `from contains "stripe" => category=NoSuchCategory`); err != nil {
|
||||
t.Fatalf("SaveRules() error = %v", err)
|
||||
}
|
||||
|
||||
var calls int32
|
||||
srv := stubJEV(t, "Newsletter", "medium", "keep", &calls)
|
||||
defer srv.Close()
|
||||
|
||||
if _, err := newTestClassifier(t, st, srv.URL).Run(ctx); err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
items, err := st.ListMessages(ctx, 10)
|
||||
if err != nil {
|
||||
t.Fatalf("ListMessages() error = %v", err)
|
||||
}
|
||||
if items[0].Category != "Newsletter" {
|
||||
t.Errorf("Category = %q, want the JEV answer Newsletter", items[0].Category)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifierFallsBackWhenJEVCategoryUnknown(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
st := newTestStore(t)
|
||||
seedMessages(t, st, "someone@example.com", 1)
|
||||
|
||||
var calls int32
|
||||
srv := stubJEV(t, "CategoryThatDoesNotExist", "medium", "keep", &calls)
|
||||
defer srv.Close()
|
||||
|
||||
if _, err := newTestClassifier(t, st, srv.URL).Run(ctx); err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
categories, err := st.CategoryNames(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("CategoryNames() error = %v", err)
|
||||
}
|
||||
items, err := st.ListMessages(ctx, 10)
|
||||
if err != nil {
|
||||
t.Fatalf("ListMessages() error = %v", err)
|
||||
}
|
||||
if items[0].Category != categories[0] {
|
||||
t.Errorf("Category = %q, want the first configured category %q", items[0].Category, categories[0])
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifierRerunsOnlyPending(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
st := newTestStore(t)
|
||||
seedMessages(t, st, "someone@example.com", 2)
|
||||
|
||||
var calls int32
|
||||
srv := stubJEV(t, "Info", "low", "keep", &calls)
|
||||
defer srv.Close()
|
||||
|
||||
classifier := newTestClassifier(t, st, srv.URL)
|
||||
if _, err := classifier.Run(ctx); err != nil {
|
||||
t.Fatalf("first Run() error = %v", err)
|
||||
}
|
||||
sum, err := classifier.Run(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("second Run() error = %v", err)
|
||||
}
|
||||
if sum.Considered != 0 {
|
||||
t.Errorf("second run considered %d, want 0 (nothing pending)", sum.Considered)
|
||||
}
|
||||
if calls != 2 {
|
||||
t.Errorf("JEV calls = %d, want 2 (only the first run)", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifierRunAllReclassifiesEverything(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
st := newTestStore(t)
|
||||
seedMessages(t, st, "someone@example.com", 2)
|
||||
|
||||
var calls int32
|
||||
srv := stubJEV(t, "Info", "low", "keep", &calls)
|
||||
defer srv.Close()
|
||||
|
||||
classifier := newTestClassifier(t, st, srv.URL)
|
||||
if _, err := classifier.Run(ctx); err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
sum, err := classifier.RunAll(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("RunAll() error = %v", err)
|
||||
}
|
||||
if sum.Considered != 2 || sum.Classified != 2 {
|
||||
t.Fatalf("RunAll() summary = %+v, want 2 considered/classified", sum)
|
||||
}
|
||||
if calls != 4 {
|
||||
t.Errorf("JEV calls = %d, want 4 (2 initial + 2 re-classified)", calls)
|
||||
}
|
||||
|
||||
var n int
|
||||
if err := st.DB().QueryRowContext(ctx, `SELECT COUNT(*) FROM classifications`).Scan(&n); err != nil {
|
||||
t.Fatalf("count classifications: %v", err)
|
||||
}
|
||||
if n != 4 {
|
||||
t.Errorf("classification rows = %d, want 4 (history is kept)", n)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifierActionLogRecordsRule(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
st := newTestStore(t)
|
||||
seedMessages(t, st, "billing@stripe.com", 1)
|
||||
if _, err := st.SaveRules(ctx, `from contains "stripe" => category=Action`); err != nil {
|
||||
t.Fatalf("SaveRules() error = %v", err)
|
||||
}
|
||||
|
||||
var calls int32
|
||||
srv := stubJEV(t, "Newsletter", "low", "keep", &calls)
|
||||
defer srv.Close()
|
||||
|
||||
if _, err := newTestClassifier(t, st, srv.URL).Run(ctx); err != nil {
|
||||
t.Fatalf("Run() error = %v", err)
|
||||
}
|
||||
|
||||
var detail string
|
||||
if err := st.DB().QueryRowContext(ctx,
|
||||
`SELECT detail FROM actions_log WHERE action = 'classify' ORDER BY id DESC LIMIT 1`).Scan(&detail); err != nil {
|
||||
t.Fatalf("read classify log: %v", err)
|
||||
}
|
||||
for _, want := range []string{"category=Action", "rule_line=1", "from contains"} {
|
||||
if !strings.Contains(detail, want) {
|
||||
t.Errorf("classify log detail %q missing %q", detail, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
306
internal/classify/jev.go
Normal file
306
internal/classify/jev.go
Normal file
@@ -0,0 +1,306 @@
|
||||
// Package classify turns stored messages into a category, an importance, a
|
||||
// suggested action, and a confidence using the TypeSafe (JEV) HTTP API, then
|
||||
// applies the user's rules (FR-3, FR-4).
|
||||
//
|
||||
// The JEV request/response shape is confirmed against the live docs
|
||||
// (https://docs.typesafe.ai/api.md) and recorded in docs/phase3-classify.md:
|
||||
//
|
||||
// POST https://api.typesafe.ai/v1/systemone
|
||||
// Authorization: Bearer <API_KEY>
|
||||
// {"state": …, "model": "jev-latest", "questions": {"‹id›": {"type": "choice", …}}}
|
||||
// → {"model": "jev-1.13.0", "answers": {"‹id›": {"type": "choice", "confidence", "probabilities"}}}
|
||||
package classify
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"math/rand"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DefaultBaseURL is TypeSafe's documented API host. JEV_BASE_URL overrides it.
|
||||
const DefaultBaseURL = "https://api.typesafe.ai"
|
||||
|
||||
// DefaultModel is the flagship System One alias documented at
|
||||
// https://docs.typesafe.ai/models.
|
||||
const DefaultModel = "jev-latest"
|
||||
|
||||
// systemOnePath is the documented evaluation endpoint.
|
||||
const systemOnePath = "/v1/systemone"
|
||||
|
||||
// ErrNotConfigured means JEV_API_KEY is not set.
|
||||
var ErrNotConfigured = errors.New("JEV_API_KEY is not configured")
|
||||
|
||||
// maxBodyChars bounds how much of a message body is sent to JEV.
|
||||
const maxBodyChars = 4000
|
||||
|
||||
// MessageInput is the message text handed to JEV.
|
||||
type MessageInput struct {
|
||||
From string
|
||||
To string
|
||||
Subject string
|
||||
Source string
|
||||
Body string
|
||||
Received string
|
||||
}
|
||||
|
||||
// CategoryOption is one Choice option: the live category name and its rubric.
|
||||
type CategoryOption struct {
|
||||
Name string
|
||||
Description string
|
||||
}
|
||||
|
||||
// Result is one classification decision. Confidence is the Choice answer's own
|
||||
// confidence, so the UI can show how sure JEV is.
|
||||
type Result struct {
|
||||
Category string
|
||||
CategoryConfidence float64
|
||||
Importance string
|
||||
ImportanceConfidence float64
|
||||
Action string
|
||||
ActionConfidence float64
|
||||
Model string
|
||||
}
|
||||
|
||||
// Client calls the TypeSafe System One endpoint.
|
||||
type Client struct {
|
||||
baseURL string
|
||||
apiKey string
|
||||
model string
|
||||
http *http.Client
|
||||
logger *slog.Logger
|
||||
}
|
||||
|
||||
// NewClient builds a JEV client. An empty baseURL falls back to DefaultBaseURL,
|
||||
// an empty model to DefaultModel. logger may be nil.
|
||||
func NewClient(baseURL, apiKey, model string, logger *slog.Logger) *Client {
|
||||
baseURL = strings.TrimRight(strings.TrimSpace(baseURL), "/")
|
||||
if baseURL == "" {
|
||||
baseURL = DefaultBaseURL
|
||||
}
|
||||
if strings.TrimSpace(model) == "" {
|
||||
model = DefaultModel
|
||||
}
|
||||
if logger == nil {
|
||||
logger = slog.New(slog.DiscardHandler)
|
||||
}
|
||||
return &Client{
|
||||
baseURL: baseURL,
|
||||
apiKey: strings.TrimSpace(apiKey),
|
||||
model: model,
|
||||
http: &http.Client{Timeout: 60 * time.Second},
|
||||
logger: logger,
|
||||
}
|
||||
}
|
||||
|
||||
// Configured reports whether an API key is present.
|
||||
func (c *Client) Configured() bool { return c.apiKey != "" }
|
||||
|
||||
// Model returns the configured model alias.
|
||||
func (c *Client) Model() string { return c.model }
|
||||
|
||||
// Classify sends one message and returns the category, importance, action, and
|
||||
// confidences from one JEV request.
|
||||
func (c *Client) Classify(ctx context.Context, msg MessageInput, categories []CategoryOption) (Result, error) {
|
||||
if !c.Configured() {
|
||||
return Result{}, ErrNotConfigured
|
||||
}
|
||||
if len(categories) == 0 {
|
||||
return Result{}, errors.New("no categories configured")
|
||||
}
|
||||
|
||||
categoryCriteria := make(map[string]any, len(categories))
|
||||
for _, opt := range categories {
|
||||
if strings.TrimSpace(opt.Name) == "" {
|
||||
continue
|
||||
}
|
||||
if strings.TrimSpace(opt.Description) == "" {
|
||||
categoryCriteria[opt.Name] = nil
|
||||
} else {
|
||||
categoryCriteria[opt.Name] = opt.Description
|
||||
}
|
||||
}
|
||||
|
||||
body := map[string]any{
|
||||
"state": map[string]any{
|
||||
"email": map[string]any{
|
||||
"from": msg.From,
|
||||
"to": msg.To,
|
||||
"source_address": msg.Source,
|
||||
"source_domain": domainOf(msg.Source),
|
||||
"subject": msg.Subject,
|
||||
"received": msg.Received,
|
||||
"body": truncate(msg.Body, maxBodyChars),
|
||||
},
|
||||
},
|
||||
"model": c.model,
|
||||
"questions": map[string]any{
|
||||
"category": map[string]any{
|
||||
"type": "choice",
|
||||
"instructions": "Which category best fits this email for a personal family inbox? Pick the single best match.",
|
||||
"criteria": categoryCriteria,
|
||||
},
|
||||
"importance": map[string]any{
|
||||
"type": "choice",
|
||||
"instructions": "How important is this email to the recipient?",
|
||||
"criteria": map[string]any{
|
||||
"low": "Can be ignored or removed without loss",
|
||||
"medium": "Worth a quick look",
|
||||
"high": "Needs the recipient's attention soon",
|
||||
},
|
||||
},
|
||||
"action": map[string]any{
|
||||
"type": "choice",
|
||||
"instructions": "What is the best single recommended action for this email?",
|
||||
"criteria": map[string]any{
|
||||
"delete": "Junk or spam; safe to remove",
|
||||
"unsubscribe": "Stop future mail from this sender",
|
||||
"reply": "Send a reply to the sender",
|
||||
"file": "Keep for reference; move out of the inbox",
|
||||
"keep": "Leave in the inbox",
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
resp, err := c.post(ctx, body)
|
||||
if err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
|
||||
res := Result{Model: resp.Model}
|
||||
if a, ok := resp.Answers["category"]; ok {
|
||||
res.Category = a.Choice
|
||||
res.CategoryConfidence = a.Confidence
|
||||
}
|
||||
if a, ok := resp.Answers["importance"]; ok {
|
||||
res.Importance = a.Choice
|
||||
res.ImportanceConfidence = a.Confidence
|
||||
}
|
||||
if a, ok := resp.Answers["action"]; ok {
|
||||
res.Action = a.Choice
|
||||
res.ActionConfidence = a.Confidence
|
||||
}
|
||||
if res.Model == "" {
|
||||
res.Model = c.model
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// answer is one System One Choice answer.
|
||||
type answer struct {
|
||||
Type string `json:"type"`
|
||||
Choice string `json:"choice"`
|
||||
Confidence float64 `json:"confidence"`
|
||||
Probabilities map[string]float64 `json:"probabilities"`
|
||||
Noul float64 `json:"noul"`
|
||||
}
|
||||
|
||||
type response struct {
|
||||
Model string `json:"model"`
|
||||
Answers map[string]answer `json:"answers"`
|
||||
Usage struct {
|
||||
InputTokens int `json:"input_tokens"`
|
||||
OutputTokens int `json:"output_tokens"`
|
||||
} `json:"usage"`
|
||||
}
|
||||
|
||||
// post sends the request with bounded retries for rate limits and outages.
|
||||
func (c *Client) post(ctx context.Context, body map[string]any) (response, error) {
|
||||
payload, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return response{}, fmt.Errorf("encode JEV request: %w", err)
|
||||
}
|
||||
|
||||
const maxAttempts = 4
|
||||
var lastErr error
|
||||
for attempt := 1; attempt <= maxAttempts; attempt++ {
|
||||
resp, retryable, err := c.do(ctx, payload)
|
||||
if err == nil {
|
||||
return resp, nil
|
||||
}
|
||||
lastErr = err
|
||||
if !retryable || attempt == maxAttempts {
|
||||
break
|
||||
}
|
||||
delay := backoff(attempt)
|
||||
c.logger.Warn("jev request failed; retrying", "attempt", attempt, "delay_ms", delay.Milliseconds(), "error", err)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return response{}, ctx.Err()
|
||||
case <-time.After(delay):
|
||||
}
|
||||
}
|
||||
return response{}, lastErr
|
||||
}
|
||||
|
||||
func (c *Client) do(ctx context.Context, payload []byte) (response, bool, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+systemOnePath, bytes.NewReader(payload))
|
||||
if err != nil {
|
||||
return response{}, false, fmt.Errorf("build JEV request: %w", err)
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+c.apiKey)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json")
|
||||
|
||||
httpResp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return response{}, true, fmt.Errorf("call JEV: %w", err)
|
||||
}
|
||||
defer func() { _ = httpResp.Body.Close() }()
|
||||
|
||||
raw, err := io.ReadAll(io.LimitReader(httpResp.Body, 1<<20))
|
||||
if err != nil {
|
||||
return response{}, true, fmt.Errorf("read JEV response: %w", err)
|
||||
}
|
||||
|
||||
if httpResp.StatusCode != http.StatusOK {
|
||||
retryable := httpResp.StatusCode == http.StatusTooManyRequests ||
|
||||
httpResp.StatusCode == http.StatusServiceUnavailable ||
|
||||
httpResp.StatusCode == 529 ||
|
||||
httpResp.StatusCode >= 500
|
||||
return response{}, retryable,
|
||||
fmt.Errorf("JEV returned %d: %s", httpResp.StatusCode, strings.TrimSpace(string(raw)))
|
||||
}
|
||||
|
||||
var out response
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return response{}, false, fmt.Errorf("decode JEV response: %w", err)
|
||||
}
|
||||
if len(out.Answers) == 0 {
|
||||
return response{}, false, fmt.Errorf("JEV response contained no answers")
|
||||
}
|
||||
return out, false, nil
|
||||
}
|
||||
|
||||
// backoff returns the delay before the next attempt, with a little jitter so
|
||||
// parallel workers do not retry in lock step.
|
||||
func backoff(attempt int) time.Duration {
|
||||
base := time.Duration(1<<(attempt-1)) * time.Second
|
||||
if base > 8*time.Second {
|
||||
base = 8 * time.Second
|
||||
}
|
||||
return base + time.Duration(rand.Int63n(int64(base/2+1)))
|
||||
}
|
||||
|
||||
func truncate(s string, max int) string {
|
||||
s = strings.TrimSpace(s)
|
||||
if len(s) <= max {
|
||||
return s
|
||||
}
|
||||
return s[:max] + "…"
|
||||
}
|
||||
|
||||
func domainOf(addr string) string {
|
||||
if i := strings.LastIndex(addr, "@"); i >= 0 && i < len(addr)-1 {
|
||||
return addr[i+1:]
|
||||
}
|
||||
return addr
|
||||
}
|
||||
154
internal/classify/jev_test.go
Normal file
154
internal/classify/jev_test.go
Normal file
@@ -0,0 +1,154 @@
|
||||
package classify
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestClientClassifyRequestAndResponse(t *testing.T) {
|
||||
var (
|
||||
gotPath string
|
||||
gotAuth string
|
||||
gotBody map[string]any
|
||||
callCount int32
|
||||
)
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
atomic.AddInt32(&callCount, 1)
|
||||
gotPath = r.URL.Path
|
||||
gotAuth = r.Header.Get("Authorization")
|
||||
if err := json.NewDecoder(r.Body).Decode(&gotBody); err != nil {
|
||||
t.Errorf("decode request: %v", err)
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{
|
||||
"model": "jev-1.13.0",
|
||||
"answers": {
|
||||
"category": {"type":"choice","choice":"Action","confidence":0.81,"probabilities":{"Action":0.81,"Info":0.19}},
|
||||
"importance": {"type":"choice","choice":"high","confidence":0.9,"probabilities":{"high":0.9}},
|
||||
"action": {"type":"choice","choice":"reply","confidence":0.7,"probabilities":{"reply":0.7}}
|
||||
},
|
||||
"usage": {"input_tokens": 100, "output_tokens": 20}
|
||||
}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := NewClient(srv.URL, "test-key", "jev-latest", nil)
|
||||
res, err := client.Classify(context.Background(), MessageInput{
|
||||
From: "favrolfe@gmail.com", To: "hello@where-woof.com",
|
||||
Subject: "Quick question", Source: "hello@where-woof.com",
|
||||
Body: "Can you send me the invoice?", Received: "2026-10-11T09:00:00Z",
|
||||
}, []CategoryOption{
|
||||
{Name: "Action", Description: "Needs Sam to do something"},
|
||||
{Name: "Info", Description: "Informational"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Classify() error = %v", err)
|
||||
}
|
||||
|
||||
if gotPath != "/v1/systemone" {
|
||||
t.Errorf("path = %q, want /v1/systemone", gotPath)
|
||||
}
|
||||
if gotAuth != "Bearer test-key" {
|
||||
t.Errorf("Authorization = %q", gotAuth)
|
||||
}
|
||||
if gotBody["model"] != "jev-latest" {
|
||||
t.Errorf("model = %v, want jev-latest", gotBody["model"])
|
||||
}
|
||||
questions, _ := gotBody["questions"].(map[string]any)
|
||||
if len(questions) != 3 {
|
||||
t.Fatalf("questions = %d, want 3", len(questions))
|
||||
}
|
||||
categoryQ, _ := questions["category"].(map[string]any)
|
||||
criteria, _ := categoryQ["criteria"].(map[string]any)
|
||||
if len(criteria) != 2 {
|
||||
t.Errorf("category criteria = %v, want the live category list", criteria)
|
||||
}
|
||||
state, _ := gotBody["state"].(map[string]any)
|
||||
if _, ok := state["email"]; !ok {
|
||||
t.Errorf("state = %v, want an email object", state)
|
||||
}
|
||||
|
||||
if res.Category != "Action" || res.Importance != "high" || res.Action != "reply" {
|
||||
t.Errorf("result = %+v", res)
|
||||
}
|
||||
if res.CategoryConfidence != 0.81 {
|
||||
t.Errorf("CategoryConfidence = %v, want 0.81", res.CategoryConfidence)
|
||||
}
|
||||
if res.Model != "jev-1.13.0" {
|
||||
t.Errorf("Model = %q, want jev-1.13.0", res.Model)
|
||||
}
|
||||
if atomic.LoadInt32(&callCount) != 1 {
|
||||
t.Errorf("call count = %d, want 1", callCount)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientRetriesRateLimit(t *testing.T) {
|
||||
var attempts int32
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if atomic.AddInt32(&attempts, 1) == 1 {
|
||||
w.WriteHeader(http.StatusTooManyRequests)
|
||||
_, _ = w.Write([]byte(`{"error":"rate limited"}`))
|
||||
return
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"model":"jev-1.13.0","answers":{
|
||||
"category":{"type":"choice","choice":"Info","confidence":1},
|
||||
"importance":{"type":"choice","choice":"low","confidence":1},
|
||||
"action":{"type":"choice","choice":"keep","confidence":1}}}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := NewClient(srv.URL, "k", "", nil)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
|
||||
res, err := client.Classify(ctx, MessageInput{Subject: "hi"}, []CategoryOption{{Name: "Info"}})
|
||||
if err != nil {
|
||||
t.Fatalf("Classify() error = %v", err)
|
||||
}
|
||||
if res.Category != "Info" {
|
||||
t.Errorf("Category = %q, want Info", res.Category)
|
||||
}
|
||||
if got := atomic.LoadInt32(&attempts); got != 2 {
|
||||
t.Errorf("attempts = %d, want 2", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientRequiresAPIKey(t *testing.T) {
|
||||
client := NewClient("http://example.invalid", "", "", nil)
|
||||
if client.Configured() {
|
||||
t.Fatal("Configured() = true without an API key")
|
||||
}
|
||||
if _, err := client.Classify(context.Background(), MessageInput{}, []CategoryOption{{Name: "Info"}}); err != ErrNotConfigured {
|
||||
t.Fatalf("err = %v, want ErrNotConfigured", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientDefaultsBaseURL(t *testing.T) {
|
||||
client := NewClient("", "k", "", nil)
|
||||
if client.baseURL != DefaultBaseURL {
|
||||
t.Errorf("baseURL = %q, want %q", client.baseURL, DefaultBaseURL)
|
||||
}
|
||||
if client.Model() != DefaultModel {
|
||||
t.Errorf("model = %q, want %q", client.Model(), DefaultModel)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientHTTPErrorSurfacesBody(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusUnauthorized)
|
||||
_, _ = w.Write([]byte(`{"error":"invalid api key"}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := NewClient(srv.URL, "bad", "", nil)
|
||||
_, err := client.Classify(context.Background(), MessageInput{}, []CategoryOption{{Name: "Info"}})
|
||||
if err == nil || !strings.Contains(err.Error(), "invalid api key") {
|
||||
t.Fatalf("err = %v, want the 401 body", err)
|
||||
}
|
||||
}
|
||||
238
internal/classify/rules.go
Normal file
238
internal/classify/rules.go
Normal file
@@ -0,0 +1,238 @@
|
||||
// User rules (FR-4). A rule overrides the JEV classification when its condition
|
||||
// matches the message.
|
||||
//
|
||||
// Syntax, one rule per line:
|
||||
//
|
||||
// <condition> => <effects>
|
||||
//
|
||||
// condition: `<field> <operator> "<value>"`, joined with " and ".
|
||||
//
|
||||
// fields: from, to, subject, source, body
|
||||
// operators: contains, equals, starts_with, ends_with
|
||||
//
|
||||
// A single `*` matches every message (use it as the last line for a default).
|
||||
// effects: comma-separated `category=<name>`, `importance=<low|medium|high>`,
|
||||
// `action=<delete|unsubscribe|reply|file|keep>`.
|
||||
//
|
||||
// The first matching rule wins and overrides only the fields it names. The rules
|
||||
// editor that writes this text lands in Phase 6; the classifier already reads it.
|
||||
package classify
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Rule is one parsed line.
|
||||
type Rule struct {
|
||||
Line int
|
||||
Raw string
|
||||
MatchAll bool
|
||||
Conditions []Condition
|
||||
Effects map[string]string
|
||||
}
|
||||
|
||||
// Condition is one `<field> <op> "<value>"` term.
|
||||
type Condition struct {
|
||||
Field string
|
||||
Op string
|
||||
Value string
|
||||
}
|
||||
|
||||
// ParseError records a line that could not be parsed. The rest of the document
|
||||
// still loads; the error is surfaced in logs so a typo is visible.
|
||||
type ParseError struct {
|
||||
Line int
|
||||
Text string
|
||||
Reason string
|
||||
}
|
||||
|
||||
func (e ParseError) Error() string {
|
||||
return fmt.Sprintf("rules line %d: %s (%q)", e.Line, e.Reason, strings.TrimSpace(e.Text))
|
||||
}
|
||||
|
||||
// RuleSet is a parsed rules document.
|
||||
type RuleSet struct {
|
||||
Rules []Rule
|
||||
Errors []ParseError
|
||||
}
|
||||
|
||||
// effectFields are the classification fields a rule may override.
|
||||
var effectFields = map[string]bool{
|
||||
"category": true,
|
||||
"importance": true,
|
||||
"action": true,
|
||||
}
|
||||
|
||||
var allowedImportance = map[string]bool{"low": true, "medium": true, "high": true}
|
||||
|
||||
var allowedAction = map[string]bool{
|
||||
"delete": true, "unsubscribe": true, "reply": true, "file": true, "keep": true,
|
||||
}
|
||||
|
||||
// ParseRules parses a rules document. Malformed lines are collected in Errors and
|
||||
// skipped; parsing never fails outright, so one typo cannot disable the rules.
|
||||
func ParseRules(text string) RuleSet {
|
||||
var set RuleSet
|
||||
for i, raw := range strings.Split(text, "\n") {
|
||||
line := strings.TrimSpace(strings.TrimSuffix(raw, "\r"))
|
||||
if line == "" || strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
rule, err := parseRule(line)
|
||||
rule.Line = i + 1
|
||||
rule.Raw = line
|
||||
if err != nil {
|
||||
set.Errors = append(set.Errors, ParseError{Line: i + 1, Text: raw, Reason: err.Error()})
|
||||
continue
|
||||
}
|
||||
set.Rules = append(set.Rules, rule)
|
||||
}
|
||||
return set
|
||||
}
|
||||
|
||||
func parseRule(line string) (Rule, error) {
|
||||
rule := Rule{Effects: map[string]string{}}
|
||||
left, right, ok := strings.Cut(line, "=>")
|
||||
if !ok {
|
||||
return rule, fmt.Errorf("missing =>")
|
||||
}
|
||||
left, right = strings.TrimSpace(left), strings.TrimSpace(right)
|
||||
if left == "" || right == "" {
|
||||
return rule, fmt.Errorf("empty condition or effect")
|
||||
}
|
||||
|
||||
if left == "*" {
|
||||
rule.MatchAll = true
|
||||
} else {
|
||||
for _, part := range strings.Split(left, " and ") {
|
||||
cond, err := parseCondition(strings.TrimSpace(part))
|
||||
if err != nil {
|
||||
return rule, err
|
||||
}
|
||||
rule.Conditions = append(rule.Conditions, cond)
|
||||
}
|
||||
}
|
||||
|
||||
for _, part := range strings.Split(right, ",") {
|
||||
part = strings.TrimSpace(part)
|
||||
if part == "" {
|
||||
continue
|
||||
}
|
||||
field, value, err := parseEffect(part)
|
||||
if err != nil {
|
||||
return rule, err
|
||||
}
|
||||
rule.Effects[field] = value
|
||||
}
|
||||
if len(rule.Effects) == 0 {
|
||||
return rule, fmt.Errorf("no effects")
|
||||
}
|
||||
return rule, nil
|
||||
}
|
||||
|
||||
// parseEffect accepts `field=value` and `field: value` to be forgiving of typing.
|
||||
func parseEffect(part string) (string, string, error) {
|
||||
field, value, ok := strings.Cut(part, "=")
|
||||
if !ok {
|
||||
field, value, ok = strings.Cut(part, ":")
|
||||
}
|
||||
if !ok {
|
||||
return "", "", fmt.Errorf("effect %q is not field=value", part)
|
||||
}
|
||||
field = strings.ToLower(strings.TrimSpace(field))
|
||||
value = unquote(strings.TrimSpace(value))
|
||||
if !effectFields[field] {
|
||||
return "", "", fmt.Errorf("unknown effect field %q", field)
|
||||
}
|
||||
if value == "" {
|
||||
return "", "", fmt.Errorf("effect %q has no value", field)
|
||||
}
|
||||
switch field {
|
||||
case "importance":
|
||||
value = strings.ToLower(value)
|
||||
if !allowedImportance[value] {
|
||||
return "", "", fmt.Errorf("importance %q must be low, medium, or high", value)
|
||||
}
|
||||
case "action":
|
||||
value = strings.ToLower(value)
|
||||
if !allowedAction[value] {
|
||||
return "", "", fmt.Errorf("action %q must be delete, unsubscribe, reply, file, or keep", value)
|
||||
}
|
||||
}
|
||||
return field, value, nil
|
||||
}
|
||||
|
||||
func parseCondition(part string) (Condition, error) {
|
||||
fields := strings.Fields(part)
|
||||
if len(fields) < 3 {
|
||||
return Condition{}, fmt.Errorf("condition %q needs a field, an operator, and a value", part)
|
||||
}
|
||||
field := strings.ToLower(fields[0])
|
||||
switch field {
|
||||
case "from", "to", "subject", "source", "body":
|
||||
default:
|
||||
return Condition{}, fmt.Errorf("unknown condition field %q", field)
|
||||
}
|
||||
op := strings.ToLower(fields[1])
|
||||
switch op {
|
||||
case "contains", "equals", "starts_with", "ends_with":
|
||||
default:
|
||||
return Condition{}, fmt.Errorf("unknown operator %q", op)
|
||||
}
|
||||
value := unquote(strings.TrimSpace(strings.Join(fields[2:], " ")))
|
||||
if value == "" {
|
||||
return Condition{}, fmt.Errorf("condition %q has no value", part)
|
||||
}
|
||||
return Condition{Field: field, Op: op, Value: value}, nil
|
||||
}
|
||||
|
||||
func unquote(v string) string {
|
||||
if len(v) >= 2 {
|
||||
if (v[0] == '"' && v[len(v)-1] == '"') || (v[0] == '\'' && v[len(v)-1] == '\'') {
|
||||
return v[1 : len(v)-1]
|
||||
}
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// Match returns the effects of the first matching rule. It reports nil when no
|
||||
// rule matches. Field lookups are case-insensitive; values compare
|
||||
// case-insensitively too.
|
||||
func (rs RuleSet) Match(fields map[string]string) *Rule {
|
||||
normalized := make(map[string]string, len(fields))
|
||||
for k, v := range fields {
|
||||
normalized[strings.ToLower(k)] = strings.ToLower(v)
|
||||
}
|
||||
for i := range rs.Rules {
|
||||
r := &rs.Rules[i]
|
||||
if r.MatchAll || allMatch(r.Conditions, normalized) {
|
||||
return r
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func allMatch(conds []Condition, fields map[string]string) bool {
|
||||
for _, c := range conds {
|
||||
if !matches(c, fields[c.Field]) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func matches(c Condition, fieldValue string) bool {
|
||||
want := strings.ToLower(c.Value)
|
||||
switch c.Op {
|
||||
case "contains":
|
||||
return strings.Contains(fieldValue, want)
|
||||
case "equals":
|
||||
return fieldValue == want
|
||||
case "starts_with":
|
||||
return strings.HasPrefix(fieldValue, want)
|
||||
case "ends_with":
|
||||
return strings.HasSuffix(fieldValue, want)
|
||||
}
|
||||
return false
|
||||
}
|
||||
119
internal/classify/rules_test.go
Normal file
119
internal/classify/rules_test.go
Normal file
@@ -0,0 +1,119 @@
|
||||
package classify
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestParseRulesSkipsCommentsAndBlanks(t *testing.T) {
|
||||
set := ParseRules(`
|
||||
# a comment
|
||||
# an indented comment
|
||||
|
||||
from contains "amazon.com" => category=Newsletter, action=unsubscribe
|
||||
`)
|
||||
if len(set.Errors) != 0 {
|
||||
t.Fatalf("Errors = %v, want none", set.Errors)
|
||||
}
|
||||
if len(set.Rules) != 1 {
|
||||
t.Fatalf("Rules len = %d, want 1", len(set.Rules))
|
||||
}
|
||||
r := set.Rules[0]
|
||||
if r.Line != 5 {
|
||||
t.Errorf("Line = %d, want 5", r.Line)
|
||||
}
|
||||
if r.Effects["category"] != "Newsletter" || r.Effects["action"] != "unsubscribe" {
|
||||
t.Errorf("Effects = %v", r.Effects)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseRulesMatchAll(t *testing.T) {
|
||||
set := ParseRules(`* => category=Info`)
|
||||
if len(set.Rules) != 1 || !set.Rules[0].MatchAll {
|
||||
t.Fatalf("expected a match-all rule, got %+v (errors %v)", set.Rules, set.Errors)
|
||||
}
|
||||
if got := set.Match(map[string]string{"subject": "anything"}); got == nil {
|
||||
t.Fatal("match-all rule did not match")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseRulesReportsBadLines(t *testing.T) {
|
||||
set := ParseRules(`
|
||||
from => category=Info
|
||||
from contains "x" => importance=urgent
|
||||
from contains "x" => bogus=1
|
||||
missing the arrow
|
||||
`)
|
||||
if len(set.Rules) != 0 {
|
||||
t.Fatalf("Rules = %+v, want none", set.Rules)
|
||||
}
|
||||
if len(set.Errors) != 4 {
|
||||
t.Fatalf("Errors len = %d, want 4: %v", len(set.Errors), set.Errors)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRuleMatchAndPrecedence(t *testing.T) {
|
||||
set := ParseRules(`
|
||||
from ends_with "@stripe.com" and subject contains "invoice" => category=Action, importance=high
|
||||
from contains "stripe" => category=Newsletter
|
||||
* => category=Info
|
||||
`)
|
||||
if len(set.Errors) != 0 {
|
||||
t.Fatalf("unexpected errors: %v", set.Errors)
|
||||
}
|
||||
|
||||
// First matching rule wins.
|
||||
got := set.Match(map[string]string{
|
||||
"from": "Billing@Stripe.com",
|
||||
"subject": "Your invoice is ready",
|
||||
})
|
||||
if got == nil || got.Effects["category"] != "Action" {
|
||||
t.Fatalf("first rule did not win: %+v", got)
|
||||
}
|
||||
|
||||
// The second rule matches on its own terms.
|
||||
got = set.Match(map[string]string{"from": "stripe-press@example.com"})
|
||||
if got == nil || got.Effects["category"] != "Newsletter" {
|
||||
t.Fatalf("second rule did not match: %+v", got)
|
||||
}
|
||||
|
||||
// The catch-all is the fallback.
|
||||
got = set.Match(map[string]string{"from": "someone@example.com"})
|
||||
if got == nil || got.Effects["category"] != "Info" {
|
||||
t.Fatalf("catch-all did not match: %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRuleOperators(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
rule string
|
||||
fields map[string]string
|
||||
wantCat string
|
||||
wantNone bool
|
||||
}{
|
||||
{"equals", `from equals "a@b.com" => category=Action`, map[string]string{"from": "A@B.com"}, "Action", false},
|
||||
{"starts_with", `subject starts_with "Hello" => category=Info`, map[string]string{"subject": "hello there"}, "Info", false},
|
||||
{"ends_with", `source ends_with "where-woof.com" => category=Newsletter`, map[string]string{"source": "hello@where-woof.com"}, "Newsletter", false},
|
||||
{"body", `body contains "unsubscribe here" => category=Newsletter`, map[string]string{"body": "Click UNSUBSCRIBE HERE to stop"}, "Newsletter", false},
|
||||
{"no match", `from contains "zzz" => category=Action`, map[string]string{"from": "a@b.com"}, "", true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
set := ParseRules(tc.rule)
|
||||
if len(set.Errors) != 0 {
|
||||
t.Fatalf("parse errors: %v", set.Errors)
|
||||
}
|
||||
got := set.Match(tc.fields)
|
||||
if tc.wantNone {
|
||||
if got != nil {
|
||||
t.Fatalf("expected no match, got %+v", got)
|
||||
}
|
||||
return
|
||||
}
|
||||
if got == nil {
|
||||
t.Fatal("expected a match, got none")
|
||||
}
|
||||
if got.Effects["category"] != tc.wantCat {
|
||||
t.Errorf("category = %q, want %q", got.Effects["category"], tc.wantCat)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -28,6 +28,16 @@ const (
|
||||
DefaultIMAPPort = 993
|
||||
DefaultIMAPLookbackDays = 14
|
||||
DefaultIMAPMaxMessages = 100
|
||||
|
||||
// JEV (TypeSafe System One) defaults. The base URL is fixed by the TypeSafe
|
||||
// docs (https://docs.typesafe.ai/api.md); JEV_BASE_URL overrides it.
|
||||
DefaultJEVBaseURL = "https://api.typesafe.ai"
|
||||
DefaultJEVModel = "jev-latest"
|
||||
|
||||
// Classifier defaults (FR-3). Concurrency bounds in-flight JEV requests;
|
||||
// max messages bounds one run so a scheduled scan stays cheap.
|
||||
DefaultClassifyConcurrency = 4
|
||||
DefaultClassifyMaxMessages = 200
|
||||
)
|
||||
|
||||
// Config is the fully resolved runtime configuration.
|
||||
@@ -66,6 +76,12 @@ type Config struct {
|
||||
JEVBaseURL string
|
||||
// JEVAPIKey authenticates to JEV. Secret.
|
||||
JEVAPIKey string
|
||||
// JEVModel is the System One model alias (default jev-latest).
|
||||
JEVModel string
|
||||
// ClassifyConcurrency bounds in-flight JEV requests per run.
|
||||
ClassifyConcurrency int
|
||||
// ClassifyMaxMessages caps how many messages one classify run processes.
|
||||
ClassifyMaxMessages int
|
||||
|
||||
// OmniRouteBaseURL is the base URL of the OmniRoute gateway.
|
||||
OmniRouteBaseURL string
|
||||
@@ -99,8 +115,9 @@ func Load() (*Config, error) {
|
||||
Password: os.Getenv("EMAIL_AGENT_PASSWORD"),
|
||||
LogLevel: strings.ToLower(getenvDefault("EMAIL_AGENT_LOG_LEVEL", "info")),
|
||||
PublicBaseURL: strings.TrimRight(getenvDefault("EMAIL_AGENT_PUBLIC_BASE_URL", "http://localhost:8790"), "/"),
|
||||
JEVBaseURL: strings.TrimRight(os.Getenv("JEV_BASE_URL"), "/"),
|
||||
JEVBaseURL: strings.TrimRight(getenvDefault("JEV_BASE_URL", DefaultJEVBaseURL), "/"),
|
||||
JEVAPIKey: os.Getenv("JEV_API_KEY"),
|
||||
JEVModel: getenvDefault("JEV_MODEL", DefaultJEVModel),
|
||||
OmniRouteBaseURL: strings.TrimRight(os.Getenv("OMNIROUTE_BASE_URL"), "/"),
|
||||
OmniRouteAPIKey: os.Getenv("OMNIROUTE_API_KEY"),
|
||||
GmailUser: os.Getenv("GMAIL_USER"),
|
||||
@@ -116,6 +133,8 @@ func Load() (*Config, error) {
|
||||
c.IMAPPort = imapPort
|
||||
c.IMAPLookbackDays = getenvDefaultInt("EMAIL_AGENT_IMAP_LOOKBACK_DAYS", DefaultIMAPLookbackDays)
|
||||
c.IMAPMaxMessages = getenvDefaultInt("EMAIL_AGENT_IMAP_MAX_MESSAGES", DefaultIMAPMaxMessages)
|
||||
c.ClassifyConcurrency = getenvDefaultInt("EMAIL_AGENT_CLASSIFY_CONCURRENCY", DefaultClassifyConcurrency)
|
||||
c.ClassifyMaxMessages = getenvDefaultInt("EMAIL_AGENT_CLASSIFY_MAX_MESSAGES", DefaultClassifyMaxMessages)
|
||||
|
||||
port, err := resolvePort()
|
||||
if err != nil {
|
||||
@@ -219,6 +238,10 @@ func (c *Config) GmailConfigured() bool {
|
||||
return strings.TrimSpace(c.GmailUser) != "" && strings.TrimSpace(c.GmailAppPassword) != ""
|
||||
}
|
||||
|
||||
// JEVConfigured reports whether JEV classification can be attempted. The
|
||||
// service still starts without a key; only classification is unavailable.
|
||||
func (c *Config) JEVConfigured() bool { return strings.TrimSpace(c.JEVAPIKey) != "" }
|
||||
|
||||
// IsProduction reports whether the service runs in the production environment.
|
||||
func (c *Config) IsProduction() bool { return c.Env == "production" }
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"time"
|
||||
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/session"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/store"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/version"
|
||||
)
|
||||
|
||||
@@ -226,18 +227,55 @@ type dashboardData struct {
|
||||
Folders []navItem
|
||||
}
|
||||
|
||||
// categoryCounts returns per-category message counts keyed by the canonical
|
||||
// category name, plus the unclassified bucket.
|
||||
func categoryCounts(items []store.MessageListItem, categories []store.Category) map[string]int {
|
||||
canonical := make(map[string]string, len(categories))
|
||||
for _, c := range categories {
|
||||
canonical[strings.ToLower(c.Name)] = c.Name
|
||||
}
|
||||
counts := map[string]int{}
|
||||
for _, it := range items {
|
||||
raw := strings.TrimSpace(it.Category)
|
||||
if raw == "" {
|
||||
counts[unclassifiedCategory]++
|
||||
continue
|
||||
}
|
||||
if name, ok := canonical[strings.ToLower(raw)]; ok {
|
||||
counts[name]++
|
||||
} else {
|
||||
counts[raw]++
|
||||
}
|
||||
}
|
||||
return counts
|
||||
}
|
||||
|
||||
// handleDashboard renders the inbox shell with real category counts.
|
||||
func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
|
||||
sess := sessionFrom(r)
|
||||
|
||||
// FR-5: counts come from the stored messages. Phase 2 has no classifier
|
||||
// yet, so every message lands under Unclassified (AGENTS.md §13.2).
|
||||
// FR-5: counts come from the stored messages, grouped by the live category
|
||||
// list (FR-14).
|
||||
items, err := s.store.ListMessages(r.Context(), 200)
|
||||
if err != nil {
|
||||
s.logger.Error("load dashboard counts", "error", err)
|
||||
items = nil
|
||||
}
|
||||
counts := categoryCounts(items)
|
||||
categories, err := s.store.ListCategories(r.Context())
|
||||
if err != nil {
|
||||
s.logger.Error("load dashboard categories", "error", err)
|
||||
}
|
||||
counts := categoryCounts(items, categories)
|
||||
|
||||
catNav := make([]navItem, 0, len(categories)+1)
|
||||
for _, c := range categories {
|
||||
catNav = append(catNav, navItem{Name: c.Name, Color: "var(--primary-soft, var(--surface-2))", Count: counts[c.Name]})
|
||||
}
|
||||
catNav = append(catNav, navItem{
|
||||
Name: unclassifiedCategory,
|
||||
Color: "var(--ink-faint)",
|
||||
Count: counts[unclassifiedCategory],
|
||||
})
|
||||
|
||||
data := dashboardData{
|
||||
Greeting: partOfDay(time.Now()),
|
||||
@@ -245,17 +283,7 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
|
||||
CSRFToken: s.csrfToken(w, r),
|
||||
Version: version.Version,
|
||||
Total: len(items),
|
||||
// Placeholders until FR-14 (editable categories) and FR-9 (folders)
|
||||
// land. Colours are the design system's decorative sticker palette.
|
||||
Categories: []navItem{
|
||||
{Name: "Action", Color: "var(--primary)", Count: counts["action"]},
|
||||
{Name: "Important", Color: "var(--sticker-orange)", Count: counts["important"]},
|
||||
{Name: "Info", Color: "var(--sticker-sky)", Count: counts["info"]},
|
||||
{Name: "Spam", Color: "var(--sticker-red)", Count: counts["spam"]},
|
||||
{Name: "Newsletter", Color: "var(--sticker-purple)", Count: counts["newsletter"]},
|
||||
{Name: "Auto-reply", Color: "var(--sticker-teal)", Count: counts["auto-reply"]},
|
||||
{Name: "Unclassified", Color: "var(--ink-faint)", Count: counts[unclassifiedCategory]},
|
||||
},
|
||||
Categories: catNav,
|
||||
Folders: []navItem{
|
||||
{Name: "Replied-to", Color: "var(--sticker-green)"},
|
||||
{Name: "Delete", Color: "var(--sticker-red)"},
|
||||
|
||||
151
internal/server/categories.go
Normal file
151
internal/server/categories.go
Normal file
@@ -0,0 +1,151 @@
|
||||
// Category management UI and handlers (FR-14).
|
||||
//
|
||||
// The category list is editable: add, rename, and remove. JEV uses whatever list
|
||||
// is current at classification time, and renaming rewrites existing
|
||||
// classifications so the triage list stays consistent.
|
||||
package server
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/store"
|
||||
)
|
||||
|
||||
type categoriesData struct {
|
||||
Categories []store.Category
|
||||
CSRFToken string
|
||||
Note string
|
||||
Error string
|
||||
Total int
|
||||
Classified int
|
||||
}
|
||||
|
||||
// handleCategories renders the category editor the inbox nav links to.
|
||||
func (s *Server) handleCategories(w http.ResponseWriter, r *http.Request) {
|
||||
s.renderCategories(w, r, http.StatusOK, r.URL.Query().Get("note"), r.URL.Query().Get("error"))
|
||||
}
|
||||
|
||||
func (s *Server) renderCategories(w http.ResponseWriter, r *http.Request, code int, note, errMsg string) {
|
||||
categories, err := s.store.ListCategories(r.Context())
|
||||
if err != nil {
|
||||
s.logger.Error("list categories", "error", err)
|
||||
http.Error(w, "could not load categories", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
coverage, err := s.store.ClassificationCoverage(r.Context())
|
||||
if err != nil {
|
||||
s.logger.Error("classification coverage", "error", err)
|
||||
}
|
||||
|
||||
data := categoriesData{
|
||||
Categories: categories,
|
||||
CSRFToken: s.csrfToken(w, r),
|
||||
Note: note,
|
||||
Error: errMsg,
|
||||
Total: coverage.Total,
|
||||
Classified: coverage.Classified,
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
w.WriteHeader(code)
|
||||
if err := s.tmpl.ExecuteTemplate(w, "categories.html", data); err != nil {
|
||||
s.logger.Error("render categories", "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// handleCategoryAdd creates a category (FR-14).
|
||||
func (s *Server) handleCategoryAdd(w http.ResponseWriter, r *http.Request) {
|
||||
if !s.csrfOK(r) {
|
||||
http.Error(w, "invalid CSRF token", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
if err := r.ParseForm(); err != nil {
|
||||
s.redirectCategories(w, r, "", "Could not read the form.")
|
||||
return
|
||||
}
|
||||
name, description := r.FormValue("name"), r.FormValue("description")
|
||||
cat, err := s.store.AddCategory(r.Context(), name, description)
|
||||
switch {
|
||||
case err == nil:
|
||||
s.logger.Info("category added", "name", cat.Name)
|
||||
_ = s.store.LogAction(r.Context(), "category_add", "Sam", "name="+cat.Name)
|
||||
s.redirectCategories(w, r, fmt.Sprintf("Added category %q.", cat.Name), "")
|
||||
case errors.Is(err, store.ErrCategoryExists):
|
||||
s.redirectCategories(w, r, "", fmt.Sprintf("A category named %q already exists.", name))
|
||||
default:
|
||||
s.redirectCategories(w, r, "", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// handleCategoryRename changes a category's name and updates existing
|
||||
// classifications (FR-14).
|
||||
func (s *Server) handleCategoryRename(w http.ResponseWriter, r *http.Request) {
|
||||
if !s.csrfOK(r) {
|
||||
http.Error(w, "invalid CSRF token", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
if err := r.ParseForm(); err != nil {
|
||||
s.redirectCategories(w, r, "", "Could not read the form.")
|
||||
return
|
||||
}
|
||||
id, err := strconv.ParseInt(r.FormValue("id"), 10, 64)
|
||||
if err != nil {
|
||||
s.redirectCategories(w, r, "", "Unknown category.")
|
||||
return
|
||||
}
|
||||
name := r.FormValue("name")
|
||||
switch err := s.store.RenameCategory(r.Context(), id, name); {
|
||||
case err == nil:
|
||||
s.logger.Info("category renamed", "id", id, "name", name)
|
||||
_ = s.store.LogAction(r.Context(), "category_rename", "Sam",
|
||||
fmt.Sprintf("id=%d name=%s", id, name))
|
||||
s.redirectCategories(w, r, fmt.Sprintf("Renamed category to %q.", name), "")
|
||||
case errors.Is(err, store.ErrCategoryExists):
|
||||
s.redirectCategories(w, r, "", fmt.Sprintf("A category named %q already exists.", name))
|
||||
default:
|
||||
s.redirectCategories(w, r, "", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// handleCategoryRemove deletes a category and moves its messages to the first
|
||||
// remaining category (FR-14).
|
||||
func (s *Server) handleCategoryRemove(w http.ResponseWriter, r *http.Request) {
|
||||
if !s.csrfOK(r) {
|
||||
http.Error(w, "invalid CSRF token", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
if err := r.ParseForm(); err != nil {
|
||||
s.redirectCategories(w, r, "", "Could not read the form.")
|
||||
return
|
||||
}
|
||||
id, err := strconv.ParseInt(r.FormValue("id"), 10, 64)
|
||||
if err != nil {
|
||||
s.redirectCategories(w, r, "", "Unknown category.")
|
||||
return
|
||||
}
|
||||
switch err := s.store.RemoveCategory(r.Context(), id); {
|
||||
case err == nil:
|
||||
s.logger.Info("category removed", "id", id)
|
||||
_ = s.store.LogAction(r.Context(), "category_remove", "Sam", fmt.Sprintf("id=%d", id))
|
||||
s.redirectCategories(w, r, "Removed category; its messages moved to the first remaining category.", "")
|
||||
case errors.Is(err, store.ErrLastCategory):
|
||||
s.redirectCategories(w, r, "", "At least one category is required.")
|
||||
default:
|
||||
s.redirectCategories(w, r, "", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// redirectCategories is POST/redirect/GET so a refresh does not resubmit.
|
||||
func (s *Server) redirectCategories(w http.ResponseWriter, r *http.Request, note, errMsg string) {
|
||||
q := url.Values{}
|
||||
if note != "" {
|
||||
q.Set("note", note)
|
||||
}
|
||||
if errMsg != "" {
|
||||
q.Set("error", errMsg)
|
||||
}
|
||||
http.Redirect(w, r, "/categories?"+q.Encode(), http.StatusSeeOther)
|
||||
}
|
||||
213
internal/server/categories_test.go
Normal file
213
internal/server/categories_test.go
Normal file
@@ -0,0 +1,213 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/session"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/store"
|
||||
)
|
||||
|
||||
// login signs in and returns the session cookie plus the CSRF cookie/token.
|
||||
func login(t *testing.T, srv *Server) (*http.Cookie, *http.Cookie, string) {
|
||||
t.Helper()
|
||||
h := srv.Handler()
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/login", nil))
|
||||
csrf := csrfFromBody(t, rec.Body.String())
|
||||
csrfCookie := findCookie(t, rec.Result().Cookies(), csrfCookieName)
|
||||
|
||||
rec = httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, loginRequest(csrf, testPassword, csrfCookie))
|
||||
sessionCookie := findCookie(t, rec.Result().Cookies(), session.CookieName)
|
||||
if sessionCookie == nil {
|
||||
t.Fatal("login did not set the session cookie")
|
||||
}
|
||||
return sessionCookie, csrfCookie, csrf
|
||||
}
|
||||
|
||||
func postForm(t *testing.T, srv *Server, path string, session, csrfCookie *http.Cookie, form url.Values) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
req := httptest.NewRequest(http.MethodPost, path, strings.NewReader(form.Encode()))
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
req.AddCookie(session)
|
||||
if csrfCookie != nil {
|
||||
req.AddCookie(csrfCookie)
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
srv.Handler().ServeHTTP(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
func getWithSession(t *testing.T, srv *Server, path string, session *http.Cookie) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
req := httptest.NewRequest(http.MethodGet, path, nil)
|
||||
req.AddCookie(session)
|
||||
rec := httptest.NewRecorder()
|
||||
srv.Handler().ServeHTTP(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
func TestCategoriesPageRenders(t *testing.T) {
|
||||
srv := newTestServer(t)
|
||||
session, _, _ := login(t, srv)
|
||||
|
||||
rec := getWithSession(t, srv, "/categories", session)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200", rec.Code)
|
||||
}
|
||||
body := rec.Body.String()
|
||||
for _, want := range []string{"Categories", "Action", "Spam", "Add category", "Back to inbox"} {
|
||||
if !strings.Contains(body, want) {
|
||||
t.Errorf("categories page missing %q", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCategoryAddRenameRemove(t *testing.T) {
|
||||
srv := newTestServer(t)
|
||||
session, csrfCookie, csrf := login(t, srv)
|
||||
|
||||
// Add (FR-14).
|
||||
rec := postForm(t, srv, "/categories/add", session, csrfCookie, url.Values{
|
||||
"csrf": {csrf}, "name": {"Finance"}, "description": {"Bills and payments"},
|
||||
})
|
||||
if rec.Code != http.StatusSeeOther {
|
||||
t.Fatalf("add status = %d, want 303 (body %s)", rec.Code, rec.Body.String())
|
||||
}
|
||||
if rec := getWithSession(t, srv, "/categories", session); !strings.Contains(rec.Body.String(), "Finance") {
|
||||
t.Fatal("added category is not rendered")
|
||||
}
|
||||
|
||||
id := categoryID(t, srv, "Finance")
|
||||
|
||||
// Rename (FR-14).
|
||||
rec = postForm(t, srv, "/categories/rename", session, csrfCookie, url.Values{
|
||||
"csrf": {csrf}, "id": {itoa(id)}, "name": {"Money"},
|
||||
})
|
||||
if rec.Code != http.StatusSeeOther {
|
||||
t.Fatalf("rename status = %d, want 303", rec.Code)
|
||||
}
|
||||
body := getWithSession(t, srv, "/categories", session).Body.String()
|
||||
if !strings.Contains(body, "Money") || strings.Contains(body, ">Finance<") {
|
||||
t.Fatalf("rename not reflected; body contains Finance=%v Money=%v",
|
||||
strings.Contains(body, "Finance"), strings.Contains(body, "Money"))
|
||||
}
|
||||
|
||||
// Remove (FR-14).
|
||||
id = categoryID(t, srv, "Money")
|
||||
rec = postForm(t, srv, "/categories/remove", session, csrfCookie, url.Values{
|
||||
"csrf": {csrf}, "id": {itoa(id)},
|
||||
})
|
||||
if rec.Code != http.StatusSeeOther {
|
||||
t.Fatalf("remove status = %d, want 303", rec.Code)
|
||||
}
|
||||
if rec := getWithSession(t, srv, "/categories", session); strings.Contains(rec.Body.String(), "Money") {
|
||||
t.Fatal("removed category is still rendered")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCategoryMutationsRequireCSRF(t *testing.T) {
|
||||
srv := newTestServer(t)
|
||||
session, _, _ := login(t, srv)
|
||||
|
||||
rec := postForm(t, srv, "/categories/add", session, nil, url.Values{"name": {"Nope"}})
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Fatalf("add without CSRF status = %d, want 403", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyAnonymousRedirects(t *testing.T) {
|
||||
srv := newTestServer(t)
|
||||
for _, path := range []string{"/actions/classify", "/actions/reclassify"} {
|
||||
req := httptest.NewRequest(http.MethodPost, path, nil)
|
||||
rec := httptest.NewRecorder()
|
||||
srv.Handler().ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusSeeOther {
|
||||
t.Errorf("%s status = %d, want 303 (redirect to login)", path, rec.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyWithoutJEVKeyReportsNotConfigured(t *testing.T) {
|
||||
srv := newTestServer(t)
|
||||
session, csrfCookie, csrf := login(t, srv)
|
||||
|
||||
rec := postForm(t, srv, "/actions/classify", session, csrfCookie, url.Values{"csrf": {csrf}})
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200", rec.Code)
|
||||
}
|
||||
if !strings.Contains(rec.Body.String(), "JEV_API_KEY") {
|
||||
t.Errorf("expected a not-configured note; body: %s", rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestDashboardLinksToCategoriesAndShowsClassify(t *testing.T) {
|
||||
srv := newTestServer(t)
|
||||
session, _, _ := login(t, srv)
|
||||
|
||||
rec := getWithSession(t, srv, "/", session)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200", rec.Code)
|
||||
}
|
||||
body := rec.Body.String()
|
||||
for _, want := range []string{`href="/categories"`, "Classify", "Re-classify all", "Action", "Auto-reply"} {
|
||||
if !strings.Contains(body, want) {
|
||||
t.Errorf("dashboard missing %q", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInboxShowsClassificationConfidence(t *testing.T) {
|
||||
srv := newTestServer(t)
|
||||
ctx := context.Background()
|
||||
|
||||
msgID, err := srv.store.UpsertMessage(ctx, &store.Message{
|
||||
GmailID: "cls-1", FromAddr: "a@b.com", Subject: "Classified mail",
|
||||
SourceAddress: "hello@where-woof.com", DateSent: time.Now().UTC(),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("seed message: %v", err)
|
||||
}
|
||||
if _, err := srv.store.SaveClassification(ctx, store.Classification{
|
||||
MessageID: msgID, Category: "Action", Importance: "high", Action: "reply",
|
||||
Confidence: 0.82, Model: "jev-1.13.0",
|
||||
}); err != nil {
|
||||
t.Fatalf("seed classification: %v", err)
|
||||
}
|
||||
|
||||
session, _, _ := login(t, srv)
|
||||
rec := getWithSession(t, srv, "/partials/inbox", session)
|
||||
body := rec.Body.String()
|
||||
for _, want := range []string{"Classified mail", "Action", "reply", "82%"} {
|
||||
if !strings.Contains(body, want) {
|
||||
t.Errorf("inbox partial missing %q", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func categoryID(t *testing.T, srv *Server, name string) int64 {
|
||||
t.Helper()
|
||||
cats, err := srv.store.ListCategories(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("ListCategories() error = %v", err)
|
||||
}
|
||||
for _, c := range cats {
|
||||
if c.Name == name {
|
||||
return c.ID
|
||||
}
|
||||
}
|
||||
t.Fatalf("category %q not found", name)
|
||||
return 0
|
||||
}
|
||||
|
||||
func itoa(v int64) string {
|
||||
return strconv.FormatInt(v, 10)
|
||||
}
|
||||
@@ -1,9 +1,8 @@
|
||||
// Inbox rendering and the manual mail scan.
|
||||
// Inbox rendering, the manual mail scan, and the classify action.
|
||||
//
|
||||
// Phase 2 lists real mail from the database. Classification is Phase 3, so
|
||||
// messages without a classification are shown under "Unclassified" with a
|
||||
// temporary "Review" recommended action; the grouping and action plumbing is
|
||||
// already category-aware.
|
||||
// Phase 3 groups messages by the live category list (FR-5) and shows the JEV
|
||||
// recommended action plus confidence on each row. Messages without a
|
||||
// classification fall under "Unclassified".
|
||||
package server
|
||||
|
||||
import (
|
||||
@@ -15,26 +14,13 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/classify"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/mailfetch"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/store"
|
||||
)
|
||||
|
||||
// unclassifiedCategory is the fallback bucket before Phase 3 adds JEV
|
||||
// categories (AGENTS.md §13.2).
|
||||
const unclassifiedCategory = "unclassified"
|
||||
|
||||
// categoryOrder is the display order for the built-in categories.
|
||||
var categoryOrder = []string{"action", "important", "info", "spam", "newsletter", "auto-reply"}
|
||||
|
||||
var categoryDisplay = map[string]string{
|
||||
"action": "Action",
|
||||
"important": "Important",
|
||||
"info": "Info",
|
||||
"spam": "Spam",
|
||||
"newsletter": "Newsletter",
|
||||
"auto-reply": "Auto-reply",
|
||||
"unclassified": "Unclassified",
|
||||
}
|
||||
// unclassifiedCategory is the bucket for messages JEV has not classified yet.
|
||||
const unclassifiedCategory = "Unclassified"
|
||||
|
||||
type messageRow struct {
|
||||
ID int64
|
||||
@@ -42,6 +28,8 @@ type messageRow struct {
|
||||
Subject string
|
||||
SourceAddr string
|
||||
Action string
|
||||
Confidence int
|
||||
Classified bool
|
||||
When string
|
||||
}
|
||||
|
||||
@@ -57,21 +45,42 @@ type inboxData struct {
|
||||
Error string
|
||||
}
|
||||
|
||||
// loadInbox reads messages and groups them for rendering.
|
||||
// loadInbox reads messages and groups them by the live category list.
|
||||
func (s *Server) loadInbox(ctx context.Context) (inboxData, error) {
|
||||
items, err := s.store.ListMessages(ctx, 200)
|
||||
if err != nil {
|
||||
return inboxData{}, err
|
||||
}
|
||||
return inboxData{Groups: groupMessages(items), Total: len(items)}, nil
|
||||
categories, err := s.store.ListCategories(ctx)
|
||||
if err != nil {
|
||||
return inboxData{}, err
|
||||
}
|
||||
return inboxData{Groups: groupMessages(items, categories), Total: len(items)}, nil
|
||||
}
|
||||
|
||||
// groupMessages buckets messages by category, ordering groups the way the
|
||||
// category list is ordered in the UI. Unknown categories (for example a
|
||||
// classification written before a rename) keep their own group; messages with
|
||||
// no classification fall into "Unclassified" last.
|
||||
func groupMessages(items []store.MessageListItem, categories []store.Category) []messageGroup {
|
||||
canonical := make(map[string]string, len(categories))
|
||||
order := make([]string, 0, len(categories))
|
||||
for _, c := range categories {
|
||||
canonical[strings.ToLower(c.Name)] = c.Name
|
||||
order = append(order, c.Name)
|
||||
}
|
||||
|
||||
// groupMessages buckets messages by category. Unknown (custom) categories get
|
||||
// their own group; messages with no classification fall into "Unclassified".
|
||||
func groupMessages(items []store.MessageListItem) []messageGroup {
|
||||
buckets := map[string][]messageRow{}
|
||||
for _, it := range items {
|
||||
key := strings.ToLower(strings.TrimSpace(it.Category))
|
||||
raw := strings.TrimSpace(it.Category)
|
||||
key := ""
|
||||
if raw != "" {
|
||||
if name, ok := canonical[strings.ToLower(raw)]; ok {
|
||||
key = name
|
||||
} else {
|
||||
key = raw
|
||||
}
|
||||
}
|
||||
if key == "" {
|
||||
key = unclassifiedCategory
|
||||
}
|
||||
@@ -85,62 +94,47 @@ func groupMessages(items []store.MessageListItem) []messageGroup {
|
||||
Subject: orDefault(it.Subject, "(no subject)"),
|
||||
SourceAddr: it.SourceAddress,
|
||||
Action: action,
|
||||
Confidence: confidencePercent(it.Confidence),
|
||||
Classified: it.HasClassification,
|
||||
When: formatWhen(it.DateSent, it.ReceivedAt),
|
||||
})
|
||||
}
|
||||
|
||||
// Order: built-in categories first, then custom ones alphabetically, then
|
||||
// Unclassified last.
|
||||
var keys []string
|
||||
for _, k := range categoryOrder {
|
||||
if _, ok := buckets[k]; ok {
|
||||
keys = append(keys, k)
|
||||
for _, name := range order {
|
||||
if _, ok := buckets[name]; ok {
|
||||
keys = append(keys, name)
|
||||
}
|
||||
}
|
||||
var custom []string
|
||||
var unknown []string
|
||||
for k := range buckets {
|
||||
if k == unclassifiedCategory || contains(categoryOrder, k) {
|
||||
if k == unclassifiedCategory || contains(order, k) {
|
||||
continue
|
||||
}
|
||||
custom = append(custom, k)
|
||||
unknown = append(unknown, k)
|
||||
}
|
||||
sort.Strings(custom)
|
||||
keys = append(keys, custom...)
|
||||
sort.Strings(unknown)
|
||||
keys = append(keys, unknown...)
|
||||
if _, ok := buckets[unclassifiedCategory]; ok {
|
||||
keys = append(keys, unclassifiedCategory)
|
||||
}
|
||||
|
||||
groups := make([]messageGroup, 0, len(keys))
|
||||
for _, k := range keys {
|
||||
groups = append(groups, messageGroup{
|
||||
Name: displayCategory(k),
|
||||
Messages: buckets[k],
|
||||
})
|
||||
groups = append(groups, messageGroup{Name: k, Messages: buckets[k]})
|
||||
}
|
||||
return groups
|
||||
}
|
||||
|
||||
func displayCategory(key string) string {
|
||||
if name, ok := categoryDisplay[key]; ok {
|
||||
return name
|
||||
// confidencePercent turns a 0–1 confidence into a whole percentage.
|
||||
func confidencePercent(v float64) int {
|
||||
if v <= 0 {
|
||||
return 0
|
||||
}
|
||||
if key == "" {
|
||||
return "Unclassified"
|
||||
if v > 1 {
|
||||
v = 1
|
||||
}
|
||||
return strings.ToUpper(key[:1]) + key[1:]
|
||||
}
|
||||
|
||||
// categoryCounts returns per-category message counts for the left nav.
|
||||
func categoryCounts(items []store.MessageListItem) map[string]int {
|
||||
counts := map[string]int{}
|
||||
for _, it := range items {
|
||||
key := strings.ToLower(strings.TrimSpace(it.Category))
|
||||
if key == "" {
|
||||
key = unclassifiedCategory
|
||||
}
|
||||
counts[key]++
|
||||
}
|
||||
return counts
|
||||
return int(v*100 + 0.5)
|
||||
}
|
||||
|
||||
func (s *Server) renderInbox(w http.ResponseWriter, r *http.Request, note string) {
|
||||
@@ -187,6 +181,53 @@ func (s *Server) handleFetchMail(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
}
|
||||
|
||||
// handleClassify classifies pending messages (FR-3) and returns the refreshed
|
||||
// list. A run can take minutes, so the write deadline is extended for this
|
||||
// handler only.
|
||||
func (s *Server) handleClassify(w http.ResponseWriter, r *http.Request) {
|
||||
s.runClassifier(w, r, false)
|
||||
}
|
||||
|
||||
// handleReclassify re-runs the classifier over every stored message. It is how
|
||||
// a category-list or rules edit is applied to mail that was already classified.
|
||||
func (s *Server) handleReclassify(w http.ResponseWriter, r *http.Request) {
|
||||
s.runClassifier(w, r, true)
|
||||
}
|
||||
|
||||
func (s *Server) runClassifier(w http.ResponseWriter, r *http.Request, all bool) {
|
||||
if !s.csrfOK(r) {
|
||||
http.Error(w, "invalid CSRF token", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
if rc := http.NewResponseController(w); rc != nil {
|
||||
_ = rc.SetWriteDeadline(time.Now().Add(15 * time.Minute))
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
var (
|
||||
sum classify.Summary
|
||||
err error
|
||||
)
|
||||
if all {
|
||||
sum, err = s.classifier.RunAll(ctx)
|
||||
} else {
|
||||
sum, err = s.classifier.Run(ctx)
|
||||
}
|
||||
switch {
|
||||
case err == nil:
|
||||
s.renderInbox(w, r, fmt.Sprintf(
|
||||
"Classified %d of %d message(s); %d failed; %d still unclassified.",
|
||||
sum.Classified, sum.Considered, sum.Failed, sum.Remaining))
|
||||
case errors.Is(err, classify.ErrNotConfigured):
|
||||
s.renderInbox(w, r, "Classification is not connected yet: set JEV_API_KEY.")
|
||||
default:
|
||||
s.logger.Error("classification run failed", "error", err)
|
||||
s.renderInbox(w, r, "Classification failed: "+err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func orDefault(v, def string) string {
|
||||
if strings.TrimSpace(v) == "" {
|
||||
return def
|
||||
|
||||
@@ -17,6 +17,7 @@ import (
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/classify"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/config"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/mailfetch"
|
||||
"gitea.lab.audasmedia.com.au/sam/email_agent/internal/session"
|
||||
@@ -39,6 +40,7 @@ type Server struct {
|
||||
tmpl *template.Template
|
||||
sessions *session.Manager
|
||||
fetcher *mailfetch.Fetcher
|
||||
classifier *classify.Classifier
|
||||
}
|
||||
|
||||
// New builds a Server. It does not start listening.
|
||||
@@ -64,6 +66,10 @@ func New(cfg *config.Config, st *store.Store, logger *slog.Logger) (*Server, err
|
||||
tmpl: tmpl,
|
||||
sessions: sessions,
|
||||
fetcher: mailfetch.NewFetcher(cfg, st, logger),
|
||||
classifier: classify.NewClassifier(
|
||||
classify.NewClient(cfg.JEVBaseURL, cfg.JEVAPIKey, cfg.JEVModel, logger),
|
||||
st, logger, cfg.ClassifyConcurrency, cfg.ClassifyMaxMessages,
|
||||
),
|
||||
}
|
||||
s.http = &http.Server{
|
||||
Addr: cfg.Addr,
|
||||
@@ -113,6 +119,14 @@ func (s *Server) Handler() http.Handler {
|
||||
mux.HandleFunc("GET /{$}", s.requireAuth(s.handleDashboard))
|
||||
mux.HandleFunc("GET /partials/inbox", s.requireAuth(s.handleInboxPartial))
|
||||
mux.HandleFunc("POST /actions/fetch", s.requireAuth(s.handleFetchMail))
|
||||
mux.HandleFunc("POST /actions/classify", s.requireAuth(s.handleClassify))
|
||||
mux.HandleFunc("POST /actions/reclassify", s.requireAuth(s.handleReclassify))
|
||||
|
||||
// FR-14: editable category list.
|
||||
mux.HandleFunc("GET /categories", s.requireAuth(s.handleCategories))
|
||||
mux.HandleFunc("POST /categories/add", s.requireAuth(s.handleCategoryAdd))
|
||||
mux.HandleFunc("POST /categories/rename", s.requireAuth(s.handleCategoryRename))
|
||||
mux.HandleFunc("POST /categories/remove", s.requireAuth(s.handleCategoryRemove))
|
||||
|
||||
return s.recoverPanic(s.logRequests(mux))
|
||||
}
|
||||
@@ -234,6 +248,10 @@ type statusRecorder struct {
|
||||
status int
|
||||
}
|
||||
|
||||
// Unwrap lets http.ResponseController reach the underlying writer (the classify
|
||||
// handler extends its write deadline for long runs).
|
||||
func (r *statusRecorder) Unwrap() http.ResponseWriter { return r.ResponseWriter }
|
||||
|
||||
func (r *statusRecorder) WriteHeader(code int) {
|
||||
r.status = code
|
||||
r.ResponseWriter.WriteHeader(code)
|
||||
|
||||
218
internal/store/categories.go
Normal file
218
internal/store/categories.go
Normal file
@@ -0,0 +1,218 @@
|
||||
// Category management (FR-14) and the classification read/write paths (FR-3).
|
||||
//
|
||||
// `categories.name` is the stable key: it is stored in `classifications.category`
|
||||
// and sent to JEV as the Choice option name. Renaming a category therefore
|
||||
// cascades to existing classifications so the triage list never shows a category
|
||||
// that no longer exists.
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ErrLastCategory is returned when removing the final category would leave the
|
||||
// classifier with nothing to choose from.
|
||||
var ErrLastCategory = errors.New("at least one category is required")
|
||||
|
||||
// ErrCategoryExists is returned when adding or renaming to a name already taken.
|
||||
var ErrCategoryExists = errors.New("a category with that name already exists")
|
||||
|
||||
// Category is one editable triage category.
|
||||
type Category struct {
|
||||
ID int64
|
||||
Name string
|
||||
Description string
|
||||
Position int
|
||||
// MessageCount is the number of messages whose latest classification is
|
||||
// this category. It is filled by ListCategories.
|
||||
MessageCount int
|
||||
}
|
||||
|
||||
// ListCategories returns the categories in display order, each with the count of
|
||||
// messages currently classified into it.
|
||||
func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
|
||||
const q = `
|
||||
SELECT c.id, c.name, c.description, c.position,
|
||||
(SELECT COUNT(*) FROM messages m
|
||||
WHERE (SELECT cc.category FROM classifications cc
|
||||
WHERE cc.message_id = m.id ORDER BY cc.id DESC LIMIT 1) = c.name)
|
||||
FROM categories c
|
||||
ORDER BY c.position, c.name COLLATE NOCASE`
|
||||
rows, err := s.db.QueryContext(ctx, q)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list categories: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []Category
|
||||
for rows.Next() {
|
||||
var c Category
|
||||
if err := rows.Scan(&c.ID, &c.Name, &c.Description, &c.Position, &c.MessageCount); err != nil {
|
||||
return nil, fmt.Errorf("scan category: %w", err)
|
||||
}
|
||||
out = append(out, c)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("list categories: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// CategoryNames returns just the names, in display order.
|
||||
func (s *Store) CategoryNames(ctx context.Context) ([]string, error) {
|
||||
cats, err := s.ListCategories(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
names := make([]string, 0, len(cats))
|
||||
for _, c := range cats {
|
||||
names = append(names, c.Name)
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
// AddCategory appends a category. The name is trimmed, length-capped, and must
|
||||
// be unique (case-insensitively).
|
||||
func (s *Store) AddCategory(ctx context.Context, name, description string) (Category, error) {
|
||||
name, err := cleanCategoryName(name)
|
||||
if err != nil {
|
||||
return Category{}, err
|
||||
}
|
||||
description = strings.TrimSpace(description)
|
||||
|
||||
var exists int
|
||||
if err := s.db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM categories WHERE name = ? COLLATE NOCASE`, name).Scan(&exists); err != nil {
|
||||
return Category{}, fmt.Errorf("check category name: %w", err)
|
||||
}
|
||||
if exists > 0 {
|
||||
return Category{}, ErrCategoryExists
|
||||
}
|
||||
|
||||
var position int
|
||||
if err := s.db.QueryRowContext(ctx,
|
||||
`SELECT COALESCE(MAX(position), 0) + 1 FROM categories`).Scan(&position); err != nil {
|
||||
return Category{}, fmt.Errorf("next category position: %w", err)
|
||||
}
|
||||
|
||||
res, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO categories (name, description, position) VALUES (?, ?, ?)`,
|
||||
name, description, position)
|
||||
if err != nil {
|
||||
return Category{}, fmt.Errorf("insert category %q: %w", name, err)
|
||||
}
|
||||
id, err := res.LastInsertId()
|
||||
if err != nil {
|
||||
return Category{}, fmt.Errorf("read new category id: %w", err)
|
||||
}
|
||||
return Category{ID: id, Name: name, Description: description, Position: position}, nil
|
||||
}
|
||||
|
||||
// RenameCategory changes the display name and rewrites existing classifications
|
||||
// so the new name is used everywhere (FR-14).
|
||||
func (s *Store) RenameCategory(ctx context.Context, id int64, newName string) error {
|
||||
old, err := s.categoryByID(ctx, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
name, err := cleanCategoryName(newName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if strings.EqualFold(name, old) {
|
||||
return nil
|
||||
}
|
||||
|
||||
var exists int
|
||||
if err := s.db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM categories WHERE name = ? COLLATE NOCASE AND id <> ?`, name, id).Scan(&exists); err != nil {
|
||||
return fmt.Errorf("check category name: %w", err)
|
||||
}
|
||||
if exists > 0 {
|
||||
return ErrCategoryExists
|
||||
}
|
||||
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin rename: %w", err)
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
if _, err := tx.ExecContext(ctx, `UPDATE categories SET name = ? WHERE id = ?`, name, id); err != nil {
|
||||
return fmt.Errorf("rename category: %w", err)
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`UPDATE classifications SET category = ? WHERE category = ? COLLATE NOCASE`, name, old); err != nil {
|
||||
return fmt.Errorf("update classifications after rename: %w", err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return fmt.Errorf("commit rename: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RemoveCategory deletes a category and moves any messages still classified into
|
||||
// it to the first remaining category, so no classification points at a category
|
||||
// that no longer exists.
|
||||
func (s *Store) RemoveCategory(ctx context.Context, id int64) error {
|
||||
name, err := s.categoryByID(ctx, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var count int
|
||||
if err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM categories`).Scan(&count); err != nil {
|
||||
return fmt.Errorf("count categories: %w", err)
|
||||
}
|
||||
if count <= 1 {
|
||||
return ErrLastCategory
|
||||
}
|
||||
|
||||
var fallback string
|
||||
if err := s.db.QueryRowContext(ctx,
|
||||
`SELECT name FROM categories WHERE id <> ? ORDER BY position, name COLLATE NOCASE LIMIT 1`, id).Scan(&fallback); err != nil {
|
||||
return fmt.Errorf("select fallback category: %w", err)
|
||||
}
|
||||
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin remove: %w", err)
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`UPDATE classifications SET category = ? WHERE category = ? COLLATE NOCASE`, fallback, name); err != nil {
|
||||
return fmt.Errorf("reassign classifications: %w", err)
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, `DELETE FROM categories WHERE id = ?`, id); err != nil {
|
||||
return fmt.Errorf("delete category: %w", err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return fmt.Errorf("commit remove: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) categoryByID(ctx context.Context, id int64) (string, error) {
|
||||
var name string
|
||||
if err := s.db.QueryRowContext(ctx,
|
||||
`SELECT name FROM categories WHERE id = ?`, id).Scan(&name); err != nil {
|
||||
return "", fmt.Errorf("read category %d: %w", id, err)
|
||||
}
|
||||
return name, nil
|
||||
}
|
||||
|
||||
// cleanCategoryName trims, caps at 40 characters, and rejects an empty name.
|
||||
func cleanCategoryName(name string) (string, error) {
|
||||
name = strings.Join(strings.Fields(strings.TrimSpace(name)), " ")
|
||||
if name == "" {
|
||||
return "", errors.New("category name must not be empty")
|
||||
}
|
||||
if len([]rune(name)) > 40 {
|
||||
return "", errors.New("category name must be 40 characters or fewer")
|
||||
}
|
||||
return name, nil
|
||||
}
|
||||
245
internal/store/categories_test.go
Normal file
245
internal/store/categories_test.go
Normal file
@@ -0,0 +1,245 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func newCategoryStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
s, err := Open(context.Background(), filepath.Join(t.TempDir(), "email_agent.db"))
|
||||
if err != nil {
|
||||
t.Fatalf("Open() error = %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = s.Close() })
|
||||
return s
|
||||
}
|
||||
|
||||
func TestDefaultCategoriesSeeded(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := newCategoryStore(t)
|
||||
|
||||
cats, err := s.ListCategories(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ListCategories() error = %v", err)
|
||||
}
|
||||
want := []string{"Action", "Important", "Info", "Spam", "Newsletter", "Auto-reply"}
|
||||
if len(cats) != len(want) {
|
||||
t.Fatalf("categories = %d, want %d: %+v", len(cats), len(want), cats)
|
||||
}
|
||||
for i, name := range want {
|
||||
if cats[i].Name != name {
|
||||
t.Errorf("category %d = %q, want %q", i, cats[i].Name, name)
|
||||
}
|
||||
if cats[i].Description == "" {
|
||||
t.Errorf("category %q has no description (JEV needs the rubric)", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSeedCategoriesDoesNotReaddRemoved(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
path := filepath.Join(t.TempDir(), "email_agent.db")
|
||||
|
||||
s1, err := Open(ctx, path)
|
||||
if err != nil {
|
||||
t.Fatalf("Open() error = %v", err)
|
||||
}
|
||||
cats, err := s1.ListCategories(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ListCategories() error = %v", err)
|
||||
}
|
||||
var spamID int64
|
||||
for _, c := range cats {
|
||||
if c.Name == "Spam" {
|
||||
spamID = c.ID
|
||||
}
|
||||
}
|
||||
if err := s1.RemoveCategory(ctx, spamID); err != nil {
|
||||
t.Fatalf("RemoveCategory() error = %v", err)
|
||||
}
|
||||
_ = s1.Close()
|
||||
|
||||
s2, err := Open(ctx, path)
|
||||
if err != nil {
|
||||
t.Fatalf("second Open() error = %v", err)
|
||||
}
|
||||
defer s2.Close()
|
||||
cats, err = s2.ListCategories(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ListCategories() after restart error = %v", err)
|
||||
}
|
||||
for _, c := range cats {
|
||||
if c.Name == "Spam" {
|
||||
t.Error("removed category came back after restart")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddCategoryRejectsDuplicate(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := newCategoryStore(t)
|
||||
|
||||
if _, err := s.AddCategory(ctx, "Finance", "Bills and payments"); err != nil {
|
||||
t.Fatalf("AddCategory() error = %v", err)
|
||||
}
|
||||
if _, err := s.AddCategory(ctx, "finance", ""); !errors.Is(err, ErrCategoryExists) {
|
||||
t.Fatalf("duplicate AddCategory() error = %v, want ErrCategoryExists", err)
|
||||
}
|
||||
if _, err := s.AddCategory(ctx, " ", ""); err == nil {
|
||||
t.Fatal("blank AddCategory() error = nil, want an error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenameCategoryCascadesToClassifications(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := newCategoryStore(t)
|
||||
|
||||
msgID, err := s.UpsertMessage(ctx, &Message{
|
||||
GmailID: "1", Subject: "hi", FromAddr: "a@b.com", DateSent: time.Now().UTC(),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("UpsertMessage() error = %v", err)
|
||||
}
|
||||
if _, err := s.SaveClassification(ctx, Classification{
|
||||
MessageID: msgID, Category: "Info", Importance: "low", Action: "keep",
|
||||
}); err != nil {
|
||||
t.Fatalf("SaveClassification() error = %v", err)
|
||||
}
|
||||
|
||||
cats, err := s.ListCategories(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ListCategories() error = %v", err)
|
||||
}
|
||||
var infoID int64
|
||||
for _, c := range cats {
|
||||
if c.Name == "Info" {
|
||||
infoID = c.ID
|
||||
}
|
||||
}
|
||||
if err := s.RenameCategory(ctx, infoID, "Updates"); err != nil {
|
||||
t.Fatalf("RenameCategory() error = %v", err)
|
||||
}
|
||||
|
||||
items, err := s.ListMessages(ctx, 10)
|
||||
if err != nil {
|
||||
t.Fatalf("ListMessages() error = %v", err)
|
||||
}
|
||||
if items[0].Category != "Updates" {
|
||||
t.Errorf("classification after rename = %q, want Updates", items[0].Category)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRemoveCategoryReassignsMessages(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := newCategoryStore(t)
|
||||
|
||||
msgID, err := s.UpsertMessage(ctx, &Message{GmailID: "1", Subject: "hi"})
|
||||
if err != nil {
|
||||
t.Fatalf("UpsertMessage() error = %v", err)
|
||||
}
|
||||
if _, err := s.SaveClassification(ctx, Classification{MessageID: msgID, Category: "Spam"}); err != nil {
|
||||
t.Fatalf("SaveClassification() error = %v", err)
|
||||
}
|
||||
|
||||
cats, err := s.ListCategories(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ListCategories() error = %v", err)
|
||||
}
|
||||
var spamID int64
|
||||
for _, c := range cats {
|
||||
if c.Name == "Spam" {
|
||||
spamID = c.ID
|
||||
}
|
||||
}
|
||||
if err := s.RemoveCategory(ctx, spamID); err != nil {
|
||||
t.Fatalf("RemoveCategory() error = %v", err)
|
||||
}
|
||||
|
||||
items, err := s.ListMessages(ctx, 10)
|
||||
if err != nil {
|
||||
t.Fatalf("ListMessages() error = %v", err)
|
||||
}
|
||||
if items[0].Category != "Action" {
|
||||
t.Errorf("reassigned category = %q, want the first remaining category Action", items[0].Category)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRemoveLastCategoryRefused(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := newCategoryStore(t)
|
||||
|
||||
for {
|
||||
cats, err := s.ListCategories(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ListCategories() error = %v", err)
|
||||
}
|
||||
if len(cats) == 1 {
|
||||
if err := s.RemoveCategory(ctx, cats[0].ID); !errors.Is(err, ErrLastCategory) {
|
||||
t.Fatalf("RemoveCategory(last) error = %v, want ErrLastCategory", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err := s.RemoveCategory(ctx, cats[0].ID); err != nil {
|
||||
t.Fatalf("RemoveCategory() error = %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCategoryCountsReflectClassifications(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := newCategoryStore(t)
|
||||
|
||||
msgID, err := s.UpsertMessage(ctx, &Message{GmailID: "1", Subject: "hi"})
|
||||
if err != nil {
|
||||
t.Fatalf("UpsertMessage() error = %v", err)
|
||||
}
|
||||
if _, err := s.SaveClassification(ctx, Classification{MessageID: msgID, Category: "Action"}); err != nil {
|
||||
t.Fatalf("SaveClassification() error = %v", err)
|
||||
}
|
||||
|
||||
cats, err := s.ListCategories(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("ListCategories() error = %v", err)
|
||||
}
|
||||
for _, c := range cats {
|
||||
want := 0
|
||||
if c.Name == "Action" {
|
||||
want = 1
|
||||
}
|
||||
if c.MessageCount != want {
|
||||
t.Errorf("category %q count = %d, want %d", c.Name, c.MessageCount, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRulesSeedAndVersioning(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := newCategoryStore(t)
|
||||
|
||||
text, version, err := s.CurrentRules(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("CurrentRules() error = %v", err)
|
||||
}
|
||||
if version != 1 || text == "" {
|
||||
t.Fatalf("seeded rules = (version %d, %d bytes), want version 1 with text", version, len(text))
|
||||
}
|
||||
|
||||
next, err := s.SaveRules(ctx, `from contains "x" => category=Info`)
|
||||
if err != nil {
|
||||
t.Fatalf("SaveRules() error = %v", err)
|
||||
}
|
||||
if next != 2 {
|
||||
t.Errorf("SaveRules() version = %d, want 2", next)
|
||||
}
|
||||
text, version, err = s.CurrentRules(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("CurrentRules() error = %v", err)
|
||||
}
|
||||
if version != 2 || text != `from contains "x" => category=Info` {
|
||||
t.Errorf("CurrentRules() = (%d, %q), want version 2 with the new text", version, text)
|
||||
}
|
||||
}
|
||||
161
internal/store/classify.go
Normal file
161
internal/store/classify.go
Normal file
@@ -0,0 +1,161 @@
|
||||
// Classification persistence and the rules read/write paths (FR-3, FR-4, FR-11).
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// Classification is one classifier output for a message.
|
||||
type Classification struct {
|
||||
ID int64
|
||||
MessageID int64
|
||||
Category string
|
||||
Importance string
|
||||
Action string
|
||||
Confidence float64
|
||||
Model string
|
||||
}
|
||||
|
||||
// SaveClassification appends a classification row. The triage list reads the
|
||||
// newest row per message, so re-classifying keeps a history.
|
||||
func (s *Store) SaveClassification(ctx context.Context, c Classification) (int64, error) {
|
||||
const q = `INSERT INTO classifications
|
||||
(message_id, category, importance, action, confidence, model)
|
||||
VALUES (?, ?, ?, ?, ?, ?)`
|
||||
res, err := s.db.ExecContext(ctx, q,
|
||||
c.MessageID, c.Category, nullString(c.Importance), nullString(c.Action),
|
||||
c.Confidence, nullString(c.Model))
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("save classification for message %d: %w", c.MessageID, err)
|
||||
}
|
||||
id, err := res.LastInsertId()
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("read classification id: %w", err)
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// PendingClassifications returns messages whose latest classification is
|
||||
// missing or points at a category that no longer exists. Oldest first so a
|
||||
// bounded run makes steady progress.
|
||||
func (s *Store) PendingClassifications(ctx context.Context, limit int) ([]Message, error) {
|
||||
return s.messagesForClassification(ctx, limit, false)
|
||||
}
|
||||
|
||||
// AllMessagesForClassification returns every message, for a full re-classify
|
||||
// after the category list or the rules change.
|
||||
func (s *Store) AllMessagesForClassification(ctx context.Context, limit int) ([]Message, error) {
|
||||
return s.messagesForClassification(ctx, limit, true)
|
||||
}
|
||||
|
||||
func (s *Store) messagesForClassification(ctx context.Context, limit int, all bool) ([]Message, error) {
|
||||
if limit < 1 {
|
||||
limit = 100
|
||||
}
|
||||
filter := "WHERE c.id IS NULL OR c.category NOT IN (SELECT name FROM categories)"
|
||||
if all {
|
||||
filter = ""
|
||||
}
|
||||
q := `
|
||||
SELECT m.id, m.gmail_id, COALESCE(m.thread_id, ''), COALESCE(m.from_addr, ''),
|
||||
COALESCE(m.to_addr, ''), COALESCE(m.subject, ''), COALESCE(m.body_text, ''),
|
||||
COALESCE(m.source_address, ''), COALESCE(m.date_sent, ''), m.received_at,
|
||||
COALESCE(m.imap_uid, 0), COALESCE(m.imap_uidvalidity, 0)
|
||||
FROM messages m
|
||||
LEFT JOIN classifications c ON c.id = (
|
||||
SELECT id FROM classifications WHERE message_id = m.id ORDER BY id DESC LIMIT 1
|
||||
)
|
||||
` + filter + `
|
||||
ORDER BY m.id ASC
|
||||
LIMIT ?`
|
||||
rows, err := s.db.QueryContext(ctx, q, limit)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list messages for classification: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []Message
|
||||
for rows.Next() {
|
||||
var (
|
||||
m Message
|
||||
dateSent string
|
||||
received string
|
||||
)
|
||||
if err := rows.Scan(
|
||||
&m.ID, &m.GmailID, &m.ThreadID, &m.FromAddr, &m.ToAddr, &m.Subject,
|
||||
&m.BodyText, &m.SourceAddress, &dateSent, &received,
|
||||
&m.IMAPUID, &m.IMAPUIDValidity,
|
||||
); err != nil {
|
||||
return nil, fmt.Errorf("scan pending message: %w", err)
|
||||
}
|
||||
m.DateSent = parseSQLTime(dateSent)
|
||||
m.ReceivedAt = parseSQLTime(received)
|
||||
out = append(out, m)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("list messages for classification: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// ClassificationCoverage reports how many messages have a usable classification.
|
||||
type ClassificationCoverage struct {
|
||||
Total int
|
||||
Classified int
|
||||
}
|
||||
|
||||
// Missing reports how many messages still need a classification.
|
||||
func (c ClassificationCoverage) Missing() int { return c.Total - c.Classified }
|
||||
|
||||
// ClassificationCoverage returns the total message count and how many have a
|
||||
// classification whose category still exists.
|
||||
func (s *Store) ClassificationCoverage(ctx context.Context) (ClassificationCoverage, error) {
|
||||
var cov ClassificationCoverage
|
||||
const q = `
|
||||
SELECT
|
||||
(SELECT COUNT(*) FROM messages),
|
||||
(SELECT COUNT(*) FROM messages m
|
||||
WHERE (SELECT cc.category FROM classifications cc
|
||||
WHERE cc.message_id = m.id ORDER BY cc.id DESC LIMIT 1)
|
||||
IN (SELECT name FROM categories))`
|
||||
if err := s.db.QueryRowContext(ctx, q).Scan(&cov.Total, &cov.Classified); err != nil {
|
||||
return cov, fmt.Errorf("classification coverage: %w", err)
|
||||
}
|
||||
return cov, nil
|
||||
}
|
||||
|
||||
// CurrentRules returns the newest rules text and its version. An empty string
|
||||
// and version 0 mean no rules are stored yet.
|
||||
func (s *Store) CurrentRules(ctx context.Context) (string, int, error) {
|
||||
var (
|
||||
text string
|
||||
version int
|
||||
)
|
||||
err := s.db.QueryRowContext(ctx,
|
||||
`SELECT text, version FROM rules ORDER BY version DESC LIMIT 1`).Scan(&text, &version)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return "", 0, nil
|
||||
}
|
||||
if err != nil {
|
||||
return "", 0, fmt.Errorf("read rules: %w", err)
|
||||
}
|
||||
return text, version, nil
|
||||
}
|
||||
|
||||
// SaveRules appends a new rules version and returns the new version number
|
||||
// (FR-11 versioning). The rules editor UI lands in Phase 6.
|
||||
func (s *Store) SaveRules(ctx context.Context, text string) (int, error) {
|
||||
var next int
|
||||
if err := s.db.QueryRowContext(ctx,
|
||||
`SELECT COALESCE(MAX(version), 0) + 1 FROM rules`).Scan(&next); err != nil {
|
||||
return 0, fmt.Errorf("next rules version: %w", err)
|
||||
}
|
||||
if _, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO rules (text, version) VALUES (?, ?)`, text, next); err != nil {
|
||||
return 0, fmt.Errorf("save rules version %d: %w", next, err)
|
||||
}
|
||||
return next, nil
|
||||
}
|
||||
@@ -159,6 +159,15 @@ func (s *Store) LogAction(ctx context.Context, action, actor, detail string) err
|
||||
return nil
|
||||
}
|
||||
|
||||
// LogMessageAction appends a per-message action to the actions_log (FR-13).
|
||||
func (s *Store) LogMessageAction(ctx context.Context, messageID int64, action, actor, detail string) error {
|
||||
const q = `INSERT INTO actions_log (message_id, action, actor, detail) VALUES (?, ?, ?, ?)`
|
||||
if _, err := s.db.ExecContext(ctx, q, messageID, action, nullString(actor), nullString(detail)); err != nil {
|
||||
return fmt.Errorf("log message action %q: %w", action, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const sqlTimeLayout = "2006-01-02 15:04:05"
|
||||
|
||||
func nullString(v string) any {
|
||||
|
||||
@@ -36,6 +36,19 @@ CREATE INDEX IF NOT EXISTS idx_messages_received_at ON messages (received_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_thread_id ON messages (thread_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_source ON messages (source_address);
|
||||
|
||||
-- FR-14: the editable category list. `name` is the stable key stored in
|
||||
-- classifications.category and sent to JEV as the Choice option name. Renaming a
|
||||
-- category cascades to existing classifications (see store.RenameCategory).
|
||||
CREATE TABLE IF NOT EXISTS categories (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
name TEXT NOT NULL COLLATE NOCASE UNIQUE,
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
position INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_categories_position ON categories (position, name);
|
||||
|
||||
-- FR-3, FR-14: classifier output per message.
|
||||
CREATE TABLE IF NOT EXISTS classifications (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
|
||||
@@ -108,6 +108,89 @@ func (s *Store) applyMigrations(ctx context.Context) error {
|
||||
return fmt.Errorf("record schema version 2: %w", err)
|
||||
}
|
||||
}
|
||||
if version < 3 {
|
||||
// FR-14: seed the default category list and a starter (comment-only)
|
||||
// rules document. Seeding happens only inside this migration so that a
|
||||
// category the user removes is not silently re-added on the next start.
|
||||
if err := s.seedCategories(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := s.seedRules(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := s.db.ExecContext(ctx,
|
||||
`INSERT OR IGNORE INTO schema_migrations (version) VALUES (3)`); err != nil {
|
||||
return fmt.Errorf("record schema version 3: %w", err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// defaultCategories are the starter categories from AGENTS.md §13.2. The
|
||||
// descriptions are shown in the UI and sent to JEV as the Choice rubrics.
|
||||
var defaultCategories = []struct{ name, description string }{
|
||||
{"Action", "Needs Sam to do something"},
|
||||
{"Important", "Worth Sam's attention, but no specific task"},
|
||||
{"Info", "Informational, no action required"},
|
||||
{"Spam", "Unwanted, deceptive, or scam mail"},
|
||||
{"Newsletter", "Bulk mail, digests, and marketing"},
|
||||
{"Auto-reply", "Automated notifications and receipts"},
|
||||
}
|
||||
|
||||
// seedCategories inserts the built-in categories when the table is empty.
|
||||
func (s *Store) seedCategories(ctx context.Context) error {
|
||||
var n int
|
||||
if err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM categories`).Scan(&n); err != nil {
|
||||
return fmt.Errorf("count categories: %w", err)
|
||||
}
|
||||
if n > 0 {
|
||||
return nil
|
||||
}
|
||||
for i, c := range defaultCategories {
|
||||
if _, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO categories (name, description, position) VALUES (?, ?, ?)`,
|
||||
c.name, c.description, i+1); err != nil {
|
||||
return fmt.Errorf("seed category %q: %w", c.name, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// starterRules documents the rule syntax (FR-4/FR-11). The editor that writes
|
||||
// real rules lands in Phase 6; the classifier already reads whatever is stored.
|
||||
const starterRules = `# Email Agent rules (FR-4). One rule per line.
|
||||
#
|
||||
# <condition> => <effects>
|
||||
#
|
||||
# condition: <field> <operator> "<value>", joined with " and ".
|
||||
# fields: from, to, subject, source, body
|
||||
# operators: contains, equals, starts_with, ends_with
|
||||
# a single "*" matches every message (use it as the last line for a default).
|
||||
#
|
||||
# effects: comma-separated category=<name>, importance=<low|medium|high>,
|
||||
# action=<delete|unsubscribe|reply|file|keep>
|
||||
#
|
||||
# The first matching rule wins and overrides the JEV classification for the
|
||||
# fields it names. Category names must exist in the category list.
|
||||
#
|
||||
# Examples (remove the leading # to enable):
|
||||
# from contains "amazon.com" => category=Newsletter, action=unsubscribe
|
||||
# subject contains "invoice" => category=Action, importance=high, action=file
|
||||
`
|
||||
|
||||
// seedRules stores the starter rules document when no rules exist yet.
|
||||
func (s *Store) seedRules(ctx context.Context) error {
|
||||
var n int
|
||||
if err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM rules`).Scan(&n); err != nil {
|
||||
return fmt.Errorf("count rules: %w", err)
|
||||
}
|
||||
if n > 0 {
|
||||
return nil
|
||||
}
|
||||
if _, err := s.db.ExecContext(ctx,
|
||||
`INSERT INTO rules (text, version) VALUES (?, ?)`, starterRules, 1); err != nil {
|
||||
return fmt.Errorf("seed rules: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
@@ -326,3 +326,52 @@ a:hover { text-decoration: underline; }
|
||||
|
||||
/* subtle, non-decorative loading pulse (no spinners) */
|
||||
.htmx-request .empty-state { opacity: .55; transition: opacity .15s ease; }
|
||||
.btn.htmx-request { opacity: .55; transition: opacity .15s ease; }
|
||||
|
||||
/* Phase 3: classifier confidence on each row */
|
||||
.msg-confidence { font: var(--type-caption); color: var(--ink-faint); white-space: nowrap; }
|
||||
|
||||
/* Phase 3: editable category list (FR-14) */
|
||||
.nav-group-title { display: flex; align-items: center; justify-content: space-between; gap: var(--space-xs); }
|
||||
.nav-edit { font: var(--type-caption); color: var(--primary); }
|
||||
|
||||
.categories-panel { margin-top: var(--space-md); }
|
||||
.categories-help { font: var(--type-body-sm); margin-bottom: var(--space-md); max-width: 68ch; }
|
||||
|
||||
.category-add {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 2fr auto;
|
||||
align-items: end;
|
||||
gap: var(--space-sm);
|
||||
padding-bottom: var(--space-md);
|
||||
border-bottom: 1px solid var(--hairline);
|
||||
}
|
||||
|
||||
.category-list { display: flex; flex-direction: column; }
|
||||
.category-row {
|
||||
display: flex;
|
||||
align-items: end;
|
||||
gap: var(--space-sm);
|
||||
padding: var(--space-sm) 0;
|
||||
border-bottom: 1px solid var(--hairline);
|
||||
}
|
||||
.category-row:last-child { border-bottom: 0; }
|
||||
.category-rename {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(0, 1fr) minmax(0, 2fr) auto auto;
|
||||
align-items: end;
|
||||
gap: var(--space-sm);
|
||||
flex: 1 1 auto;
|
||||
}
|
||||
.category-desc { min-width: 0; padding-bottom: var(--space-xs); }
|
||||
.category-desc .muted { display: block; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.category-rename .count { white-space: nowrap; padding-bottom: var(--space-xs); }
|
||||
.category-remove { flex: 0 0 auto; }
|
||||
.btn-danger { color: var(--sticker-red); border-color: var(--sticker-red); }
|
||||
|
||||
@media (max-width: 759px) {
|
||||
.category-add { grid-template-columns: 1fr; }
|
||||
.category-row { flex-direction: column; align-items: stretch; }
|
||||
.category-rename { grid-template-columns: 1fr; }
|
||||
.category-remove .btn { width: 100%; }
|
||||
}
|
||||
|
||||
104
web/templates/categories.html
Normal file
104
web/templates/categories.html
Normal file
@@ -0,0 +1,104 @@
|
||||
<!doctype html>
|
||||
<html lang="en" data-theme="light">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Categories · Email Agent</title>
|
||||
<link rel="stylesheet" href="/static/tokens.css">
|
||||
<link rel="stylesheet" href="/static/app.css">
|
||||
<script>
|
||||
(function () {
|
||||
try {
|
||||
var t = localStorage.getItem("ea-theme");
|
||||
if (!t) t = matchMedia("(prefers-color-scheme: dark)").matches ? "dark" : "light";
|
||||
document.documentElement.setAttribute("data-theme", t);
|
||||
} catch (e) {}
|
||||
})();
|
||||
function eaToggleTheme() {
|
||||
var el = document.documentElement;
|
||||
var next = el.getAttribute("data-theme") === "dark" ? "light" : "dark";
|
||||
el.setAttribute("data-theme", next);
|
||||
try { localStorage.setItem("ea-theme", next); } catch (e) {}
|
||||
}
|
||||
</script>
|
||||
</head>
|
||||
<body>
|
||||
<header class="nav-bar">
|
||||
<span class="wordmark">Email Agent</span>
|
||||
<div class="nav-actions">
|
||||
<a class="btn btn-utility" href="/">Back to inbox</a>
|
||||
<button class="btn btn-utility theme-toggle" type="button" onclick="eaToggleTheme()" aria-label="Toggle dark mode">
|
||||
<span class="icon-light" aria-hidden="true">☾</span>
|
||||
<span class="icon-dark" aria-hidden="true">☀</span>
|
||||
</button>
|
||||
<form method="post" action="/logout">
|
||||
<input type="hidden" name="csrf" value="{{.CSRFToken}}">
|
||||
<button class="btn btn-utility" type="submit">Log out</button>
|
||||
</form>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<main class="page">
|
||||
<h1 class="greeting">Categories</h1>
|
||||
|
||||
{{if .Note}}<div class="scan-note" role="status">{{.Note}}</div>{{end}}
|
||||
{{if .Error}}<div class="alert alert-error" role="alert">{{.Error}}</div>{{end}}
|
||||
|
||||
<section class="panel categories-panel" aria-label="Category list">
|
||||
<div class="panel-head">
|
||||
<h2>Category list</h2>
|
||||
<span class="muted">{{.Classified}} of {{.Total}} message(s) classified</span>
|
||||
</div>
|
||||
|
||||
<div class="panel-body">
|
||||
<p class="muted categories-help">
|
||||
JEV classifies each message against this list. Renaming a category updates
|
||||
every message already filed under it. Removing one moves those messages to
|
||||
the first remaining category. At least one category is required.
|
||||
</p>
|
||||
|
||||
<form class="category-add" method="post" action="/categories/add">
|
||||
<input type="hidden" name="csrf" value="{{.CSRFToken}}">
|
||||
<div class="field">
|
||||
<label for="new-name">New category</label>
|
||||
<input class="text-input" id="new-name" name="name" maxlength="40" required
|
||||
placeholder="Finance" autocomplete="off">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="new-description">Description (used by JEV)</label>
|
||||
<input class="text-input" id="new-description" name="description" maxlength="200"
|
||||
placeholder="Bills, statements, and payments" autocomplete="off">
|
||||
</div>
|
||||
<button class="btn btn-primary" type="submit">Add category</button>
|
||||
</form>
|
||||
|
||||
<div class="category-list">
|
||||
{{range .Categories}}
|
||||
<div class="category-row">
|
||||
<form class="category-rename" method="post" action="/categories/rename">
|
||||
<input type="hidden" name="csrf" value="{{$.CSRFToken}}">
|
||||
<input type="hidden" name="id" value="{{.ID}}">
|
||||
<div class="field">
|
||||
<label for="name-{{.ID}}">Name</label>
|
||||
<input class="text-input" id="name-{{.ID}}" name="name" value="{{.Name}}" maxlength="40" required>
|
||||
</div>
|
||||
<div class="category-desc">
|
||||
<span class="muted">{{.Description}}</span>
|
||||
</div>
|
||||
<span class="count pill">{{.MessageCount}} message(s)</span>
|
||||
<button class="btn btn-utility" type="submit">Rename</button>
|
||||
</form>
|
||||
<form method="post" action="/categories/remove" class="category-remove"
|
||||
onsubmit="return confirm('Remove this category? Its messages move to the first remaining category.');">
|
||||
<input type="hidden" name="csrf" value="{{$.CSRFToken}}">
|
||||
<input type="hidden" name="id" value="{{.ID}}">
|
||||
<button class="btn btn-utility btn-danger" type="submit">Remove</button>
|
||||
</form>
|
||||
</div>
|
||||
{{end}}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
@@ -44,7 +44,10 @@
|
||||
<aside class="panel" aria-label="Categories and folders">
|
||||
<div class="nav-groups">
|
||||
<div class="nav-group">
|
||||
<div class="nav-group-title">Categories</div>
|
||||
<div class="nav-group-title">
|
||||
<span>Categories</span>
|
||||
<a class="nav-edit" href="/categories">Edit</a>
|
||||
</div>
|
||||
<div class="nav-group-items">
|
||||
<a class="nav-item nav-item-active" href="#">
|
||||
<span class="dot" style="background: var(--primary)"></span>
|
||||
@@ -53,7 +56,7 @@
|
||||
</a>
|
||||
{{range .Categories}}
|
||||
<a class="nav-item" href="#">
|
||||
<span class="dot" style="background: {{.Color}}"></span>
|
||||
<span class="dot" style="background: {{.Color}}; {{if not .Count}}opacity:.35;{{end}}"></span>
|
||||
<span>{{.Name}}</span>
|
||||
<span class="count">{{.Count}}</span>
|
||||
</a>
|
||||
@@ -84,6 +87,17 @@
|
||||
hx-headers='{"X-CSRF-Token": "{{.CSRFToken}}"}'>
|
||||
Check mail
|
||||
</button>
|
||||
<button class="btn btn-primary" type="button"
|
||||
hx-post="/actions/classify" hx-target="#inbox-list" hx-swap="innerHTML"
|
||||
hx-headers='{"X-CSRF-Token": "{{.CSRFToken}}"}'>
|
||||
Classify
|
||||
</button>
|
||||
<button class="btn btn-utility" type="button"
|
||||
hx-post="/actions/reclassify" hx-target="#inbox-list" hx-swap="innerHTML"
|
||||
hx-confirm="Re-classify every stored message with the current categories and rules?"
|
||||
hx-headers='{"X-CSRF-Token": "{{.CSRFToken}}"}'>
|
||||
Re-classify all
|
||||
</button>
|
||||
<button class="btn btn-utility" type="button"
|
||||
hx-get="/partials/inbox" hx-target="#inbox-list" hx-swap="innerHTML">
|
||||
Refresh
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
</div>
|
||||
<div class="msg-meta">
|
||||
<span class="pill action-pill">{{.Action}}</span>
|
||||
{{if .Classified}}<span class="msg-confidence" title="Classifier confidence">{{.Confidence}}%</span>{{end}}
|
||||
{{if .When}}<span class="msg-when">{{.When}}</span>{{end}}
|
||||
</div>
|
||||
</div>
|
||||
@@ -27,7 +28,7 @@
|
||||
{{else}}
|
||||
<div class="empty-state">
|
||||
<div class="empty-title">No messages yet</div>
|
||||
<p class="empty-copy">Use <strong>Check mail</strong> to fetch from Gmail. New messages appear here, grouped by category, with a recommended action on each row.</p>
|
||||
<p class="empty-copy">Use <strong>Check mail</strong> to fetch from Gmail, then <strong>Classify</strong> to let JEV sort each message.</p>
|
||||
<span class="pill">No stored messages</span>
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
Reference in New Issue
Block a user