email_agent/docs/phase5-replies.md

Phase 5 — Reply drafts (templates, LLM draft, edit, send)

Status: complete. Built, tested, deployed, and the approved real test reply was sent and confirmed to have updated the thread. Sam approved the test send via issue interaction 080c500d-5968-49ad-9c40-0b2d3f445cc0 on AUD-71. Parent AUD-68. Phase 5 of AGENTS.md §11.

DoD: a reply is drafted, edited, and sent, and the thread updates. — met (§2.6).

1. What was built

• Message detail pane (FR-8). Selecting a message opens the body, the recommended action, the reply identity, a template picker, and the draft. It is an HTMX fragment (GET /partials/message) rendered into #message-detail.
• Template-based LLM draft (FR-8). internal/reply calls the OmniRoute gateway's OpenAI-compatible /chat/completions endpoint (the only non-JEV model path, §10). Four built-in templates — quick-ack, answer, confirm, decline — shape the prompt; the action default is answer for a reply recommendation and quick-ack otherwise. Sam can add a free-text note for the model. The generated body is stored in drafts.
• Edit then send. Sam edits the draft in a textarea, can Save draft, then Send reply. Sending is a deliberate UI action: the handler requires a CSRF token, the UI confirmation, and the guard field confirm=yes. There is no auto-send and no scheduled send (§10).
• Reply identity (FR-15). From: is "Sam Rolfe" <address the mail arrived at> (X-Pm-Original-To). When that is unknown it falls back to "Sam Rolfe" <GMAIL_USER>. The SMTP envelope sender is always the authenticated Gmail account. To: is the original sender.
• Threading. The reply sets In-Reply-To and References from the original Message-ID, which Phase 5 now stores (messages.message_id_header, migration v5). After a send the service checks [Gmail]/Sent Mail for the original Gmail thread id and reports whether the thread updated.
• SMTP send, IPv4 forced (§9). internal/mailops sends over smtp.gmail.com:587 with STARTTLS and the app password, dialling tcp4 exactly like the IMAP read path.
• Filing (FR-9). After a successful send the original message moves to its Replied-to/<address> label.
• FR-13 logging. draft, draft_edit, reply_sent, the move to the Replied-to label, and the summary rows are appended to actions_log.
• Duplicate guard. A draft that is already sent cannot be sent again.

New environment variables (all optional; defaults shown): EMAIL_AGENT_SMTP_HOST (smtp.gmail.com), EMAIL_AGENT_SMTP_PORT (587), OMNIROUTE_BASE_URL (http://192.168.20.13:20129/v1), OMNIROUTE_MODEL (auto/best-chat). OMNIROUTE_API_KEY and the Gmail credentials are the only secrets.

2. DoD evidence

2.1 Automated tests

gofmt -l clean, go vet ./... clean, go test ./... green (in golang:1.25-alpine). New coverage:

Package	Tests
internal/reply	chat-completions request shape (auth, x-opencode-session, model, non-streaming); not-configured; HTTP error; template default by action; prompt includes template/instruction/classification/message; fence and Subject: stripping; empty reply rejected
internal/mailops	FR-15 identity (arrived-at vs Gmail fallback); ReplyTo parsing; Re: prefixing; RFC822 headers (From/To/Subject/In-Reply-To/References/Message-ID, CRLF); Send without credentials; dialSMTP4 reaches an IPv4 listener; VerifyThreadUpdated selects [Gmail]/Sent Mail and searches the thread
internal/store	draft round trip (save/edit/mark sent); latest-draft miss; MessageByID returns the Message-ID; migration v5 on a hand-built v4 database adds messages.message_id_header and drafts.sent_at
internal/server	detail pane renders the reply identity and body; detail requires auth; draft requires CSRF; draft is generated and stored; edit saves; send requires confirmation; send calls the sender with the right To/From/Subject/threading headers, marks the draft sent, moves the message to Replied-to, and verifies the thread; already-sent draft refused; missing Message-ID warns

A fake LLM client (an httptest server) and fake sender/generator are used, so the suite needs no network and sends no mail.

2.2 Live, deployed (schema v5, email-agent recreated on .13)

Check	Result
GET /health	200 {"status":"ok","env":"production","database":"ok"}
Container log	database ready … schema_version: 5 (migration v5 ran in place on the Phase 1–4 volume)
docker exec email-agent email-agent -check-smtp	smtp check ok: ipv4 + starttls + auth (no mail sent) — Gmail submission path works over IPv4 with the app password
OmniRoute POST /v1/chat/completions (model auto/best-chat)	200; a probe returned ok
Login over public HTTPS (https://mail.lab.audasmedia.com.au)	303, session + CSRF cookies
POST /actions/draft for message 96	200; OmniRoute produced Got it. and the draft was stored
POST /actions/draft/save	200; the edited body persisted
POST /actions/send without confirm=yes	200; refused with "Sending needs confirmation"; no mail sent
POST /actions/send without a CSRF token	403

The draft → edit path is therefore verified end to end over the public HTTPS URL, and the send path is verified up to (but not including) DATA.

2.3 Gmail "Send mail as" verification (AGENTS.md §13 "Remaining")

AGENTS.md recorded send-as as "reported as already set up in Gmail … not verifiable from the sent folder". Gmail exposes no read-only alias list over IMAP/SMTP, so the check is a real send: Gmail accepts the From header only when the address is a registered send-as alias, and rejects it during DATA otherwise.

Result: verified. The approved test reply was sent as "Sam Rolfe" <hello@where-woof.com> (the X-Pm-Original-To address) from the samuelrolfe@gmail.com account. Gmail accepted it and stored it in the Sent mailbox ([Google Mail]/Sent Mail) with that exact From header, so hello@where-woof.com is a working send-as alias. The SMTP connection, STARTTLS upgrade, and app-password authentication were already verified separately with -check-smtp.

2.4 Draft generated, edited, and sent

Draft 2 for message 99 ("Test email", arrived at hello@where-woof.com), generated through the live UI path, edited, and then sent after Sam's approval:

Thanks, got it. This came through to hello@where-woof.com. Since it's a test, no action needed on my side. If you need anything specific, let me know.

Sam

After the send the draft status is sent, the message left the inbox and now lives under the Replied-to/hello@where-woof.com label (1 row in the Replied-to view), and the message history shows draft, reply_sent, and the move rows (FR-13).

2.5 Thread update after the sent reply

Result: verified — the reply joined the original thread. Read directly from Gmail over IMAP (read-only):

Message	From	To	X-GM-THRID	In-Reply-To / References
Original (99)	sam rolfe <samuelrolfe@gmail.com>	hello@where-woof.com	1878738835102629816	—
Sent reply	"Sam Rolfe" <hello@where-woof.com>	samuelrolfe@gmail.com	1878738835102629816	<CAMAgT7S04G5E=…@mail.gmail.com>

The reply carries the original Message-ID in In-Reply-To and References and Gmail assigned it the same thread id, so the thread updated. The app's own check now agrees:

$ docker exec email-agent email-agent -check-thread 99
message=99 thread=1878738835102629816 sent_reply_in_thread=true

Status code sent_reply_in_thread=true. No third party received the test: the only recipient was samuelrolfe@gmail.com.

2.6 Bug found by the live test and fixed

The first send reported "Could not yet confirm the thread update". The cause was not threading: VerifyThreadUpdated hardcoded Gmail's mailbox as [Gmail]/Sent Mail, but this account exposes the special-use mailboxes as [Google Mail]/…. The fix discovers the mailbox flagged \Sent (RFC 6154) with LIST, falling back to the common names. New coverage: internal/mailfetch (parseList, ListMailboxes, SentMailbox) and internal/mailops (discovered sent mailbox, discovery error). -check-thread <id> was added so the check can be exercised against real Gmail without sending mail. After the fix, -check-thread 99 returns true (above).

3. Configuration

OMNIROUTE_API_KEY and GMAIL_USER/GMAIL_APP_PASSWORD come from ~/.config/environment.d/10-secrets.conf and are passed in by docker-compose. No secret value is committed. The other Phase 5 settings have working defaults.

4. Deployment

On .13, from the repo, with the host secrets in scope:

set -a; . ~/.config/environment.d/10-secrets.conf; set +a
docker compose build
docker compose up -d          # migration v5 runs on start; data volume kept
docker exec email-agent email-agent -check-smtp   # safe: no mail sent

Two commits were needed: the Phase 5 implementation and the \Sent mailbox discovery fix from §2.6.

5. Rollback

1. git revert <sent-mailbox-fix commit> <phase-5 commit> and docker compose up -d --build.
2. Migration v5 is additive (messages.message_id_header, drafts.sent_at); the Phase 4 binary ignores the new columns.
3. A sent reply cannot be unsent. Gmail's Sent copy and the thread stay; the Replied-to/<address> label can be removed in Gmail by hand.

6. Not done / deferred

• A template editor (user-defined templates) is not in the Phase 5 requirements; templates are built in. The Phase 6 rules editor still feeds the classifier.
• 1-month body retention (§7) is still not implemented.
• Rich-text or HTML replies are out of scope; replies are plain text.