From 449fdd02c584310010a6c1892074d913d3d25293 Mon Sep 17 00:00:00 2001 From: Sam Rolfe Date: Sat, 10 Oct 2026 17:06:43 +1100 Subject: [PATCH] Freeze Hub inbound API v1; choose Go for the service - docs/hub-api.md: language decision (Go 1.25) with reason and the frozen v1 routes POST /ask, POST /approve, GET /health, including request/response schemas, bearer-token auth, timeouts, error shapes, logging, versioning, and a safety confirmation that no route is a high-consequence control. - Update README, hub_brief.md open items, and the outline docs so they no longer say the API is undecided or the language is open. Co-Authored-By: Paperclip --- README.md | 3 +- docs/hub-api.md | 510 ++++++++++++++++++++++++++++++ docs/outline/00-index.md | 3 +- docs/outline/02-config-and-env.md | 7 +- docs/outline/03-code-map.md | 8 +- hub_brief.md | 3 + 6 files changed, 528 insertions(+), 6 deletions(-) create mode 100644 docs/hub-api.md diff --git a/README.md b/README.md index 6b7bcf7..bea14b9 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ shared by the phone app, the house voice, chat, and the admin app. ## Demonstrated Skills & Tech Stack -- **Core Technologies**: Go or Python service · HTTP JSON API · MQTT · LLM +- **Core Technologies**: Go 1.25 service · HTTP JSON API · MQTT · LLM intent routing · OmniRoute gateway · Docker - **Key Tools**: Jervis (voice) · Home Assistant · Paperclip · Artemis · internet use control · Apprise / NTFY · OmniRoute @@ -82,5 +82,6 @@ flowchart LR The design brief is `hub_brief.md`. Project instructions are `AGENTS.md`. The interface between the Hub and every data project is `docs/ask-contract.md`. +The frozen inbound API (ways in to the Hub) is `docs/hub-api.md`. Every other project is a **peer**, not a sub-project. A high-consequence control (for example, cutting the kids' internet) does **not** route through the Hub. diff --git a/docs/hub-api.md b/docs/hub-api.md new file mode 100644 index 0000000..562c784 --- /dev/null +++ b/docs/hub-api.md @@ -0,0 +1,510 @@ +# The Hub Inbound API (v1) + +**Purpose:** one frozen interface from every way in to the Hub. +**Applies to:** phone app, ESP32 voice, dsh chat, admin app, console portal. +**Status:** frozen. Build the Hub server and every client to this document. +**Decided:** 2026-10-10. + +`hub_brief.md` is the source of truth for the design. This document freezes the +wire format. If this document and the brief disagree, the brief wins. + +--- + +## 1. Service language (decided) + +**Go 1.25.** + +**Reason (one line):** The Hub is a small, always-on gateway on the fast path, so +a single static Go binary gives hard timeouts, no runtime dependency drift, +instant start, and a tiny container, and nothing in the Hub needs the Python +data or ML stack because every model call goes through OmniRoute over HTTP. + +Build notes: + +- Standard library `net/http` with `context` deadlines. Small dependency list. +- Multi-stage Docker build. Run on a static or distroless base image. +- Deploy as one container on `192.168.20.13` (the agreed host). +- One binary, one port, one folder. Do not merge a destination into the Hub. + +Go 1.25 is available from nixpkgs (a derivation is already in the local store). +Docker 29 runs on the build host. + +## 2. Two different `/ask` endpoints + +The name `/ask` appears twice. They are different services on different ports. +Do not confuse them. + +| Endpoint | Caller | Server | Defined in | +|---|---|---|---| +| `POST /ask` | A way in (phone, voice, chat, admin) | **The Hub** | This document | +| `POST /ask` | The Hub | A data project (email, school, internet control) | `docs/ask-contract.md` | + +This document freezes only the **Hub** endpoint. The Hub sends the project +endpoint with the shape in `docs/ask-contract.md`. + +## 3. Transport and authentication + +### Base + +- Base URL: `http://192.168.20.13:8787` (port 8787 is the proposed default, + `HUB_PORT`). Confirm the port against the house port list before deploy. +- Plain HTTP on the private LAN for v1. Put a TLS reverse proxy in front later. +- Request and response body: JSON, `application/json; charset=utf-8`. +- Maximum request body: 64 KiB. Larger bodies get `413`. + +### Authentication + +Every request carries a bearer token, except `GET /health`: + +``` +Authorization: Bearer +``` + +- One token per **client**, not per person. Store each token in + `~/.config/environment.d/10-secrets.conf`. +- The Hub holds the accepted tokens. It compares them in constant time. +- The token names the device or channel. The token does **not** name the person. + +| Env key | Client | +|---|---| +| `HUB_TOKEN_PHONE` | Phone app. | +| `HUB_TOKEN_ESP32` | House voice (ESP32, "Hey Jervis"). | +| `HUB_TOKEN_DSH` | dsh chat. | +| `HUB_TOKEN_CONSOLE` | Console portal. | +| `HUB_TOKEN_ADMIN` | Admin app. The only trusted door. | + +### Auth matrix + +| Route | Accepted tokens | +|---|---| +| `POST /ask` | Any client token. | +| `POST /approve` | `HUB_TOKEN_ADMIN` only. | +| `GET /health` | None. | + +### Voice is shared, not trusted + +Voice is open to everyone in the house. The ESP32 token proves the **device**, +not the speaker. The Hub never treats a voice request as proof of identity. A +restricted action always needs the admin app. + +### One identity + +There is one identity: Sam. The `asker` field must be `"sam"`. Any other value +gets `422 unsupported_asker`. Keep the field so a second person can be added +later. + +## 4. Common response rules + +### Success and partial success + +A well-formed, authorized, processed request returns HTTP `200`, even when the +Hub cannot route the request. The top-level `status` field says what happened. + +| `status` | Meaning | +|---|---| +| `ok` | A reply is ready. | +| `needs_clarification` | The Hub needs one short answer. `answer` holds the question. | +| `needs_approval` | A Hub-owned approval is required. `approval` holds the id. | +| `no_match` | The Hub cannot route the request. | +| `refused` | The request is a high-consequence control. The Hub does not do it. | +| `unavailable` | A destination or dependency is down. | + +### Error envelope + +An error uses a non-200 status and this shape: + +```json +{ + "status": "error", + "error": { + "code": "bad_request", + "message": "Field 'text' is required.", + "request_id": "01J8Z3K4M5N6P7Q8R9S0T1V2W3" + } +} +``` + +`message` is one short sentence. It is safe to speak aloud. It holds no secret, +no internal path, and no stack trace. + +### HTTP status and error codes + +| HTTP | `code` | Use | +|---|---|---| +| 400 | `bad_request` | Bad JSON, missing field, or bad enum value. | +| 401 | `unauthorized` | Missing or invalid bearer token. | +| 403 | `forbidden` | Valid token, wrong route. For example, a non-admin calls `/approve`. | +| 403 | `source_mismatch` | The `source` field does not match the token. | +| 404 | `not_found` | Unknown path, or unknown `approval_id`. | +| 409 | `not_hub_owned` | The approval belongs to another service. | +| 409 | `already_decided` | The approval was already decided. | +| 409 | `approval_expired` | The approval passed its `expires` time. | +| 413 | `payload_too_large` | Body over 64 KiB. | +| 415 | `unsupported_media_type` | Body is not `application/json`. | +| 422 | `unprocessable` | Valid JSON, bad content. For example, empty `text`. | +| 422 | `unsupported_asker` | `asker` is not `"sam"`. | +| 429 | `rate_limited` | Too many requests. See `Retry-After`. | +| 500 | `internal` | Unexpected Hub fault. | +| 502 | `destination_error` | A destination failed to answer. | +| 503 | `unavailable` | A dependency is down (OmniRoute, MQTT). | +| 504 | `timeout` | A deadline passed. | + +### Rate limit + +Cap each token at 30 requests per minute. On breach, return `429` with a +`Retry-After` header in seconds. This protects the Hub. It is not a security +control. + +### Versioning + +Add fields. Do not remove or rename a field. Clients must ignore unknown fields. +A breaking change needs a new path, for example `/ask/v2`, `/approve/v2`, or +`/health/v2`. + +--- + +## 5. `POST /ask` + +A way in sends words to the Hub. The Hub runs the five steps: hear, understand, +check, route, answer. A way in needs only this route to ask. + +### Request + +```json +{ + "text": "what does Harry have on today?", + "asker": "sam", + "source": "esp32", + "person": "harry", + "want": "speech", + "device": "kitchen-esp32", + "request_id": "01J8Z3K4M5N6P7Q8R9S0T1V2W3" +} +``` + +| Field | Required | Type | Meaning | +|---|---|---|---| +| `text` | yes | string | The words the user said, or typed. 1 to 1000 characters. | +| `asker` | yes | string | Who asked. Must be `"sam"` in v1. | +| `source` | yes | enum | Where it came from. See below. | +| `person` | no | string | Who the request is about. Defaults to `asker`. | +| `want` | no | enum | Reply shape the caller prefers: `text`, `speech`, `both`. Default comes from `source`. | +| `device` | no | string | Device name, for speech routing. | +| `request_id` | no | string | Client id. The Hub makes one when absent. | + +`source` values: `esp32`, `phone`, `chat`, `console`, `admin`. + +The token must belong to the `source`. For example, `HUB_TOKEN_DSH` cannot claim +`source: "admin"`. A mismatch gets `403 source_mismatch`. + +### Response — a normal reply + +```json +{ + "status": "ok", + "request_id": "01J8Z3K4M5N6P7Q8R9S0T1V2W3", + "answer": "Harry has maths and a music lesson today.", + "shape": "speech", + "speech": { "text": "Harry has maths and a music lesson today.", "target": "house" }, + "route": { "kind": "project", "project": "compass_nhs" }, + "approval": null, + "clarify": null, + "expires": "2026-10-10T09:00:00Z" +} +``` + +| Field | Required | Meaning | +|---|---|---| +| `status` | yes | See the table in section 4. | +| `request_id` | yes | Echo of the request id, or a new one. | +| `answer` | yes | One short sentence. Safe to speak aloud. | +| `shape` | yes | `text`, `speech`, `notification`, or `none`. | +| `speech` | no | TTS detail. Present when `shape` is `speech`. | +| `route` | no | What the Hub did. For logging and debug. | +| `approval` | no | Present when `status` is `needs_approval`. | +| `clarify` | no | Present when `status` is `needs_clarification`. | +| `expires` | no | When the answer stops being true. | + +`shape` values: + +- `text` — text only. Chat and console. +- `speech` — the Hub also speaks the reply. +- `notification` — a phone or desktop notification. +- `none` — no reply body. The action ran. + +`speech.target` values: `house` (house speakers), `phone` (the asking phone), +`none`. + +`route.kind` values: `direct` (Home Assistant, speakers), `project` (a data +project through `/ask`), `fanout` (several projects), `paperclip`, `clarify`, +`no_match`, `refused`. + +### Response — the Hub needs a short answer + +```json +{ + "status": "needs_clarification", + "request_id": "01J8Z3K4M5N6P7Q8R9S0T1V2W3", + "answer": "School or email?", + "shape": "speech", + "speech": { "text": "School or email?", "target": "house" }, + "route": { "kind": "clarify" }, + "clarify": { "question": "School or email?", "options": ["school", "email"] } +} +``` + +### Response — a Hub-owned approval is required + +A slow, recorded action (a Paperclip job) needs a yes. The Hub does not run it +yet. It returns an approval id. + +```json +{ + "status": "needs_approval", + "request_id": "01J8Z3K4M5N6P7Q8R9S0T1V2W3", + "answer": "I need a yes on the admin app to file that job.", + "shape": "speech", + "speech": { "text": "I need a yes on the admin app to file that job.", "target": "house" }, + "route": { "kind": "paperclip" }, + "approval": { + "approval_id": "ap_01J8Z3K4M5N6P7Q8R9S0T1V2W3", + "action": "paperclip_job", + "summary": "File a job: plan the kitchen lights.", + "reason": "A slow, multi-step job needs a yes.", + "expires": "2026-10-10T09:10:00Z" + } +} +``` + +`approval.action` values in v1: `paperclip_job`. Add values later. Never remove +one. + +### Response — a high-consequence control + +The Hub does not route a control action. It refuses and points at the admin app. + +```json +{ + "status": "refused", + "request_id": "01J8Z3K4M5N6P7Q8R9S0T1V2W3", + "answer": "Use the admin app to change internet limits.", + "shape": "speech", + "speech": { "text": "Use the admin app to change internet limits.", "target": "house" }, + "route": { "kind": "refused", "reason": "high_consequence_control" } +} +``` + +### Response — no match + +```json +{ + "status": "no_match", + "request_id": "01J8Z3K4M5N6P7Q8R9S0T1V2W3", + "answer": "I do not know where to send that.", + "shape": "text", + "route": { "kind": "no_match" } +} +``` + +### Behaviour rules + +1. The Hub never parses a project's data. It forwards the question. +2. The Hub never changes the meaning of an answer. It only changes the shape. +3. The Hub never places an LLM in the path of a fast or consequential action + (brief rule 4). Routing uses an LLM. The action call does not. +4. One project down is not an outage. Return `status: "unavailable"` and say so. +5. Every request is logged. See section 10. + +## 6. `POST /approve` + +The admin app answers "Did you ask for this?". This route records one yes or no +for a **Hub-owned** approval and then runs the approved action. + +In v1 the only Hub-owned approval is a Paperclip job. The Hub files the job once, +then returns the result. + +### Request + +```json +{ + "approval_id": "ap_01J8Z3K4M5N6P7Q8R9S0T1V2W3", + "decision": "approve", + "asker": "sam", + "device": "sam-phone" +} +``` + +| Field | Required | Type | Meaning | +|---|---|---|---| +| `approval_id` | yes | string | The id from the `/ask` response. | +| `decision` | yes | enum | `approve` or `deny`. | +| `asker` | yes | string | Who decided. Must be `"sam"` in v1. | +| `device` | no | string | The admin device. | + +### Response + +```json +{ + "status": "ok", + "approval_id": "ap_01J8Z3K4M5N6P7Q8R9S0T1V2W3", + "decision": "approve", + "result": { + "kind": "paperclip_job", + "status": "filed", + "detail": "Issue filed." + }, + "answer": "Filed the job." +} +``` + +`result.status` values: `filed`, `denied`, `failed`. + +### Rules + +1. An approval is single use. A second decision gets `409 already_decided`. +2. An unknown id gets `404 not_found`. +3. An id owned by another service gets `409 not_hub_owned`. The admin app must + call that service directly. +4. An expired approval gets `409 approval_expired`. The user must ask again. +5. Only `HUB_TOKEN_ADMIN` may call this route. + +### Parental controls are not here + +A parental control change is a high-consequence control. The Hub does **not** +execute it (brief rule 11). The admin app shows the confirmation and calls the +internet control service directly. This route is not in that path. + +## 7. `GET /health` + +Liveness only. No authentication. + +### Response — healthy + +```json +{ + "ok": true, + "service": "hub", + "version": "1", + "uptime_s": 4321, + "checks": { + "omniroute": "ok", + "mqtt": "ok", + "log": "ok" + } +} +``` + +### Response — degraded + +```json +{ + "ok": false, + "service": "hub", + "version": "1", + "uptime_s": 4321, + "checks": { + "omniroute": "ok", + "mqtt": "down", + "log": "ok" + } +} +``` + +HTTP `200` when `ok` is true. HTTP `503` when `ok` is false. + +Each `checks` value is `ok`, `down`, or `unknown`. A check never blocks the +health reply for longer than 1 second. The health reply holds no secret and no +personal data. + +## 8. Timeouts + +Every timeout is a hard cap. The Hub cancels the work when the cap passes. + +| Step | Cap | +|---|---| +| Read the request body | 1 s | +| Routing LLM call (OmniRoute) | 3 s | +| One project `/ask` call | 3 s | +| Fanout (all projects, in parallel) | 4 s | +| One destination command (Home Assistant, MQTT) | 2 s | +| `POST /ask` total | 8 s | +| `POST /approve` total | 5 s | +| `GET /health` total | 1 s | + +House TTS is out of band. The Hub starts it and does not wait for the speakers. +Phone TTS is part of the reply. + +Data project rules stay as written in `docs/ask-contract.md`: a project answers +in under 3 seconds, or it returns `answer: null`. + +## 9. Reply shape and TTS target + +The `want` field is a preference. The Hub picks the final shape. It uses `source` +and `want`: + +| `source` | Default `shape` | Default `speech.target` | +|---|---|---| +| `esp32` | `speech` | `house` | +| `phone` | `speech` | `phone` | +| `chat` | `text` | `none` | +| `console` | `text` | `none` | +| `admin` | `text` | `none` | + +`want: "both"` returns text and adds speech. `want: "text"` never speaks. +A spoken reply is always one short sentence. + +This settles the open question "when does phone TTS get used?": the Hub speaks on +the asking phone when the source is the phone, and on the house speakers when the +source is the house voice. The caller may override with `want`. Artemis +transport (stream or file) is still open. See `hub_brief.md`. + +## 10. Logging + +The Hub writes one append-only log (brief rule 13). + +One line per request. One line holds: + +| Field | Example | +|---|---| +| `time` | `2026-10-10T08:55:03Z` | +| `request_id` | `01J8Z3K4M5N6P7Q8R9S0T1V2W3` | +| `source` | `esp32` | +| `asker` | `sam` | +| `text` | `what does Harry have on today?` | +| `route` | `project:compass_nhs` | +| `status` | `ok` | +| `latency_ms` | `412` | +| `shape` | `speech` | + +Rules: + +- Append only. Never edit a line. +- Cap the size. Rotate at a fixed size. Keep a fixed number of days. +- No secrets. Never log a token. +- No full personal data. Log the question and the route, not a project's data. +- Never log the `Authorization` header. + +## 11. Safety confirmation + +The three frozen routes are not high-consequence control actions. + +| Route | High-consequence control? | Why | +|---|---|---| +| `POST /ask` | No | Routes a query or a fast command. Refuses a control action with `status: "refused"`. | +| `POST /approve` | No | Decides a Hub-owned approval only. Files a Paperclip job. Runs no parental control. | +| `GET /health` | No | Read only. | + +A parental control change goes: admin app → internet control service, directly. +The Hub is not in that path. See brief rules 7, 8, and 11. + +## 12. Not in v1 + +`POST /notify` (a project tells the Hub) is part of the design. Its shape is +defined in `docs/ask-contract.md`. It is not one of the three frozen Hub-inbound +routes in this document. Freeze it in a later task. + +`routing.md` (the human-edited routing rules) is still open. `POST /ask` depends +on it. See `hub_brief.md`, "What is still open", item 5. + +The Artemis transport (stream or file) is still open. See `hub_brief.md`, +"What is still open", item 3. diff --git a/docs/outline/00-index.md b/docs/outline/00-index.md index a4bfc05..49c628b 100644 --- a/docs/outline/00-index.md +++ b/docs/outline/00-index.md @@ -14,6 +14,7 @@ Read this index to inject the exact file you need. Do not search all files. | **Code Structure Map** | `docs/outline/03-code-map.md` | Shows the planned folder layout and entry points. | | **AI Agent Runbook** | `docs/outline/04-agent-runbook.md` | Gives steps to build, test, and restart Hub work. | | **Ask Contract** | `docs/ask-contract.md` | The interface between the Hub and every data project. | +| **Hub Inbound API** | `docs/hub-api.md` | The frozen v1 API from every way in to the Hub. | | **Design Brief** | `hub_brief.md` | The agreed design. This file is the source of truth. | | **Project Rules** | `AGENTS.md` | Mission, rules, build order, and project relations. | @@ -29,7 +30,7 @@ hold a personality or a memory. The Hub does not do the destination work. - **Env File**: `~/.config/environment.d/10-secrets.conf` - **Config Directory**: `docs/` -- **Active Ports**: Hub port not allocated yet. +- **Active Ports**: Hub 8787 (`HUB_PORT`, proposed default — confirm before deploy). ## Agent Restart Steps diff --git a/docs/outline/02-config-and-env.md b/docs/outline/02-config-and-env.md index f792cd7..e7b0e5f 100644 --- a/docs/outline/02-config-and-env.md +++ b/docs/outline/02-config-and-env.md @@ -17,6 +17,11 @@ Relevant keys: | `OL_API_KEY` | Outline docs. | | `VIKUNJA_TOKEN` | Vikunja tasks. | | `GITEA_TOKEN` | Gitea repository. | +| `HUB_TOKEN_PHONE` | Hub token for the phone app. | +| `HUB_TOKEN_ESP32` | Hub token for the house voice. | +| `HUB_TOKEN_DSH` | Hub token for dsh chat. | +| `HUB_TOKEN_CONSOLE` | Hub token for the console portal. | +| `HUB_TOKEN_ADMIN` | Hub token for the admin app. The only trusted door. | ## Services and Ports @@ -28,7 +33,7 @@ Relevant keys: | Home Assistant | 192.168.20.30 | 8123 | Devices and timers. | | MQTT | 192.168.20.13 | 1883 | Voice topics. | | Apprise | 192.168.20.35 | 8210 | Notifications. | -| Hub | not allocated | — | This service. | +| Hub | 192.168.20.13 | 8787 | This service. Proposed default (`HUB_PORT`). | ## Config Paths diff --git a/docs/outline/03-code-map.md b/docs/outline/03-code-map.md index de94862..20b2eec 100644 --- a/docs/outline/03-code-map.md +++ b/docs/outline/03-code-map.md @@ -16,9 +16,11 @@ central_ai_llm_master/ diagrams/ rendered images archify/ Archify JSON input outline/ AI context documents + ask-contract.md Hub to project interface + hub-api.md frozen inbound API (v1) vikunja-tasks.json human task list src/ Hub service code (to build) - main.* HTTP entry point + main.go HTTP entry point hear/ takes voice or text understand/ LLM intent routing check/ authorization and approval @@ -36,11 +38,11 @@ central_ai_llm_master/ | `POST /approve` | Admin app confirms a restricted action. | | `GET /health` | Liveness check. | -The API contract is not final. See `hub_brief.md`. +The inbound API is frozen. See `docs/hub-api.md`. ## Primary Dependencies -- HTTP server (Go or Python — decide before build). +- HTTP server (Go 1.25, standard library `net/http`). - HTTP client for OmniRoute, Home Assistant, Paperclip. - MQTT client for voice topics. - Tavily API client for web search. diff --git a/hub_brief.md b/hub_brief.md index f5f984a..dcf9717 100644 --- a/hub_brief.md +++ b/hub_brief.md @@ -221,8 +221,11 @@ Each project is a **peer**. These are talk directions, not ownership. ## What is still open 1. **Service tokens.** One bearer token per project, stored in `environment.d`. + The **scheme** is frozen in `docs/hub-api.md`. Only the token values remain. 2. **Job contract.** Exact fields for a Paperclip job (title, brief, project, agent). 3. **Artemis voice.** Stream or file. When does phone TTS get used? + The **phone TTS rule** is decided in `docs/hub-api.md`, section 9. + The Artemis transport is still open. 4. **Paperclip layout.** One company or several. Which projects. 5. **Routing rules.** The first draft of `routing.md`.