Phase 9: publish the documentation set (README, map, Outline, Vikunja)

- README: Phase 9 status, the architecture diagram + map link, and the explicit "Teams (FR-15) is DEFERRED" statement. - docs/architecture.mmd + docs/diagrams/architecture.png (mmdc). - docs/archify/architecture.architecture.json -> docs/architecture-map.html (Archify showcase, 9/9 checks). - docs/outline/: the five AI-context documents (STE style). - docs/vikunja-tasks.json: the live Vikunja board (project 21). Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-15 19:58:55 +11:00
parent 06f759c37b
commit b4bbd6d3f9
11 changed files with 14534 additions and 6 deletions
--- a/docs/outline/00-index.md
+++ b/docs/outline/00-index.md
@@ -0,0 +1,43 @@
+# Compass NHS — AI Document Index
+
+> **Target**: AI Agents
+> **Writing Style**: ASD-STE100 Simplified Technical English.
+
+## Document Catalog (Inject Specific Document)
+
+Read this index first. Inject the exact file you need. Do not search every file.
+
+| Document Title | Path | Scope |
+| :--- | :--- | :--- |
+| **System Overview** | `docs/outline/01-system-overview.md` | States the purpose, the main parts and the live services. |
+| **Configuration and Environment** | `docs/outline/02-config-and-env.md` | Lists variables, paths, ports and secrets. |
+| **Code Map** | `docs/outline/03-code-map.md` | Shows the folder layout, entry points and dependencies. |
+| **Agent Runbook** | `docs/outline/04-agent-runbook.md` | Gives the commands to build, test, deploy and restart. |
+
+## System Summary
+
+This project reads the school systems for two children, Harry and Finn. It stores
+what is due, finds overdue work, writes important events to the family calendar,
+and sends morning briefs.
+
+The service is one Go binary with SQLite. It runs in its own container on `.13`,
+port 8792, behind Caddy on `.35` at `school.lab.audasmedia.com.au`.
+
+**Teams (FR-15) is DEFERRED.** The project does not scan Teams. Compass answers
+what is due and overdue.
+
+## Core Configuration
+
+- **Env File**: `environment.d/school-service.env` (gitignored)
+- **Env Example**: `environment.d/school-service.env.example`
+- **Config Code**: `internal/config/`
+- **Active Ports**: `8792` (service), `8210` (Apprise on `.35`), `8788` (Hub)
+- **Database**: SQLite in the Docker volume `compass-teams-nhs-data`
+
+## Agent Restart Steps
+
+1. Read the Document Catalog above. Inject only the required files.
+2. Check the checkout: `git status`.
+3. Read the recent history: `git log -n 5 --oneline`.
+4. Run the checks: `make test` and `make verify-phase8`.
+5. Read the current state: `curl -s http://127.0.0.1:8792/health`.
--- a/docs/outline/01-system-overview.md
+++ b/docs/outline/01-system-overview.md
@@ -0,0 +1,70 @@
+# Compass NHS — System Overview
+
+## Purpose
+
+The service reads the school systems for Harry and Finn. It answers three
+questions: what is due, what is overdue, and what important events are coming.
+
+Sam sees both children on one page. The family receives the briefs before work is
+late. Nobody logs into Compass each morning.
+
+## Users
+
+| User | Role |
+| :--- | :--- |
+| Sam | Owner. Uses the dashboard. |
+| Jo, Harry, Finn | Read-only. Receive the briefs. |
+
+## Main Parts
+
+| Part | Function |
+| :--- | :--- |
+| Go service | Serves the dashboard and the Hub API. Holds the state. |
+| SQLite store | Keeps children, items, scans, notifications and calendar pushes. |
+| patchright scraper | Logs into Compass and fetches the pages. |
+| n8n workflow | Runs the scan and the sends at 06:00 daily. |
+| Apprise | Sends the briefs on Email and WhatsApp. |
+| Home Assistant | Holds the calendar `calendar.school`. |
+| Hub | Calls the read-only `/ask` endpoint. |
+
+## The Data Flow
+
+1. n8n triggers `scripts/morning-brief.sh` at 06:00.
+2. The script scans Compass for each child and stores the raw pages.
+3. The parser turns the pages into items: tasks, messages and events.
+4. The classifier marks important events. OmniRoute helps only here.
+5. The brief builder makes one brief per boy and one combined brief.
+6. Apprise sends the briefs on Email and WhatsApp.
+7. The calendar sync writes today's and future important events to Home Assistant.
+
+## Read-Only Rule
+
+The service never writes to the school. It logs in and fetches pages only. The
+scraper never posts, messages or submits anything.
+
+## Fail Loudly
+
+A missing scan or a failed scan is never an empty day. The dashboard, the briefs
+and `/ask` each state the real scan state. A failed scan says so in plain words.
+
+## Teams
+
+**Teams (FR-15) is DEFERRED.** The project does not scan Teams. This is a known
+gap, not a defect. Compass gives the schedule, the tasks and the due dates.
+
+## Live Targets
+
+| System | Address |
+| :--- | :--- |
+| Dashboard | `https://school.lab.audasmedia.com.au` |
+| Service container | `.13`, port 8792 |
+| Health | `http://127.0.0.1:8792/health` |
+| Apprise API | `.35`, port 8210 |
+| Hub inbound | `http://192.168.20.13:8788/ask` |
+
+## Repository
+
+- Git: `ssh://git@gitea.lab.audasmedia.com.au:2222/sam/compass_teams_nhs.git`
+- Branch: `main`
+- Spec: `AGENTS.md`
+- Interactive map: `https://maps.lab.audasmedia.com.au/compass_teams_nhs/docs/`
--- a/docs/outline/02-config-and-env.md
+++ b/docs/outline/02-config-and-env.md
@@ -0,0 +1,101 @@
+# Compass NHS — Configuration and Environment
+
+## Rules for Agents
+
+1. Never write a credential, token, cookie or session value into a document.
+2. Keep every secret in `environment.d/school-service.env`. The file is gitignored.
+3. Copy the shape from `environment.d/school-service.env.example`. Never copy a value.
+4. Docker Compose interpolates `$VAR` inside `env_file`. Escape each bcrypt `$` as `$$`.
+
+## Paths
+
+| Item | Path |
+| :--- | :--- |
+| Real env file | `environment.d/school-service.env` |
+| Env example | `environment.d/school-service.env.example` |
+| Config code | `internal/config/` |
+| SQLite schema | `internal/store/migrations/` |
+| Service binary | `/app/school` inside the container |
+| Database | `/data/school.db` inside the container |
+| Docker volume | `compass-teams-nhs-data` |
+
+## Ports
+
+| Port | System | Host |
+| :--- | :--- | :--- |
+| 8792 | This service | `.13` |
+| 8210 | Apprise API | `.35` |
+| 8788 | Hub inbound API | `.13` |
+| 8123 | Home Assistant | `.30` |
+| 5678 | n8n | `.13` |
+
+Caddy on `.35` publishes `school.lab.audasmedia.com.au` and proxies to
+`192.168.20.13:8792`.
+
+## Variables
+
+### Service
+
+| Variable | Meaning |
+| :--- | :--- |
+| `SCHOOL_ADDR` | Listen address. Default `:8792`. |
+| `SCHOOL_DB_PATH` | SQLite path. Container value `/data/school.db`. |
+| `SCHOOL_SESSION_SECRET` | Signs the session cookie. Secret. Minimum 16 characters. |
+| `SCHOOL_ADMIN_USERNAME` | The single dashboard login. |
+| `SCHOOL_ADMIN_PASSWORD_HASH` | Bcrypt hash. Prefer this over the plaintext. |
+| `SCHOOL_ADMIN_PASSWORD` | Plaintext fallback. |
+| `SCHOOL_COOKIE_SECURE` | `true` behind HTTPS. |
+| `SCHOOL_SESSION_TTL` | Session life. Default `12h`. |
+| `SCHOOL_TIMEZONE` | Family timezone for due dates. |
+| `SCHOOL_LOG_LEVEL` | Log level. |
+| `SCHOOL_APP_ENV` | Environment name. |
+
+### Home Assistant Calendar
+
+| Variable | Meaning |
+| :--- | :--- |
+| `SCHOOL_HA_URL` | Home Assistant base URL. |
+| `SCHOOL_HA_TOKEN` | Long-lived access token. Secret. |
+| `SCHOOL_HA_CALENDAR_ENTITY` | Calendar entity. Default `calendar.school`. |
+| `SCHOOL_HA_INSECURE` | Skips TLS verification. Use only for the internal `.lab` URL. |
+| `SCHOOL_HA_TIMEOUT` | Per-request timeout. Default `20s`. |
+
+### Apprise
+
+| Variable | Meaning |
+| :--- | :--- |
+| `SCHOOL_APPRISE_URL` | Apprise API base. Default `http://192.168.20.35:8210`. |
+| `SCHOOL_APPRISE_KEY` | Stored configuration key. Secret. |
+| `SCHOOL_APPRISE_TOKEN` | Optional bearer token. |
+| `SCHOOL_APPRISE_TIMEOUT` | Per-request timeout. Default `15s`. |
+| `SCHOOL_APPRISE_CHANNELS` | `email,whatsapp`. |
+| `SCHOOL_APPRISE_TAGS` | Recipients as `Name=tag-base`. |
+| `SCHOOL_APPRISE_PARENTS` | Who receives the combined brief. `Jo,Sam`. |
+| `SCHOOL_NOTIFY_TOKEN` | Bearer token for `POST /notify/morning`. Secret. |
+
+### Hub API
+
+| Variable | Meaning |
+| :--- | :--- |
+| `SCHOOL_HUB_TOKEN` | Shared bearer token. Secret. Empty turns the Hub endpoints off with `503`. |
+
+The Hub holds the same token as `HUB_PROJECT_COMPASS_NHS_TOKEN`. The Hub reaches
+this service through `HUB_PROJECTS=compass_nhs=http://192.168.20.13:8792`.
+
+## Secrets Outside This Repo
+
+| Secret | Location |
+| :--- | :--- |
+| Apprise destinations (address, number, SMTP) | Apprise stored configuration on `.35` |
+| Hub token copy | `~/.config/environment.d/10-secrets.conf` on `.13` |
+| Compass account and password | Paperclip secret or the env file |
+| n8n trigger SSH key | `/home/sam/.ssh/id_ed25519_school_scan` |
+
+The Apprise destinations stay outside this repo on purpose. This service names a
+tag such as `harry_email` only.
+
+## Environment Access for an Agent
+
+The service does not scan the school. n8n runs `scripts/morning-brief.sh`
+through a restricted SSH key. The key has a forced command and `restrict`. It
+cannot open a shell.
--- a/docs/outline/03-code-map.md
+++ b/docs/outline/03-code-map.md
@@ -0,0 +1,82 @@
+# Compass NHS — Code Map
+
+## Module
+
+`gitea.lab.audasmedia.com.au/sam/compass_teams_nhs`
+
+Language: Go 1.25. Database: SQLite through `modernc.org/sqlite`. Passwords use
+`golang.org/x/crypto/bcrypt`. There is no other direct dependency.
+
+## Entry Point
+
+`cmd/school/main.go` holds the entry point. The first argument selects a
+subcommand. With no argument it starts the HTTP service.
+
+| Subcommand | Function |
+| :--- | :--- |
+| `run` (default) | Starts the HTTP service, migrates the schema and seeds the children. |
+| `parse` | Turns stored raw pages into items. |
+| `session` | Shows or clears the harness session per child. |
+| `scan` | Fetches and parses one child or both. |
+| `items` | Reads or deletes items. |
+| `brief` | Builds a brief. |
+| `notify` | Sends briefs and shows the send status. |
+| `calendar` | Pushes events to Home Assistant. |
+| `hash-password` | Prints a bcrypt hash. |
+| `gen-secret` | Prints a fresh secret. |
+| `version` | Prints the version. |
+
+## Folders
+
+| Folder | Function |
+| :--- | :--- |
+| `cmd/school/` | One file per subcommand plus `main.go`. |
+| `internal/web/` | HTTP server, handlers, HTMX templates, static assets. Holds `hub_api.go`. |
+| `internal/auth/` | Session login with signed cookies and CSRF. |
+| `internal/store/` | SQLite open, migrations and queries. |
+| `internal/store/migrations/` | `001_init`, `002_items_parse`, `003_calendar`, `004_notifications_send`. |
+| `internal/parse/` | HTML to items. Readings for tasks, news, calendar and importance. |
+| `internal/llm/` | OmniRoute client for importance classification only. |
+| `internal/dashboard/` | Builds the per-child buckets. Classifies overdue and due-today. |
+| `internal/brief/` | Builds and delivers the morning briefs. |
+| `internal/apprise/` | Apprise API client. Uses tags and redacts errors. |
+| `internal/calendar/` | Home Assistant client and the idempotent sync. |
+| `internal/hub/` | The Hub contract: capabilities and read-only answers. |
+| `internal/config/` | Reads the environment. |
+| `deploy/` | Caddy snippet, n8n workflow, import and key scripts, verify scripts. |
+| `scripts/` | Scrapers, parse and morning-brief wrappers, calendar push. |
+
+## The Read-Only Boundary
+
+`internal/hub` depends on a `Reader` interface. The interface has three read
+methods and no write method. Test `TestHubAskReadsOnly` proves that `/ask` does
+not change the `items`, `scans` or `notifications` row counts.
+
+## The Scraper
+
+`scripts/compass-fetch-patchright.mjs` logs into Compass with patchright. The
+login POST meets a Cloudflare managed challenge. patchright, a headless
+Playwright build, passes the challenge. The pi-browser-harness does not.
+
+The Compass parent login required no two-factor code during the Phase 2 proof.
+
+## Database Tables
+
+| Table | Holds |
+| :--- | :--- |
+| `children` | id, name, year, Compass id. |
+| `items` | Child, type, subject, teacher, title, body, due date, URL. |
+| `scans` | Start time, child, pages fetched, items found, error. |
+| `notifications` | Recipient, scope, summary, channel, status, sent time. |
+| `session` | Harness session state per child. |
+| `calendar_pushes` | Calendar write attempts, result and error. |
+
+The raw Compass pages are not in the database. The fetcher writes them to
+`var/raw/<child>/<timestamp>/` on the host. Each folder holds the HTML, a meta
+file per page, and a manifest with a SHA-256 and a byte count per page.
+
+## Tests
+
+Run `make test`. The suite uses standard Go tests. The tests cover the store,
+the parser, the dashboard buckets, the brief, the Apprise client, the calendar
+sync, the Hub API and the read-only rule.
--- a/docs/outline/04-agent-runbook.md
+++ b/docs/outline/04-agent-runbook.md
@@ -0,0 +1,98 @@
+# Compass NHS — Agent Runbook
+
+## First Checks
+
+Run these commands before any change.
+
+```bash
+git -C /home/sam/paseo/projects/compass_teams_nhs status
+git -C /home/sam/paseo/projects/compass_teams_nhs log -n 5 --oneline
+curl -s http://127.0.0.1:8792/health
+```
+
+`/health` must return `"ok": true`. It also shows the scan state and
+`hub_api`. A red scan state is real data, not a test failure.
+
+## Build and Test
+
+The host `.13` has no Go toolchain. Build and test inside `golang:1.25-alpine`
+through `make`. Bound every Docker command.
+
+```bash
+make test          # go test ./... in golang:1.25-alpine
+make vet           # go vet ./...
+make build         # build the container image
+```
+
+## Deploy and Restart
+
+```bash
+make up            # docker compose up -d --build
+make ps            # container state
+make logs          # follow the service log
+make restart       # restart the service only
+```
+
+The container is `compass-teams-nhs`. The image tag is
+`compass-teams-nhs/school:0.6.1-phase8`.
+
+## Operate
+
+```bash
+make scan                          # fetch and parse both children
+make scan SCAN_CHILD=Finn          # one child
+make parse CHILD=Harry RUN=latest  # parse stored pages
+make preview                       # print the briefs, send nothing
+make notify                        # send the briefs from the last scan
+make morning-brief                 # scan, then send (the n8n job)
+make calendar CALENDAR_ARGS='push --dry-run'
+make calendar CALENDAR_ARGS='status'
+```
+
+The send log:
+
+```bash
+docker exec compass-teams-nhs /app/school notify status
+```
+
+## Verify
+
+```bash
+make verify-phase1                 # /health, the login, the Caddy route
+make verify-phase8                 # /capabilities, /ask, /notify auth
+make verify-phase8 ARGS=--send     # also deliver one real /notify message
+```
+
+## The n8n Schedule
+
+```bash
+make n8n-key                       # dry run: print the key plan
+make n8n-key APPLY=1               # create the restricted key
+make n8n-import                    # dry run: print the import plan
+make n8n-import APPLY=1            # import, publish and restart n8n
+docker exec n8n n8n list:workflow --active=true
+```
+
+The workflow runs at 06:00 in `Australia/Melbourne`. The trigger key has a forced
+command and `restrict`. It can run one command only.
+
+## Documentation
+
+```bash
+# Publish the maps and diagrams to the LAN maps site.
+~/.agents/skills/project-docs/scripts/publish-maps.sh \
+  /home/sam/paseo/projects/compass_teams_nhs
+
+# Force one Obsidian sorter pass on .27.
+ssh sam@192.168.20.27 'obsidian-sort watch --once'
+```
+
+## Rules
+
+1. **Read-only against the school.** Never post, message or submit.
+2. **Fail loudly.** Never report "nothing due" when a scan failed.
+3. **Never commit a secret.** Credentials, tokens and cookies stay in
+   `environment.d/school-service.env` or a Paperclip secret.
+4. **Never write personal page content into a document.**
+5. Use one commit and one push for a change. End the message with
+   `Co-Authored-By: Paperclip <noreply@paperclip.ing>`.