- §9 Technology: scraper is patchright (headless Playwright); the pi-browser-harness gets a Cloudflare 403 managed challenge on the login POST. - §13 Resolved: add the patchright decision and the 2FA finding (the Compass parent login required no two-factor or one-time code during the Phase 2 proof). - §13 Remaining: Teams two-factor auth is still unverified. Co-Authored-By: Paperclip <noreply@paperclip.ing>
248 lines
12 KiB
Markdown
248 lines
12 KiB
Markdown
# Compass / NHS (school)
|
|
|
|
> Build spec for Paperclip. Follow the phases in section 11 in order. Do not skip a
|
|
> phase's Definition of Done. When something is unclear, ask Sam — do not guess.
|
|
|
|
---
|
|
|
|
## 1. Purpose and outcome
|
|
|
|
Two school systems, for Harry and Finn:
|
|
|
|
- **Compass** — the daily schedule and planner: what is on, what is overdue, notices.
|
|
- **Teams** — homework and assignments.
|
|
|
|
Store both, surface overdue and pending work, write important events to Sam's Home
|
|
Assistant calendar, and send the morning briefs.
|
|
|
|
**Outcome (the single success statement):**
|
|
> Sam sees, in one page, what each child has due and what is overdue — without
|
|
> logging into Compass — and the family is told before work is late.
|
|
|
|
## 2. Scope
|
|
|
|
**In scope**
|
|
- Scan **Compass** per child: the daily schedule, notices, and due dates.
|
|
- Scan **Teams** per child: homework and assignments. **Scope may be limited** — pick
|
|
the smallest section that still answers "what is overdue". Decide in Phase 2.
|
|
- Store and display per child.
|
|
- Detect overdue and pending work.
|
|
- Write important events to Sam's Home Assistant calendar.
|
|
- Send the morning briefs via Apprise.
|
|
- Answer the Hub.
|
|
|
|
**Out of scope**
|
|
- The **React calendar** — that is a separate project. Do not build a calendar UI here.
|
|
- The school's own systems. Read-only.
|
|
|
|
## 2a. Prerequisites (verify before Phase 1)
|
|
|
|
Check these first. The values come from the prerequisite check on `.13` (AUD-102).
|
|
Items marked **not verified** need a decision or a test.
|
|
|
|
| Need | Status / note |
|
|
|---|---|
|
|
| **Go toolchain** | **MISSING on `.13`** — `go` is not in the PATH, not in `nix-env`, and not in `/etc/nixos/configuration.nix`. Sam must add `go` to `environment.systemPackages` and run `nixos-rebuild switch`. Phase 1 can build in the `golang` Docker image instead. |
|
|
| **Docker + compose** | Present on `.13`. Docker client and server 29.6.2, Compose 5.3.1. Run as user `sam` (in the `docker` group, no sudo). |
|
|
| **Port 8792** | Free on `.13`. No listener and no container maps this port. Ports 8790 and 8791 are in use by other services. |
|
|
| **Caddy route on `.35`** | Caddy runs as the Docker container `caddy` (v2.8.4) on `sam-Ubuntu1` (`.35`). Routes are site blocks in `/Docker/Containers/caddy/Caddyfile`, bind-mounted to `/etc/caddy/Caddyfile`. Use `school.lab.audasmedia.com.au { import basic-auth; reverse_proxy 192.168.20.13:8792 }`. **No `school.*` block exists yet** — Phase 1 must add one. Reload with `docker exec caddy caddy reload --config /etc/caddy/Caddyfile`. DNS for the name resolves (wildcard) to `144.6.86.11`. |
|
|
| **pi-browser-harness** | Present on `.13` at `/home/sam/.agents/extensions/pi-browser-harness`, version 0.11.0. It is a **pi extension**, not a CLI. Load it as a package extension, then use the commands `/browser-setup` and `/browser-profile`. It exposes `browser_*` tools and drives a long-lived headless Chrome over CDP (`--user-data-dir=/home/sam/.chrome-pi`, debug port 9222). Session reuse works. **The extension is not enabled in this repo's `.pi/settings.json`** — Phase 2 must enable it before the login proof. |
|
|
| **Compass URL + reachability** | **`https://nhs-vic.compass.education/login.aspx`** (Northcote High School). DNS goes to Cloudflare. TCP and TLS reach the edge, but the page returns `HTTP/2 403` with `cf-mitigated: challenge` (Cloudflare bot challenge). A real browser session is necessary. This is the Phase 2 risk. |
|
|
| **Compass credentials + 2FA** | **NOT AVAILABLE** — no Compass, school, or child key exists in `~/.config/environment.d/10-secrets.conf` or its backups, and no school-named secret is in the instance config. Sam must supply the account name, the username, and the password as a **Paperclip secret** (or in `10-secrets.conf`). Never write them in this file. **2FA is not verified** — no login was attempted. Confirm whether Compass enforces 2FA or new-device email verification, and define the manual step. |
|
|
| **Apprise** | Running on `.35:8210` (container `apprise`, host 8210 to container 8000). `GET /` and `GET /status` return 200. The Caddy route `apprise.lab.audasmedia.com.au` has basic auth. |
|
|
|
|
**Gating read-out**
|
|
|
|
- **Phase 1** is unblocked except for Go. Build in the `golang` Docker image, or ask Sam to install Go on the host.
|
|
- **Phase 2** is blocked until Sam supplies a Compass credential and confirms the 2FA step. Phase 2 must also enable the `pi-browser-harness` extension in this repo.
|
|
- The Cloudflare challenge is the main technical risk. Prove it first.
|
|
|
|
**Not verified**
|
|
|
|
- Whether Compass enforces 2FA.
|
|
- The exact per-child account model.
|
|
- Whether the headless Chrome profile clears the Cloudflare challenge (no login was attempted).
|
|
|
|
## 3. Users and roles
|
|
|
|
| User | Can do |
|
|
|---|---|
|
|
| **Sam** | Everything. Session login. |
|
|
| **Jo / Harry / Finn** | Receive notifications. Read-only. |
|
|
|
|
## 4. Functional requirements
|
|
|
|
| ID | Requirement |
|
|
|---|---|
|
|
| FR-1 | Scan Compass with the **pi-browser-harness** on an n8n schedule, once per child. |
|
|
| FR-2 | Log in and reuse a stored session. A manual step is allowed when the school demands it. |
|
|
| FR-3 | Parse messages, homework, due dates, and calendar events into structured items. |
|
|
| FR-4 | Use JEV or the LLM to structure the page content and classify importance. |
|
|
| FR-5 | Store every item with its child, source page, type, subject, due date, and link. |
|
|
| FR-6 | Detect overdue work and work due today and this week. |
|
|
| FR-7 | Show a per-child view: overdue, due today, due this week, and messages. |
|
|
| FR-8 | Write **important school events** to **Sam's Home Assistant calendar**. The Hub or Jervis may also push into it. |
|
|
| FR-9 | Send a **morning brief to each boy** (Harry, Finn): what is on today, overdue work, pending work, and school notices. |
|
|
| FR-10 | Provide a **Delete** action and a **Send** action on items and summaries. |
|
|
| FR-11 | Hub API: `GET /capabilities`, `POST /ask` (read-only), `POST /notify`. |
|
|
| FR-12 | Fail **loudly** on a scrape failure. Never report "nothing due" when the scan failed. |
|
|
| FR-13 | Log every scan: time, child, pages, items, errors. |
|
|
| FR-14 | Send a **combined brief** for both boys to **Jo and Sam** via Apprise. |
|
|
| FR-15 | Scan **Teams** for homework and assignments, with the smallest useful scope. |
|
|
|
|
## 5. Screens and UX flow
|
|
|
|
**Screens**
|
|
1. **Login** — session form.
|
|
2. **Child dashboard** — a tab per child: overdue, due today, due this week, messages.
|
|
3. **Item detail** — subject, teacher, due date, link, and full text.
|
|
4. **Send panel** — choose recipients, preview the brief, send.
|
|
5. **Scan history** — last scans, items found, errors.
|
|
|
|
**Main flow**
|
|
```
|
|
n8n triggers the morning scan → items stored
|
|
→ each boy gets his own morning brief (today, overdue, pending, notices)
|
|
→ Jo and Sam get one combined brief
|
|
→ important events are written to Sam's Home Assistant calendar
|
|
```
|
|
|
|
**Output style:** brief, concise, lists only. No fluff. Overdue first.
|
|
|
|
## 6. Look and feel
|
|
|
|
Reuse the Family Home Lab design system.
|
|
|
|
- Source: `/home/sam/Docker/Containers/family-home-lab/DESIGN.md`
|
|
- Tokens: `/home/sam/Docker/Containers/family-home-lab/portal/static/tokens.css`
|
|
- Copy `tokens.css` into this project.
|
|
|
|
Canvas `#f6f5f4`, one action blue `#0075de`, Inter, 8px grid, pill CTAs, hairline
|
|
borders. Dark mode. Works on a phone browser.
|
|
|
|
## 7. Data model
|
|
|
|
SQLite inside the service. Suggested tables:
|
|
|
|
| Table | Holds |
|
|
|---|---|
|
|
| `children` | id, name, year, compass id |
|
|
| `items` | child, type (`message`, `task`, `event`), subject, teacher, title, body, due-at, url |
|
|
| `scans` | started-at, child, pages fetched, items found, error |
|
|
| `notifications` | recipient, summary text, sent-at |
|
|
| `session` | harness session state per child |
|
|
|
|
Retention: keep items while the term is current. Scans are append-only and size-capped.
|
|
|
|
## 8. Integrations
|
|
|
|
| System | Use |
|
|
|---|---|
|
|
| **pi-browser-harness** | Log in and fetch Compass pages. Already built and working. |
|
|
| **JEV** | Structure page content; classify importance. HTTP API. |
|
|
| **OmniRoute** | All other models. |
|
|
| **n8n** | Triggers the daily scan. |
|
|
| **Apprise** | Per-boy morning briefs, and one combined brief to Jo and Sam. Running on `.35:8210`. |
|
|
| **Calendar** | **Sam's Home Assistant calendar.** The Hub or Jervis pushes into HA. |
|
|
| **Hub** | `POST /ask`, `POST /notify`, `GET /capabilities`. |
|
|
|
|
The Hub interface is frozen in `central_ai_llm_master/docs/ask-contract.md`.
|
|
`/ask` is read-only.
|
|
|
|
## 9. Technology (locked)
|
|
|
|
| Item | Value |
|
|
|---|---|
|
|
| Backend | **Go** |
|
|
| UI | **HTMX** + server-rendered HTML |
|
|
| Database | **SQLite** |
|
|
| Scraper | **patchright** (headless Playwright). The **pi-browser-harness** cannot do the daily scan: it gets a Cloudflare **403 managed challenge** on the login POST. |
|
|
| Design | Family Home Lab `tokens.css` |
|
|
| Host | `.13`, Docker |
|
|
| Container | **This project's own** image and compose file. |
|
|
| Port | Allocate one (suggest **8792**). Record it once chosen. |
|
|
| URL | `school.lab.audasmedia.com.au` via Caddy on `.35` |
|
|
| Auth | **Separate session login.** |
|
|
|
|
## 10. Rules and constraints
|
|
|
|
1. **Read-only against the school.** Never post, message, or submit anything.
|
|
2. **A failed scan must be loud.** Never show an empty day as "nothing due".
|
|
3. School credentials and session state are secrets. Never commit them.
|
|
4. Models go through **OmniRoute**, except JEV.
|
|
5. `/ask` is read-only.
|
|
6. Output is brief: lists, no fluff, overdue first.
|
|
7. Writing a calendar event needs **no** approval.
|
|
8. Own folder, own container, own port.
|
|
|
|
## 11. Build order
|
|
|
|
**Phase 1 — Skeleton**
|
|
Go service, SQLite, config, session login, Dockerfile, compose, Caddy route.
|
|
*DoD:* the app runs on `.13`, login works, `/health` returns ok.
|
|
|
|
**Phase 2 — Prove the login and the fetch**
|
|
Point the pi-browser-harness at Compass. Log in once. Fetch the pages needed for one
|
|
child. Store the raw pages.
|
|
*DoD:* a real Compass page for one child is fetched and stored. **This is the risk
|
|
phase — a failed login stops the project, so prove it early.**
|
|
|
|
**Phase 3 — Parse into items**
|
|
Turn pages into structured items: messages, tasks, events, due dates, links.
|
|
*DoD:* real homework and messages for one child appear as structured rows.
|
|
|
|
**Phase 4 — Both children**
|
|
Repeat for Harry and Finn, with per-child sessions.
|
|
*DoD:* both children's items are stored and separated.
|
|
|
|
**Phase 5 — Dashboard**
|
|
Per-child view with overdue, due today, due this week, and messages.
|
|
*DoD:* Sam can see, per child, what is overdue and due, on a phone.
|
|
|
|
**Phase 6 — Calendar write**
|
|
Write school events to the calendar.
|
|
*DoD:* a real school event appears in the calendar.
|
|
|
|
**Phase 7 — Notify and send**
|
|
Apprise notifications, recipient selection, and the Send panel.
|
|
*DoD:* a brief is sent to a real recipient.
|
|
|
|
**Phase 8 — Hub**
|
|
`/capabilities`, `/ask`, `/notify`.
|
|
*DoD:* the Hub asks "what does Harry have on today at school?" and gets a correct answer.
|
|
|
|
**Phase 9 — Documentation**
|
|
*DoD:* project-docs run; Gitea, Obsidian, Outline, Vikunja, and the map are updated.
|
|
|
|
## 12. Acceptance criteria
|
|
|
|
- The scan runs daily and stores real items for both children.
|
|
- Overdue work is detected and shown first.
|
|
- A failed login shows an error, not an empty day.
|
|
- A school event can be written to the calendar.
|
|
- A brief can be sent to a real recipient from the app.
|
|
- The Hub answers a school question correctly.
|
|
- Runs in its own container on its own port, behind Caddy.
|
|
|
|
## 13. Decisions and remaining questions
|
|
|
|
Resolved:
|
|
1. **React calendar dropped.** Separate project.
|
|
2. **Login** is on-screen, via the **pi-browser-harness** or JEV.
|
|
3. **Go + HTMX + SQLite**, own container, `school.lab.audasmedia.com.au`, session login.
|
|
4. **Calendar = Home Assistant (Sam's).** Important events are written there.
|
|
5. **Recipients:** each boy gets his own morning brief; Jo and Sam get one combined brief.
|
|
6. **Focus:** Compass gives the daily schedule; Teams gives homework and assignments.
|
|
7. **Trigger time: 6:00 am.**
|
|
8. **Apprise channels: Email and WhatsApp.**
|
|
9. **Scraper = patchright.** The daily scan uses patchright (headless Playwright).
|
|
The pi-browser-harness cannot pass the Cloudflare managed challenge on the login POST.
|
|
10. **2FA:** the Compass parent login required no two-factor or one-time code during
|
|
the Phase 2 proof.
|
|
|
|
Remaining:
|
|
1. **Teams scope:** find the smallest section that answers "what is overdue". Teams
|
|
may be hard to reach. This is a decision for the brief and for Phase 2.
|
|
2. **Teams two-factor auth:** not verified. Confirm whether Teams needs 2FA, then
|
|
define the manual step.
|