Turn the raw pages stored by Phase 2 into rows in the SQLite items table
(AGENTS.md §11 Phase 3, FR-3/FR-4/FR-5/FR-12).
- internal/parse: dependency-free HTML reader plus extractors for the ExtJS
learning-task grid (subject, title, category, kind, due date, status), the
home news feed (author, posted time, title, body) and the calendar week grid
(day column + grid offset give the time; label gives title/room/teacher).
- internal/parse: fail loudly. A challenge page, login page, changed layout or
unknown due-date format is an explicit error, never an empty day.
- FR-4: structure is parsed deterministically; importance is classified by an
OmniRoute model (default openrouter/google/gemini-3.5-flash-lite) with a
deterministic fallback. The classifier used is printed and recorded.
- internal/llm: small OpenAI-compatible OmniRoute client (handles SSE replies).
- internal/store: migration 002 adds source_page, importance and external_ref
with a unique (child_id, source, type, external_ref) index; UpsertItems,
CountItemsByType, SampleItems and ChildByName.
- cmd/school parse: reads var/raw/<child>/<run>, upserts items, records the scan
row (ok=0 plus the error on failure) and prints counts and samples.
- scripts/parse-compass.sh and make parse run it in a one-off container against
the service's own volume without restarting the service.
- docs/phase-3-parse-notes.md records the approach, the FR-4 decision and the
known gaps.
Evidence on the live database: 56 rows for Harry (20 task, 1 message, 35 event).
Re-running the same raw pages inserts 0 and updates 56.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- §9 Technology: scraper is patchright (headless Playwright); the
pi-browser-harness gets a Cloudflare 403 managed challenge on the login POST.
- §13 Resolved: add the patchright decision and the 2FA finding (the Compass
parent login required no two-factor or one-time code during the Phase 2 proof).
- §13 Remaining: Teams two-factor auth is still unverified.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- add scripts/compass-fetch-patchright.mjs: headless patchright logs in with the
parent account (no 2FA) and stores the per-child pages with a sha256 manifest
- add scripts/package.json + lockfile for the patchright dependency
- docs: the harness clears the Cloudflare GET but not the login POST; patchright
passes it; record the real per-child URLs and the overdue counts
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- retry the login on a fresh browser context (stale cf_chl_rc_ni cookies loop
the next challenge) and fail loudly with cloudflare_login_post_challenge
- detect the Compass login-rejection message and second-factor prompts
- docs: record that Cloudflare's managed challenge blocks the headless login
POST (GET passes), what was tried, and the unblock options
Co-Authored-By: Paperclip <noreply@paperclip.ing>
- scripts/compass-fetch.mjs drives pi-browser-harness (daemon socket) to log
in once and store raw Compass pages with a sha256/byte-count manifest
- clear the Cloudflare managed challenge via a desktop-UA override plus an
isolated browser context (the default HeadlessChrome UA loops the challenge)
- docs/phase-2-harness-notes.md records the harness entrypoint, the Compass
login form, and the stored-page evidence
- gitignore local runtime state (var/, sqlite files)
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Phase 1 of AGENTS.md §11: the app runs on .13 in its own container on port
8792, behind Caddy at school.lab.audasmedia.com.au, with its own session login
and GET /health.
- Go service (Go 1.25, modernc SQLite, no cgo) with its own image.
- SQLite schema per AGENTS.md §7: children, items, scans, notifications,
session (internal/store/migrations/001_init.sql).
- Separate session login (signed cookie, CSRF double-submit), not SSO.
- HTMX server-rendered shell using the Family Home Lab tokens.css: canvas
#f6f5f4, action blue #0075de, Inter, 8px grid, hairline borders, pill CTAs,
dark mode, phone-friendly.
- Fail loudly (FR-12): a missing or failed scan shows as an explicit warning or
error banner, never as an empty day.
- Dockerfile + docker-compose.yml (own container, own port 8792; uses
network_mode bridge because .13 has no free default Docker address pool).
- Caddy snippet for .35 in deploy/caddy/, deployment notes, and
deploy/verify-phase1.sh.
No school scan in this phase; the Compass login and fetch are Phase 2.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Record the verified Phase 1-2 prerequisites from AUD-102: Go toolchain
(missing), Docker + compose, port 8792, the Caddy route mechanism,
pi-browser-harness, Compass URL and reachability, the credential/2FA
state, and Apprise. Mark the unverified items.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
