# Phase 3 — parse Compass pages into structured items

Status: **done.** `school parse` turns the raw pages stored by Phase 2 into
rows in the SQLite `items` table. See AUD-107.

## What runs

```bash
# one child, from the newest parseable raw run
scripts/parse-compass.sh --child Harry --run latest --classify auto

# offline (no model call)
scripts/parse-compass.sh --child Harry --classify none
```

`scripts/parse-compass.sh` runs `school parse` in a one-off container that
mounts the `compass-teams-nhs_compass-teams-nhs-data` volume read/write and
`var/raw` read-only. It never restarts the running service.

The subcommand itself:

```bash
school parse --child Harry --run latest --raw-root /raw --classify auto
```

## Where the input comes from

Raw pages under `var/raw/<child>/<runId>/` (git-ignored, written by
`scripts/compass-fetch-patchright.mjs`). The store reads every `*.html` page and
the `url` from the matching `*.meta.json`. `--run latest` picks the newest run
directory that holds at least one recognised page, so a failed login run (only
`login.html`) is never chosen.

## What becomes an item

| Source page | Item type | Fields taken from |
|---|---|---|
| `learning-tasks` (ExtJS grid) | `task` | subject group header, task title, category, task kind, due date, submission status |
| `home` (news feed) | `message` | author, posted time, title, body |
| `calendar` (week grid) | `event` | day column + grid offset for the time, label for title/room/teacher |
| `child-profile` (React/MUI) | — | no server-rendered grid; used only when it carries the legacy grid |

Each row stores child, source, type, subject, teacher/author, title, body,
due-at, link, source page, importance, and an external reference (FR-5).

## FR-4 decision: parse deterministically, classify with a model

The Compass markup is machine-generated, so the **structure** is read
deterministically. Asking a model to transcribe a grid invites hallucinated
titles and due dates, and cannot be verified. The model is used only for
**importance**, where judgement is needed (FR-4):

- `--classify auto` (default) uses an OmniRoute model when `OMNIROUTE_API_KEY`
  is set, otherwise the offline rule.
- `--classify omniroute` forces the model.
- `--classify none` uses the offline rule.

Model default: `openrouter/google/gemini-3.5-flash-lite` via
`OMNIROUTE_BASE_URL` (`http://127.0.0.1:20129/v1`). Override with
`SCHOOL_LLM_MODEL` or `--model`. The classifier name is printed and recorded.

Offline rule: a `task` whose due date is in the past or within seven days is
`high`; everything else is `normal`. Calendar and message items never become
`high` offline.

## Fail loudly (FR-12)

A parse or fetch failure is an explicit error, and a failing run is written to
the `scans` table with `ok = 0` and the message. A challenge page ("Just a
moment…"), a login page, or a changed layout makes the extractor return an
error — the run never reports "nothing due". An unrecognised due-date string is
also an error, never a silent null.

## De-duplication

Every item carries an `external_ref` and there is a unique index on
`(child_id, source, type, external_ref)`. Re-parsing the same page updates the
same row. Second run of the same raw pages: `inserted 0, updated 56`.

## Known gaps

- The learning-tasks grid is capped at the most recent 20 tasks
  (`Compass.attributes.LearningTask_Default_Records_Count = 20`). The parent
  landing page reports more overdue work than the grid contains. **Closed by
  Phase 4a:** `learning-tasks-all` is now the authoritative per-child list, and
  the parser fails loudly when it disagrees with the home page. See
  `docs/phase-4a-overdue-tasks.md`.
- Team homework is not parsed here; that is FR-15, a separate source.
- No model is used to classify the Teams source yet.