Sam Rolfe
5d263987dd
The Learning Tasks grid is capped at 20 rows and, on the parent page, mixes both children, so it cannot answer "what is overdue". The parent home page reported 6 overdue tasks for Harry while the stored grid held 1. Fetch the per-child task list (read-only) from POST /Services/LearningTasks.svc/GetAllLearningTasksByUserId, parse it into task rows, and refuse to store it when the overdue count disagrees with the home page summary (FR-12). - fetcher: store learning-tasks-all.json + meta (userId, status, task count) - parse: new learning-tasks-all page, JSON parser, home-page reconciliation - store: drop superseded grid-derived task rows before the upsert - docs: docs/phase-4a-overdue-tasks.md; close the Phase 3 known gap Verified with live runs: Harry 70 tasks / 6 overdue, Finn 42 tasks / 2 overdue, both equal to the home page; a tampered home page stops the parse. Note: the fetcher and store.go files also carry the per-child session work that was already uncommitted in this shared workspace (AUD-108). Co-Authored-By: Paperclip <noreply@paperclip.ing>
113 lines
4.2 KiB
Markdown
113 lines
4.2 KiB
Markdown
# Phase 4a — the overdue-filtered learning-tasks view
|
|
|
|
Status: **done.** The authoritative per-child learning-task list is fetched and
|
|
parsed, and the parser fails loudly when it disagrees with the home page. See
|
|
AUD-114.
|
|
|
|
## Problem
|
|
|
|
The Learning Tasks grid is capped at the most recent 20 records
|
|
(`Compass.attributes.LearningTask_Default_Records_Count = 20`). On the parent
|
|
account the same grid mixes both children. The parent landing page reported
|
|
**6 overdue learning tasks** for Harry, but the stored grid held 1 `Overdue` row
|
|
for him. The dashboard's "overdue" column cannot be built from that page.
|
|
|
|
## The endpoint
|
|
|
|
The grid's ExtJS store reads one service:
|
|
|
|
```
|
|
POST /Services/LearningTasks.svc/GetAllLearningTasksByUserId
|
|
Content-Type: application/json
|
|
{ "userId": 27038, "page": 1, "start": 0, "limit": 1000 }
|
|
```
|
|
|
|
- **No `?sessionstate=readonly` query.** Adding it makes the service answer 500.
|
|
- The request must run **inside the logged-in page** (`fetch`, `credentials:
|
|
include`). A request from the browser context's own API client is answered 403
|
|
by the Compass request capture.
|
|
- The response is `{"d":{"data":[...],"total":n}}`. `total` is the number of
|
|
tasks for that child across the current and past enrolments.
|
|
|
|
The service returns **every** learning task for the child, with the submission
|
|
status, so the overdue count can be checked against the home page.
|
|
|
|
## Status codes
|
|
|
|
`Compass.enums.LearningTasksSubmissionStatus`:
|
|
|
|
| Code | Label |
|
|
|---|---|
|
|
| 1 | Pending |
|
|
| 2 | Overdue |
|
|
| 3 | On Time |
|
|
| 4 | Received Late |
|
|
|
|
`LearningTasks_OverdueShowAsPending = false` on the parent account, so an
|
|
`Overdue` row stays `Overdue`.
|
|
|
|
## Due dates
|
|
|
|
Most tasks carry `dueDateTimestamp` at the task level (`2026-10-09T05:00:00Z`,
|
|
UTC). A few carry it on the student row instead, so the parser reads both and
|
|
falls back to the student value.
|
|
|
|
## Fetcher
|
|
|
|
`scripts/compass-fetch-patchright.mjs` stores the response as
|
|
`learning-tasks-all.json` plus `learning-tasks-all.meta.json`. The meta records
|
|
the `userId`, the HTTP status, the byte count, the SHA-256, and the task count,
|
|
so the Go parser can tie the list to the child and to the home page. A non-200
|
|
response or a non-JSON body is an explicit fetch error (FR-12).
|
|
|
|
```bash
|
|
cd scripts && npm install
|
|
COMPASS_USERNAME=... COMPASS_PASSWORD=... \
|
|
node scripts/compass-fetch-patchright.mjs --child Harry --user-id 27038
|
|
```
|
|
|
|
Read-only against the school.
|
|
|
|
## Parser
|
|
|
|
`internal/parse/tasks_json.go` turns the JSON into `task` rows:
|
|
|
|
- one row per task for the child's `userId` (a sibling's rows are skipped),
|
|
- `external_ref = learning-task|<taskId>` (stable; the task id is unique),
|
|
- the submission status and due date are written into the item body, so the
|
|
offline importance rule marks an overdue task `high`.
|
|
|
|
When a run holds `learning-tasks-all`, the parser **ignores the legacy HTML
|
|
grid** (`learning-tasks` and `child-profile`): the grid is capped and would store
|
|
the same task a second time. `school parse` then deletes the child's old
|
|
grid-derived task rows before the upsert.
|
|
|
|
## Fail loudly (FR-12)
|
|
|
|
`parse.Parse` reads the child's overdue count from the home page (the
|
|
`N overdue learning tasks` link that carries the child's `userId`) and compares
|
|
it with the number of `Overdue` rows in the JSON list.
|
|
|
|
- They must be equal. A disagreement stops the parse, records a failed scan
|
|
(`ok = 0`), and stores nothing.
|
|
- A home page with no record of the child also stops the parse; a missing
|
|
summary is never read as zero.
|
|
|
|
## Verification (2026-10-15)
|
|
|
|
| Child | Compass id | Tasks | Overdue | Home page |
|
|
|---|---|---|---|---|
|
|
| Harry | 27038 | 70 | 6 | 6 overdue |
|
|
| Finn | 31460 | 42 | 2 | 2 overdue |
|
|
|
|
- `school parse --child Harry --classify none` →
|
|
`overdue 6 (reconciled against the home page)`, `Harry task 70`.
|
|
- `school parse --child Finn --classify none` →
|
|
`overdue 2 (reconciled against the home page)`, `Finn task 42`.
|
|
- Tampered home page (6 → 5): `parse: Harry: home page reports 5 overdue
|
|
learning task(s) but the stored learning-task list has 6; refusing to store a
|
|
partial list (FR-12)` and exit 1.
|
|
|
|
Raw pages stay under `var/raw/<child>/<runId>/` and are git-ignored. The service
|
|
response holds the family's personal data; it is never committed.
|