Phase 7: telemetry and usage (daily rollup + week dashboard)
Build on Phases 1–6. Telemetry already arrived on
restrict/<device>/telemetry; this phase stores it in a daily rollup and
shows it.
- store: migration 003 adds telemetry_daily (device, day, app seconds,
site seconds). InsertTelemetry maintains the affected days in the same
transaction; RebuildRollups backfills and repairs it. UsageRange and
TopUsage serve the week grid and top lists. CapAudit enforces the
size-capped append-only audit log from AGENTS.md §7.
- web: GET /devices/{id}/usage — Monday-to-Sunday totals plus the top 10
apps and top 10 sites, using CSP-safe native <progress> bars.
- theme.js: light/dark from the OS preference, with a nav toggle, loaded
as an external script so the strict CSP still holds.
- cmd/control: `rollup` subcommand; rollup rebuild and audit cap at
start-up and every 15 minutes.
- Version 0.5.0-phase7 (compose, Makefile, docs).
Tests: go test ./... and go vet ./... pass in golang:1.25-alpine.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
70
README.md
70
README.md
@@ -5,8 +5,8 @@ Internet Use Control project (`SYSTEM_MAP.md`). It holds schedules, blocklists,
|
||||
per-device state and usage, and serves the web dashboard. Only it publishes
|
||||
device config.
|
||||
|
||||
This repository currently contains **Phase 6 — Schedules, blocklists,
|
||||
allowlists** (AGENTS.md §11), on top of Phases 1–3:
|
||||
This repository currently contains **Phase 7 — Telemetry and usage**
|
||||
(AGENTS.md §11), on top of Phases 1–6:
|
||||
|
||||
- Go service with its own image (`Dockerfile`) and compose file.
|
||||
- SQLite schema for the whole data model (§7), applied through versioned
|
||||
@@ -34,10 +34,14 @@ allowlists** (AGENTS.md §11), on top of Phases 1–3:
|
||||
process) and `site` (hostname) entries. While a device is blocked, the device
|
||||
blocks every blocklist value **except** the allowlist values (Spotify at
|
||||
night). That allow decision is in the published config.
|
||||
- **Usage**: telemetry reports are stored per device, day, app and domain and
|
||||
rolled up into a daily summary. The dashboard shows a week at a time — the
|
||||
daily total plus the top apps and top sites — with dark mode and phone-sized
|
||||
touch targets.
|
||||
- Every edit is written to the append-only audit log (FR-15).
|
||||
|
||||
Enforcement itself (the kids' clients), the admin API, usage screens and the Hub
|
||||
arrive in later phases. Nothing here touches the router, Pi-hole, or DNS.
|
||||
Enforcement itself (the kids' clients), the admin API and the Hub arrive in
|
||||
later phases. Nothing here touches the router, Pi-hole, or DNS.
|
||||
|
||||
## Quick start (on `.13`)
|
||||
|
||||
@@ -53,7 +57,7 @@ docker compose up -d --build
|
||||
|
||||
# 3. Prove it.
|
||||
curl -s http://127.0.0.1:8791/health
|
||||
# {"status":"ok","service":"control-service","version":"0.4.0-phase6",...}
|
||||
# {"status":"ok","service":"control-service","version":"0.5.0-phase7",...}
|
||||
```
|
||||
|
||||
Open <http://127.0.0.1:8791/> in a browser. It redirects to `/login`. Sign in
|
||||
@@ -90,7 +94,7 @@ reads only the process environment.
|
||||
Generate a bcrypt hash without host Go:
|
||||
|
||||
```bash
|
||||
docker run --rm --entrypoint /app/control webrestrict/control:0.4.0-phase6 \
|
||||
docker run --rm --entrypoint /app/control webrestrict/control:0.5.0-phase7 \
|
||||
hash-password 'your-password'
|
||||
```
|
||||
|
||||
@@ -172,7 +176,7 @@ List the MQTT identity of every device (tokens, no secrets):
|
||||
docker exec webrestrict-control /app/control mqtt-clients
|
||||
```
|
||||
|
||||
## Endpoints (Phase 2–3)
|
||||
## Endpoints (Phase 2–7)
|
||||
|
||||
| Method | Path | Notes |
|
||||
|---|---|---|
|
||||
@@ -193,11 +197,46 @@ docker exec webrestrict-control /app/control mqtt-clients
|
||||
| `GET` | `/devices/{id}/lists` | Blocklist / allowlist editor. |
|
||||
| `POST` | `/devices/{id}/lists` | Adds one entry (`table=block|allow`, `kind=app|site`, `value`). |
|
||||
| `POST` | `/devices/{id}/lists/{entryID}/delete` | Removes one entry. |
|
||||
| `GET` | `/devices/{id}/usage` | Usage for one device: a week of daily totals, top apps and top sites. `?week=YYYY-MM-DD` selects the week (any day in it; default today). |
|
||||
|
||||
Every form POST needs the `csrf` field from the session cookie. State changes
|
||||
append to the `audit` table and republish the device's retained config. Devices
|
||||
are never hard-deleted (§10 rule 10).
|
||||
|
||||
## Telemetry and usage (Phase 7)
|
||||
|
||||
Usage arrives on `restrict/<device>/telemetry` and is stored in `telemetry`:
|
||||
one row per (device, day, app, domain), where a repeat report for the same key
|
||||
replaces the earlier total (a rolling daily total, not a delta).
|
||||
|
||||
`telemetry_daily` is the **daily rollup** — one row per (device, day) — written
|
||||
together with the detail row and rebuildable from it with `control rollup`.
|
||||
Apps and domains are summed into separate columns on purpose: a report may
|
||||
carry both for the same session, so a single total would double-count the time.
|
||||
The rollup is derived data, never a second source of truth.
|
||||
|
||||
`GET /devices/{id}/usage` shows one Monday-to-Sunday week: the app total and the
|
||||
site total per day, then the top 10 apps and top 10 sites for the week. The
|
||||
screen uses the Family Home Lab tokens, follows the OS light/dark preference
|
||||
(there is a toggle in the nav bar), and works on a phone browser.
|
||||
|
||||
Retention:
|
||||
|
||||
- The daily rollup is kept long; the detail table is small and kept too.
|
||||
Nothing is hard-deleted (AGENTS.md §10 rule 10).
|
||||
- The append-only `audit` log is **size-capped** at 20 000 rows by the service's
|
||||
maintenance loop, oldest first (§7). It is never edited in place.
|
||||
|
||||
Rebuild the rollup from the detail table at any time (idempotent, safe while the
|
||||
service is up; the service also does it at start-up and every 15 minutes):
|
||||
|
||||
```bash
|
||||
# inside the container
|
||||
docker exec webrestrict-control /app/control rollup
|
||||
# or against a database file on the host
|
||||
/app/control rollup --db /data/control.db
|
||||
```
|
||||
|
||||
## Schedules, blocklists and allowlists (Phase 6)
|
||||
|
||||
A **schedule** is a set of weekly blocking windows per device: a weekday
|
||||
@@ -239,14 +278,14 @@ broker on `127.0.0.1:18883` and removes it afterwards.
|
||||
## Layout
|
||||
|
||||
```
|
||||
cmd/control/ service entry point (+ hash-password, mqtt-clients)
|
||||
cmd/control/ service entry point (+ hash-password, mqtt-clients, rollup)
|
||||
internal/config/ environment configuration
|
||||
internal/store/ SQLite open, migrations, audit, devices + state
|
||||
devices.go device registry, desired state, lock expiry
|
||||
schedules.go weekly windows and the effective state
|
||||
reports.go presence and telemetry written from broker reports
|
||||
reports.go presence, telemetry, the daily rollup, top lists, audit cap
|
||||
lists.go allow/block lists and device lookup by topic
|
||||
migrations/001_init.sql, 002_telemetry_unique.sql
|
||||
migrations/001_init.sql, 002_telemetry_unique.sql, 003_usage_rollup.sql
|
||||
internal/auth/ signed-cookie session login + CSRF
|
||||
internal/mqtt/ the broker link: retained config out, reports in
|
||||
payload.go topic and payload contract for the kids' clients
|
||||
@@ -254,8 +293,9 @@ internal/mqtt/ the broker link: retained config out, reports in
|
||||
internal/web/ HTTP handlers, templates, static assets
|
||||
devices.go device registry + manual lock/unlock handlers
|
||||
schedules.go weekly schedule and block/allow editors
|
||||
templates/ login, dashboard, device form, editors, HTMX partials
|
||||
static/ tokens.css (Family Home Lab), app.css, htmx, Inter fonts
|
||||
usage.go usage screen: week grid, top apps, top sites
|
||||
templates/ login, dashboard, device form, editors, usage, partials
|
||||
static/ tokens.css (Family Home Lab), app.css, theme.js, htmx, fonts
|
||||
Dockerfile own image (multi-stage, CGO-free)
|
||||
docker-compose.yml own container, port 8791
|
||||
environment.d/ control-service.env.example (real file gitignored)
|
||||
@@ -269,12 +309,14 @@ deploy/mosquitto/ TLS listener, credentials, topic ACL, verifier
|
||||
`webrestrict.lab.audasmedia.com.au`. The terminating Caddy runs on `.35`
|
||||
(sam-Ubuntu1); see that file for the two proxy patterns and the apply commands.
|
||||
|
||||
## Non-goals (Phase 6)
|
||||
## Non-goals (Phase 7)
|
||||
|
||||
- No group/global list editing from the UI. The data model supports a NULL
|
||||
`device_id` that applies to every device, and the effective state merges it,
|
||||
but the editor manages per-device rows only.
|
||||
- No usage screens (Phase 7). Telemetry is stored, not yet shown.
|
||||
- No HTTP telemetry ingest. Devices report over MQTT; an HTTP path arrives if a
|
||||
client needs it.
|
||||
- No usage charts or export beyond the week grid and the top lists shown.
|
||||
- No admin API (Phase 8).
|
||||
- No kids' clients (Phases 4–5). The topic and payload contract above is what
|
||||
they implement. The clients apply the allowlist exception; the Control Service
|
||||
|
||||
Reference in New Issue
Block a user