sam/internet_use_control
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 3: MQTT transport — retained config, status/telemetry, LWT presence, TLS/ACL design

Browse Source 
- internal/mqtt: publish each device's desired state retained on
  restrict/<device>/config; subscribe to restrict/+/status and
  restrict/+/telemetry and write reports to SQLite; publish on every state
  change, republish on connect and every 15 minutes from SQLite, so the broker
  is repaired from the source of truth.
- store: presence (devices.online and last_seen, audited only on transitions),
  telemetry upsert per (device, day, app, domain) via migration 002, allow and
  block lists read into the config payload, device lookup by topic.
- config: MQTT settings. TLS plus per-device credentials are required; the
  pre-TLS shared broker needs an explicit CONTROL_MQTT_INSECURE escape hatch.
- web: republish config after lock, unlock, expiry, create and edit; broker card
  on the status partial.
- deploy/mosquitto: TLS listener, topic ACL, credential provisioning and a
  verifier that proves the policy on a throwaway broker. Not applied to the
  shared Mosquitto on .13 — that change needs Sam's approval.
- cmd/control: mqtt-clients prints the MQTT identity of every device.
- paho.mqtt.golang v1.5.1, which pulls golang.org/x/net and with it the
  golang.org/x/crypto bump to v0.42.0.

Verified on .13: retained config delivered to a fresh subscriber; a client
killed without DISCONNECT drove the broker's retained will and the service
flipped the device offline; telemetry landed in SQLite; make mqtt-verify passed
13/13; the service connected over TLS with credentials and a CA.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Sam Rolfe
2026-10-13 14:37:43 +11:00
parent 7c88d3f52b
commit 14b304e0b0
31 changed files with 2624 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

126
README.md
View File

@@ -5,8 +5,8 @@ Internet Use Control project (`SYSTEM_MAP.md`). It holds schedules, blocklists,
per-device state and usage, and serves the web dashboard. Only it publishes
device config.
This repository currently contains **Phase 2 — Devices and state**
(AGENTS.md §11), on top of the Phase 1 skeleton:
This repository currently contains **Phase 3 — MQTT** (AGENTS.md §11), on top of
Phases 1 and 2:
- Go service with its own image (`Dockerfile`) and compose file.
- SQLite schema for the whole data model (§7), applied through versioned
@@ -22,9 +22,13 @@ This repository currently contains **Phase 2 — Devices and state**
- **Dashboard cards**: name, child, platform, status, current state, time left.
- **Manual lock/unlock** from the dashboard, persisted in `state` and written
to the append-only `audit` log (FR-15).
- **MQTT**: each device's desired state is published *retained* on
`restrict/<device>/config`; device status and telemetry are subscribed and
written to SQLite; presence comes from the broker's Last Will and Testament.
The broker is transport only — SQLite stays the source of truth.
Enforcement, MQTT, schedules, blocklists and the clients arrive in later
phases. Nothing here touches the router, Pi-hole, or DNS.
Enforcement, schedules, blocklists, the admin API and the kids' clients arrive
in later phases. Nothing here touches the router, Pi-hole, or DNS.
## Quick start (on `.13`)
@@ -40,7 +44,7 @@ docker compose up -d --build
# 3. Prove it.
curl -s http://127.0.0.1:8791/health
# {"status":"ok","service":"control-service","version":"0.2.0-phase2",...}
# {"status":"ok","service":"control-service","version":"0.3.0-phase3",...}
```
Open <http://127.0.0.1:8791/> in a browser. It redirects to `/login`. Sign in
@@ -65,15 +69,91 @@ reads only the process environment.
| `CONTROL_ADMIN_PASSWORD_HASH` | — | bcrypt hash (option B, preferred) |
| `CONTROL_APP_ENV` | `production` | Label shown in the footer |
| `CONTROL_LOG_LEVEL` | `info` | `debug`/`info`/`warn`/`error` |
| `CONTROL_MQTT_ENABLED` | `true` | Turn the broker link on or off |
| `CONTROL_MQTT_BROKER` | `tcp://host.docker.internal:1883` | Broker URL; `tls://…:8883` once the TLS listener exists |
| `CONTROL_MQTT_CLIENT_ID` | `control-service` | MQTT client id |
| `CONTROL_MQTT_USERNAME` | — | Control Service broker account |
| `CONTROL_MQTT_PASSWORD` | — | its password |
| `CONTROL_MQTT_CA_FILE` | `/etc/webrestrict/tls/ca.crt` | PEM bundle that verifies the broker (TLS only) |
| `CONTROL_MQTT_INSECURE` | `false` | **Temporary.** Allow a plaintext, credential-less broker |
Generate a bcrypt hash without host Go:
```bash
docker run --rm --entrypoint /app/control webrestrict/control:0.2.0-phase2 \
docker run --rm --entrypoint /app/control webrestrict/control:0.3.0-phase3 \
hash-password 'your-password'
```
## Endpoints (Phase 2)
`CONTROL_MQTT_USERNAME`/`CONTROL_MQTT_PASSWORD` are required unless
`CONTROL_MQTT_INSECURE=true`, and a non-TLS broker is refused unless
`CONTROL_MQTT_INSECURE=true` (AGENTS.md §10 rule 6). The shared Mosquitto on
`.13` has no TLS listener yet, so the running deployment sets the escape hatch
and logs a warning. `deploy/mosquitto/README.md` holds the TLS listener, the
per-device credentials and the topic ACL, verified but not yet applied — see
that file for the apply and rollback steps.
## MQTT (Phase 3)
Broker: the shared **Mosquitto on `.13`**. A second broker is not added
(§10 rule 6).
| Topic | Direction | Retained | Payload |
|---|---|---|---|
| `restrict/<device>/config` | service → device | yes | desired state |
| `restrict/<device>/status` | device → service | yes | presence (`online`) |
| `restrict/<device>/telemetry` | device → service | no | usage totals for a day |
- `<device>` is `restrict/<slug>_<id>` from `devices.topic`, which is stored when
the device is created and never changes. `<id>` is its MQTT username.
- The service publishes each device's config **retained** on startup, after every
lock, unlock, expiry, create and edit, and again every 15 minutes so a broker
that lost its retained store is repaired from SQLite.
- A device registers the will `{"online": false}` on its status topic, so the
broker reports the drop the moment the connection dies. That is presence
(FR-6); no polling.
- Reports are written to SQLite. Telemetry is a rolling daily total per
(device, day, app, domain), so a repeated report replaces the row rather than
adding to it (migration 002).
### Config payload (v1)
```json
{"v":1,"device_id":1,"name":"Harry's phone","platform":"android",
"locked":true,"until":"2026-10-13T21:00:00Z","allow":[],"block":[],
"generated_at":"2026-10-13T03:30:27Z"}
```
`allow` and `block` are always arrays, never `null`. They are read from the
`allowlist` and `blocklist` tables, which get their editors in Phase 6; until
then they are empty. A client must ignore unknown fields, so this shape can grow.
### Status payload (device → service)
```json
{"online":true,"app":"com.spotify.music","at":"2026-10-13T03:30:27Z"}
```
The will the device registers on the same topic is `{"online":false}`. A report
with no `online` field counts as online, because the device just talked to us.
### Telemetry payload (device → service)
```json
{"day":"2026-10-13",
"apps":[{"name":"com.spotify.music","seconds":600}],
"domains":[{"name":"youtube.com","seconds":120}]}
```
A flat single-row form (`{"app":"…","seconds":600}`) is also accepted. Seconds
are totals for `day`, in `YYYY-MM-DD`; a missing or malformed day uses today.
List the MQTT identity of every device (tokens, no secrets):
```bash
docker exec webrestrict-control /app/control mqtt-clients
```
## Endpoints (Phase 2–3)
| Method | Path | Notes |
|---|---|---|
@@ -91,7 +171,10 @@ docker run --rm --entrypoint /app/control webrestrict/control:0.2.0-phase2 \
| `POST` | `/devices/{id}/unlock` | Unlocks now. |
Every form POST needs the `csrf` field from the session cookie. State changes
append to the `audit` table. Devices are never hard-deleted (§10 rule 10).
append to the `audit` table and republish the device's retained config. Devices
are never hard-deleted (§10 rule 10).
Phase 3 adds no HTTP endpoints.
## Development
@@ -103,16 +186,28 @@ make vet # go vet ./...
make build # build the container image
make up # docker compose up -d --build
make logs # follow container logs
make mqtt-verify # prove TLS + credentials + ACLs on a throwaway broker
```
`make test` and `make vet` run the container as your own user and keep the module
cache in `.gocache/` (gitignored), so nothing in the repository ends up owned by
root. `make mqtt-verify` never touches the shared Mosquitto: it stages a scratch
broker on `127.0.0.1:18883` and removes it afterwards.
## Layout
```
cmd/control/ service entry point (+ hash-password helper)
cmd/control/ service entry point (+ hash-password, mqtt-clients)
internal/config/ environment configuration
internal/store/ SQLite open, migrations, audit, devices + state
internal/store/migrations/001_init.sql
devices.go device registry, desired state, lock expiry
reports.go presence and telemetry written from broker reports
lists.go allow/block lists and device lookup by topic
migrations/001_init.sql, 002_telemetry_unique.sql
internal/auth/ signed-cookie session login + CSRF
internal/mqtt/ the broker link: retained config out, reports in
payload.go topic and payload contract for the kids' clients
link.go paho client, subscriptions, publish, TLS verification
internal/web/ HTTP handlers, templates, static assets
devices.go device registry + manual lock/unlock handlers
templates/ login, dashboard, device form, HTMX partials
@@ -121,6 +216,7 @@ Dockerfile own image (multi-stage, CGO-free)
docker-compose.yml own container, port 8791
environment.d/ control-service.env.example (real file gitignored)
deploy/caddy/ Caddy vhost for webrestrict.lab.audasmedia.com.au
deploy/mosquitto/ TLS listener, credentials, topic ACL, verifier
```
## Caddy
@@ -129,10 +225,12 @@ deploy/caddy/ Caddy vhost for webrestrict.lab.audasmedia.com.au
`webrestrict.lab.audasmedia.com.au`. The terminating Caddy runs on `.35`
(sam-Ubuntu1); see that file for the two proxy patterns and the apply commands.
## Non-goals (Phase 2)
## Non-goals (Phase 3)
- No MQTT publishing (Phase 3).
- No schedules, blocklists or allowlists (Phase 6).
- No telemetry or usage screens (Phase 7).
- No schedules, blocklists or allowlists (Phase 6). The config payload reads
those tables, but nothing edits them yet.
- No usage screens (Phase 7). Telemetry is stored, not yet shown.
- No admin API (Phase 8).
- No kids' clients (Phases 4–5). The topic and payload contract above is what
they implement.
- No Hub contact (Phase 9).