Follow-up to the Phase 3 commit (14b304e), after Sam accepted the confirmation
on AUD-80. The operational change itself lives outside this repository; this
commit updates the documentation so it matches reality.
Applied on .13 (backups: mosquitto.conf.bak.aud80-20261013-152648 and
docker-compose.yml.bak.aud80-20261013-152648):
- mosquitto.conf gains per_listener_settings true and
include_dir /mosquitto/config/conf.d; its original 1883 listener and
allow_anonymous true are untouched.
- docker-compose.yml publishes 8883 and mounts the new conf.d and tls
directories (the compose mounts mosquitto.conf as a single file, so the
include_dir fails without them).
- conf.d/webrestrict.{conf,acl,passwd} and tls/{ca.crt,server.crt,server.key}
are installed from deploy/mosquitto/.
- The Control Service now uses tls://host.docker.internal:8883 with the
control-service account and CONTROL_MQTT_INSECURE=false.
Verified live after the restart: the 1883 plane still carries anonymous traffic,
8883 refuses anonymous clients and a wrong password, each device reads only its
own retained config, a device cannot write another device's status or its own
config, and a client killed without DISCONNECT drove the retained will into an
offline transition recorded in SQLite.
Docs: deploy/mosquitto/README.md records the exact host diff, the runbook gap
that caused one restart-loop (the missing bind mounts), the per_listener_settings
deprecation caveat and the rollback. README.md and TRANSPORT_DESIGN.md now
describe the applied state.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
227 lines
9.8 KiB
Markdown
227 lines
9.8 KiB
Markdown
# Phase 3 MQTT security for the shared Mosquitto
|
|
|
|
> **APPLIED 2026-10-13 (AUD-80).** The shared Mosquitto on `.13` now runs the
|
|
> 8883 listener below next to its original anonymous `1883` listener, and the
|
|
> Control Service connects over TLS with credentials. Backups:
|
|
> `/home/sam/Docker/Containers/mosquitto/mosquitto.conf.bak.aud80-20261013-152648`
|
|
> and `docker-compose.yml.bak.aud80-20261013-152648`. See "What was changed on
|
|
> `.13`" at the end of this file.
|
|
|
|
This directory holds the **broker-side** half of Phase 3 (AGENTS.md §11, §10
|
|
rule 6; `TRANSPORT_DESIGN.md` "Security"). The Go half lives in
|
|
`internal/mqtt/`.
|
|
|
|
The transport itself — retained config per device, status and telemetry
|
|
subscriptions, LWT presence — is implemented and running against the shared
|
|
Mosquitto on `.13:1883`. What follows is the credential and TLS layer that
|
|
onboards the kids' devices.
|
|
|
|
## What is here
|
|
|
|
| File | What it is |
|
|
|---|---|
|
|
| `conf.d/webrestrict.conf` | the TLS listener on 8883 with authentication |
|
|
| `conf.d/webrestrict.acl` | the topic ACL: one device, one topic set |
|
|
| `conf.d/webrestrict.passwd` | **generated** hashed accounts (gitignored) |
|
|
| `generate-tls.sh` | builds the lab CA and the broker certificate |
|
|
| `build-config.sh` | writes the scratch broker config used by the verifier |
|
|
| `provision-clients.sh` | creates one broker account per registered device |
|
|
| `verify-isolation.sh` | proves the ACL on a throwaway broker |
|
|
| `clients.env` | **generated** device credentials (gitignored) |
|
|
|
|
## The design
|
|
|
|
Two listeners on the one broker. No second broker (§10 rule 6), no router
|
|
change, no DNS change.
|
|
|
|
| Listener | Auth | Who uses it |
|
|
|---|---|---|
|
|
| `1883` (existing, unchanged) | anonymous | n8n and the rest of the lab |
|
|
| `8883` (new, TLS) | required account | the kids' devices and the Control Service |
|
|
|
|
`per_listener_settings true` is what keeps the two planes apart: the plain
|
|
listener stays anonymous, the TLS listener requires credentials. It must appear
|
|
in the main `mosquitto.conf` **before the first `listener` line**.
|
|
|
|
### Topic ACL
|
|
|
|
```acl
|
|
user control-service
|
|
topic write restrict/+/config
|
|
topic read restrict/+/status
|
|
topic read restrict/+/telemetry
|
|
|
|
pattern read restrict/%u/config
|
|
pattern write restrict/%u/status
|
|
pattern write restrict/%u/telemetry
|
|
```
|
|
|
|
- Only the Control Service publishes config (§10 rule 5). It cannot read config
|
|
and cannot write status or telemetry.
|
|
- A device's MQTT username **is** its topic token — the middle level of its
|
|
config topic. For `restrict/harry_phone_2/config` the account is
|
|
`harry_phone_2`.
|
|
- `%u` expands to that username, so a device can only ever touch its own topics.
|
|
Adding a device is therefore one account, no ACL edit.
|
|
- A device publishing to its own config topic is dropped by the broker, so a
|
|
child cannot publish a fake "unlocked".
|
|
|
|
List the tokens the Control Service expects:
|
|
|
|
```bash
|
|
docker exec webrestrict-control /app/control mqtt-clients
|
|
```
|
|
|
|
## Prove it without touching anything shared
|
|
|
|
```bash
|
|
make mqtt-verify # TLS + CA + scratch broker on 127.0.0.1:18883
|
|
```
|
|
|
|
`verify-isolation.sh` stages a throwaway copy of the config, runs its own
|
|
Mosquitto container on a spare port, and asserts the whole policy: the plain
|
|
listener is still anonymous, the TLS listener rejects anonymous clients and a
|
|
wrong password, a device reads only its own config, writes only its own status
|
|
and telemetry, cannot write another device's topics, and **cannot overwrite its
|
|
own config to fake an unlock**.
|
|
|
|
## Applying it (approval required first)
|
|
|
|
Applying it changes files outside this project (`mosquitto.conf` and the
|
|
Mosquitto `docker-compose.yml`), adds port 8883 to a family-facing service, and
|
|
restarts that service — a change to a shared household service, so it needs
|
|
Sam's approval first. Everything below is additive and a backup is taken before
|
|
anything is written.
|
|
|
|
```bash
|
|
# 1. Credentials and certificates.
|
|
docker exec webrestrict-control /app/control mqtt-clients # tokens, no secrets
|
|
deploy/mosquitto/provision-clients.sh # writes conf.d/webrestrict.passwd + clients.env
|
|
make mqtt-tls # writes tls/{ca,server}.{crt,key}
|
|
|
|
# 2. Copy the material into the shared broker's config directory.
|
|
BROKER_DIR=/home/sam/Docker/Containers/mosquitto
|
|
sudo -u sam cp "$BROKER_DIR/mosquitto.conf" \
|
|
"$BROKER_DIR/mosquitto.conf.bak.$(date +%Y%m%d-%H%M%S)"
|
|
install -d -m 755 "$BROKER_DIR/conf.d" "$BROKER_DIR/tls"
|
|
install -m 644 deploy/mosquitto/conf.d/webrestrict.conf "$BROKER_DIR/conf.d/"
|
|
install -m 644 deploy/mosquitto/conf.d/webrestrict.acl "$BROKER_DIR/conf.d/"
|
|
install -m 644 deploy/mosquitto/conf.d/webrestrict.passwd "$BROKER_DIR/conf.d/"
|
|
install -m 644 deploy/mosquitto/tls/* "$BROKER_DIR/tls/"
|
|
# then add to mosquitto.conf (leave `listener 1883 0.0.0.0` and
|
|
# `allow_anonymous true` exactly as they are):
|
|
# per_listener_settings true (before the first listener)
|
|
# include_dir /mosquitto/config/conf.d
|
|
|
|
# 3. Mount the new directories, publish 8883, restart, bounded.
|
|
# The shared compose mounts mosquitto.conf as a single file, so the new
|
|
# directories need their own lines or the broker will not start
|
|
# ("Error: Unable to open include_dir"). Add to the mosquitto service:
|
|
#
|
|
# ports:
|
|
# - "8883:8883"
|
|
# volumes:
|
|
# - /home/sam/Docker/Containers/mosquitto/conf.d:/mosquitto/config/conf.d:ro
|
|
# - /home/sam/Docker/Containers/mosquitto/tls:/mosquitto/config/tls:ro
|
|
#
|
|
# then:
|
|
cd "$BROKER_DIR"
|
|
timeout 120 docker compose up -d
|
|
timeout 30 docker logs --tail 20 mosquitto # expect both listeners to open
|
|
|
|
# 4. Prove the plain plane still works (n8n depends on it).
|
|
timeout 40 docker run --rm --network host --entrypoint sh eclipse-mosquitto:latest -c \
|
|
'mosquitto_sub -h 127.0.0.1 -p 1883 -t lab/after -C 1 -W 10 & sleep 1; \
|
|
mosquitto_pub -h 127.0.0.1 -p 1883 -t lab/after -m ok; wait'
|
|
|
|
# 5. Point the Control Service at TLS.
|
|
# environment.d/control-service.env:
|
|
# CONTROL_MQTT_BROKER=tls://host.docker.internal:8883
|
|
# CONTROL_MQTT_INSECURE=false
|
|
# CONTROL_MQTT_USERNAME=control-service
|
|
# CONTROL_MQTT_PASSWORD=<from clients.env>
|
|
timeout 120 docker compose up -d --build
|
|
timeout 30 docker logs --tail 30 webrestrict-control # expect "mqtt connected" over tls
|
|
```
|
|
|
|
### Notes from the live apply (2026-10-13)
|
|
|
|
- The broker logs `The 'per_listener_settings' option is now deprecated and will
|
|
be removed in version 3.0` and the matching `allow_anonymous` notice. It still
|
|
works on the `eclipse-mosquitto:latest` image, which is 2.1.0.1 today.
|
|
`per_listener_settings true` is kept because it is the only way on 2.1.0 to
|
|
give the two listeners different `password_file`/`acl_file` values: the
|
|
per-listener `listener_*` prefix exists only for `allow_anonymous`. **Pin the
|
|
image tag before Mosquitto 3.0 arrives**, or rework the two planes.
|
|
- The bind mounts in step 3 are not optional. The shared compose mounts
|
|
`mosquitto.conf` as a single file, so `include_dir` fails without them and the
|
|
broker restart-loops.
|
|
- The password file and the TLS key are installed mode 0644 because the broker
|
|
runs as uid 1883 and that user does not exist on the host. See `tls/README.md`.
|
|
|
|
### Roll back
|
|
|
|
```bash
|
|
cd /home/sam/Docker/Containers/mosquitto
|
|
cp mosquitto.conf.bak.aud80-<timestamp> mosquitto.conf
|
|
cp docker-compose.yml.bak.aud80-<timestamp> docker-compose.yml
|
|
rm -rf conf.d tls
|
|
timeout 120 docker compose up -d
|
|
```
|
|
|
|
Rolling back the broker needs no change in this project: set
|
|
`CONTROL_MQTT_INSECURE=true` and `CONTROL_MQTT_BROKER=tcp://host.docker.internal:1883`
|
|
in `environment.d/control-service.env` and restart the service.
|
|
|
|
## What was changed on `.13` (2026-10-13)
|
|
|
|
Files outside this project were edited for the first time. The backup copies are
|
|
listed above; everything is additive.
|
|
|
|
`/home/sam/Docker/Containers/mosquitto/mosquitto.conf` — the two original lines
|
|
(`listener 1883 0.0.0.0`, `allow_anonymous true`) are untouched:
|
|
|
|
```diff
|
|
log_dest file /mosquitto/log/mosquitto.log
|
|
+
|
|
+# Internet Use Control (AUD-80). Two listeners, two auth planes. Mosquitto
|
|
+# requires per_listener_settings before the first listener, and it is what keeps
|
|
+# the 1883 listener below anonymous while 8883 requires an account.
|
|
+per_listener_settings true
|
|
+
|
|
listener 1883 0.0.0.0
|
|
allow_anonymous true # IMPORTANT: For production, configure authentication (username/password)
|
|
+
|
|
+# Internet Use Control device plane (AUD-80): TLS on 8883, account required,
|
|
+# topic ACL. Roll back by deleting this line and the conf.d directory.
|
|
+include_dir /mosquitto/config/conf.d
|
|
```
|
|
|
|
`/home/sam/Docker/Containers/mosquitto/docker-compose.yml`:
|
|
|
|
```diff
|
|
ports:
|
|
- "1883:1883" # Standard MQTT port
|
|
+ - "8883:8883" # Internet Use Control device plane (TLS, AUD-80)
|
|
- "9001:9001" # Websockets port (optional, good for web clients)
|
|
volumes:
|
|
- /home/sam/Docker/Containers/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
|
|
+ - /home/sam/Docker/Containers/mosquitto/conf.d:/mosquitto/config/conf.d:ro
|
|
+ - /home/sam/Docker/Containers/mosquitto/tls:/mosquitto/config/tls:ro
|
|
- /home/sam/Docker/Containers/mosquitto/data:/mosquitto/data
|
|
- /home/sam/Docker/Containers/mosquitto/log:/mosquitto/log
|
|
```
|
|
|
|
New on the host, copied from this directory:
|
|
`conf.d/webrestrict.{conf,acl,passwd}` and `tls/{ca.crt,server.crt,server.key}`.
|
|
|
|
`environment.d/control-service.env` in this project now points at
|
|
`tls://host.docker.internal:8883` with the `control-service` account and
|
|
`CONTROL_MQTT_INSECURE=false`.
|
|
|
|
Verified on the live broker after the restart: the `1883` plane still carries
|
|
anonymous traffic, `8883` refuses anonymous clients and a wrong password, each
|
|
device reads only its own retained config, a device's write to another device's
|
|
status and to its own config is dropped, and a client killed without DISCONNECT
|
|
drove the retained will into an offline transition in the Control Service.
|