99 lines
5.4 KiB
Markdown
99 lines
5.4 KiB
Markdown
---
|
|
created: 2026-09-06
|
|
modified: 2026-09-06
|
|
title: Error Tracking
|
|
summary: Log of network/system errors encountered and their fixes
|
|
type: note
|
|
client: sam
|
|
project: devops
|
|
status: active
|
|
priority: 5
|
|
---
|
|
|
|
# Error Tracking
|
|
|
|
A running log of errors encountered across the network, with root causes and fixes. Append new entries as they occur.
|
|
|
|
---
|
|
|
|
# 2026-09-06 — Router: "Another management entity is configuring the Broadband Router"
|
|
|
|
## Symptom
|
|
While in the router admin (192.168.20.1), saw:
|
|
|
|
> Broadband Router Error — Another management entity is currently configuring the Broadband Router. Please close the Broadband Router Configuration window, and try again in a few minutes.
|
|
|
|
Error appeared repeatedly; also coincided with `.13` losing all network connectivity (wired + Wi-Fi + Tailscale unreachable from `.27`).
|
|
|
|
## Root cause
|
|
Not a second admin session and not a rogue device — **`.13`'s own flapping Wi-Fi adapter (`wlp4s0`) was seen by the router as a "management entity"**:
|
|
|
|
- `.13` has **two MACs / two IPs on the router table** (root of the confusion):
|
|
- **Wired** `enp42s0` = MAC **`34:5a:60:a4:cc:8c`** → `192.168.20.13` (LAN)
|
|
- **Wi-Fi** `wlp4s0` = MAC **`a0:f3:c1:d0:e3:2b`** → `192.168.20.46` (2.4G)
|
|
- Wi-Fi kept going DORMANT / re-registering and the default route flipped onto it → router interpreted the churn as a second configuring device.
|
|
- `sudo reboot` on `.13` cleared it. Error has **not returned** since.
|
|
|
|
## Verification / non-causes
|
|
- Router login log showed **no login except `.27`** → no foreign admin session.
|
|
- Client table showed only known devices (ESP32s, Pis, `.27`/`.35`/`.51`/`.13`, etc.).
|
|
- Not caused by Langfuse/Clickhouse removal (unrelated; those never touch networking).
|
|
|
|
## Potential fix — pin the Wi-Fi MAC address
|
|
If the error (or Wi-Fi flakiness on `.13`) recurs, the likely fix is to stop the Wi-Fi adapter's MAC churn:
|
|
|
|
- On `.13`, disable **MAC address randomization** in NetworkManager for `wlp4s0`, forcing a stable MAC so the router sees one consistent identity per interface:
|
|
```
|
|
# /etc/NetworkManager/conf.d/ (or via home-manager)
|
|
[connection]
|
|
wifi.cloned-mac-address=permanent # or "stable"
|
|
# per-interface: wifi.mac-randomization-disabled
|
|
```
|
|
- On NixOS this is done in `hardware-configuration.nix` / `networking` options (e.g. `networking.wireless` or NetworkManager `connection` settings in home-manager), or simply disable the Wi-Fi interface entirely if wired is preferred: `sudo nmcli radio wifi off`.
|
|
|
|
## Related notes
|
|
- `.13` network layout: `enp42s0` = wired `.13`, `wlp4s0` = Wi-Fi `.46` (2.4G). Default route prefers Wi-Fi (lower metric) — worth reviewing if flakiness returns.
|
|
- `192.168.80.0/20` bridge on `.13` is normal Docker (`wherewoof-db_default` subnet) — not an issue.
|
|
## 2026-09-06 — Voice agent: "feed the fish" did nothing, LED red
|
|
|
|
### Symptom
|
|
The voice command "feed the fish" returned no action. The ESP32 LED went red.
|
|
|
|
### Root cause
|
|
Two failures blocked the request path.
|
|
|
|
- OmniRoute combo `voice-fast` used old model IDs. The new IDs are `openai/*`.
|
|
The combo returned 503 "all accounts inactive".
|
|
- The agent MQTT intake connected once. A network flap broke that connection.
|
|
The agent did not retry, so it lost the ESP32 text feed.
|
|
|
|
### Fix
|
|
- Recreated the `voice-fast` combo with three healthy OpenRouter models.
|
|
- Changed `app.py` MQTT intake to retry and reconnect. Backup: `app.py.bak-mqtt-20260906`.
|
|
- Run `sudo systemctl restart voice-agent` to load the change.
|
|
- DeepSeek key is valid but the account balance is zero. Add credit at platform.deepseek.com.
|
|
|
|
---
|
|
|
|
# 2026-09-09 — Garage S3 web returns 404/403 for site images (anonymous hosting)
|
|
|
|
## Symptom
|
|
A Go + htmx site (being built for `kontra.day`) tried to serve uploaded images through **Garage S3 web** (`:3902`) and got `404` on the bucket root and `403 Forbidden: Garage does not support anonymous access yet` in `docker logs family-home-lab-garage-1`.
|
|
|
|
## Root cause
|
|
Two things misled the diagnosis:
|
|
|
|
1. **Wrong claim (from an earlier AI):** it said `[s3_web]` was absent and the web server wasn't running. **False** — `[s3_web]` IS in `garage.toml` (`bind_addr = "[::]:3902"`, `root_domain = ".web.lab.audasmedia.com.au"`, `index = "index.html"`) and the log shows `Web server listening on http://[::]:3902`. Verified: `ss -tln` shows 3902 bound (Garage web server), 3903 admin, 3909 garage-webui.
|
|
2. **Real issue:** Garage S3 Web is **not anonymous static hosting**. It requires **per-bucket web access + a read-scoped access key**. Without it, every request gets `403 Forbidden: Garage does not support anonymous access yet`, and the bucket root 404s with no index/perm.
|
|
|
|
## Fix / correct pattern
|
|
- **Do NOT** point site/app code at `:3902` for anonymous browsing.
|
|
- **Use the S3 API** (`:3900`, region `garage`) with a **read-scoped key** for uploads/reads.
|
|
- **Serve images** through the app or **Caddy** (reverse-proxy to Garage with the read key, or a Caddy `file_server` static dir) — not via `:3902` directly.
|
|
- If web-serving a bucket is truly needed, enable **per-bucket web access + a guest/read key** in Garage (not a global switch).
|
|
|
|
## Verification / non-causes
|
|
- `[s3_web]` present; web server confirmed running (log line + port 3902 bound) → the earlier "no s3_web / not running" diagnosis was wrong.
|
|
- Port map: 3900 S3 API · 3902 S3 web · 3903 admin API · 3909 garage-webui. No port conflict.
|
|
- Documented in the Garage section of [[Backup System — Borg, Kopia & Restic]] and the kontra.day site handoff.
|