diff --git a/docs/secret-rotation-2026-11-11.md b/docs/secret-rotation-2026-11-11.md new file mode 100644 index 0000000..7754811 --- /dev/null +++ b/docs/secret-rotation-2026-11-11.md @@ -0,0 +1,88 @@ +# Secret rotation plan — 2026-11-11 + +**Why:** the compromised GIMP container had `/home/sam` mounted **read-only for 5 days** +(2026-11-06 → 2026-11-11) and ran as uid 1000 (`sam`). Everything below was **readable**, +and LMMS additionally had a reverse shell giving arbitrary command execution as root inside +a container with a write mount into `/home/sam/Docker/Containers/music`. + +Treat **every credential listed here as compromised**. See `incident-2026-11-11.md`. + +## Exposure inventory (what was readable) + +| Store | Contents | Volume | +|---|---|---| +| `~/.ssh/id_ed25519`, `id_ed25519_rsync`, `id_ed25519_school_scan` | **3 private keys** | full host/remote access | +| `~/.config/environment.d/10-secrets.conf` | **47 keys** (API keys, tokens, passwords) | see table below | +| …`10-secrets.conf.bak*` | **12 historical copies** — old AND current secrets | doubles the exposure | +| `~/.aws-credentials.local` | AWS access keys | cloud billing | +| `~/.secrets` (1.5 KB) | misc credentials | varies | +| `~/.gemini/mcp-oauth-tokens-v2.json` | OAuth tokens | Google account scope | +| `~/.npmrc` | npm registry token | supply-chain publish risk | +| **20 `.env` files** (~200 keys total) | per-project service secrets | see counts below | + +Largest `.env` files: `sams-home-network` (43 keys), `wherewoof-admin` (28), `voice-agent` (21), +`family-home-lab` (19), `langfuse` (14), `outline` (12). + +## Priority 1 — rotate TODAY (direct access or money) + +| Credential | Why urgent | Rotate where | Then | +|---|---|---|---| +| `id_ed25519`, `id_ed25519_rsync`, `id_ed25519_school_scan` | Attacker can SSH as you | `ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_new`; remove old pubkeys from `authorized_keys` on every target | Update `.ssh/config`, rsync targets, school-scan host | +| `GITHUB_TOKEN` | Code push/steal, supply chain | github.com → Settings → Developer settings → Tokens | `gh auth login` to refresh | +| `GITEA_TOKEN` | Same, self-hosted repos | Gitea → Settings → Applications → regenerate | update `10-secrets.conf` | +| `HUB_TOKEN_ADMIN`, `HUB_TOKEN_DSH`, `HUB_TOKEN_CONSOLE`, `HUB_TOKEN_PHONE`, `HUB_TOKEN_ESP32`, `HUB_PROJECT_*_TOKEN` | Full control of your hub/API layer | hub admin UI → revoke + reissue per client | update ESP32 firmware + phone config | +| `SC_TOKEN` | Service-controller = start/stop anything | regenerate in `10-secrets.conf` | also `family-home-lab/.env`, restart controller | +| `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, `DEEPSEEK_API_KEY`, `GEMINI_API_KEY`, `MOONSHOT_API_KEY`, `MINIMAX_API_KEY` (+`_GROUP_ID`,`_SUBSCRIPTION_KEY`), `OPENCODE_API_KEY`, `TOKENROUTER_API_KEY`, `OMNIROUTE_API_KEY`, `STITCH_API_KEY`, `JEV_API_KEY`, `MOUSE_API_KEY`, `OL_API_KEY`, `TYPESAFE_API_KEY`, `TAVILY_API_KEY` | **Billing** — check usage dashboards for abuse | each provider console | update `10-secrets.conf` + dsh configs | +| `GMAIL_APP_PASSWORD`, `EMAIL_AGENT_PASSWORD`, `EMAIL_AGENT_SESSION_SECRET` | Mailbox takeover, spam/phishing from your address | Google account → App passwords → revoke all | update voice-agent + email_agent | +| `COMPASS_PASSWORD`, `PASEO_PASSWORD` | Account takeover | respective apps | update `.env`/conf | +| `MQTT_USER`/`MQTT_PASS` | Control smart-home devices | Mosquitto `passwd` file | update all ESP/HA clients | +| `AWS` keys in `~/.aws-credentials.local` | Cloud spend, data | IAM → deactivate access key → create new | update profile | +| `~/.npmrc` token | Publish malicious packages under your name | npmjs.com → Access Tokens → revoke | `npm login` | +| `~/.gemini/mcp-oauth-tokens-v2.json` | Google scope | revoke at myaccount.google.com → Security → Third-party access | re-auth | +| `VIKUNJA_TOKEN`, `N8N_API_KEY`, `N8N_MCP_TOKEN`, `PAPERCLIP_API_KEY`, `PAPERCLIP_BOARD_KEY`, `APPRISE_URLS` | Task/automation control; apprise URLs embed tokens | each app's settings | update consumers | + +## Priority 2 — rotate this week (internal blast radius) + +- `POSTGRES_PASSWORD` / `POSTGRES_USER` / `POSTGRES_DB` (family-home-lab) +- `SESSION_SECRET`, `ADMIN_PASSWORD` (portal — invalidate all sessions, log everyone out) +- `GARAGE_RPC_SECRET`, `GARAGE_ADMIN_TOKEN`, `GARAGE_METRICS_TOKEN`, `S3_ACCESS_KEY`, `S3_SECRET_KEY` +- `RABBITMQ` / `REDIS` passwords if set in `.env` +- `langfuse`, `outline`, `worldmonitor`, `omniroute`, `wherewoof-*` DB/API keys +- `WEBTOP_PASSWORD` (new, created after the incident — **not** exposed, no action) + +## Priority 3 — no action (not secrets) + +`MOZ_ENABLE_WAYLAND`, `GITEA_URL`, `GMAIL_USER`, `COMPASS_USERNAME`, `HUB_PROJECTS`, +`PASEO_HOST`, `PI_NTFY_TOPIC` (topic name only), ports/paths. + +## Hygiene fixes (do alongside rotation) + +1. **Delete the 12 `10-secrets.conf.bak*` files** — they hold superseded secrets and multiply + exposure. Keep **one** encrypted backup instead. +2. `chmod 600` every file under `~/.config/environment.d` and confirm `~/.ssh/id_*` are `600` + (currently correct; `.pub` files at 644 are fine). +3. **Never mount `/home/sam` into a container again** — removed from GIMP already. Mount only + the specific directory needed. +4. Consider `sops-nix` / `agenix` so secrets are encrypted at rest, and so a read-only home + mount can never expose them in plaintext. +5. Audit `.env` files for values that don't need to be secrets at all. + +## Rotation order that avoids downtime + +1. Revoke first (kills attacker access), even if it breaks things for a few minutes. +2. Regenerate + write into `10-secrets.conf`. +3. `systemctl --user daemon-reload` / re-login for `environment.d` to apply. +4. Restart consumers (portal, controller, dsh, voice-agent, n8n, worldmonitor…). +5. Verify each service health endpoint, then update the `.env` copies. +6. Only then delete the old `.bak` files. + +## Verification checklist + +- [ ] `ssh -T git@github.com` works with a new key; old key rejected on target hosts +- [ ] Gitea push works with new token +- [ ] Portal login works; old sessions invalidated (forced logout) +- [ ] Service-controller `/health` OK with new `SC_TOKEN` +- [ ] dsh + models respond (new AI provider keys) +- [ ] MQTT devices reconnect (new broker creds) +- [ ] Email agent sends a test message +- [ ] No unexpected spend on provider billing dashboards for 2026-11-06 → 11-11