Files
family_home_lab/docs/secret-rotation-2026-11-11.md

6.1 KiB

Secret rotation plan — 2026-11-11

Why: the compromised GIMP container had /home/sam mounted read-only for 5 days (2026-11-06 → 2026-11-11) and ran as uid 1000 (sam). Everything below was readable, and LMMS additionally had a reverse shell giving arbitrary command execution as root inside a container with a write mount into /home/sam/Docker/Containers/music.

Treat every credential listed here as compromised. See incident-2026-11-11.md.

Exposure inventory (what was readable)

Store Contents Volume
~/.ssh/id_ed25519, id_ed25519_rsync, id_ed25519_school_scan 3 private keys full host/remote access
~/.config/environment.d/10-secrets.conf 47 keys (API keys, tokens, passwords) see table below
…10-secrets.conf.bak* 12 historical copies — old AND current secrets doubles the exposure
~/.aws-credentials.local AWS access keys cloud billing
~/.secrets (1.5 KB) misc credentials varies
~/.gemini/mcp-oauth-tokens-v2.json OAuth tokens Google account scope
~/.npmrc npm registry token supply-chain publish risk
20 .env files (~200 keys total) per-project service secrets see counts below

Largest .env files: sams-home-network (43 keys), wherewoof-admin (28), voice-agent (21), family-home-lab (19), langfuse (14), outline (12).

Priority 1 — rotate TODAY (direct access or money)

Credential Why urgent Rotate where Then
id_ed25519, id_ed25519_rsync, id_ed25519_school_scan Attacker can SSH as you ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_new; remove old pubkeys from authorized_keys on every target Update .ssh/config, rsync targets, school-scan host
GITHUB_TOKEN Code push/steal, supply chain github.com → Settings → Developer settings → Tokens gh auth login to refresh
GITEA_TOKEN Same, self-hosted repos Gitea → Settings → Applications → regenerate update 10-secrets.conf
HUB_TOKEN_ADMIN, HUB_TOKEN_DSH, HUB_TOKEN_CONSOLE, HUB_TOKEN_PHONE, HUB_TOKEN_ESP32, HUB_PROJECT_*_TOKEN Full control of your hub/API layer hub admin UI → revoke + reissue per client update ESP32 firmware + phone config
SC_TOKEN Service-controller = start/stop anything regenerate in 10-secrets.conf also family-home-lab/.env, restart controller
OPENROUTER_API_KEY, OPENAI_API_KEY, DEEPSEEK_API_KEY, GEMINI_API_KEY, MOONSHOT_API_KEY, MINIMAX_API_KEY (+_GROUP_ID,_SUBSCRIPTION_KEY), OPENCODE_API_KEY, TOKENROUTER_API_KEY, OMNIROUTE_API_KEY, STITCH_API_KEY, JEV_API_KEY, MOUSE_API_KEY, OL_API_KEY, TYPESAFE_API_KEY, TAVILY_API_KEY Billing — check usage dashboards for abuse each provider console update 10-secrets.conf + dsh configs
GMAIL_APP_PASSWORD, EMAIL_AGENT_PASSWORD, EMAIL_AGENT_SESSION_SECRET Mailbox takeover, spam/phishing from your address Google account → App passwords → revoke all update voice-agent + email_agent
COMPASS_PASSWORD, PASEO_PASSWORD Account takeover respective apps update .env/conf
MQTT_USER/MQTT_PASS Control smart-home devices Mosquitto passwd file update all ESP/HA clients
AWS keys in ~/.aws-credentials.local Cloud spend, data IAM → deactivate access key → create new update profile
~/.npmrc token Publish malicious packages under your name npmjs.com → Access Tokens → revoke npm login
~/.gemini/mcp-oauth-tokens-v2.json Google scope revoke at myaccount.google.com → Security → Third-party access re-auth
VIKUNJA_TOKEN, N8N_API_KEY, N8N_MCP_TOKEN, PAPERCLIP_API_KEY, PAPERCLIP_BOARD_KEY, APPRISE_URLS Task/automation control; apprise URLs embed tokens each app's settings update consumers

Priority 2 — rotate this week (internal blast radius)

  • POSTGRES_PASSWORD / POSTGRES_USER / POSTGRES_DB (family-home-lab)
  • SESSION_SECRET, ADMIN_PASSWORD (portal — invalidate all sessions, log everyone out)
  • GARAGE_RPC_SECRET, GARAGE_ADMIN_TOKEN, GARAGE_METRICS_TOKEN, S3_ACCESS_KEY, S3_SECRET_KEY
  • RABBITMQ / REDIS passwords if set in .env
  • langfuse, outline, worldmonitor, omniroute, wherewoof-* DB/API keys
  • WEBTOP_PASSWORD (new, created after the incident — not exposed, no action)

Priority 3 — no action (not secrets)

MOZ_ENABLE_WAYLAND, GITEA_URL, GMAIL_USER, COMPASS_USERNAME, HUB_PROJECTS, PASEO_HOST, PI_NTFY_TOPIC (topic name only), ports/paths.

Hygiene fixes (do alongside rotation)

  1. Delete the 12 10-secrets.conf.bak* files — they hold superseded secrets and multiply exposure. Keep one encrypted backup instead.
  2. chmod 600 every file under ~/.config/environment.d and confirm ~/.ssh/id_* are 600 (currently correct; .pub files at 644 are fine).
  3. Never mount /home/sam into a container again — removed from GIMP already. Mount only the specific directory needed.
  4. Consider sops-nix / agenix so secrets are encrypted at rest, and so a read-only home mount can never expose them in plaintext.
  5. Audit .env files for values that don't need to be secrets at all.

Rotation order that avoids downtime

  1. Revoke first (kills attacker access), even if it breaks things for a few minutes.
  2. Regenerate + write into 10-secrets.conf.
  3. systemctl --user daemon-reload / re-login for environment.d to apply.
  4. Restart consumers (portal, controller, dsh, voice-agent, n8n, worldmonitor…).
  5. Verify each service health endpoint, then update the .env copies.
  6. Only then delete the old .bak files.

Verification checklist

  • ssh -T git@github.com works with a new key; old key rejected on target hosts
  • Gitea push works with new token
  • Portal login works; old sessions invalidated (forced logout)
  • Service-controller /health OK with new SC_TOKEN
  • dsh + models respond (new AI provider keys)
  • MQTT devices reconnect (new broker creds)
  • Email agent sends a test message
  • No unexpected spend on provider billing dashboards for 2026-11-06 → 11-11