network_security/AGENTS.md

Network Security

Build spec for Paperclip. Follow the phases in section 11 in order. Do not skip a phase's Definition of Done. When something is unclear, ask Sam — do not guess.

---

1. Purpose and outcome

Make the lab fail closed. Today a new service is exposed to the internet the moment it exists, and the only thing protecting it is remembering to add auth. That is backwards.

Outcome (the single success statement):

Exposing a service is a deliberate act. By default it is protected — and scans against the lab are blocked and visible, not absorbed.

2. Scope

In scope

• Move admin UIs off the public internet.
• One authentication layer in front of the whole *.lab wildcard.
• Intrusion detection and auto-banning.
• Secret scanning, code scanning, image scanning.
• A correct-by-default auth helper for our own FastAPI apps.

Out of scope

• Rewriting application features.
• Changing what *.lab hosts are — only how they are reached.

3. Why this exists — the incident

On 2026-10-13 an automated scanner found console.lab.audasmedia.com.au on the public internet and exploited a critical authentication bypass in the family console:

if username is None and basic is not None:
    username = basic.username     # password never verified

HTTPBasic only parses the header. The code trusted the username alone, so curl -u sam:anything authenticated as admin. The attacker used it to drive Jervis (the house voice) with secret-exfiltration prompts for two days, unnoticed.

The bypass is fixed. This project fixes the conditions that made it matter.

4. Prerequisites

Need	Status / note
Caddy	.35, already the single reverse proxy for *.lab
Tailscale	Installed on .13, .27, .51, and the phone (sams-pixel-8a)
Docker	Everywhere it is needed
A decision	*.lab stays public for genuine public sites. Only admin UIs move.

5. Functional requirements

ID	Requirement
FR-1	Inventory. Produce a table of every *.lab host: what it is, its backend, and whether it has auth.
FR-2	Admin UIs are not public. Console, Portainer, Gitea, Paperclip, and every other admin surface are reachable only over Tailscale.
FR-3	Default-deny at Caddy. Every *.lab host passes through forward-auth. A host is protected unless it is on an explicit public allowlist.
FR-4	SSO + 2FA (Authelia) in front of the wildcard, with per-host policy.
FR-5	CrowdSec reads the Caddy and sshd logs, detects scans, and auto-bans via a Caddy bouncer.
FR-6	Secret scanning. gitleaks as a pre-commit hook on every repo; trufflehog over existing history.
FR-7	Code scanning. semgrep in CI/pre-commit for our own code.
FR-8	Image scanning. trivy over the running container images.
FR-9	A correct-by-default auth helper for FastAPI apps, plus a test asserting valid username + wrong password → 401.
FR-10	A baseline scan with nuclei against .lab, so we find issues before the bots do.

6. Screens and UX flow

• Authelia portal — login + 2FA page in front of protected hosts.
• CrowdSec dashboard — who is being banned, and why.
• The inventory table (FR-1) — a document, not a UI.

7. Look and feel

Reuse the Family Home Lab design system for any page we build ourselves.

• /home/sam/Docker/Containers/family-home-lab/DESIGN.md
• /home/sam/Docker/Containers/family-home-lab/portal/static/tokens.css

Authelia and CrowdSec ship their own UIs — do not restyle them.

8. Integrations

System	Use
Caddy (.35)	forward-auth, the CrowdSec bouncer, the public allowlist
Tailscale	admin access without public exposure
Authelia	SSO + 2FA
CrowdSec	log parsing + decisions + bouncer
Apprise (.35:8210)	alerts on bans and scan spikes
Gitea	pre-commit hooks, CI

9. Technology (locked)

Item	Value
Reverse proxy	Caddy (existing, .35)
Auth	Authelia (forward-auth)
IDS/IPS	CrowdSec + Caddy bouncer
Remote access	Tailscale
Secret scan	gitleaks (pre-commit), trufflehog (history)
SAST	semgrep
Image scan	trivy
DAST	nuclei
Container	One per service. Own folder, own compose file.

10. Rules

1. Fail closed. A new host is protected by default; exposure is explicit.
2. Never remove auth to "fix" a problem. Fix the auth instead.
3. Do not break genuinely public sites — they go on the allowlist.
4. Alerts go through Apprise.
5. One commit and push per phase.
6. Bound every docker command (timeout, --entrypoint sh when running a shell inside an image).

11. Build order

Phase 1 — Inventory Table of every *.lab host: purpose, backend, auth, public or not. DoD: the table exists and is complete. Nothing changes yet.

Phase 2 — Admin UIs off the public internet Move admin surfaces to Tailscale-only. Console, Portainer, Gitea, Paperclip first. DoD: admin UIs are unreachable from the public internet and reachable over Tailscale — proven both ways.

Phase 3 — CrowdSec + Caddy bouncer DoD: a deliberate scan from an external host is banned, and the ban is visible in the dashboard.

Phase 4 — Authelia forward-auth + allowlist DoD: a newly created *.lab host is protected without anyone remembering to add auth. Public sites still work.

Phase 5 — Secret scanning gitleaks pre-commit on every repo; trufflehog over history. DoD: findings triaged; every real leak rotated.

Phase 6 — Code and image scanning semgrep + trivy. DoD: both run in CI; findings triaged.

Phase 7 — Auth helper + test A shared FastAPI dependency that verifies the password, plus the test. DoD: the test fails against the old bypass pattern and passes against the fix.

Phase 8 — Baseline scan + documentation nuclei against .lab; write it all up. DoD: no high findings unreviewed; docs published.

12. Acceptance criteria

• No admin UI is reachable from the public internet.
• A new *.lab host is protected by default.
• A scan from outside is banned automatically and visible.
• Repo history is scanned for secrets and findings are rotated.
• Semgrep, trivy, and the auth test run in CI.
• A nuclei baseline exists.

13. Decisions and open questions

Resolved:

1. *.lab stays public for genuine public sites. Only admin UIs move.
2. Scanning is accepted as normal — the goal is control, not zero noise.
3. Authelia over Authentik (lighter, fits Caddy forward-auth).

Remaining:

1. Which hosts are genuinely public? (Needs Sam's list for the allowlist.)
2. Should CrowdSec run in ban mode from the start, or observe first?
3. Where do the alerts go — Apprise to which channel?
4. Do we add nuclei/semgrep to Gitea CI, or run them by hand?