network_security/AGENTS.md

# Network Security

> Build spec for Paperclip. Follow the phases in section 11 in order. Do not skip a
> phase's Definition of Done. When something is unclear, ask Sam — do not guess.

---

## 1. Purpose and outcome

Make the lab **fail closed**. Today a new service is exposed to the internet the
moment it exists, and the only thing protecting it is remembering to add auth.
That is backwards.

**Outcome (the single success statement):**

> Every service stays reachable from anywhere — but reaching it means
authenticating. Scans against the lab are blocked and visible, not absorbed.

## 2. Scope

**In scope**
- **Authenticate** the public surface — reachability stays, exposure does not.
- One authentication layer in front of the `*.lab` wildcard, with a short explicit
  unauthenticated allowlist.
- Intrusion detection and auto-banning.
- Secret scanning, code scanning, image scanning.
- A correct-by-default auth helper for our own FastAPI apps.

**Out of scope**
- Rewriting application features.
- Changing what `*.lab` hosts *are* — only how they are reached.

## 3. Why this exists — the incident

On **2026-10-13** an automated scanner found `console.lab.audasmedia.com.au` on the
public internet and exploited a **critical authentication bypass** in the family
console:

```python
if username is None and basic is not None:
    username = basic.username     # password never verified
```

`HTTPBasic` only parses the header. The code trusted the username alone, so
`curl -u sam:anything` authenticated as admin. The attacker used it to drive
Jervis (the house voice) with secret-exfiltration prompts for two days, unnoticed.

The bypass is fixed. **This project fixes the conditions that made it matter.**

## 4. Prerequisites

| Need | Status / note |
|---|---|
| Caddy | `.35`, already the single reverse proxy for `*.lab` |
| Tailscale | Installed on `.13`, `.27`, `.51`, and the phone (`sams-pixel-8a`) |
| Docker | Everywhere it is needed |
| A decision | `*.lab` stays **public** for genuine public sites. Only admin UIs move. |

## 5. Functional requirements

| ID | Requirement |
|---|---|
| FR-1 | **Inventory.** Produce a table of every `*.lab` host: what it is, its backend, and whether it has auth. |
| FR-2 | **Every public host is authenticated.** Console, Paperclip, Gitea and the rest stay **reachable from the internet**, because they must work from any machine with no client software. They sit behind **forward-auth (SSO + 2FA)**, or their own login where that is stronger. Only an explicit allowlist is unauthenticated (`maps.lab`). |
| FR-3 | **Default-deny at Caddy.** Every `*.lab` host passes through **forward-auth**. A host is protected unless it is on an explicit public allowlist. |
| FR-4 | **SSO + 2FA** (Authelia) in front of the wildcard, with per-host policy. |
| FR-5 | **CrowdSec** reads the Caddy and sshd logs, detects scans, and **auto-bans** via a Caddy bouncer. |
| FR-6 | **Secret scanning.** `gitleaks` as a pre-commit hook on every repo; `trufflehog` over existing history. |
| FR-7 | **Code scanning.** `semgrep` in CI/pre-commit for our own code. |
| FR-8 | **Image scanning.** `trivy` over the running container images. |
| FR-9 | **A correct-by-default auth helper** for FastAPI apps, plus a test asserting **valid username + wrong password → 401**. |
| FR-10 | **A baseline scan** with `nuclei` against `.lab`, so we find issues before the bots do. |

## 6. Screens and UX flow

- **Authelia portal** — login + 2FA page in front of protected hosts.
- **CrowdSec dashboard** — who is being banned, and why.
- **The inventory table** (FR-1) — a document, not a UI.

## 7. Look and feel

Reuse the Family Home Lab design system for any page we build ourselves.

- `/home/sam/Docker/Containers/family-home-lab/DESIGN.md`
- `/home/sam/Docker/Containers/family-home-lab/portal/static/tokens.css`

Authelia and CrowdSec ship their own UIs — do not restyle them.

## 8. Integrations

| System | Use |
|---|---|
| **Caddy** (`.35`) | forward-auth, the CrowdSec bouncer, the public allowlist |
| **Tailscale** | admin access without public exposure |
| **Authelia** | SSO + 2FA |
| **CrowdSec** | log parsing + decisions + bouncer |
| **Apprise** (`.35:8210`) | alerts on bans and scan spikes |
| **Gitea** | pre-commit hooks, CI |

## 9. Technology (locked)

| Item | Value |
|---|---|
| Reverse proxy | **Caddy** (existing, `.35`) |
| Auth | **Authelia** (forward-auth) |
| IDS/IPS | **CrowdSec** + Caddy bouncer |
| Remote access | **Tailscale** |
| Secret scan | **gitleaks** (pre-commit), **trufflehog** (history) |
| SAST | **semgrep** |
| Image scan | **trivy** |
| DAST | **nuclei** |
| Container | One per service. Own folder, own compose file. |

## 10. Rules

1. **Fail closed means authenticated, not hidden.** Public reachability is
expected and required. A new host is protected **by default**; the unauthenticated
public allowlist is explicit and short.
2. **Never remove auth to "fix" a problem.** Fix the auth instead.
3. Do not break genuinely public sites — they go on the allowlist.
4. Alerts go through **Apprise**.
5. One commit and push per phase.
6. Bound every docker command (`timeout`, `--entrypoint sh` when running a shell
   inside an image).

## 11. Build order

**Phase 1 — Inventory**
Table of every `*.lab` host: purpose, backend, auth, public or not.
*DoD:* the table exists and is complete. **Nothing changes yet.**

**Phase 2 — Protect the public surface**
Put **Authelia forward-auth (SSO + 2FA)** in front of the sensitive hosts. They
stay **publicly reachable** — that is the point. Console, Paperclip and Gitea must
work from any browser, on any machine, with no client software installed.

Per-host policy:

| Host | Policy |
|---|---|
| console, Paperclip, Portainer | forward-auth + **2FA** (owner only) |
| Gitea | **public read** for anonymous viewers; forward-auth + 2FA for **write** |
| `maps.lab` | public allowlist, **no auth** — Obsidian and other apps embed it |
| anything not needed publicly | simply not routed publicly |

*DoD:* signing in from a machine **with no Tailscale and no VPN** works. An
unauthenticated request to a protected host is refused. Public read still works on
Gitea and maps.

**Phase 3 — CrowdSec + Caddy bouncer**
*DoD:* a deliberate scan from an external host is **banned**, and the ban is visible
in the dashboard.

**Phase 4 — Authelia forward-auth + allowlist**
*DoD:* a **newly created** `*.lab` host is protected **without anyone remembering
to add auth**. Public sites still work.

**Phase 5 — Secret scanning**
`gitleaks` pre-commit on every repo; `trufflehog` over history.
*DoD:* findings triaged; every real leak rotated.

**Phase 6 — Code and image scanning**
`semgrep` + `trivy`.
*DoD:* both run in CI; findings triaged.

**Phase 7 — Auth helper + test**
A shared FastAPI dependency that verifies the password, plus the test.
*DoD:* the test **fails** against the old bypass pattern and passes against the fix.

**Phase 8 — Baseline scan + documentation**
`nuclei` against `.lab`; write it all up.
*DoD:* no high findings unreviewed; docs published.

## 12. Acceptance criteria

- Every protected host refuses unauthenticated access **and** works from a machine
  with no Tailscale and no VPN.
- Gitea read is public for employers; Gitea write requires 2FA.
- A new `*.lab` host is protected by default.
- A scan from outside is banned automatically and visible.
- Repo history is scanned for secrets and findings are rotated.
- Semgrep, trivy, and the auth test run in CI.
- A `nuclei` baseline exists.

## 13. Decisions and open questions

Resolved:
1. **Everything stays public.** Console, Paperclip and Gitea MUST work from an
   arbitrary machine on the road, with no Tailscale and no VPN. The fix is
   **stronger auth**, not hiding them. Tailscale is for SSH and for services that
   genuinely do not need public access.
2. `*.lab` stays **public**. The unauthenticated allowlist is short and explicit
   (`maps.lab`).
2. Scanning is accepted as normal — the goal is **control**, not zero noise.
3. Authelia over Authentik (lighter, fits Caddy forward-auth).

**`maps.lab` — PUBLIC BY DESIGN (decided 2026-10-14).**

Obsidian embeds the Archify maps and Gitea content, so `maps.lab` must stay
reachable without auth. Accepted: it discloses internal hostnames, ports and
service names to anyone who asks.

Rationale from Sam: *"the network info is not really that bad — if someone gets
in they have that anyway. No keys, passwords etc matter."*

Rules that follow from this:
- **NEVER publish a secret, key, password, token or credential to `maps.lab`.**
  It is world-readable. (Publishing a *public* SSH key is fine.)
- Directories named after secrets (for example `inmotionhosting_S3_email`) must
  not contain sensitive material.
- Optional hardening, not required: turn off directory `browse` so the tree index
  is not listable while direct links keep working.

Remaining:
1. Which other hosts are **genuinely public**? (Needs Sam's list for the allowlist.)
2. Should CrowdSec run in **ban** mode from the start, or observe first?
3. Where do the alerts go — Apprise to which channel?
4. Do we add `nuclei`/`semgrep` to Gitea CI, or run them by hand?