Initial: network security spec (fail-closed, forward-auth, CrowdSec, scanning)
This commit is contained in:
178
AGENTS.md
Normal file
178
AGENTS.md
Normal file
@@ -0,0 +1,178 @@
|
||||
# Network Security
|
||||
|
||||
> Build spec for Paperclip. Follow the phases in section 11 in order. Do not skip a
|
||||
> phase's Definition of Done. When something is unclear, ask Sam — do not guess.
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose and outcome
|
||||
|
||||
Make the lab **fail closed**. Today a new service is exposed to the internet the
|
||||
moment it exists, and the only thing protecting it is remembering to add auth.
|
||||
That is backwards.
|
||||
|
||||
**Outcome (the single success statement):**
|
||||
|
||||
> Exposing a service is a deliberate act. By default it is protected — and scans
|
||||
> against the lab are blocked and visible, not absorbed.
|
||||
|
||||
## 2. Scope
|
||||
|
||||
**In scope**
|
||||
- Move admin UIs off the public internet.
|
||||
- One authentication layer in front of the whole `*.lab` wildcard.
|
||||
- Intrusion detection and auto-banning.
|
||||
- Secret scanning, code scanning, image scanning.
|
||||
- A correct-by-default auth helper for our own FastAPI apps.
|
||||
|
||||
**Out of scope**
|
||||
- Rewriting application features.
|
||||
- Changing what `*.lab` hosts *are* — only how they are reached.
|
||||
|
||||
## 3. Why this exists — the incident
|
||||
|
||||
On **2026-10-13** an automated scanner found `console.lab.audasmedia.com.au` on the
|
||||
public internet and exploited a **critical authentication bypass** in the family
|
||||
console:
|
||||
|
||||
```python
|
||||
if username is None and basic is not None:
|
||||
username = basic.username # password never verified
|
||||
```
|
||||
|
||||
`HTTPBasic` only parses the header. The code trusted the username alone, so
|
||||
`curl -u sam:anything` authenticated as admin. The attacker used it to drive
|
||||
Jervis (the house voice) with secret-exfiltration prompts for two days, unnoticed.
|
||||
|
||||
The bypass is fixed. **This project fixes the conditions that made it matter.**
|
||||
|
||||
## 4. Prerequisites
|
||||
|
||||
| Need | Status / note |
|
||||
|---|---|
|
||||
| Caddy | `.35`, already the single reverse proxy for `*.lab` |
|
||||
| Tailscale | Installed on `.13`, `.27`, `.51`, and the phone (`sams-pixel-8a`) |
|
||||
| Docker | Everywhere it is needed |
|
||||
| A decision | `*.lab` stays **public** for genuine public sites. Only admin UIs move. |
|
||||
|
||||
## 5. Functional requirements
|
||||
|
||||
| ID | Requirement |
|
||||
|---|---|
|
||||
| FR-1 | **Inventory.** Produce a table of every `*.lab` host: what it is, its backend, and whether it has auth. |
|
||||
| FR-2 | **Admin UIs are not public.** Console, Portainer, Gitea, Paperclip, and every other admin surface are reachable **only over Tailscale**. |
|
||||
| FR-3 | **Default-deny at Caddy.** Every `*.lab` host passes through **forward-auth**. A host is protected unless it is on an explicit public allowlist. |
|
||||
| FR-4 | **SSO + 2FA** (Authelia) in front of the wildcard, with per-host policy. |
|
||||
| FR-5 | **CrowdSec** reads the Caddy and sshd logs, detects scans, and **auto-bans** via a Caddy bouncer. |
|
||||
| FR-6 | **Secret scanning.** `gitleaks` as a pre-commit hook on every repo; `trufflehog` over existing history. |
|
||||
| FR-7 | **Code scanning.** `semgrep` in CI/pre-commit for our own code. |
|
||||
| FR-8 | **Image scanning.** `trivy` over the running container images. |
|
||||
| FR-9 | **A correct-by-default auth helper** for FastAPI apps, plus a test asserting **valid username + wrong password → 401**. |
|
||||
| FR-10 | **A baseline scan** with `nuclei` against `.lab`, so we find issues before the bots do. |
|
||||
|
||||
## 6. Screens and UX flow
|
||||
|
||||
- **Authelia portal** — login + 2FA page in front of protected hosts.
|
||||
- **CrowdSec dashboard** — who is being banned, and why.
|
||||
- **The inventory table** (FR-1) — a document, not a UI.
|
||||
|
||||
## 7. Look and feel
|
||||
|
||||
Reuse the Family Home Lab design system for any page we build ourselves.
|
||||
|
||||
- `/home/sam/Docker/Containers/family-home-lab/DESIGN.md`
|
||||
- `/home/sam/Docker/Containers/family-home-lab/portal/static/tokens.css`
|
||||
|
||||
Authelia and CrowdSec ship their own UIs — do not restyle them.
|
||||
|
||||
## 8. Integrations
|
||||
|
||||
| System | Use |
|
||||
|---|---|
|
||||
| **Caddy** (`.35`) | forward-auth, the CrowdSec bouncer, the public allowlist |
|
||||
| **Tailscale** | admin access without public exposure |
|
||||
| **Authelia** | SSO + 2FA |
|
||||
| **CrowdSec** | log parsing + decisions + bouncer |
|
||||
| **Apprise** (`.35:8210`) | alerts on bans and scan spikes |
|
||||
| **Gitea** | pre-commit hooks, CI |
|
||||
|
||||
## 9. Technology (locked)
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| Reverse proxy | **Caddy** (existing, `.35`) |
|
||||
| Auth | **Authelia** (forward-auth) |
|
||||
| IDS/IPS | **CrowdSec** + Caddy bouncer |
|
||||
| Remote access | **Tailscale** |
|
||||
| Secret scan | **gitleaks** (pre-commit), **trufflehog** (history) |
|
||||
| SAST | **semgrep** |
|
||||
| Image scan | **trivy** |
|
||||
| DAST | **nuclei** |
|
||||
| Container | One per service. Own folder, own compose file. |
|
||||
|
||||
## 10. Rules
|
||||
|
||||
1. **Fail closed.** A new host is protected by default; exposure is explicit.
|
||||
2. **Never remove auth to "fix" a problem.** Fix the auth instead.
|
||||
3. Do not break genuinely public sites — they go on the allowlist.
|
||||
4. Alerts go through **Apprise**.
|
||||
5. One commit and push per phase.
|
||||
6. Bound every docker command (`timeout`, `--entrypoint sh` when running a shell
|
||||
inside an image).
|
||||
|
||||
## 11. Build order
|
||||
|
||||
**Phase 1 — Inventory**
|
||||
Table of every `*.lab` host: purpose, backend, auth, public or not.
|
||||
*DoD:* the table exists and is complete. **Nothing changes yet.**
|
||||
|
||||
**Phase 2 — Admin UIs off the public internet**
|
||||
Move admin surfaces to Tailscale-only. Console, Portainer, Gitea, Paperclip first.
|
||||
*DoD:* admin UIs are **unreachable from the public internet** and reachable over
|
||||
Tailscale — proven both ways.
|
||||
|
||||
**Phase 3 — CrowdSec + Caddy bouncer**
|
||||
*DoD:* a deliberate scan from an external host is **banned**, and the ban is visible
|
||||
in the dashboard.
|
||||
|
||||
**Phase 4 — Authelia forward-auth + allowlist**
|
||||
*DoD:* a **newly created** `*.lab` host is protected **without anyone remembering
|
||||
to add auth**. Public sites still work.
|
||||
|
||||
**Phase 5 — Secret scanning**
|
||||
`gitleaks` pre-commit on every repo; `trufflehog` over history.
|
||||
*DoD:* findings triaged; every real leak rotated.
|
||||
|
||||
**Phase 6 — Code and image scanning**
|
||||
`semgrep` + `trivy`.
|
||||
*DoD:* both run in CI; findings triaged.
|
||||
|
||||
**Phase 7 — Auth helper + test**
|
||||
A shared FastAPI dependency that verifies the password, plus the test.
|
||||
*DoD:* the test **fails** against the old bypass pattern and passes against the fix.
|
||||
|
||||
**Phase 8 — Baseline scan + documentation**
|
||||
`nuclei` against `.lab`; write it all up.
|
||||
*DoD:* no high findings unreviewed; docs published.
|
||||
|
||||
## 12. Acceptance criteria
|
||||
|
||||
- No admin UI is reachable from the public internet.
|
||||
- A new `*.lab` host is protected by default.
|
||||
- A scan from outside is banned automatically and visible.
|
||||
- Repo history is scanned for secrets and findings are rotated.
|
||||
- Semgrep, trivy, and the auth test run in CI.
|
||||
- A `nuclei` baseline exists.
|
||||
|
||||
## 13. Decisions and open questions
|
||||
|
||||
Resolved:
|
||||
1. `*.lab` stays **public** for genuine public sites. Only admin UIs move.
|
||||
2. Scanning is accepted as normal — the goal is **control**, not zero noise.
|
||||
3. Authelia over Authentik (lighter, fits Caddy forward-auth).
|
||||
|
||||
Remaining:
|
||||
1. Which hosts are **genuinely public**? (Needs Sam's list for the allowlist.)
|
||||
2. Should CrowdSec run in **ban** mode from the start, or observe first?
|
||||
3. Where do the alerts go — Apprise to which channel?
|
||||
4. Do we add `nuclei`/`semgrep` to Gitea CI, or run them by hand?
|
||||
Reference in New Issue
Block a user