sam/network_security
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial: network security spec (fail-closed, forward-auth, CrowdSec, scanning)

Browse Source
This commit is contained in:
Sam Rolfe
2026-10-14 11:23:50 +11:00
commit f689be3475
3 changed files with 224 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.gitignore vendored Normal file
View File

@@ -0,0 +1,5 @@
.env
*.local
__pycache__/
node_modules/
.pi/memory-vault/

178
AGENTS.md Normal file
View File

@@ -0,0 +1,178 @@
# Network Security
> Build spec for Paperclip. Follow the phases in section 11 in order. Do not skip a
> phase's Definition of Done. When something is unclear, ask Sam — do not guess.
---
## 1. Purpose and outcome
Make the lab **fail closed**. Today a new service is exposed to the internet the
moment it exists, and the only thing protecting it is remembering to add auth.
That is backwards.
**Outcome (the single success statement):**
> Exposing a service is a deliberate act. By default it is protected — and scans
> against the lab are blocked and visible, not absorbed.
## 2. Scope
**In scope**
- Move admin UIs off the public internet.
- One authentication layer in front of the whole `*.lab` wildcard.
- Intrusion detection and auto-banning.
- Secret scanning, code scanning, image scanning.
- A correct-by-default auth helper for our own FastAPI apps.
**Out of scope**
- Rewriting application features.
- Changing what `*.lab` hosts *are* — only how they are reached.
## 3. Why this exists — the incident
On **2026-10-13** an automated scanner found `console.lab.audasmedia.com.au` on the
public internet and exploited a **critical authentication bypass** in the family
console:
```python
if username is None and basic is not None:
username = basic.username # password never verified
```
`HTTPBasic` only parses the header. The code trusted the username alone, so
`curl -u sam:anything` authenticated as admin. The attacker used it to drive
Jervis (the house voice) with secret-exfiltration prompts for two days, unnoticed.
The bypass is fixed. **This project fixes the conditions that made it matter.**
## 4. Prerequisites
| Need | Status / note |
|---|---|
| Caddy | `.35`, already the single reverse proxy for `*.lab` |
| Tailscale | Installed on `.13`, `.27`, `.51`, and the phone (`sams-pixel-8a`) |
| Docker | Everywhere it is needed |
| A decision | `*.lab` stays **public** for genuine public sites. Only admin UIs move. |
## 5. Functional requirements
| ID | Requirement |
|---|---|
| FR-1 | **Inventory.** Produce a table of every `*.lab` host: what it is, its backend, and whether it has auth. |
| FR-2 | **Admin UIs are not public.** Console, Portainer, Gitea, Paperclip, and every other admin surface are reachable **only over Tailscale**. |
| FR-3 | **Default-deny at Caddy.** Every `*.lab` host passes through **forward-auth**. A host is protected unless it is on an explicit public allowlist. |
| FR-4 | **SSO + 2FA** (Authelia) in front of the wildcard, with per-host policy. |
| FR-5 | **CrowdSec** reads the Caddy and sshd logs, detects scans, and **auto-bans** via a Caddy bouncer. |
| FR-6 | **Secret scanning.** `gitleaks` as a pre-commit hook on every repo; `trufflehog` over existing history. |
| FR-7 | **Code scanning.** `semgrep` in CI/pre-commit for our own code. |
| FR-8 | **Image scanning.** `trivy` over the running container images. |
| FR-9 | **A correct-by-default auth helper** for FastAPI apps, plus a test asserting **valid username + wrong password → 401**. |
| FR-10 | **A baseline scan** with `nuclei` against `.lab`, so we find issues before the bots do. |
## 6. Screens and UX flow
- **Authelia portal** — login + 2FA page in front of protected hosts.
- **CrowdSec dashboard** — who is being banned, and why.
- **The inventory table** (FR-1) — a document, not a UI.
## 7. Look and feel
Reuse the Family Home Lab design system for any page we build ourselves.
- `/home/sam/Docker/Containers/family-home-lab/DESIGN.md`
- `/home/sam/Docker/Containers/family-home-lab/portal/static/tokens.css`
Authelia and CrowdSec ship their own UIs — do not restyle them.
## 8. Integrations
| System | Use |
|---|---|
| **Caddy** (`.35`) | forward-auth, the CrowdSec bouncer, the public allowlist |
| **Tailscale** | admin access without public exposure |
| **Authelia** | SSO + 2FA |
| **CrowdSec** | log parsing + decisions + bouncer |
| **Apprise** (`.35:8210`) | alerts on bans and scan spikes |
| **Gitea** | pre-commit hooks, CI |
## 9. Technology (locked)
| Item | Value |
|---|---|
| Reverse proxy | **Caddy** (existing, `.35`) |
| Auth | **Authelia** (forward-auth) |
| IDS/IPS | **CrowdSec** + Caddy bouncer |
| Remote access | **Tailscale** |
| Secret scan | **gitleaks** (pre-commit), **trufflehog** (history) |
| SAST | **semgrep** |
| Image scan | **trivy** |
| DAST | **nuclei** |
| Container | One per service. Own folder, own compose file. |
## 10. Rules
1. **Fail closed.** A new host is protected by default; exposure is explicit.
2. **Never remove auth to "fix" a problem.** Fix the auth instead.
3. Do not break genuinely public sites — they go on the allowlist.
4. Alerts go through **Apprise**.
5. One commit and push per phase.
6. Bound every docker command (`timeout`, `--entrypoint sh` when running a shell
inside an image).
## 11. Build order
**Phase 1 — Inventory**
Table of every `*.lab` host: purpose, backend, auth, public or not.
*DoD:* the table exists and is complete. **Nothing changes yet.**
**Phase 2 — Admin UIs off the public internet**
Move admin surfaces to Tailscale-only. Console, Portainer, Gitea, Paperclip first.
*DoD:* admin UIs are **unreachable from the public internet** and reachable over
Tailscale — proven both ways.
**Phase 3 — CrowdSec + Caddy bouncer**
*DoD:* a deliberate scan from an external host is **banned**, and the ban is visible
in the dashboard.
**Phase 4 — Authelia forward-auth + allowlist**
*DoD:* a **newly created** `*.lab` host is protected **without anyone remembering
to add auth**. Public sites still work.
**Phase 5 — Secret scanning**
`gitleaks` pre-commit on every repo; `trufflehog` over history.
*DoD:* findings triaged; every real leak rotated.
**Phase 6 — Code and image scanning**
`semgrep` + `trivy`.
*DoD:* both run in CI; findings triaged.
**Phase 7 — Auth helper + test**
A shared FastAPI dependency that verifies the password, plus the test.
*DoD:* the test **fails** against the old bypass pattern and passes against the fix.
**Phase 8 — Baseline scan + documentation**
`nuclei` against `.lab`; write it all up.
*DoD:* no high findings unreviewed; docs published.
## 12. Acceptance criteria
- No admin UI is reachable from the public internet.
- A new `*.lab` host is protected by default.
- A scan from outside is banned automatically and visible.
- Repo history is scanned for secrets and findings are rotated.
- Semgrep, trivy, and the auth test run in CI.
- A `nuclei` baseline exists.
## 13. Decisions and open questions
Resolved:
1. `*.lab` stays **public** for genuine public sites. Only admin UIs move.
2. Scanning is accepted as normal — the goal is **control**, not zero noise.
3. Authelia over Authentik (lighter, fits Caddy forward-auth).
Remaining:
1. Which hosts are **genuinely public**? (Needs Sam's list for the allowlist.)
2. Should CrowdSec run in **ban** mode from the start, or observe first?
3. Where do the alerts go — Apprise to which channel?
4. Do we add `nuclei`/`semgrep` to Gitea CI, or run them by hand?

41
NOTES.md Normal file
View File

@@ -0,0 +1,41 @@
The principle to build around
Fail closed. A new service should be protected by default, not by remembering. Right now
.lab
is open-by-default — that's what made the console bypass matter.
Tier 1 — do these first (highest value, least work)
1. Move admin UIs to Tailscale, keep
.lab
for public sites only.
You already have Tailscale on
.13
,
.27
,
.51
and the phone. Admin dashboards (console, Portainer, Gitea, Paperclip) don't need the public internet — you get remote access anyway, with no exposure. Zero new software. This alone removes most of your attack surface.
2. Put one auth layer in front of the whole wildcard.
Caddy forward-auth → Authelia (or Authentik). Every *.lab host goes through one policy: SSO, 2FA, default-deny, with an explicit allowlist for genuinely public sites. Then "I forgot to add auth" can't happen.
3. Add CrowdSec.
It tails your Caddy and sshd logs, detects scans like the one you just saw, and auto-bans via a Caddy bouncer. You already accept scanning — this turns it from noise into blocked traffic, and gives you a dashboard of who's probing.
Tier 2 — stop the two failure modes you've already hit
4. gitleaks + trufflehog.
You had a key leak on Gitea. gitleaks as a pre-commit hook stops new leaks; trufflehog scans existing repos and history for what's already there. Cheapest possible fix for a problem that already cost you once.
5. semgrep.
SAST for your own code — exactly the class of bug found today. It runs in seconds, has auth/security rulesets, and fits a pre-commit or CI step.
6. trivy.
Scans your container images and filesystems for known CVEs. With ~50 containers you're running a lot of images you never inspect.
Tier 3 — worth doing, not urgent
7. nuclei — run your own scans against
.lab
so you find the issues before the bots do. Also tells you what's actually exposed.
8. A correct-by-default auth helper + one test.
For your FastAPI apps: a shared dependency that verifies the password, and a test that asserts valid username + wrong password → 401. That exact test would have caught today's bug on the first run.
What you already have
•
npq, sfw, vet — npm supply-chain (on
.13
).
•
Tailscale — underused; see Tier 1 item 1.
•
Caddy — the natural place for forward-auth and the CrowdSec bouncer.