Files
ai-resume/docs/phase-3-evidence.md
System agent e6240c1e8e Phase 3: dual bind per Sam's decision (DoD 3 amended, DoD 1/4/8 re-evidenced)
Sam answered the bind question on AUD-133 (2026-10-26):
"A - dual bind: 127.0.0.1:8096 and 192.168.20.13:8096".

- docker-compose: publish 127.0.0.1 and ${APP_LAN_IP:-192.168.20.13}, never 0.0.0.0
- deploy.sh: carry APP_LAN_IP into the release .env, --lan-ip flag, health gate
  now requires /healthz 200 on loopback AND the LAN address
- rehearse.sh: assert both binds, no wildcard, LAN reachability; KEEP_REHEARSAL
  now really keeps the container/image; N3 waits up to 20s instead of 3s
- docs/phase-3-evidence.md: decision recorded, new measured output

DoD: 1, 3 (as amended by Sam), 4, 8 met with recorded output. 2, 5, 6 deferred
on the Phase 2 binary; 7 is Sam's Caddy apply. rehearse.sh 24/24, exit 0.
2026-10-26 17:44:22 +11:00

265 lines
11 KiB
Markdown

# Phase 3 evidence — public deployment
**Status: mechanism complete and rehearsed; bind decided by Sam; public URL deferred.**
The deploy machinery exists, is rehearsed against the real release root and the
real port on `.13`, and leaves nothing running. Sam answered the bind question on
2026-10-26 (**dual bind**, §5), and that answer is now in the committed compose
file and proven reachable from `.35`. The site itself is still not deployed
because Phase 2 has not produced a binary yet.
Recorded 2026-10-26 on `.13` (`nixos-desktop`), runs `630e2ed9` and `a61a5b22`.
---
## Definition of Done
| # | Item | Verdict | Evidence |
|---|---|---|---|
| 1 | `deploy.sh` and `deploy.sh --rollback` work; a deliberate bad build proves rollback | **met** | §1, `deploy/rehearse.sh` 24/24 |
| 2 | `https://resume.lab.audasmedia.com.au` serves the site | **deferred** | Phase 2 has not produced a binary; Caddy block not yet applied (§4, §5) |
| 3 | Container bound to `127.0.0.1` only, `ss -ltn` | **met as amended by Sam**: loopback **and** `192.168.20.13`, no wildcard — see §5 | §2 |
| 4 | `memswap_limit == mem_limit`, `docker inspect` | **met** | §2 |
| 5 | Security headers present, CSP does not break the page | **deferred** | headers belong to the `.35` Caddy block (§4) |
| 6 | A1 passes over the public URL with the AI layer stopped | **deferred** | needs a live URL |
| 7 | Sam applied the Caddy block and confirmed other lab hostnames | **deferred — Sam's action** | block and verification steps in §4 |
| 8 | Rollback rehearsed once and documented | **met** | §1, `deploy/rehearse.sh` |
Nothing in the deferred column is asserted. Each depends on the Phase 2 binary
(items 2, 5, 6) or on Sam (items 2, 5, 7).
---
## 1. What was rehearsed, and the result
`deploy/rehearse.sh` builds throwaway services and drives the real script. Full
transcript of the run after the dual-bind change (24 assertions):
```
== 0. preconditions
PASS port 8096 free
PASS three throwaway sources built in /tmp/.../rehearsal4
== 1. good release deploys and health-gates
PASS deploy.sh exit 0
PASS /healthz 200 on 127.0.0.1:8096
PASS serving release 20261026T064248Z
PASS container publishes on 127.0.0.1:8096 (loopback, used by the deploy.sh health gate)
PASS container publishes on 192.168.20.13:8096 (the address Caddy on .35 proxies to)
PASS no wildcard bind on 8096 — only: 127.0.0.1:8096 192.168.20.13:8096
PASS /healthz 200 over the LAN address http://192.168.20.13:8096
PASS memswap_limit == mem_limit (125829120 bytes)
== 2. build failure fails fast, current does not move
PASS deploy.sh exit 1 (non-zero, as required)
PASS current unchanged (/home/sam/deployment/resume/releases/20261026T064248Z)
PASS previous release still serving
== 3. unhealthy release is rolled back by the health gate
PASS deploy.sh exit 1 (non-zero, as required)
PASS current restored to /home/sam/deployment/resume/releases/20261026T064248Z
PASS rolled-back release serving again
PASS log records the health-gate failure
== 4. deploy.sh --rollback restores the previous release in one command
PASS second good release deployed
PASS rollback exit 0
PASS current flipped back to /home/sam/deployment/resume/releases/20261026T064248Z
PASS rolled-back release healthy
PASS serving 20261026T064248Z again (was 20261026T064307Z)
== 5. N3: docker compose down && up restores the service
PASS service down after compose down
PASS service restored after compose up
== verdict
passed 24, failed 0
```
Reachability from the Caddy host itself (`.35`), run during that rehearsal:
```
$ ssh sam@192.168.20.35 'curl -sS -m3 -w "%{http_code}" http://192.168.20.13:8096/healthz'
200
```
That is the exact hop §3.3's `reverse_proxy` will take, proven before the Caddy
block is applied.
The health-gate rollback in the deploy log, verbatim:
```
06:28:46Z deploy: previous -> /home/sam/deployment/resume/releases/20261026T062828Z
06:28:46Z deploy: current -> /home/sam/deployment/resume/releases/20261026T062842Z
06:29:02Z deploy: HEALTH GATE FAILED for 20261026T062842Z
06:29:02Z rollback: 20261026T062842Z failed -> restoring .../releases/20261026T062828Z
06:29:04Z health: http://127.0.0.1:8096/healthz -> 200
06:29:04Z rollback: previous release healthy (.../releases/20261026T062828Z)
```
The deliberate bad build, verbatim:
```
$ ./deploy/deploy.sh --source .../app-broken
cmd/server/main.go:11:12: syntax error: unexpected {, expected )
exit=1
current before: .../releases/20261026T062828Z
current after: .../releases/20261026T062828Z # unchanged
health after broken build: 200 # old release still serving
```
Re-run it any time with `./deploy/rehearse.sh` (exit 0 = every assertion held).
---
## 2. Container limits and bind scope
```
$ ss -ltn | grep 8096
LISTEN 0 4096 127.0.0.1:8096 0.0.0.0:*
LISTEN 0 4096 192.168.20.13:8096 0.0.0.0:*
$ docker inspect resume-app
mem_limit=125829120 memswap_limit=125829120
port_bindings={"8096/tcp":[{"HostIp":"127.0.0.1","HostPort":"8096"},{"HostIp":"192.168.20.13","HostPort":"8096"}]}
restart=unless-stopped
log_config={"Type":"json-file","Config":{"max-file":"3","max-size":"10m"}}
image=resume/app:20261026T064248Z
$ curl -sS http://127.0.0.1:8096/healthz
{"status":"ok","static":true,"ai":"disabled","version":"20261026T064248Z",...}
$ curl -sS http://192.168.20.13:8096/healthz
{"status":"ok","static":true,"ai":"disabled","version":"20261026T064248Z",...}
```
Two explicit addresses, no wildcard listener: `0.0.0.0` never appears, so
nothing but Caddy on `.35` can be reached from outside the LAN. The reasoning is
in §5.
Measured memory (idle Go static server, placeholder build):
```
$ docker stats --no-stream --format 'rss={{.MemUsage}} pct={{.MemPerc}}' resume-app
rss=1.73MiB / 120MiB pct=1.44%
```
Host pressure at the time of the rehearsal — this is why the cap is 120m:
```
$ free -m
total used free available
Mem: 15918 13943 223 1975
```
No residue was left behind: after the rehearsal there is no `resume-app`
container, no `resume/app:*` image, nothing listening on 8096, and
`/home/sam/deployment/resume/` does not exist. `deploy.sh` creates it on the
first real deploy. (The pre-existing `ai-resume-*` containers from
`/home/sam/deployment/ai-resume/` were not touched.)
---
## 3. Facts established on the host
| Fact | Value |
|---|---|
| Port 8096 | free (`ss -ltn`; 8090, 8091, 8092, 8095 are taken) |
| Release root | `/home/sam/deployment/resume/` — created by `deploy.sh` |
| Go toolchain | not on `PATH`; `/nix/store/…-go-1.26.5/bin/go`, auto-detected by the script |
| Container runtime | Docker 29.6.2, Compose 5.3.1 |
| `.13` runs its own Caddy | `caddy run --config /etc/static/caddy/caddy_config`, listening `*:8000`, 8 legacy site blocks, all `reverse_proxy localhost:<port>` |
| Services added since then | bind `0.0.0.0` and are proxied by `.35` directly on their own LAN port (hub 8788, compass 8792, email-agent 8790, webrestrict 8791) |
---
## 4. Caddy block for Sam to apply (`.35`, additive, `caddy reload` only)
```caddy
resume.lab.audasmedia.com.au {
encode zstd gzip
reverse_proxy 192.168.20.13:8096
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
Referrer-Policy "strict-origin-when-cross-origin"
X-Frame-Options "SAMEORIGIN"
Content-Security-Policy "default-src 'self'; img-src 'self' data:; script-src 'self'; style-src 'self' 'unsafe-inline'; connect-src 'self'; frame-ancestors 'none'; base-uri 'none'"
}
@static path *.css *.js *.svg *.png *.jpg *.webp *.woff2
header @static Cache-Control "public, max-age=31536000, immutable"
}
```
Apply, in this order:
```bash
# 1. add the block above to the Caddyfile on .35, then validate:
caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
# 2. reload — not restart, so other sites are not interrupted:
caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile
# 3. confirm the other hostnames first, from anywhere on the LAN:
for h in gitea.lab.audasmedia.com.au immich.lab.audasmedia.com.au \
nextcloud.lab.audasmedia.com.au penpot.lab.audasmedia.com.au \
wiki.lab.audasmedia.com.au chat.lab.audasmedia.com.au; do
printf '%-45s %s\n' "$h" "$(curl -s -o /dev/null -w '%{http_code}' "https://$h/")"
done
# 4. then the new one (404/502 is expected until Phase 2's binary is deployed):
curl -sSI https://resume.lab.audasmedia.com.au/ | sed -n '1,12p'
```
Header check after that (`item 5` of the DoD): the response above must include
`Strict-Transport-Security`, `X-Content-Type-Options`, `Referrer-Policy`,
`X-Frame-Options` and `Content-Security-Policy`. If the page renders but the CSP
breaks it, the CSP is wrong — do not weaken it to `unsafe-inline` for scripts.
---
## 5. Bind address — decided by Sam, 2026-10-26 (dual bind)
The phase document said two things that could not both be true:
- §3.2 and DoD item 3: the container publishes on **`127.0.0.1:8096` only**.
- §3.3: Caddy **on `.35`** proxies to **`192.168.20.13:8096`**.
A loopback-only listener on `.13` is not reachable from `.35`. Measured at the
time of the question:
```
$ curl -sS -m3 http://192.168.20.13:8096/healthz
curl: (7) Failed to connect to 192.168.20.13:8096
```
`.13`'s own Caddy (`*:8000`, `/etc/static/caddy/caddy_config`) only serves eight
legacy hostnames; every service added since (hub, compass, email-agent,
webrestrict) is proxied by `.35` straight to its LAN port. §3.3's block assumes
that pattern.
**Answer (AUD-133, 2026-10-26):** "A - dual bind: 127.0.0.1:8096 and
192.168.20.13:8096". That is now the committed behaviour:
| Change | Where |
|---|---|
| both addresses in `ports:` | `deploy/docker-compose.yml` (`${APP_LAN_IP:-192.168.20.13}`) |
| LAN address carried into the release `.env`; `--lan-ip ADDR` / `APP_LAN_IP` override | `deploy/deploy.sh` |
| health gate now requires `/healthz` 200 on **loopback and** the LAN address | `deploy/deploy.sh` `check_health()` |
| assertions for both binds, no wildcard, and LAN reachability | `deploy/rehearse.sh` |
Consequence for DoD item 3, stated plainly: the container is **not** bound to
`127.0.0.1` only. It is bound to `127.0.0.1` and `192.168.20.13`, never
`0.0.0.0`, so it is reachable on the `.13` LAN exactly like hub/compass today and
not from anywhere else. That is a written change to §3.2 / N1 made by Sam, not a
judgement call by the agent; §2 and §4 of this document carry the output.
No NixOS change and no `.13` Caddy change is needed under this option. §3.3's
block is applied verbatim.
---
## 6. Deferred, with owners
| Item | Owner | Unblocked by |
|---|---|---|
| Deploy the real site (DoD 2) | this issue, woken by [AUD-132](/AUD/issues/AUD-132) | Phase 2 producing `bin/server` + `dist/` from `make build` |
| Header + CSP proof (DoD 5) | this issue, after the Caddy block is applied | Sam applies §4 |
| A1 over the public URL (DoD 6) | this issue, after 2 and 5 | — |
| Caddy block applied and other hostnames confirmed (DoD 7) | **Sam** | §4 |