sam/ai-resume
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8b3209fab5bcbd81ed6fc9f2842fe7bc6de05aba
ai-resume/docs/phase-3-evidence.md
System 8b3209fab5 Phase 3: release script, container limits, rehearsed rollback 
Adds the deployment mechanism for the portfolio site. The site itself is not
deployed yet: Phase 2 has not produced a binary.

- deploy/deploy.sh: build + test (fail fast) -> releases/<ts> -> atomic
  `current` symlink flip -> docker compose up -d -> /healthz gate ->
  flip back to `previous` and exit non-zero on failure. `--rollback`
  restores the previous release in one command. Auto-detects the Go
  toolchain in /nix/store when `go` is not on PATH.
- deploy/docker-compose.yml: 127.0.0.1:8096, mem_limit == memswap_limit ==
  120m, capped json-file logs (10m x 3), restart unless-stopped.
- deploy/Dockerfile: runtime image only (alpine + prebuilt payload), so a
  release needs no toolchain, no network and no package install.
- deploy/rehearse.sh: repeatable proof (21 assertions, exit 0) of deploy,
  build fail-fast, health-gate rollback, --rollback and N3 restart.
- deploy/README.md + docs/phase-3-evidence.md: how it works, the build
  contract Phase 2 must satisfy, the Caddy block for Sam, and the open
  bind-address question (loopback-only vs reachable from .35).

Satisfies DoD items 1, 3 (container), 4, 8. Items 2, 5, 6, 7 need the Phase 2
binary and Sam's Caddy change; recorded as deferred with owners, not claimed.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-26 17:31:13 +11:00

232 lines
9.8 KiB
Markdown
Raw Blame History

# Phase 3 evidence — public deployment
**Status: mechanism complete and rehearsed; public URL deferred.**
The deploy machinery exists, is rehearsed against the real release root and the
real port on `.13`, and leaves nothing running. The site itself is not deployed
because Phase 2 has not produced a binary yet, and one decision is needed from
Sam before the container can be reachable by Caddy on `.35`.
Recorded 2026-10-26 on `.13` (`nixos-desktop`), run `630e2ed9`.
---
## Definition of Done
| # | Item | Verdict | Evidence |
|---|---|---|---|
| 1 | `deploy.sh` and `deploy.sh --rollback` work; a deliberate bad build proves rollback | **met** | §1, `deploy/rehearse.sh` 21/21 |
| 2 | `https://resume.lab.audasmedia.com.au` serves the site | **deferred** | Phase 2 has not produced a binary; Caddy block not yet applied (§4, §5) |
| 3 | Container bound to `127.0.0.1` only, `ss -ltn` | **met for the container**; conflicts with reachability from `.35` — see §5 | §2 |
| 4 | `memswap_limit == mem_limit`, `docker inspect` | **met** | §2 |
| 5 | Security headers present, CSP does not break the page | **deferred** | headers belong to the `.35` Caddy block (§4) |
| 6 | A1 passes over the public URL with the AI layer stopped | **deferred** | needs a live URL |
| 7 | Sam applied the Caddy block and confirmed other lab hostnames | **deferred — Sam's action** | block and verification steps in §4 |
| 8 | Rollback rehearsed once and documented | **met** | §1, `deploy/rehearse.sh` |
Nothing in the deferred column is asserted. Each depends on the Phase 2 binary
(items 2, 5, 6) or on Sam (items 2, 5, 7).
---
## 1. What was rehearsed, and the result
`deploy/rehearse.sh` builds throwaway services and drives the real script. Full
transcript:
```
== 0. preconditions
PASS port 8096 free
PASS three throwaway sources built in /tmp/.../resume-rehearsal-snpEjy
== 1. good release deploys and health-gates
PASS deploy.sh exit 0
PASS /healthz 200 on 127.0.0.1:8096
PASS serving release 20261026T062957Z
PASS container publishes on 127.0.0.1 only
PASS memswap_limit == mem_limit (125829120 bytes)
== 2. build failure fails fast, current does not move
PASS deploy.sh exit 1 (non-zero, as required)
PASS current unchanged (/home/sam/deployment/resume/releases/20261026T062957Z)
PASS previous release still serving
== 3. unhealthy release is rolled back by the health gate
PASS deploy.sh exit 1 (non-zero, as required)
PASS current restored to /home/sam/deployment/resume/releases/20261026T062957Z
PASS rolled-back release serving again
PASS log records the health-gate failure
== 4. deploy.sh --rollback restores the previous release in one command
PASS second good release deployed
PASS rollback exit 0
PASS current flipped back to /home/sam/deployment/resume/releases/20261026T062957Z
PASS rolled-back release healthy
PASS serving 20261026T062957Z again (was 20261026T063021Z)
== 5. N3: docker compose down && up restores the service
PASS service down after compose down
PASS service restored after compose up
== verdict
passed 21, failed 0
```
The health-gate rollback in the deploy log, verbatim:
```
06:28:46Z deploy: previous -> /home/sam/deployment/resume/releases/20261026T062828Z
06:28:46Z deploy: current -> /home/sam/deployment/resume/releases/20261026T062842Z
06:29:02Z deploy: HEALTH GATE FAILED for 20261026T062842Z
06:29:02Z rollback: 20261026T062842Z failed -> restoring .../releases/20261026T062828Z
06:29:04Z health: http://127.0.0.1:8096/healthz -> 200
06:29:04Z rollback: previous release healthy (.../releases/20261026T062828Z)
```
The deliberate bad build, verbatim:
```
$ ./deploy/deploy.sh --source .../app-broken
cmd/server/main.go:11:12: syntax error: unexpected {, expected )
exit=1
current before: .../releases/20261026T062828Z
current after: .../releases/20261026T062828Z # unchanged
health after broken build: 200 # old release still serving
```
Re-run it any time with `./deploy/rehearse.sh` (exit 0 = every assertion held).
---
## 2. Container limits and bind scope
```
$ ss -ltn | grep 8096
LISTEN 0 4096 127.0.0.1:8096 0.0.0.0:*
$ docker inspect -f '...' resume-app
name=/resume-app
mem_limit=125829120 bytes # 120 MiB
memswap_limit=125829120 bytes # 120 MiB — equal, so no host swap
port_bindings={"8096/tcp":[{"HostIp":"127.0.0.1","HostPort":"8096"}]}
restart=unless-stopped
log_config={"Type":"json-file","Config":{"max-file":"3","max-size":"10m"}}
```
Measured memory (idle Go static server, placeholder build):
```
rss=1.5MiB / 120MiB pct=1.25%
rss=6.9MiB / 120MiB pct=5.78% # just after start, before settling
```
Host pressure at the time of the rehearsal — this is why the cap is 120m:
```
$ free -m
total used free available
Mem: 15918 14255 227 1663
```
No residue was left behind: after the rehearsal there is no `resume-app`
container, no `resume/app:*` image, nothing listening on 8096, and
`/home/sam/deployment/resume/` does not exist. `deploy.sh` creates it on the
first real deploy. (The pre-existing `ai-resume-*` containers from
`/home/sam/deployment/ai-resume/` were not touched.)
---
## 3. Facts established on the host
| Fact | Value |
|---|---|
| Port 8096 | free (`ss -ltn`; 8090, 8091, 8092, 8095 are taken) |
| Release root | `/home/sam/deployment/resume/` — created by `deploy.sh` |
| Go toolchain | not on `PATH`; `/nix/store/…-go-1.26.5/bin/go`, auto-detected by the script |
| Container runtime | Docker 29.6.2, Compose 5.3.1 |
| `.13` runs its own Caddy | `caddy run --config /etc/static/caddy/caddy_config`, listening `*:8000`, 8 legacy site blocks, all `reverse_proxy localhost:<port>` |
| Services added since then | bind `0.0.0.0` and are proxied by `.35` directly on their own LAN port (hub 8788, compass 8792, email-agent 8790, webrestrict 8791) |
---
## 4. Caddy block for Sam to apply (`.35`, additive, `caddy reload` only)
```caddy
resume.lab.audasmedia.com.au {
encode zstd gzip
reverse_proxy 192.168.20.13:8096
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
Referrer-Policy "strict-origin-when-cross-origin"
X-Frame-Options "SAMEORIGIN"
Content-Security-Policy "default-src 'self'; img-src 'self' data:; script-src 'self'; style-src 'self' 'unsafe-inline'; connect-src 'self'; frame-ancestors 'none'; base-uri 'none'"
}
@static path *.css *.js *.svg *.png *.jpg *.webp *.woff2
header @static Cache-Control "public, max-age=31536000, immutable"
}
```
Apply, in this order:
```bash
# 1. add the block above to the Caddyfile on .35, then validate:
caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
# 2. reload — not restart, so other sites are not interrupted:
caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile
# 3. confirm the other hostnames first, from anywhere on the LAN:
for h in gitea.lab.audasmedia.com.au immich.lab.audasmedia.com.au \
nextcloud.lab.audasmedia.com.au penpot.lab.audasmedia.com.au \
wiki.lab.audasmedia.com.au chat.lab.audasmedia.com.au; do
printf '%-45s %s\n' "$h" "$(curl -s -o /dev/null -w '%{http_code}' "https://$h/")"
done
# 4. then the new one (404/502 is expected until Phase 2's binary is deployed):
curl -sSI https://resume.lab.audasmedia.com.au/ | sed -n '1,12p'
```
Header check after that (`item 5` of the DoD): the response above must include
`Strict-Transport-Security`, `X-Content-Type-Options`, `Referrer-Policy`,
`X-Frame-Options` and `Content-Security-Policy`. If the page renders but the CSP
breaks it, the CSP is wrong — do not weaken it to `unsafe-inline` for scripts.
---
## 5. Open question for Sam — bind address vs Caddy reachability
The phase document says two things that cannot both be true today:
- §3.2 and DoD item 3: the container publishes on **`127.0.0.1:8096` only**.
- §3.3: Caddy **on `.35`** proxies to **`192.168.20.13:8096`**.
A loopback-only listener on `.13` is not reachable from `.35`. Measured:
```
$ curl -sS -m3 http://192.168.20.13:8096/healthz
curl: (7) Failed to connect to 192.168.20.13:8096
```
`.13`'s own Caddy (`*:8000`, `/etc/static/caddy/caddy_config`) only serves eight
legacy hostnames; every service added since (hub, compass, email-agent,
webrestrict) publishes on `0.0.0.0` and is proxied by `.35` directly to its LAN
port. That is the working pattern, and it is what §3.3's block assumes.
| Option | What changes | Consequence |
|---|---|---|
| **A (recommended, matches every current service)** | publish `127.0.0.1:8096` **and** `192.168.20.13:8096` | `deploy.sh` keeps its loopback health gate; `.35` reaches the app; §3.3 applied verbatim; no NixOS change. Exposure is the `.13` LAN, same as hub/compass today. DoD item 3 / N1 can then only claim "not on `0.0.0.0`" — needs your written change |
| **B (strict loopback)** | add `http://resume.lab.audasmedia.com.au:8000 { reverse_proxy localhost:8096 }` to `.13`'s Caddyfile, and `.35`'s block becomes `reverse_proxy 192.168.20.13:8000` | Satisfies N1 literally. But `/etc/static/caddy/caddy_config` is NixOS-generated — it needs a NixOS config change and `nixos-rebuild`, which this phase forbids and only you can do |
The committed `docker-compose.yml` currently follows the phase document (option
**A**'s loopback half only: `127.0.0.1:8096`). Choosing A or B is a one-line
change in `deploy/docker-compose.yml` plus, for B, the NixOS change on `.13`.
---
## 6. Deferred, with owners
| Item | Owner | Unblocked by |
|---|---|---|
| Deploy the real site (DoD 2) | this issue, woken by [AUD-132](/AUD/issues/AUD-132) | Phase 2 producing `bin/server` + `dist/` from `make build` |
| Header + CSP proof (DoD 5) | this issue, after the Caddy block is applied | Sam applies §4 |
| A1 over the public URL (DoD 6) | this issue, after 2 and 5 | — |
| Caddy block applied and other hostnames confirmed (DoD 7) | **Sam** | §4 |
| Bind-address decision (§5) | **Sam** | A or B above |
Reference in New Issue View Git Blame Copy Permalink