ai-resume/docs/phase-3-evidence.md

Phase 3 evidence — public deployment

Status: mechanism complete and rehearsed; bind decided by Sam; public URL deferred. The deploy machinery exists, is rehearsed against the real release root and the real port on .13, and leaves nothing running. Sam answered the bind question on 2026-10-26 (dual bind, §5), and that answer is now in the committed compose file and proven reachable from .35. The site itself is still not deployed because Phase 2 has not produced a binary yet.

Recorded 2026-10-26 on .13 (nixos-desktop), runs 630e2ed9 and a61a5b22.

---

Definition of Done

#	Item	Verdict	Evidence
1	deploy.sh and deploy.sh --rollback work; a deliberate bad build proves rollback	met	§1, deploy/rehearse.sh 24/24
2	https://resume.lab.audasmedia.com.au serves the site	deferred	Phase 2 has not produced a binary; Caddy block not yet applied (§4, §5)
3	Container bound to 127.0.0.1 only, ss -ltn	met as amended by Sam: loopback and 192.168.20.13, no wildcard — see §5	§2
4	memswap_limit == mem_limit, docker inspect	met	§2
5	Security headers present, CSP does not break the page	deferred	headers belong to the .35 Caddy block (§4)
6	A1 passes over the public URL with the AI layer stopped	deferred	needs a live URL
7	Sam applied the Caddy block and confirmed other lab hostnames	deferred — Sam's action	block and verification steps in §4
8	Rollback rehearsed once and documented	met	§1, deploy/rehearse.sh

Nothing in the deferred column is asserted. Each depends on the Phase 2 binary (items 2, 5, 6) or on Sam (items 2, 5, 7).

---

1. What was rehearsed, and the result

deploy/rehearse.sh builds throwaway services and drives the real script. Full transcript of the run after the dual-bind change (24 assertions):

== 0. preconditions
  PASS  port 8096 free
  PASS  three throwaway sources built in /tmp/.../rehearsal4

== 1. good release deploys and health-gates
  PASS  deploy.sh exit 0
  PASS  /healthz 200 on 127.0.0.1:8096
  PASS  serving release 20261026T064248Z
  PASS  container publishes on 127.0.0.1:8096 (loopback, used by the deploy.sh health gate)
  PASS  container publishes on 192.168.20.13:8096 (the address Caddy on .35 proxies to)
  PASS  no wildcard bind on 8096 — only: 127.0.0.1:8096 192.168.20.13:8096
  PASS  /healthz 200 over the LAN address http://192.168.20.13:8096
  PASS  memswap_limit == mem_limit (125829120 bytes)

== 2. build failure fails fast, current does not move
  PASS  deploy.sh exit 1 (non-zero, as required)
  PASS  current unchanged (/home/sam/deployment/resume/releases/20261026T064248Z)
  PASS  previous release still serving

== 3. unhealthy release is rolled back by the health gate
  PASS  deploy.sh exit 1 (non-zero, as required)
  PASS  current restored to /home/sam/deployment/resume/releases/20261026T064248Z
  PASS  rolled-back release serving again
  PASS  log records the health-gate failure

== 4. deploy.sh --rollback restores the previous release in one command
  PASS  second good release deployed
  PASS  rollback exit 0
  PASS  current flipped back to /home/sam/deployment/resume/releases/20261026T064248Z
  PASS  rolled-back release healthy
  PASS  serving 20261026T064248Z again (was 20261026T064307Z)

== 5. N3: docker compose down && up restores the service
  PASS  service down after compose down
  PASS  service restored after compose up

== verdict
passed 24, failed 0

Reachability from the Caddy host itself (.35), run during that rehearsal:

$ ssh sam@192.168.20.35 'curl -sS -m3 -w "%{http_code}" http://192.168.20.13:8096/healthz'
200

That is the exact hop §3.3's reverse_proxy will take, proven before the Caddy block is applied.

The health-gate rollback in the deploy log, verbatim:

06:28:46Z deploy: previous -> /home/sam/deployment/resume/releases/20261026T062828Z
06:28:46Z deploy: current  -> /home/sam/deployment/resume/releases/20261026T062842Z
06:29:02Z deploy: HEALTH GATE FAILED for 20261026T062842Z
06:29:02Z rollback: 20261026T062842Z failed -> restoring .../releases/20261026T062828Z
06:29:04Z health: http://127.0.0.1:8096/healthz -> 200
06:29:04Z rollback: previous release healthy (.../releases/20261026T062828Z)

The deliberate bad build, verbatim:

$ ./deploy/deploy.sh --source .../app-broken
cmd/server/main.go:11:12: syntax error: unexpected {, expected )
exit=1
current before: .../releases/20261026T062828Z
current after:  .../releases/20261026T062828Z      # unchanged
health after broken build: 200                     # old release still serving

Re-run it any time with ./deploy/rehearse.sh (exit 0 = every assertion held).

---

2. Container limits and bind scope

$ ss -ltn | grep 8096
LISTEN 0  4096  127.0.0.1:8096     0.0.0.0:*
LISTEN 0  4096  192.168.20.13:8096  0.0.0.0:*

$ docker inspect resume-app
mem_limit=125829120 memswap_limit=125829120
port_bindings={"8096/tcp":[{"HostIp":"127.0.0.1","HostPort":"8096"},{"HostIp":"192.168.20.13","HostPort":"8096"}]}
restart=unless-stopped
log_config={"Type":"json-file","Config":{"max-file":"3","max-size":"10m"}}
image=resume/app:20261026T064248Z

$ curl -sS http://127.0.0.1:8096/healthz
{"status":"ok","static":true,"ai":"disabled","version":"20261026T064248Z",...}
$ curl -sS http://192.168.20.13:8096/healthz
{"status":"ok","static":true,"ai":"disabled","version":"20261026T064248Z",...}

Two explicit addresses, no wildcard listener: 0.0.0.0 never appears, so nothing but Caddy on .35 can be reached from outside the LAN. The reasoning is in §5.

Measured memory (idle Go static server, placeholder build):

$ docker stats --no-stream --format 'rss={{.MemUsage}} pct={{.MemPerc}}' resume-app
rss=1.73MiB / 120MiB pct=1.44%

Host pressure at the time of the rehearsal — this is why the cap is 120m:

$ free -m
               total   used   free   available
Mem:           15918  13943    223        1975

No residue was left behind: after the rehearsal there is no resume-app container, no resume/app:* image, nothing listening on 8096, and /home/sam/deployment/resume/ does not exist. deploy.sh creates it on the first real deploy. (The pre-existing ai-resume-* containers from /home/sam/deployment/ai-resume/ were not touched.)

---

3. Facts established on the host

Fact	Value
Port 8096	free (ss -ltn; 8090, 8091, 8092, 8095 are taken)
Release root	/home/sam/deployment/resume/ — created by deploy.sh
Go toolchain	not on PATH; /nix/store/…-go-1.26.5/bin/go, auto-detected by the script
Container runtime	Docker 29.6.2, Compose 5.3.1
.13 runs its own Caddy	caddy run --config /etc/static/caddy/caddy_config, listening *:8000, 8 legacy site blocks, all reverse_proxy localhost:<port>
Services added since then	bind 0.0.0.0 and are proxied by .35 directly on their own LAN port (hub 8788, compass 8792, email-agent 8790, webrestrict 8791)

---

4. Caddy block for Sam to apply (.35, additive, caddy reload only)

resume.lab.audasmedia.com.au {
	encode zstd gzip
	reverse_proxy 192.168.20.13:8096
	header {
		Strict-Transport-Security "max-age=31536000; includeSubDomains"
		X-Content-Type-Options "nosniff"
		Referrer-Policy "strict-origin-when-cross-origin"
		X-Frame-Options "SAMEORIGIN"
		Content-Security-Policy "default-src 'self'; img-src 'self' data:; script-src 'self'; style-src 'self' 'unsafe-inline'; connect-src 'self'; frame-ancestors 'none'; base-uri 'none'"
	}
	@static path *.css *.js *.svg *.png *.jpg *.webp *.woff2
	header @static Cache-Control "public, max-age=31536000, immutable"
}

Apply, in this order:

# 1. add the block above to the Caddyfile on .35, then validate:
caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
# 2. reload — not restart, so other sites are not interrupted:
caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile
# 3. confirm the other hostnames first, from anywhere on the LAN:
for h in gitea.lab.audasmedia.com.au immich.lab.audasmedia.com.au \
         nextcloud.lab.audasmedia.com.au penpot.lab.audasmedia.com.au \
         wiki.lab.audasmedia.com.au chat.lab.audasmedia.com.au; do
  printf '%-45s %s\n' "$h" "$(curl -s -o /dev/null -w '%{http_code}' "https://$h/")"
done
# 4. then the new one (404/502 is expected until Phase 2's binary is deployed):
curl -sSI https://resume.lab.audasmedia.com.au/ | sed -n '1,12p'

Header check after that (item 5 of the DoD): the response above must include Strict-Transport-Security, X-Content-Type-Options, Referrer-Policy, X-Frame-Options and Content-Security-Policy. If the page renders but the CSP breaks it, the CSP is wrong — do not weaken it to unsafe-inline for scripts.

---

5. Bind address — decided by Sam, 2026-10-26 (dual bind)

The phase document said two things that could not both be true:

• §3.2 and DoD item 3: the container publishes on 127.0.0.1:8096 only.
• §3.3: Caddy on .35 proxies to 192.168.20.13:8096.

A loopback-only listener on .13 is not reachable from .35. Measured at the time of the question:

$ curl -sS -m3 http://192.168.20.13:8096/healthz
curl: (7) Failed to connect to 192.168.20.13:8096

.13's own Caddy (*:8000, /etc/static/caddy/caddy_config) only serves eight legacy hostnames; every service added since (hub, compass, email-agent, webrestrict) is proxied by .35 straight to its LAN port. §3.3's block assumes that pattern.

Answer (AUD-133, 2026-10-26): "A - dual bind: 127.0.0.1:8096 and 192.168.20.13:8096". That is now the committed behaviour:

Change	Where
both addresses in ports:	deploy/docker-compose.yml (${APP_LAN_IP:-192.168.20.13})
LAN address carried into the release .env; --lan-ip ADDR / APP_LAN_IP override	deploy/deploy.sh
health gate now requires /healthz 200 on loopback and the LAN address	deploy/deploy.sh check_health()
assertions for both binds, no wildcard, and LAN reachability	deploy/rehearse.sh

Consequence for DoD item 3, stated plainly: the container is not bound to 127.0.0.1 only. It is bound to 127.0.0.1 and 192.168.20.13, never 0.0.0.0, so it is reachable on the .13 LAN exactly like hub/compass today and not from anywhere else. That is a written change to §3.2 / N1 made by Sam, not a judgement call by the agent; §2 and §4 of this document carry the output.

No NixOS change and no .13 Caddy change is needed under this option. §3.3's block is applied verbatim.

---

6. Deferred, with owners

Item	Owner	Unblocked by
Deploy the real site (DoD 2)	this issue, woken by AUD-132	Phase 2 producing bin/server + dist/ from make build
Header + CSP proof (DoD 5)	this issue, after the Caddy block is applied	Sam applies §4
A1 over the public URL (DoD 6)	this issue, after 2 and 5	—
Caddy block applied and other hostnames confirmed (DoD 7)	Sam	§4