Restore Runbook — bring it all back

How to restore every machine/data from local backups (.23) or from S3 (house-fire scenario). Linked from Backup Architecture — Offsite to AWS S3 and Backup Action Plan — Close the Gaps + Restore.

Restore principle

Machine died, .23 alive → restore from local repos (fast, minutes–hours).
House fire / .23 gone → restore from S3 (photos use DEEP_ARCHIVE → hours to retrieve, then download; Borg repos STANDARD → fast).

1. Restore .27 desktop (from Borg `borg/.27`)

Local (fast):

export BORG_PASSPHRASE=$(cat /home/sam/.borg-passphrase)
borg extract sam@192.168.20.13:/mnt/ubuntu_storage_3TB/backup/borg/.27::<archive-name>

From S3 (if .13 gone):

# on any machine with aws cli + borg
source /home/sam/.aws-offsite-env.sh
aws s3 sync s3://sam-offsite-backup/borg/.27/ ./borg-repo/
export BORG_PASSPHRASE=$(cat ~/.borg-passphrase)
borg extract ./borg-repo::<archive-name>
# or borg mount to browse

2. Restore .13 (from Borg `borg/.13`)

Local (fast):

export BORG_PASSPHRASE=$(cat /home/sam/.borg-passphrase)
borg list /mnt/ubuntu_storage_3TB/backup/borg/.13        # find archive
borg extract /mnt/ubuntu_storage_3TB/backup/borg/.13::<archive>   # run from /

From S3 (house fire):

aws s3 sync s3://sam-offsite-backup/borg/.13/ ./borg-repo/
borg extract ./borg-repo::<archive>

Contents restored: ~/Docker, ~/deployment, voice/audio dirs, dotfiles, .ssh, .agents, .pi, ~/bin, .config, pocketbase, family-home-lab, photo-pipeline. (prefect DB excluded by design — restore a prefect DB dump separately if needed.)

3. Restore .35 containers / Gitea (from Kopia/Restic on .23)

Local (repos on .23 via .35):

Backrest (Restic): open http://<.35>:9898 (Backrest UI) → restore plan → pick snapshot → restore /docker-volumes, /docker-containers, /host_fs/mnt/hd.
Kopia: docker exec kopia kopia restore <snapshot> /restore-target (snapshots of /host_fs/Docker/Containers, /host_fs/var/lib/docker/volumes, trilium-data).
Gitea: its data is /Docker/Containers/gitea → inside the Docker_Containers restic snapshot → restore then docker compose up -d in /Docker/Containers/gitea/. From S3: the restic/kopia repos are NOT offsited (decision 2026-09-01) — .35 containers restore requires .23. They are rebuildable from configs (which are in borg/.13 + Gitea) + Docker Compose files.

4. Restore photos (by_date master) — from S3 DEEP_ARCHIVE

⚠️ DEEP_ARCHIVE restore takes HOURS (12–48h) — plan for it.

aws s3 ls s3://sam-offsite-backup/photos/by_date/ --recursive | wc -l   # sanity
# Option 1: bulk
aws s3 sync s3://sam-offsite-backup/photos/by_date/ /mnt/ubuntu_storage_3TB/archive/03_photos/by_date/
# Option 2: single file (one-off, pay per retrieve)
aws s3api restore-object --bucket sam-offsite-backup --key photos/by_date/<path> --restore-request '{"Days":7,"GlacierJobParameters":{"Tier":"Bulk"}}'

Actual file data downloads only after the restore job completes.
Immich originals (photos/immich/, if added later) same procedure.

5. Restore email / DNS

DNS: Cloudflare is authoritative — nothing to restore (cloud-managed).
Email: Cloudflare Email Routing + SES — cloud services, nothing local to restore.

Test-restore cadence (recommended)

Monthly: restore ONE small file from S3 (e.g. a borg chunk or a single photo) → proves read path works.
Quarterly: restore one complete small Borg archive → verify integrity (borg check).

Key commands (quick reference)

Task	Command
List .27 archives	`borg list sam@192.168.20.13:/mnt/ubuntu_storage_3TB/backup/borg/.27`
List .13 archives	`export BORG_PASSPHRASE=$(cat ~/.borg-passphrase); borg list /mnt/ubuntu_storage_3TB/backup/borg/.13`
Backrest UI	`http://192.168.20.35:9898`
Check S3 sizes	`aws s3 ls s3://sam-offsite-backup --summarize`

Part of Migration Plan — InMotion to Local + AWS Offsite. Track in Migration Work Log.

4.3 KiB Raw Blame History Unescape Escape