105 lines
4.3 KiB
Markdown
105 lines
4.3 KiB
Markdown
---
|
||
created: 2026-09-01
|
||
modified: 2026-09-01
|
||
type: runbook
|
||
client: sam
|
||
project: web-host-migration
|
||
status: active
|
||
priority: 1
|
||
tags:
|
||
- backup
|
||
- restore
|
||
- runbook
|
||
- borg
|
||
- kopia
|
||
- restic
|
||
- s3
|
||
aliases:
|
||
- restore-runbook
|
||
- how-to-restore
|
||
id: 1848129028-RR
|
||
---
|
||
# Restore Runbook — bring it all back
|
||
|
||
> How to restore every machine/data from local backups (`.23`) or from **S3** (house-fire scenario).
|
||
> Linked from [[Backup Architecture — Offsite to AWS S3]] and [[Backup Action Plan — Close the Gaps + Restore]].
|
||
|
||
## Restore principle
|
||
- **Machine died, .23 alive** → restore from **local repos** (fast, minutes–hours).
|
||
- **House fire / .23 gone** → restore from **S3** (photos use DEEP_ARCHIVE → hours to retrieve, then download; Borg repos STANDARD → fast).
|
||
|
||
---
|
||
|
||
## 1. Restore .27 desktop (from Borg `borg/.27`)
|
||
|
||
**Local (fast):**
|
||
```bash
|
||
export BORG_PASSPHRASE=$(cat /home/sam/.borg-passphrase)
|
||
borg extract sam@192.168.20.13:/mnt/ubuntu_storage_3TB/backup/borg/.27::<archive-name>
|
||
```
|
||
**From S3 (if .13 gone):**
|
||
```bash
|
||
# on any machine with aws cli + borg
|
||
source /home/sam/.aws-offsite-env.sh
|
||
aws s3 sync s3://sam-offsite-backup/borg/.27/ ./borg-repo/
|
||
export BORG_PASSPHRASE=$(cat ~/.borg-passphrase)
|
||
borg extract ./borg-repo::<archive-name>
|
||
# or borg mount to browse
|
||
```
|
||
|
||
## 2. Restore .13 (from Borg `borg/.13`)
|
||
|
||
**Local (fast):**
|
||
```bash
|
||
export BORG_PASSPHRASE=$(cat /home/sam/.borg-passphrase)
|
||
borg list /mnt/ubuntu_storage_3TB/backup/borg/.13 # find archive
|
||
borg extract /mnt/ubuntu_storage_3TB/backup/borg/.13::<archive> # run from /
|
||
```
|
||
**From S3 (house fire):**
|
||
```bash
|
||
aws s3 sync s3://sam-offsite-backup/borg/.13/ ./borg-repo/
|
||
borg extract ./borg-repo::<archive>
|
||
```
|
||
Contents restored: `~/Docker`, `~/deployment`, voice/audio dirs, dotfiles, `.ssh`, `.agents`, `.pi`, `~/bin`, `.config`, pocketbase, family-home-lab, photo-pipeline. (prefect DB excluded by design — restore a prefect DB dump separately if needed.)
|
||
|
||
## 3. Restore .35 containers / Gitea (from Kopia/Restic on .23)
|
||
|
||
**Local (repos on .23 via .35):**
|
||
- **Backrest (Restic):** open `http://<.35>:9898` (Backrest UI) → restore plan → pick snapshot → restore `/docker-volumes`, `/docker-containers`, `/host_fs/mnt/hd`.
|
||
- **Kopia:** `docker exec kopia kopia restore <snapshot> /restore-target` (snapshots of `/host_fs/Docker/Containers`, `/host_fs/var/lib/docker/volumes`, trilium-data).
|
||
- **Gitea:** its data is `/Docker/Containers/gitea` → inside the Docker_Containers restic snapshot → restore then `docker compose up -d` in `/Docker/Containers/gitea/`.
|
||
**From S3:** the restic/kopia repos are NOT offsited (decision 2026-09-01) — .35 containers restore requires .23. They are rebuildable from configs (which are in borg/.13 + Gitea) + Docker Compose files.
|
||
|
||
## 4. Restore photos (by_date master) — from S3 DEEP_ARCHIVE
|
||
|
||
**⚠️ DEEP_ARCHIVE restore takes HOURS (12–48h) — plan for it.**
|
||
```bash
|
||
aws s3 ls s3://sam-offsite-backup/photos/by_date/ --recursive | wc -l # sanity
|
||
# Option 1: bulk
|
||
aws s3 sync s3://sam-offsite-backup/photos/by_date/ /mnt/ubuntu_storage_3TB/archive/03_photos/by_date/
|
||
# Option 2: single file (one-off, pay per retrieve)
|
||
aws s3api restore-object --bucket sam-offsite-backup --key photos/by_date/<path> --restore-request '{"Days":7,"GlacierJobParameters":{"Tier":"Bulk"}}'
|
||
```
|
||
- Actual file data downloads **only after** the restore job completes.
|
||
- **Immich originals** (`photos/immich/`, if added later) same procedure.
|
||
|
||
## 5. Restore email / DNS
|
||
- DNS: Cloudflare is authoritative — nothing to restore (cloud-managed).
|
||
- Email: Cloudflare Email Routing + SES — cloud services, nothing local to restore.
|
||
|
||
---
|
||
|
||
## Test-restore cadence (recommended)
|
||
- **Monthly:** restore ONE small file from S3 (e.g. a borg chunk or a single photo) → proves read path works.
|
||
- **Quarterly:** restore one complete small Borg archive → verify integrity (`borg check`).
|
||
|
||
## Key commands (quick reference)
|
||
| Task | Command |
|
||
|---|---|
|
||
| List .27 archives | `borg list sam@192.168.20.13:/mnt/ubuntu_storage_3TB/backup/borg/.27` |
|
||
| List .13 archives | `export BORG_PASSPHRASE=$(cat ~/.borg-passphrase); borg list /mnt/ubuntu_storage_3TB/backup/borg/.13` |
|
||
| Backrest UI | `http://192.168.20.35:9898` |
|
||
| Check S3 sizes | `aws s3 ls s3://sam-offsite-backup --summarize` |
|
||
|
||
---
|
||
*Part of [[Migration Plan — InMotion to Local + AWS Offsite]]. Track in [[Migration Work Log]].* |