From 61a501716e00163adc8c963e6dd69bebe457b666 Mon Sep 17 00:00:00 2001
From: Sam Rolfe <samuelrolfe@gmail.com>
Date: Sun, 6 Sep 2026 18:27:43 +1000
Subject: [PATCH] readme: Mermaid pipeline diagram + flow explanation (renders
 in Gitea)

---
 README.md | 41 ++++++++++++++++++++++++-----------------
 1 file changed, 24 insertions(+), 17 deletions(-)

diff --git a/README.md b/README.md
index 9af36f5..04bd297 100644
--- a/README.md
+++ b/README.md
@@ -16,25 +16,32 @@ server (.13). It downloads Google Takeout exports, fingerprints every image
 
 ## The pipeline
 
-```
-Google Takeout export (manual — Google has no export API)
-        │  drop manifest of URLs into /mnt/data/takeout/incoming/
-        ▼
-photo-watch   (Prefect, every 15 min — watches incoming/)
-        │  downloads archives (aria2c, resume), tracks sha256, extracts
-        ▼
-photo-ingest  (fingerprint DB: sha256 exact + phash/dhash near-dup detection)
-        │
-        ▼
-photo-quality-scan  (CleanVision: blurry, dark, light, grayscale, low-info…)
-        │  verdicts → 01_keep / 02_review / 03_delete staging
-        ▼
-Apprise notification  →  you review in the dashboard
-        │  Keep → moves file to 01_keep · Reject → 03_delete (never auto-deleted)
-        ▼
-immich-import  (scp → server CLI upload with checksum dedup)
+```mermaid
+flowchart LR
+    TO[Google Takeout<br/>134GB / 14 zips] -->|drop manifest| W[photo-watch<br/>every 15 min]
+    W -->|download + extract| I[photo-ingest<br/>sha256 + dhash dedup]
+    I -->|fingerprints| Q[quality-scan<br/>PIL blurry/dark]
+    Q -->|verdicts| R[Review dashboard<br/>photo-filter.home.lab]
+    R -->|approved| M[merge_orphans<br/>EXIF date routing]
+    M --> B[(by_date master<br/>35K files)]
+    B -->|incremental import| IM[Immich<br/>.35 library]
+    R -->|rejected| T[(trash → purge)]
+    B -.->|offsite| S3[AWS S3<br/>Glacier lifecycle]
 ```
 
+### The flow explained
+
+1. **Takeout** — download Google Photos export archives (14 zips, ~134GB)
+2. **photo-watch** — watches incoming/, extracts, fingerprints
+3. **photo-ingest** — sha256 + dhash; content dedup (never doubles)
+4. **quality-scan** — PIL verdicts: blurry/dark flagged
+5. **Review** — approve / reject in the dashboard
+6. **merge_orphans** — approved → by_date (EXIF-date routed, rename-on-collision)
+7. **Immich import** — incremental (only new files, checksum dedup)
+8. **Rejected** → trash → purge · **by_date** → Borg daily + AWS S3 offsite
+
+> Interactive map: see [`docs/photo-pipeline-map.html`](docs/photo-pipeline-map.html)
+> (published: https://maps.lab.audasmedia.com.au/google_cloud_and_images/docs/)
 ## Deployments (Prefect, pool `photo-pool`)
 
 | Flow | Purpose | Trigger |