photo-pipeline/README.md

photo-pipeline

Prefect-orchestrated photo ingestion: Takeout download → fingerprint DB (dedup) → quality scan → Immich.

Components

• photo_db.py — SQLite fingerprint DB (sha256 exact + phash/dhash near dupes; Takeout archive tracking)
• takeout_fetch.py — flow: download (aria2c resume) → track sha256 → extract
• photo_ingest.py — flow: scan dir → check dups → register new
• prefect.yaml — deployments (fetch, ingest) on photo-pool

NixOS note

Venv python needs LD_LIBRARY_PATH (gcc libstdc++ + zlib) — see run-python.sh / run-prefect.sh.

Run

prefect deployment run "takeout-fetch/fetch" --param manifest=/path/urls.txt --param export_id=photos-2026-08 prefect deployment run "photo-ingest/ingest" --param base_dir=/mnt/data/takeout/photos-2026-08