Initial: fingerprint DB + Takeout fetch + ingest flows

This commit is contained in:
2026-08-07 20:41:32 +10:00
commit 16123d00f1
10 changed files with 614 additions and 0 deletions

16
README.md Normal file
View File

@@ -0,0 +1,16 @@
# photo-pipeline
Prefect-orchestrated photo ingestion: Takeout download → fingerprint DB (dedup) → quality scan → Immich.
## Components
- `photo_db.py` — SQLite fingerprint DB (sha256 exact + phash/dhash near dupes; Takeout archive tracking)
- `takeout_fetch.py` — flow: download (aria2c resume) → track sha256 → extract
- `photo_ingest.py` — flow: scan dir → check dups → register new
- `prefect.yaml` — deployments (fetch, ingest) on photo-pool
## NixOS note
Venv python needs LD_LIBRARY_PATH (gcc libstdc++ + zlib) — see run-python.sh / run-prefect.sh.
## Run
prefect deployment run "takeout-fetch/fetch" --param manifest=/path/urls.txt --param export_id=photos-2026-08
prefect deployment run "photo-ingest/ingest" --param base_dir=/mnt/data/takeout/photos-2026-08