sam/family_home_lab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4cf702cd5fff3daaf886f260bd90980e3c201b92
family_home_lab/plan.md

10 KiB
Raw Blame History

Family Home Lab Console — Plan

Adapted from Obsidian notes: FAMILY Enterprise Homelab AI Multimedia Suite, FAMILY S3 Storage Integration & Blueprint, FAMILY DeepSeek Harness (dsh) Home Lab Setup, Home Network Map Overview, Backup System — Borg, Kopia & Restic.

Status: FINAL v3 — approved. Build phase: repo-local code complete (2026-08-24), deployment pending.

1. Goal

A single family console ("landing page") at a local URL where each of the four users (Sam, Jo, Harry, Finn) logs in and gets:

  • A personalised dashboard of available AI + media tools
  • One-click launch of each tool (inline iframe where possible, else window/new tab)
  • Per-user session handling and tool permissions

The console is also the parent interface that the DeepSeek Harness instances hang off (dsh itself is built by another agent — we only define the integration contract here).

2. Where it runs (adapted to my network)

Concern Generic blueprint said Adapted decision
Host machine unspecified .13 nixos-desktop — always-on server, existing Docker host (15.5 GB RAM), already runs Open WebUI/OmniRoute/Langfuse
Reverse proxy dedicated Caddy container in compose stack Reuse existing Caddy on .35 — it already proxies *.home.lab; no duplicate proxy/ports 80/443 conflict
Local DNS Pi-hole records Not needed — using public domain *.lab.audasmedia.com.au (real TLD, always resolves; *.home.lab failed as it's not a registered TLD)
LLM routing deploy omniroute container in stack Reuse existing OmniRoute .13:20128/20129
Object storage deploy Garage in stack Fresh Garage deployment in our own stack, data on /mnt/data/family-home-lab/garage/ (new subdir; Takeout landing zone untouched). Old garage-garage-1 test instance retired after data check

Directory layout (on .13)

/home/sam/Docker/Containers/family-home-lab/   # compose project (matches existing convention)
├── docker-compose.yml
├── .env                      # secrets (git-ignored)
├── garage/garage.toml
├── portal/                   # custom FastAPI app
│   ├── main.py               # async web controller
│   ├── auth.py               # user login/session handling
│   ├── database.py           # PostgreSQL + pgvector
│   ├── tasks.py              # background workers (Celery)
│   ├── templates/            # HTMX/Jinja2 frontend
│   └── static/
/mnt/data/family-home-lab/    # heavy data (NOT backed by root drive)
├── garage/garage-data        # S3 object data
├── garage-meta/
└── shared-media/

Repo lives here (family_home_lab working dir) with subfolders per tool category: video/ image/ audio/ docs/ ai/ chat/ — no dsh folder (owned by other agent).

Queue: RabbitMQ container deployed fresh inside this stack (verified none exists on network).

3. Core stack

Component Technology Rationale
Console portal FastAPI + Jinja2 + HTMX lightweight, employable skillset, no SPA build chain needed
Auth per-user accounts (bcrypt password hash), signed HTTP-only session cookies; admin panel for Sam to manage users simple, self-contained; no external IdP overhead for 4 users
Database pgvector/pgvector:pg16 user profiles + preferences now; semantic memory vectors later
Task queue Celery + Redis (downgrade from blueprint's RabbitMQ unless queue UI is wanted) lighter on 15.5 GB RAM shared with existing services; swap-in RabbitMQ is trivial if wanted for resume value
Object storage Garage v1.x S3-compatible pure OSS Rust S3; buckets per user + shared-media; boto3 from portal
Proxy/DNS existing Caddy (.35) + Pi-hole records consistent with rest of network

URLs (final — public domain, no local DNS needed)

All domains resolve via public DNS lab.audasmedia.com.au → 144.6.86.11 → router → .35 Caddy → internal hosts.

Domain Target Tool
console.lab.audasmedia.com.au .35 → .13:8500 The family console (this project)
photo.lab.audasmedia.com.au .13 photopea container Image editing
video.lab.audasmedia.com.au .13 video editor container Video editing
audio.lab.audasmedia.com.au .13 audio container Audio editing
s3.lab.audasmedia.com.au .13 Garage API S3 endpoint (internal/API use)
dsh instances dsh-<user>.lab.audasmedia.com.au Built by other agent — see §6

4. Tool catalogue (phase 1)

Each tool = its own subfolder here, own URL, dockerised with WebUI, mounted against Garage buckets or bind-mounted media dirs.

  • Image: Photopea (self-hosted)
  • Video: KdenLive via LinuxServer webtop/KasmVNC image ⚠️ verify current image name — LSIO deprecated some standalone rdesktop apps
  • Audio: Audacity via LinuxServer webtop image (Zrythm optional phase 2)
  • AI chat: OmniRoute dashboard (.13:20128, omniroute.home.lab) linked/inlined. Open WebUI — dropped; dsh instances will replace it.
  • Docs / Chat: future phases (dsh per-user instances are the chat layer)

5. S3 storage integration (final)

  • Fresh Garage v1.x container in our stack. data_dir = /mnt/data/family-home-lab/garage-data, metadata_dir = /mnt/data/family-home-lab/garage-meta. Ports 3900 (S3 API) / 3902 (web) — old test instance stopped first to free them.
  • Buckets: sam, jo, harry, finn, shared-media.
  • Portal uploads/downloads via boto3 (S3v4 signatures); tools either bind-mount or use rclone/s3fs mounts where they can't talk S3 natively.
  • Backups: add /mnt/data/family-home-lab/ to .13's existing Borg job source list → covered by rsync→.35→.23 chain.
  • /mnt/data's existing Takeout dirs (01_keep/02_review/03_delete/takeout/.thumbs) are never touched.

6. DeepSeek Harness integration contract (NOT building dsh)

Update the dsh Obsidian note to require:

  • Each dsh instance reachable at dsh-<user>.home.lab via .35 Caddy (subdomains, not subpaths — avoids WebSocket/path issues).
  • Console links to the correct instance after login; optionally auto-auth via one-time token passed in query param (to be designed with the dsh agent).
  • dsh containers must be iframe-friendly (correct CSP/X-Frame-Options headers set by Caddy) if embedded inline in the console.
  • Session handling stays inside dsh; the console only handles entry-point routing.

7. Deployment

  1. Stop/retire old garage-garage-1 test instance (after du sanity check of its volume) ✅ done
  2. Apply Caddyfile entries on .35 from deploy/caddy/Caddyfile.snippet (tracked via filechanges extension)
  3. docker compose up -d from /home/sam/Docker/Containers/family-home-lab/ on .13
  4. Create Garage buckets + access keys; first-run admin setup creates Sam's admin account, then Jo/Harry/Finn
  5. Add /mnt/data/family-home-lab/ to Borg backup source list on .13

DNS note: public domain *.lab.audasmedia.com.au is used — no Pi-hole local records required. *.home.lab domains are deprecated (not a registered TLD; resolution unreliable).

8. Resolved decisions

  1. Queue: RabbitMQ (own container in our stack; verified no conflict — Mosquitto :1883 was the false memory)
  2. Video/audio editors: browser-streamed desktop containers ✓
  3. Portal port: 8500 on .13 ✓
  4. Auth: username + password (bcrypt), signed session cookies, all four users ✓
  5. Phase-1 tools: Photopea, video editor, audio editor, OmniRoute link ✓ (Open WebUI dropped — dsh replaces)
  6. Storage: fresh Garage on /mnt/data/family-home-lab/; old test instance retired after du check ✓
  7. Compose location: /home/sam/Docker/Containers/family-home-lab/ ✓
  8. Frontend: FastAPI + Jinja2 + HTMX, server-rendered, no SPA build chain ✓

9. Verified facts (live checks)

  • .13 containers include: omniroute (:20128/29), garage (:3900/3902), n8n, airflow, langfuse, litellm (:4000), mosquitto (:1883), pihole (:53/:8080), pocketbase (:8090), prefect (:4200), minio (:9010/:9090), postgres (:5432/:5433/:5434). No RabbitMQ. Port 8500 free ✓.

10. Build status (repo-local, 2026-08-24)

Item State
portal/ FastAPI app (main/auth/db/tasks/tools/config) ✅ written & compiles
Templates + tokens.css/app.css (9 pages, HTMX polling) ✅ written, all render
docker-compose.yml (portal/db/redis/rabbitmq/garage/worker) ✅ written, config -q valid
Tool containers (photo/video/audio) ✅ media tools LIVE: photo/video/audio.lab.audasmedia.com.au (Photopea/KdenLive/Audacity)
garage/garage.toml + .env.example ✅ written + applied live
First-run admin bootstrap (env) ✅ implemented + verified
Local smoke test (login→dashboard→admin→add-user) ✅ passed — caught+fixed 2 bugs (session_scope, non-fatal broker call)
Deploy to .13 + Garage buckets/keys ✅ core stack live on console.lab.audasmedia.com.au
Caddy rules on .35 ✅ console + photo/video/audio + dsh-sam/jo/harry/finn applied & reloaded
DeepSeek Harness ✅ built here (dsh/) + 4 instances live: dsh-{user}.lab.audasmedia.com.au → .13:3081-3084 (auto/best-chat via OmniRoute)
Tool page workspace files ✅ /tool/{id} lists user's S3 bucket + shared-media
Borg backup path on .13 ⏳ needs sudo edit to backup.nix (snippet provided)

Deployed 2026-08-24: console live + media tools live. Buckets sam, jo-media, harry, finn, shared-media (S3 needs ≥3-char names → jo→jo-media). Media host ports reassigned: video 8083 (8081=taken by airflow), audio 8084 (3000=taken by a NixOS service); photopea 8487. Images: shtse8/photopea:1.0, lscr.io/linuxserver/kdenlive, lscr.io/linuxserver/audacity. OmniRoute console link uses proper URL omniroute.lab.audasmedia.com.au.

Audio / GPU follow-ups (2026-08-25)

  • Zrythm DAW: install NATIVELY on NixOS via pkgs.zrythm (no docker/webtop — no apt/AppImage). Access via RustDesk (RustDesk server already runs on .35: hbbs/hbbr). DAW audio latency over remote is ok for composing, not live multitrack recording.
  • AMD RX 6600 being added to .13: configure amdgpu driver in NixOS so containers/ML can use it (MuScriptor/Basic Pitch/vision speedup). Currently .13 has no working NVIDIA/AMD driver or container runtime (nvidia-smi fails; docker only has runc).
Reference in New Issue View Git Blame Copy Permalink