154 lines
10 KiB
Markdown
154 lines
10 KiB
Markdown
# Family Home Lab Console — Plan
|
|
|
|
> Adapted from Obsidian notes: [[FAMILY Enterprise Homelab AI Multimedia Suite]], [[FAMILY S3 Storage Integration & Blueprint]], [[FAMILY DeepSeek Harness (dsh) Home Lab Setup]], [[Home Network Map Overview]], [[Backup System — Borg, Kopia & Restic]].
|
|
>
|
|
> **Status: FINAL v3 — approved. Build phase: repo-local code complete (2026-08-24), deployment pending.**
|
|
|
|
---
|
|
|
|
## 1. Goal
|
|
|
|
A single family console ("landing page") at a local URL where each of the four users
|
|
(**Sam, Jo, Harry, Finn**) logs in and gets:
|
|
|
|
- A personalised dashboard of available AI + media tools
|
|
- One-click launch of each tool (inline iframe where possible, else window/new tab)
|
|
- Per-user session handling and tool permissions
|
|
|
|
The console is also the parent interface that the DeepSeek Harness instances hang off
|
|
(dsh itself is built by another agent — we only define the integration contract here).
|
|
|
|
## 2. Where it runs (adapted to my network)
|
|
|
|
| Concern | Generic blueprint said | Adapted decision |
|
|
|---|---|---|
|
|
| Host machine | unspecified | **`.13` nixos-desktop** — always-on server, existing Docker host (15.5 GB RAM), already runs Open WebUI/OmniRoute/Langfuse |
|
|
| Reverse proxy | dedicated Caddy container in compose stack | **Reuse existing Caddy on `.35`** — it already proxies `*.home.lab`; no duplicate proxy/ports 80/443 conflict |
|
|
| Local DNS | ~~Pi-hole records~~ **Not needed** — using public domain `*.lab.audasmedia.com.au` (real TLD, always resolves; `*.home.lab` failed as it's not a registered TLD) |
|
|
| LLM routing | deploy omniroute container in stack | **Reuse existing OmniRoute** `.13:20128/20129` |
|
|
| Object storage | deploy Garage in stack | **Fresh Garage deployment in our own stack**, data on **`/mnt/data/family-home-lab/garage/`** (new subdir; Takeout landing zone untouched). Old `garage-garage-1` test instance retired after data check |
|
|
|
|
### Directory layout (on .13)
|
|
|
|
```text
|
|
/home/sam/Docker/Containers/family-home-lab/ # compose project (matches existing convention)
|
|
├── docker-compose.yml
|
|
├── .env # secrets (git-ignored)
|
|
├── garage/garage.toml
|
|
├── portal/ # custom FastAPI app
|
|
│ ├── main.py # async web controller
|
|
│ ├── auth.py # user login/session handling
|
|
│ ├── database.py # PostgreSQL + pgvector
|
|
│ ├── tasks.py # background workers (Celery)
|
|
│ ├── templates/ # HTMX/Jinja2 frontend
|
|
│ └── static/
|
|
/mnt/data/family-home-lab/ # heavy data (NOT backed by root drive)
|
|
├── garage/garage-data # S3 object data
|
|
├── garage-meta/
|
|
└── shared-media/
|
|
```
|
|
|
|
Repo lives here (`family_home_lab` working dir) with subfolders per tool category:
|
|
`video/ image/ audio/ docs/ ai/ chat/` — **no dsh folder** (owned by other agent).
|
|
|
|
Queue: RabbitMQ container deployed fresh inside this stack (verified none exists on network).
|
|
|
|
## 3. Core stack
|
|
|
|
| Component | Technology | Rationale |
|
|
|---|---|---|
|
|
| Console portal | FastAPI + Jinja2 + HTMX | lightweight, employable skillset, no SPA build chain needed |
|
|
| Auth | per-user accounts (bcrypt password hash), signed HTTP-only session cookies; admin panel for Sam to manage users | simple, self-contained; no external IdP overhead for 4 users |
|
|
| Database | `pgvector/pgvector:pg16` | user profiles + preferences now; semantic memory vectors later |
|
|
| Task queue | Celery + Redis (downgrade from blueprint's RabbitMQ unless queue UI is wanted) | lighter on 15.5 GB RAM shared with existing services; swap-in RabbitMQ is trivial if wanted for resume value |
|
|
| Object storage | **Garage v1.x** S3-compatible | pure OSS Rust S3; buckets per user + `shared-media`; boto3 from portal |
|
|
| Proxy/DNS | existing Caddy (.35) + Pi-hole records | consistent with rest of network |
|
|
|
|
### URLs (final — public domain, no local DNS needed)
|
|
|
|
All domains resolve via public DNS `lab.audasmedia.com.au` → `144.6.86.11` → router → `.35` Caddy → internal hosts.
|
|
|
|
| Domain | Target | Tool |
|
|
|---|---|---|
|
|
| `console.lab.audasmedia.com.au` | .35 → `.13:8500` | The family console (this project) |
|
|
| `photo.lab.audasmedia.com.au` | .13 photopea container | Image editing |
|
|
| `video.lab.audasmedia.com.au` | .13 video editor container | Video editing |
|
|
| `audio.lab.audasmedia.com.au` | .13 audio container | Audio editing |
|
|
| `s3.lab.audasmedia.com.au` | .13 Garage API | S3 endpoint (internal/API use) |
|
|
| dsh instances | `dsh-<user>.lab.audasmedia.com.au` | Built by other agent — see §6 |
|
|
|
|
## 4. Tool catalogue (phase 1)
|
|
|
|
Each tool = its own subfolder here, own URL, dockerised with WebUI, mounted against
|
|
Garage buckets or bind-mounted media dirs.
|
|
|
|
- **Image:** Photopea (self-hosted)
|
|
- **Video:** KdenLive via LinuxServer webtop/KasmVNC image ⚠️ *verify current image name — LSIO deprecated some standalone rdesktop apps*
|
|
- **Audio:** Audacity via LinuxServer webtop image (Zrythm optional phase 2)
|
|
- **AI chat:** OmniRoute dashboard (`.13:20128`, `omniroute.home.lab`) linked/inlined. ~~Open WebUI~~ — dropped; dsh instances will replace it.
|
|
- **Docs / Chat:** future phases (dsh per-user instances are the chat layer)
|
|
|
|
## 5. S3 storage integration (final)
|
|
|
|
- Fresh **Garage v1.x** container in our stack. `data_dir = /mnt/data/family-home-lab/garage-data`, `metadata_dir = /mnt/data/family-home-lab/garage-meta`. Ports 3900 (S3 API) / 3902 (web) — old test instance stopped first to free them.
|
|
- Buckets: `sam`, `jo`, `harry`, `finn`, `shared-media`.
|
|
- Portal uploads/downloads via boto3 (S3v4 signatures); tools either bind-mount or use rclone/s3fs mounts where they can't talk S3 natively.
|
|
- Backups: add `/mnt/data/family-home-lab/` to .13's existing Borg job source list → covered by rsync→.35→.23 chain.
|
|
- `/mnt/data`'s existing Takeout dirs (`01_keep/02_review/03_delete/takeout/.thumbs`) are never touched.
|
|
|
|
## 6. DeepSeek Harness integration contract (NOT building dsh)
|
|
|
|
Update the dsh Obsidian note to require:
|
|
|
|
- Each dsh instance reachable at `dsh-<user>.home.lab` via .35 Caddy (subdomains, not subpaths — avoids WebSocket/path issues).
|
|
- Console links to the correct instance after login; optionally auto-auth via one-time token passed in query param (to be designed with the dsh agent).
|
|
- dsh containers must be iframe-friendly (correct CSP/X-Frame-Options headers set by Caddy) if embedded inline in the console.
|
|
- Session handling stays inside dsh; the console only handles entry-point routing.
|
|
|
|
## 7. Deployment
|
|
|
|
1. Stop/retire old `garage-garage-1` test instance (after `du` sanity check of its volume) ✅ done
|
|
2. Apply Caddyfile entries on .35 from `deploy/caddy/Caddyfile.snippet` (tracked via filechanges extension)
|
|
3. `docker compose up -d` from `/home/sam/Docker/Containers/family-home-lab/` on .13
|
|
4. Create Garage buckets + access keys; first-run admin setup creates Sam's admin account, then Jo/Harry/Finn
|
|
5. Add `/mnt/data/family-home-lab/` to Borg backup source list on .13
|
|
|
|
> **DNS note:** public domain `*.lab.audasmedia.com.au` is used — no Pi-hole local records required. `*.home.lab` domains are deprecated (not a registered TLD; resolution unreliable).
|
|
|
|
## 8. Resolved decisions
|
|
|
|
1. Queue: **RabbitMQ** (own container in our stack; verified no conflict — Mosquitto :1883 was the false memory)
|
|
2. Video/audio editors: browser-streamed desktop containers ✓
|
|
3. Portal port: **8500** on .13 ✓
|
|
4. Auth: username + password (bcrypt), signed session cookies, all four users ✓
|
|
5. Phase-1 tools: Photopea, video editor, audio editor, OmniRoute link ✓ (Open WebUI dropped — dsh replaces)
|
|
6. Storage: fresh Garage on `/mnt/data/family-home-lab/`; old test instance retired after `du` check ✓
|
|
7. Compose location: `/home/sam/Docker/Containers/family-home-lab/` ✓
|
|
8. Frontend: FastAPI + Jinja2 + **HTMX**, server-rendered, no SPA build chain ✓
|
|
|
|
## 9. Verified facts (live checks)
|
|
- .13 containers include: omniroute (:20128/29), garage (:3900/3902), n8n, airflow, langfuse, litellm (:4000), mosquitto (:1883), pihole (:53/:8080), pocketbase (:8090), prefect (:4200), minio (:9010/:9090), postgres (:5432/:5433/:5434). No RabbitMQ. Port **8500 free** ✓.
|
|
|
|
## 10. Build status (repo-local, 2026-08-24)
|
|
|
|
| Item | State |
|
|
|---|---|
|
|
| `portal/` FastAPI app (main/auth/db/tasks/tools/config) | ✅ written & compiles |
|
|
| Templates + `tokens.css`/`app.css` (9 pages, HTMX polling) | ✅ written, all render |
|
|
| `docker-compose.yml` (portal/db/redis/rabbitmq/garage/worker) | ✅ written, `config -q` valid |
|
|
| Tool containers (photo/video/audio) | ✅ media tools LIVE: photo/video/audio.lab.audasmedia.com.au (Photopea/KdenLive/Audacity) |
|
|
| `garage/garage.toml` + `.env.example` | ✅ written + applied live |
|
|
| First-run admin bootstrap (env) | ✅ implemented + verified |
|
|
| Local smoke test (login→dashboard→admin→add-user) | ✅ passed — caught+fixed 2 bugs (session_scope, non-fatal broker call) |
|
|
| Deploy to .13 + Garage buckets/keys | ✅ core stack live on console.lab.audasmedia.com.au |
|
|
| Caddy rules on .35 | ✅ console + photo/video/audio + dsh-sam/jo/harry/finn applied & reloaded |
|
|
| DeepSeek Harness | ✅ built here (dsh/) + 4 instances live: dsh-{user}.lab.audasmedia.com.au → .13:3081-3084 (auto/best-chat via OmniRoute) |
|
|
| Tool page workspace files | ✅ /tool/{id} lists user's S3 bucket + shared-media |
|
|
| Borg backup path on .13 | ⏳ needs sudo edit to backup.nix (snippet provided) |
|
|
|
|
**Deployed 2026-08-24:** console live + media tools live. Buckets `sam, jo-media, harry, finn, shared-media` (S3 needs ≥3-char names → `jo`→`jo-media`). Media host ports reassigned: **video 8083** (8081=taken by airflow), **audio 8084** (3000=taken by a NixOS service); photopea 8487. Images: shtse8/photopea:1.0, lscr.io/linuxserver/kdenlive, lscr.io/linuxserver/audacity. OmniRoute console link uses proper URL omniroute.lab.audasmedia.com.au.
|
|
|
|
## Audio / GPU follow-ups (2026-08-25)
|
|
- Zrythm DAW: install NATIVELY on NixOS via `pkgs.zrythm` (no docker/webtop — no apt/AppImage). Access via **RustDesk** (RustDesk server already runs on .35: hbbs/hbbr). DAW audio latency over remote is ok for composing, not live multitrack recording.
|
|
- AMD **RX 6600** being added to **.13**: configure `amdgpu` driver in NixOS so containers/ML can use it (MuScriptor/Basic Pitch/vision speedup). Currently .13 has no working NVIDIA/AMD driver or container runtime (nvidia-smi fails; docker only has runc).
|