obsidian-vault/000 daily/FAMILY Enterprise Homelab AI Multimedia Suite. Systems Architecture & Blueprint.md

created, modified, type, tags, aliases

created	modified	type	tags	aliases
2026-08-21 21:37	2026-08-21 21:37	note	ai family tools

FAMILY Enterprise Homelab AI Multimedia Suite. Systems Architecture & Blueprint

Build status: see FAMILY Home Lab Console — Build Status (portal written repo-local, deployment pending).

SUPERSEDED by final plan in /home/sam/home_network/custom_tools/family_home_lab/plan.md (approved). Key changes from this blueprint: host = .13; reuse existing Caddy on .35 + Pi-hole DNS (no Caddy container); RabbitMQ deployed fresh (verified none exists); Garage S3 data on /mnt/data/family-home-lab/ (see FAMILY S3 Storage Integration & Blueprint); compose at /home/sam/Docker/Containers/family-home-lab/; portal port 8500; frontend FastAPI+HTMX; Open WebUI retired in favour of DeepSeek Harness instances (FAMILY DeepSeek Harness (dsh) Home Lab Setup); users Sam/Jo/Harry/Finn with username+password auth. Original blueprint below for reference.

Enterprise Homelab AI Multimedia Suite: Systems Architecture & Blueprint

This document outlines the deployment strategy for a self-hosted, custom-built multimedia and AI workspace. It features individual family login profiles, an asynchronous RabbitMQ queue, and persistent user understanding via PostgreSQL with pgvector.

1. Directory Structure (Managed via Pi)

/home/user/ai-studio/
├── docker-compose.yml
├── caddy/
│   └── Caddyfile
├── gateway-app/            # Custom FastAPI Web Application
│   ├── main.py            # Async Web Controller
│   ├── database.py        # PostgreSQL Connection Layer
│   ├── tasks.py           # Background Worker Tasks (Celery)
│   ├── templates/         # HTMX / Frontend Layouts
│   └── static/
└── storage/
    ├── shared_media/      # Central Media Asset Pool
    ├── mum_workspace/
    └── son_workspace/

2. Core Production Infrastructure Stack (docker-compose.yml)

version: '3.8'

services:
  # 1. NETWORK PROXY
  caddy:
    image: caddy:2-alpine
    container_name: network_proxy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./caddy/Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_config:/config
    network_mode: host

  # 2. THE MASTER ENTRY PORTAL (Custom Gateway UI)
  studio-portal:
    image: python:3.11-slim
    container_name: studio_portal_app
    restart: unless-stopped
    working_dir: /app
    command: >
      sh -c "pip install fastapi uvicorn psycopg2-binary celery jinja2 python-multipart && 
             uvicorn main:app --host 0.0.0.0 --port 8000"
    ports:
      - "8000:8000"
    volumes:
      - ./gateway-app:/app
      - ./storage:/app/storage
    depends_on:
      - studio-db
      - studio-rabbitmq

  # 3. ASYNCHRONOUS MEDIA WORKER (Fueled by RabbitMQ)
  media-worker:
    image: python:3.11-slim
    container_name: async_media_worker
    restart: unless-stopped
    working_dir: /app
    command: celery -A tasks worker --loglevel=info
    volumes:
      - ./gateway-app:/app
      - ./storage:/app/storage
      - /var/run/docker.sock:/var/run/docker.sock # Safe container orchestration loop
    depends_on:
      - studio-rabbitmq

  # 4. INDUSTRIAL MESSAGE BROKER (RabbitMQ)
  studio-rabbitmq:
    image: rabbitmq:3-management-alpine
    container_name: studio_message_broker
    restart: unless-stopped
    ports:
      - "5672:5672"   # RabbitMQ message port
      - "15672:15672" # Management Web UI dashboard
    environment:
      - RABBITMQ_DEFAULT_USER=studio_broker
      - RABBITMQ_DEFAULT_PASS=broker_secure_pass

  # 5. ENTERPRISE COGNITIVE DATABASE
  studio-db:
    image: pgvector/pgvector:pg16 # PostgreSQL natively equipped with AI Vector support
    container_name: studio_cognitive_db
    restart: unless-stopped
    environment:
      - POSTGRES_USER=studio_admin
      - POSTGRES_PASSWORD=studio_secure_pass
      - POSTGRES_DB=studio_memory_matrix
    volumes:
      - postgres_data:/var/lib/postgresql/data

  # 6. OMNIROUTE GATEWAY
  omniroute:
    image: omniroute/gateway:latest
    container_name: omniroute_gateway
    restart: unless-stopped
    ports:
      - "20128:20128"
    environment:
      - OPENAI_API_KEY=your_secure_cloud_key
      - GEMINI_API_KEY=your_secure_cloud_key
    volumes:
      - ./omniroute/config:/app/config

  # 7. MULTIMEDIA STUDIO CONTAINERS
  photopea:
    image: shtse8/photopea:1.0
    container_name: photopea_studio
    ports:
      - "8487:8887"

  kdenlive-studio:
    image: lscr.io/linuxserver/kdenlive:latest # HTML5 Streamed Pro Video Studio
    container_name: pro_video_studio
    ports:
      - "8081:3000"
    environment:
      - PUID=1000
      - PGID=1000
    volumes:
      - ./storage/shared_media:/config

  # AUDIO STUDIO A: Browser-Streamed Audacity
  audacity-studio:
    image: lscr.io/linuxserver/audacity:latest 
    container_name: pro_audio_audacity
    ports:
      - "3000:3000"
    environment:
      - PUID=1000
      - PGID=1000
    volumes:
      - ./storage/shared_media:/config

  # AUDIO STUDIO B: Browser-Streamed Pro DAW (Zrythm)
  zrythm-studio:
    image: lscr.io/linuxserver/zrythm:latest # Full multi-track timeline automation DAW
    container_name: pro_audio_zrythm
    ports:
      - "3001:3000"
    environment:
      - PUID=1000
      - PGID=1000
    volumes:
      - ./storage/shared_media:/config

volumes:
  postgres_data:
  caddy_data:
  caddy_config:

3. Cognitive Engine: Relational Vector Schema (database.py)

import psycopg2

def init_db():
    conn = psycopg2.connect("host=studio-db dbname=studio_memory_matrix user=studio_admin password=studio_secure_pass")
    cur = conn.cursor()
    
    # Enable the Vector extension explicitly
    cur.execute("CREATE EXTENSION IF NOT EXISTS vector;")
    
    # User Profile table
    cur.execute("""
        CREATE TABLE IF NOT EXISTS user_profiles (
            user_id VARCHAR PRIMARY KEY,
            preference_matrix JSONB,
            updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
        );
    """)
    
    # Semantic Memory Table with Vector Embeddings
    cur.execute("""
        CREATE TABLE IF NOT EXISTS user_memories (
            memory_id SERIAL PRIMARY KEY,
            user_id VARCHAR,
            summary TEXT,
            embedding vector(1536), -- Standard cloud embedding dimensions
            created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
        );
    """)
    conn.commit()
    cur.close()
    conn.close()

4. Asynchronous Task Routing Backend (tasks.py)

from celery import Celery
import subprocess

# Configured to use RabbitMQ as the robust message broker
celery_app = Celery('studio_tasks', broker='amqp://studio_broker:broker_secure_pass@studio-rabbitmq:5672//')

@celery_app.task
def process_video_rotation(input_file, output_file):
    """
    Executes a fast, headless FFmpeg run inside RabbitMQ queue context.
    Prevents the main FastAPI frontend from lagging during large transfers.
    """
    subprocess.run([
        "docker", "run", "--rm",
        "-v", "/home/user/ai-studio/storage/shared_media:/media",
        "jrottenberg/ffmpeg",
        "-i", f"/media/{input_file}",
        "-vf", "transpose=1",
        f"/media/{output_file}"
    ])

5. Network Routing Configuration (Caddyfile)

# Core Application Portal Entry Point
ds.home.lab {
    reverse_proxy 127.0.0.1:8000
}

# Image Design Lab
photo.home.lab {
    reverse_proxy 127.0.0.1:8487
}

# Pro Video Editor Layout
video.home.lab {
    reverse_proxy 127.0.0.1:8081
}

# Pro Audio Station A (Audacity Wrapper)
audacity.home.lab {
    reverse_proxy 127.0.0.1:3000
}

# Pro Audio Station B (Zrythm DAW Studio)
zrythm.home.lab {
    reverse_proxy 127.0.0.1:3001
}