Console Portal Architecture & Integration Spec

1. System Overview

The Family Home Lab Console Portal (portal/) serves as the central unified dashboard, authentication gateway, file manager, and service control plane for the home lab (console.lab.audasmedia.com.au).

It is built with FastAPI, Jinja2 templates, HTMX, and PostgreSQL, running as container family-home-lab-portal-1 on host .13:8500.

2. Technology Stack & Architecture

 Browser (HTMX / Vanilla JS)
      │
      ▼
 Caddy Reverse Proxy (.35:443)  ──[HTTPS]──►  Portal Container (.13:8500)
                                                    │
        ┌───────────────────┬───────────────────────┼───────────────────┐
        ▼                   ▼                       ▼                   ▼
 PostgreSQL (pg16)    Redis / RabbitMQ         Garage (S3)       service-controller
 (portal DB)          (task queue / cache)     (.13:3900)        (.13:8443)

Framework: FastAPI (async Python 3.12)
Frontend: Jinja2 HTML templates + HTMX (interactivity) + CSS custom tokens (DESIGN.md sticker system)
Database: PostgreSQL (pgvector/pgvector:pg16), SQLAlchemy async ORM (asyncpg)
Queue / Async Tasks: Celery + Redis + RabbitMQ
Object Storage: Garage S3 (:3900)
Service Control: Host-side service-controller (:8443)

3. Database Schema & User Entitlements

The user model (portal/database.py) defines authentication credentials, administrative roles, and feature capability flags:

`User` Entity Schema

class User(Base):
    __tablename__ = "users"

    id: Mapped[int] = mapped_column(primary_key=True)
    username: Mapped[str] = mapped_column(String(50), unique=True)
    password_hash: Mapped[str] = mapped_column(String(255))
    full_name: Mapped[str] = mapped_column(String(100))
    is_admin: Mapped[bool] = mapped_column(Boolean, default=False)

    # Per-user capability flags
    can_chat: Mapped[bool] = mapped_column(Boolean, default=True)
    can_image: Mapped[bool] = mapped_column(Boolean, default=True)
    can_video: Mapped[bool] = mapped_column(Boolean, default=True)
    can_audio: Mapped[bool] = mapped_column(Boolean, default=True)
    can_docs: Mapped[bool] = mapped_column(Boolean, default=False)

Access Control Rules (`tools_for_user(user)`)

Admins (is_admin=True): Unrestricted access to all tools, categories, and service toggles.
Non-Admin Family Members:
- Open categories: media (Media & Fun), data, storage, home, home-mgmt, dev, portfolio.
- Feature-gated categories: chat/ai (can_chat), image (can_image), video (can_video), audio (can_audio), docs (can_docs).
- Gated tools: Infrastructure (infra) and Network (network) tools marked admin=True are filtered out.

4. Tool Catalogue (`portal/tools.py`)

Tools are registered in TOOLS = _cat() using the Tool dataclass:

@dataclass(frozen=True)
class Tool:
    id: str
    name: str
    category: str
    description: str
    url: str
    mode: str = "tab"             # "embed" or "tab"
    status: str = "online"         # initial visual default
    external: bool = True
    private: bool = False
    lan: bool = False              # LAN-only reachable badge
    login: bool = False            # Has separate login badge
    admin: bool = False            # Admins only
    admin_service: bool = False    # Service control admin only
    service_id: str | None = None  # Controller allowlist ID
    ram_mb: int | None = None      # Estimated idle RAM footprint
    togglable: bool = False        # Displays ON/OFF service strip

5. ON/OFF Service Controller Integration

To allow family members to free up RAM on .13 without exposing /var/run/docker.sock to the web portal, all container/systemd actions pass through a token-gated, host-side controller process.

Architecture

Host Process: service-controller runs natively on host .13 as a systemd user unit (family-service-controller.service) under user sam on 127.0.0.1:8443.
Security Boundary: The controller enforces an allowlist (deploy/service-controller/services.toml). Requests for unlisted services return 404.
Portal Proxy Endpoints:
- GET /api/services: Fetches live status for allowlisted services from http://host.docker.internal:8443/services.
- POST /service/{id}/{action}: Sends start or stop command with X-Controller-Token header.

Togglable Services Allowlist

Service ID	Target / Kind	Idle RAM	Group	Allowed Users
`gimp`	`family-home-lab-gimp-1` (Container)	~233 MB	`media`	`sam`, `finn`, `harry`, `jo`
`video-editor`	`family-home-lab-video-editor-1` (Container)	~237 MB	`media`	`sam`, `finn`, `harry`, `jo`
`audio-editor`	`family-home-lab-audio-editor-1` (Container)	~201 MB	`media`	`sam`, `finn`, `harry`, `jo`
`lmms`	`lmms` (Container)	~798 MB	`media`	`sam`, `finn`, `harry`, `jo`
`paperclip`	`paperclipai` (systemd user)	~495 MB	`admin`	`sam`
`prefect-worker`	`prefect-worker` (systemd user)	~129 MB	`pipeline`	`sam`
`photo-dashboard`	`photo-dashboard` (systemd user)	~112 MB	`pipeline`	`sam`
`worldmonitor`	4-container group (Container)	~120 MB	`admin`	`sam`

6. Garage S3 & File Management

The portal includes an integrated S3 file manager (portal/s3.py, portal/templates/files.html):

Connects to internal Garage S3 endpoint (http://garage:3900).
Each user gets a personal bucket (sam, jo, harry, finn) plus access to shared-media.
File actions: Upload, Download, Delete, Preview, Create Directory.

7. Operational & Deployment Commands

Deployment directory on host .13: /home/sam/Docker/Containers/family-home-lab

# Sync local changes from .27 to .13
rsync -avz --exclude='.git' --exclude='.venv' \
    /home/sam/home_network/custom_tools/family_home_lab/ \
    sam@192.168.20.13:/home/sam/Docker/Containers/family-home-lab/

# Rebuild and restart portal container on .13
ssh sam@192.168.20.13 "cd /home/sam/Docker/Containers/family-home-lab && docker compose build portal && docker compose up -d portal"

6.3 KiB Raw Blame History