docs: add comprehensive technical architecture specs for DSH, Portal, and Host Ports
This commit is contained in:
139
docs/portal-architecture.md
Normal file
139
docs/portal-architecture.md
Normal file
@@ -0,0 +1,139 @@
|
||||
# Console Portal Architecture & Integration Spec
|
||||
|
||||
## 1. System Overview
|
||||
|
||||
The **Family Home Lab Console Portal** (`portal/`) serves as the central unified dashboard, authentication gateway, file manager, and service control plane for the home lab (`console.lab.audasmedia.com.au`).
|
||||
|
||||
It is built with **FastAPI**, **Jinja2 templates**, **HTMX**, and **PostgreSQL**, running as container `family-home-lab-portal-1` on host `.13:8500`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Technology Stack & Architecture
|
||||
|
||||
```
|
||||
Browser (HTMX / Vanilla JS)
|
||||
│
|
||||
▼
|
||||
Caddy Reverse Proxy (.35:443) ──[HTTPS]──► Portal Container (.13:8500)
|
||||
│
|
||||
┌───────────────────┬───────────────────────┼───────────────────┐
|
||||
▼ ▼ ▼ ▼
|
||||
PostgreSQL (pg16) Redis / RabbitMQ Garage (S3) service-controller
|
||||
(portal DB) (task queue / cache) (.13:3900) (.13:8443)
|
||||
```
|
||||
|
||||
- **Framework**: FastAPI (async Python 3.12)
|
||||
- **Frontend**: Jinja2 HTML templates + HTMX (interactivity) + CSS custom tokens (`DESIGN.md` sticker system)
|
||||
- **Database**: PostgreSQL (`pgvector/pgvector:pg16`), SQLAlchemy async ORM (`asyncpg`)
|
||||
- **Queue / Async Tasks**: Celery + Redis + RabbitMQ
|
||||
- **Object Storage**: Garage S3 (`:3900`)
|
||||
- **Service Control**: Host-side `service-controller` (`:8443`)
|
||||
|
||||
---
|
||||
|
||||
## 3. Database Schema & User Entitlements
|
||||
|
||||
The user model (`portal/database.py`) defines authentication credentials, administrative roles, and feature capability flags:
|
||||
|
||||
### `User` Entity Schema
|
||||
```python
|
||||
class User(Base):
|
||||
__tablename__ = "users"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
username: Mapped[str] = mapped_column(String(50), unique=True)
|
||||
password_hash: Mapped[str] = mapped_column(String(255))
|
||||
full_name: Mapped[str] = mapped_column(String(100))
|
||||
is_admin: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
|
||||
# Per-user capability flags
|
||||
can_chat: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
can_image: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
can_video: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
can_audio: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
can_docs: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
```
|
||||
|
||||
### Access Control Rules (`tools_for_user(user)`)
|
||||
- **Admins (`is_admin=True`)**: Unrestricted access to all tools, categories, and service toggles.
|
||||
- **Non-Admin Family Members**:
|
||||
- Open categories: `media` (Media & Fun), `data`, `storage`, `home`, `home-mgmt`, `dev`, `portfolio`.
|
||||
- Feature-gated categories: `chat`/`ai` (`can_chat`), `image` (`can_image`), `video` (`can_video`), `audio` (`can_audio`), `docs` (`can_docs`).
|
||||
- Gated tools: Infrastructure (`infra`) and Network (`network`) tools marked `admin=True` are filtered out.
|
||||
|
||||
---
|
||||
|
||||
## 4. Tool Catalogue (`portal/tools.py`)
|
||||
|
||||
Tools are registered in `TOOLS = _cat()` using the `Tool` dataclass:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class Tool:
|
||||
id: str
|
||||
name: str
|
||||
category: str
|
||||
description: str
|
||||
url: str
|
||||
mode: str = "tab" # "embed" or "tab"
|
||||
status: str = "online" # initial visual default
|
||||
external: bool = True
|
||||
private: bool = False
|
||||
lan: bool = False # LAN-only reachable badge
|
||||
login: bool = False # Has separate login badge
|
||||
admin: bool = False # Admins only
|
||||
admin_service: bool = False # Service control admin only
|
||||
service_id: str | None = None # Controller allowlist ID
|
||||
ram_mb: int | None = None # Estimated idle RAM footprint
|
||||
togglable: bool = False # Displays ON/OFF service strip
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. ON/OFF Service Controller Integration
|
||||
|
||||
To allow family members to free up RAM on `.13` without exposing `/var/run/docker.sock` to the web portal, all container/systemd actions pass through a **token-gated, host-side controller process**.
|
||||
|
||||
### Architecture
|
||||
1. **Host Process**: `service-controller` runs natively on host `.13` as a systemd user unit (`family-service-controller.service`) under user `sam` on `127.0.0.1:8443`.
|
||||
2. **Security Boundary**: The controller enforces an allowlist (`deploy/service-controller/services.toml`). Requests for unlisted services return `404`.
|
||||
3. **Portal Proxy Endpoints**:
|
||||
- `GET /api/services`: Fetches live status for allowlisted services from `http://host.docker.internal:8443/services`.
|
||||
- `POST /service/{id}/{action}`: Sends `start` or `stop` command with `X-Controller-Token` header.
|
||||
|
||||
### Togglable Services Allowlist
|
||||
| Service ID | Target / Kind | Idle RAM | Group | Allowed Users |
|
||||
| :--- | :--- | :--- | :--- | :--- |
|
||||
| `gimp` | `family-home-lab-gimp-1` (Container) | ~233 MB | `media` | `sam`, `finn`, `harry`, `jo` |
|
||||
| `video-editor` | `family-home-lab-video-editor-1` (Container) | ~237 MB | `media` | `sam`, `finn`, `harry`, `jo` |
|
||||
| `audio-editor` | `family-home-lab-audio-editor-1` (Container) | ~201 MB | `media` | `sam`, `finn`, `harry`, `jo` |
|
||||
| `lmms` | `lmms` (Container) | ~798 MB | `media` | `sam`, `finn`, `harry`, `jo` |
|
||||
| `paperclip` | `paperclipai` (systemd user) | ~495 MB | `admin` | `sam` |
|
||||
| `prefect-worker` | `prefect-worker` (systemd user) | ~129 MB | `pipeline` | `sam` |
|
||||
| `photo-dashboard` | `photo-dashboard` (systemd user) | ~112 MB | `pipeline` | `sam` |
|
||||
| `worldmonitor` | 4-container group (Container) | ~120 MB | `admin` | `sam` |
|
||||
|
||||
---
|
||||
|
||||
## 6. Garage S3 & File Management
|
||||
|
||||
The portal includes an integrated S3 file manager (`portal/s3.py`, `portal/templates/files.html`):
|
||||
- Connects to internal Garage S3 endpoint (`http://garage:3900`).
|
||||
- Each user gets a personal bucket (`sam`, `jo`, `harry`, `finn`) plus access to `shared-media`.
|
||||
- File actions: Upload, Download, Delete, Preview, Create Directory.
|
||||
|
||||
---
|
||||
|
||||
## 7. Operational & Deployment Commands
|
||||
|
||||
Deployment directory on host `.13`: `/home/sam/Docker/Containers/family-home-lab`
|
||||
|
||||
```bash
|
||||
# Sync local changes from .27 to .13
|
||||
rsync -avz --exclude='.git' --exclude='.venv' \
|
||||
/home/sam/home_network/custom_tools/family_home_lab/ \
|
||||
sam@192.168.20.13:/home/sam/Docker/Containers/family-home-lab/
|
||||
|
||||
# Rebuild and restart portal container on .13
|
||||
ssh sam@192.168.20.13 "cd /home/sam/Docker/Containers/family-home-lab && docker compose build portal && docker compose up -d portal"
|
||||
```
|
||||
Reference in New Issue
Block a user