Sam Rolfe
2609ab5286
The dropdown is built from the hardcoded MODEL_CHOICES list in app.py, not from OmniRoute, so 'ollama/qwen2.5*' existed upstream but could never appear in DSH. - add ollama/qwen2.5:14b and ollama/qwen2.5-coder:14b to MODEL_CHOICES + MODEL_LABELS - labels note that Ollama must be started first, since it now boots off on .27 - docs/dsh-architecture.md: new section on model selection, incl. the rebuild requirement (app.py is baked into the image via COPY . .) and the ordering trap that a model can be live in OmniRoute yet invisible in DSH Verified: /api/models lists both ids on 3081-3084, and real streamed replies come back from both local models through DSH.
155 lines
7.2 KiB
Markdown
155 lines
7.2 KiB
Markdown
# DeepSeek Harness (DSH) — Architecture & Integration Spec
|
|
|
|
## 1. System Overview
|
|
|
|
**DeepSeek Harness (`dsh`)** is a single-user, multi-tenant streaming chat web application that serves as the AI chat interface for Family Home Lab members. Instead of running a single shared application with multi-user authentication, DSH is deployed as **isolated, per-user container instances** on `.13`.
|
|
|
|
The portal (`console.lab.audasmedia.com.au`) embeds each user's dedicated DSH instance inline via iframe or links directly to their subdomains (`https://dsh-{username}.lab.audasmedia.com.au`).
|
|
|
|
---
|
|
|
|
## 2. Multi-Tenancy & Port Allocation
|
|
|
|
Each family member gets a dedicated DSH container instance running on host `.13`:
|
|
|
|
| User | Subdomain | Host Port | Container Name | Host Workspace Directory |
|
|
| :--- | :--- | :--- | :--- | :--- |
|
|
| **Sam** | `dsh-sam.lab.audasmedia.com.au` | `3081` | `dsh-sam` | `/mnt/data/family-home-lab/dsh/sam` |
|
|
| **Jo** | `dsh-jo.lab.audasmedia.com.au` | `3082` | `dsh-jo` | `/mnt/data/family-home-lab/dsh/jo` |
|
|
| **Harry** | `dsh-harry.lab.audasmedia.com.au` | `3083` | `dsh-harry` | `/mnt/data/family-home-lab/dsh/harry` |
|
|
| **Finn** | `dsh-finn.lab.audasmedia.com.au` | `3084` | `dsh-finn` | `/mnt/data/family-home-lab/dsh/finn` |
|
|
|
|
### Reverse Proxy & Routing
|
|
- Caddy on `.35` terminates HTTPS for `dsh-*.lab.audasmedia.com.au` and proxies traffic to `.13:{port}`.
|
|
- Portal `tools.py` dynamically resolves the user's DSH URL via `_dsh_for_user(user)` based on their authenticated portal session.
|
|
|
|
---
|
|
|
|
## 3. Storage & Session Persistence
|
|
|
|
Chat history and sessions are stored directly as JSON files inside each container's `/workspace/sessions/` directory (mapped to host `/mnt/data/family-home-lab/dsh/{user}`):
|
|
|
|
### Session Directory Structure
|
|
```
|
|
/workspace/
|
|
├── sessions/
|
|
│ ├── main.json
|
|
│ ├── session-172840192.json
|
|
│ └── ...
|
|
└── uploads/
|
|
```
|
|
|
|
### Session File Schema (`sessions/{id}.json`)
|
|
```json
|
|
{
|
|
"id": "session-172840192",
|
|
"name": "Project discussion",
|
|
"updated": "2026-10-08T19:30:00Z",
|
|
"messages": [
|
|
{
|
|
"role": "user",
|
|
"content": "Summarise the home lab setup",
|
|
"timestamp": 172840191
|
|
},
|
|
{
|
|
"role": "assistant",
|
|
"content": "The family home lab consists of...",
|
|
"timestamp": 172840192
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Multi-Model Routing & External Integration Table
|
|
|
|
DSH routes user requests across specialized AI services and local house infrastructure:
|
|
|
|
| Function | Backend Endpoint | Model / Engine | Environment Variable |
|
|
| :--- | :--- | :--- | :--- |
|
|
| **Text Chat (SSE Stream)** | `http://192.168.20.13:20129/v1` | `auto/best-chat` (OmniRoute) | `DSH_LLM_BASE`, `DSH_LLM_MODEL` |
|
|
| **Vision (Image Ingest)** | `https://openrouter.ai/api/v1` | `openai/gpt-5` | `DSH_VISION_BASE`, `DSH_VISION_MODEL` |
|
|
| **Speech-to-Text (STT)** | `http://192.168.20.13:5000/transcribe` | `faster-whisper` (`voice_whisper`) | `DSH_WHISPER_URL` |
|
|
| **Voice Agent (Action/TTS)** | `http://192.168.20.13:8501/voice` | Jervis Voice Agent | `DSH_VOICE_AGENT_URL` |
|
|
| **Image Generation** | `https://openrouter.ai/api/v1` | `openai/gpt-image-1` | `DSH_IMAGE_MODEL` |
|
|
| **Video Generation** | `https://openrouter.ai/api/v1` | `z-ai/glm-5v-turbo` | `DSH_VIDEO_MODEL` |
|
|
|
|
---
|
|
|
|
## 5. Model Selection (the chat dropdown)
|
|
|
|
**The model list is hardcoded in `dsh/app.py`** (`MODEL_CHOICES`, ~line 660). DSH does
|
|
**not** discover models from OmniRoute. `GET /api/models` simply returns that list
|
|
plus `MODEL_LABELS` for the friendly names.
|
|
|
|
Consequence: a model can exist in OmniRoute and still be invisible in DSH until it
|
|
is added to `MODEL_CHOICES` **and** `MODEL_LABELS`.
|
|
|
|
A chosen model is validated server-side (`if active_model not in MODEL_CHOICES:
|
|
active_model = LLM_MODEL`), so an unlisted id silently falls back to the default.
|
|
|
|
### Adding a model
|
|
|
|
1. Confirm it works through OmniRoute first — `curl http://127.0.0.1:20129/v1/models`
|
|
and then a real `chat/completions` call with a short timeout.
|
|
2. Add the id to `MODEL_CHOICES` and a friendly string to `MODEL_LABELS` in
|
|
`dsh/app.py`.
|
|
3. Deploy: copy `app.py` to `.13:/home/sam/Docker/Containers/dsh/`, then
|
|
`docker compose build && docker compose up -d` in that directory (the image
|
|
bakes `app.py` via `COPY . .`, so a rebuild is required — restarting is not enough).
|
|
4. Verify: `curl -s http://127.0.0.1:3081/api/models` should list the new id.
|
|
|
|
### Local Ollama models
|
|
|
|
The `.27` models are exposed by OmniRoute as `ollama/qwen2.5:14b` and
|
|
`ollama/qwen2.5-coder:14b` (`ollama-local/…` also exists but is duplicated).
|
|
Because Ollama now starts **off** at boot, these fail until it is started from the
|
|
family console; hence the `needs Ollama on` label. Inference is CPU-only on `.27`,
|
|
so expect it to be slower than the cloud routes.
|
|
|
|
---
|
|
|
|
## 6. API Endpoints & Request Flow
|
|
|
|
### Primary API Routes (`dsh/app.py`)
|
|
|
|
| Method | Route | Description |
|
|
| :--- | :--- | :--- |
|
|
| `GET` | `/` | Serves main single-page chat UI (`templates/chat.html`). |
|
|
| `POST` | `/chat/stream` | Primary chat endpoint. Receives `{message, session_id}`,Streams response via Server-Sent Events (SSE). |
|
|
| `POST` | `/transcribe` | Uploads `.wav`/`.mp3`/`.m4a` file; forwards to `voice_whisper` (`:5000`); returns transcribed text. |
|
|
| `POST` | `/upload-image` | Uploads image; forwards to Vision model (`openai/gpt-5`) with user prompt; returns image description. |
|
|
| `POST` | `/voice-agent` | Forwards prompt to Jervis Voice Agent (`:8501`); triggers house audio/action. |
|
|
| `POST` | `/generate-image` | Generates image via OpenRouter API and saves output to workspace. |
|
|
| `POST` | `/generate-video` | Generates video via OpenRouter API and saves output to workspace. |
|
|
| `GET` | `/sessions` | Returns list of all session summaries for the current user. |
|
|
| `POST` | `/sessions/new` | Creates a new conversation session. |
|
|
| `POST` | `/sessions/{id}/delete` | Deletes a conversation session file. |
|
|
|
|
---
|
|
|
|
## 8. Embedding & Security Headers
|
|
|
|
To support seamless inline framing inside the Family Home Lab Console (`/tool/dsh`):
|
|
- **Content Security Policy (CSP)**: `frame-ancestors https://console.lab.audasmedia.com.au`
|
|
- **X-Frame-Options**: Explicitly omitted (not set to `DENY` or `SAMEORIGIN`), allowing Caddy and Portal to frame DSH cleanly.
|
|
|
|
---
|
|
|
|
## 9. Environment Variables Reference
|
|
|
|
| Variable | Default Value | Purpose |
|
|
| :--- | :--- | :--- |
|
|
| `DSH_USER` | `user` | Name of the user (e.g., `sam`, `jo`). |
|
|
| `DSH_SLOT` | `Welcome to DeepSeek Harness` | Subtitle/greeting shown in the header. |
|
|
| `DSH_LLM_BASE` | `http://192.168.20.13:20129/v1` | OmniRoute OpenAI-compatible base URL. |
|
|
| `DSH_LLM_MODEL` | `auto/best-chat` | Default model passed to OmniRoute. |
|
|
| `DSH_LLM_KEY` | `""` | Optional API key for OmniRoute. |
|
|
| `DSH_SYSTEM_PROMPT` | `"You are {USER_NAME}'s helpful DeepSeek assistant..."` | Custom system prompt per user instance. |
|
|
| `DSH_VISION_BASE` | `https://openrouter.ai/api/v1` | OpenRouter base URL for multimodal vision. |
|
|
| `DSH_VISION_MODEL` | `openai/gpt-5` | Multimodal model for image analysis. |
|
|
| `DSH_WHISPER_URL` | `http://192.168.20.13:5000/transcribe` | Local faster-whisper STT service. |
|
|
| `DSH_VOICE_AGENT_URL` | `http://192.168.20.13:8501/voice` | Jervis Voice Agent execution endpoint. |
|
|
| `DSH_HISTORY_DIR` | `/workspace` | Mount point for history and session storage. |
|