dsh: offer the local .27 Ollama models in the chat model dropdown
The dropdown is built from the hardcoded MODEL_CHOICES list in app.py, not from OmniRoute, so 'ollama/qwen2.5*' existed upstream but could never appear in DSH. - add ollama/qwen2.5:14b and ollama/qwen2.5-coder:14b to MODEL_CHOICES + MODEL_LABELS - labels note that Ollama must be started first, since it now boots off on .27 - docs/dsh-architecture.md: new section on model selection, incl. the rebuild requirement (app.py is baked into the image via COPY . .) and the ordering trap that a model can be live in OmniRoute yet invisible in DSH Verified: /api/models lists both ids on 3081-3084, and real streamed replies come back from both local models through DSH.
This commit is contained in:
@@ -77,7 +77,40 @@ DSH routes user requests across specialized AI services and local house infrastr
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 5. API Endpoints & Request Flow
|
## 5. Model Selection (the chat dropdown)
|
||||||
|
|
||||||
|
**The model list is hardcoded in `dsh/app.py`** (`MODEL_CHOICES`, ~line 660). DSH does
|
||||||
|
**not** discover models from OmniRoute. `GET /api/models` simply returns that list
|
||||||
|
plus `MODEL_LABELS` for the friendly names.
|
||||||
|
|
||||||
|
Consequence: a model can exist in OmniRoute and still be invisible in DSH until it
|
||||||
|
is added to `MODEL_CHOICES` **and** `MODEL_LABELS`.
|
||||||
|
|
||||||
|
A chosen model is validated server-side (`if active_model not in MODEL_CHOICES:
|
||||||
|
active_model = LLM_MODEL`), so an unlisted id silently falls back to the default.
|
||||||
|
|
||||||
|
### Adding a model
|
||||||
|
|
||||||
|
1. Confirm it works through OmniRoute first — `curl http://127.0.0.1:20129/v1/models`
|
||||||
|
and then a real `chat/completions` call with a short timeout.
|
||||||
|
2. Add the id to `MODEL_CHOICES` and a friendly string to `MODEL_LABELS` in
|
||||||
|
`dsh/app.py`.
|
||||||
|
3. Deploy: copy `app.py` to `.13:/home/sam/Docker/Containers/dsh/`, then
|
||||||
|
`docker compose build && docker compose up -d` in that directory (the image
|
||||||
|
bakes `app.py` via `COPY . .`, so a rebuild is required — restarting is not enough).
|
||||||
|
4. Verify: `curl -s http://127.0.0.1:3081/api/models` should list the new id.
|
||||||
|
|
||||||
|
### Local Ollama models
|
||||||
|
|
||||||
|
The `.27` models are exposed by OmniRoute as `ollama/qwen2.5:14b` and
|
||||||
|
`ollama/qwen2.5-coder:14b` (`ollama-local/…` also exists but is duplicated).
|
||||||
|
Because Ollama now starts **off** at boot, these fail until it is started from the
|
||||||
|
family console; hence the `needs Ollama on` label. Inference is CPU-only on `.27`,
|
||||||
|
so expect it to be slower than the cloud routes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. API Endpoints & Request Flow
|
||||||
|
|
||||||
### Primary API Routes (`dsh/app.py`)
|
### Primary API Routes (`dsh/app.py`)
|
||||||
|
|
||||||
@@ -96,7 +129,7 @@ DSH routes user requests across specialized AI services and local house infrastr
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 6. Embedding & Security Headers
|
## 8. Embedding & Security Headers
|
||||||
|
|
||||||
To support seamless inline framing inside the Family Home Lab Console (`/tool/dsh`):
|
To support seamless inline framing inside the Family Home Lab Console (`/tool/dsh`):
|
||||||
- **Content Security Policy (CSP)**: `frame-ancestors https://console.lab.audasmedia.com.au`
|
- **Content Security Policy (CSP)**: `frame-ancestors https://console.lab.audasmedia.com.au`
|
||||||
@@ -104,7 +137,7 @@ To support seamless inline framing inside the Family Home Lab Console (`/tool/ds
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 7. Environment Variables Reference
|
## 9. Environment Variables Reference
|
||||||
|
|
||||||
| Variable | Default Value | Purpose |
|
| Variable | Default Value | Purpose |
|
||||||
| :--- | :--- | :--- |
|
| :--- | :--- | :--- |
|
||||||
|
|||||||
@@ -665,6 +665,11 @@ MODEL_CHOICES = ["auto/best-chat",
|
|||||||
"auto/best-reasoning",
|
"auto/best-reasoning",
|
||||||
"auto/best-coding",
|
"auto/best-coding",
|
||||||
"auto/best-vision",
|
"auto/best-vision",
|
||||||
|
# Local models on .27, reached through OmniRoute. Free to run, but
|
||||||
|
# Ollama must be started first (family console -> AI Tools ->
|
||||||
|
# Ollama Local LLM). CPU inference, so slower than the cloud ones.
|
||||||
|
"ollama/qwen2.5:14b",
|
||||||
|
"ollama/qwen2.5-coder:14b",
|
||||||
"openrouter/deepseek/deepseek-v4-flash",
|
"openrouter/deepseek/deepseek-v4-flash",
|
||||||
"openrouter/deepseek/deepseek-v4-pro"]
|
"openrouter/deepseek/deepseek-v4-pro"]
|
||||||
MODEL_LABELS = {
|
MODEL_LABELS = {
|
||||||
@@ -676,6 +681,8 @@ MODEL_LABELS = {
|
|||||||
"auto/best-reasoning": "Reasoning · OmniRoute best",
|
"auto/best-reasoning": "Reasoning · OmniRoute best",
|
||||||
"auto/best-coding": "Coding · OmniRoute best",
|
"auto/best-coding": "Coding · OmniRoute best",
|
||||||
"auto/best-vision": "Multimodal · OmniRoute vision",
|
"auto/best-vision": "Multimodal · OmniRoute vision",
|
||||||
|
"ollama/qwen2.5:14b": "Local · Qwen2.5 14B (.27, needs Ollama on)",
|
||||||
|
"ollama/qwen2.5-coder:14b": "Local · Qwen2.5 Coder 14B (.27, needs Ollama on)",
|
||||||
"openrouter/deepseek/deepseek-v4-flash": "DeepSeek v4 flash (OpenRouter)",
|
"openrouter/deepseek/deepseek-v4-flash": "DeepSeek v4 flash (OpenRouter)",
|
||||||
"openrouter/deepseek/deepseek-v4-pro": "DeepSeek v4 pro (OpenRouter)",
|
"openrouter/deepseek/deepseek-v4-pro": "DeepSeek v4 pro (OpenRouter)",
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user