sam/obsidian-vault
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

sam-4screen-desktop 2026-9-6:8:24:8

Browse Source
This commit is contained in:
Sam Rolfe
2026-09-06 08:24:08 +10:00
parent ece0c7a603
commit e1bb4a558f
2 changed files with 58 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
.obsidian/workspace.json vendored
View File

@@ -13,12 +13,12 @@
"state": { "state": {
"type": "markdown", "type": "markdown",
"state": { "state": {
"file": "000 daily/basketball training.md", "file": "300 areas/360 Dev-Ops Network Computers/Home Voice Assistant System.md",
"mode": "source", "mode": "source",
"source": false "source": false
}, },
"icon": "lucide-file", "icon": "lucide-file",
"title": "basketball training" "title": "Home Voice Assistant System"
} }
} }
] ]
@@ -55,7 +55,7 @@
"state": { "state": {
"type": "search", "type": "search",
"state": { "state": {
"query": "basketb", "query": "Jervi",
"matchingCase": false, "matchingCase": false,
"explainSearch": false, "explainSearch": false,
"collapseAll": false, "collapseAll": false,
@@ -76,8 +76,7 @@
"title": "Bookmarks" "title": "Bookmarks"
} }
} }
], ]
"currentTab": 1
} }
], ],
"direction": "horizontal", "direction": "horizontal",
@@ -184,7 +183,7 @@
"state": { "state": {
"type": "file-properties", "type": "file-properties",
"state": { "state": {
"file": "000 daily/basketball training.md" "file": "300 areas/360 Dev-Ops Network Computers/Home Voice Assistant System.md"
}, },
"icon": "lucide-info", "icon": "lucide-info",
"title": "File properties" "title": "File properties"
@@ -210,8 +209,9 @@
"templater-obsidian:Templater": false "templater-obsidian:Templater": false
} }
}, },
"active": "8018aa52f0591df9", "active": "7066c9273c6fb936",
"lastOpenFiles": [ "lastOpenFiles": [
"000 daily/basketball training.md",
"200 projects/220 Web Host Migration/000 — Migration Overview.md", "200 projects/220 Web Host Migration/000 — Migration Overview.md",
"200 projects/220 Web Host Migration/Garage Web UI — handoff.md", "200 projects/220 Web Host Migration/Garage Web UI — handoff.md",
"200 projects/220 Web Host Migration/Project Ops — Build Plan (Vikunja+Outline+Obsidian).md", "200 projects/220 Web Host Migration/Project Ops — Build Plan (Vikunja+Outline+Obsidian).md",
@@ -237,7 +237,6 @@
"000 daily/FAMILY S3 Storage Integration & Blueprint.md", "000 daily/FAMILY S3 Storage Integration & Blueprint.md",
"000 daily/FAMILY DeepSeek Harness (dsh) Home Lab Setup.md", "000 daily/FAMILY DeepSeek Harness (dsh) Home Lab Setup.md",
"000 daily/FAMILY Enterprise Homelab AI Multimedia Suite. Systems Architecture & Blueprint.md", "000 daily/FAMILY Enterprise Homelab AI Multimedia Suite. Systems Architecture & Blueprint.md",
"200 projects/220 Web Host Migration/Backup Architecture — Offsite to AWS S3.md",
"000 daily/ThinkPad Recommendations", "000 daily/ThinkPad Recommendations",
"500 archive/510 Daily", "500 archive/510 Daily",
"300 areas/305 Ideas Businesses", "300 areas/305 Ideas Businesses",

72
300 areas/360 Dev-Ops Network Computers/Home Voice Assistant System.md
View File

@@ -1,11 +1,12 @@
--- ---
created: 2026-08-30 created: 2026-08-30
modified: 2026-08-30 modified: 2026-09-05
type: note type: note
tags: tags:
- voice - voice
- system - system
- ai - ai
- project
aliases: aliases:
- Jervis - Jervis
- Voice Assistant - Voice Assistant
@@ -13,34 +14,61 @@ aliases:
# Home Voice Assistant System (Jervis) # Home Voice Assistant System (Jervis)
> **Source of truth:** Gitea repo `sam/voice-assistant` → `https://gitea.lab.audasmedia.com.au/sam/voice-assistant` > **Human-readable overview of the local voice-assistant project.**
> Docs: `docs/overview.md`, `docs/agent-api.md`, `docs/clients.md` (in that repo). > - **AI/project tracking:** Vikunja → *Jervis Voice Assistant (Whisper ESP HA Flow)* (project id 5)
> - **AI documentation (for bots):** Outline → *Voice Assistant (Whisper ESP HA)* collection → [Project Overview](/doc/project-overview-genev9IHFT)
> - **API/docs (Gitea):** `sam/voice-assistant` → overview / agent-api / clients
> - **Tasks/progress:** see the Vikunja project; docs live in Outline; this note is the human map.
## TL;DR ## What it is (TL;DR)
Local, private voice assistant. "Hey Jervis" → natural language → one LLM brain (voice-agent on `.13:8501`) A private, local voice assistant. Say **"Hey Jervis"** (ESP32 wake word) then a natural-language request — a local **LLM agent** decides intent and calls tools: it controls Home Assistant, manages shopping/tasks/calendar, plays music, answers questions, and speaks back through the house speakers (piper → Snapcast).
decides intent → calls tools (time/weather/shopping/HA control/reminders/character voices) → speaks via piper → Snapcast.
**Command it from anywhere:** `POST http://192.168.20.13:8501/voice {"text":"add milk to the shopping list"}` (reply is spoken). **Command it from anywhere:** `POST http://192.168.20.13:8501/voice {"text": "add milk to the shopping list"}` — the reply is spoken.
## Architecture
```
ESP32 (wake word, mic) → MQTT voice/audio_stream
→ voice_bridge + whisper (STT → text)
→ Jervis agent (FastAPI :8501, LLM tool-calling)
├─ Home Assistant (lights, timers, calendar, HA todo lists)
├─ OmniRoute LLM (voice-fast: OpenRouter → opencode-go → deepseek flash)
├─ Mopidy → Spotify → Snapcast (music)
├─ Apprise (family WhatsApp/Telegram/ntfy/email)
├─ Vikunja (project tasks)
└─ reply → piper → Snapcast speakers
```
## Components (host : port) ## Components (host : port)
| Component | Host | Port | Role | | Component | Host | Port | Role |
|---|---|---|---| |---|---|---|---|
| ESP32-S3 (wake word) | — | — | mic, LED | | ESP32-S3 (voice_assistant.ino) | devices | — | wake word "Hey Jervis", mic, LED |
| voice_bridge / voice_whisper | .13 | docker/:5000 | STT | | voice_bridge / voice_whisper | .13 | docker / :5000 | speech → text (faster-whisper) |
| **voice-agent (LLM brain)** | .13 | **:8501** | tool-calling | | **Jervis agent** | .13 | **:8501** | LLM tool-calling brain (systemd) |
| OmniRoute | .13 | :20129 | LLM gateway | | OmniRoute | .13 | :20129 | LLM gateway (voice-fast route) |
| piper_tts | .13 | :10200 | TTS | | Home Assistant | .30 | :8123 | control, timers, calendar, todo lists |
| Snapcast | .13 | :1780 | audio | | Mopidy + Spotify | .13 | :6600 / librespot | music |
| Home Assistant | .30 | :8123 | control | | piper_tts | .13 | :10200 | TTS → Snapcast |
| KitchenOwl | .35 | owl.lab | shopping | | Snapcast | .13 | :1780 | multi-room audio |
| mosquitto | .13/.30 | :1883 | MQTT | | Apprise | .35 | :8210 | notifications |
| Vikunja | .35 | vikunja.lab | project tasks/stages (tracker) |
| Outline | .13 | :3000 | project documentation (AI) |
See the repo `docs/overview.md` for the full diagram + `docs/agent-api.md` for the API contract + `docs/clients.md` for type/web/Android. ## Status (2026-09-05)
- **Voice working end-to-end**: ESP32 wake word → whisper → agent → piper → Snapcast (fixed docker piper, HA→.13 SSH auth, resampler, MQTT client-id).
- **HA integration**: lights/scenes via Assist; shopping lists via HA todo (`add_shopping_item`); timers + spoken reminders (HA `timer` → automation → piper); calendar alerts spoken 30 min before (HA automation); add-calendar tool.
- **Music**: play/pause/skip/volume via Mopidy → Spotify → Snapcast (credentials.json device-auth).
- **Q&A + web search**: Tavily-backed `web_search`; fast LLM route `voice-fast` (~2–3s).
- **Family messaging**: Apprise (sam/finn/harry/joanna; WhatsApp/email/Telegram/ntfy).
- **AI tooling**: `project-ops` skill gives every pi agent Vikunja + Outline access (keys in `10-secrets.conf`).
## Status (2026-08-30) ## Tools & systems (employment/skills)
- Voice-over-Snapcast restored (piper docker + HA→.13 SSH auth fixed); full-length speech (pure-python resampler). FastAPI · Python · Docker · MQTT · Home Assistant (REST/Assist/automations) · Whisper · piper TTS · Snapcast · Mopidy / librespot / Spotify · OmniRoute LLM routing · Tavily web search · Apprise · Vikunja & Outline APIs · NixOS (systemd services, flakes) · git-filter-repo (secret hygiene) · ESP32 (Arduino/ESP-IDF).
- Agent persistent as NixOS systemd service; tools verified: time, weather, shopping (KitchenOwl), HA control, reminders, feed-fish, character voices (Trump etc.).
- **Music working**: `play_music` → Mopidy → Spotify → Snapcast (gst-plugin-spotify + device-auth credentials.json; Radiohead/Pink Floyd verified). Fast LLM route `voice-fast` (OpenRouter→opencode-go→deepseek flash) cuts response latency to ~2-3s. ## Key configs & links
- Next: pause/stop music tools, Apprise messaging (sam/jo/harry/finn), calendar (N8N), extended TTS voices. See [[secrets_fixes]] for credential issues. - **Vikunja project**: `https://vikunja.lab.audasmedia.com.au` → *Jervis Voice Assistant (Whisper ESP HA Flow)*
- **Outline collection**: `https://outline.lab.audasmedia.com.au` → *Voice Assistant (Whisper ESP HA)*
- **Gitea**: `sam/voice-assistant` (overview / agent-api / clients docs)
- **Repo layout / plan**: `~/chats/homeassistant` (plan.md, AGENT.md); agent at `/home/sam/voice-agent/`
- **Secrets**: `~/.config/environment.d/10-secrets.conf` on .27/.13/.51 (VIKUNJA_TOKEN, OL_API_KEY, etc.) — see also the `secrets_fixes` note.
See also: [[Docker Containers]], [[Home Network Map Overview]]. See also: [[Docker Containers]], [[Home Network Map Overview]].