Add overview, agent-api, clients docs for Jervis voice agent

2026-08-30 14:42:11 +10:00
commit 4b86c18622
5 changed files with 283 additions and 0 deletions
--- a/docs/overview.md
+++ b/docs/overview.md
@@ -0,0 +1,93 @@
+# Jervis Voice Assistant — System Overview
+
+> Single source of truth for the local voice-assistant system on Sam's home lab.
+> Companion docs: **`agent-api.md`** (how to send a command / get a reply) and **`clients.md`** (type-it, web, Android gateways).
+> Component repos: `voice_assistant` (ESP32 firmware), `voice_bridge` (STT bridge), `voice_agent` (this brain), `speech_piper` (TTS).
+
+---
+
+## What it does
+
+A fully-local, private voice assistant. You say **"Hey Jervis"**, then a natural-language request; it answers, controls the house, manages your shopping list, and speaks back through your speakers. The **design principle: one LLM "brain"** decides what to do, and calls tools. Home Assistant is *one of those tools* — not the decision maker.
+
+---
+
+## Architecture
+
+```
+┌─ Input ───────────────────────────────────────────────────────────────┐
+│  ESP32-S3  ("Hey Jervis" wake word, INMP441 mic, WS2812 LED ring)     │
+│    └→ MQTT voice/audio_stream  (broker .13:1883)                      │
+│  voice_bridge (docker .13) — wake word, buffer audio                  │
+│    └→ HTTP :5000                                                      │
+│  voice_whisper (docker .13, faster-whisper) — STT audio→text          │
+│    └→ publish text → MQTT homeassistant/voice/text  (broker .30)      │
+│  ALSO: any client → POST http://.13:8501/voice {text}  (see clients.md)│
+└───────────────────────────────────────────────────────────────────────┘
+                          │  text
+                          ▼
+┌─ Brain ── voice-agent (systemd service on .13:8501) ─────────────────┐
+│  LLM = OmniRoute (OpenAI-compatible, tool-calling)                   │
+│  Model: auto/best-fast                                               │
+│  LLM decides intent → calls a TOOL:                                  │
+│    • get_time / get_weather / answer          (Open-Meteo, LLM)      │
+│    • add_shopping_item    → KitchenOwl API   (shopping list)         │
+│    • control_ha / ha_service_cmd  → Home Assistant REST API          │
+│    • feed_fish            → HA input_button.fish_feeder              │
+│    • set_reminder         → HA persistent_notification               │
+│    • speak_as(voice,text) → piper per-character voice                │
+│    • play_music           → Mopidy (planned)                         │
+└───────────────────────────────────────────────────────────────────────┘
+                          │  reply text
+                          ▼
+┌─ Output ─────────────────────────────────────────────────────────────┐
+│  piper_tts (docker .13) → speak_direct.sh → Snapcast → all speakers  │
+└───────────────────────────────────────────────────────────────────────┘
+```
+
+**Key design decision:** the agent is the single brain; **Home Assistant is a passive device the agent commands via its REST API** — no per-action HA automation/subscription is needed for the agent's commands (one long-lived HA token enables all HA control).
+
+---
+
+## Components (`host/IP/port/repo`)
+
+| Component | Host/IP | Port | Repo / path | Role |
+|---|---|---|---|---|
+| ESP32-S3 | — | — | `sam/voice_assistant` | Wake word, mic, LED |
+| voice_bridge | `.13` | docker | `sam/voice_bridge` | Wake word + audio buffer |
+| voice_whisper | `.13` | `:5000` | `sam/voice_bridge` | faster-whisper STT |
+| **voice-agent** | `.13` | **`:8501`** | `sam/voice-assistant` | LLM brain + tools |
+| OmniRoute | `.13` | `:20129` (API) | — | LLM gateway (OpenAI-compatible) |
+| piper_tts | `.13` | `:10200` | `sam/speech_piper` | TTS (Wyoming) |
+| Snapcast | `.13` | `:1780` | — | Multi-room audio |
+| Mopidy / librespot | `.13` | `:6600`/`:6680` | — | Music sources |
+| Home Assistant | `.30` | `:8123` (REST) | — | Home control |
+| KitchenOwl | `.35` | `owl.lab.audasmedia.com.au` | — | Shopping list |
+| mosquitto | `.13` + `.30` | `:1883` | — | MQTT brokers |
+
+---
+
+## Data flow summary
+
+1. ESP32 streams audio to **voice_bridge** (MQTT on `.13`).
+2. **voice_bridge** detects the wake word, buffers, sends to **voice_whisper**.
+3. **voice_whisper** (faster-whisper) returns text.
+4. Text is published to MQTT `homeassistant/voice/text` (broker `.30`) **and/or** POSTed to the agent.
+5. **voice-agent** (LLM) decides intent and calls a tool.
+6. Reply text → **piper** → **Snapcast** → speakers.
+
+---
+
+## Run / manage
+
+- Agent service (persistent, auto-starts): `systemctl status voice-agent` (NixOS systemd unit declared in `/etc/nixos/configuration.nix`). Restart: `sudo systemctl restart voice-agent`.
+- Logs: `journalctl -u voice-agent` / `journalctl -u snapserver`.
+- Firewall: port `8501` is in `allowedTCPPorts` in the NixOS config.
+
+## Repository map (other related projects)
+
+- `sam/voice_assistant` — ESP32 firmware
+- `sam/voice_bridge` — STT bridge + whisper
+- `sam/speech_piper` — TTS scripts/models → Snapcast
+- `sam/voice-assistant` — **this** overview + API contract + clients
+- `sam/family_home_lab` — console portal / dsh