151 lines
6.1 KiB
Markdown
151 lines
6.1 KiB
Markdown
# The Hub — a short brief
|
|
|
|
**Purpose:** tell every project where it fits and the rules it must follow.
|
|
**Written:** 2026-10-10
|
|
**Status:** agreed direction, not a build order.
|
|
|
|
---
|
|
|
|
## The idea in one line
|
|
|
|
**Share the plumbing. Keep the brains separate. Put one small hub in the middle.**
|
|
|
|
Every way in (phone, house voice, chat) reaches every ability (lights, internet,
|
|
Artemis, Paperclip) through one place — built once, not repeated.
|
|
|
|
---
|
|
|
|
## What already works (do not rebuild)
|
|
|
|
| System | What it is |
|
|
|---|---|
|
|
| **Jervis** | The house voice. ESP32 "Hey Jervis" → STT → agent → TTS → speakers. Controls HA, timers, reminders, shopping, calendar, music, weather, web search, family messages. |
|
|
| **dsh** | Per-person chat (sam / jo / harry / finn). Memory, images, video, documents. |
|
|
| **Console portal** | Logins and dashboard for the family. |
|
|
| **Front Door Cam** | Its own separate ESP32. Visitor detection, Gemini sees the person, voice asks them to speak. |
|
|
| **OmniRoute** | The model gateway. All models go through here. |
|
|
| **Home Assistant** | Devices, timers, calendar, notifications. |
|
|
| **Shared** | Apprise, Snapcast, STT/TTS, n8n, secrets in `environment.d`. |
|
|
|
|
These stay. The hub is added, not a replacement.
|
|
|
|
---
|
|
|
|
## What is new
|
|
|
|
- The **Hub** (this brief).
|
|
- **Email agent** — its own pipeline.
|
|
- **School (Compass / Teams)** — its own pipeline.
|
|
- **Internet use control** — a destination the hub calls.
|
|
- **Artemis** — a destination the hub calls.
|
|
- **Kotlin phone app** — a new way in (wake word, voice, controls).
|
|
- **Admin app + kids app** — parental control, admin only.
|
|
|
|
---
|
|
|
|
## What the Hub is
|
|
|
|
One service that does five steps:
|
|
|
|
1. **Hear** — takes voice or text from any way in.
|
|
2. **Understand** — the LLM works out what was asked.
|
|
3. **Check** — is this person allowed to ask for this?
|
|
4. **Route** — sends it to the right destination.
|
|
5. **Answer** — returns a short reply in the right shape for the device.
|
|
|
|
The reply can be **text**, **spoken audio (TTS)**, **Artemis commands**, **a phone
|
|
notification**, **an email**, or a mix. TTS can play on the **house speakers** or
|
|
on the **phone**.
|
|
|
|
### What the Hub is NOT
|
|
|
|
- **Not a central brain.** It does not hold one personality or one memory.
|
|
- **Not Paperclip.** It never waits minutes for an answer.
|
|
- **Not a destination.** It does not turn off the internet itself. It sends the
|
|
request to the service that does.
|
|
|
|
### The dividing line
|
|
|
|
One question sorts every request:
|
|
|
|
> **Does this need a decision, or does it need to happen?**
|
|
|
|
- **Happen** (fast, certain) → straight to the destination: HA, internet control,
|
|
Artemis, speakers. No LLM in the path of the action.
|
|
- **Decision** (slow, multi-step, worth recording) → Paperclip.
|
|
|
|
**Paperclip is the work bench, not the switchboard.** It is never in the path of
|
|
a fast or high-consequence command.
|
|
|
|
---
|
|
|
|
## Where each system fits
|
|
|
|
| System | Role | With the Hub |
|
|
|---|---|---|
|
|
| **Jervis** | House voice | Becomes the **voice** (speak / announce), not a list of tools. Its action tools move into the Hub over time. |
|
|
| **dsh** | Chat | Calls the Hub for actions instead of keeping its own copies. |
|
|
| **Console portal** | Login + dashboard | Unchanged. May show Hub status and approvals. |
|
|
| **Front Door Cam** | Visitor intercom | Stays separate. May use the Hub only to speak. |
|
|
| **Email agent** | Own pipeline (n8n) | Standalone. May report results through the Hub. |
|
|
| **School agent** | Own pipeline (n8n) | Standalone. May report results through the Hub. |
|
|
| **Internet control** | Destination | The Hub calls its API. Enforcement happens on the kids' devices. |
|
|
| **Artemis** | Destination | The Hub calls its API. Artemis touches the phone screen. |
|
|
| **Paperclip** | Work bench | The Hub files jobs and hears the answer back. |
|
|
| **Phone app** | Way in | Sends voice to the Hub. Shows replies, notifications, approvals. |
|
|
|
|
---
|
|
|
|
## Rules every project follows
|
|
|
|
1. **All models go through OmniRoute.** Never call a provider directly.
|
|
2. **All secrets come from `environment.d`.** Never hard-code.
|
|
3. **Own service, own port, own folder.** Do not merge into another project.
|
|
4. **No LLM in the path of a fast or consequential action.** Deterministic calls
|
|
for anything that turns something off.
|
|
5. **Paperclip only for slow, multi-step, recorded work.** Never make a person
|
|
wait on it.
|
|
6. **Replies return to the source, and stay short.** Speaker → one line. Phone →
|
|
notification. Chat → text. No long-winded answers.
|
|
7. **Restricted commands need tap-to-approve** on the admin app. This includes
|
|
parental controls and any job filed to Paperclip.
|
|
8. **Only the admin app can change parental controls.** Kids' apps and desktop
|
|
exes may only obey and report.
|
|
9. **New work registers with the Hub.** A capability is added once, and every way
|
|
in gets it.
|
|
|
|
---
|
|
|
|
## Safety
|
|
|
|
- **Voice is shared and unauthenticated.** Anyone in the house can speak.
|
|
- **The admin app is the only trusted door** for restricted actions.
|
|
- **A spoken keycode is not the lock.** It can be overheard and misheard. Voice
|
|
may *ask*; the admin app **confirms** with "Did you ask for this?".
|
|
- **Approval is required** for parental controls and Paperclip jobs.
|
|
|
|
---
|
|
|
|
## Build order
|
|
|
|
1. **Thin slice, end to end.** Issue a Paperclip API key, file one job from a
|
|
script, and get the result announced back. Proves the round trip.
|
|
2. **Hub skeleton.** Hear → understand → route → answer, with one destination
|
|
(HA or speakers) working.
|
|
3. **Approval flow.** "Did you ask for this?" on the admin app.
|
|
4. **Internet control.** Standalone service with a plain API. Device-side
|
|
enforcement. Admin-only.
|
|
5. **Artemis.** Standalone service with a plain API. Screen control only.
|
|
6. **Move Jervis's tools** into the Hub. Jervis becomes voice-only.
|
|
7. **Phone Kotlin app.** Wake word, voice to the Hub, replies and approvals.
|
|
8. **Email and school pipelines** stay independent and connect when ready.
|
|
|
|
---
|
|
|
|
## What is still open
|
|
|
|
- Exact API contract for a job (title, brief, project, agent).
|
|
- How Artemis receives voice (stream vs file) and when phone TTS is used.
|
|
- Which Paperclip company / projects to use.
|
|
- Identity: how the Hub knows which family member is speaking.
|