The Hub — a short brief

Purpose: tell every project where it fits and the rules it must follow. Written: 2026-10-10 Status: agreed direction, not a build order.

The idea in one line

Share the plumbing. Keep the brains separate. Put one small hub in the middle.

Every way in (phone, house voice, chat) reaches every ability (lights, internet, Artemis, Paperclip) through one place — built once, not repeated.

What already works (do not rebuild)

System	What it is
Jervis	The house voice. ESP32 "Hey Jervis" → STT → agent → TTS → speakers. Controls HA, timers, reminders, shopping, calendar, music, weather, web search, family messages.
dsh	Per-person chat (sam / jo / harry / finn). Memory, images, video, documents.
Console portal	Logins and dashboard for the family.
Front Door Cam	Its own separate ESP32. Visitor detection, Gemini sees the person, voice asks them to speak.
OmniRoute	The model gateway. All models go through here.
Home Assistant	Devices, timers, calendar, notifications.
Shared	Apprise, Snapcast, STT/TTS, n8n, secrets in `environment.d`.

These stay. The hub is added, not a replacement.

What is new

The Hub (this brief).
Email agent — its own pipeline.
School (Compass / Teams) — its own pipeline.
Internet use control — a destination the hub calls.
Artemis — a destination the hub calls.
Kotlin phone app — a new way in (wake word, voice, controls).
Admin app + kids app — parental control, admin only.

What the Hub is

One service that does five steps:

Hear — takes voice or text from any way in.
Understand — the LLM works out what was asked.
Check — is this person allowed to ask for this?
Route — sends it to the right destination.
Answer — returns a short reply in the right shape for the device.

The reply can be text, spoken audio (TTS), Artemis commands, a phone notification, an email, or a mix. TTS can play on the house speakers or on the phone.

What the Hub is NOT

Not a central brain. It does not hold one personality or one memory.
Not Paperclip. It never waits minutes for an answer.
Not a destination. It does not turn off the internet itself. It sends the request to the service that does.

The dividing line

One question sorts every request:

Does this need a decision, or does it need to happen?

Happen (fast, certain) → straight to the destination: HA, internet control, Artemis, speakers. No LLM in the path of the action.
Decision (slow, multi-step, worth recording) → Paperclip.

Paperclip is the work bench, not the switchboard. It is never in the path of a fast or high-consequence command.

Where each system fits

System	Role	With the Hub
Jervis	House voice	Becomes the voice (speak / announce), not a list of tools. Its action tools move into the Hub over time.
dsh	Chat	Calls the Hub for actions instead of keeping its own copies.
Console portal	Login + dashboard	Unchanged. May show Hub status and approvals.
Front Door Cam	Visitor intercom	Stays separate. May use the Hub only to speak.
Email agent	Own pipeline (n8n)	Standalone. May report results through the Hub.
School agent	Own pipeline (n8n)	Standalone. May report results through the Hub.
Internet control	Destination	The Hub calls its API. Enforcement happens on the kids' devices.
Artemis	Destination	The Hub calls its API. Artemis touches the phone screen.
Paperclip	Work bench	The Hub files jobs and hears the answer back.
Phone app	Way in	Sends voice to the Hub. Shows replies, notifications, approvals.

Rules every project follows

All models go through OmniRoute. Never call a provider directly.
All secrets come from environment.d. Never hard-code.
Own service, own port, own folder. Do not merge into another project.
No LLM in the path of a fast or consequential action. Deterministic calls for anything that turns something off.
Paperclip only for slow, multi-step, recorded work. Never make a person wait on it.
Replies return to the source, and stay short. Speaker → one line. Phone → notification. Chat → text. No long-winded answers.
Restricted commands need tap-to-approve on the admin app. This includes parental controls and any job filed to Paperclip.
Only the admin app can change parental controls. Kids' apps and desktop exes may only obey and report.
New work registers with the Hub. A capability is added once, and every way in gets it.

Safety

Voice is shared and unauthenticated. Anyone in the house can speak.
The admin app is the only trusted door for restricted actions.
A spoken keycode is not the lock. It can be overheard and misheard. Voice may ask; the admin app confirms with "Did you ask for this?".
Approval is required for parental controls and Paperclip jobs.

Build order

Thin slice, end to end. Issue a Paperclip API key, file one job from a script, and get the result announced back. Proves the round trip.
Hub skeleton. Hear → understand → route → answer, with one destination (HA or speakers) working.
Approval flow. "Did you ask for this?" on the admin app.
Internet control. Standalone service with a plain API. Device-side enforcement. Admin-only.
Artemis. Standalone service with a plain API. Screen control only.
Move Jervis's tools into the Hub. Jervis becomes voice-only.
Phone Kotlin app. Wake word, voice to the Hub, replies and approvals.
Email and school pipelines stay independent and connect when ready.

What is still open

Exact API contract for a job (title, brief, project, agent).
How Artemis receives voice (stream vs file) and when phone TTS is used.
Which Paperclip company / projects to use.
Identity: how the Hub knows which family member is speaking.

6.1 KiB Raw Blame History