62 lines
3.7 KiB
Markdown
62 lines
3.7 KiB
Markdown
|
|
> **Editor's note (2026-10-10).** This note describes the Kotlin "Jeeves" phone
|
|
> app. It moved here from `artemis_control`, because it belongs to the phone app,
|
|
> not to the screen-control layer.
|
|
>
|
|
> **What still holds:**
|
|
> - The phone app records your voice and sends it to the home server.
|
|
> - The server decides what to do, and sends instructions back.
|
|
> - Native phone actions (contacts, dialing, audio) run in the app.
|
|
>
|
|
> **What changed:**
|
|
> - The "Home AI Server" is now **the Hub** (`central_ai_llm_master`).
|
|
> - **Artemis is separate.** It controls the screen over **ADB + Tailscale** from
|
|
> `.13`. It does **not** go through this app. See `../artemis_control/`.
|
|
> - The Hub calls Artemis **directly**. The phone app is only a way **in**.
|
|
|
|
🌐 High-Level System Architecture Flow
|
|
|
|
[🗣️ Your Voice] ──> (📱 Jeeves App) ──> [🏠 Home AI Server (LLM)]
|
|
│
|
|
┌───────────────────────────────┴───────────────────────────────┐
|
|
▼ [If Native Phone Feature] ▼ [If Screen Control]
|
|
(📱 Jeeves App) (🤖 ARTEMIS CONTROL)
|
|
│ │
|
|
[⚡ Native Kotlin Execution] [🖱️ ADB input over Tailscale]
|
|
(Contacts, Phone Calls, Mic Audio) (Spotify, Maps, other apps)
|
|
|
|
NOTE: The two right-hand branches are now SEPARATE projects.
|
|
Jeeves = kotlin_phone_app_admin. Artemis = artemis_control.
|
|
|
|
🔄 Step-by-Step Execution Sequence
|
|
|
|
|
|
1. Input & Intent Catching
|
|
|
|
• Action: You speak into your phone: "Find Sarah and call her."
|
|
• Process: Your custom Jeeves Kotlin App converts the speech to text and instantly forwards it to your Home AI Server over your local network or Tailscale tunnel.
|
|
|
|
2. The LLM Brain Check
|
|
|
|
• Process: Your Home LLM interprets the request. It identifies the intent and figures out the best way to execute it:
|
|
• Option A (Native System Tasks): If it requires basic phone features (Contacts, Dialing), it builds a Code Function Hook to send back to your app.
|
|
• Option B (Third-Party Tasks): If it requires a closed app interface (Telegram, Spotify), it sends instructions to Google ARTEMIS.
|
|
|
|
🔀 The Split Execution Paths
|
|
|
|
|
|
Path A: Handling Android Native Tasks (Contacts & Calling)
|
|
|
|
1. The Hook-Back: The Home LLM Server sends a quick JSON payload back to your running phone app:json
|
|
{ "action": "QUERY_CONTACT", "target": "Sarah" }
|
|
Use code with caution.
|
|
2. Local Lookup: The Jeeves App intercepts this and uses Android's native database hooks to find Sarah's number (+1-555-0199) in milliseconds.
|
|
3. The Call Setup: The app automatically triggers a phone call using native Android system commands without touching the screen.
|
|
4. The Audio Loop: Your app records your microphone audio, passes it to your server for real-time processing, receives the generated MiniMax TTS audio file back, and plays it instantly over your phone's speakerphone.
|
|
|
|
Path B: Handling Third-Party Apps (Telegram, Spotify, etc.)
|
|
|
|
1. The Hand-Off: Your Home LLM Server realizes it cannot natively inject data into a closed application. It kicks the instruction over to Google ARTEMIS running on your Linux host.
|
|
2. The Remote Bridge: ARTEMIS connects to your phone over the air via the encrypted Tailscale wire.
|
|
3. Visual Interaction: ARTEMIS physically wakes up your phone screen, launches Telegram or Spotify, types out text strings, and clicks the layout buttons live in front of your eyes like a human operator.
|