Initial spec: AGENTS.md build spec + archived original notes
This commit is contained in:
61
GOOGLE_NOTES.md
Normal file
61
GOOGLE_NOTES.md
Normal file
@@ -0,0 +1,61 @@
|
||||
|
||||
> **Editor's note (2026-10-10).** This note describes the Kotlin "Jeeves" phone
|
||||
> app. It moved here from `artemis_control`, because it belongs to the phone app,
|
||||
> not to the screen-control layer.
|
||||
>
|
||||
> **What still holds:**
|
||||
> - The phone app records your voice and sends it to the home server.
|
||||
> - The server decides what to do, and sends instructions back.
|
||||
> - Native phone actions (contacts, dialing, audio) run in the app.
|
||||
>
|
||||
> **What changed:**
|
||||
> - The "Home AI Server" is now **the Hub** (`central_ai_llm_master`).
|
||||
> - **Artemis is separate.** It controls the screen over **ADB + Tailscale** from
|
||||
> `.13`. It does **not** go through this app. See `../artemis_control/`.
|
||||
> - The Hub calls Artemis **directly**. The phone app is only a way **in**.
|
||||
|
||||
🌐 High-Level System Architecture Flow
|
||||
|
||||
[🗣️ Your Voice] ──> (📱 Jeeves App) ──> [🏠 Home AI Server (LLM)]
|
||||
│
|
||||
┌───────────────────────────────┴───────────────────────────────┐
|
||||
▼ [If Native Phone Feature] ▼ [If Screen Control]
|
||||
(📱 Jeeves App) (🤖 ARTEMIS CONTROL)
|
||||
│ │
|
||||
[⚡ Native Kotlin Execution] [🖱️ ADB input over Tailscale]
|
||||
(Contacts, Phone Calls, Mic Audio) (Spotify, Maps, other apps)
|
||||
|
||||
NOTE: The two right-hand branches are now SEPARATE projects.
|
||||
Jeeves = kotlin_phone_app_admin. Artemis = artemis_control.
|
||||
|
||||
🔄 Step-by-Step Execution Sequence
|
||||
|
||||
|
||||
1. Input & Intent Catching
|
||||
|
||||
• Action: You speak into your phone: "Find Sarah and call her."
|
||||
• Process: Your custom Jeeves Kotlin App converts the speech to text and instantly forwards it to your Home AI Server over your local network or Tailscale tunnel.
|
||||
|
||||
2. The LLM Brain Check
|
||||
|
||||
• Process: Your Home LLM interprets the request. It identifies the intent and figures out the best way to execute it:
|
||||
• Option A (Native System Tasks): If it requires basic phone features (Contacts, Dialing), it builds a Code Function Hook to send back to your app.
|
||||
• Option B (Third-Party Tasks): If it requires a closed app interface (Telegram, Spotify), it sends instructions to Google ARTEMIS.
|
||||
|
||||
🔀 The Split Execution Paths
|
||||
|
||||
|
||||
Path A: Handling Android Native Tasks (Contacts & Calling)
|
||||
|
||||
1. The Hook-Back: The Home LLM Server sends a quick JSON payload back to your running phone app:json
|
||||
{ "action": "QUERY_CONTACT", "target": "Sarah" }
|
||||
Use code with caution.
|
||||
2. Local Lookup: The Jeeves App intercepts this and uses Android's native database hooks to find Sarah's number (+1-555-0199) in milliseconds.
|
||||
3. The Call Setup: The app automatically triggers a phone call using native Android system commands without touching the screen.
|
||||
4. The Audio Loop: Your app records your microphone audio, passes it to your server for real-time processing, receives the generated MiniMax TTS audio file back, and plays it instantly over your phone's speakerphone.
|
||||
|
||||
Path B: Handling Third-Party Apps (Telegram, Spotify, etc.)
|
||||
|
||||
1. The Hand-Off: Your Home LLM Server realizes it cannot natively inject data into a closed application. It kicks the instruction over to Google ARTEMIS running on your Linux host.
|
||||
2. The Remote Bridge: ARTEMIS connects to your phone over the air via the encrypted Tailscale wire.
|
||||
3. Visual Interaction: ARTEMIS physically wakes up your phone screen, launches Telegram or Spotify, types out text strings, and clicks the layout buttons live in front of your eyes like a human operator.
|
||||
Reference in New Issue
Block a user