sam-4screen-desktop 2026-10-14:16:6:28

.graphifyignore

@@ -3,6 +3,7 @@
 #  hard-skipped by graphify, but .obsidian/ is NOT — dot-dirs are allowed)
 
 .obsidian/
+.pi/
 graphify-out/
 graphify-out-temp/
 .trash/

.obsidian/workspace.json

@@ -212,19 +212,20 @@
   },
   "active": "26813e16e829b798",
   "lastOpenFiles": [
+    "010 inbox/Ai Resume RAG Architecture Ideas.md",
+    "200 projects/210 AI Resume/Ai Resume RAG Architecture Ideas.md",
+    "000 daily/Daily Notes and To Do.md",
     "010 inbox/Graphify.md",
     "010 inbox/Internet Use Control — Kids' Screen Time Limits.md",
     "010 inbox/Email Agent — Family Inbox Triage.md",
     "100 inbox/Local LLM Infrastructure & Qwen2.5 Deployment.md",
     "010 inbox/Email & DNS Setup — Domains, Mailboxes & Records.md",
     "300 areas/360 Dev-Ops Network Computers/Email & DNS Setup — Domains, Mailboxes & Records.md",
-    "000 daily/Daily Notes and To Do.md",
     "010 inbox/Central AI Hub.md",
     "010 inbox/Family Holiday Camping Wilsons Prom.md",
     "400 Personal Family/470 Holidays Travel/Family Holiday Camping Wilsons Prom.md",
     "010 inbox/MiniMax Voice — Cloud STT and TTS with Local Fallback.md",
     "200 projects/Tools Software WebUI/MiniMax Voice — Cloud STT and TTS with Local Fallback.md",
-    "010 inbox/Untitled.md",
     "300 areas/355 AI Research and Techniques/Ai memory management.md",
     "200 projects/Tools Software WebUI/Family Console.md",
     "200 projects/Tools Software WebUI/Family Home Lab.md",
@@ -239,7 +240,6 @@
     "010 inbox/Hindsight AI Resume Tool.md",
     "200 projects/210 AI Resume/Hindsight AI Resume Tool.md",
     "010 inbox/Alibaba Code Review AI Tool.md",
-    "300 areas/395 AI Tools to try/Alibaba Code Review AI Tool.md",
     "010 inbox/_WATCHER",
     "300 areas/355 AI Research and Techniques",
     "300 areas/355 Ai Research and Techniques",

010 inbox/_sort log.md

@@ -20069,3 +20069,28 @@ What the watcher did, newest last.
 ### 2026-10-14 13:23:07
 - **proposed** `Graphify.md` → `200 projects/Tools Software WebUI` (confidence 0.36)
 -   - Low JEV confidence — recommend human confirmation
+
+### 2026-10-14 15:59:02
+- **held** `Untitled.md` — BLOCKED: untitled notes are never moved automatically. Rename it first if it is real content.
+
+### 2026-10-14 15:59:12
+- **held** `Untitled.md` — BLOCKED: untitled notes are never moved automatically. Rename it first if it is real content.
+
+### 2026-10-14 15:59:22
+- **held** `Untitled.md` — BLOCKED: untitled notes are never moved automatically. Rename it first if it is real content.
+
+### 2026-10-14 15:59:32
+- **held** `Untitled.md` — BLOCKED: untitled notes are never moved automatically. Rename it first if it is real content.
+
+### 2026-10-14 15:59:42
+- **held** `Untitled.md` — BLOCKED: untitled notes are never moved automatically. Rename it first if it is real content.
+
+### 2026-10-14 15:59:52
+- **proposed** `Ai Resume RAG Architecture Ideas.md` → `200 projects/210 AI Resume` (confidence 0.61)
+
+### 2026-10-14 16:00:13
+- **re-proposing** `Ai Resume RAG Architecture Ideas.md` — you edited it
+- **proposed** `Ai Resume RAG Architecture Ideas.md` → `200 projects/210 AI Resume` (folder already set — kept) (confidence 0.91)
+
+### 2026-10-14 16:00:53
+- **filed** `Ai Resume RAG Architecture Ideas.md` → `200 projects/210 AI Resume/Ai Resume RAG Architecture Ideas.md` (type project)

100 Table of Contents/Projects.md

@@ -40,6 +40,7 @@ SORT title ASC
 - [[AI Resume - Content Organization Plan]]
 - [[Local Hybrid Vector + Graph RAG Setup]]
 - [[Hindsight AI Resume Tool]]
+- [[Ai Resume RAG Architecture Ideas]]
 
 ### Web Host Migration
 - [[Migration Plan — InMotion to Local + AWS Offsite]]

200 projects/210 AI Resume/Ai Resume RAG Architecture Ideas.md

@@ -0,0 +1,140 @@
+---
+title: Ai Resume RAG Architecture Ideas
+type: project
+status: active
+created: "2026-10-14 15:58"
+updated: "2026-09-26"
+tags:
+  - architecture
+  - pipeline
+  - ideas
+modified: "2026-10-14 15:58"
+---
+
+# [[Ai Resume RAG Architecture Ideas]]
+
+# Enterprise Homelab AI Knowledge Engine: Architecture & Workflows
+
+This document outlines the blueprint for a professional-grade, resource-efficient, local-to-cloud hybrid RAG (Retrieval-Augmented Generation) infrastructure [3.2]. Built on a local Docker network, it orchestrates data pipelines from self-hosted tools (Gitea, Outline, Obsidian, Wiki.js, IoT systems) and routes semantic context to low-cost cloud LLMs.
+
+---
+
+## 1. Architectural Overview & Component Responsibilities
+
+The system decouples data extraction, stateful scheduling, data persistence, and processing layers [3.2]. This guarantees low memory overhead on the host machine while matching industry-standard data design patterns.
+
+*   **Extraction Layer (n8n & Python Scrapers):** Captures change events and extracts raw content via API parsing, web scraping, or webhook listeners.
+*   **Orchestration Layer (Prefect):** Manages data pipelines, handling fault tolerance, rate limiting, document chunking, and batch schema execution.
+*   **Semantic Router Layer (LangGraph / LangChain):** Coordinates state machines to receive user queries, perform pre-filtered database retrieval, and manage cloud LLM context boundaries.
+*   **Storage Layer (Qdrant Vector Database):** Persists mathematical representations (embeddings) of text blocks along with explicit schema definitions.
+*   **Inference Layer (Cloud APIs via OpenRouter, DeepSeek, Gemini):** Executes logic, token generation, and codebase debugging tasks statelessly over HTTPS.
+
+---
+
+## 2. Component Workflow Implementations
+
+### A. Data Ingestion & Extraction (n8n & Python)
+Instead of executing heavy brute-force loops, data extraction runs in two structural forms:
+1.  **Event-Driven (n8n):** Listens to live inbound HTTP payloads (e.g., Gitea webhooks on code push events, Outline page revisions). n8n captures the operational change event payload and acts as a trigger to offload heavy lifting to the ETL pipeline.
+2.  **Scheduled Batch Jobs (Python Scrapers):** Custom headless scripts built with modern parsers (such as `Crawl4AI` or `BeautifulSoup4` coupled with `html2text`) to sweep legacy local web portals, hobby forums, or IoT data repositories. HTML is stripped of CSS stylesheets, tracking components, and sidebars, condensing raw input into text files.
+
+### B. Enterprise ETL Orchestration (Prefect)
+Prefect takes raw string text payloads from the extraction layer and runs them through deterministic python validation tasks:
+*   **Retry & Error Budgets:** Implements graceful back-offs when hitting remote token pools or local database limits.
+*   **Token-Aware Chunking:** Uses text splitters (like `RecursiveCharacterTextSplitter`) to split long files into target dimensions (e.g., 500-1000 characters) with an intentional safety overlap (e.g., 10%–15%) to avoid cutting sentences or code definitions in half.
+*   **Batch Pipelining:** Aggregates multiple document updates into single payloads to lower total HTTP requests across network connections.
+
+### C. The Metadata Layer & Vector Storage (Qdrant)
+To guarantee quick queries across massive local document footprints, data must be structurally categorized before it is saved into Qdrant. Every text block pushed to the database is accompanied by a structured metadata schema.
+
+#### The Metadata Schema Format:
+```json
+{
+  "source": "gitea | outline | obsidian | iot | hobby-web",
+  "project_name": "home-automation",
+  "file_path_or_url": "https://gitea.local",
+  "last_updated": "2026-10-14T15:55:00Z",
+  "content_type": "source_code | documentation | log_summary"
+}
+```
+
+#### Why Qdrant is Selected:
+Qdrant acts as the system's indexing anchor. Written in Rust, it requires trivial idle resources (~200MB RAM), supports native Docker setups, and exposes comprehensive payload filtering capabilities. This allows the system to combine text math (vectors) with standard database query filters.
+
+### D. The Vector Embedding Lifecycle
+Raw text chunks cannot be parsed directly by Qdrant. They must go through a vector tokenization loop:
+1.  **Text Transform:** Prefect submits a block of text to an external embedding endpoint (e.g., Gemini's free tier embedding endpoint or OpenRouter's text-embedding variants).
+2.  **Numerical Array Return:** The endpoint converts the text into a fixed-length numerical array (e.g., 1536 floating-point numbers reflecting semantic context).
+3.  **Qdrant Upsert:** Prefect packages the numerical vector array alongside the source text string and its corresponding metadata object, writing it to a specific storage collection block inside Qdrant.
+
+### E. Semantic Processing & Retrieval State Machines (LangGraph)
+When a user asks a question, LangGraph acts as the cognitive director, controlling how data travels between the vector store and the cloud LLM.
+
+1.  **Query Extraction Node:** Evaluates the user's intent. If a user asks: *"Fix the bug in my home-automation repository,"* LangGraph flags that the request is specifically tied to code infrastructure.
+2.  **Metadata-Filtered Retrieval Node:** Instead of executing a blind vector scan against millions of lines of text (which risks pulling in unrelated data from Obsidian notes or IoT metrics), LangGraph passes a specific metadata condition directly to Qdrant:
+    *   *Condition:* Find nearest vector coordinates *WHERE* `source == "gitea"` *AND* `project_name == "home-automation"`.
+3.  **Context Construction Node:** Qdrant returns only the most relevant, targeted code snippets within milliseconds. LangGraph packages these exact blocks into a unified system context window payload.
+4.  **Cloud LLM Call Node:** The finalized packet is sent statelessly over a secure outbound HTTPS connection to low-cost APIs (DeepSeek-V3/R1, OpenRouter, or Gemini). The LLM processes the contextualized prompt and fires back the perfect answer instantly.
+
+---
+
+## 3. Production-Ready Deployment: Docker Compose Topology
+
+This composition maps your local storage environment seamlessly into your running n8n, Prefect, and LangGraph network spaces.
+
+```yaml
+version: '3.8'
+
+services:
+  # The central vector memory engine
+  qdrant:
+    image: qdrant/qdrant:latest
+    container_name: local-ai-qdrant
+    ports:
+      - "6333:6333" # Main REST/HTTP API endpoint
+      - "6334:6334" # High-performance gRPC endpoint
+    volumes:
+      - qdrant_data_storage:/qdrant/storage
+    environment:
+      - QDRANT__SERVICE__ANALYTICS_TRACKING=false
+    restart: unless-stopped
+    networks:
+      - homelab-ai-mesh
+
+  # Your custom LangGraph running node/worker environment
+  langgraph-agent-worker:
+    build:
+      context: ./infra-agent-code
+      dockerfile: Dockerfile
+    container_name: langgraph-agent-worker
+    depends_on:
+      - qdrant
+    environment:
+      - QDRANT_HOST=http://qdrant:6333
+      - OPENROUTER_API_KEY=your_key_here
+      - DEEPSEEK_API_KEY=your_key_here
+      - GEMINI_API_KEY=your_key_here
+    volumes:
+      - /mnt/shared_local_data/obsidian_vault:/data/obsidian:ro # Read-only mount for safety
+    restart: unless-stopped
+    networks:
+      - homelab-ai-mesh
+
+volumes:
+  qdrant_data_storage:
+
+networks:
+  homelab-ai-mesh:
+    external: true # Bridges directly with your running n8n and Prefect network interfaces
+```
+
+---
+
+## 4. Architectural Summary for Professional Interviews
+
+When presenting this architecture to industry engineering leads, highlight these technical choices to demonstrate production maturity:
+
+1.  **Strict Resource Decoupling:** *"Instead of spinning up heavy local GPU servers or bloated monolithic RAG frameworks, I isolated the platform layers [3.2]. Compute costs are completely externalized to commodity cloud APIs via OpenRouter/DeepSeek, while data hygiene, scheduling, and storage remain entirely local inside Docker [3.2]."*
+2.  **Hybrid Metadata Optimization:** *"To ensure sub-second response speeds and completely prevent model hallucinations across distinct applications, I built custom Python ETL workflows in Prefect that tag all ingested data from Gitea, Outline, and local sites with strict metadata schemas. This allows my LangGraph pipelines to execute pre-filtered vector queries before the context hits the LLM."*
+3.  **Environment Agnostic Design:** *"The entire infrastructure interacts via standard OpenAI-compatible API schemas and container definitions. The codebase remains fully portable, meaning the system can scale from a single local workstation node to an enterprise Kubernetes tier simply by updating network configuration keys."*
+