Local AI Coding Harness Architecture

This note details the high-level architecture for building a private, cost-optimised, and highly customizable local alternative to Cursor IDE using Neovim, Pi.dev, Qdrant, and Fastembed.

System Architecture

The environment relies on a structured, three-tier local loop that tightly integrates your editor, your agent orchestrator, and your vector pipeline:

1. The Interaction Tier (Neovim & Terminal)

Editor Layer: Your primary workspace runs inside Neovim buffers, navigating the filesystem and editing source files.
Execution Layer: A terminal window hosts the active Pi.dev agent instance, responding to commands and monitoring your developer workspace.

2. The Agent Tier (Pi.dev Engine)

Context Request: When a debugging or coding task requires deep structural knowledge, the Pi agent runs a registered tool called search_codebase.
Context Return: The tool returns the top 3 most relevant structural code chunks directly into Pi's active context window, preventing token bloat.

3. The Retrieval Tier (Local Docker RAG)

Save Hook Handler: A Git hook or save utility uses Tree-sitter AST parsers to divide your source code into cleanly isolated function and class scopes.
Fastembed Vectorizer: The chunks are processed locally via CPU-optimised embedding models, converting plain text code blocks into multi-dimensional vectors.
Qdrant Storage Node: A lightweight Qdrant vector database running inside a background Docker container indexing payloads, ready for instant semantic retrieval.

Workflow Overview

Instead of constantly streaming entire code repositories to expensive proprietary cloud servers, this setup routes contextual sub-tasks through a local pipeline. You retain full control over context boundaries, model selections, and operational expenses.

1. Ingestion & Chunking (AST Scope-Splitting)

Trigger: A file-save event or a Git pre-commit hook triggers a local TypeScript indexing script.
Parsing: The script uses lightweight abstract syntax tree (AST) parsing via Tree-sitter to slice code by logical boundaries (classes, methods, functions) rather than arbitrary text character counts.
Context Wrapping: Each chunk is decorated with metadata including file path, line numbers, and its parent structural scope.

2. Private Local Embeddings

Execution: The chunked code blocks pass into @qdrant/fastembed entirely on your local machine.
Model: Uses a fast, CPU-optimized model (e.g., bge-small-en-v1.5) to transform code into vector representations.
Cost: Fully local execution results in $0.00 infrastructure or token costs for repository indexing.

3. Vector Storage via Qdrant

Infrastructure: A lightweight Qdrant vector database runs silently in a background Docker container.
Storage: Vector points are stored locally, paired with their code payloads and metadata, allowing for near-instant semantic lookup.

4. Pi.dev Agent Automation

The Bridge: A custom TypeScript tool (search_codebase) is registered in your Pi profile.
Agent Flow: When the Pi agent encounters an execution error or needs architectural context, it queries the local Qdrant collection.
Parallelization: Pi orchestrates low-cost sub-agents in parallel tabs to lint code, run tests, and check types, preserving expensive frontier models purely for complex engineering decisions.

Key Advantages Over Out-of-the-Box IDEs

Asymmetric Cost Efficiency: Saves hundreds of dollars monthly by using tiered model routing and semantic RAG filtering over global code dumps.
Deterministic Control: The agent's action loop is managed via local scripts and shell guardrails—no unexpected cloud system prompt updates.
Zero-Trust Security: Private codebases, environment variables, and proprietary logic never leave your machine during index cycles.

4.2 KiB Raw Blame History