obsidian-vault/300 areas/350 AI/Pi Agent Extensions & Skills.md

created, modified, type, tags, aliases

created	modified	type	tags	aliases
2026-05-16	2026-05-16	note	ai dev-ops

Pi Agent Extensions & Skills

Source Repositories

Source	Location
Gitea (package)	git:https://gitea.lab.audasmedia.com.au/sam/pi-config
Local filesystem	~/.agents/
Project settings	sys_config/.pi/settings.json, ai_setup/.pi/settings.json

---

Extensions

Extension	Source	Purpose
pi-config	~/.agents	/config-add, /config-remove, /config-show, /config-setup — manage which extensions/skills are active in a project
tavily-search	Gitea	tavily_search — web search via Tavily API (AI-optimized)
web-fetch	~/.agents	web_fetch — fetch any URL and return clean markdown (HTML, PDF, JS-rendered with Jina fallback)
ask-user-question	~/.agents	ask_user_question — LLM presents structured multiple-choice / text questions with keyboard UI
video-extract	~/.agents	video_extract — extract frames from YouTube/local video + full Gemini analysis (requires ffmpeg + yt-dlp + GEMINI_API_KEY)
filechanges	~/.agents	/filechanges, /filechanges-accept, /filechanges-decline — tracks every file LLM edits/writes, diff review, revert
pi-subagents (nicopreme)	npm (global)	Spawn child Pi agents with chains, parallel execution, async dispatch. /agents command. Integrates with pi-prompt-template-model for delegated prompt execution
pi-prompt-template-model	npm (global)	Model-switching prompt templates with frontmatter. See #Prompt Templates section below
pi-mcp-adapter	npm (global)	Single proxy tool (~200 tokens) replaces hundreds of MCP tool definitions. /mcp command for management. Lazy server connections
pi-graphify	~/.agents	Knowledge graph tools: build, query, path tracing, explain, watch, add, update
plannotator	~/.agents	Interactive plan review with browser UI, annotations, code review
caveman	~/.agents	Ultra-compressed communication mode
markitdown	~/.agents	Convert files (PDF, Word, Excel, PPTX, images, HTML, etc.) to Markdown. Image analysis via Qwen 2.5 VL 72B on OpenRouter.
smart-router	~/.agents	Dynamic prompt routing + thinking levels — analyzes intent, routes to optimal model, sets thinking level per task complexity. /lock-model and /unlock-model for manual override. Triggers Headroom compression for heavy analysis/code/devops contexts.

---

Skills

Skill	Purpose
nixos-workflow	STRICT workflow for managing Pi assets via Gitea on NixOS
system-architect	Multi-machine NixOS infrastructure (Snapcast, MQTT, Docker, Nvim)
obsidian-cli	Interact with Obsidian vault (notes, search, plugin dev, theme dev)
graphify	Full-pipeline knowledge graph orchestration
caveman	Caveman communication mode
openspec-propose	Propose new changes with design docs, specs, tasks
openspec-apply-change	Implement tasks from an OpenSpec change
openspec-archive-change	Archive completed changes
openspec-explore	Explore ideas and clarify requirements
npm-security	Scan packages with SafeDep Vet, check typosquatting with npq, wrap installs with Socket Firewall
markitdown	Convert files (PDF, Word, Excel, PowerPoint, images, HTML, CSV, JSON, XML, ZIP, EPubs, YouTube) to Markdown for LLM consumption. Image analysis via Qwen 2.5 VL 72B on OpenRouter.

---

markitdown

Convert various file formats to Markdown. Useful for feeding documents and images into LLMs.

What it converts

Format	Input	Notes
PDF	.pdf	Preserves structure (headings, lists, tables)
Word	.docx	mammoth + lxml
PowerPoint	.pptx	python-pptx
Excel	.xlsx, .xls	openpyxl + pandas
Images	.jpg, .png, etc.	EXIF metadata (free) + LLM vision description (via OpenRouter)
HTML	.html	beautifulsoup4
CSV / JSON / XML	.csv, .json, .xml	Structured data → Markdown tables
ZIP	.zip	Iterates contents, converts each file
EPubs	.epub
YouTube	URLs	Transcript extraction

CLI usage

# Convert file to Markdown (stdout)
markitdown document.pdf

# Write to file
markitdown document.pdf -o document.md

# Image with LLM vision description
markitdown-vision photo.jpg

Image analysis

Two levels:

1. EXIF metadata only (free, no API key): markitdown photo.jpg
2. LLM vision description (via OpenRouter, requires API key): markitdown-vision photo.jpg

The markitdown-vision wrapper auto-sources OPENROUTER_API_KEY from ~/.config/environment.d/10-secrets.conf and uses qwen/qwen2.5-vl-72b-instruct.

Missing / can be added

Feature	What's needed
Audio transcription	pip install markitdown[audio-transcription] (pydub + speechrecognition)
Azure AI Document Intelligence	pip install markitdown[az-doc-intel] + Azure credentials
Azure Content Understanding	pip install markitdown[az-content-understanding] + Azure credentials
markitdown-ocr plugin	Installed but needs OpenRouter key enabled to activate

---

Security Tools (npm Global)

Three tools installed globally at ~/.local/share/npm-global/bin/ to guard package installs.

SafeDep Vet (vet)

Scans local directories for multi-language malware signatures. Catches obfuscated code, suspicious imports, base64 payloads.

# Scan a cloned repo before touching it
vet scan -D . --format json --filter "package.malware == true"

# Scan package metadata from npm registry
vet scan package <name> --format json

Socket Firewall (socket)

Wraps npm/pip installs with real-time scanning. Blocks malicious packages at install time.

# Safe npm install
socket npm install <package>

# Safe pip install
socket pip install -r requirements.txt

npq

Checks package names against typosquatting lists before install. Lightweight, local, no phoning home.

npq check <package> --json

Workflow

1. vet scan → checks for malware in the code/package
2. npq check → checks the package name for typosquatting
3. socket install → wraps the actual install with runtime scanning

The npm-security skill instructs the Pi agent to follow this workflow before any install.

---

Smart Router

The smart-router extension (~/.agents/extensions/smart-router/) is a prompt interceptor that analyzes every incoming prompt and dynamically routes it to the most appropriate model based on intent. It also sets thinking levels and triggers Headroom context compression for heavy workloads.

How it works

1. Every prompt is intercepted before the agent loop starts
2. A fast model (openrouter/free) analyzes the intent
3. The prompt is classified into tags (read, discuss, search, devops-low, devops-high, code-analysis-low, code-analysis-high, codewrite-low, codewrite-high)
4. The router selects the optimal model + thinking level based on tag
5. For analysis/code/devops tags, large contexts (>5K tokens) are compressed via Headroom before reaching the LLM
6. Routing + compression status appears in the footer status bar (e.g. 🎯 devops-low → opencode-go/deepseek-v4-flash, 📦 73%)

Routing table

Tag	Route Key	Model	Thinking	Use case
read, discuss, search	free-core	openrouter/free	—	Reading docs, chat, web search
devops-low	economy-devops	opencode-go/deepseek-v4-flash	—	Simple YAML, Docker, bash
devops-high	precision-devops	opencode-go/deepseek-v4-flash	—	Complex multi-container, server crashes
code-analysis-low	free-core	openrouter/free	—	Finding bugs in short files
code-analysis-high	context-heavy	openrouter/free	—	Refactoring large codebases (262K context)
codewrite-low	economy-code	opencode-go/deepseek-v4-flash	—	Boilerplate, simple functions
codewrite-high (React)	precision-react	opencode-go/deepseek-v4-pro	high	Complex React/JS
codewrite-high (other)	precision-code-high	opencode-go/deepseek-v4-pro	high	Complex code, dense logic
Short prompts (<15 chars)	router-eval	openrouter/free	—	Quick responses

Thinking levels

Only set for tasks that genuinely benefit. Everything else keeps default (off):

Thinking Level	Tags	Effect
(off)	read, discuss, search, devops-low, devops-high, code-analysis-low, code-analysis-high, codewrite-low	Fast, no extended reasoning
high	codewrite-high, precision-react	Full reasoning for genuinely complex code — worth the latency and token cost

Manual override

# Lock to a specific model (disables routing)
/lock-model openrouter/anthropic/claude-3.5-sonnet

# Re-enable dynamic routing
/unlock-model

Note on model ID format: /lock-model takes provider/model-id. Some OpenRouter models include the provider prefix in their ID (e.g. openrouter/owl-alpha has id = "openrouter/owl-alpha"). The handler tries both id and provider/id to find the model.

What about /do-* commands?

The /do-* prompt templates are no longer needed for model selection. The smart-router handles all routing automatically. However, /do-img and /do-make-img are still available for image analysis/generation since those use specialized tool calls rather than just model selection.

Configuration

Model mappings are defined in ~/.agents/extensions/smart-router/index.ts. To change routing, thinking, or compression behavior, edit the MODELS, THINKING, or COMPRESS_TAGS tables and run /reload.

Files

File	Purpose
~/.agents/extensions/smart-router/index.ts	Extension source
~/.agents/extensions/smart-router/package.json	npm deps (headroom-ai)
~/.pi/agent/extensions/smart-router/index.ts	Synced copy

---

Headroom

Headroom is a context compression layer that reduces prompt token usage by 60-95% for heavy analysis/code/devops workloads. It runs as a Docker container on the server (192.168.20.13) and is selectively triggered by the smart-router.

How it works

1. Smart-router analyzes the prompt → determines if compression is needed
2. Tags read, discuss, search never trigger compression (these are fast paths)
3. For all other tags, if accumulated context exceeds ~5K tokens, the smart-router calls compress()
4. Messages are sent to the Headroom proxy at 192.168.20.13:8787
5. Headroom compresses the context (using SmartCrusher for JSON, CodeCompressor for AST, Kompress-base ML for text)
6. Compressed messages are returned and forwarded to the LLM
7. If the proxy is down, messages pass through unchanged (graceful fallback)

Architecture

Desktop (.27)                         Server (.13)
─────────────                         ────────────
smart-router analyzes prompt          headroom proxy (Docker)
    │                                     │
    │  if compress needed:                │
    │  compress(messages) ──────────────► │
    │  HTTP POST 192.168.20.13:8787       │
    │  ◄────────── compressed messages    │
    │                                     │
    │  send to LLM                        │
    │                                     │
    │  if proxy down: pass through        │

Compression thresholds

Condition	Action
Tag is read/discuss/search	Skip — no compression
Context < 5K tokens	Skip — too small to benefit
Context ≥ 5K tokens + analysis/code/devops tag	Compress
Proxy unreachable	Pass through unchanged

Management

# Check status
ssh 192.168.20.13 "docker ps --filter name=headroom"

# View logs
ssh 192.168.20.13 "docker logs --tail 20 headroom"

# Restart
ssh 192.168.20.13 "docker restart headroom"

# Update image
ssh 192.168.20.13 "cd /home/sam/Docker/Containers/headroom && docker compose pull && docker compose up -d"

Files

File	Purpose
~/.agents/extensions/smart-router/index.ts	Compression logic (compress() import + context event handler)
/home/sam/Docker/Containers/headroom/docker-compose.yml	Docker service definition (on .13)
/home/sam/Docker/Containers/headroom/.env	Environment file (on .13)

---

MCP Servers

pi-mcp-adapter connects Pi to external services via the Model Context Protocol.

Config file: ~/.config/mcp/mcp.json

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/home/sam"]
    }
  }
}

Find MCP servers at:

• github.com/modelcontextprotocol/servers
• smithery.ai — community registry

Usage:

• /mcp — interactive panel to manage servers
• mcp({ search: "..." }) — search available tools
• mcp({ tool: "tool_name", args: '{}' }) — call a tool
• Servers are lazy (connect on first use, disconnect after 10 min idle)

---

Configuration Files

Global (~/.pi/agent/settings.json)

• Nix store symlink — managed via /etc/nixos/home/sam/home.nix
• Contains: providers (opencode-go, openrouter, google), packages (pi-memctx, pi-prompt-template-model, Gitea)
• Read-only — cannot be modified by pi install or /config-add

Project (<project-dir>/.pi/settings.json)

• Overrides global settings (arrays replace, not merge)
• Contains: ~/.agents package (extensions + skills), Gitea package (tavily-search)
• Modified via /config-add / /config-remove commands

Per-folder Memory (via pi-memctx)

• Memory stored in <chat-folder>/.pi/memory-vault/packs/
• Workspace map at ~/.pi/agent/memory-vault/00-system/workspace-map.json
• Each chat folder has isolated memory (prevents sibling directory contamination)

---

Useful Commands

Command	What it does
/lock-model <id>	Lock routing to a specific model (e.g. openrouter/anthropic/claude-3.5-sonnet)
/unlock-model	Re-enable dynamic prompt routing
/config-setup	One-shot: creates .pi/, settings.json, memory vault in current folder
/config-add ext <name>	Activate an extension from ~/.agents
/config-add skill <name>	Activate a skill from ~/.agents
/config-show	Show active extensions and skills
/memctx-init	Scan folder, build initial memory pack
/memctx-status	Show memory status
/memctx-refresh	Re-scan and enrich memory
/filechanges	Review changed files, diffs, accept/decline
/filechanges-accept	Accept all changes
/filechanges-decline	Revert all changes
markitdown <file>	Convert file to Markdown (PDF, Word, Excel, PPTX, images, HTML, etc.)
markitdown-vision <file>	Describe image using Qwen 2.5 VL 72B via OpenRouter

---

Skipped / Bookmarked

Extension/Skill	Reason
web-search (amosblomqvist)	❌ Redundant — Tavily does this
subagents (amosblomqvist)	❌ Redundant — pi-subagents already installed
bash-guard (amosblomqvist)	❌ Too aggressive — would interrupt flow
google-image-search (amosblomqvist)	❌ Would need Google Search API + CSE setup
pdf-reader (amosblomqvist)	⏳ Bookmarked — Python + pymupdf setup needed
notify (mitsuhiko)	⏳ Minor QoL — desktop notifications on task complete
audio/voice	⏳ Not practical

---

Tasks

• Rebuild NixOS to activate new packages (Google provider, ffmpeg/yt-dlp, pi-prompt-template-model, pi-mcp-adapter, pi-subagents) ✅ 2026-06-11
• Add MCP servers to ~/.config/mcp/mcp.json or .mcp.json as needed (Home Assistant, databases, etc.)
• Run /reload in Pi to activate filechanges and new templates
• Add more prompt templates to ~/.pi/agent/prompts/ as needed
• Verify video-extract works with Gemini
• Add markitdown skill to Obsidian skills page
• Add smart-router extension and update Obsidian docs
• Deploy Headroom Docker compression on .13, integrate with smart-router
• Clean up workspace-map.json entries for any stale memory packs