obsidian-vault/300 areas/350 AI/Pi Agent Extensions & Skills.md

created, modified, type, tags, aliases

created	modified	type	tags	aliases
2026-05-16	2026-05-16	note	ai dev-ops

Pi Agent Extensions & Skills

Source Repositories

Source	Location
Gitea (package)	git:https://gitea.lab.audasmedia.com.au/sam/pi-config
Local filesystem	~/.agents/
Project settings	sys_config/.pi/settings.json, ai_setup/.pi/settings.json

---

Extensions

Extension	Source	Purpose
pi-config	~/.agents	/config-add, /config-remove, /config-show, /config-setup — manage which extensions/skills are active in a project
tavily-search	Gitea	tavily_search — web search via Tavily API (AI-optimized)
web-fetch	~/.agents	web_fetch — fetch any URL and return clean markdown (HTML, PDF, JS-rendered with Jina fallback)
ask-user-question	~/.agents	ask_user_question — LLM presents structured multiple-choice / text questions with keyboard UI
video-extract	~/.agents	video_extract — extract frames from YouTube/local video + full Gemini analysis (requires ffmpeg + yt-dlp + GEMINI_API_KEY)
filechanges	~/.agents	/filechanges, /filechanges-accept, /filechanges-decline — tracks every file LLM edits/writes, diff review, revert
pi-subagents (nicopreme)	npm (global)	Spawn child Pi agents with chains, parallel execution, async dispatch. /agents command. Integrates with pi-prompt-template-model for delegated prompt execution
pi-prompt-template-model	npm (global)	Model-switching prompt templates with frontmatter. See #Prompt Templates section below
pi-mcp-adapter	npm (global)	Single proxy tool (~200 tokens) replaces hundreds of MCP tool definitions. /mcp command for management. Lazy server connections
pi-graphify	~/.agents	Knowledge graph tools: build, query, path tracing, explain, watch, add, update
plannotator	~/.agents	Interactive plan review with browser UI, annotations, code review
caveman	~/.agents	Ultra-compressed communication mode
markitdown	~/.agents	Convert files (PDF, Word, Excel, PPTX, images, HTML, etc.) to Markdown. Image analysis via Qwen 2.5 VL 72B on OpenRouter.
smart-router	~/.agents	Dynamic prompt routing — analyzes intent and routes to optimal model. /lock-model and /unlock-model for manual override.

---

Skills

Skill	Purpose
nixos-workflow	STRICT workflow for managing Pi assets via Gitea on NixOS
system-architect	Multi-machine NixOS infrastructure (Snapcast, MQTT, Docker, Nvim)
obsidian-cli	Interact with Obsidian vault (notes, search, plugin dev, theme dev)
graphify	Full-pipeline knowledge graph orchestration
caveman	Caveman communication mode
openspec-propose	Propose new changes with design docs, specs, tasks
openspec-apply-change	Implement tasks from an OpenSpec change
openspec-archive-change	Archive completed changes
openspec-explore	Explore ideas and clarify requirements
npm-security	Scan packages with SafeDep Vet, check typosquatting with npq, wrap installs with Socket Firewall
markitdown	Convert files (PDF, Word, Excel, PowerPoint, images, HTML, CSV, JSON, XML, ZIP, EPubs, YouTube) to Markdown for LLM consumption. Image analysis via Qwen 2.5 VL 72B on OpenRouter.

---

markitdown

Convert various file formats to Markdown. Useful for feeding documents and images into LLMs.

What it converts

Format	Input	Notes
PDF	.pdf	Preserves structure (headings, lists, tables)
Word	.docx	mammoth + lxml
PowerPoint	.pptx	python-pptx
Excel	.xlsx, .xls	openpyxl + pandas
Images	.jpg, .png, etc.	EXIF metadata (free) + LLM vision description (via OpenRouter)
HTML	.html	beautifulsoup4
CSV / JSON / XML	.csv, .json, .xml	Structured data → Markdown tables
ZIP	.zip	Iterates contents, converts each file
EPubs	.epub
YouTube	URLs	Transcript extraction

CLI usage

# Convert file to Markdown (stdout)
markitdown document.pdf

# Write to file
markitdown document.pdf -o document.md

# Image with LLM vision description
markitdown-vision photo.jpg

Image analysis

Two levels:

1. EXIF metadata only (free, no API key): markitdown photo.jpg
2. LLM vision description (via OpenRouter, requires API key): markitdown-vision photo.jpg

The markitdown-vision wrapper auto-sources OPENROUTER_API_KEY from ~/.config/environment.d/10-secrets.conf and uses qwen/qwen2.5-vl-72b-instruct.

Missing / can be added

Feature	What's needed
Audio transcription	pip install markitdown[audio-transcription] (pydub + speechrecognition)
Azure AI Document Intelligence	pip install markitdown[az-doc-intel] + Azure credentials
Azure Content Understanding	pip install markitdown[az-content-understanding] + Azure credentials
markitdown-ocr plugin	Installed but needs OpenRouter key enabled to activate

---

Security Tools (npm Global)

Three tools installed globally at ~/.local/share/npm-global/bin/ to guard package installs.

SafeDep Vet (vet)

Scans local directories for multi-language malware signatures. Catches obfuscated code, suspicious imports, base64 payloads.

# Scan a cloned repo before touching it
vet scan -D . --format json --filter "package.malware == true"

# Scan package metadata from npm registry
vet scan package <name> --format json

Socket Firewall (socket)

Wraps npm/pip installs with real-time scanning. Blocks malicious packages at install time.

# Safe npm install
socket npm install <package>

# Safe pip install
socket pip install -r requirements.txt

npq

Checks package names against typosquatting lists before install. Lightweight, local, no phoning home.

npq check <package> --json

Workflow

1. vet scan → checks for malware in the code/package
2. npq check → checks the package name for typosquatting
3. socket install → wraps the actual install with runtime scanning

The npm-security skill instructs the Pi agent to follow this workflow before any install.

---

Smart Router

The smart-router extension (~/.agents/extensions/smart-router/) is a prompt interceptor that analyzes every incoming prompt and dynamically routes it to the most appropriate model based on intent. It replaces the old /do-* prompt templates with automatic, invisible routing.

How it works

1. Every prompt is intercepted before the agent loop starts
2. A free model (openrouter/owl-alpha) analyzes the intent
3. The prompt is classified into one of 10 tags (read, discuss, search, devops-low, devops-high, code-analysis-low, code-analysis-high, codewrite-low, codewrite-high)
4. The router selects the optimal model based on tag + language
5. The selected model is set via pi.setModel()
6. Routing decisions appear in the footer status bar (e.g. 🎯 devops-low → qwen/qwen3.6-flash)

Routing table

Tag	Model	Use case
read, discuss, search	openrouter/owl-alpha	Reading docs, general chat, web search
devops-low	qwen/qwen3.6-flash	Simple YAML, Docker, bash
devops-high	qwen/qwen-2.5-72b-instruct	Complex multi-container, server crashes
code-analysis-low	openrouter/owl-alpha	Finding bugs in short files
code-analysis-high	moonshotai/kimi-k2.6	Refactoring large codebases (262K context)
codewrite-low	deepseek/deepseek-v4-flash	Boilerplate, simple functions
codewrite-high (React)	qwen/qwen3-coder-plus	Complex React/JS
codewrite-high (other)	deepseek/deepseek-v4-pro	Complex PHP, dense logic
Short prompts (<15 chars)	openrouter/owl-alpha	Quick responses, greetings

Manual override

# Lock to a specific model (disables routing)
/lock-model openrouter/anthropic/claude-3.5-sonnet

# Re-enable dynamic routing
/unlock-model

When locked, all prompts go directly to the specified model until /unlock-model is called.

What about /do-* commands?

The /do-* prompt templates are no longer needed for model selection. The smart-router handles all routing automatically. However, /do-img and /do-make-img are still available for image analysis/generation since those use specialized tool calls rather than just model selection.

Configuration

Model mappings are defined in ~/.agents/extensions/smart-router/index.ts. To change which model a tag routes to, edit the MODELS table and run /reload.

Files

File	Purpose
~/.agents/extensions/smart-router/index.ts	Extension source
~/.pi/agent/extensions/smart-router/index.ts	Synced copy (Gitea backup)

---

MCP Servers

pi-mcp-adapter connects Pi to external services via the Model Context Protocol.

Config file: ~/.config/mcp/mcp.json

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/home/sam"]
    }
  }
}

Find MCP servers at:

• github.com/modelcontextprotocol/servers
• smithery.ai — community registry

Usage:

• /mcp — interactive panel to manage servers
• mcp({ search: "..." }) — search available tools
• mcp({ tool: "tool_name", args: '{}' }) — call a tool
• Servers are lazy (connect on first use, disconnect after 10 min idle)

---

Configuration Files

Global (~/.pi/agent/settings.json)

• Nix store symlink — managed via /etc/nixos/home/sam/home.nix
• Contains: providers (opencode-go, openrouter, google), packages (pi-memctx, pi-prompt-template-model, Gitea)
• Read-only — cannot be modified by pi install or /config-add

Project (<project-dir>/.pi/settings.json)

• Overrides global settings (arrays replace, not merge)
• Contains: ~/.agents package (extensions + skills), Gitea package (tavily-search)
• Modified via /config-add / /config-remove commands

Per-folder Memory (via pi-memctx)

• Memory stored in <chat-folder>/.pi/memory-vault/packs/
• Workspace map at ~/.pi/agent/memory-vault/00-system/workspace-map.json
• Each chat folder has isolated memory (prevents sibling directory contamination)

---

Useful Commands

Command	What it does
/lock-model <id>	Lock routing to a specific model (e.g. openrouter/anthropic/claude-3.5-sonnet)
/unlock-model	Re-enable dynamic prompt routing
/config-setup	One-shot: creates .pi/, settings.json, memory vault in current folder
/config-add ext <name>	Activate an extension from ~/.agents
/config-add skill <name>	Activate a skill from ~/.agents
/config-show	Show active extensions and skills
/memctx-init	Scan folder, build initial memory pack
/memctx-status	Show memory status
/memctx-refresh	Re-scan and enrich memory
/filechanges	Review changed files, diffs, accept/decline
/filechanges-accept	Accept all changes
/filechanges-decline	Revert all changes
markitdown <file>	Convert file to Markdown (PDF, Word, Excel, PPTX, images, HTML, etc.)
markitdown-vision <file>	Describe image using Qwen 2.5 VL 72B via OpenRouter

---

Skipped / Bookmarked

Extension/Skill	Reason
web-search (amosblomqvist)	❌ Redundant — Tavily does this
subagents (amosblomqvist)	❌ Redundant — pi-subagents already installed
bash-guard (amosblomqvist)	❌ Too aggressive — would interrupt flow
google-image-search (amosblomqvist)	❌ Would need Google Search API + CSE setup
pdf-reader (amosblomqvist)	⏳ Bookmarked — Python + pymupdf setup needed
notify (mitsuhiko)	⏳ Minor QoL — desktop notifications on task complete
audio/voice	⏳ Not practical

---

Tasks

• Rebuild NixOS to activate new packages (Google provider, ffmpeg/yt-dlp, pi-prompt-template-model, pi-mcp-adapter, pi-subagents)
• Add MCP servers to ~/.config/mcp/mcp.json or .mcp.json as needed (Home Assistant, databases, etc.)
• Run /reload in Pi to activate filechanges and new templates
• Add more prompt templates to ~/.pi/agent/prompts/ as needed
• Verify video-extract works with Gemini
• Add markitdown skill to Obsidian skills page
• Add smart-router extension and update Obsidian docs
• Clean up workspace-map.json entries for any stale memory packs