obsidian-vault/300 areas/350 AI/Pi Agent Extensions & Skills.md

created, modified, type, tags, aliases

created	modified	type	tags	aliases
2026-05-16	2026-05-16	note	ai dev-ops

Pi Agent Extensions & Skills

Source Repositories

Source	Location
Gitea (package)	git:https://gitea.lab.audasmedia.com.au/sam/pi-config
Local filesystem	~/.agents/
Project settings	sys_config/.pi/settings.json, ai_setup/.pi/settings.json

---

Extensions

Extension	Source	Purpose
pi-config	~/.agents	/config-add, /config-remove, /config-show, /config-setup — manage which extensions/skills are active in a project
tavily-search	Gitea	tavily_search — web search via Tavily API (AI-optimized)
web-fetch	~/.agents	web_fetch — fetch any URL and return clean markdown (HTML, PDF, JS-rendered with Jina fallback)
ask-user-question	~/.agents	ask_user_question — LLM presents structured multiple-choice / text questions with keyboard UI
video-extract	~/.agents	video_extract — extract frames from YouTube/local video + full Gemini analysis (requires ffmpeg + yt-dlp + GEMINI_API_KEY)
filechanges	~/.agents	/filechanges, /filechanges-accept, /filechanges-decline — tracks every file LLM edits/writes, diff review, revert
pi-subagents (nicopreme)	npm (global)	Spawn child Pi agents with chains, parallel execution, async dispatch. /agents command. Integrates with pi-prompt-template-model for delegated prompt execution
pi-prompt-template-model	npm (global)	Model-switching prompt templates with frontmatter. See #Prompt Templates section below
pi-mcp-adapter	npm (global)	Single proxy tool (~200 tokens) replaces hundreds of MCP tool definitions. /mcp command for management. Lazy server connections
pi-graphify	~/.agents	Knowledge graph tools: build, query, path tracing, explain, watch, add, update
plannotator	~/.agents	Interactive plan review with browser UI, annotations, code review
caveman	~/.agents	Ultra-compressed communication mode
markitdown	~/.agents	Convert files (PDF, Word, Excel, PPTX, images, HTML, etc.) to Markdown. Image analysis via Qwen 2.5 VL 72B on OpenRouter.

---

Skills

Skill	Purpose
nixos-workflow	STRICT workflow for managing Pi assets via Gitea on NixOS
system-architect	Multi-machine NixOS infrastructure (Snapcast, MQTT, Docker, Nvim)
obsidian-cli	Interact with Obsidian vault (notes, search, plugin dev, theme dev)
graphify	Full-pipeline knowledge graph orchestration
caveman	Caveman communication mode
openspec-propose	Propose new changes with design docs, specs, tasks
openspec-apply-change	Implement tasks from an OpenSpec change
openspec-archive-change	Archive completed changes
openspec-explore	Explore ideas and clarify requirements
npm-security	Scan packages with SafeDep Vet, check typosquatting with npq, wrap installs with Socket Firewall
markitdown	Convert files (PDF, Word, Excel, PowerPoint, images, HTML, CSV, JSON, XML, ZIP, EPubs, YouTube) to Markdown for LLM consumption. Image analysis via Qwen 2.5 VL 72B on OpenRouter.

---

markitdown

Convert various file formats to Markdown. Useful for feeding documents and images into LLMs.

What it converts

Format	Input	Notes
PDF	.pdf	Preserves structure (headings, lists, tables)
Word	.docx	mammoth + lxml
PowerPoint	.pptx	python-pptx
Excel	.xlsx, .xls	openpyxl + pandas
Images	.jpg, .png, etc.	EXIF metadata (free) + LLM vision description (via OpenRouter)
HTML	.html	beautifulsoup4
CSV / JSON / XML	.csv, .json, .xml	Structured data → Markdown tables
ZIP	.zip	Iterates contents, converts each file
EPubs	.epub
YouTube	URLs	Transcript extraction

CLI usage

# Convert file to Markdown (stdout)
markitdown document.pdf

# Write to file
markitdown document.pdf -o document.md

# Image with LLM vision description
markitdown-vision photo.jpg

Image analysis

Two levels:

1. EXIF metadata only (free, no API key): markitdown photo.jpg
2. LLM vision description (via OpenRouter, requires API key): markitdown-vision photo.jpg

The markitdown-vision wrapper auto-sources OPENROUTER_API_KEY from ~/.config/environment.d/10-secrets.conf and uses qwen/qwen2.5-vl-72b-instruct.

Missing / can be added

Feature	What's needed
Audio transcription	pip install markitdown[audio-transcription] (pydub + speechrecognition)
Azure AI Document Intelligence	pip install markitdown[az-doc-intel] + Azure credentials
Azure Content Understanding	pip install markitdown[az-content-understanding] + Azure credentials
markitdown-ocr plugin	Installed but needs OpenRouter key enabled to activate

---

Security Tools (npm Global)

Three tools installed globally at ~/.local/share/npm-global/bin/ to guard package installs.

SafeDep Vet (vet)

Scans local directories for multi-language malware signatures. Catches obfuscated code, suspicious imports, base64 payloads.

# Scan a cloned repo before touching it
vet scan -D . --format json --filter "package.malware == true"

# Scan package metadata from npm registry
vet scan package <name> --format json

Socket Firewall (socket)

Wraps npm/pip installs with real-time scanning. Blocks malicious packages at install time.

# Safe npm install
socket npm install <package>

# Safe pip install
socket pip install -r requirements.txt

npq

Checks package names against typosquatting lists before install. Lightweight, local, no phoning home.

npq check <package> --json

Workflow

1. vet scan → checks for malware in the code/package
2. npq check → checks the package name for typosquatting
3. socket install → wraps the actual install with runtime scanning

The npm-security skill instructs the Pi agent to follow this workflow before any install.

---

Prompt Templates

Prompt templates are .md files in ~/.pi/agent/prompts/ with YAML frontmatter. Each defines a slash command that auto-switches model, thinking level, and injects skills — then restores your previous model when done.

Installed Templates

Command	Tier	Model	Use
/do-quick	⚡ flash	deepseek-v4-flash (opencode-go)	General everyday tasks
/do-think	🧠 ultra	openrouter/deepseek/deepseek-v4-pro	Deep thinking, hard problems
/do-check	🔷 pro	opencode-go/deepseek-v4-pro	Code review, quality checks
/do-nix-check	⚡ flash	deepseek-v4-flash (opencode-go)	Nix flake checks, config debug
/do-note	⚡ flash	deepseek-v4-flash (opencode-go)	Create Obsidian .md with frontmatter
/do-sort	🔷 pro	opencode-go/deepseek-v4-pro	Classify files, suggest Obsidian folder
/do-read	⚡ flash	deepseek-v4-flash (opencode-go)	Batch read and synthesize many files
/do-debug	🧠 ultra	openrouter/deepseek/deepseek-v4-pro	Complex NixOS/infra debugging
/do-vault	🔷 pro	opencode-go/deepseek-v4-pro	Obsidian vault setup and management
/do-chat	⚡ flash	deepseek-v4-flash (opencode-go)	Chat with automatic web search
/do-img	🧠 ultra	openrouter/deepseek/deepseek-v4-pro	Image analysis and description
/do-make-img	⚡ flash	deepseek-v4-flash (opencode-go)	Generate images via OpenRouter (GPT/Image Mini, Gemini)

Audio/Voice note: Pi runs in a text-based terminal (TUI) with no microphone access or audio playback. Qwen supports audio processing, but it's not practical in Pi's current architecture regardless of model.

How to use

/do-quick what's the weather?
/do-think design the architecture for this feature
/do-check review the changes in this file
/do-nix-check analyze the nix flake check output
/do-note document the new service setup
/do-sort read this config file and tell me where it goes
/do-read skim these three config files and compare them
/do-debug why is my NixOS build failing?
/do-vault set up a template for daily notes
/do-chat what's new in NixOS 25.11?
/do-img what's wrong in this screenshot?
/do-make-img a futuristic cityscape with neon lights

Template format

---
description: Quick task — fast model, minimal thinking
model: openrouter/deepseek-v4-flash
restore: true
---
$@

Key frontmatter fields:

• model — provider/model-id or comma-separated fallback list
• thinking — off, minimal, low, medium, high, xhigh
• restore — (default true) restore previous model when done
• skill — auto-inject a skill
• deterministic — run a command/script before the LLM turn (e.g. nix eval .#checks)

Adding new templates

Create a new .md file in ~/.pi/agent/prompts/ with frontmatter. The filename becomes the command name. Restart Pi or run any extension command to pick them up.

---

MCP Servers

pi-mcp-adapter connects Pi to external services via the Model Context Protocol.

Config file: ~/.config/mcp/mcp.json

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/home/sam"]
    }
  }
}

Find MCP servers at:

• github.com/modelcontextprotocol/servers
• smithery.ai — community registry

Usage:

• /mcp — interactive panel to manage servers
• mcp({ search: "..." }) — search available tools
• mcp({ tool: "tool_name", args: '{}' }) — call a tool
• Servers are lazy (connect on first use, disconnect after 10 min idle)

---

Configuration Files

Global (~/.pi/agent/settings.json)

• Nix store symlink — managed via /etc/nixos/home/sam/home.nix
• Contains: providers (opencode-go, openrouter, google), packages (pi-memctx, pi-prompt-template-model, Gitea)
• Read-only — cannot be modified by pi install or /config-add

Project (<project-dir>/.pi/settings.json)

• Overrides global settings (arrays replace, not merge)
• Contains: ~/.agents package (extensions + skills), Gitea package (tavily-search)
• Modified via /config-add / /config-remove commands

Per-folder Memory (via pi-memctx)

• Memory stored in <chat-folder>/.pi/memory-vault/packs/
• Workspace map at ~/.pi/agent/memory-vault/00-system/workspace-map.json
• Each chat folder has isolated memory (prevents sibling directory contamination)

---

Useful Commands

Command	What it does
/do-quick <task>	⚡ Fast model for everyday tasks
/do-think <task>	🧠 Ultra model for deep analysis
/do-check <path>	🔷 Pro model for code review
/do-nix-check <task>	⚡ Nix-specific analysis
/do-note <description>	⚡ Create Obsidian note with frontmatter
/do-sort <path>	🔷 Classify file, suggest Obsidian folder
/do-read <paths>	⚡ Batch read and synthesize files
/do-debug <issue>	🧠 Complex NixOS/infra debugging
/do-vault <task>	🔷 Obsidian vault setup and management
/do-chat <topic>	⚡ Conversation with web search
/do-img <prompt>	🧠 Image analysis and description
/do-make-img <prompt>	⚡ Generate image via OpenRouter (GPT/Image Mini, Gemini)
/config-setup	One-shot: creates .pi/, settings.json, memory vault in current folder
/config-add ext <name>	Activate an extension from ~/.agents
/config-add skill <name>	Activate a skill from ~/.agents
/config-show	Show active extensions and skills
/memctx-init	Scan folder, build initial memory pack
/memctx-status	Show memory status
/memctx-refresh	Re-scan and enrich memory
/filechanges	Review changed files, diffs, accept/decline
/filechanges-accept	Accept all changes
/filechanges-decline	Revert all changes
markitdown <file>	Convert file to Markdown (PDF, Word, Excel, PPTX, images, HTML, etc.)
markitdown-vision <file>	Describe image using Qwen 2.5 VL 72B via OpenRouter

---

Skipped / Bookmarked

Extension/Skill	Reason
web-search (amosblomqvist)	❌ Redundant — Tavily does this
subagents (amosblomqvist)	❌ Redundant — pi-subagents already installed
bash-guard (amosblomqvist)	❌ Too aggressive — would interrupt flow
google-image-search (amosblomqvist)	❌ Would need Google Search API + CSE setup
pdf-reader (amosblomqvist)	⏳ Bookmarked — Python + pymupdf setup needed
notify (mitsuhiko)	⏳ Minor QoL — desktop notifications on task complete
audio/voice	⏳ Not practical

---

Tasks

• Rebuild NixOS to activate new packages (Google provider, ffmpeg/yt-dlp, pi-prompt-template-model, pi-mcp-adapter, pi-subagents)
• Add MCP servers to ~/.config/mcp/mcp.json or .mcp.json as needed (Home Assistant, databases, etc.)
• Run /reload in Pi to activate filechanges and new templates
• Add more prompt templates to ~/.pi/agent/prompts/ as needed
• Verify video-extract works with Gemini
• Add markitdown skill to Obsidian skills page
• Clean up workspace-map.json entries for any stale memory packs