obsidian-sorter

Automated note classification and filing for an Obsidian vault.

Notes enter through 010 inbox/, the sorter decides where they belong, and — once approved — writes them into place and updates the Table of Contents.

Design goal: no dependencies. The vault uses simple, uniform YAML; the tool ships a small purpose-built parser so it deploys onto hosts without pip (NixOS .13, .27, .51).

---

🛠 Demonstrated Skills & Tech Stack

• Core Technologies: Python 3.10+ using the standard library only — no PyYAML, no pip dependencies; systemd user services; YAML frontmatter.
• Key Tools: TypeSafe JEV (POST api.typesafe.ai/v1/systemone) for semantic classification, stdlib urllib for HTTP, mmdc + Archify for diagrams, rsync for deployment, Git + Gitea for version control.
• Engineering Capabilities: a deterministic rule engine with a probabilistic second tier, atomic file writes (tempfile + os.replace), idempotent index updates, a human-in-the-loop approval gate, and a purpose-built YAML parser written to avoid runtime dependencies on NixOS hosts.

📐 System Architecture

flowchart LR
    A["Sam writes a note"] --> B["010 inbox/"]
    B --> C{"Watcher<br/>systemd user service<br/>polls every 10s"}
    C --> D["Tier 1 — rules<br/>folder + keyword hints<br/>user tags win"]
    D -->|"confident"| F["Proposal"]
    D -->|"unsure"| E["Tier 2 — TypeSafe JEV<br/>bucket + folder + tags"]
    E --> F
    F --> G["Proposal written<br/>into the note frontmatter"]
    G --> H["Sam reviews in Obsidian"]
    H -->|"ticks approved"| I["Writer"]
    H -->|"edits folder first"| G
    I --> J["Destination folder"]
    I --> K["Table of Contents"]

Interactive Architecture Map: maps.lab.audasmedia.com.au/obsidian-sorter/docs/obsidian-sorter-map.html

---

The taxonomy (simplified 2026-09-24)

The type of a note mirrors the folder it lives in. Nothing is guessed from content — the destination decides the type.

Folder	Type	Meaning
200 projects/	project	Work in progress — something being built
300 areas/	area	Ongoing interest with documentation and notes
400 Personal Family/	family	Family / non-work
100 Table of Contents/	index	Navigation notes only
000 daily/	daily	Chronological daily notes
500 archive/	archive	Inactive material
Clippings/	clipping	Browser-extension captures
010 inbox/	note	Unsorted placeholder

Everything else has been retired: subject, domain, tool, service, container, machine, storage, backup-system, network-component, website, person. Do not reintroduce them.

The lifecycle

being built      ->  200 projects/...        ->  project
built and running->  300 areas/...           ->  area

A note moves between the two when the work finishes. Its areas: field records which area it belongs to while it is still a project.

---

Status

Phase	Scope	State
A	Vault scanner + deterministic classifier	✅
B	JEV (TypeSafe) semantic tier	✅
C	Writer: frontmatter, move, TOC insert, approval gate	✅
D	MCP tool wrapper	pending
E	Pi skill + Gitea distribution	pending
F	Validation command	pending

---

🚀 Quick Start

Nothing to install — pure standard library, Python 3.10+.

Set the API key for the semantic tier:

export TYPESAFE_API_KEY=...        # required for --jev / benchmark

---

The workflow (how a note gets filed)

The watcher runs as a systemd user service on .27, always on.

you save a note into 010 inbox/          type: note
        |
        v   watcher notices within ~10s
    it classifies the note and WRITES THE PROPOSAL INTO THE NOTE

        type: note
        proposed_folder: 200 projects/250 Electronics
        proposed_toc: 100 Table of Contents/Projects.md → Ideas
        proposed_tags: [solar, 12v, bird, bath]
        confidence: 0.69
        approved: false          <-- a checkbox in Obsidian
        |
        v   you open the note and tick the box (or edit the proposal first)
        approved: true
        |
        v   watcher notices within ~10s
    it files the note, strips the admin fields, updates the TOC
        |
        v
you see the note appear in the right folder, listed in the index

Nothing is ever filed without your approval. You can:

• tick approved to accept as-is
• edit proposed_folder (or proposed_tags) first, then tick it
• leave it alone — it waits in the inbox indefinitely

Admin fields (proposed_folder, proposed_toc, proposed_tags, confidence, approved) are removed when the note is filed, so the finished note is clean.

A running log is written to 010 inbox/_sort log.md.

Managing the watcher

systemctl --user status  obsidian-sorter    # is it running?
systemctl --user stop    obsidian-sorter    # pause it
systemctl --user start   obsidian-sorter    # resume
systemctl --user disable obsidian-sorter    # don't start on boot
journalctl --user -u obsidian-sorter -f     # live output

Logs: ~/.local/state/obsidian-sorter.log

---

📝 Usage & Configuration

Manual use

obsidian-sort is installed on .27 (in ~/bin):

obsidian-sort              # show what would happen (dry run, no writes)
obsidian-sort apply        # file the notes, asks to confirm
obsidian-sort apply --yes  # file without asking
obsidian-sort scan         # vault summary

Or call the module directly from anywhere the code is deployed:

python3 -m sorter --vault ~/obsidian --inbox "010 inbox" watch --once
python3 -m sorter --vault ~/obsidian --inbox "010 inbox" sort-inbox --jev --apply

---

Usage reference

Read-only

# Vault shape: counts, types, what needs classification
python3 -m sorter --vault ~/obsidian scan

# Classify one note (rules only, free)
python3 -m sorter --vault ~/obsidian propose "010 inbox/My Note.md"

# Classify with the semantic tier
python3 -m sorter --vault ~/obsidian --inbox "010 inbox" \
    propose "010 inbox/My Note.md" --jev

Filing (writes)

Dry run by default — nothing is written without --apply.

# Show the plan
python3 -m sorter --vault ~/obsidian --inbox "010 inbox" \
    sort "010 inbox/My Note.md" --jev

# Apply it
python3 -m sorter --vault ~/obsidian --inbox "010 inbox" \
    sort "010 inbox/My Note.md" --jev --apply

# Whole inbox, no prompt (for agents)
python3 -m sorter --vault ~/obsidian --inbox "010 inbox" \
    sort-inbox --jev --apply --yes

The plan lists every change explicitly:

Plan for: 010 inbox/50 Legendary Websites.md

  → 300 areas/390 Web Pages and Tools To Try/50 Legendary Websites.md
    title  : 50 Legendary Websites
    type   : area   status: active
    toc    : 100 Table of Contents/Areas.md → Web Pages & Tools To Try

  changes:
  + create  300 areas/390 Web Pages and Tools To Try/50 Legendary Websites.md
  ~ update  100 Table of Contents/Areas.md   (add [[50 Legendary Websites]] under 'Web Pages & Tools To Try')
  - remove  010 inbox/50 Legendary Websites.md   (moved out of inbox)

Measuring the semantic tier

python3 -m sorter --vault ~/obsidian --inbox "010 inbox" benchmark --all --quiet

---

Safety guarantees (all tested)

Guarantee	Behaviour
Dry run by default	--apply required to write anything
Untitled* never moved	Plan marked BLOCKED, exit code 2, file untouched
Idempotent TOC	Re-running never duplicates an index entry
Never overwrites	An existing destination blocks the plan
Frontmatter merged	Existing tags, aliases, id survive; created preserved
Body untouched	The author's words are never rewritten
Atomic writes	temp file + os.replace — no partial files
.trash/ off limits	Files under .trash are never removed

---

Testing

Runs against an isolated copy — the mirror is never modified.

python3 tests/run_tests.py            # full suite (uses JEV)
python3 tests/run_tests.py --no-jev   # mechanics only, no API cost
python3 tests/run_tests.py --keep     # keep the temp vault for inspection

Covers: dry-run safety, all three buckets routing to the correct sub-folder, frontmatter merging, the Untitled guard, destination collisions, .trash protection, inbox clearing, idempotency, and duplicate index entries.

---

Development

# Read-only mirror of the live vault for testing
rsync -a --delete \
  --exclude='.git' --exclude='.trash' --exclude='.venv' \
  --exclude='graphify-out*' --exclude='.obsidian' \
  sam@192.168.20.27:~/obsidian/ ./test-vault/

test-vault/ is a throwaway mirror and is gitignored.

Layout

File	Purpose
sorter/frontmatter.py	Minimal YAML reader/writer (no PyYAML)
sorter/models.py	Types, Note, Proposal, folder→type mapping
sorter/vault.py	Vault scanning, index introspection
sorter/classify.py	Tier 1 rules
sorter/jev.py	TypeSafe JEV client + cost tracking
sorter/tier2.py	Tier 2 semantic classifier + hybrid
sorter/writer.py	Plans, frontmatter merge, TOC insert, apply
sorter/cli.py	Command line