33ec9b5bc5
Classifies notes from an inbox folder and files them, with a human approval step. Zero third-party dependencies. - type mirrors folder (200 projects / 300 areas / 400 Personal Family) - TypeSafe JEV for semantic classification (~$0.0001 per note) - watcher writes a proposal INTO the note; the owner ticks a checkbox - systemd user service on .27, 10s poll - 15/15 functional tests
131 lines
5.0 KiB
Markdown
131 lines
5.0 KiB
Markdown
# HANDOFF — Obsidian Sorter
|
|
|
|
> Written 2026-09-24. Pick this up cold; everything you need is here.
|
|
|
|
## What this is
|
|
|
|
An automated system that files notes into Sam's Obsidian vault.
|
|
|
|
Notes go into `010 inbox/`. A **watcher** proposes a destination by writing it
|
|
*into the note's frontmatter*. Sam ticks a checkbox in Obsidian to approve.
|
|
The watcher then files it and updates the Table of Contents.
|
|
|
|
**Nothing is ever filed without approval** — deliberately, so Sam can edit a note
|
|
before it moves.
|
|
|
|
## Where things live
|
|
|
|
| Thing | Location |
|
|
|-------|----------|
|
|
| Vault (live) | `.27` → `~/obsidian` — git-synced to Gitea `sam/obsidian-vault` |
|
|
| Sorter code (dev) | `.13` → `~/paseo/projects/obsidian-sorter/` |
|
|
| Sorter code (deployed) | `.27` → `~/obsidian-sorter/` |
|
|
| Watcher service | `.27` → `systemctl --user obsidian-sorter` (enabled + lingering) |
|
|
| Command | `.27` → `~/bin/obsidian-sort` |
|
|
| Sort log | `.27` → `~/obsidian/010 inbox/_sort log.md` |
|
|
| Service log | `.27` → `~/.local/state/obsidian-sorter.log` |
|
|
| Design | `~/paseo/projects/obsidian-cleanup-tool/SORTER-PLAN.md` |
|
|
|
|
## THE TAXONOMY — final, do not "improve" it
|
|
|
|
**Type mirrors folder.** Nothing is inferred from content.
|
|
|
|
| Folder | Type |
|
|
|--------|------|
|
|
| `200 projects/` | `project` |
|
|
| `300 areas/` | `area` |
|
|
| `400 Personal Family/` | `family` |
|
|
| `100 Table of Contents/` | `index` |
|
|
| `000 daily/` | `daily` |
|
|
| `500 archive/` | `archive` |
|
|
| `Clippings/` | `clipping` |
|
|
| `010 inbox/` | `note` (unsorted placeholder only) |
|
|
|
|
**RETIRED — never reintroduce:** `subject`, `domain`, `tool`, `service`,
|
|
`container`, `machine`, `storage`, `backup-system`, `network-component`,
|
|
`website`, `person`.
|
|
|
|
Sam's lifecycle model:
|
|
```
|
|
IDEA (not started) -> 200 projects/260 Ideas to Build Create Make
|
|
BEING BUILT -> 200 projects/250 Electronics
|
|
FINISHED -> 300 areas/...
|
|
```
|
|
|
|
## The workflow
|
|
|
|
1. Note lands in `010 inbox/` with `type: note`
|
|
2. Watcher (polls every 10s) classifies it and **writes the proposal into the note**:
|
|
```yaml
|
|
proposed_folder: 200 projects/250 Electronics
|
|
proposed_toc: 100 Table of Contents/Projects.md → Ideas
|
|
proposed_tags: [solar, 12v, bird, bath]
|
|
confidence: 0.69
|
|
approved: false # a checkbox in Obsidian
|
|
```
|
|
3. Sam ticks `approved` (or edits the proposal first)
|
|
4. Watcher files it: move, set type from folder, merge tags, **strip admin
|
|
fields**, update the TOC
|
|
|
|
## Implementation notes
|
|
|
|
- **Zero dependencies.** Custom YAML parser (`sorter/frontmatter.py`) — no PyYAML,
|
|
because NixOS has no pip. HTTP via stdlib `urllib`.
|
|
- **JEV** (TypeSafe, `POST api.typesafe.ai/v1/systemone`) does semantic
|
|
classification — ~$0.0001/note. Key read from `~/.config/environment.d/*.conf`
|
|
(not in the env for non-login SSH shells).
|
|
- **Classifies by FOLDER, not type.** JEV picks the folder; the type follows.
|
|
- **Sam's own tags win.** A note tagged `project`/`family` is constrained to that
|
|
bucket — JEV cannot override it (`explicit_bucket_from_tags`).
|
|
- **Folder meanings are taught explicitly** in `FOLDER_NOTES` (tier2.py), e.g.
|
|
*"260 Ideas = UNSTARTED; 250 Electronics = ACTIVELY BEING BUILT"*.
|
|
**This is the place to fix any future misfiling.**
|
|
- JEV is **probabilistic** (~74% folder accuracy). Tests assert the *bucket*
|
|
strictly, the sub-folder only with `--strict-folders`.
|
|
- Files starting with `_` are ignored by the inbox scanner, so `_sort log.md`
|
|
and `_review` files never get sorted.
|
|
|
|
## Safety guarantees (15/15 tests)
|
|
|
|
Dry run by default · `Untitled*` never moved · never overwrites an existing
|
|
destination · frontmatter merged (user `tags`/`aliases`/`id`/`created` survive) ·
|
|
body never rewritten · atomic writes · `.trash/` untouched · idempotent TOC.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
# on .27
|
|
obsidian-sort # dry run — show what would happen
|
|
obsidian-sort apply # file the notes (asks to confirm)
|
|
obsidian-sort scan # vault summary
|
|
systemctl --user status obsidian-sorter
|
|
journalctl --user -u obsidian-sorter -f
|
|
|
|
# tests (on .13, against the mirror)
|
|
cd ~/paseo/projects/obsidian-sorter
|
|
python3 tests/run_tests.py
|
|
python3 tests/run_tests.py --strict-folders # may flake (JEV variance)
|
|
|
|
# refresh the local mirror from the live vault
|
|
rsync -a --delete --exclude='.git' --exclude='.trash' --exclude='.venv' \
|
|
--exclude='graphify-out*' --exclude='.obsidian' \
|
|
sam@192.168.20.27:~/obsidian/ ./test-vault/
|
|
|
|
# deploy code to .27
|
|
rsync -a --delete --exclude='test-vault' --exclude='__pycache__' \
|
|
--exclude='.git' --exclude='*.pyc' ./ sam@192.168.20.27:~/obsidian-sorter/
|
|
```
|
|
|
|
## Tag vocabulary
|
|
|
|
Cleaned 2026-09-24: 214 → 195 unique tags; lowercased everything, merged
|
|
`tools`→`tool`, split one malformed comma-joined tag into 9, fixed typos,
|
|
removed all retired-type tags. Tool: `tools/normalize_tags.py`.
|
|
|
|
## Remaining work
|
|
|
|
- **Phase D** — MCP tool so Pi agents can call the sorter
|
|
- **Phase E** — Pi skill ("sort my inbox") + distribute via Gitea to .13/.27/.51
|
|
- Add a watcher test to the suite (currently only verified manually)
|
|
- Sam is still testing with real notes — **no Pi integration until he's satisfied**
|