sam/obsidian-vault
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

sam-4screen-desktop 2026-5-29:16:51:53

Browse Source
This commit is contained in:
Sam Rolfe
2026-05-29 16:51:53 +10:00
parent 8fb44c6331
commit 8bf182ca50
4 changed files with 273 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

214
100 inbox/Filesystem Consolidation Plan.md Normal file
View File

@@ -0,0 +1,214 @@
---
created: 2026-05-28
modified: 2026-05-28
type: note
tags: [dev-ops, filesystem, consolidation, planning]
aliases: [consolidation-plan, file-system-redesign]
---
# Filesystem Consolidation Plan
> DRAFT — Planning document for reorganising files across all machines.
---
## Principles
1. **One source of truth** — no duplicate data on multiple drives
2. **Always-on drives for active data** — rarely-used data goes on removable/archive drives
3. **NTFS for cross-platform** — anything that needs Samba/NFS access from Windows/Mac
4. **Backup repos stay where they are** — changing backup infrastructure is risky
5. **Consolidate by access frequency** — hot/warm/cold tiers
---
## Current Inventory — What Needs Homes
### Always-On Services (need drives mounted 24/7)
| Service | Machine | Current Mount | Size | Notes |
|---------|---------|--------------|------|-------|
| **Immich photos** | .35 | `/mnt/hd/immich/` | ~? | 40K photos, need fast reliable access |
| **Jellyfin media** | .35 | `/mnt/hd/...` | ? | Movies, TV shows, music |
| **Docker containers** | .35 | `/Docker/Containers/` | 23.7G | Already on system drive |
| **Docker data** | .35 | `/mnt/hd/*/` | ? | Lidarr, Radarr, Sonarr, etc. data |
| **Websites** | .13 | `/var/www/` | ? | Active web projects |
| **Docker on .13** | .13 | `/home/sam/Docker/` | ? | |
### Personal Data (frequent access)
| Data | Current Location(s) | Size | Notes |
|------|-------------------|------|-------|
| **Dev projects** | .27 `~/...` | ? | NixOS, coding projects |
| **Old web work** | .13 `ALL_TRANFERS/` + .27 `/mnt/backup/` | ? | htdocs, PHP, etc. — **DUPLICATED** |
| **Arduino / IoT** | .13 `ALL_TRANFERS` + SmartDrive | ? | **DUPLICATED** |
| **Flash AS3.0** | .13 `ALL_TRANFERS` | ? | Rarely accessed |
| **Music** | .13 + .27 | ? | **DUPLICATED** |
| **Sheet music** | ? | ? | Could move to Obsidian |
| **Family notes** | ? | ? | Could move to Obsidian |
| **TODO lists** | Obsidian + ? | — | Already in Obsidian |
| **Movies/TV** | .35 `/mnt/hd/` + .13 | ? | Jellyfin media |
| **Display screen + website builder** | ? | ? | Large project? |
### Archive / Rarely Accessed (cold storage)
| Data | Current Location(s) | Size | Notes |
|------|-------------------|------|-------|
| **Sorted photos (2003–2023)** | .13 `sorted_pictures_holder/` | ~? | 31K photos, already organized |
| **Old subject-organized photos** | .27 `/mnt/backup/` + .13 | ~? | **DUPLICATED** |
| **Integral300 recovery .img** | .27 `/mnt/tempbackup/` | 298G | Keep or delete? |
| **Integral300 backup** | .27 SmartDrive `integral300_backup_...` | 105G | Already extracted — delete? |
| **Windows old drives** | .13 `/mnt/storage/` | 271G used | Old Windows files |
| **Old Windows data drive** | .13 `/mnt/data/` | 1.5T used | ALL_TRANFERS, etc. — **DUPLICATED** |
### Backup Repos (stay where they are)
| Repo | Machine | Path | Size |
|------|---------|------|------|
| Borg .27 repo | .13 | `/mnt/ubuntu_storage_3TB/backup/borg/.27` | ~? |
| Borg .13 repo | .13 | `/mnt/ubuntu_storage_3TB/backup/borg/.13` | ~? |
| Kopia repo | .23 (via .35) | `/mnt/nfs_usb_3tb/kopia/` | 46G |
| Restic repo | .23 (via .35) | `/mnt/nfs_usb_3tb/backups/restic/` | 13G |
---
## Drive Role Assignment
### Always-On Drives (mounted 24/7)
| Machine | Drive | Mount | Role | Filesystem |
|---------|-------|-------|------|------------|
| **.35** | `vda1` (My Passport) | `/mnt/hd` | **Active data** — Immich, Jellyfin, Docker service data, current projects | NTFS (cross-platform) |
| **.13** | `sdd1` (ubuntu_storage_3TB) | `/mnt/ubuntu_storage_3TB` | **Backup target** — Borg repos only | ext4 |
| **.13** | `sdb2` (Data) | `/mnt/data` | **Archive** — old photos, old web work, ALL_TRANFERS | NTFS (cross-platform) |
| **.13** | `sda2` (Storage) | `/mnt/storage` | **Windows legacy** — old Windows files | NTFS (cross-platform) |
### Warm-Swap Drives (plug in when needed)
| Drive | Size | Free | Role | Label |
|-------|------|------|------|-------|
| **SmartDrive** | 687G | 548G | **Transfer / temporary** — moving data between machines | `SmartDrive` |
| **TempBackup** | 458G | 136G | **Offline backup** — secondary copy of critical data | `TempBackup` |
| **Back Up** | 224G | 158G | **Legacy** — old photos (read-only NTFS) | `Back Up` (NTFS) |
| **MaxtorBackup** | 1.4T | 1.3T | **Archive target** — empty, ready for use | `MaxtorBackup` |
---
## Proposed Folder Hierarchy
### On `.35` — `/mnt/hd/` (My Passport — NTFS) — Active Data
This is the hot data tier — always accessible via Docker, Samba, etc.
```
/mnt/hd/
├── services/ ← Docker service data (bind-mounted)
│ ├── immich/ ── Already here
│ ├── audiobookshelf/
│ ├── lidarr/
│ ├── radarr/
│ ├── sonarr/
│ ├── qbittorrent/
│ ├── readarr/
│ ├── headphones/
│ ├── spotweb/
│ ├── nextcloud/ ── Future?
│ └── paperless/ ── Already on NFS
├── media/ ← Jellyfin media library
│ ├── movies/
│ ├── tv/
│ ├── music/
│ ├── audiobooks/
│ ├── podcasts/
│ └── books/
└── projects/ ← Current active projects
├── web/
├── arduino/
├── nixos/
└── ...
```
### On `.13` — `/mnt/data/` (NTFS) — Archive & Warm Data
This is the archive tier — old projects, old photos, everything rarely accessed but worth keeping.
```
/mnt/data/
├── archive/
│ ├── photos/ ← Consolidated photo archive
│ │ ├── sorted_pictures_holder/ ── From ALL_TRANFERS
│ │ └── old_subject_organized/ ── From Back Up drive + PICTURES_GROUPED_SUBJECT_OLD
│ ├── web_work/ ← Old htdocs, PHP, Flash AS3.0 projects
│ ├── arduino_iot/
│ ├── music/
│ ├── sheet_music/
│ ├── family_notes/
│ └── integral300/ ← Recovery .img moved here from .27
└── transfer/ ← Staging area for moving data between machines
```
### On `.13` — `/mnt/ubuntu_storage_3TB/` (ext4) — Backup Repository
```
/mnt/ubuntu_storage_3TB/
└── backup/
├── borg/
│ ├── .13/ ← .13 local backups
│ └── .27/ ← .27 backups via SSH
└── restic/ ← Future: secondary restic target?
```
---
## Consolidation Actions
### Phase 1 — Eliminate Duplicates
| What | Duplicated Where | Action |
|------|-----------------|--------|
| Old subject-organized photos | .27 `/mnt/backup/` + .13 `PICTURES_GROUPED_SUBJECT_OLD` | Delete from .27 (keep .13 copy) |
| Integral300 extracted backup | .27 SmartDrive `integral300_backup_...` | Delete — recovery .img is the source of truth |
| Arduino / IoT files | .13 ALL_TRANFERS + SmartDrive | Consolidate to .13 archive |
| Old web work (htdocs) | Multiple places | Consolidate to .13 `/mnt/data/archive/web_work/` |
### Phase 2 — Consolidate to Right Drives
| What | Move From | Move To | When |
|------|----------|---------|------|
| Integral300 recovery .img | .27 `/mnt/tempbackup/` | .13 `/mnt/data/archive/integral300/` | Phase 2 |
| Sorted photos | .13 `ALL_TRANFERS/Pictures/sorted_pictures_holder/` | .13 `/mnt/data/archive/photos/` | Phase 2 |
| Old web work | .13 `ALL_TRANFERS/` | .13 `/mnt/data/archive/web_work/` | Phase 2 |
### Phase 3 — System Rebalancing
| Action | Reason |
|--------|--------|
| Mount MaxtorBackup as **offline backup** repository | Provides a second physical copy of backup repos |
| Configure periodic rsync from .23 backup repos → MaxtorBackup | .23 has no backup — this covers it |
| Clean up old empty stub directories on .35 `/mnt/hd/` | Remove Western Digital installer files, System Volume Information |
---
## Always-Connected vs Hot-Swap
| Drive | Connection | Why |
|-------|-----------|-----|
| **.35** My Passport (vda, /mnt/hd) | **Always on** | Immich, Jellyfin, Docker services need this |
| **.13** ubuntu_storage_3TB | **Always on** | Backup target — Borg runs daily |
| **.13** Data (sdb) | **Always on** | Archive data, accessible to network |
| **.13** Storage (sda) | **Always on** | Old Windows files |
| **.27** SmartDrive | **Hot-swap** | Only needed during data transfers |
| **.27** TempBackup | **Hot-swap** | Temporary storage / cache |
| **.27** Back Up | **Hot-swap** | Legacy — eventually retire |
| **MaxtorBackup** | **Hot-swap** | Offline backup — plug in monthly |
---
## Open Questions
- [ ] Should Immich and Jellyfin media live on the same drive? (single point of failure)
- [ ] Should we move old photo archive to MaxtorBackup and rotate it offsite?
- [ ] Is the Integral300 recovery .img still needed (298G)? It matches the extracted files.
- [ ] What about `.13`'s `/var/www/` websites — should they be backed up?
- [ ] Should `.23` (file-server) get its own backup?
- [ ] Naming convention for drives: should we re-label drives with consistent names?

40
100 inbox/Filesystem Drive Map.md
View File

@@ -62,13 +62,13 @@ aliases: [drive-map, filesystem-map]
| Logical Name | Device | Size | Used | Avail | Mount | Label | Note |
|-------------|--------|------|------|-------|-------|-------|------|
| **System Root** | `sda2` | 442G | 165G | 256G (40%) | `/` | — | Ubuntu 24.04 |
| **System Root** | `sda2` | 442G | 165G | 256G (40%) | `/` | — | Ubuntu 24.04 VM |
### Not Mounted
### Attached Virtual Disk (My Passport — Data Drive)
| Logical Name | Device | Size | FSType | Label | Note |
|-------------|--------|------|--------|-------|------|
| **My Passport** | `vda1` | 1.8T | ntfs | `My Passport` | Western Digital USB drive — needs mounting to check content |
| Logical Name | Device | Size | Used | Avail | Mount | Label | UUID | Note |
|-------------|--------|------|------|-------|-------|-------|------|------|
| **My Passport** | `vda1` | 1.8T | — | — | **`/mnt/hd`** | `My Passport` | `AE5A06ED...` | NTFS, physical WD drive passed through to VM. **Contains all real Docker service data.** Previously mounted at `/mnt/my_passport` — fixed 2026-05-28. |
### NFS Mount (from .23)
@@ -89,21 +89,23 @@ aliases: [drive-map, filesystem-map]
| `downloads/` | — | Download stubs (subdirs only, empty) |
| `kopia_0.23.0_linux_amd64.deb` | 17M | Kopia installer |
### Immich (Photo Library — DATA LOST)
### Immich (Photo Library — RECOVERED)
| Path | Status |
|------|--------|
| `/mnt/hd/immich/` | ❌ Empty (data lost Apr 21 2026) |
| PostgreSQL DB | ✅ Metadata for 40,769 assets survives (Dec 2024 – Jan 2025) |
| Docker containers | ✅ Running (crashed, needs folder init) |
| `/mnt/hd/immich/` | ✅ **Recovered 2026-05-28** — all 40K photos found on My Passport |
| PostgreSQL DB | ✅ Metadata intact |
| Docker containers | ✅ Running again after mount fix |
| **Cause** | My Passport was mounted at `/mnt/my_passport` instead of `/mnt/hd` — Docker looked at `/mnt/hd` which was empty stubs on root |
### Backup Role
| Direction | Method | Target | Schedule |
|-----------|--------|--------|----------|
| `.35 → .23` | Kopia (host) | `/mnt/nfs_usb_3tb/kopia/` | Weekly (Sun 8am) |
| `.35 → .23` | Backrest/Restic (Docker) | `/mnt/nfs_usb_3tb/backups/restic/` | Daily? |
| **Gap** | **`/mnt/hd/` NOT backed up** | — | ❌ No backup covers host-level paths |
| `.35 → .23` | Kopia | `/mnt/nfs_usb_3tb/kopia/` | Weekly Sun 22:00 |
| `.35 → .23` | Backrest (Restic) | `/mnt/nfs_usb_3tb/backups/restic/` | Weekly Sun 23:00 |
| `.35` VM-level | Proxmox Backup Server (.48) | — | Full VM backup |
| **`/mnt/hd/`** | **Backrest (Restic)** | **Same restic repo** | ✅ **Added 2026-05-28** — `/host_fs/mnt/hd` now in `Docker_Containers` plan |
---
@@ -187,15 +189,13 @@ No SSH access available. Contents fully visible via .35's CIFS mount.
### Key Gaps to Address
1. **Immich photos — lost forever** unless on disconnected drive or MaxtorBackup (.13)
2. **`/mnt/hd/` on .35 is not backed up** — any Docker host bind-mount data is unprotected
1. ✅ **Immich photos recovered** — found on My Passport, now mounted at `/mnt/hd`
2. ✅ **`/mnt/hd/` backup added** — `/host_fs/mnt/hd` added to Backrest (Restic) plan
3. **.23 has no backup** — single point of failure for .35's backup repos
4. **MaxtorBackup on .13 is unmounted** — needs investigation
5. **My Passport on .35 is unmounted** — needs investigation
4. **MaxtorBackup on .13 is unmounted** — 1.4T ext4, 1% used (~11G, likely empty)
### Next Steps
- [ ] Check MaxtorBackup (`/dev/sde1` on .13 — 1.4T ext4)
- [ ] Check My Passport (`/dev/vda1` on .35 — 1.8T ntfs)
- [ ] Decide if `/mnt/hd/` on .35 should be added to Kopia/Restic backup scope
- [ ] Consider if .23 needs its own backup
- [ ] Confirm Backrest backup of `/host_fs/mnt/hd` runs successfully on Sunday
- [ ] Consider if `.23` file-server needs its own backup
- [ ] Check MaxtorBackup (`/dev/sde1` on .13) if needed