family_home_lab/docs/audio-plan.md

Audio & Music — Implementation Plan (Draft)

For Sam's son (music student). Goal: give the family lab a remote music-production workspace (DAWs) and an audio → sheet-music / MIDI transcription pipeline. Installing extra software in containers is approved.

1. Feasibility of the proposed tools (verified)

Tool	What it is	Remote-app container?	Fit
Zrythm	Open-source DAW (GPLv3), Pro-grade: mixed, automation, piano roll, module lanes	❌ no public image exists. Installable into a LinuxServer webtop container via its official installer (zrythm.org install.sh / apt repo).	✅ son's main DAW
LMMS	FL-Studio-style open-source DAW (beats / MIDI / virtual instruments)	✅ apt install lmms inside a webtop container (Ubuntu).	✅ secondary / beats
Spotify Basic Pitch	Audio→MIDI neural net (polyphonic, pitch bends). pip install basic-pitch. Best on a single instrument.	Runs as a Celery task (no UI).	✅ fast, lightweight first-pass MIDI
MuScriptor (Kyutai + Mirelo)	State-of-the-art multi-instrument transcription → MIDI + MusicXML + engraved PDF + guitar tabs in one run. muscriptor transcribe audio.wav --format sheets.	CLI / heavy model (1.4B).	✅ the "sheet music" engine you described
MuseScore	Open-source notation/engraver (MuScriptor uses it internally).	apt install musescore, also headless CLI.	✅ fallback: MIDI→MusicXML→PDF locally

2. Key caveats to decide on

• MuScriptor licensing: code is MIT, but model weights are CC BY-NC 4.0 (non-commercial). That's fine for personal/family use — but not for anything commercial.
• MuScriptor resources: 1.4B-parameter model. Feasible on CPU but slow; a GPU makes it practical. We have no GPU confirmed on .13 → treat MuScriptor as best-effort/beta, queue it in Celery, or throttle.
• Basic Pitch scale: fast on CPU, but "best on one instrument at a time". Good default for quick single-part transcription.
• Webtop note: the LinuxServer webtop terminal grants root inside the container. Safe on the trusted LAN behind Caddy; don't expose it to the public internet.

3. Proposed architecture

Family-music user (son/family)
   │  console.lab.audasmedia.com.au  (portal)
   ├── "Music (LMMS)"   → lmms.lab.audasmedia.com.au   (webtop:8086? )  remote DAW
   ├── "Zrythm"         → zrythm.lab.audasmedia.com.au (webtop:8087? )  remote DAW
   └── "Transcriber"    → console form:  pick audio (Garage) / upload
                          │  enqueue Celery task
                          ▼
                    worker (existing family-home-lab-worker)
                       ├─ Basic Pitch  → MIDI     (fast, single-instrument)
                       └─ MuScriptor   → MIDI+MusicXML+PDF+tabs   (beta, slow)
                          ▼
                    writes results back to Garage bucket + shared-media
                          ▼
                    Transcriber page lists/plays/downloads results

• Input & output live in Garage (sam/…/shared-media buckets), backed by Borg.
• Uses the existing RabbitMQ+Celery worker — no new queue.
• DAWs are separate webtop containers on free host ports with their own Caddy rows.

4. Blocker found on the DAWs (ports)

Originlab snippet used :3001 for zrythm — but that port is taken by langfuse. We'll use fresh free ports (e.g. 8085/8086/8087) and correct Caddy/nearby URLs.

5. Suggested build order (each is independently useful)

1. Transcriber (Basic Pitch) — quick win: upload/pick audio → MIDI; show result in console. (Easy, CPU, no license.)
2. LMMS remote DAW — apt install lmms on a webtop container, free port + Caddy row.
3. Zrythm remote DAW — custom webtop image with Zrythm installed.
4. MuScriptor sheet-music — add the multi-instrument + MusicXML/PDF stage (needs HF login/license; beta if no GPU).

6. Open decisions (need your call)

• Which transcriber to prioritize: Basic Pitch (fast, MIDI) vs MuScriptor (full sheet music, heavier) — or pair them.
• LMMS and/or Zrythm both? (I suggest both — they cover different use cases.)
• Confirm we may use the webtop image which gives container-root to the terminal.