# Audio & Music — Implementation Plan (Draft)

> For Sam's son (music student). Goal: give the family lab a remote music-production
> workspace (DAWs) and an **audio → sheet-music / MIDI transcription** pipeline.
> Installing extra software in containers is approved.

## 1. Feasibility of the proposed tools (verified)

| Tool | What it is | Remote-app container? | Fit |
|---|---|---|---|
| **Zrythm** | Open-source DAW (GPLv3), Pro-grade: mixed, automation, piano roll, module lanes | ❌ no public image **exists**. Installable into a LinuxServer **webtop** container via its official installer (zrythm.org `install.sh` / apt repo). | ✅ son's main DAW |
| **LMMS** | FL-Studio-style open-source DAW (beats / MIDI / virtual instruments) | ✅ `apt install lmms` inside a webtop container (Ubuntu). | ✅ secondary / beats |
| **Spotify Basic Pitch** | Audio→**MIDI** neural net (polyphonic, pitch bends). `pip install basic-pitch`. Best on a single instrument. | Runs as a **Celery task** (no UI). | ✅ fast, lightweight first-pass MIDI |
| **MuScriptor** (Kyutai + Mirelo) | State-of-the-art **multi-instrument** transcription → **MIDI + MusicXML + engraved PDF + guitar tabs** in one run. `muscriptor transcribe audio.wav --format sheets`. | CLI / heavy model (1.4B). | ✅ the "sheet music" engine you described |
| **MuseScore** | Open-source notation/engraver (MuScriptor uses it internally). | `apt install musescore`, also headless CLI. | ✅ fallback: MIDI→MusicXML→PDF locally |

## 2. Key caveats to decide on

- **MuScriptor licensing:** code is **MIT**, but model weights are **CC BY-NC 4.0** (non-commercial). That's fine for personal/family use — but not for anything commercial.
- **MuScriptor resources:** 1.4B-parameter model. Feasible on CPU but **slow**; a GPU makes it practical. We have no GPU confirmed on `.13` → treat MuScriptor as best-effort/beta, queue it in Celery, or throttle.
- **Basic Pitch scale:** fast on CPU, but "best on one instrument at a time". Good default for quick single-part transcription.
- **Webtop note:** the LinuxServer webtop terminal grants root inside the container. Safe on the trusted LAN behind Caddy; don't expose it to the public internet.

## 3. Proposed architecture

```
Family-music user (son/family)
   │  console.lab.audasmedia.com.au  (portal)
   ├── "Music (LMMS)"   → lmms.lab.audasmedia.com.au   (webtop:8086? )  remote DAW
   ├── "Zrythm"         → zrythm.lab.audasmedia.com.au (webtop:8087? )  remote DAW
   └── "Transcriber"    → console form:  pick audio (Garage) / upload
                          │  enqueue Celery task
                          ▼
                    worker (existing family-home-lab-worker)
                       ├─ Basic Pitch  → MIDI     (fast, single-instrument)
                       └─ MuScriptor   → MIDI+MusicXML+PDF+tabs   (beta, slow)
                          ▼
                    writes results back to Garage bucket + shared-media
                          ▼
                    Transcriber page lists/plays/downloads results
```

- Input & output live in Garage (`sam`/…/`shared-media` buckets), backed by Borg.
- Uses the **existing** RabbitMQ+Celery worker — no new queue.
- DAWs are separate webtop containers on **free host ports** with their own Caddy rows.

## 4. Blocker found on the DAWs (ports)
Originlab snippet used **:3001** for zrythm — but that port is **taken by langfuse**. 
We'll use fresh free ports (e.g. **8085/8086/8087**) and correct Caddy/nearby URLs.

## 5. Suggested build order (each is independently useful)
1. **Transcriber (Basic Pitch)** — quick win: upload/pick audio → MIDI; show result in console. (Easy, CPU, no license.)
2. **LMMS remote DAW** — `apt install lmms` on a webtop container, free port + Caddy row.
3. **Zrythm remote DAW** — custom webtop image with Zrythm installed.
4. **MuScriptor sheet-music** — add the multi-instrument + MusicXML/PDF stage (needs HF login/license; beta if no GPU).

## 6. Open decisions (need your call)
- Which transcriber to prioritize: **Basic Pitch** (fast, MIDI) vs **MuScriptor** (full sheet music, heavier) — or pair them.
- LMMS and/or Zrythm both? (I suggest both — they cover different use cases.)
- Confirm we may use the webtop image which gives container-root to the terminal.