101 lines
3.3 KiB
Markdown
101 lines
3.3 KiB
Markdown
---
|
|
created: 2026-08-24 09:39
|
|
modified: 2026-08-24 09:39
|
|
type: note
|
|
tags:
|
|
- homelab
|
|
- storage
|
|
- s3
|
|
- garage
|
|
- family
|
|
aliases: []
|
|
---
|
|
# [[FAMILY S3 Storage Integration & Blueprint]]
|
|
|
|
> **FINAL PLAN (approved 2026-08 session).** Implementation tracked in `/home/sam/home_network/custom_tools/family_home_lab/plan.md`. Supersedes the generic blueprint below.
|
|
|
|
## Final architecture
|
|
|
|
| Item | Decision |
|
|
|---|---|
|
|
| Engine | Fresh **Garage v1.x** container inside the family-home-lab Docker Compose stack |
|
|
| Host | `.13` nixos-desktop (always-on server) |
|
|
| Compose location | `/home/sam/Docker/Containers/family-home-lab/` |
|
|
| Data location | `/mnt/data/family-home-lab/garage-data/` + `/mnt/data/family-home-lab/garage-meta/` — **new subdirs only**. The Takeout landing zone (`01_keep`, `02_review`, `03_delete`, `takeout`, `.thumbs`) is never touched or reformatted |
|
|
| Ports | `3900` (S3 API), `3902` (web) — old test instance `garage-garage-1` retired first to free these |
|
|
| Region | `homelab` |
|
|
| Buckets | `sam`, `jo`, `harry`, `finn`, `shared-media` |
|
|
| Access | Portal (FastAPI) uses boto3, S3v4 signatures, endpoint `.13:3900` |
|
|
| Backups | `/mnt/data/family-home-lab/` added to .13's existing Borg job → rsync→.35→.23 chain |
|
|
|
|
## Migration steps
|
|
|
|
1. Sanity-check old test instance data: `sudo du -sh /var/lib/docker/volumes/garage_garage_data/_data/*`
|
|
2. Stop & remove old `garage-garage-1` (compose at `/home/sam/deployment/garage/`)
|
|
3. Deploy new Garage in family-home-lab stack with data dirs on `/mnt/data`
|
|
4. Create buckets + per-user S3 access keys via `garage` admin CLI
|
|
5. Verify portal upload/download round-trip
|
|
6. Add path to Borg backup sources
|
|
|
|
## Consumers
|
|
|
|
- **Family console portal** (`console.home.lab`) — uploads/downloads per-user buckets
|
|
- Media tools (Photopea/video/audio containers) — bind-mount or rclone-mount bucket paths
|
|
- DeepSeek Harness instances ([[FAMILY DeepSeek Harness (dsh) Home Lab Setup]]) — optional workspace persistence in user buckets
|
|
|
|
---
|
|
|
|
## Original generic blueprint (reference only)
|
|
|
|
This document details the architectural expansion of the self-hosted AI and multimedia workspace. It introduces a pure open-source, local S3 object storage tier using **Garage** (Rust-based), replacing legacy volume mappings with decoupled web APIs to unify storage across network nodes.
|
|
|
|
### Garage engine config (final form)
|
|
|
|
```toml
|
|
metadata_dir = "/var/lib/garage/meta" # → mapped to /mnt/data/family-home-lab/garage-meta
|
|
data_dir = "/var/lib/garage/data" # → mapped to /mnt/data/family-home-lab/garage-data
|
|
db_engine = "sqlite"
|
|
replication_factor = 1
|
|
rpc_bind_addr = "[::]:3901"
|
|
|
|
[s3_api]
|
|
s3_region = "homelab"
|
|
api_bind_addr = "[::]:3900"
|
|
|
|
[s3_web]
|
|
bind_addr = "[::]:3902"
|
|
root_domain = "web.s3.home.lab"
|
|
```
|
|
|
|
### Caddy routing (.35)
|
|
|
|
```caddy
|
|
s3.home.lab {
|
|
reverse_proxy 192.168.20.13:3900
|
|
}
|
|
```
|
|
|
|
### Full original compose example
|
|
|
|
<details><summary>expand</summary>
|
|
|
|
```yaml
|
|
version: '3.8'
|
|
services:
|
|
studio-s3:
|
|
image: dxflrs/garage:v1.0.0
|
|
container_name: studio_garage_s3
|
|
restart: unless-stopped
|
|
ports:
|
|
- "3900:3900"
|
|
- "3901:3901"
|
|
volumes:
|
|
- ./garage/garage.toml:/etc/garage.toml
|
|
- /mnt/data/family-home-lab/garage-meta:/var/lib/garage/meta
|
|
- /mnt/data/family-home-lab/garage-data:/var/lib/garage/data
|
|
environment:
|
|
- GARAGE_RPC_SECRET=<generate-fresh>
|
|
```
|
|
|
|
</details>
|