diff --git a/stacks/backup/.env.example b/stacks/backup/.env.example new file mode 100644 index 00000000..2e4e303f --- /dev/null +++ b/stacks/backup/.env.example @@ -0,0 +1,4 @@ +# Backup Stack +BACKUP_SOURCE=/opt/homelab +BACKUP_DEST_LOCAL=/opt/homelab-backups +DUPLICATI_PASSWORD= diff --git a/stacks/backup/README.md b/stacks/backup/README.md new file mode 100644 index 00000000..0de96903 --- /dev/null +++ b/stacks/backup/README.md @@ -0,0 +1,158 @@ +# Backup & DR Stack + +3-2-1 backup strategy for HomeLab Stack — encrypted cloud backup (Duplicati) + local backup repository (Restic) + automated scripts. + +## What's Included + +| Service | Version | URL | Purpose | +|---------|---------|-----|---------| +| Duplicati | v2.0.8.1 | `backup.` | Encrypted backup UI (cloud/S3/B2/SFTP) | +| Restic REST Server | 0.13.0 | `restic.` | Local restic backup repository | + +## 3-2-1 Backup Strategy + +``` +Data Sources + │ + ├──► [Duplicati] ──► Cloud (S3/B2/SFTP/R2) [Copy 1: remote] + │ Local /backups [Copy 2: local] + │ + └──► [scripts/backup.sh] ──► Docker volumes + PostgreSQL dumps + MariaDB dumps + Config tarballs [Copy 3: local] + │ + [Restic REST Server] ──► Deduplicated backup repo [Medium 2: separate volume] +``` + +## Quick Start + +```bash +cd stacks/base && docker compose up -d +cd ../backup +ln -sf ../../.env .env +docker compose up -d +``` + +## Configuration + +### Environment Variables + +| Variable | Required | Default | Description | +|----------|----------|---------|-------------| +| `DOMAIN` | Yes | — | Base domain | +| `BACKUP_SOURCE` | No | `/opt/homelab` | Path to backup (mounted read-only) | +| `BACKUP_DEST_LOCAL` | No | `/opt/homelab-backups` | Local backup destination | +| `DUPLICATI_PASSWORD` | No | — | Duplicati web UI password | +| `TZ` | No | `Asia/Shanghai` | Timezone | + +### Duplicati Setup + +1. Visit `https://backup.` +2. Add backup job: Settings → Add Backup +3. Select source: `/source` (maps to `BACKUP_SOURCE`) +4. Select destination: Choose one: + - **Local**: `/backups` + - **S3**: MinIO at `https://s3.` + - **B2**: Backblaze B2 (need keyID + appKey) + - **SFTP**: Remote server +5. Set encryption passphrase (important!) +6. Schedule: Daily at 2:00 AM +7. Retention: Keep 7 daily, 4 weekly, 12 monthly + +### Restic Setup + +Use with the `restic` CLI from any machine: + +```bash +export RESTIC_REPOSITORY=rest:https://restic.${DOMAIN}/myrepo +export RESTIC_PASSWORD=my-encryption-password + +# Initialize repo (first time only) +restic init + +# Backup +restic backup /path/to/data + +# Restore +restic restore latest --target /path/to/restore + +# List snapshots +restic snapshots +``` + +### Automated Backups with backup.sh + +The repo includes `scripts/backup.sh` for automated backups: + +```bash +# Full backup (volumes + databases + configs) +./scripts/backup.sh + +# Add to crontab for daily 2AM backup +echo '0 2 * * * /opt/homelab/scripts/backup.sh >> /var/log/homelab-backup.log 2>&1' | crontab - +``` + +### Backup Notifications + +Configure Duplicati to send notifications via ntfy: + +1. In Duplicati: Settings → Default options → Send notifications +2. Type: Custom HTTP URL +3. URL: `http://ntfy:80/homelab-backup` + +## Disaster Recovery + +### Full Restore Procedure (new host) + +```bash +# 1. Install Docker + Compose +# 2. Clone repo +git clone /opt/homelab + +# 3. Restore .env from backup +tar xzf /opt/homelab-backups//configs.tar.gz -C /opt/homelab + +# 4. Start base stack +cd /opt/homelab/stacks/base && docker compose up -d + +# 5. Start databases and restore data +cd ../databases && docker compose up -d +cat /postgresql_all.sql | docker exec -i homelab-postgres psql -U postgres + +# 6. Start remaining stacks in order +cd ../sso && docker compose up -d +cd ../storage && docker compose up -d +cd ../productivity && docker compose up -d +# ... etc +``` + +### Restore Order + +1. Base (Traefik) → 2. Databases → 3. SSO → 4. All others + +### Estimated RTO + +| Component | Time | +|-----------|------| +| Base + DB | ~5 min | +| SSO | ~3 min | +| Storage | ~5 min | +| Productivity | ~5 min | +| Other stacks | ~10 min | +| **Total** | **~30 min** | + +## Health Check + +```bash +docker compose ps --format "table {{.Name}}\t{{.Status}}" +``` + +## Troubleshooting + +| Problem | Solution | +|---------|----------| +| Duplicati can't access source | Check `BACKUP_SOURCE` mount is correct | +| Restic REST returns 401 | No auth by default; add `--htpasswd-file` for auth | +| Backup too large | Duplicati uses deduplication; first backup is largest | +| Restore fails | Check encryption passphrase matches | diff --git a/stacks/backup/docker-compose.yml b/stacks/backup/docker-compose.yml new file mode 100644 index 00000000..615e9b50 --- /dev/null +++ b/stacks/backup/docker-compose.yml @@ -0,0 +1,63 @@ +services: + duplicati: + image: linuxserver/duplicati:v2.0.8.1-2.0.8.1.2 + container_name: duplicati + restart: unless-stopped + networks: + - proxy + volumes: + - duplicati-config:/config + - ${BACKUP_SOURCE:-/opt/homelab}:/source:ro + - ${BACKUP_DEST_LOCAL:-/opt/homelab-backups}:/backups + environment: + - PUID=${PUID:-1000} + - PGID=${PGID:-1000} + - TZ=${TZ:-Asia/Shanghai} + - CLI_ARGS=--webservice-password=${DUPLICATI_PASSWORD:-} + labels: + - traefik.enable=true + - "traefik.http.routers.duplicati.rule=Host(`backup.${DOMAIN}`)" + - traefik.http.routers.duplicati.entrypoints=websecure + - traefik.http.routers.duplicati.tls=true + - traefik.http.routers.duplicati.tls.certresolver=letsencrypt + - traefik.http.services.duplicati.loadbalancer.server.port=8200 + - traefik.http.routers.duplicati.middlewares=authentik-forwardauth@docker + healthcheck: + test: [CMD-SHELL, "curl -sf http://localhost:8200/api/v1/serverstate || exit 1"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 30s + + rest-server: + image: restic/rest-server:0.13.0 + container_name: rest-server + restart: unless-stopped + networks: + - proxy + volumes: + - rest-server-data:/data + environment: + - OPTIONS=--prometheus + - TZ=${TZ:-Asia/Shanghai} + labels: + - traefik.enable=true + - "traefik.http.routers.rest-server.rule=Host(`restic.${DOMAIN}`)" + - traefik.http.routers.rest-server.entrypoints=websecure + - traefik.http.routers.rest-server.tls=true + - traefik.http.routers.rest-server.tls.certresolver=letsencrypt + - traefik.http.services.rest-server.loadbalancer.server.port=8000 + healthcheck: + test: [CMD-SHELL, "curl -sf http://localhost:8000/ || exit 1"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 15s + +networks: + proxy: + external: true + +volumes: + duplicati-config: + rest-server-data: