Files
ansible/roles/docker/README.md
T
PhilandClaude Opus 5.5 558b6d1a43 HeyForm: MongoDB 4.4 -> 7.0, back on the real data
4.4 is end-of-life. Moved by mongodump/mongorestore into a new volume
(heyform_database7); the 4.4 volume stays for rollback.

The restore also fixes a data mix-up. From Nov 2024 the compose file set
MONGO_URI='mongodb://mongo:27017/heyform' in list form, so the quotes
became part of the value and HeyForm used a database literally named
"heyform'". The v3 upgrade (a1a79af) dropped the quotes, which switched
HeyForm to the old "heyform" database from the first week of setup, so
the real Repairs and Volunteer forms (91 submissions) disappeared. The new
volume holds "heyform'" restored as "heyform". Tested first with v3 on a
throwaway Mongo 7.0: both forms load.

Backups: ~/backups/heyform-mongo4.4-20261109 on the server (dump of both
databases + tar of the 4.4 volume).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 16:18:33 -07:00

144 lines
8.3 KiB
Markdown

# Docker Role
Installs Docker and deploys all 24 services. Each service is a Docker Compose
stack in `/opt/stacks/<service>/` (Dockge lives in `/opt/dockge/`), built from
Jinja2 templates in this role.
## What runs before the services
`tasks/main.yml` does the system setup, then imports each category:
1. Base packages (`python3-pip`, `virtualenv`, `duf`, …).
2. Shell setup (`shell.yml`): zsh as the login shell, Atuin (installed and
logged in once, dotfile sync on), and `bat`.
3. Docker CE from Docker's apt repository.
4. `ufw-docker.sh` copied to `/usr/local/bin/` and its rules installed (UFW is
restarted once after install).
5. The shared `lava` network (see below).
6. `/opt/stacks` and `/opt/dockge`.
## Services by category
Tags: each category file has a category tag, each service its own tag.
### Infrastructure (`infrastructure`)
| Tag | Service | Notes |
|---|---|---|
| `caddy` | Caddy | Fixed IP 172.20.0.5. Ports 80/443 (+443/udp). `Caddyfile.j2` changes restart it via a handler; the stack itself is deployed with `recreate: never`. Static sites are served from `/opt/stacks/caddy/site/`. |
| `pocketid` | Pocket ID v2 | Passkey OIDC provider at `id.`. OIDC clients are created in its web UI; their IDs and secrets go in the vault. |
| `dockge` | Dockge | Compose UI on port 5001, managing `/opt/stacks`. |
### Development (`development`)
| Tag | Service | Notes |
|---|---|---|
| `gitea` | Gitea (`latest`) + runner (`gitea/runner`) + BuildKit | Fixed IP 172.20.0.6 so the ufw-docker rule for SSH (host port 222) stays valid. Pocket ID login is set up in Gitea's admin UI; OpenID 2.0 login is off. Jobs run inside the runner container (label `docker:host`), which has the Caddy site folders at `/sites` but no Docker socket. Image builds use the rootless `buildkitd` container through `unix:///run/buildkit/buildkit/buildkitd.sock` (`docker buildx create --driver remote`). |
| `bytestash` | ByteStash | Pocket ID through env vars. |
### Media (`media`)
| Tag | Service | Notes |
|---|---|---|
| `audiobookshelf` | Audiobookshelf | Pocket ID set up in the app's settings. Podcasts are not used. |
| `kavita` | Kavita | Ebooks at `books.`. Also handles OPDS and KOReader sync (replaced Calibre-Web, dir2opds and kosync). Pocket ID set up in the app's settings. |
| `karakeep`, `hoarder` | Karakeep | Files are still named `hoarder*`. Web, headless Chrome and Meilisearch. Pocket ID through `hoarder-env.j2`. |
| `manyfold` | Manyfold | Pocket ID through env vars. |
### Productivity (`productivity`)
| Tag | Service | Notes |
|---|---|---|
| `paperlessngx` | Paperless-ngx | Image tag from `paperless_version` in `group_vars/docker/services.yml` (`latest`, auto-updated). Redis 8, Gotenberg `:8` (9 breaks the API), Tika pinned to 3.x by hand (4 is incompatible). Pocket ID through django-allauth (`paperlessngx.env.j2`). Rejects duplicate documents. Daily S3 backup of the data and media volumes. |
| `baikal` | Baikal | CalDAV/CardDAV. Caddy redirects the `.well-known` paths. |
| `syncthing` | Syncthing | Ports on the Tailscale IP only. |
| `heyform` | HeyForm v3 | `latest` (stable releases only), auto-updated. MongoDB 7.0 (Percona) and Redis 8 (v3 needs it). Port 9157. Public sign-up is off; people join through workspace invitations. On `forms.nerder.land`. |
| `dawarich` | Dawarich | App, Sidekiq, PostGIS, Redis. Based on the upstream production compose file. Image tag from `dawarich_version` (pinned during the 1.7 → 1.22 step-by-step upgrade). |
| `obsidian-livesync` | CouchDB for Obsidian LiveSync | Port 5984 on the Tailscale IP only. Has a restart handler. |
| `gokapi` | Gokapi | Pocket ID login. `config.json` is only written on a fresh install, because Gokapi rewrites it on upgrade. To change it, delete `/opt/stacks/gokapi/config/config.json` and redeploy. |
### Communication (`communication`)
| Tag | Service | Notes |
|---|---|---|
| `gotosocial` | GoToSocial | Pocket ID through env vars. Account domain is the apex; Caddy serves WebFinger there. Daily S3 backup of its volume. |
| `listmonk` | Listmonk | Postgres 17. On `listmonk.nerder.land`. |
| `starpunk` | StarPunk | Micropub server for the main site. Image built by the StarPunk repo's Gitea Actions on `vX.Y.Z` tags; deploy a release by bumping `starpunk_version`. Admin login via Pocket ID; issues its own Micropub tokens. Crossposts new notes to GoToSocial (token in `vault_starpunk.gotosocial_token`) Bluesky as phil.thesatelliteoflove.com (app password in `vault_starpunk.bluesky_app_password`) and Threads as @philskents (Meta app "starpunk"; long-lived token in `vault_starpunk.threads_access_token`, refreshed daily by StarPunk itself). |
### Monitoring (`monitoring`)
| Tag | Service | Notes |
|---|---|---|
| `changedetection` | Changedetection | With a sockpuppetbrowser for JavaScript pages. |
| `appriseapi` | Apprise API | Port 8000 on the Tailscale IP only. Other containers reach it at `http://apprise:8000`. |
| `gotify` | Gotify + iGotify assistant | Push notifications. Dockcheck sends its update notices here. |
| `cronmaster` | Cronmaster | Cron UI, port 40123 on the Tailscale IP only. |
## Networking
### Shared network (`lava`)
Every stack joins the external network `lava` (`172.20.0.0/16`). Docker only
hands out dynamic addresses from `172.20.128.0/17`, so `172.20.0.x` is free
for fixed IPs: Caddy (.5) and Gitea (.6). Both need fixed IPs because the
ufw-docker rules point at container IPs. The role creates the network if it
is missing and warns if it lacks the reserved range; `migrate-lava-network.yml`
fixes an old network.
### Hairpinning
Containers can't reach the server's own public IP, so any app that calls
another of our domains (usually Pocket ID for OIDC) maps it to Caddy:
```yaml
extra_hosts:
- "{{ subdomains.pocketid }}:{{ docker.hairpin_ip }}"
```
### Firewall
`ufw-docker.sh` makes UFW apply to published container ports. Only Caddy
(80/tcp, 443/tcp, 443/udp) and Gitea (22/tcp, published as 222) are allowed
in from the internet; the Caddy and Gitea task files add those rules.
## Updates
Containers labelled `mag37.dockcheck.update: true` are updated automatically
each morning by dockcheck (set up in the cron role). Containers without the
label (StarPunk, Listmonk, HeyForm, Dockge and most databases) only
get a Gotify notice. Dawarich's database and Redis are skipped entirely.
StarPunk uses a pinned version tag.
## Adding a service
1. `templates/<service>-compose.yml.j2` (plus `-env.j2` if needed). Join the
external `{{ docker.network_name }}` network and add `extra_hosts` for any
of our domains the app calls.
2. `tasks/<category>/<service>.yml`: make `/opt/stacks/<service>`, template
the files, deploy with `community.docker.docker_compose_v2`. Use
`diff: false` or `no_log: true` on tasks that write secrets.
3. Import it in `tasks/<category>/main.yml` with a tag named after the service.
4. Subdomain in `group_vars/all/domains.yml`, route in `templates/Caddyfile.j2`,
A record in `dns.yml`.
5. Secrets under `vault_<service>` with `bin/vault-run`.
## Files
```
roles/docker/
├── tasks/
│ ├── main.yml # System setup, network, category imports
│ ├── shell.yml # zsh, Atuin, bat
│ ├── infrastructure/ # caddy, pocketid (+ dockge in main.yml)
│ ├── development/ # gitea, bytestash
│ ├── media/ # audiobookshelf, kavita, hoarder (Karakeep), manyfold
│ ├── productivity/ # paperlessngx, baikal, syncthing, heyform, dawarich, obsidian-livesync, gokapi
│ ├── communication/ # gotosocial, listmonk, starpunk
│ └── monitoring/ # changedetection, appriseapi, gotify, cronmaster
├── templates/ # <service>-compose.yml.j2, env and config templates, Caddyfile.j2
├── files/ufw-docker.sh # UFW integration for Docker
└── handlers/main.yml # restart caddy, restart obsidian-livesync
```
## Troubleshooting
- **OIDC login fails with a connection error:** the container is missing the
`extra_hosts` entry for `id.thesatelliteoflove.com`.
- **OIDC discovery fails:** check the issuer URL has no trailing slash.
- **Caddy fails with "Address already in use":** another container took
172.20.0.5; the network lacks the reserved range. Run
`migrate-lava-network.yml`.
- **Gitea SSH stops working after a recreate:** Gitea lost its fixed IP;
redeploy with `--tags gitea` to restore it and the firewall rule.