The in-memory metrics system covered one gunicorn process at a time, was lost on restart and was never used. Remove /admin/metrics-dashboard, /admin/metrics, starpunk/monitoring, the HTTP/query timing, memory monitor, feed counters, METRICS_* settings and psutil. Keep /health, /admin/health (without its metrics block), pool stats and the feed cache. ADR-072. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
267 lines
12 KiB
Markdown
267 lines
12 KiB
Markdown
# Deployment Architecture
|
||
|
||
**Version**: 2.10.1 (2026-11-03)
|
||
**Status**: Current
|
||
|
||
How StarPunk is built, configured and run in production. The production site
|
||
is a single container behind Caddy on one host, managed with Ansible. This
|
||
document stays generic: hostnames, credentials and inventory live in the
|
||
deployment repository, not here.
|
||
|
||
## Overview
|
||
|
||
```
|
||
git tag vX.Y.Z ──► Gitea Actions ──► test stage (pytest) ──► runtime image
|
||
│ push :vX.Y.Z, :latest
|
||
▼
|
||
Gitea container registry
|
||
│ pull (tag set in Ansible)
|
||
▼
|
||
Internet ──► Caddy (TLS, headers, X-Forwarded-For) ──► container :8000 (gunicorn × 4)
|
||
│
|
||
/data volume (host)
|
||
```
|
||
|
||
## Container image (`Containerfile`)
|
||
|
||
A multi-stage build on `python:3.14-slim`; it works with Podman and Docker.
|
||
|
||
| Stage | Purpose |
|
||
|-------|---------|
|
||
| `builder` | Copies `uv` from `ghcr.io/astral-sh/uv`, creates `/opt/venv` and installs `requirements.txt` into it |
|
||
| `test` | Copies the venv, installs `requirements-test.txt`, copies the source and runs `python -m pytest -q -p no:cacheprovider tests` with test values for `SESSION_SECRET`, `SITE_URL` and `ADMIN_ME`. The stage only builds if the tests pass. |
|
||
| runtime (last, the default target) | Creates user `starpunk` (UID/GID 1000) and `/data/notes`, copies the venv and the application to `/app`, sets `DATA_PATH=/data`, `NOTES_PATH=/data/notes`, `DATABASE_PATH=/data/starpunk.db`, runs as `starpunk`, exposes port 8000 |
|
||
|
||
The runtime image's command is gunicorn:
|
||
|
||
```
|
||
gunicorn --bind 0.0.0.0:8000 --workers 4 --worker-class sync
|
||
--worker-tmp-dir /dev/shm --max-requests 1000 --max-requests-jitter 50
|
||
--timeout 30 --graceful-timeout 30
|
||
--access-logfile - --error-logfile - --log-level info app:app
|
||
```
|
||
|
||
Each of the four worker processes builds its own Flask app, so each runs
|
||
migrations (only one applies them, see [database.md](database.md)) and starts
|
||
its own background threads (see [overview.md](overview.md#background-workers)).
|
||
Workers are recycled after about 1000 requests; a recycled process starts
|
||
new threads, and the file locks pass to another process if the holder exits.
|
||
|
||
**Health check**: the image's `HEALTHCHECK` runs every 30 seconds (3-second
|
||
timeout, 10-second start period, 3 retries) and calls
|
||
`http://localhost:8000/health` with httpx. `/health` returns
|
||
`{"status": "ok", "version": …}` plus webmention-worker statistics, without
|
||
authentication.
|
||
|
||
## CI/CD (`.gitea/workflows/build-container.yml`)
|
||
|
||
- **Triggers**: pushing a tag matching `vX.Y.Z`, or a manual run.
|
||
- **Runner**: a Gitea Actions runner with no Docker daemon. Images are built
|
||
by a rootless BuildKit daemon reached over its socket
|
||
(`docker buildx create --driver remote`).
|
||
- **Steps**:
|
||
1. Log in to the Gitea container registry with the repository secret
|
||
`REGISTRY_TOKEN` (a token with only the `write:package` scope; the
|
||
automatic Actions token is refused by the registry).
|
||
2. `docker buildx build --target test`: runs the test suite. A failure
|
||
stops the workflow before anything is pushed.
|
||
3. Build the runtime image and push it tagged with the git tag
|
||
(e.g. `v2.10.1`) and `latest`. The image name is the repository path in
|
||
lower case.
|
||
4. Remove the builder and log out, even after a failure.
|
||
|
||
There is no CI on branches or pull requests; run the tests locally before
|
||
merging (see [technology-stack.md](technology-stack.md#testing)).
|
||
|
||
## Release process
|
||
|
||
1. Finish the change on a branch, including the version bump in
|
||
`starpunk/__init__.py` (`__version__`, `__version_info__`) and a
|
||
`CHANGELOG.md` entry.
|
||
2. Merge to `main`.
|
||
3. Tag the merge commit `vX.Y.Z` and push the tag. CI tests, builds and
|
||
pushes the image.
|
||
4. In the Ansible-managed deployment, change the StarPunk image tag to the new
|
||
version and apply it. The container is recreated from the new image; new
|
||
migrations run automatically at start-up.
|
||
5. Check `/health` reports the new version.
|
||
|
||
Rolling back means deploying the previous tag. Migrations are not reversed
|
||
automatically, so a release with a migration needs a database backup first
|
||
if a rollback might be needed.
|
||
|
||
## Runtime layout
|
||
|
||
### Data volume (`/data`)
|
||
|
||
A host directory is mounted at `/data`, owned by UID 1000. Everything
|
||
persistent is there; the image itself holds no state.
|
||
|
||
| Path | Contents |
|
||
|------|----------|
|
||
| `starpunk.db` (+ `-wal`, `-shm`) | SQLite database in WAL mode |
|
||
| `notes/YYYY/MM/<slug>.md` | Note content |
|
||
| `.trash/YYYY/MM/` | Soft-deleted note files (restorable with Micropub undelete) |
|
||
| `media/YYYY/MM/` | Uploaded images and their variants; `media/.tmp/` while processing |
|
||
| `logs/starpunk.log` | Application log, rotated at 10 MB, 10 files kept |
|
||
| `webmention-worker.lock`, `websub-worker.lock` | `flock` files choosing the process that runs each worker |
|
||
| `threads_token.json` (+ `.lock`) | Current Threads access token, refreshed daily (only with Threads configured) |
|
||
| `debug/` | Failed uploads, only with `DEBUG_SAVE_FAILED_UPLOADS=true`; cleaned up after `DEBUG_FILE_MAX_AGE_DAYS` |
|
||
|
||
The application does not make backups. A backup must include the database
|
||
(copied with SQLite's online backup because of WAL), `notes/` and `media/`.
|
||
Backups are handled by the deployment, outside the container.
|
||
|
||
### Reverse proxy
|
||
|
||
Caddy terminates TLS and proxies to the container's port 8000. Publish that
|
||
port only where the proxy can reach it (for example on the host's loopback
|
||
interface, as `compose.yaml` does), never publicly. Caddy adds security headers
|
||
(HSTS, CSP, X-Frame-Options, Referrer-Policy and so on; see
|
||
`Caddyfile.example`, and `nginx.conf.example` for nginx) and sets
|
||
`X-Forwarded-For`.
|
||
|
||
Set `TRUSTED_PROXY=true` behind the proxy. With it, StarPunk takes the client
|
||
address from the first entry of `X-Forwarded-For`, which the webmention and
|
||
WebSub rate limits need; without it, every client appears as the proxy's
|
||
address and shares one limit. Do not set it if the container port can be
|
||
reached without the proxy, because the header could then be forged.
|
||
|
||
`SITE_URL` must be the public `https://` URL: it is used for absolute links,
|
||
feed and WebSub topic URLs, the IndieAuth issuer and redirect URIs, and to
|
||
decide whether the session cookie is `Secure`.
|
||
|
||
## Configuration
|
||
|
||
Configuration is read from environment variables (and a `.env` file if
|
||
present) by `starpunk/config.py` at start-up. In production the variables
|
||
come from the Ansible-managed environment, with secrets from its vault.
|
||
`.env.example` documents them for local use. Booleans are `true`/`false`
|
||
unless noted.
|
||
|
||
### Required
|
||
|
||
| Variable | Default | Notes |
|
||
|----------|---------|-------|
|
||
| `SESSION_SECRET` | none | Required always. Signs sessions; also Flask's `SECRET_KEY` unless `FLASK_SECRET_KEY` is set |
|
||
| `SITE_URL` | `http://localhost:5000` | Public URL; a trailing `/` is added |
|
||
| `ADMIN_ME` | none | Owner's identity URL (h-card, Micropub `me`). Required unless `DEV_MODE` |
|
||
| `OIDC_ISSUER`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`, `OIDC_ADMIN_SUB` | empty | Admin login via OIDC. Required unless `DEV_MODE` (or testing) |
|
||
|
||
### Site and authentication
|
||
|
||
| Variable | Default | Notes |
|
||
|----------|---------|-------|
|
||
| `SITE_NAME` | `StarPunk` | |
|
||
| `SITE_AUTHOR` | `Unknown` | |
|
||
| `SITE_DESCRIPTION` | `A minimal IndieWeb CMS` | |
|
||
| `SESSION_LIFETIME` | `30` | Days |
|
||
| `FLASK_SECRET_KEY` | `SESSION_SECRET` | |
|
||
| `OIDC_PROVIDER_NAME` | `single sign-on` | Label on the login button |
|
||
| `INDIEAUTH_TOKEN_LIFETIME_DAYS` | `90` | Lifetime of Micropub access tokens |
|
||
| `DEV_MODE` | `false` | Enables `/dev/login`; never in production |
|
||
| `DEV_ADMIN_ME` | empty | Required when `DEV_MODE=true` |
|
||
|
||
`TOKEN_ENDPOINT` and `INDIELOGIN_URL` are v1 settings; if present they are
|
||
ignored with a warning.
|
||
|
||
### Paths, runtime and logging
|
||
|
||
| Variable | Default | Notes |
|
||
|----------|---------|-------|
|
||
| `DATA_PATH` | `./data` (image: `/data`) | Root of all stored data |
|
||
| `DATABASE_PATH` | `./data/starpunk.db` (image: `/data/starpunk.db`) | |
|
||
| `NOTES_PATH` | `./data/notes` (image: `/data/notes`) | Created at start-up, but note files are always written to `DATA_PATH/notes` |
|
||
| `FLASK_ENV` | `production` | Reported in `/health` and admin diagnostics; the image sets `production` |
|
||
| `FLASK_DEBUG` | `0` | Debug off unless `1` (`flask run --debug` sets it locally); the image sets `0` |
|
||
| `LOG_LEVEL` | `INFO` | `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` |
|
||
| `VERSION` | package `__version__` | Overrides the reported version; normally unset |
|
||
|
||
### Feeds
|
||
|
||
| Variable | Default | Notes |
|
||
|----------|---------|-------|
|
||
| `FEED_MAX_ITEMS` | `50` | Notes per feed |
|
||
| `FEED_CACHE_SECONDS` | `300` | Cache TTL and `Cache-Control: max-age` |
|
||
| `FEED_CACHE_ENABLED` | `true` | |
|
||
| `FEED_CACHE_MAX_SIZE` | `50` | Cached feed documents per process |
|
||
|
||
### Webmention
|
||
|
||
| Variable | Default | Notes |
|
||
|----------|---------|-------|
|
||
| `WEBMENTION_ENABLED` | `true` | Endpoint, `Link` header and sending |
|
||
| `WEBMENTION_WORKER_ENABLED` | `true` | |
|
||
| `WEBMENTION_POLL_INTERVAL` | `10` | Seconds |
|
||
| `WEBMENTION_ERROR_BACKOFF` | `60` | Seconds after a worker error |
|
||
| `WEBMENTION_BATCH_SIZE` | `5` | Items per pass |
|
||
| `WEBMENTION_AUTO_APPROVE` | `true` | Valid webmentions are shown without moderation |
|
||
| `WEBMENTION_RATE_LIMIT_ENABLED` | `true` | |
|
||
| `WEBMENTION_RATE_LIMIT_IP` | `60` | Requests per window per client IP |
|
||
| `WEBMENTION_RATE_LIMIT_DOMAIN` | `10` | Requests per window per source domain |
|
||
| `WEBMENTION_RATE_LIMIT_WINDOW` | `60` | Seconds |
|
||
| `TRUSTED_PROXY` | `false` | Use `X-Forwarded-For` for the client IP (see above) |
|
||
|
||
### WebSub
|
||
|
||
| Variable | Default | Notes |
|
||
|----------|---------|-------|
|
||
| `WEBSUB_ENABLED` | `true` | Discovery links, hub and worker |
|
||
| `WEBSUB_HUB_URL` | empty | Use an external hub (StarPunk then sends publish pings) instead of the built-in one |
|
||
| `WEBSUB_MAX_SUBSCRIPTIONS` | `1000` | Total subscriptions |
|
||
| `WEBSUB_MAX_PER_HOST` | `25` | Subscriptions per callback host |
|
||
| `WEBSUB_POLL_INTERVAL` | `5` | Seconds |
|
||
|
||
### Crossposting (POSSE)
|
||
|
||
A target is enabled when its credentials are set; the syndication worker
|
||
starts only if at least one is.
|
||
|
||
| Variable | Default | Notes |
|
||
|----------|---------|-------|
|
||
| `GOTOSOCIAL_URL`, `GOTOSOCIAL_ACCESS_TOKEN` | empty | GoToSocial or any Mastodon-API server |
|
||
| `GOTOSOCIAL_NAME` | `GoToSocial` | Label in the UI and `syndicate-to` |
|
||
| `GOTOSOCIAL_VISIBILITY` | `public` | |
|
||
| `GOTOSOCIAL_MAX_CHARS` | `5000` | |
|
||
| `GOTOSOCIAL_MAX_MEDIA` | `4` | |
|
||
| `BLUESKY_HANDLE`, `BLUESKY_APP_PASSWORD` | empty | |
|
||
| `BLUESKY_SERVICE` | `https://bsky.social` | |
|
||
| `BLUESKY_NAME` | `Bluesky` | |
|
||
| `BLUESKY_MAX_CHARS` | `300` | |
|
||
| `THREADS_ACCESS_TOKEN` | empty | Long-lived token; refreshed automatically into `threads_token.json` |
|
||
| `THREADS_USERNAME` | empty | |
|
||
| `THREADS_NAME` | `Threads` | |
|
||
| `THREADS_MAX_CHARS` | `500` | |
|
||
| `THREADS_MAX_MEDIA` | `4` | |
|
||
| `SYNDICATION_POLL_INTERVAL` | `10` | Seconds |
|
||
|
||
### Debugging
|
||
|
||
The `METRICS_*` variables were removed in v2.12.0 and are ignored if set.
|
||
|
||
| Variable | Default | Notes |
|
||
|----------|---------|-------|
|
||
| `DEBUG_SAVE_FAILED_UPLOADS` | `false` | Keep failed uploads in `debug/` |
|
||
| `DEBUG_FILE_MAX_AGE_DAYS` | `7` | |
|
||
| `DEBUG_FILE_MAX_SIZE_MB` | `100` | |
|
||
|
||
Fixed values: the maximum request body is 50 MB (`MAX_CONTENT_LENGTH`). The
|
||
connection pool size and timeout (`DB_POOL_SIZE`, `DB_TIMEOUT`) are not read
|
||
from the environment.
|
||
|
||
## Running locally
|
||
|
||
```bash
|
||
uv venv && uv pip install -r requirements-dev.txt
|
||
cp .env.example .env # set SESSION_SECRET; DEV_MODE=true and DEV_ADMIN_ME for local login
|
||
flask --app app.py run # or: gunicorn app:app
|
||
```
|
||
|
||
`compose.yaml` builds and runs the image with `./container-data` mounted at
|
||
`/data`; its image tag and `VERSION` values are out of date, so override them
|
||
or build with `podman build -t starpunk .`.
|
||
|
||
---
|
||
|
||
**Last Updated**: 2026-11-03 (v2.10.1)
|