Files
StarPunk/docs/architecture/deployment.md
T
PhilandClaude Opus 5.5 742e2e3ec3 Remove the metrics dashboard and monitoring package (v2.12.0)
The in-memory metrics system covered one gunicorn process at a time, was
lost on restart and was never used. Remove /admin/metrics-dashboard,
/admin/metrics, starpunk/monitoring, the HTTP/query timing, memory monitor,
feed counters, METRICS_* settings and psutil. Keep /health, /admin/health
(without its metrics block), pool stats and the feed cache. ADR-072.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 16:07:47 -07:00

267 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Deployment Architecture
**Version**: 2.10.1 (2026-11-03)
**Status**: Current
How StarPunk is built, configured and run in production. The production site
is a single container behind Caddy on one host, managed with Ansible. This
document stays generic: hostnames, credentials and inventory live in the
deployment repository, not here.
## Overview
```
git tag vX.Y.Z ──► Gitea Actions ──► test stage (pytest) ──► runtime image
│ push :vX.Y.Z, :latest
▼
Gitea container registry
│ pull (tag set in Ansible)
▼
Internet ──► Caddy (TLS, headers, X-Forwarded-For) ──► container :8000 (gunicorn × 4)
│
/data volume (host)
```
## Container image (`Containerfile`)
A multi-stage build on `python:3.14-slim`; it works with Podman and Docker.
| Stage | Purpose |
|-------|---------|
| `builder` | Copies `uv` from `ghcr.io/astral-sh/uv`, creates `/opt/venv` and installs `requirements.txt` into it |
| `test` | Copies the venv, installs `requirements-test.txt`, copies the source and runs `python -m pytest -q -p no:cacheprovider tests` with test values for `SESSION_SECRET`, `SITE_URL` and `ADMIN_ME`. The stage only builds if the tests pass. |
| runtime (last, the default target) | Creates user `starpunk` (UID/GID 1000) and `/data/notes`, copies the venv and the application to `/app`, sets `DATA_PATH=/data`, `NOTES_PATH=/data/notes`, `DATABASE_PATH=/data/starpunk.db`, runs as `starpunk`, exposes port 8000 |
The runtime image's command is gunicorn:
```
gunicorn --bind 0.0.0.0:8000 --workers 4 --worker-class sync
--worker-tmp-dir /dev/shm --max-requests 1000 --max-requests-jitter 50
--timeout 30 --graceful-timeout 30
--access-logfile - --error-logfile - --log-level info app:app
```
Each of the four worker processes builds its own Flask app, so each runs
migrations (only one applies them, see [database.md](database.md)) and starts
its own background threads (see [overview.md](overview.md#background-workers)).
Workers are recycled after about 1000 requests; a recycled process starts
new threads, and the file locks pass to another process if the holder exits.
**Health check**: the image's `HEALTHCHECK` runs every 30 seconds (3-second
timeout, 10-second start period, 3 retries) and calls
`http://localhost:8000/health` with httpx. `/health` returns
`{"status": "ok", "version": …}` plus webmention-worker statistics, without
authentication.
## CI/CD (`.gitea/workflows/build-container.yml`)
- **Triggers**: pushing a tag matching `vX.Y.Z`, or a manual run.
- **Runner**: a Gitea Actions runner with no Docker daemon. Images are built
by a rootless BuildKit daemon reached over its socket
(`docker buildx create --driver remote`).
- **Steps**:
1. Log in to the Gitea container registry with the repository secret
`REGISTRY_TOKEN` (a token with only the `write:package` scope; the
automatic Actions token is refused by the registry).
2. `docker buildx build --target test`: runs the test suite. A failure
stops the workflow before anything is pushed.
3. Build the runtime image and push it tagged with the git tag
(e.g. `v2.10.1`) and `latest`. The image name is the repository path in
lower case.
4. Remove the builder and log out, even after a failure.
There is no CI on branches or pull requests; run the tests locally before
merging (see [technology-stack.md](technology-stack.md#testing)).
## Release process
1. Finish the change on a branch, including the version bump in
`starpunk/__init__.py` (`__version__`, `__version_info__`) and a
`CHANGELOG.md` entry.
2. Merge to `main`.
3. Tag the merge commit `vX.Y.Z` and push the tag. CI tests, builds and
pushes the image.
4. In the Ansible-managed deployment, change the StarPunk image tag to the new
version and apply it. The container is recreated from the new image; new
migrations run automatically at start-up.
5. Check `/health` reports the new version.
Rolling back means deploying the previous tag. Migrations are not reversed
automatically, so a release with a migration needs a database backup first
if a rollback might be needed.
## Runtime layout
### Data volume (`/data`)
A host directory is mounted at `/data`, owned by UID 1000. Everything
persistent is there; the image itself holds no state.
| Path | Contents |
|------|----------|
| `starpunk.db` (+ `-wal`, `-shm`) | SQLite database in WAL mode |
| `notes/YYYY/MM/<slug>.md` | Note content |
| `.trash/YYYY/MM/` | Soft-deleted note files (restorable with Micropub undelete) |
| `media/YYYY/MM/` | Uploaded images and their variants; `media/.tmp/` while processing |
| `logs/starpunk.log` | Application log, rotated at 10 MB, 10 files kept |
| `webmention-worker.lock`, `websub-worker.lock` | `flock` files choosing the process that runs each worker |
| `threads_token.json` (+ `.lock`) | Current Threads access token, refreshed daily (only with Threads configured) |
| `debug/` | Failed uploads, only with `DEBUG_SAVE_FAILED_UPLOADS=true`; cleaned up after `DEBUG_FILE_MAX_AGE_DAYS` |
The application does not make backups. A backup must include the database
(copied with SQLite's online backup because of WAL), `notes/` and `media/`.
Backups are handled by the deployment, outside the container.
### Reverse proxy
Caddy terminates TLS and proxies to the container's port 8000. Publish that
port only where the proxy can reach it (for example on the host's loopback
interface, as `compose.yaml` does), never publicly. Caddy adds security headers
(HSTS, CSP, X-Frame-Options, Referrer-Policy and so on; see
`Caddyfile.example`, and `nginx.conf.example` for nginx) and sets
`X-Forwarded-For`.
Set `TRUSTED_PROXY=true` behind the proxy. With it, StarPunk takes the client
address from the first entry of `X-Forwarded-For`, which the webmention and
WebSub rate limits need; without it, every client appears as the proxy's
address and shares one limit. Do not set it if the container port can be
reached without the proxy, because the header could then be forged.
`SITE_URL` must be the public `https://` URL: it is used for absolute links,
feed and WebSub topic URLs, the IndieAuth issuer and redirect URIs, and to
decide whether the session cookie is `Secure`.
## Configuration
Configuration is read from environment variables (and a `.env` file if
present) by `starpunk/config.py` at start-up. In production the variables
come from the Ansible-managed environment, with secrets from its vault.
`.env.example` documents them for local use. Booleans are `true`/`false`
unless noted.
### Required
| Variable | Default | Notes |
|----------|---------|-------|
| `SESSION_SECRET` | none | Required always. Signs sessions; also Flask's `SECRET_KEY` unless `FLASK_SECRET_KEY` is set |
| `SITE_URL` | `http://localhost:5000` | Public URL; a trailing `/` is added |
| `ADMIN_ME` | none | Owner's identity URL (h-card, Micropub `me`). Required unless `DEV_MODE` |
| `OIDC_ISSUER`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`, `OIDC_ADMIN_SUB` | empty | Admin login via OIDC. Required unless `DEV_MODE` (or testing) |
### Site and authentication
| Variable | Default | Notes |
|----------|---------|-------|
| `SITE_NAME` | `StarPunk` | |
| `SITE_AUTHOR` | `Unknown` | |
| `SITE_DESCRIPTION` | `A minimal IndieWeb CMS` | |
| `SESSION_LIFETIME` | `30` | Days |
| `FLASK_SECRET_KEY` | `SESSION_SECRET` | |
| `OIDC_PROVIDER_NAME` | `single sign-on` | Label on the login button |
| `INDIEAUTH_TOKEN_LIFETIME_DAYS` | `90` | Lifetime of Micropub access tokens |
| `DEV_MODE` | `false` | Enables `/dev/login`; never in production |
| `DEV_ADMIN_ME` | empty | Required when `DEV_MODE=true` |
`TOKEN_ENDPOINT` and `INDIELOGIN_URL` are v1 settings; if present they are
ignored with a warning.
### Paths, runtime and logging
| Variable | Default | Notes |
|----------|---------|-------|
| `DATA_PATH` | `./data` (image: `/data`) | Root of all stored data |
| `DATABASE_PATH` | `./data/starpunk.db` (image: `/data/starpunk.db`) | |
| `NOTES_PATH` | `./data/notes` (image: `/data/notes`) | Created at start-up, but note files are always written to `DATA_PATH/notes` |
| `FLASK_ENV` | `production` | Reported in `/health` and admin diagnostics; the image sets `production` |
| `FLASK_DEBUG` | `0` | Debug off unless `1` (`flask run --debug` sets it locally); the image sets `0` |
| `LOG_LEVEL` | `INFO` | `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` |
| `VERSION` | package `__version__` | Overrides the reported version; normally unset |
### Feeds
| Variable | Default | Notes |
|----------|---------|-------|
| `FEED_MAX_ITEMS` | `50` | Notes per feed |
| `FEED_CACHE_SECONDS` | `300` | Cache TTL and `Cache-Control: max-age` |
| `FEED_CACHE_ENABLED` | `true` | |
| `FEED_CACHE_MAX_SIZE` | `50` | Cached feed documents per process |
### Webmention
| Variable | Default | Notes |
|----------|---------|-------|
| `WEBMENTION_ENABLED` | `true` | Endpoint, `Link` header and sending |
| `WEBMENTION_WORKER_ENABLED` | `true` | |
| `WEBMENTION_POLL_INTERVAL` | `10` | Seconds |
| `WEBMENTION_ERROR_BACKOFF` | `60` | Seconds after a worker error |
| `WEBMENTION_BATCH_SIZE` | `5` | Items per pass |
| `WEBMENTION_AUTO_APPROVE` | `true` | Valid webmentions are shown without moderation |
| `WEBMENTION_RATE_LIMIT_ENABLED` | `true` | |
| `WEBMENTION_RATE_LIMIT_IP` | `60` | Requests per window per client IP |
| `WEBMENTION_RATE_LIMIT_DOMAIN` | `10` | Requests per window per source domain |
| `WEBMENTION_RATE_LIMIT_WINDOW` | `60` | Seconds |
| `TRUSTED_PROXY` | `false` | Use `X-Forwarded-For` for the client IP (see above) |
### WebSub
| Variable | Default | Notes |
|----------|---------|-------|
| `WEBSUB_ENABLED` | `true` | Discovery links, hub and worker |
| `WEBSUB_HUB_URL` | empty | Use an external hub (StarPunk then sends publish pings) instead of the built-in one |
| `WEBSUB_MAX_SUBSCRIPTIONS` | `1000` | Total subscriptions |
| `WEBSUB_MAX_PER_HOST` | `25` | Subscriptions per callback host |
| `WEBSUB_POLL_INTERVAL` | `5` | Seconds |
### Crossposting (POSSE)
A target is enabled when its credentials are set; the syndication worker
starts only if at least one is.
| Variable | Default | Notes |
|----------|---------|-------|
| `GOTOSOCIAL_URL`, `GOTOSOCIAL_ACCESS_TOKEN` | empty | GoToSocial or any Mastodon-API server |
| `GOTOSOCIAL_NAME` | `GoToSocial` | Label in the UI and `syndicate-to` |
| `GOTOSOCIAL_VISIBILITY` | `public` | |
| `GOTOSOCIAL_MAX_CHARS` | `5000` | |
| `GOTOSOCIAL_MAX_MEDIA` | `4` | |
| `BLUESKY_HANDLE`, `BLUESKY_APP_PASSWORD` | empty | |
| `BLUESKY_SERVICE` | `https://bsky.social` | |
| `BLUESKY_NAME` | `Bluesky` | |
| `BLUESKY_MAX_CHARS` | `300` | |
| `THREADS_ACCESS_TOKEN` | empty | Long-lived token; refreshed automatically into `threads_token.json` |
| `THREADS_USERNAME` | empty | |
| `THREADS_NAME` | `Threads` | |
| `THREADS_MAX_CHARS` | `500` | |
| `THREADS_MAX_MEDIA` | `4` | |
| `SYNDICATION_POLL_INTERVAL` | `10` | Seconds |
### Debugging
The `METRICS_*` variables were removed in v2.12.0 and are ignored if set.
| Variable | Default | Notes |
|----------|---------|-------|
| `DEBUG_SAVE_FAILED_UPLOADS` | `false` | Keep failed uploads in `debug/` |
| `DEBUG_FILE_MAX_AGE_DAYS` | `7` | |
| `DEBUG_FILE_MAX_SIZE_MB` | `100` | |
Fixed values: the maximum request body is 50 MB (`MAX_CONTENT_LENGTH`). The
connection pool size and timeout (`DB_POOL_SIZE`, `DB_TIMEOUT`) are not read
from the environment.
## Running locally
```bash
uv venv && uv pip install -r requirements-dev.txt
cp .env.example .env # set SESSION_SECRET; DEV_MODE=true and DEV_ADMIN_ME for local login
flask --app app.py run # or: gunicorn app:app
```
`compose.yaml` builds and runs the image with `./container-data` mounted at
`/data`; its image tag and `VERSION` values are out of date, so override them
or build with `podman build -t starpunk .`.
---
**Last Updated**: 2026-11-03 (v2.10.1)