Compare commits
111
Commits
v1.4.2-rc.1
...
v2.13.4
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d312fa0074 | ||
|
|
647f6c723b | ||
|
|
81c730813c | ||
|
|
113f28aff1 | ||
|
|
76b81d6ad9 | ||
|
|
44f77a3adf | ||
|
|
b247de011d | ||
|
|
65fc246d55 | ||
|
|
f3ed1b3488 | ||
|
|
d400e4d838 | ||
|
|
55ce6a1fdd | ||
|
|
6aaac982d5 | ||
|
|
2654692d15 | ||
|
|
f8a429cdc1 | ||
|
|
afef1fc9a5 | ||
|
|
b450c0c648 | ||
|
|
13936749de | ||
|
|
742e2e3ec3 | ||
|
|
d6a916ba7d | ||
|
|
e654dd84de | ||
|
|
abb75f080e | ||
|
|
d5157a6059 | ||
|
|
393228002b | ||
|
|
874465e103 | ||
|
|
3f5e7dc735 | ||
|
|
75946be7b2 | ||
|
|
9f17e145ff | ||
|
|
d73dd0a5b0 | ||
|
|
f85be50072 | ||
|
|
0716984958 | ||
|
|
fa551de197 | ||
|
|
b2043286fa | ||
|
|
409b8b31b4 | ||
|
|
b520b8b4d3 | ||
|
|
720a1e6d6f | ||
|
|
c89c2cd5ca | ||
|
|
ab6ff60f5c | ||
|
|
57682165a4 | ||
|
|
8c51b52ed6 | ||
|
|
42c77e34b7 | ||
|
|
438465592c | ||
|
|
c8a7ef309f | ||
|
|
0640730416 | ||
|
|
51f711ea4a | ||
|
|
6a76ee26af | ||
|
|
43ec7cb8be | ||
|
|
4aaa7a8122 | ||
|
|
0cfc9965df | ||
|
|
106495d114 | ||
|
|
3c1e109358 | ||
|
|
d43361d505 | ||
|
|
f924413934 | ||
|
|
4b2ae4c7fc | ||
|
|
d3148063e3 | ||
|
|
39c8259f3f | ||
|
|
1ce88b38a5 | ||
|
|
4ffe36f9d3 | ||
|
|
a551770c76 | ||
|
|
d0bfc80c2e | ||
|
|
d16463ef8a | ||
|
|
4b56f7cd4b | ||
|
|
2474628646 | ||
|
|
0fe341161e | ||
|
|
a62077ab30 | ||
|
|
fade75fd2d | ||
|
|
0dd59fbc18 | ||
|
|
11e09a0d02 | ||
|
|
565adaeecf | ||
|
|
53d10ed271 | ||
|
|
e8c1b03980 | ||
|
|
0d3a5cd519 | ||
|
|
88088e10e1 | ||
|
|
391bdb9db3 | ||
|
|
c51cf726b0 | ||
|
|
2616ef5060 | ||
|
|
0ae16bd1fc | ||
|
|
cc09098741 | ||
|
|
16925663a3 | ||
|
|
e92c27a84c | ||
|
|
ed63e70c46 | ||
|
|
57b02ad6c7 | ||
|
|
94e8c672df | ||
|
|
011f7cc87d | ||
|
|
f46130569f | ||
|
|
b4e6cf3cc3 | ||
|
|
2d0568d9c2 | ||
|
|
f117913797 | ||
|
|
b84cf23e1a | ||
|
|
d3fa5b7a2a | ||
|
|
06a8aabc01 | ||
|
|
4ee2c189ae | ||
|
|
c94cb377d3 | ||
|
|
2bd971f3d6 | ||
|
|
84e693fe57 | ||
|
|
975046abc7 | ||
|
|
21fa7acfbb | ||
|
|
b689e02e64 | ||
|
|
1b45a64920 | ||
|
|
3f1f82a749 | ||
|
|
92e7bdd342 | ||
|
|
0acefa4670 | ||
|
|
9dcc5c5710 | ||
|
|
7be2fb0f62 | ||
|
|
730eb8d58c | ||
|
|
6682339a86 | ||
|
|
d416463242 | ||
|
|
25b8cbd79d | ||
|
|
042505d5a6 | ||
|
|
72f3d4a55e | ||
|
|
e8ff0a0dcb | ||
|
|
9bc6780a8e |
No files matched your search
+186
-70
@@ -1,102 +1,218 @@
|
||||
# StarPunk Configuration Template
|
||||
# Copy this file to .env and fill in your values
|
||||
# DO NOT commit .env to version control
|
||||
# StarPunk configuration template (v2.10.1)
|
||||
#
|
||||
# Copy to .env and fill in your values. DO NOT commit .env.
|
||||
# Every variable here is read by starpunk/config.py. Uncommented lines are
|
||||
# the ones you normally set; commented lines show the built-in default and
|
||||
# only need uncommenting to change it. Real environment variables take
|
||||
# precedence over .env.
|
||||
|
||||
# =============================================================================
|
||||
# SITE CONFIGURATION
|
||||
# SITE
|
||||
# =============================================================================
|
||||
|
||||
# Public URL where your site is hosted (no trailing slash)
|
||||
# Public URL of the site (a trailing slash is added if missing)
|
||||
# (default: http://localhost:5000)
|
||||
SITE_URL=http://localhost:5000
|
||||
|
||||
# Your site name (appears in RSS feed and page titles)
|
||||
# Site name: page titles and feeds (default: StarPunk)
|
||||
SITE_NAME=My StarPunk Site
|
||||
|
||||
# Your name (appears as author in RSS feed)
|
||||
# Author name; must be non-empty but is not currently displayed: pages use
|
||||
# the h-card discovered at ADMIN_ME (default: Unknown)
|
||||
SITE_AUTHOR=Your Name
|
||||
|
||||
# Site description (appears in RSS feed)
|
||||
# Feed description (default: A minimal IndieWeb CMS)
|
||||
SITE_DESCRIPTION=My personal IndieWeb site
|
||||
|
||||
# =============================================================================
|
||||
# AUTHENTICATION
|
||||
# =============================================================================
|
||||
|
||||
# Your IndieWeb identity URL (REQUIRED)
|
||||
# This is YOUR personal website URL that you authenticate with
|
||||
# Example: https://yourname.com or https://github.com/yourname
|
||||
# Your identity URL (REQUIRED unless DEV_MODE=true). Used as the author h-card
|
||||
# source and as the "me" of Micropub tokens. It should link to this site's
|
||||
# IndieAuth metadata so Micropub apps can find StarPunk (ADR-064).
|
||||
ADMIN_ME=https://your-website.com
|
||||
|
||||
# Session secret key (REQUIRED - GENERATE A RANDOM VALUE)
|
||||
# Session secret (REQUIRED: the app refuses to start without it)
|
||||
# Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
|
||||
SESSION_SECRET=REPLACE_WITH_RANDOM_SECRET
|
||||
|
||||
# Session lifetime in days (default: 30)
|
||||
SESSION_LIFETIME=30
|
||||
# Flask secret key (default: SESSION_SECRET)
|
||||
# FLASK_SECRET_KEY=
|
||||
|
||||
# IndieLogin service URL (usually don't change this)
|
||||
INDIELOGIN_URL=https://indielogin.com
|
||||
# Admin session lifetime in days (default: 30)
|
||||
# SESSION_LIFETIME=30
|
||||
|
||||
# Admin login via OpenID Connect (all four REQUIRED unless DEV_MODE=true)
|
||||
# Register StarPunk as a confidential client with your provider (e.g. Pocket ID)
|
||||
# using the callback URL: <SITE_URL>/auth/callback
|
||||
OIDC_ISSUER=https://id.example.com
|
||||
OIDC_CLIENT_ID=
|
||||
OIDC_CLIENT_SECRET=
|
||||
# The provider's subject ("sub") of the one account allowed to log in.
|
||||
# Pocket ID: the user's ID shown in the admin user list.
|
||||
OIDC_ADMIN_SUB=
|
||||
# Name shown on the login button (default: single sign-on)
|
||||
OIDC_PROVIDER_NAME=Pocket ID
|
||||
|
||||
# Lifetime of the Micropub access tokens StarPunk's IndieAuth server issues,
|
||||
# in days (default: 90)
|
||||
# INDIEAUTH_TOKEN_LIFETIME_DAYS=90
|
||||
|
||||
# =============================================================================
|
||||
# DATA STORAGE
|
||||
# DATA STORAGE (the container image sets /data, /data/notes, /data/starpunk.db)
|
||||
# =============================================================================
|
||||
|
||||
# Base data directory (relative to project root)
|
||||
# Base data directory: notes (DATA_PATH/notes), media, logs,
|
||||
# threads_token.json (default: ./data)
|
||||
DATA_PATH=./data
|
||||
|
||||
# Notes directory (where markdown files are stored)
|
||||
# Created at startup; note files are always written under DATA_PATH/notes,
|
||||
# so keep this equal to that (default: ./data/notes)
|
||||
NOTES_PATH=./data/notes
|
||||
|
||||
# SQLite database path
|
||||
# SQLite database (default: ./data/starpunk.db)
|
||||
DATABASE_PATH=./data/starpunk.db
|
||||
|
||||
# =============================================================================
|
||||
# FLASK CONFIGURATION
|
||||
# FLASK / LOGGING
|
||||
# =============================================================================
|
||||
|
||||
# Environment: development or production
|
||||
FLASK_ENV=development
|
||||
|
||||
# Debug mode: 1 (on) or 0 (off)
|
||||
# NEVER use debug mode in production
|
||||
FLASK_DEBUG=1
|
||||
|
||||
# Flask secret key (falls back to SESSION_SECRET if not set)
|
||||
FLASK_SECRET_KEY=
|
||||
|
||||
# =============================================================================
|
||||
# RSS FEED CONFIGURATION
|
||||
# =============================================================================
|
||||
|
||||
# Maximum number of items in RSS feed (default: 50)
|
||||
FEED_MAX_ITEMS=50
|
||||
|
||||
# Feed cache duration in seconds (default: 300 = 5 minutes)
|
||||
FEED_CACHE_SECONDS=300
|
||||
|
||||
# =============================================================================
|
||||
# CONTAINER CONFIGURATION
|
||||
# =============================================================================
|
||||
|
||||
# Environment: development or production
|
||||
ENVIRONMENT=production
|
||||
|
||||
# Number of Gunicorn workers (default: 4)
|
||||
# Recommendation: (2 x CPU cores) + 1
|
||||
WORKERS=4
|
||||
|
||||
# Worker timeout in seconds (default: 30)
|
||||
WORKER_TIMEOUT=30
|
||||
|
||||
# Max requests per worker before restart (prevents memory leaks)
|
||||
MAX_REQUESTS=1000
|
||||
|
||||
# =============================================================================
|
||||
# DEVELOPMENT OPTIONS
|
||||
# =============================================================================
|
||||
|
||||
# Log level: DEBUG, INFO, WARNING, ERROR, CRITICAL
|
||||
# development or production (default: production)
|
||||
FLASK_ENV=production
|
||||
# 1 = debug on, 0 = off (default: 0). `flask run --debug` turns it on locally.
|
||||
FLASK_DEBUG=0
|
||||
# DEBUG, INFO, WARNING, ERROR or CRITICAL (default: INFO)
|
||||
LOG_LEVEL=INFO
|
||||
# Overrides the version shown in the footer and /health (default: the
|
||||
# package version, starpunk.__version__). Normally leave unset.
|
||||
# VERSION=
|
||||
|
||||
# Enable SQL query logging (development only)
|
||||
SQL_ECHO=0
|
||||
# =============================================================================
|
||||
# DEVELOPMENT LOGIN (never in production)
|
||||
# =============================================================================
|
||||
|
||||
# DEV_MODE=true adds /dev/login, which logs you in as DEV_ADMIN_ME without
|
||||
# OIDC, and makes ADMIN_ME and OIDC_* optional (default: false)
|
||||
# DEV_MODE=false
|
||||
# Identity used by /dev/login (required when DEV_MODE=true)
|
||||
# DEV_ADMIN_ME=http://localhost:5000/
|
||||
|
||||
# =============================================================================
|
||||
# REVERSE PROXY
|
||||
# =============================================================================
|
||||
|
||||
# Behind a reverse proxy (Caddy, nginx): take the client address from
|
||||
# X-Forwarded-For. Without it every visitor shares the proxy's address, so one
|
||||
# busy client can rate-limit everyone on /webmention and /websub.
|
||||
# Only enable when the proxy sets X-Forwarded-For itself (Caddy does).
|
||||
# (default: false)
|
||||
TRUSTED_PROXY=false
|
||||
|
||||
# CSRF tokens on admin forms, logout and the IndieAuth consent (default: true).
|
||||
# Leave on; tests turn it off unless they test it.
|
||||
CSRF_ENABLED=true
|
||||
|
||||
# =============================================================================
|
||||
# FEEDS
|
||||
# =============================================================================
|
||||
|
||||
# Items per feed (default: 50)
|
||||
# FEED_MAX_ITEMS=50
|
||||
# Feed cache lifetime in seconds; changes to notes clear it (default: 300)
|
||||
# FEED_CACHE_SECONDS=300
|
||||
# FEED_CACHE_ENABLED=true
|
||||
# Cached feed variants kept (default: 50)
|
||||
# FEED_CACHE_MAX_SIZE=50
|
||||
|
||||
# =============================================================================
|
||||
# WEBMENTION
|
||||
# =============================================================================
|
||||
|
||||
# Receive webmentions at /webmention (default: true)
|
||||
# WEBMENTION_ENABLED=true
|
||||
# Background worker that verifies received webmentions and sends outgoing
|
||||
# ones (default: true)
|
||||
# WEBMENTION_WORKER_ENABLED=true
|
||||
# Seconds between queue checks (default: 10) and after an error (default: 60)
|
||||
# WEBMENTION_POLL_INTERVAL=10
|
||||
# WEBMENTION_ERROR_BACKOFF=60
|
||||
# Items per worker pass (default: 5)
|
||||
# WEBMENTION_BATCH_SIZE=5
|
||||
# Show verified mentions without moderation (default: true)
|
||||
# WEBMENTION_AUTO_APPROVE=true
|
||||
# Rate limits on /webmention: requests per window per client IP (default: 60)
|
||||
# and per source domain (default: 10); window in seconds (default: 60)
|
||||
# WEBMENTION_RATE_LIMIT_ENABLED=true
|
||||
# WEBMENTION_RATE_LIMIT_IP=60
|
||||
# WEBMENTION_RATE_LIMIT_DOMAIN=10
|
||||
# WEBMENTION_RATE_LIMIT_WINDOW=60
|
||||
|
||||
# =============================================================================
|
||||
# WEBSUB (v2.10.0)
|
||||
# =============================================================================
|
||||
|
||||
# Built-in hub at /websub for this site's feeds. Set WEBSUB_HUB_URL to use
|
||||
# another hub instead (StarPunk then pings it on changes). (default: true)
|
||||
WEBSUB_ENABLED=true
|
||||
# WEBSUB_HUB_URL=https://hub.example.com/
|
||||
# Subscriptions in total, pending included (default: 1000)
|
||||
# WEBSUB_MAX_SUBSCRIPTIONS=1000
|
||||
# Per callback server (resolved address) (default: 25)
|
||||
# WEBSUB_MAX_PER_HOST=25
|
||||
# Seconds between hub worker passes (default: 5)
|
||||
# WEBSUB_POLL_INTERVAL=5
|
||||
|
||||
# =============================================================================
|
||||
# CROSSPOSTING / POSSE (optional; a target is enabled when its credentials
|
||||
# are set)
|
||||
# =============================================================================
|
||||
|
||||
# GoToSocial or any Mastodon-API server (v2.1.0, ADR-065). Create an app
|
||||
# token with scopes: write:statuses write:media, plus read:search
|
||||
# write:favourites for native likes and boosts (v2.5.0), plus read:statuses
|
||||
# read:accounts (or read) for backfeed (v2.13.0)
|
||||
GOTOSOCIAL_URL=
|
||||
GOTOSOCIAL_ACCESS_TOKEN=
|
||||
# Name shown in the editor and Micropub apps (default: GoToSocial)
|
||||
# GOTOSOCIAL_NAME=GoToSocial
|
||||
# public, unlisted or private (default: public)
|
||||
# GOTOSOCIAL_VISIBILITY=public
|
||||
# Server limits (defaults: 5000 characters, 4 images)
|
||||
# GOTOSOCIAL_MAX_CHARS=5000
|
||||
# GOTOSOCIAL_MAX_MEDIA=4
|
||||
|
||||
# Bluesky (v2.2.0, ADR-066). Use an app password, not your account password.
|
||||
BLUESKY_HANDLE=
|
||||
BLUESKY_APP_PASSWORD=
|
||||
# PDS / entryway (default: https://bsky.social)
|
||||
# BLUESKY_SERVICE=https://bsky.social
|
||||
# BLUESKY_NAME=Bluesky
|
||||
# BLUESKY_MAX_CHARS=300
|
||||
|
||||
# Threads (v2.6.0, ADR-069). A long-lived user token with threads_basic,
|
||||
# threads_content_publish and threads_delete (plus threads_read_replies and
|
||||
# threads_manage_insights for backfeed, v2.13.0); StarPunk refreshes it daily and
|
||||
# keeps the current one in DATA_PATH/threads_token.json.
|
||||
# How to get one: docs/design/v2.6.0/2026-10-31-threads-crossposting.md
|
||||
THREADS_ACCESS_TOKEN=
|
||||
# Your Threads username (for the Micropub syndicate-to uid)
|
||||
THREADS_USERNAME=
|
||||
# THREADS_NAME=Threads
|
||||
# THREADS_MAX_CHARS=500
|
||||
# THREADS_MAX_MEDIA=4
|
||||
|
||||
# Seconds between syndication worker passes (default: 10)
|
||||
# SYNDICATION_POLL_INTERVAL=10
|
||||
|
||||
# Backfeed (v2.13.0, ADR-073): bring likes, reposts, quotes and replies to the
|
||||
# crossposts back to the note as responses. Needs the read scopes above.
|
||||
# BACKFEED_ENABLED=false
|
||||
# Stop checking a note's responses after this many days (default: 90)
|
||||
# BACKFEED_MAX_AGE_DAYS=90
|
||||
|
||||
# =============================================================================
|
||||
# DEBUGGING
|
||||
# =============================================================================
|
||||
|
||||
# Keep uploads that fail processing in DATA_PATH/debug for diagnosis
|
||||
# (default: false); cleaned up by age (default: 7 days) and total size
|
||||
# (default: 100 MB)
|
||||
# DEBUG_SAVE_FAILED_UPLOADS=false
|
||||
# DEBUG_FILE_MAX_AGE_DAYS=7
|
||||
# DEBUG_FILE_MAX_SIZE_MB=100
|
||||
@@ -1,58 +1,46 @@
|
||||
# Gitea Actions workflow for StarPunk
|
||||
# Builds and pushes container images on version tags
|
||||
|
||||
# Runs the tests, then builds and pushes container images on version tags.
|
||||
#
|
||||
# The runner has no Docker daemon (that would give workflows root on the
|
||||
# server). Images are built by the rootless BuildKit container next to it,
|
||||
# reached through its socket.
|
||||
name: Build Container
|
||||
|
||||
on:
|
||||
# Trigger on version tags
|
||||
push:
|
||||
tags:
|
||||
- 'v[0-9]+.[0-9]+.[0-9]+'
|
||||
# Allow manual trigger from Gitea UI
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
REGISTRY: git.thesatelliteoflove.com
|
||||
BUILDKIT: unix:///run/buildkit/buildkit/buildkitd.sock
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: docker
|
||||
steps:
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
if command -v apk > /dev/null; then
|
||||
apk add --no-cache nodejs npm docker git
|
||||
elif command -v apt-get > /dev/null; then
|
||||
apt-get update && apt-get install -y nodejs npm docker.io git
|
||||
elif command -v yum > /dev/null; then
|
||||
yum install -y nodejs npm docker git
|
||||
fi
|
||||
|
||||
run: apk add --no-cache nodejs git docker-cli docker-cli-buildx
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Extract registry URL
|
||||
id: registry
|
||||
run: |
|
||||
# Extract hostname from server URL (remove protocol)
|
||||
REGISTRY_URL=$(echo "${{ github.server_url }}" | sed 's|https://||' | sed 's|http://||')
|
||||
echo "url=${REGISTRY_URL}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Login to Gitea Container Registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ steps.registry.outputs.url }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
# The automatic Actions token can't log in to the container registry, so
|
||||
# this uses a personal access token with only the write:package scope,
|
||||
# stored as the repository secret REGISTRY_TOKEN.
|
||||
- name: Log in to the Gitea registry
|
||||
run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login "$REGISTRY" -u "${{ github.repository_owner }}" --password-stdin
|
||||
- name: Use the BuildKit socket
|
||||
run: docker buildx create --name ci --driver remote "$BUILDKIT" --use
|
||||
# Builds the Containerfile's test stage, which runs pytest; a failure
|
||||
# stops the workflow before anything is pushed
|
||||
- name: Run tests
|
||||
run: docker buildx build --target test --progress plain -f Containerfile .
|
||||
- name: Build and push
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: .
|
||||
file: ./Containerfile
|
||||
push: true
|
||||
tags: |
|
||||
${{ steps.registry.outputs.url }}/${{ github.repository }}:${{ github.ref_name }}
|
||||
${{ steps.registry.outputs.url }}/${{ github.repository }}:latest
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
run: |
|
||||
image="$REGISTRY/$(echo '${{ github.repository }}' | tr '[:upper:]' '[:lower:]')"
|
||||
docker buildx build --push -f Containerfile \
|
||||
-t "$image:${{ github.ref_name }}" -t "$image:latest" .
|
||||
- name: Clean up
|
||||
if: always()
|
||||
run: |
|
||||
docker buildx rm ci || true
|
||||
docker logout "$REGISTRY" || true
|
||||
+587
-1
@@ -7,14 +7,600 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.13.4] - 2026-11-10
|
||||
|
||||
### Added
|
||||
|
||||
- The home page, tag pages and the Likes page show each note's like, repost and reply counts next to its date (♥ 3 🔁 1 💬 2), linking to the note's responses. Notes with none show nothing
|
||||
|
||||
### Changed
|
||||
|
||||
- Clicking anywhere on a reply opens it where it was posted (the reply on Bluesky, GoToSocial or Threads, or the page that sent the webmention). The author link and links in the reply still go to their own pages
|
||||
|
||||
### Fixed
|
||||
|
||||
- The "Liked", "Reposted" and "In reply to" lines on notes picked up styles meant for v2.13.2's counts line (both used the class `reaction`). The counts line's classes are now `response-*`
|
||||
|
||||
## [2.13.3] - 2026-11-10
|
||||
|
||||
### Changed
|
||||
|
||||
- Likes and reposts by people who opted out of backfeed (`#nobridge` or `#nobot` in their profile, or Bluesky's "don't show me to logged-out users" label) are counted in a note's totals, without their name or avatar ("+1 on Bluesky"). Before, they were left out entirely, so totals were lower than on the silo
|
||||
|
||||
## [2.13.2] - 2026-11-10
|
||||
|
||||
### Changed
|
||||
|
||||
- Likes, reposts and bookmarks are no longer rows of faces under "Responses". A note shows one line of counts under the post (♥ likes, 🔁 reposts, 💬 replies, bookmarks), and hovering or tapping a count lists who, with where each came from. Threads' anonymous counts are part of the totals ("+1 on Threads"), so the separate "On Threads" line is gone. The Responses section now holds replies, quotes and mentions
|
||||
- Web actions (https://indieweb.org/webactions): each like, repost and reply count is an `<indie-action>` for the note, with links to do the same on the note's crossposts ("Like on Bluesky · GoToSocial · Threads"). The icons show even before anyone has responded, when the note has crossposts
|
||||
|
||||
### Fixed
|
||||
|
||||
- Avatars that GoToSocial hasn't cached (it redirects to the original server) are saved; the redirect is followed through the safe fetcher
|
||||
|
||||
## [2.13.1] - 2026-11-10
|
||||
|
||||
### Fixed
|
||||
|
||||
- Avatars from your own GoToSocial server are saved again. When StarPunk reaches it through the same proxy, its name resolves to an internal address, which the safe fetcher refused. That one server (`GOTOSOCIAL_URL`) is now fetched directly, as crossposting already does
|
||||
- Bluesky avatars are saved again. Bluesky's image CDN refuses some servers' IPv4 addresses (HTTP 403), so avatars are now downloaded from the account's own server (`com.atproto.sync.getBlob`)
|
||||
|
||||
## [2.13.0] - 2026-11-10
|
||||
|
||||
Backfeed: responses to crossposts come back to the note (ADR-073).
|
||||
|
||||
### Added
|
||||
|
||||
- Likes, reposts, quotes and replies to a note's crossposts on GoToSocial and Bluesky, and replies on Threads, are shown on the note as responses, labelled "via Bluesky" (or GoToSocial, Threads). StarPunk checks each crosspost itself: every 15 minutes on a note's first day, hourly in its first week, daily until 30 days, weekly until `BACKFEED_MAX_AGE_DAYS` (90). Withdrawn likes and deleted replies drop off at the next check
|
||||
- Threads like, repost and quote counts, shown as a line such as "On Threads: 12 likes · 3 reposts" (the Threads API doesn't say who)
|
||||
- A Quotes group in a note's responses
|
||||
- **Backfill responses** on the admin Webmentions page checks every crosspost once, however old the note; each note's edit page shows when its responses were last checked, with **Check now**
|
||||
- The admin Webmentions list can be filtered by where a response came from, and shows deleted ones under **Deleted**
|
||||
- Settings `BACKFEED_ENABLED` (default off) and `BACKFEED_MAX_AGE_DAYS`. Backfeed needs read access: a GoToSocial token with `read:statuses` and `read:accounts` (or `read`), and a Threads token with `threads_read_replies` and `threads_manage_insights`. Bluesky app passwords already work
|
||||
|
||||
### Changed
|
||||
|
||||
- Response authors' avatars are saved on this site (96 px WebP in `DATA_PATH/avatars`) and shown from there, instead of being loaded from other sites. Until a copy is saved the initial is shown
|
||||
- Deleting a webmention in the admin keeps it as a deleted record, so a re-sent webmention or the next check doesn't bring it back
|
||||
- People with `#nobridge` or `#nobot` in their profile, and Bluesky accounts that ask not to be shown to logged-out visitors, are not backfed; only public responses are
|
||||
- A webmention for a response that was already backfed (for example a reply also sent by Bridgy) is ignored, so it isn't shown twice
|
||||
- The Threads client uses `graph.threads.com`, the host in Meta's current documentation
|
||||
|
||||
## [2.12.1] - 2026-11-10
|
||||
|
||||
### Fixed
|
||||
|
||||
- Micropub `q=category` (the tag list apps use for autocomplete) only lists tags on posts that still exist. It used to include tags that an edit had removed from every post, and tags only used on deleted posts. Drafts still count; a deleted post's tags come back if it is undeleted
|
||||
|
||||
## [2.12.0] - 2026-11-03
|
||||
|
||||
The metrics dashboard is gone (ADR-072).
|
||||
|
||||
### Removed
|
||||
|
||||
- The admin metrics dashboard (`/admin/metrics-dashboard`, the "Metrics" admin link) and its JSON endpoint `/admin/metrics`. Its numbers were kept in memory per server process, so they covered a quarter of the traffic and vanished on every restart
|
||||
- Everything that only fed it: request and database-query timing, the memory monitor thread, feed-generation counters (`starpunk/monitoring/`), and the `X-Request-ID` response header. Slow queries are no longer logged
|
||||
- The `METRICS_ENABLED`, `METRICS_SLOW_QUERY_THRESHOLD`, `METRICS_SAMPLING_RATE`, `METRICS_BUFFER_SIZE` and `METRICS_MEMORY_INTERVAL` settings (ignored if still set), and the `psutil` dependency
|
||||
- The `metrics` section of `/admin/health`; its database, pool, filesystem and disk checks are unchanged, as is `/health`
|
||||
|
||||
## [2.11.0] - 2026-11-03
|
||||
|
||||
Maintenance release: dependency upgrades and the High-priority backlog items.
|
||||
|
||||
### Security
|
||||
|
||||
- Every dependency is upgraded to its current release. The previous set had 97 published vulnerabilities (Pillow 64, PyJWT 22, gunicorn 4, and Flask, Markdown, python-dotenv, pytest and pillow-heif); the new set has none known. Notable versions: Flask 3.1, gunicorn 26, Pillow 13, PyJWT 2.15, httpx 0.28, nh3 0.3, Markdown 3.11, pillow-heif 1.8, psutil 7
|
||||
- Admin forms, logout and the IndieAuth consent screen now carry a CSRF token (an HMAC of your login session, no storage needed); requests without it are refused. Endpoints that apps call with tokens (Micropub, Webmention, WebSub, the token endpoint) are unaffected
|
||||
- Fetching an IndieAuth app's page and fetching pages and images for crosspost link cards now use the same protected fetcher as webmentions: every redirect is checked against private addresses, the connection goes to the checked address, and the whole fetch has a time limit
|
||||
- Debug mode is off and the environment is production unless set otherwise (`flask run --debug` still turns debug on locally); the container image sets both explicitly
|
||||
|
||||
### Changed
|
||||
|
||||
- Runs on Python 3.14 (was 3.11, which reaches end of life in 2027)
|
||||
- Rate limits on `/webmention` and `/websub` are counted in the database and shared by all server processes. Before, each of the four processes counted separately, so the real limit was four times the setting
|
||||
- Test dependencies are in `requirements-test.txt` (used by CI); `pytest` is no longer installed in the production image, and `lxml` and `python-dateutil`, which StarPunk uses directly, are now pinned
|
||||
- `uv` is pinned in the Containerfile instead of `latest`
|
||||
|
||||
### Fixed
|
||||
|
||||
- Deleting a note or a photo left its tag, photo and image-size rows behind, because SQLite only applies the schema's cascades when foreign keys are switched on. They now are; migration 018 removes rows left by earlier deletes (production had three)
|
||||
- Deprecated `datetime.utcnow()` and sqlite3's default date adapter (both removed in a future Python) replaced; the stored formats are unchanged
|
||||
- `compose.yaml` pinned version 0.6.0 (which would override the real version shown) and set an unused variable; a stray duplicate `dev_auth.py` and a `test.ini` with a private host name were removed
|
||||
|
||||
## [2.10.2] - 2026-11-03
|
||||
|
||||
### Fixed
|
||||
|
||||
- The context shown above replies, likes and reposts (the post being replied to: its title, author and text) was never fetched since v2.7.0: a change to the shared fetcher wasn't carried over, so every new context was saved as an error and the note showed only the bare link. Relative links on the other page now also resolve against its address after redirects
|
||||
|
||||
## [2.10.1] - 2026-11-03
|
||||
|
||||
WebSub fixes from an adversarial review of v2.10.0 against the [W3C Recommendation](https://www.w3.org/TR/websub/) (details in ADR-071, Amendments).
|
||||
|
||||
### Security
|
||||
|
||||
- One subscription request could stop the hub for good: a callback that answered verification with a large page, or a callback URL with a control character, crashed the worker on every pass. Each queued item is now handled on its own
|
||||
- A subscriber sending its response one byte at a time could hold the hub for hours. Every request the hub (and webmention fetching) makes now has a hard time limit
|
||||
- The subscription limits only counted verified subscriptions and could be dodged with subdomains. They now count pending requests too, per callback server address, and at most 200 requests can wait for verification
|
||||
- Callbacks must be printable URLs of reasonable length, not on this site, and on ports 80, 443 or 1024 and up
|
||||
|
||||
### Fixed
|
||||
|
||||
- New posts with photos were sent to subscribers (and cached in the feeds) without their photos, and photo-only edits sent the old feed. Notes with photos are announced once the photos are attached, and the feed cache follows photo and caption changes
|
||||
- A subscriber whose deliveries failed for three days was removed; WebSub says to keep it until its lease ends. Deliveries are retried and given up, the subscription stays
|
||||
- A temporary DNS failure or an error rendering the feed lost the delivery instead of retrying it. A feed that no longer exists now ends its subscriptions with a `hub.mode=denied` notice
|
||||
- Verification sent our normalised topic (`/tag/python`) instead of the one requested (`/tag/Python`), so strict subscribers refused it. Percent-encoded topics (`/%66eed.atom`) are now understood, the Content-Type check ignores case, unsubscribe requests ignore `hub.lease_seconds`, and an empty `hub.secret` is refused instead of turning signatures off
|
||||
- A newer subscription request arriving while an older one was being verified was lost (migration 017)
|
||||
- Unsubscribing works while `WEBSUB_HUB_URL` points to another hub
|
||||
- Feed ETags differed between server processes, so conditional requests rarely matched; `/feed` now sends `Vary: Accept`
|
||||
- A like no longer causes pushes of the home page and feeds, which don't show likes; each feed is rendered once per delivery round, not once per subscriber
|
||||
- Internationalised callback domains can be verified
|
||||
|
||||
### Changed
|
||||
|
||||
- `.env.example` documents `TRUSTED_PROXY` (needed behind a reverse proxy for per-client rate limits) and the WebSub settings
|
||||
|
||||
## [2.10.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
|
||||
- **WebSub** ([ADR-071](docs/decisions/ADR-071-websub.md)): readers that support it get new posts, edits and deletions within seconds instead of polling. StarPunk runs its own hub at `/websub` for its own feeds: the home page, `/feed.atom`, `/feed.rss`, `/feed.json`, `/likes` and each tag page. Each advertises the hub and itself (`Link` headers, and links inside the page or feed). Subscriptions are verified with a challenge, leases expire, deliveries carry the full feed, signed with `X-Hub-Signature` when the subscriber gave a secret, and failures are retried for about nine hours. Requests to subscribers go through the same private-address protections as webmentions, and there are limits per subscriber host and in total. Set `WEBSUB_HUB_URL` to use another hub instead; `WEBSUB_ENABLED=false` turns it off
|
||||
- The admin dashboard shows WebSub subscribers per feed
|
||||
|
||||
### Fixed
|
||||
|
||||
- Feeds could be up to five minutes out of date after posting, editing or deleting, and each server process cached its own copy. Any change to a public note now refreshes the feeds everywhere at once
|
||||
- The memory monitor stopped after the first request (the same bug the webmention worker had before v2.6.8)
|
||||
|
||||
### Changed
|
||||
|
||||
- The RSS feed calls itself `/feed.rss` instead of the older `/feed.xml`. Both addresses still work
|
||||
|
||||
## [2.9.0] - 2026-11-03
|
||||
|
||||
Micropub review against the [W3C Recommendation](https://www.w3.org/TR/micropub/).
|
||||
|
||||
### Changed
|
||||
|
||||
- **Micropub posts are only crossposted to the targets the client asks for.** If a client doesn't send `mp-syndicate-to`, nothing is crossposted (before, everything was). Posts made in the admin editor are unchanged
|
||||
- Posts sent with `visibility` private or unlisted are saved as drafts instead of being published (StarPunk has no private posts)
|
||||
- A token with only the `draft` scope can create drafts, and nothing else
|
||||
- `q=source` and `q=category` need a token that can post (`create`, `update`, `delete` or `draft`); before, any token could list every post, drafts included
|
||||
- Sending the access token in both the header and the body is refused (400)
|
||||
|
||||
### Added
|
||||
|
||||
- Bookmarks: Micropub `bookmark-of` is stored and shown ("🔖 Bookmarked …", `u-bookmark-of`), returned by `q=source`, listed in `q=config`, and crossposts lead with the bookmarked link (migration 015)
|
||||
- A post with only a `summary` uses it as the text; a post with only a `name` is a titled post with no body
|
||||
|
||||
### Fixed
|
||||
|
||||
- Quill and other JSON clients put `mp-syndicate-to` and `mp-slug` inside `properties`; they were ignored, so posts crossposted everywhere and slugs were not used
|
||||
- HTML in posts is sanitized: scripts, event handlers, iframes and `javascript:` links are removed when notes are shown. Formatting, images, code highlighting, footnotes and microformats are kept
|
||||
- A photo that can't be downloaded or found made the post without it while reporting success. Create and update now fail with 400 and change nothing
|
||||
- Remote photos are fetched the same way as webmentions: every redirect is checked for private addresses and the connection goes to the checked address
|
||||
- JSON values that weren't lists were read a character at a time (`"category": "solo"` became tags s, o, l). Single values are now treated as a one-item list; `delete` values in updates must be lists
|
||||
- Posts written as HTML are returned by `q=source` as `{"html": …}`, as the spec requires; empty `content` is left out
|
||||
- A client-supplied `published` time is stored as UTC like every other post; a date that can't be read is refused instead of silently replaced with now
|
||||
- Malformed JSON gets 400 instead of 500, and error messages no longer include Python internals
|
||||
- `video` and `audio` (not supported) are refused with 400 instead of being dropped; person tags in `category` are ignored instead of failing the post
|
||||
- An update with no `replace`, `add` or `delete` is refused
|
||||
- `Authorization: bearer …` is accepted (the scheme is case-insensitive), and 401 responses include `WWW-Authenticate`
|
||||
|
||||
## [2.8.1] - 2026-11-03
|
||||
|
||||
### Fixed
|
||||
|
||||
- Starting the server after an upgrade with a database migration could fail: several server processes applied the same migration at once and one stopped with "duplicate column name", taking the server down until it restarted (this happened deploying v2.8.0, for about two seconds). The migration lock was released right after it was taken, and again before each migration was recorded. Migrations now run entirely inside the lock
|
||||
|
||||
## [2.8.0] - 2026-11-03
|
||||
|
||||
Microformats2 review: the markup now matches the [microformats2](https://microformats.org/wiki/microformats2) vocabularies, so IndieWeb readers and tools see what the page shows.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Replies to a note weren't marked up as its comments, so readers and threading tools saw none. They are now `u-comment h-cite`s with the replier's h-card (name, link, photo) and the reply's HTML
|
||||
- Likes, reposts, bookmarks and mentions on a note had no microformats at all. Each is now a `u-like`, `u-repost`, `u-bookmark` or `u-mention` h-cite with its author, including likes past the eight faces shown
|
||||
- Responses link to and are dated by the original post (its own URL and published time) instead of the address that sent the webmention and when it arrived. New database columns hold these (migration 014)
|
||||
- Post times had no time zone (and production showed microseconds), so readers could shift them by hours. Times on pages and in Micropub `q=source` are now UTC with a `Z`, matching the Atom feed
|
||||
- A note starting with a hashtag (`#indieweb is great…`) was treated as a titled article: its text became a heading and the body was empty, and a note starting with a bare `#` line got its slug as a title. Only a Markdown heading (`#` then a space) makes a title now
|
||||
- A webmention from one of a note's own crossposts (for example your GoToSocial copy, sent back through Bridgy) showed up as a response on the note. It is now ignored
|
||||
- Micropub and IndieAuth discovery links disappeared from pages when webmentions were turned off
|
||||
- A new site had no `rel="me"` link to its author, so readers couldn't tell whose site it was; pages now always link `rel="me"` and `rel="author"` to the author's home page
|
||||
- Finding the author's profile could pick someone else's h-card from their home page. It now follows the [representative h-card](https://microformats.org/wiki/representative-h-card-parsing) rules
|
||||
- A webmention's author is found by the [authorship algorithm](https://indieweb.org/authorship-spec), from the entry on the source page that actually responds to the note: an author given as a name or link, and the author of a feed the entry is in, now count
|
||||
|
||||
### Added
|
||||
|
||||
- Tag pages and the likes page have a feed URL and author, like the home page
|
||||
- Note previews have `u-uid`
|
||||
|
||||
## [2.7.2] - 2026-11-03
|
||||
|
||||
### Added
|
||||
|
||||
- When the replies, likes and other responses shown on a note change (a webmention arrives, is updated or removed, or you approve, hide or delete one), the note's webmentions are sent again, as the Webmention spec recommends, so the sites it links to can pick up the conversation. A response that is checked again without changing sends nothing
|
||||
|
||||
### Security
|
||||
|
||||
- Webmention fetches and sends connect to the exact address that passed the private-address check. Before, the hostname was looked up a second time when connecting, so a DNS server that changed its answer could still point StarPunk at a private address. TLS certificates are still checked against the hostname. IPv4 addresses are preferred when a host has both
|
||||
|
||||
## [2.7.1] - 2026-11-03
|
||||
|
||||
### Changed
|
||||
|
||||
- Releases run the full test suite first: CI builds the Containerfile's new `test` stage (same Python image as production) and stops before pushing an image if any test fails
|
||||
|
||||
### Fixed
|
||||
|
||||
- Eight tests that had been failing: the image validation tests still expected bytes (validation has returned an image object since v1.5.0), and a migration test started the app in production mode
|
||||
|
||||
## [2.7.0] - 2026-11-03
|
||||
|
||||
Webmention now follows the [W3C Recommendation](https://www.w3.org/TR/webmention/) throughout.
|
||||
|
||||
### Changed
|
||||
|
||||
- Webmentions are only accepted for published notes. Mentions of the home page or other pages were stored but never shown; they're now refused with 400
|
||||
- Deleted notes answer 410 Gone with a "This post has been deleted" page instead of 404
|
||||
- A received mention that is sent again keeps its moderation: one you hid stays hidden, one you approved stays approved
|
||||
|
||||
### Fixed
|
||||
|
||||
- Sending: endpoint discovery now handles `rel` values with several words (`rel="webmention other"`), any capitalisation, `rel` after other Link parameters, and several Link headers, and takes the first `<link>` or `<a>` in page order
|
||||
- Sending: the endpoint is looked up again every time (the 1–24 hour cache broke the spec's rule for updated posts)
|
||||
- Sending: editing a note notifies every page it mentioned before, including an old reply target and links the edit removed. Deleting or restoring a note notifies them too, so they can remove or restore the mention
|
||||
- Sending: a redirect from the endpoint is no longer treated as success (the POST was turned into a GET); "429 Too Many Requests" is retried
|
||||
- Receiving: the source must link to the exact target URL (in a link, image, or other HTML attribute, any JSON value, or plain text). A longer URL that starts with the target, or the target mentioned in plain text or an HTML comment, no longer counts
|
||||
- Receiving: relative links in the source are resolved, using the URL after redirects
|
||||
- Receiving: the source is fetched with an `Accept` header and decoded in its declared character set
|
||||
- Receiving: if the source can't be reached when a mention is sent again, the existing mention stays up instead of disappearing
|
||||
- Receiving: the target's host is compared without regard to case or a default port, and `#fragment`s are dropped, so one mention isn't stored twice
|
||||
- Mention snippets cut at 500 characters no longer leave HTML tags open, and the "Mentions" list no longer shows raw HTML tags
|
||||
- The `Link: rel="webmention"` header is added alongside other Link headers instead of replacing them
|
||||
- Links at the top level of a code block were still sent webmentions
|
||||
|
||||
### Security
|
||||
|
||||
- Fetches for webmentions check every redirect, not just the first URL, so a public page can't redirect StarPunk to a private address. Every address a hostname resolves to (IPv4 and IPv6) must be public; carrier-grade NAT, `0.0.0.0/8`, multicast, reserved and IPv4-mapped addresses are blocked too. Each fetch has a total time limit and a 1 MB size limit, including endpoint discovery
|
||||
|
||||
## [2.6.9] - 2026-11-03
|
||||
|
||||
### Fixed
|
||||
|
||||
- Received webmentions failed when the sender's profile photo had alt text (Bridgy, Mastodon and most IndieWeb sites do this). The photo is now stored as its URL
|
||||
- Outgoing webmentions were never sent: endpoint discovery crashed on every target that had an endpoint, and the target was then marked as having none. Discovery now works, and network errors during discovery are retried instead of giving up
|
||||
- Each server process ran its own webmention worker, so a webmention could be processed or sent more than once. Only one process handles the queues at a time now; if it stops, another takes over
|
||||
|
||||
### Security
|
||||
|
||||
- A webmention sender could set its author link to a `javascript:` URL, which was shown as a clickable link on the note page. Author links and photos are now only kept if they are http or https URLs
|
||||
|
||||
## [2.6.8] - 2026-11-03
|
||||
|
||||
### Fixed
|
||||
|
||||
- Webmentions were never processed. The background worker checked URLs against this site's settings without the app loaded, so every incoming webmention was marked "error" ("Working outside of application context") and every outgoing one kept failing and retrying. The worker now runs with the app loaded. A received webmention showed up in the admin list but never on the note
|
||||
- The webmention worker also stopped as soon as any request finished, so after startup it often wasn't running at all. It now runs for the life of the server process
|
||||
|
||||
## [2.6.7] - 2026-11-02
|
||||
|
||||
### Fixed
|
||||
|
||||
- Admin pages were misaligned: the notes table ran past the right edge of the header and admin menu, because every page sat in the 40rem reading column and the table couldn't shrink to fit it. Admin pages now use a 60rem frame for the header, menu, content and footer alike, long post previews wrap, and on narrow screens tables scroll sideways instead of spilling out
|
||||
|
||||
## [2.6.6] - 2026-11-02
|
||||
|
||||
### Fixed
|
||||
|
||||
- Crossposts left out photos that were embedded in the post text instead of attached. Some Micropub clients (OutPost, for one) upload a photo to the media endpoint and then put it in `content` as an image rather than sending it as `photo`; both are valid Micropub. Images in the text that are in this site's media library are now sent to GoToSocial, Bluesky and Threads like attached photos, with their alt text, after any attached ones and within each target's photo limit. GoToSocial's text no longer includes the image Markdown. Images hosted on other sites are still left out
|
||||
|
||||
## [2.6.5] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- A Micropub post that failed partway (for example, the same photo listed twice) answered "Failed to create post" but stayed published without its photos, so a client retrying would post it twice. Photos are now looked up before the post is made, and a post that fails after that is removed again
|
||||
- After such a failure, every request failed with "database is locked" for about 10 seconds: database connections now roll back unfinished writes when they're returned to the pool
|
||||
- The same photo listed twice in one post is shown once instead of failing the post
|
||||
- Replies didn't read as replies to IndieWeb readers and webmention receivers: on note pages `u-in-reply-to` sat inside an unrelated `h-cite`, and the home page didn't mark replies at all, so [Post Type Discovery](https://www.w3.org/TR/post-type-discovery/) saw notes (or photo posts). The reply context is now the entry's `u-in-reply-to h-cite`, and home page entries show "↩ In reply to"
|
||||
- Photos without alt text used "Image" as their description, which IndieWeb readers passed on as if you'd written it. They now use the post's text (up to 150 characters), and fall back to "Image" only when the post has no text
|
||||
- Home page entries always have an `e-content`, so a photo post without text never gets an implied name (which readers would show as its caption)
|
||||
|
||||
### Changed
|
||||
|
||||
- Photo posts no longer need text. Micropub doesn't require `content`, and `q=config` already offered a Photo post type with only `photo`. A Micropub update may also clear the text of a post that keeps its photos. Crossposts of a photo post without text start with its link, and its page is titled "Photo"
|
||||
|
||||
## [2.6.4] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- Tags keep the order you entered them in. The first tag is the note's main tag: it's the Threads topic, and Bluesky keeps the first tags when space runs out. Before, both used the alphabetically first tag. Public pages still list tags alphabetically
|
||||
- Opening a note in the editor or updating it via Micropub reordered its tags alphabetically on save; the editor and Micropub `q=source` now show, and keep, the entered order
|
||||
|
||||
## [2.6.3] - 2026-11-01
|
||||
|
||||
### Changed
|
||||
|
||||
- **A note's own link gets the preview card** (ADR-070). When a note links somewhere (say, an article), the crosspost's card shows that page instead of the note. Bluesky cards are built from the page's title, description and image; Threads previews it from the text. Notes without links keep their own card
|
||||
|
||||
### Fixed
|
||||
|
||||
- Markdown links (`[text](url)`) lost their URL on Bluesky and Threads. On Bluesky the link text is now clickable; on Threads it's written "text (url)"
|
||||
- Notes with many links failed on Threads (Meta allows 5 per post); extra links now keep just their text
|
||||
|
||||
## [2.6.2] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- New notes failed their first Threads crosspost with "Invalid Link Attachment" (Meta rejects an explicit link preview for a page it hasn't fetched yet) and only went out on the retry a minute later. StarPunk no longer sends the explicit link preview; Threads still shows one, built from the note's link in the text
|
||||
|
||||
## [2.6.1] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- Threads crossposts gave up after Meta's publish step returned a one-off `HTTP 400 Fatal`; publish errors are now retried (a retry checks for an existing post first, so nothing is posted twice). Only a bad token or missing permission stops retries
|
||||
- Threads errors now include Meta's error code, subcode, user message and trace id, so the editor shows why a post failed
|
||||
- Wait a moment before the first readiness check on a Threads post, since Meta can report it ready too early
|
||||
|
||||
## [2.6.0] - 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
- **Crossposting to Threads** (ADR-069): set `THREADS_ACCESS_TOKEN` (and optionally `THREADS_USERNAME`) to add a Threads target. Text posts get a link preview; 1 image posts as an image, 2–4 as a carousel, with alt text; the note's first tag becomes the post's topic. Shortened to 500 characters. Edits don't follow (the API can't edit); unpublish and delete remove the post. Likes aren't crossposted to Threads; reposts post "🔁 title url" (with the comment, if any)
|
||||
- The Threads token (60-day lifetime) is refreshed automatically once a day and kept in `data/threads_token.json`, so it survives restarts; setting a new token in the config replaces it. Setup steps: `docs/design/v2.6.0/`
|
||||
|
||||
## [2.5.0] - 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
- **Likes and reposts** via Micropub `like-of` / `repost-of` (ADR-068); a comment is optional
|
||||
- Likes are kept off the home page and feeds and listed on a new `/likes` page (linked in the header)
|
||||
- Reposts appear in the stream and feeds with a "🔁 Reposted" line
|
||||
- The liked/reposted post's title and author are fetched and shown; it is sent a webmention
|
||||
- GoToSocial: likes/reposts of fediverse posts become native favourites/boosts; reposts of other pages (or with a comment) post "🔁 title url"
|
||||
- Bluesky: likes/reposts of Bluesky posts become native likes/reposts; a repost with a comment is a quote post; reposts of other pages post with a link card
|
||||
- Likes of ordinary web pages are not crossposted; deleting a like/repost undoes it everywhere
|
||||
- Migration 013: `notes.like_of`, `notes.repost_of`
|
||||
|
||||
### Fixed
|
||||
|
||||
- Outgoing webmentions named posts `https://site//note/<slug>` (double slash)
|
||||
|
||||
### Notes
|
||||
|
||||
- The GoToSocial token needs the extra scopes `read:search` and `write:favourites`
|
||||
|
||||
## [2.4.0] - 2026-10-31
|
||||
|
||||
### Added
|
||||
|
||||
- **Micropub update** (`action=update`, JSON): replace/add/delete for `content`, `name`, `category`, `photo`, `in-reply-to`, `post-status`; edits follow to crossposts (ADR-067)
|
||||
- **Micropub delete and undelete**: soft delete, restorable; crossposts are removed and re-posted
|
||||
- **Drafts** via `post-status: draft`; **replies** via `in-reply-to`
|
||||
- Photos uploaded in a multipart create request; photos hosted elsewhere are downloaded (public addresses only, 20 MB cap)
|
||||
- `q=source` without `url` lists recent posts (drafts included, `limit`/`offset`); `properties[]` filter; returns photos, syndication links, reply and post status
|
||||
- `q=category` (tag suggestions, optional `filter`); `q=config` lists supported queries and post types
|
||||
- Media endpoint accepts the `media` scope
|
||||
|
||||
### Fixed
|
||||
|
||||
- Media endpoint was advertised at `/micropub/media` but served at `/media` (404 for clients); now both work
|
||||
- Unsupported Micropub types (e.g. `h=event`) were silently stored as notes; now rejected
|
||||
|
||||
## [2.3.3] - 2026-10-31
|
||||
|
||||
### Fixed
|
||||
|
||||
- Micropub `name` was ignored, so titles sent by Micropub clients never appeared. It is now stored as the note's `# Heading` first line, which is how StarPunk titles work
|
||||
- Micropub `q=source` returned `/notes/<slug>` links (a 404); it now returns the real permalink `/note/<slug>`
|
||||
- Micropub `q=source` reported every note's first line as its `name`; it now returns `name` only for titled notes, with the heading left out of `content`, matching what was posted
|
||||
- Titled notes showed their title twice: as the page heading and again at the top of the body (and of the home-page summary)
|
||||
|
||||
## [2.3.2] - 2026-10-31
|
||||
|
||||
### Changed
|
||||
|
||||
- Header link reads "Admin" (was "Dashboard") when logged in
|
||||
|
||||
### Fixed
|
||||
|
||||
- Public pages never knew you were logged in (only `/admin` routes checked the session), so the header always said "Log in"
|
||||
|
||||
## [2.3.1] - 2026-10-31
|
||||
|
||||
### Fixed
|
||||
|
||||
- "Log in" is back in the header nav (2.3.0 had moved it to the footer, where it was easy to miss)
|
||||
|
||||
## [2.3.0] - 2026-10-31
|
||||
|
||||
### Changed
|
||||
|
||||
- **Public pages restyled**: compact header with the site name from `SITE_NAME`, serif reading font for notes, one-line metadata (date · #tags · Also on …), dark mode via `prefers-color-scheme`, footer pinned to the bottom, "Log in" moved to the footer
|
||||
- Home and tag pages show short notes in full; titled notes show a summary and "Read more"
|
||||
- Author h-card kept for parsers but visually hidden on this single-author site
|
||||
- Page titles use the note text and `SITE_NAME` instead of the slug and "StarPunk"
|
||||
|
||||
### Fixed
|
||||
|
||||
- Every note's metadata was styled as the page footer (centered, ruled, padded)
|
||||
- Previews cut the note's HTML at 300 characters, which could break markup
|
||||
- Bare URLs in notes are now links
|
||||
|
||||
## [2.2.0] - 2026-10-31
|
||||
|
||||
### Added
|
||||
|
||||
- **Crossposting to Bluesky**, ADR-066
|
||||
- Plain-text note (Markdown stripped) with clickable link and hashtag facets, shortened to 300 characters with the permalink kept
|
||||
- Images (up to 4, shrunk under Bluesky's 1 MB limit) with alt text; notes without images get a link card
|
||||
- Unpublishing, unticking and deleting remove the Bluesky post; edits don't follow (Bluesky has no edits)
|
||||
- Logs in with an app password: `BLUESKY_HANDLE`, `BLUESKY_APP_PASSWORD` (optional `BLUESKY_SERVICE`, default `https://bsky.social`)
|
||||
- Shown in the editor and Micropub `syndicate-to` next to GoToSocial
|
||||
|
||||
## [2.1.0] - 2026-10-31
|
||||
|
||||
### Added
|
||||
|
||||
- **Crossposting to GoToSocial** (or any Mastodon-API server), ADR-065
|
||||
- Full note as Markdown with images (alt text from captions), permalink and hashtags
|
||||
- Edits, unpublishing and deletes follow the note
|
||||
- On by default for new notes; untick in the editor, or send `mp-syndicate-to` from Micropub apps
|
||||
- Micropub `q=syndicate-to` and `q=config` list the target
|
||||
- Editor shows crosspost status, errors and a retry button; note pages link the copy with `u-syndication`
|
||||
- Configure with `GOTOSOCIAL_URL` and `GOTOSOCIAL_ACCESS_TOKEN` (scopes `write:statuses write:media`)
|
||||
- Migration 012: `syndications`
|
||||
|
||||
### Fixed
|
||||
|
||||
- Micropub `Location` header pointed at `/notes/<slug>` (404); now the real permalink
|
||||
- "Delete Note" on the edit page submitted the edit form (nested forms)
|
||||
- Search results linked to `/notes/<slug>` (404); now `/note/<slug>`
|
||||
|
||||
## [2.0.2] - 2026-10-31
|
||||
|
||||
### Fixed
|
||||
|
||||
- Container builds in Gitea Actions log in to the registry with the
|
||||
`REGISTRY_TOKEN` secret (a `write:package` token); the automatic Actions
|
||||
token is refused by the registry. No application changes.
|
||||
|
||||
## [2.0.1] - 2026-10-31
|
||||
|
||||
### Fixed
|
||||
|
||||
- Page links built from the request said `http://` behind an HTTPS reverse proxy
|
||||
(RSS/OPML `<link>`s and microformats `u-url`/`u-uid` permalinks). All absolute
|
||||
URLs in templates now come from `SITE_URL` via `absolute_url_for()`.
|
||||
- Container builds in Gitea Actions: build through the server's rootless BuildKit
|
||||
and log in to the public registry name (the old workflow never succeeded).
|
||||
|
||||
## [2.0.0] - 2026-10-31
|
||||
|
||||
### Changed (breaking)
|
||||
|
||||
- **Admin login uses OpenID Connect** instead of IndieAuth (ADR-064)
|
||||
- Works with Pocket ID or any OIDC provider; authorization code flow with PKCE and nonce
|
||||
- Only the account whose subject matches `OIDC_ADMIN_SUB` can log in
|
||||
- New required settings: `OIDC_ISSUER`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`, `OIDC_ADMIN_SUB`
|
||||
- `INDIELOGIN_URL` and `TOKEN_ENDPOINT` are ignored
|
||||
- **Micropub tokens are issued by StarPunk**, not an external IndieAuth server
|
||||
- Your `ADMIN_ME` page must point Micropub apps at StarPunk's
|
||||
`/.well-known/oauth-authorization-server` (or its `/auth/authorize` and `/auth/token`)
|
||||
- Apps authorized through an external server must be authorized again
|
||||
|
||||
### Added
|
||||
|
||||
- Built-in IndieAuth server: metadata, consent screen, token, revocation endpoints
|
||||
- **Connected Apps** admin page (`/admin/tokens`) to review and revoke Micropub tokens
|
||||
- Discovery links (`micropub`, `indieauth-metadata`, `authorization_endpoint`, `token_endpoint`) on every page
|
||||
- Migration 011: `oidc_login_state`, `indieauth_codes`, `indieauth_tokens`
|
||||
|
||||
### Removed
|
||||
|
||||
- IndieAuth endpoint discovery and external token verification (`starpunk/auth_external.py`)
|
||||
|
||||
### Notes
|
||||
|
||||
- Existing notes, media, tags, webmentions, author profile and admin sessions are preserved
|
||||
- v1.6.0 source (webmentions) recovered from the production image; it had not been committed
|
||||
|
||||
## [1.5.0] - 2025-12-17
|
||||
|
||||
### Added
|
||||
|
||||
- **Timestamp-Based Slugs** - Default slug generation now uses timestamp format (ADR-062)
|
||||
- Format: YYYYMMDDHHMMSS (e.g., 20251217143045)
|
||||
- Unique collision handling with numeric suffix (-1, -2, etc.)
|
||||
- More semantic and sortable than random slugs
|
||||
- Custom slugs via `mp-slug` or web UI still supported
|
||||
- Applies to all new notes created via Micropub or web interface
|
||||
|
||||
### Changed
|
||||
|
||||
- **Debug File Management** - Enhanced control and cleanup for failed uploads
|
||||
- Debug file saving now controlled by `DEBUG_SAVE_FAILED_UPLOADS` config (default: false in production)
|
||||
- Automatic cleanup of debug files older than 7 days on app startup
|
||||
- Disk space protection with 100MB limit for debug directory
|
||||
- Filename sanitization in debug paths to prevent path traversal
|
||||
- Debug feature designed for development/troubleshooting only
|
||||
|
||||
### Fixed
|
||||
|
||||
- **IndieAuth Authentication** - Corrected W3C IndieAuth specification compliance
|
||||
- Authentication now discovers endpoints from user's profile URL per specification
|
||||
- Uses `response_type=id` for authentication-only flow (identity verification)
|
||||
- Removed hardcoded indielogin.com service URL
|
||||
- URL comparison handles trailing slashes and case differences correctly
|
||||
- User-friendly error messages when endpoint discovery fails
|
||||
- DEPRECATED: `INDIELOGIN_URL` config no longer used (warning shown if set)
|
||||
|
||||
- **Feed Generation Performance** - Eliminated N+1 query pattern in feed generation
|
||||
- Implemented batch loading for note media in `_get_cached_notes()`
|
||||
- Single query loads media for all 50 feed notes instead of 50 separate queries
|
||||
- Significant performance improvement for RSS/Atom/JSON Feed generation
|
||||
- Feed cache still prevents repeated queries for same feed requests
|
||||
- Other N+1 patterns in lower-traffic areas deferred (see BACKLOG.md)
|
||||
|
||||
- **Variant Generation Atomicity** - Fixed orphaned files on database failure
|
||||
- Variant files now written to temporary location first
|
||||
- Files moved to final location only after successful database commit
|
||||
- Prevents disk bloat from failed variant generation
|
||||
- Rollback mechanism cleans up temporary files on error
|
||||
- Database and filesystem state now consistent
|
||||
|
||||
### Technical Details
|
||||
|
||||
- **Migration 010**: Add debug cleanup tracking table (optional)
|
||||
- **New Functions**:
|
||||
- `cleanup_old_debug_files()` - Automatic debug file cleanup
|
||||
- `sanitize_filename()` - Safe filename generation for debug paths
|
||||
- `get_media_for_notes()` - Batch media loading to prevent N+1 queries
|
||||
- **Modified Functions**:
|
||||
- `generate_slug()` - Timestamp-based default slug generation
|
||||
- `save_media()` - Debug file handling with sanitization
|
||||
- `generate_all_variants()` - Atomic variant generation with temp files
|
||||
- `_get_cached_notes()` - Batch media loading integration
|
||||
|
||||
### Configuration
|
||||
|
||||
- `DEBUG_SAVE_FAILED_UPLOADS` - Enable debug file saving (default: false in production)
|
||||
- `DEBUG_MAX_AGE_DAYS` - Debug file retention period (default: 7)
|
||||
- `DEBUG_MAX_SIZE_MB` - Debug directory size limit (default: 100)
|
||||
|
||||
### Testing
|
||||
|
||||
- Enhanced MPO format test coverage
|
||||
- All 5 broken tests removed (test suite cleanup)
|
||||
- Brittle test assertions fixed
|
||||
- Complete test coverage for new batch loading functions
|
||||
- Atomic variant generation tests with rollback scenarios
|
||||
|
||||
### Standards Compliance
|
||||
|
||||
- ADR-062: Timestamp-Based Default Slug Generation
|
||||
- Maintains backward compatibility with custom slugs
|
||||
- No breaking changes to Micropub or web interface
|
||||
|
||||
### Related Documentation
|
||||
|
||||
- ADR-062: Timestamp-Based Default Slug Generation
|
||||
- Implementation reports in `docs/design/v1.5.0/`
|
||||
- Architect reviews documenting all design decisions
|
||||
|
||||
## [1.4.2] - 2025-12-16
|
||||
|
||||
### Added
|
||||
|
||||
- HEIC/HEIF image format support for iPhone photo uploads
|
||||
- Automatic HEIC to JPEG conversion (browsers cannot display HEIC)
|
||||
- MPO (Multi-Picture Object) format support for iPhone depth/portrait photos
|
||||
- Automatic HEIC/MPO to JPEG conversion (browsers cannot display these formats)
|
||||
- Graceful error handling if pillow-heif library not installed
|
||||
|
||||
### Changed
|
||||
|
||||
- Increased maximum input image dimensions from 4096x4096 to 12000x12000 to support modern phone cameras (48MP+); images are still optimized to smaller sizes for web delivery
|
||||
|
||||
### Dependencies
|
||||
|
||||
- Added `pillow-heif` for HEIC image processing
|
||||
|
||||
@@ -6,7 +6,11 @@ This file contains operational instructions for Claude agents working on this pr
|
||||
|
||||
- We use **uv** for Python virtual environment management
|
||||
- All Python commands must be run with `uv run` prefix
|
||||
- Example: `uv run pytest`, `uv run flask run`
|
||||
- Example: `uv run pytest`, `uv run flask --app app.py run --debug`
|
||||
- There is no `pyproject.toml`: create the environment with `uv venv --python 3.14 && uv pip install -r requirements-dev.txt`
|
||||
- Tests need `SESSION_SECRET` set: `SESSION_SECRET=test uv run pytest`. The full suite takes about 6 minutes, 5 of them in `tests/test_migration_race_condition.py`; add `--ignore=tests/test_migration_race_condition.py` while iterating, but run everything before a release
|
||||
- Local login without OIDC: `DEV_MODE=true` and `DEV_ADMIN_ME` in `.env`, then `/dev/login`
|
||||
- Setup details: `docs/standards/development-setup.md`
|
||||
|
||||
## Agent Protocol (All Agents)
|
||||
|
||||
@@ -42,12 +46,18 @@ When invoking the agent-developer, always remind it to:
|
||||
|
||||
4. **Version number management**
|
||||
- Increment version numbers according to `docs/standards/versioning-strategy.md`
|
||||
- Update version in `starpunk/__init__.py`
|
||||
- Update version in `starpunk/__init__.py` (both `__version__` and `__version_info__`)
|
||||
|
||||
5. **Follow git protocol**
|
||||
- Adhere to git branching strategy in `docs/standards/git-branching-strategy.md`
|
||||
- Create feature branches for non-trivial changes
|
||||
- Write clear commit messages
|
||||
- Create feature branches for non-trivial changes, in their own git worktree (`<prefix>/vX.Y.Z-description`, prefixes `feat/`, `fix/`, `ci/`, `docs/`)
|
||||
- Write clear commit messages (`feat: ... (vX.Y.Z)`)
|
||||
|
||||
## Release Process
|
||||
|
||||
Follow "Release process" in `docs/standards/git-branching-strategy.md`. In short: work in a worktree on a `feat/`/`fix/` branch → bump `__version__` and `__version_info__` → `CHANGELOG.md` entry → design note in `docs/design/vX.Y.Z/` (or ADR amendment) → full test suite → commit → `git merge --no-ff` into `main` ("Merge branch '<branch>' for vX.Y.Z release") → annotated tag `vX.Y.Z` → push `main` and the tag. CI (`.gitea/workflows/build-container.yml`) runs the tests and publishes the image; production is then updated from the separate Ansible deployment repository. Back up the database before deploying a release with a migration.
|
||||
|
||||
Merging to `main`, tagging, pushing and deploying happen only when the project owner asks for a release.
|
||||
|
||||
## Documentation
|
||||
|
||||
|
||||
+27
-4
@@ -6,10 +6,10 @@
|
||||
# ============================================================================
|
||||
# Build Stage - Install dependencies in virtual environment
|
||||
# ============================================================================
|
||||
FROM python:3.11-slim AS builder
|
||||
FROM python:3.14-slim AS builder
|
||||
|
||||
# Install uv for fast dependency installation
|
||||
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
|
||||
COPY --from=ghcr.io/astral-sh/uv:0.9.28 /uv /usr/local/bin/uv
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
@@ -23,9 +23,30 @@ RUN uv venv /opt/venv && \
|
||||
uv pip install --no-cache -r requirements.txt
|
||||
|
||||
# ============================================================================
|
||||
# Runtime Stage - Minimal production image
|
||||
# Test Stage - Runs the test suite (CI builds it with --target test)
|
||||
# ============================================================================
|
||||
FROM python:3.11-slim
|
||||
FROM python:3.14-slim AS test
|
||||
|
||||
COPY --from=ghcr.io/astral-sh/uv:0.9.28 /uv /usr/local/bin/uv
|
||||
COPY --from=builder /opt/venv /opt/venv
|
||||
COPY requirements-test.txt .
|
||||
RUN uv pip install --no-cache --python /opt/venv/bin/python -r requirements-test.txt
|
||||
|
||||
ENV PATH="/opt/venv/bin:$PATH" \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
SESSION_SECRET=ci-test-secret \
|
||||
SITE_URL=http://localhost:5000 \
|
||||
ADMIN_ME=https://test.example.com
|
||||
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN python -m pytest -q -p no:cacheprovider tests
|
||||
|
||||
# ============================================================================
|
||||
# Runtime Stage - Minimal production image (default target: keep it last)
|
||||
# ============================================================================
|
||||
FROM python:3.14-slim
|
||||
|
||||
# Create non-root user for security
|
||||
# UID/GID 1000 is standard for first user on most systems
|
||||
@@ -40,6 +61,8 @@ COPY --from=builder /opt/venv /opt/venv
|
||||
ENV PATH="/opt/venv/bin:$PATH" \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
FLASK_ENV=production \
|
||||
FLASK_DEBUG=0 \
|
||||
FLASK_APP=app.py \
|
||||
DATA_PATH=/data \
|
||||
NOTES_PATH=/data/notes \
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 [Your Name]
|
||||
Copyright (c) 2025-2026 Phil Skentelbery
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
|
||||
@@ -1,205 +1,117 @@
|
||||
# StarPunk
|
||||
|
||||
A minimal, self-hosted IndieWeb CMS for publishing notes with RSS syndication.
|
||||
|
||||
**Current Version**: 1.2.0
|
||||
|
||||
## Versioning
|
||||
|
||||
StarPunk follows [Semantic Versioning 2.0.0](https://semver.org/):
|
||||
- Version format: `MAJOR.MINOR.PATCH`
|
||||
- Current: `1.2.0` (stable release)
|
||||
- Check version: `python -c "from starpunk import __version__; print(__version__)"`
|
||||
- See changes: [CHANGELOG.md](CHANGELOG.md)
|
||||
- Versioning strategy: [docs/standards/versioning-strategy.md](docs/standards/versioning-strategy.md)
|
||||
|
||||
## Philosophy
|
||||
A minimal, self-hosted, single-user IndieWeb site for publishing notes.
|
||||
Flask, SQLite and markdown files; one container.
|
||||
|
||||
"Every line of code must justify its existence. When in doubt, leave it out."
|
||||
|
||||
StarPunk is designed for a single user who wants to:
|
||||
- Publish short notes to their personal website
|
||||
- Own their content (notes stored as portable markdown files)
|
||||
- Syndicate via RSS
|
||||
- Support IndieWeb standards (Micropub, IndieAuth)
|
||||
- Run on minimal resources
|
||||
The current version is in [`starpunk/__init__.py`](starpunk/__init__.py);
|
||||
what changed in each release is in [CHANGELOG.md](CHANGELOG.md).
|
||||
|
||||
## Features
|
||||
|
||||
- **File-based storage**: Notes are markdown files, owned by you
|
||||
- **IndieAuth authentication**: Use your own website as identity
|
||||
- **Micropub support**: Full W3C Micropub specification compliance
|
||||
- **Media attachments**: Upload and display images with your notes
|
||||
- **Microformats2**: Full h-entry, h-card, and h-feed markup for IndieWeb compatibility
|
||||
- **Author discovery**: Automatic profile discovery from your IndieWeb identity
|
||||
- **RSS, ATOM, JSON Feed**: Multiple syndication formats with Media RSS support
|
||||
- **Custom slugs**: Control your note permalinks
|
||||
- **No database lock-in**: SQLite for metadata, files for content
|
||||
- **Self-hostable**: Run on your own server
|
||||
- **Minimal dependencies**: Core dependencies, no build tools
|
||||
**Publishing**
|
||||
- Notes in markdown, stored as plain files with metadata in SQLite
|
||||
- Admin editor with drafts, custom slugs, tags and photo uploads
|
||||
(resized variants, HEIC/MPO converted to JPEG, captions)
|
||||
- Post types: notes, titled posts, replies, likes, reposts and bookmarks
|
||||
- Full-text search (SQLite FTS5) and tag pages
|
||||
|
||||
## Requirements
|
||||
**IndieWeb**
|
||||
- [Micropub](https://www.w3.org/TR/micropub/): create, update, delete and
|
||||
undelete; media endpoint; `q=config`, `source`, `syndicate-to`, `category`
|
||||
- Built-in [IndieAuth](https://indieauth.spec.indieweb.org/) server for
|
||||
Micropub apps (consent screen, tokens, revocation, Connected Apps page)
|
||||
- Admin login through any OpenID Connect provider (e.g. Pocket ID)
|
||||
- [Webmention](https://www.w3.org/TR/webmention/): receiving (verified,
|
||||
moderated, rate-limited) and sending; replies, likes and reposts shown on
|
||||
each note
|
||||
- [WebSub](https://www.w3.org/TR/websub/) with a built-in hub, so readers get
|
||||
new posts within seconds
|
||||
- [Microformats2](https://microformats.org/wiki/h-entry): h-entry, h-feed and
|
||||
an h-card discovered from your own site
|
||||
|
||||
- Python 3.11 or higher
|
||||
- 500MB disk space
|
||||
- Linux, macOS, or Windows with WSL2
|
||||
**Syndication**
|
||||
- Feeds: RSS 2.0 (with Media RSS), Atom 1.0, JSON Feed 1.1, content
|
||||
negotiation at `/feed`, OPML export; tag pages and the likes page are
|
||||
h-feeds readers can follow
|
||||
- POSSE crossposting to GoToSocial/Mastodon, Bluesky and Threads, with
|
||||
`u-syndication` links back; edits and deletes follow, native likes and
|
||||
boosts where the target supports them
|
||||
|
||||
## Quick Start
|
||||
**Operations**
|
||||
- Migrations applied automatically at startup
|
||||
- `/health` endpoint, `/admin/health` diagnostics, structured logs
|
||||
- Fetches of user-supplied URLs are limited to public addresses, size and
|
||||
time
|
||||
|
||||
## Quick start
|
||||
|
||||
### Container
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/YOUR_USERNAME/starpunk.git
|
||||
cd starpunk
|
||||
|
||||
# Install uv (package manager)
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
|
||||
# Create virtual environment
|
||||
uv venv .venv --python 3.11
|
||||
|
||||
# Install dependencies
|
||||
uv pip install -r requirements.txt
|
||||
|
||||
# Configure
|
||||
cp .env.example .env
|
||||
# Edit .env and set ADMIN_ME and SESSION_SECRET
|
||||
|
||||
# Initialize database
|
||||
mkdir -p data/notes
|
||||
.venv/bin/python -c "from starpunk.database import init_db; init_db()"
|
||||
# Note: Database also auto-initializes on first run if not present
|
||||
|
||||
# Run development server
|
||||
.venv/bin/flask --app app.py run --debug
|
||||
|
||||
# Visit http://localhost:5000
|
||||
git clone https://git.thesatelliteoflove.com/phil/StarPunk.git && cd StarPunk
|
||||
cp .env.example .env # set SITE_URL, ADMIN_ME, SESSION_SECRET, OIDC_*
|
||||
docker build -t starpunk -f Containerfile .
|
||||
docker run -d --name starpunk -p 127.0.0.1:8000:8000 --env-file .env \
|
||||
-e FLASK_ENV=production -e FLASK_DEBUG=0 \
|
||||
-v ./starpunk-data:/data starpunk
|
||||
```
|
||||
|
||||
The container runs as UID 1000, so the data directory must be writable by
|
||||
it. Release images are published by CI as
|
||||
`git.thesatelliteoflove.com/phil/starpunk:vX.Y.Z`. Put a reverse proxy
|
||||
with HTTPS in front (see `Caddyfile.example` or `nginx.conf.example`) and
|
||||
set `TRUSTED_PROXY=true`. Register `<SITE_URL>/auth/callback` as the
|
||||
redirect URI with your OIDC provider. Details:
|
||||
[docs/architecture/deployment.md](docs/architecture/deployment.md).
|
||||
|
||||
### Local development
|
||||
|
||||
```bash
|
||||
uv venv --python 3.14
|
||||
uv pip install -r requirements-dev.txt
|
||||
cp .env.example .env # set SESSION_SECRET; add DEV_MODE=true and DEV_ADMIN_ME
|
||||
uv run flask --app app.py run --debug # http://localhost:5000
|
||||
SESSION_SECRET=test uv run pytest # tests (about 6 minutes)
|
||||
```
|
||||
|
||||
With `DEV_MODE=true`, `/dev/login` logs you in without OIDC. Never enable
|
||||
it on a public server. Full guide:
|
||||
[docs/standards/development-setup.md](docs/standards/development-setup.md).
|
||||
|
||||
## Configuration
|
||||
|
||||
All configuration is in the `.env` file. Required settings:
|
||||
All settings are environment variables (or a `.env` file), each listed with
|
||||
its default in [`.env.example`](.env.example). Required in production:
|
||||
`SESSION_SECRET`, `SITE_URL`, `ADMIN_ME`, `OIDC_ISSUER`, `OIDC_CLIENT_ID`,
|
||||
`OIDC_CLIENT_SECRET` and `OIDC_ADMIN_SUB`. Crossposting is enabled by
|
||||
setting a target's credentials.
|
||||
|
||||
- `ADMIN_ME` - Your IndieWeb identity URL (e.g., https://yoursite.com)
|
||||
- `SESSION_SECRET` - Random secret key (generate with `python3 -c "import secrets; print(secrets.token_hex(32))"`)
|
||||
- `SITE_URL` - Public URL of your site
|
||||
## Data and backups
|
||||
|
||||
See `.env.example` for all options.
|
||||
Everything lives in the data directory (`/data` in the container): the
|
||||
SQLite database, note files in `notes/`, uploaded media and logs. Back it
|
||||
up regularly, and always before upgrading to a release with a migration.
|
||||
|
||||
## Project Structure
|
||||
## Documentation
|
||||
|
||||
```
|
||||
starpunk/
|
||||
├── app.py # Application entry point
|
||||
├── starpunk/ # Application code
|
||||
├── data/ # Your notes and database (gitignored)
|
||||
│ ├── notes/ # Markdown files
|
||||
│ └── starpunk.db # SQLite database
|
||||
├── static/ # CSS and JavaScript
|
||||
├── templates/ # HTML templates
|
||||
└── tests/ # Test suite
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Publishing Notes
|
||||
|
||||
**Via Web Interface**:
|
||||
1. Navigate to `/admin`
|
||||
2. Login with your IndieWeb identity
|
||||
3. Create notes in markdown
|
||||
|
||||
**Via Micropub Client**:
|
||||
1. Configure client with your site URL
|
||||
2. Authenticate via IndieAuth
|
||||
3. Publish from any Micropub-compatible app
|
||||
|
||||
### Backing Up Your Data
|
||||
|
||||
Your notes are stored as plain markdown files in `data/notes/`. Back up this directory:
|
||||
|
||||
```bash
|
||||
# Simple backup
|
||||
tar -czf backup.tar.gz data/
|
||||
|
||||
# Or use rsync
|
||||
rsync -av data/ /backup/starpunk/
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
See [docs/standards/development-setup.md](docs/standards/development-setup.md) for detailed setup.
|
||||
|
||||
```bash
|
||||
# Install dev dependencies
|
||||
uv pip install -r requirements-dev.txt
|
||||
|
||||
# Run tests
|
||||
.venv/bin/pytest
|
||||
|
||||
# Format code
|
||||
.venv/bin/black starpunk/ tests/
|
||||
|
||||
# Lint
|
||||
.venv/bin/flake8 starpunk/ tests/
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
StarPunk uses a hybrid storage approach:
|
||||
- **Notes content**: Markdown files (portable, human-readable)
|
||||
- **Metadata**: SQLite database (fast queries)
|
||||
|
||||
This gives you both portability AND performance.
|
||||
|
||||
See [docs/architecture/](docs/architecture/) for complete documentation.
|
||||
|
||||
## IndieWeb Compliance
|
||||
|
||||
StarPunk implements:
|
||||
- [Micropub](https://micropub.spec.indieweb.org/) - Publishing API
|
||||
- [IndieAuth](https://www.w3.org/TR/indieauth/) - Authentication
|
||||
- [Microformats2](http://microformats.org/) - h-entry, h-card, h-feed markup
|
||||
- [RSS 2.0](https://www.rssboard.org/rss-specification) with Media RSS extensions
|
||||
- [ATOM 1.0](https://validator.w3.org/feed/docs/atom.html) - Syndication format
|
||||
- [JSON Feed 1.1](https://jsonfeed.org/version/1.1) - Modern feed format
|
||||
|
||||
## Deployment
|
||||
|
||||
### Production Setup
|
||||
|
||||
```bash
|
||||
# Install gunicorn
|
||||
uv pip install gunicorn
|
||||
|
||||
# Run with gunicorn
|
||||
.venv/bin/gunicorn -w 4 -b 127.0.0.1:8000 app:app
|
||||
|
||||
# Configure nginx/Caddy for HTTPS
|
||||
# Set up systemd for process management
|
||||
# Enable regular backups of data/ directory
|
||||
```
|
||||
|
||||
See [docs/standards/deployment-standards.md](docs/standards/deployment-standards.md) for details.
|
||||
| Topic | Where |
|
||||
|---|---|
|
||||
| How the docs are organized | [docs/DOCUMENTATION.md](docs/DOCUMENTATION.md) |
|
||||
| System overview | [docs/architecture/overview.md](docs/architecture/overview.md) |
|
||||
| Technology stack | [docs/architecture/technology-stack.md](docs/architecture/technology-stack.md) |
|
||||
| Database and migrations | [docs/architecture/database.md](docs/architecture/database.md) |
|
||||
| Deployment | [docs/architecture/deployment.md](docs/architecture/deployment.md) |
|
||||
| Authentication (OIDC, IndieAuth) | [docs/architecture/simplified-auth-architecture.md](docs/architecture/simplified-auth-architecture.md) |
|
||||
| IndieWeb protocols | [docs/architecture/indieweb-protocols.md](docs/architecture/indieweb-protocols.md) |
|
||||
| Feeds and crossposting | [docs/architecture/syndication-architecture.md](docs/architecture/syndication-architecture.md) |
|
||||
| Decisions (ADRs) | [docs/decisions/INDEX.md](docs/decisions/INDEX.md) |
|
||||
| Per-release designs and reports | [docs/design/](docs/design/) |
|
||||
| Backlog | [docs/projectplan/BACKLOG.md](docs/projectplan/BACKLOG.md) |
|
||||
| Development standards and release process | [docs/standards/INDEX.md](docs/standards/INDEX.md) |
|
||||
|
||||
## License
|
||||
|
||||
MIT License - see LICENSE file
|
||||
|
||||
## Credits
|
||||
|
||||
Built with:
|
||||
- [Flask](https://flask.palletsprojects.com/) - Web framework
|
||||
- [python-markdown](https://python-markdown.github.io/) - Markdown processing
|
||||
- [feedgen](https://feedgen.kiesow.be/) - RSS generation
|
||||
- [httpx](https://www.python-httpx.org/) - HTTP client
|
||||
- [IndieLogin](https://indielogin.com/) - Authentication service
|
||||
|
||||
## Contributing
|
||||
|
||||
This is a personal project optimized for single-user use. If you want additional features, consider forking!
|
||||
|
||||
## Support
|
||||
|
||||
- Documentation: [docs/](docs/)
|
||||
- Issues: GitHub Issues
|
||||
- IndieWeb: [indieweb.org](https://indieweb.org/)
|
||||
[MIT](LICENSE): use, change and share it for anything, including
|
||||
commercially, as long as the copyright and license notice stay with it.
|
||||
+2
-7
@@ -11,12 +11,11 @@
|
||||
# docker compose logs -f
|
||||
# docker compose down
|
||||
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
starpunk:
|
||||
# Container configuration
|
||||
image: starpunk:0.6.0
|
||||
# Built locally from this checkout (release images: see README)
|
||||
image: starpunk:local
|
||||
container_name: starpunk
|
||||
|
||||
# Build configuration
|
||||
@@ -49,10 +48,6 @@ services:
|
||||
- NOTES_PATH=/data/notes
|
||||
- DATABASE_PATH=/data/starpunk.db
|
||||
|
||||
# Application metadata
|
||||
- VERSION=0.6.0
|
||||
- ENVIRONMENT=production
|
||||
|
||||
# Volume mounts for persistent data
|
||||
# All application data stored in ./container-data on host
|
||||
volumes:
|
||||
|
||||
-103
@@ -1,103 +0,0 @@
|
||||
"""
|
||||
Development authentication module for StarPunk
|
||||
|
||||
WARNING: This module provides a development-only authentication mechanism
|
||||
that bypasses IndieLogin. It should NEVER be enabled in production.
|
||||
|
||||
This module is separate from production auth (starpunk/auth.py) to maintain
|
||||
clear architectural boundaries and enable easy security audits.
|
||||
|
||||
Security measures:
|
||||
- Only active when DEV_MODE=true
|
||||
- Returns 404 if DEV_MODE=false
|
||||
- Requires DEV_ADMIN_ME configuration
|
||||
- Logs prominent warnings
|
||||
- Cannot coexist with production SITE_URL
|
||||
- Visual warnings in UI
|
||||
|
||||
Functions:
|
||||
is_dev_mode: Check if development mode is enabled
|
||||
validate_dev_config: Validate development configuration
|
||||
create_dev_session: Create session without authentication
|
||||
"""
|
||||
|
||||
from flask import current_app
|
||||
|
||||
from starpunk.auth import create_session
|
||||
|
||||
|
||||
def is_dev_mode() -> bool:
|
||||
"""
|
||||
Check if development mode is enabled
|
||||
|
||||
Returns:
|
||||
True if DEV_MODE is enabled, False otherwise
|
||||
|
||||
Example:
|
||||
>>> from starpunk.dev_auth import is_dev_mode
|
||||
>>> if is_dev_mode():
|
||||
... print("Development mode active")
|
||||
"""
|
||||
return current_app.config.get("DEV_MODE", False)
|
||||
|
||||
|
||||
def validate_dev_config() -> None:
|
||||
"""
|
||||
Validate development mode configuration
|
||||
|
||||
Checks that DEV_MODE configuration is valid and safe:
|
||||
- DEV_ADMIN_ME must be set if DEV_MODE is true
|
||||
- Warns if DEV_MODE is enabled with production-like SITE_URL
|
||||
|
||||
Raises:
|
||||
ValueError: If DEV_MODE is true but DEV_ADMIN_ME is not set
|
||||
|
||||
Logs:
|
||||
WARNING: If DEV_MODE is enabled with HTTPS SITE_URL
|
||||
"""
|
||||
dev_mode = current_app.config.get("DEV_MODE", False)
|
||||
|
||||
if dev_mode:
|
||||
# Require DEV_ADMIN_ME
|
||||
dev_admin_me = current_app.config.get("DEV_ADMIN_ME")
|
||||
if not dev_admin_me:
|
||||
raise ValueError("DEV_MODE=true requires DEV_ADMIN_ME to be set")
|
||||
|
||||
# Warn if production-like configuration detected
|
||||
site_url = current_app.config.get("SITE_URL", "")
|
||||
if site_url.startswith("https://"):
|
||||
current_app.logger.warning(
|
||||
"WARNING: DEV_MODE is enabled with production SITE_URL. "
|
||||
"This is likely a misconfiguration. "
|
||||
"DEV_MODE should only be used in local development."
|
||||
)
|
||||
|
||||
|
||||
def create_dev_session(me: str) -> str:
|
||||
"""
|
||||
Create development session without authentication
|
||||
|
||||
WARNING: This bypasses IndieLogin authentication entirely.
|
||||
Only use in development environments.
|
||||
|
||||
Args:
|
||||
me: User identity URL (from DEV_ADMIN_ME config)
|
||||
|
||||
Returns:
|
||||
Session token (same format as production sessions)
|
||||
|
||||
Logs:
|
||||
WARNING: Logs that dev session was created without authentication
|
||||
|
||||
Example:
|
||||
>>> token = create_dev_session("https://example.com")
|
||||
>>> # Session created without IndieLogin verification
|
||||
"""
|
||||
current_app.logger.warning(
|
||||
f"DEV MODE: Creating session for {me} without authentication. "
|
||||
f"This should NEVER happen in production!"
|
||||
)
|
||||
|
||||
# Use production session creation (same session format)
|
||||
# This ensures dev sessions work identically to production
|
||||
return create_session(me)
|
||||
+29
-71
@@ -1,82 +1,40 @@
|
||||
# Architecture Documentation Index
|
||||
|
||||
This directory contains architectural documentation, system design overviews, component diagrams, and architectural patterns for StarPunk CMS.
|
||||
This directory describes the **current** design of StarPunk (v2.10.1). Per
|
||||
[../DOCUMENTATION.md](../DOCUMENTATION.md) it is updated at the end of each
|
||||
release. Decisions and their history are in [../decisions/](../decisions/);
|
||||
per-release designs and reports are in [../design/](../design/).
|
||||
|
||||
## Core Architecture
|
||||
## Documents
|
||||
|
||||
### System Overview
|
||||
- **[overview.md](overview.md)** - Complete system architecture and design principles
|
||||
- **[technology-stack.md](technology-stack.md)** - Current technology stack and dependencies
|
||||
- **[technology-stack-legacy.md](technology-stack-legacy.md)** - Historical technology decisions
|
||||
| Document | Description |
|
||||
|----------|-------------|
|
||||
| [overview.md](overview.md) | System overview: purpose, request flow, modules and routes, background workers, storage, auth model, URL surface, caching, security posture |
|
||||
| [technology-stack.md](technology-stack.md) | Current runtime, Python dependencies, standards implemented, testing and CI |
|
||||
| [database.md](database.md) | SQLite schema after migrations 001–019, migration runner and its lock, connection pool |
|
||||
| [deployment.md](deployment.md) | Container image stages, CI workflow, release process, `/data` layout, reverse proxy, all environment variables |
|
||||
| [simplified-auth-architecture.md](simplified-auth-architecture.md) | Admin login via OpenID Connect and the built-in IndieAuth server that issues Micropub tokens (ADR-064) |
|
||||
| [indieweb-protocols.md](indieweb-protocols.md) | How Micropub, IndieAuth, Webmention, microformats2, WebSub and the feeds are implemented, with compliance status and known gaps |
|
||||
| [syndication-architecture.md](syndication-architecture.md) | Crossposting (POSSE) to GoToSocial, Bluesky and Threads: syndications table, worker, reconcile rules |
|
||||
| [technology-stack-legacy.md](technology-stack-legacy.md) | Historical: the original v0.x technology choices (IndieLogin era); not current |
|
||||
|
||||
### Feature-Specific Architecture
|
||||
## Where to start
|
||||
|
||||
#### IndieAuth & Authentication
|
||||
- **[indieauth-assessment.md](indieauth-assessment.md)** - Assessment of IndieAuth implementation
|
||||
- **[indieauth-client-diagnosis.md](indieauth-client-diagnosis.md)** - IndieAuth client diagnostic analysis
|
||||
- **[indieauth-endpoint-discovery.md](indieauth-endpoint-discovery.md)** - Endpoint discovery architecture
|
||||
- **[indieauth-identity-page.md](indieauth-identity-page.md)** - Identity page architecture
|
||||
- **[indieauth-questions-answered.md](indieauth-questions-answered.md)** - Architectural Q&A for IndieAuth
|
||||
- **[indieauth-removal-architectural-review.md](indieauth-removal-architectural-review.md)** - Review of custom IndieAuth removal
|
||||
- **[indieauth-removal-implementation-guide.md](indieauth-removal-implementation-guide.md)** - Implementation guide for removal
|
||||
- **[indieauth-removal-phases.md](indieauth-removal-phases.md)** - Phased removal approach
|
||||
- **[indieauth-removal-plan.md](indieauth-removal-plan.md)** - Overall removal plan
|
||||
- **[indieauth-token-verification-diagnosis.md](indieauth-token-verification-diagnosis.md)** - Token verification diagnostic analysis
|
||||
- **[simplified-auth-architecture.md](simplified-auth-architecture.md)** - Simplified authentication architecture
|
||||
- **[endpoint-discovery-answers.md](endpoint-discovery-answers.md)** - Endpoint discovery implementation Q&A
|
||||
- **New to the code**: overview.md, then technology-stack.md.
|
||||
- **Changing the schema**: database.md (migrations must be safe inside one
|
||||
transaction).
|
||||
- **Releasing or operating the site**: deployment.md.
|
||||
- **Working on auth, Micropub, Webmention or WebSub**:
|
||||
simplified-auth-architecture.md and indieweb-protocols.md, plus the ADRs
|
||||
they link to.
|
||||
|
||||
#### Database & Migrations
|
||||
- **[database-migration-architecture.md](database-migration-architecture.md)** - Database migration system architecture
|
||||
- **[migration-fix-quick-reference.md](migration-fix-quick-reference.md)** - Quick reference for migration fixes
|
||||
- **[migration-race-condition-answers.md](migration-race-condition-answers.md)** - Race condition resolution Q&A
|
||||
## Related documentation
|
||||
|
||||
#### Syndication
|
||||
- **[syndication-architecture.md](syndication-architecture.md)** - RSS feed and syndication architecture
|
||||
|
||||
## Version-Specific Architecture
|
||||
|
||||
### v1.0.0
|
||||
- **[v1.0.0-release-validation.md](v1.0.0-release-validation.md)** - Release validation architecture
|
||||
|
||||
### v1.1.0
|
||||
- **[v1.1.0-feature-architecture.md](v1.1.0-feature-architecture.md)** - Feature architecture for v1.1.0
|
||||
- **[v1.1.0-implementation-decisions.md](v1.1.0-implementation-decisions.md)** - Implementation decisions
|
||||
- **[v1.1.0-search-ui-validation.md](v1.1.0-search-ui-validation.md)** - Search UI validation
|
||||
- **[v1.1.0-validation-report.md](v1.1.0-validation-report.md)** - Overall validation report
|
||||
|
||||
### v1.1.1
|
||||
- **[v1.1.1-architecture-overview.md](v1.1.1-architecture-overview.md)** - Architecture overview for v1.1.1
|
||||
|
||||
## Phase Documentation
|
||||
- **[phase1-completion-guide.md](phase1-completion-guide.md)** - Phase 1 completion guide
|
||||
- **[phase-5-validation-report.md](phase-5-validation-report.md)** - Phase 5 validation report
|
||||
|
||||
## Review Documentation
|
||||
- **[review-v1.0.0-rc.5.md](review-v1.0.0-rc.5.md)** - Architectural review of v1.0.0-rc.5
|
||||
|
||||
## How to Use This Documentation
|
||||
|
||||
### For New Developers
|
||||
1. Start with **overview.md** to understand the system
|
||||
2. Review **technology-stack.md** for current technologies
|
||||
3. Read feature-specific architecture docs relevant to your work
|
||||
|
||||
### For Architects
|
||||
1. Review version-specific architecture for historical context
|
||||
2. Consult feature-specific docs when making changes
|
||||
3. Update relevant docs when architecture changes
|
||||
|
||||
### For Contributors
|
||||
1. Read **overview.md** for system understanding
|
||||
2. Consult specific architecture docs for areas you're working on
|
||||
3. Follow patterns documented in architecture files
|
||||
|
||||
## Related Documentation
|
||||
- **[../decisions/](../decisions/)** - Architectural Decision Records (ADRs)
|
||||
- **[../design/](../design/)** - Detailed design documents
|
||||
- **[../standards/](../standards/)** - Coding standards and conventions
|
||||
- [../decisions/](../decisions/): Architectural Decision Records (ADRs)
|
||||
- [../design/](../design/): designs, questions and reports per version
|
||||
- [../standards/](../standards/): coding, testing and versioning standards
|
||||
- [../projectplan/](../projectplan/): backlog and release plans
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-11-25
|
||||
**Maintained By**: Documentation Manager Agent
|
||||
**Last Updated**: 2026-11-03 (v2.10.1)
|
||||
@@ -0,0 +1,241 @@
|
||||
# Database Architecture
|
||||
|
||||
**Version**: 2.11.0 (2026-11-03)
|
||||
**Status**: Current. Verified against a database created by `create_app()`
|
||||
with migrations 001–019.
|
||||
|
||||
StarPunk uses one SQLite file, `starpunk.db` under `DATA_PATH` (`/data` in
|
||||
the container), in WAL mode. Note *content* is not in the database: it lives
|
||||
in Markdown files (`notes/YYYY/MM/<slug>.md`) and the `notes` table points to
|
||||
them. See [overview.md](overview.md) for how the rest of the system uses it.
|
||||
|
||||
## How the schema is built
|
||||
|
||||
On every start, `init_db()` (`starpunk/database/init.py`) does two things:
|
||||
|
||||
1. Runs `INITIAL_SCHEMA_SQL` (`starpunk/database/schema.py`). This is the
|
||||
frozen v1.0.0 baseline, written with `CREATE … IF NOT EXISTS`: `notes`,
|
||||
`sessions`, `tokens`, `authorization_codes`, `auth_state` and their
|
||||
indexes. It must not be changed; every later change is a migration.
|
||||
2. Runs `run_migrations()` (`starpunk/migrations.py`) over the SQL files in
|
||||
`migrations/`, in file-name order.
|
||||
|
||||
### Migration runner
|
||||
|
||||
- Applied migrations are recorded in `schema_migrations`
|
||||
(`migration_name`, `applied_at`).
|
||||
- **Locking**: all four gunicorn processes run `create_app()` at the same
|
||||
time. Each opens a direct (unpooled) connection with a 30-second busy
|
||||
timeout and runs `BEGIN IMMEDIATE`, which takes SQLite's write lock. The
|
||||
whole run (creating `schema_migrations`, checking what is pending, applying
|
||||
every pending migration and recording it) happens in that one transaction
|
||||
and is committed at the end, so a migration is applied once and atomically.
|
||||
Statements are executed one by one with `conn.execute()` (split with
|
||||
`sqlite3.complete_statement`, so trigger bodies stay whole), never with
|
||||
`executescript()`, which would commit and drop the lock part-way (the cause
|
||||
of the v2.8.0 deploy failure fixed in v2.8.1).
|
||||
- A process that finds the database locked retries with exponential backoff
|
||||
(up to 10 attempts, 120 seconds in total), then finds everything already
|
||||
applied and returns.
|
||||
- A failing migration rolls the whole run back and raises `MigrationError`,
|
||||
which stops the app from starting.
|
||||
- **Fresh databases**: when `schema_migrations` is empty, the runner checks
|
||||
each of the early migrations (001–003) against the actual schema and only
|
||||
records those whose effect is already present; migration 002 is reduced to
|
||||
creating its indexes. All later migrations run normally.
|
||||
|
||||
Background: [ADR-031](../decisions/ADR-031-database-migration-system-redesign.md),
|
||||
[ADR-032](../decisions/ADR-032-initial-schema-sql-implementation.md),
|
||||
[ADR-033](../decisions/ADR-033-database-migration-redesign.md),
|
||||
[ADR-037](../decisions/ADR-037-migration-race-condition-fix.md),
|
||||
[ADR-041](../decisions/ADR-041-database-migration-conflict-resolution.md).
|
||||
|
||||
### Migrations
|
||||
|
||||
| File | Version | Change |
|
||||
|------|---------|--------|
|
||||
| `001_add_code_verifier_to_auth_state.sql` | 0.8 | `auth_state.code_verifier` (removed again by 003) |
|
||||
| `002_secure_tokens_and_authorization_codes.sql` | 1.0.0-rc | Hashed `tokens` and `authorization_codes` (dropped by 004) |
|
||||
| `003_remove_code_verifier_from_auth_state.sql` | 1.0 | Drops `auth_state.code_verifier` |
|
||||
| `004_drop_token_tables.sql` | 1.0 | Drops `tokens` and `authorization_codes` |
|
||||
| `005_add_fts5_search.sql` | 1.1 | `notes_fts` (FTS5) and the `notes_fts_delete` trigger |
|
||||
| `006_add_author_profile.sql` | 1.2 | `author_profile` (h-card cache) |
|
||||
| `007_add_media_support.sql` | 1.2 | `media`, `note_media` |
|
||||
| `008_add_tags.sql` | 1.3 | `tags`, `note_tags` |
|
||||
| `009_add_media_variants.sql` | 1.4 | `media_variants` |
|
||||
| `010_add_webmentions.sql` | 1.6 | `notes.in_reply_to`, `webmentions`, `webmention_queue`, `reply_contexts` |
|
||||
| `011_oidc_login_and_indieauth_server.sql` | 2.0 | `oidc_login_state`, `indieauth_codes`, `indieauth_tokens` |
|
||||
| `012_add_syndications.sql` | 2.1 | `syndications` |
|
||||
| `013_add_likes_reposts.sql` | 2.5 | `notes.like_of`, `notes.repost_of` |
|
||||
| `014_webmention_source_details.sql` | 2.8 | `webmentions.source_post_url`, `webmentions.source_published` |
|
||||
| `015_bookmarks_and_content_format.sql` | 2.9 | `notes.bookmark_of`, `notes.content_format` |
|
||||
| `016_websub.sql` | 2.10.0 | `websub_subscriptions`, `websub_intents`, `websub_deliveries`, `site_state` |
|
||||
| `017_websub_hardening.sql` | 2.10.1 | `websub_intents.request_id`, `requested_topic`, `callback_ip`; `websub_subscriptions.callback_ip` |
|
||||
| `018_foreign_key_cleanup.sql` | 2.11.0 | Removes rows orphaned while foreign keys weren't enforced |
|
||||
| `019_shared_rate_limits.sql` | 2.11.0 | `rate_limits` (counters shared by all processes) |
|
||||
|
||||
New migrations are numbered `NNN_description.sql`, must be safe to run inside
|
||||
one transaction, and should use `IF NOT EXISTS` where SQLite allows it.
|
||||
|
||||
## Tables
|
||||
|
||||
### Content
|
||||
|
||||
**`notes`**: one row per note; the content is in the file at `file_path`.
|
||||
|
||||
| Column | Purpose |
|
||||
|--------|---------|
|
||||
| `id`, `slug` (unique), `file_path` (unique) | Identity and location of the Markdown file (relative to `DATA_PATH`) |
|
||||
| `published` | 0 = draft, 1 = public |
|
||||
| `created_at`, `updated_at` | Timestamps (UTC) |
|
||||
| `deleted_at` | Set on soft delete (file moved to `.trash/`); cleared by undelete |
|
||||
| `content_hash` | SHA-256 of the file content |
|
||||
| `in_reply_to`, `like_of`, `repost_of`, `bookmark_of` | URL the note responds to; set for replies, likes, reposts, bookmarks |
|
||||
| `content_format` | `'html'` when a Micropub client sent HTML (so `q=source` returns HTML); NULL means Markdown |
|
||||
|
||||
Indexes on `created_at DESC`, `published`, `slug`, `deleted_at`, `like_of`.
|
||||
|
||||
**`notes_fts`**: FTS5 virtual table (`slug` unindexed, `title`, `content`;
|
||||
`porter unicode61` tokenizer), with its shadow tables `notes_fts_config`,
|
||||
`_content`, `_data`, `_docsize`, `_idx`. Rows use the note `id` as `rowid`.
|
||||
Inserts and updates are done by `starpunk/search.py` (the database cannot
|
||||
read note files); the trigger `notes_fts_delete` removes the row when a note
|
||||
row is deleted. The index is rebuilt at start-up if it is empty.
|
||||
|
||||
**`tags`** (`name` normalised and unique, `display_name` as written) and
|
||||
**`note_tags`** (`note_id`, `tag_id`, unique pair, cascading deletes).
|
||||
|
||||
**`media`**: an uploaded image: original `filename`, `stored_filename`
|
||||
(UUID), `path` (`YYYY/MM/<uuid>.<ext>` under `media/`), `mime_type`, `size`,
|
||||
`width`, `height`, `uploaded_at`.
|
||||
|
||||
**`media_variants`**: resized copies per image (`variant_type` one of
|
||||
`thumb`, `small`, `medium`, `large`, `original`; `path`, dimensions, size);
|
||||
unique per (`media_id`, `variant_type`), deleted with the image.
|
||||
|
||||
**`note_media`**: attaches media to notes with `display_order` and
|
||||
`caption` (alt text); unique per (`note_id`, `media_id`).
|
||||
|
||||
### Authentication
|
||||
|
||||
**`sessions`**: admin sessions. `session_token_hash` (SHA-256 of the cookie
|
||||
value), `me`, `created_at`, `expires_at`, `last_used_at`, `user_agent`,
|
||||
`ip_address`.
|
||||
|
||||
**`oidc_login_state`**: one row per OIDC login in progress: `state`
|
||||
(primary key), `nonce`, `code_verifier` (PKCE), `next_url`, `expires_at`.
|
||||
Consumed once on callback.
|
||||
|
||||
**`indieauth_codes`**: authorization codes issued to Micropub clients:
|
||||
`code_hash`, `client_id`, `redirect_uri`, `scope`, `code_challenge`,
|
||||
`expires_at`, `used_at` (single use).
|
||||
|
||||
**`indieauth_tokens`**: access tokens: `token_hash`, `client_id`, `scope`,
|
||||
`created_at`, `expires_at`, `last_used_at`, `revoked_at`. Listed and revoked
|
||||
in `/admin/tokens`.
|
||||
|
||||
**Legacy tables, unused**: `auth_state` (v1 IndieLogin login state), `tokens`
|
||||
and `authorization_codes` (v1 token tables). No code reads or writes them.
|
||||
Migration 004 drops `tokens` and `authorization_codes`, but
|
||||
`INITIAL_SCHEMA_SQL` recreates them (empty) on the next start, so a running
|
||||
database normally contains them. They are harmless; v2 auth uses the tables
|
||||
above.
|
||||
|
||||
### IndieWeb interactions
|
||||
|
||||
**`webmentions`**: received webmentions, one per (`source_url`,
|
||||
`target_url`).
|
||||
|
||||
| Column | Purpose |
|
||||
|--------|---------|
|
||||
| `target_note_id` | The note mentioned (SET NULL if the note is deleted) |
|
||||
| `mention_type` | `reply`, `like`, `repost`, `bookmark`, `mention` |
|
||||
| `author_name`, `author_url`, `author_photo`, `content_snippet` | From the source's mf2 (content sanitised) |
|
||||
| `source_post_url`, `source_published` | The source entry's own `u-url` and `dt-published` |
|
||||
| `validation_status` | `pending`, `valid`, `invalid`, `error` (set by the worker) |
|
||||
| `moderation_status` | `pending`, `approved`, `hidden`, `deleted` (auto-approved when `WEBMENTION_AUTO_APPROVE`) |
|
||||
| `validation_error`, `validated_at`, `received_at`, `updated_at` | Bookkeeping |
|
||||
|
||||
**`webmention_queue`**: outgoing webmentions, one per (`source_url`,
|
||||
`target_url`): `source_note_id`, discovered `endpoint_url`, `status`
|
||||
(`pending`, `sending`, `sent`, `failed`, `no_endpoint`), `retry_count`,
|
||||
`next_retry_at`, `last_error`, `started_at`, `sent_at`.
|
||||
|
||||
**`reply_contexts`**: cache of the posts that replies (and other responses)
|
||||
point to: `url` (unique), `title`, `content_snippet`, `published_at`, author
|
||||
fields, `site_name`, `fetch_status` (`pending`, `fetched`, `error`),
|
||||
`fetch_error`, `fetched_at`.
|
||||
|
||||
**`author_profile`**: cache of the owner's h-card discovered from `ADMIN_ME`:
|
||||
`me`, `name`, `photo`, `url`, `note`, `rel_me_links` (JSON), `discovered_at`,
|
||||
`cached_until` (24 hours).
|
||||
|
||||
### Crossposting
|
||||
|
||||
**`syndications`**: one row per (`note_id`, `target`), where `target` is a
|
||||
configured target id (GoToSocial, Bluesky, Threads). `enabled` says whether
|
||||
the note should be on that target; `remote_id` and `remote_url` identify the
|
||||
remote post. `state` (`pending`, `synced`, `failed`), `version` (bumped on
|
||||
every note change, so an edit made during a send is not lost), `attempts`,
|
||||
`last_error`, `next_attempt_at` and `claimed_until` (a 5-minute lease that
|
||||
lets any gunicorn process take the row exclusively) drive the worker. See
|
||||
[syndication-architecture.md](syndication-architecture.md).
|
||||
|
||||
### WebSub
|
||||
|
||||
**`websub_subscriptions`**: verified subscriptions to this site's topics:
|
||||
`topic`, `callback` (unique pair), `secret`, `lease_seconds`, `expires_at`,
|
||||
`verified_at`, `last_success_at`, `last_error`, `callback_ip` (the address
|
||||
the callback resolved to, for per-address limits).
|
||||
|
||||
**`websub_intents`**: subscribe and unsubscribe requests waiting for
|
||||
verification of intent: `topic`, `callback` (unique pair), `mode`, `secret`,
|
||||
`lease_seconds`, `attempts`, `next_attempt_at`, `request_id` (so verifying an
|
||||
older request cannot consume a newer one), `requested_topic` (the topic as
|
||||
sent, echoed in verification), `callback_ip`.
|
||||
|
||||
**`websub_deliveries`**: content waiting to be pushed: one row per
|
||||
subscription (built-in hub) or one per topic with `subscription_id` NULL (a
|
||||
publish ping to an external hub); partial unique indexes keep at most one
|
||||
pending row of each kind. `attempts`, `next_attempt_at`, `last_error`.
|
||||
|
||||
**`site_state`**: key/value pairs. `content_version` is bumped whenever a
|
||||
public note changes; every gunicorn process compares it to decide whether
|
||||
its cached feed data is stale.
|
||||
|
||||
### Bookkeeping
|
||||
|
||||
**`schema_migrations`**: applied migrations (see above).
|
||||
|
||||
**`rate_limits`** (v2.11.0): request counts for `/webmention` and `/websub`,
|
||||
one row per key (`<limiter>:<ip or domain>`) and time window, so every
|
||||
server process enforces one shared limit. Old windows are pruned as it goes.
|
||||
|
||||
## Connections
|
||||
|
||||
- **Request connections**: `starpunk/database/pool.py`. `init_pool()` creates
|
||||
one `ConnectionPool` per process (up to 5 idle connections, 10-second busy
|
||||
timeout; `DB_POOL_SIZE` and `DB_TIMEOUT` can be set only through the
|
||||
`create_app()` config dict). Connections use `sqlite3.Row` rows,
|
||||
`check_same_thread=False`, `PRAGMA journal_mode=WAL` and (since v2.11.0)
|
||||
`PRAGMA foreign_keys=ON`, so `ON DELETE CASCADE`/`SET NULL` run. `get_db()`
|
||||
stores one connection in `flask.g` per request or app context; on teardown
|
||||
it goes back to the pool, rolled back first if a transaction was left
|
||||
open.
|
||||
- **Direct connections**: the migration runner, the FTS start-up check, the
|
||||
webmention worker and sender, reply-context fetching and search helpers
|
||||
open their own `sqlite3.connect(db_path)` connections.
|
||||
- WAL mode lets readers carry on while one writer writes. The four gunicorn
|
||||
processes and the worker threads share the one file; writes are short, so
|
||||
contention is low.
|
||||
|
||||
## Backups
|
||||
|
||||
The application does not back itself up. A consistent copy needs the
|
||||
database (use SQLite's online backup, e.g. `sqlite3 starpunk.db ".backup …"`,
|
||||
because of WAL) together with `notes/` and `media/`; see
|
||||
[deployment.md](deployment.md).
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2026-11-03 (v2.10.1)
|
||||
@@ -0,0 +1,266 @@
|
||||
# Deployment Architecture
|
||||
|
||||
**Version**: 2.10.1 (2026-11-03)
|
||||
**Status**: Current
|
||||
|
||||
How StarPunk is built, configured and run in production. The production site
|
||||
is a single container behind Caddy on one host, managed with Ansible. This
|
||||
document stays generic: hostnames, credentials and inventory live in the
|
||||
deployment repository, not here.
|
||||
|
||||
## Overview
|
||||
|
||||
```
|
||||
git tag vX.Y.Z ──► Gitea Actions ──► test stage (pytest) ──► runtime image
|
||||
│ push :vX.Y.Z, :latest
|
||||
▼
|
||||
Gitea container registry
|
||||
│ pull (tag set in Ansible)
|
||||
▼
|
||||
Internet ──► Caddy (TLS, headers, X-Forwarded-For) ──► container :8000 (gunicorn × 4)
|
||||
│
|
||||
/data volume (host)
|
||||
```
|
||||
|
||||
## Container image (`Containerfile`)
|
||||
|
||||
A multi-stage build on `python:3.14-slim`; it works with Podman and Docker.
|
||||
|
||||
| Stage | Purpose |
|
||||
|-------|---------|
|
||||
| `builder` | Copies `uv` from `ghcr.io/astral-sh/uv`, creates `/opt/venv` and installs `requirements.txt` into it |
|
||||
| `test` | Copies the venv, installs `requirements-test.txt`, copies the source and runs `python -m pytest -q -p no:cacheprovider tests` with test values for `SESSION_SECRET`, `SITE_URL` and `ADMIN_ME`. The stage only builds if the tests pass. |
|
||||
| runtime (last, the default target) | Creates user `starpunk` (UID/GID 1000) and `/data/notes`, copies the venv and the application to `/app`, sets `DATA_PATH=/data`, `NOTES_PATH=/data/notes`, `DATABASE_PATH=/data/starpunk.db`, runs as `starpunk`, exposes port 8000 |
|
||||
|
||||
The runtime image's command is gunicorn:
|
||||
|
||||
```
|
||||
gunicorn --bind 0.0.0.0:8000 --workers 4 --worker-class sync
|
||||
--worker-tmp-dir /dev/shm --max-requests 1000 --max-requests-jitter 50
|
||||
--timeout 30 --graceful-timeout 30
|
||||
--access-logfile - --error-logfile - --log-level info app:app
|
||||
```
|
||||
|
||||
Each of the four worker processes builds its own Flask app, so each runs
|
||||
migrations (only one applies them, see [database.md](database.md)) and starts
|
||||
its own background threads (see [overview.md](overview.md#background-workers)).
|
||||
Workers are recycled after about 1000 requests; a recycled process starts
|
||||
new threads, and the file locks pass to another process if the holder exits.
|
||||
|
||||
**Health check**: the image's `HEALTHCHECK` runs every 30 seconds (3-second
|
||||
timeout, 10-second start period, 3 retries) and calls
|
||||
`http://localhost:8000/health` with httpx. `/health` returns
|
||||
`{"status": "ok", "version": …}` plus webmention-worker statistics, without
|
||||
authentication.
|
||||
|
||||
## CI/CD (`.gitea/workflows/build-container.yml`)
|
||||
|
||||
- **Triggers**: pushing a tag matching `vX.Y.Z`, or a manual run.
|
||||
- **Runner**: a Gitea Actions runner with no Docker daemon. Images are built
|
||||
by a rootless BuildKit daemon reached over its socket
|
||||
(`docker buildx create --driver remote`).
|
||||
- **Steps**:
|
||||
1. Log in to the Gitea container registry with the repository secret
|
||||
`REGISTRY_TOKEN` (a token with only the `write:package` scope; the
|
||||
automatic Actions token is refused by the registry).
|
||||
2. `docker buildx build --target test`: runs the test suite. A failure
|
||||
stops the workflow before anything is pushed.
|
||||
3. Build the runtime image and push it tagged with the git tag
|
||||
(e.g. `v2.10.1`) and `latest`. The image name is the repository path in
|
||||
lower case.
|
||||
4. Remove the builder and log out, even after a failure.
|
||||
|
||||
There is no CI on branches or pull requests; run the tests locally before
|
||||
merging (see [technology-stack.md](technology-stack.md#testing)).
|
||||
|
||||
## Release process
|
||||
|
||||
1. Finish the change on a branch, including the version bump in
|
||||
`starpunk/__init__.py` (`__version__`, `__version_info__`) and a
|
||||
`CHANGELOG.md` entry.
|
||||
2. Merge to `main`.
|
||||
3. Tag the merge commit `vX.Y.Z` and push the tag. CI tests, builds and
|
||||
pushes the image.
|
||||
4. In the Ansible-managed deployment, change the StarPunk image tag to the new
|
||||
version and apply it. The container is recreated from the new image; new
|
||||
migrations run automatically at start-up.
|
||||
5. Check `/health` reports the new version.
|
||||
|
||||
Rolling back means deploying the previous tag. Migrations are not reversed
|
||||
automatically, so a release with a migration needs a database backup first
|
||||
if a rollback might be needed.
|
||||
|
||||
## Runtime layout
|
||||
|
||||
### Data volume (`/data`)
|
||||
|
||||
A host directory is mounted at `/data`, owned by UID 1000. Everything
|
||||
persistent is there; the image itself holds no state.
|
||||
|
||||
| Path | Contents |
|
||||
|------|----------|
|
||||
| `starpunk.db` (+ `-wal`, `-shm`) | SQLite database in WAL mode |
|
||||
| `notes/YYYY/MM/<slug>.md` | Note content |
|
||||
| `.trash/YYYY/MM/` | Soft-deleted note files (restorable with Micropub undelete) |
|
||||
| `media/YYYY/MM/` | Uploaded images and their variants; `media/.tmp/` while processing |
|
||||
| `logs/starpunk.log` | Application log, rotated at 10 MB, 10 files kept |
|
||||
| `webmention-worker.lock`, `websub-worker.lock` | `flock` files choosing the process that runs each worker |
|
||||
| `threads_token.json` (+ `.lock`) | Current Threads access token, refreshed daily (only with Threads configured) |
|
||||
| `debug/` | Failed uploads, only with `DEBUG_SAVE_FAILED_UPLOADS=true`; cleaned up after `DEBUG_FILE_MAX_AGE_DAYS` |
|
||||
|
||||
The application does not make backups. A backup must include the database
|
||||
(copied with SQLite's online backup because of WAL), `notes/` and `media/`.
|
||||
Backups are handled by the deployment, outside the container.
|
||||
|
||||
### Reverse proxy
|
||||
|
||||
Caddy terminates TLS and proxies to the container's port 8000. Publish that
|
||||
port only where the proxy can reach it (for example on the host's loopback
|
||||
interface, as `compose.yaml` does), never publicly. Caddy adds security headers
|
||||
(HSTS, CSP, X-Frame-Options, Referrer-Policy and so on; see
|
||||
`Caddyfile.example`, and `nginx.conf.example` for nginx) and sets
|
||||
`X-Forwarded-For`.
|
||||
|
||||
Set `TRUSTED_PROXY=true` behind the proxy. With it, StarPunk takes the client
|
||||
address from the first entry of `X-Forwarded-For`, which the webmention and
|
||||
WebSub rate limits need; without it, every client appears as the proxy's
|
||||
address and shares one limit. Do not set it if the container port can be
|
||||
reached without the proxy, because the header could then be forged.
|
||||
|
||||
`SITE_URL` must be the public `https://` URL: it is used for absolute links,
|
||||
feed and WebSub topic URLs, the IndieAuth issuer and redirect URIs, and to
|
||||
decide whether the session cookie is `Secure`.
|
||||
|
||||
## Configuration
|
||||
|
||||
Configuration is read from environment variables (and a `.env` file if
|
||||
present) by `starpunk/config.py` at start-up. In production the variables
|
||||
come from the Ansible-managed environment, with secrets from its vault.
|
||||
`.env.example` documents them for local use. Booleans are `true`/`false`
|
||||
unless noted.
|
||||
|
||||
### Required
|
||||
|
||||
| Variable | Default | Notes |
|
||||
|----------|---------|-------|
|
||||
| `SESSION_SECRET` | none | Required always. Signs sessions; also Flask's `SECRET_KEY` unless `FLASK_SECRET_KEY` is set |
|
||||
| `SITE_URL` | `http://localhost:5000` | Public URL; a trailing `/` is added |
|
||||
| `ADMIN_ME` | none | Owner's identity URL (h-card, Micropub `me`). Required unless `DEV_MODE` |
|
||||
| `OIDC_ISSUER`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`, `OIDC_ADMIN_SUB` | empty | Admin login via OIDC. Required unless `DEV_MODE` (or testing) |
|
||||
|
||||
### Site and authentication
|
||||
|
||||
| Variable | Default | Notes |
|
||||
|----------|---------|-------|
|
||||
| `SITE_NAME` | `StarPunk` | |
|
||||
| `SITE_AUTHOR` | `Unknown` | |
|
||||
| `SITE_DESCRIPTION` | `A minimal IndieWeb CMS` | |
|
||||
| `SESSION_LIFETIME` | `30` | Days |
|
||||
| `FLASK_SECRET_KEY` | `SESSION_SECRET` | |
|
||||
| `OIDC_PROVIDER_NAME` | `single sign-on` | Label on the login button |
|
||||
| `INDIEAUTH_TOKEN_LIFETIME_DAYS` | `90` | Lifetime of Micropub access tokens |
|
||||
| `DEV_MODE` | `false` | Enables `/dev/login`; never in production |
|
||||
| `DEV_ADMIN_ME` | empty | Required when `DEV_MODE=true` |
|
||||
|
||||
`TOKEN_ENDPOINT` and `INDIELOGIN_URL` are v1 settings; if present they are
|
||||
ignored with a warning.
|
||||
|
||||
### Paths, runtime and logging
|
||||
|
||||
| Variable | Default | Notes |
|
||||
|----------|---------|-------|
|
||||
| `DATA_PATH` | `./data` (image: `/data`) | Root of all stored data |
|
||||
| `DATABASE_PATH` | `./data/starpunk.db` (image: `/data/starpunk.db`) | |
|
||||
| `NOTES_PATH` | `./data/notes` (image: `/data/notes`) | Created at start-up, but note files are always written to `DATA_PATH/notes` |
|
||||
| `FLASK_ENV` | `production` | Reported in `/health` and admin diagnostics; the image sets `production` |
|
||||
| `FLASK_DEBUG` | `0` | Debug off unless `1` (`flask run --debug` sets it locally); the image sets `0` |
|
||||
| `LOG_LEVEL` | `INFO` | `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` |
|
||||
| `VERSION` | package `__version__` | Overrides the reported version; normally unset |
|
||||
|
||||
### Feeds
|
||||
|
||||
| Variable | Default | Notes |
|
||||
|----------|---------|-------|
|
||||
| `FEED_MAX_ITEMS` | `50` | Notes per feed |
|
||||
| `FEED_CACHE_SECONDS` | `300` | Cache TTL and `Cache-Control: max-age` |
|
||||
| `FEED_CACHE_ENABLED` | `true` | |
|
||||
| `FEED_CACHE_MAX_SIZE` | `50` | Cached feed documents per process |
|
||||
|
||||
### Webmention
|
||||
|
||||
| Variable | Default | Notes |
|
||||
|----------|---------|-------|
|
||||
| `WEBMENTION_ENABLED` | `true` | Endpoint, `Link` header and sending |
|
||||
| `WEBMENTION_WORKER_ENABLED` | `true` | |
|
||||
| `WEBMENTION_POLL_INTERVAL` | `10` | Seconds |
|
||||
| `WEBMENTION_ERROR_BACKOFF` | `60` | Seconds after a worker error |
|
||||
| `WEBMENTION_BATCH_SIZE` | `5` | Items per pass |
|
||||
| `WEBMENTION_AUTO_APPROVE` | `true` | Valid webmentions are shown without moderation |
|
||||
| `WEBMENTION_RATE_LIMIT_ENABLED` | `true` | |
|
||||
| `WEBMENTION_RATE_LIMIT_IP` | `60` | Requests per window per client IP |
|
||||
| `WEBMENTION_RATE_LIMIT_DOMAIN` | `10` | Requests per window per source domain |
|
||||
| `WEBMENTION_RATE_LIMIT_WINDOW` | `60` | Seconds |
|
||||
| `TRUSTED_PROXY` | `false` | Use `X-Forwarded-For` for the client IP (see above) |
|
||||
|
||||
### WebSub
|
||||
|
||||
| Variable | Default | Notes |
|
||||
|----------|---------|-------|
|
||||
| `WEBSUB_ENABLED` | `true` | Discovery links, hub and worker |
|
||||
| `WEBSUB_HUB_URL` | empty | Use an external hub (StarPunk then sends publish pings) instead of the built-in one |
|
||||
| `WEBSUB_MAX_SUBSCRIPTIONS` | `1000` | Total subscriptions |
|
||||
| `WEBSUB_MAX_PER_HOST` | `25` | Subscriptions per callback host |
|
||||
| `WEBSUB_POLL_INTERVAL` | `5` | Seconds |
|
||||
|
||||
### Crossposting (POSSE)
|
||||
|
||||
A target is enabled when its credentials are set; the syndication worker
|
||||
starts only if at least one is.
|
||||
|
||||
| Variable | Default | Notes |
|
||||
|----------|---------|-------|
|
||||
| `GOTOSOCIAL_URL`, `GOTOSOCIAL_ACCESS_TOKEN` | empty | GoToSocial or any Mastodon-API server |
|
||||
| `GOTOSOCIAL_NAME` | `GoToSocial` | Label in the UI and `syndicate-to` |
|
||||
| `GOTOSOCIAL_VISIBILITY` | `public` | |
|
||||
| `GOTOSOCIAL_MAX_CHARS` | `5000` | |
|
||||
| `GOTOSOCIAL_MAX_MEDIA` | `4` | |
|
||||
| `BLUESKY_HANDLE`, `BLUESKY_APP_PASSWORD` | empty | |
|
||||
| `BLUESKY_SERVICE` | `https://bsky.social` | |
|
||||
| `BLUESKY_NAME` | `Bluesky` | |
|
||||
| `BLUESKY_MAX_CHARS` | `300` | |
|
||||
| `THREADS_ACCESS_TOKEN` | empty | Long-lived token; refreshed automatically into `threads_token.json` |
|
||||
| `THREADS_USERNAME` | empty | |
|
||||
| `THREADS_NAME` | `Threads` | |
|
||||
| `THREADS_MAX_CHARS` | `500` | |
|
||||
| `THREADS_MAX_MEDIA` | `4` | |
|
||||
| `SYNDICATION_POLL_INTERVAL` | `10` | Seconds |
|
||||
|
||||
### Debugging
|
||||
|
||||
The `METRICS_*` variables were removed in v2.12.0 and are ignored if set.
|
||||
|
||||
| Variable | Default | Notes |
|
||||
|----------|---------|-------|
|
||||
| `DEBUG_SAVE_FAILED_UPLOADS` | `false` | Keep failed uploads in `debug/` |
|
||||
| `DEBUG_FILE_MAX_AGE_DAYS` | `7` | |
|
||||
| `DEBUG_FILE_MAX_SIZE_MB` | `100` | |
|
||||
|
||||
Fixed values: the maximum request body is 50 MB (`MAX_CONTENT_LENGTH`). The
|
||||
connection pool size and timeout (`DB_POOL_SIZE`, `DB_TIMEOUT`) are not read
|
||||
from the environment.
|
||||
|
||||
## Running locally
|
||||
|
||||
```bash
|
||||
uv venv && uv pip install -r requirements-dev.txt
|
||||
cp .env.example .env # set SESSION_SECRET; DEV_MODE=true and DEV_ADMIN_ME for local login
|
||||
flask --app app.py run # or: gunicorn app:app
|
||||
```
|
||||
|
||||
`compose.yaml` builds and runs the image with `./container-data` mounted at
|
||||
`/data`; its image tag and `VERSION` values are out of date, so override them
|
||||
or build with `podman build -t starpunk .`.
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2026-11-03 (v2.10.1)
|
||||
@@ -0,0 +1,524 @@
|
||||
# IndieWeb and W3C Protocols
|
||||
|
||||
How StarPunk implements each protocol it speaks, as of v2.10.1, with its
|
||||
compliance status and the gaps that remain. Each section links to the
|
||||
decision records and review documents behind it.
|
||||
|
||||
| Protocol | Role | Status | Main code |
|
||||
|---|---|---|---|
|
||||
| [Micropub](#micropub) | Server (posts and media) | Reviewed against the Recommendation in v2.9.0; deliberate deviations listed | `starpunk/micropub.py`, `starpunk/routes/micropub.py` |
|
||||
| [IndieAuth](#indieauth) | Authorization server for Micropub clients | Authorization code + PKCE, metadata, revocation | `starpunk/indieauth.py`, `starpunk/routes/indieauth.py` |
|
||||
| [Webmention](#webmention) | Sender and receiver | Every MUST and SHOULD checked in v2.7.0 and v2.7.2; known limits listed | `starpunk/webmention_*.py`, `starpunk/routes/webmention.py` |
|
||||
| [Microformats2](#microformats2) | Publisher and consumer | Reviewed in v2.8.0 | `templates/`, `starpunk/mf2_utils.py`, `starpunk/author_discovery.py` |
|
||||
| [WebSub](#websub) | Publisher and built-in hub (own topics only) | Adversarial review in v2.10.1 | `starpunk/websub.py`, `starpunk/websub_worker.py`, `starpunk/routes/websub.py` |
|
||||
| [Feeds](#feeds) | RSS 2.0, Atom 1.0, JSON Feed 1.1, OPML 2.0 | Stable | `starpunk/feeds/`, `starpunk/routes/public.py` |
|
||||
|
||||
Crossposting to other networks (POSSE) is covered in
|
||||
[syndication-architecture.md](syndication-architecture.md).
|
||||
|
||||
## Shared infrastructure: the SSRF-safe fetcher
|
||||
|
||||
Webmention (fetching sources, discovering endpoints, sending), WebSub
|
||||
(verification and delivery) and Micropub (downloading remote photos) all
|
||||
make requests to URLs chosen by someone else. They use
|
||||
`starpunk/webmention_ssrf.py`:
|
||||
|
||||
- `validate_url_for_fetch()`: http(s) only. Every address from
|
||||
`getaddrinfo` must be public. Private, loopback, link-local, multicast,
|
||||
reserved, unspecified, CGNAT (`100.64.0.0/10`), `0.0.0.0/8` and
|
||||
IPv4-mapped forms are blocked. It returns the checked address, preferring
|
||||
IPv4. `DEV_MODE` allows localhost.
|
||||
- **DNS pinning** (v2.7.2): each request connects to the checked IP, with
|
||||
the original host in the `Host` header and as the TLS SNI name. The
|
||||
certificate is still verified against the real hostname, so a rebinding
|
||||
DNS server can't send the second lookup elsewhere.
|
||||
- `safe_get()` follows up to 10 redirects by hand and checks each hop. The
|
||||
response is capped at 1 MB by default.
|
||||
- `safe_post()` doesn't follow redirects.
|
||||
- **Hard deadlines** (v2.10.1): the whole exchange, not each read, is
|
||||
limited to 10 s. When the deadline passes, the client is closed, so a
|
||||
server trickling bytes can't hold a worker.
|
||||
- DNS resolution failures raise `ResolutionError` (a `ValueError`), which
|
||||
callers can treat as temporary.
|
||||
- User-Agent: `StarPunk/<version> Webmention (+<SITE_URL>)`.
|
||||
|
||||
Workers that drain queues (Webmention, WebSub) run as one daemon thread per
|
||||
gunicorn process. Each takes a non-blocking `flock` on a lock file in the
|
||||
data directory, so only one process works the queue at a time and another
|
||||
takes over if it dies.
|
||||
|
||||
---
|
||||
|
||||
## Micropub
|
||||
|
||||
Spec: https://www.w3.org/TR/micropub/. Decisions:
|
||||
[ADR-028](../decisions/ADR-028-micropub-implementation.md),
|
||||
[ADR-029](../decisions/ADR-029-micropub-indieauth-integration.md),
|
||||
[ADR-057](../decisions/ADR-057-media-attachment-model.md) (photos),
|
||||
[ADR-067](../decisions/ADR-067-micropub-update-delete.md) (update/delete),
|
||||
[ADR-068](../decisions/ADR-068-likes-and-reposts.md) (likes/reposts).
|
||||
Review: [v2.9.0 Micropub review](../design/v2.9.0/2026-11-03-micropub-review.md).
|
||||
|
||||
### Endpoints and discovery
|
||||
|
||||
| Endpoint | Purpose |
|
||||
|---|---|
|
||||
| `GET /micropub?q=...` | Queries |
|
||||
| `POST /micropub` | Create, update, delete, undelete |
|
||||
| `POST /micropub/media` (and legacy `POST /media`) | Media endpoint |
|
||||
|
||||
Every page has `<link rel="micropub">` plus the IndieAuth discovery links
|
||||
(see [IndieAuth](#indieauth)). Authentication, token placement, scopes and
|
||||
error codes are described in
|
||||
[simplified-auth-architecture.md §3](simplified-auth-architecture.md#3-token-use-at-the-micropub-and-media-endpoints).
|
||||
|
||||
### Create
|
||||
|
||||
Accepts form-encoded, multipart (with photo files) and JSON bodies. Only
|
||||
`h=entry` / `type: ["h-entry"]` is accepted; other types get 400. Success is
|
||||
**201** with `Location` set to the permalink `/note/<slug>`.
|
||||
|
||||
| Property / command | Handling |
|
||||
|---|---|
|
||||
| `content` | String, or `{"html": ...}` / `{"text": ...}`. Text is preferred. HTML-only content is stored with `content_format = 'html'` (migration 015) and rendered through `render_markdown()` (Markdown, then nh3 allowlist sanitization) |
|
||||
| `summary` | Used as the text when there is no `content` |
|
||||
| `name` | Stored as a `# name` heading on the first line (StarPunk has no title column) |
|
||||
| `category` | Tags (strings only; person tags as h-card objects are ignored) |
|
||||
| `in-reply-to`, `like-of`, `repost-of`, `bookmark-of` | URL or h-cite. At most one of like/repost/bookmark. Content is optional for these |
|
||||
| `photo` | URLs, `{"value", "alt"}` objects, or uploaded files; at most 4 (ADR-057), duplicates collapsed. Own `/media/` URLs are looked up; other URLs are downloaded through `safe_get` (20 MB cap). Any photo that can't be resolved means 400 and nothing is created |
|
||||
| `published` | ISO 8601; converted to naive UTC; invalid gets 400 |
|
||||
| `post-status` | `published` (default) or `draft`; others get 400 |
|
||||
| `visibility` | `public`; `private` and `unlisted` are saved as drafts (StarPunk has no private posts); others get 400 |
|
||||
| `video`, `audio` | 400 (not supported) |
|
||||
| `location`, `checkin`, others | Ignored (allowed by 3.3.5) |
|
||||
| `mp-slug` | Custom slug |
|
||||
| `mp-syndicate-to` | Crosspost targets. **Absent means none** (v2.9.0). See [syndication-architecture.md](syndication-architecture.md#choosing-targets) |
|
||||
|
||||
`mp-*` keys are read as commands from the top level or from inside JSON
|
||||
`properties`, and are never stored. JSON property values must be arrays; a
|
||||
bare scalar is wrapped on create. If anything fails after the note is
|
||||
created, the note is removed and 500 is returned, so the client never
|
||||
believes a half-made post failed.
|
||||
|
||||
Post types advertised in `q=config`: note, article, photo, reply, like,
|
||||
repost, bookmark.
|
||||
|
||||
### Update, delete, undelete
|
||||
|
||||
- **Update** (JSON only, `update` scope) targets a `/note/<slug>` URL on
|
||||
this site.
|
||||
- `replace` works on `content`, `name`, `category`, `in-reply-to`,
|
||||
`post-status` and `photo`.
|
||||
- `add` works on `category` and `photo` only.
|
||||
- `delete` takes a list of property names, or values to remove for
|
||||
`category` and `photo`.
|
||||
- Values must be arrays. An update without any operation gets 400.
|
||||
Unsupported properties get 400.
|
||||
- Photos are resolved before anything changes. Success is **204**.
|
||||
- Edits follow crossposts and are re-announced to Webmention targets and
|
||||
WebSub subscribers.
|
||||
- **Delete** (`delete` scope) is soft: the note is moved to the trash, its
|
||||
page returns 410, and `undelete` restores it. Both return **204**.
|
||||
|
||||
### Queries
|
||||
|
||||
| Query | Scope | Result |
|
||||
|---|---|---|
|
||||
| `q=config` | any token | `media-endpoint`, `syndicate-to`, `q` list, `post-types` |
|
||||
| `q=syndicate-to` | any token | `{"syndicate-to": [{"uid", "name"}]}` |
|
||||
| `q=source&url=...[&properties[]=...]` | create/update/delete/draft | One post as mf2 JSON. Content comes back as `{"html": ...}` if it was authored as HTML. Includes `syndication` URLs and `post-status` |
|
||||
| `q=source` without `url` | same | `{"items": [...]}`, drafts included, `limit` (default 10, max 50) and `offset` |
|
||||
| `q=category[&filter=...]` | same | Tags in use, most used first |
|
||||
|
||||
Unknown or missing `q` gets 400.
|
||||
|
||||
### Media endpoint
|
||||
|
||||
`POST /micropub/media`, multipart with one part named `file`, `media` or
|
||||
`create` scope. It returns **201** with `Location: <SITE_URL>/media/<path>`.
|
||||
Images are JPEG, PNG, GIF, WebP or HEIC, up to 50 MB in. They are optimized
|
||||
and get responsive variants (`starpunk/media.py`,
|
||||
[ADR-058](../decisions/ADR-058-image-optimization-strategy.md)). Validation
|
||||
failures get 400.
|
||||
|
||||
### Known deviations and gaps
|
||||
|
||||
- **403 for `insufficient_scope`** (RFC 6750) where the Micropub error table
|
||||
says 401. Kept on purpose.
|
||||
- `add` is only supported for `category` and `photo`. `published`,
|
||||
`syndication` and `location` can't be updated. `mp-syndicate-to` is only
|
||||
read on create.
|
||||
- `location` / `checkin` are ignored, and person tags are dropped.
|
||||
- `video` and `audio` are refused.
|
||||
- The `access_token` query parameter is still accepted on GET.
|
||||
- Only notes (`/note/<slug>`) can be the target of update/delete/source.
|
||||
|
||||
---
|
||||
|
||||
## IndieAuth
|
||||
|
||||
StarPunk is its own IndieAuth authorization server for Micropub clients,
|
||||
issuing tokens for the single identity `ADMIN_ME`. The admin approves
|
||||
clients on a consent screen protected by OIDC login. Supported: the
|
||||
authorization code flow with mandatory PKCE S256, `iss` in responses (RFC
|
||||
9207), the metadata endpoint, a profile-only flow, a token endpoint, RFC
|
||||
7009 revocation, and legacy token verification (`GET /auth/token`). Client
|
||||
`redirect_uri`s must be same-origin or published by the client (JSON
|
||||
metadata or `rel=redirect_uri`).
|
||||
|
||||
Full description, scopes and gaps:
|
||||
[simplified-auth-architecture.md](simplified-auth-architecture.md).
|
||||
Decision: [ADR-064](../decisions/ADR-064-oidc-admin-login-and-builtin-indieauth-server.md).
|
||||
|
||||
Known gaps: `scopes_supported` doesn't list `draft`. There are no
|
||||
introspection, userinfo or refresh tokens. The client metadata fetch
|
||||
doesn't use the pinned SSRF fetcher.
|
||||
|
||||
---
|
||||
|
||||
## Webmention
|
||||
|
||||
Spec: https://www.w3.org/TR/webmention/ (Recommendation, 12 January 2017).
|
||||
Decision: [ADR-063](../decisions/ADR-063-webmention-implementation.md).
|
||||
Reviews: [v2.6.9 fixes](../design/v2.6.9/2026-11-03-webmention-processing-fixes.md),
|
||||
[v2.7.0 compliance](../design/v2.7.0/2026-11-03-webmention-spec-compliance.md),
|
||||
[v2.7.2 remaining SHOULDs](../design/v2.7.2/2026-11-03-webmention-shoulds.md).
|
||||
|
||||
### Endpoint discovery (advertising)
|
||||
|
||||
With `WEBMENTION_ENABLED` (default true), every HTML response carries
|
||||
`Link: </webmention>; rel="webmention"`, and `base.html` has
|
||||
`<link rel="webmention" href="/webmention">`. When it is off, the receiving
|
||||
route isn't registered at all.
|
||||
|
||||
### Receiving
|
||||
|
||||
`POST /webmention` (`routes/webmention.py`):
|
||||
|
||||
1. Rate limits: 60 requests per minute per client IP and 10 per minute per
|
||||
source domain (`WEBMENTION_RATE_LIMIT_*`). Over the limit gets 429 with
|
||||
`Retry-After`. Counts are shared by all server processes (`rate_limits`
|
||||
table, v2.11.0). The client IP
|
||||
comes from `X-Forwarded-For` only when `TRUSTED_PROXY=true`.
|
||||
2. Form-encoded (JSON also accepted). `source` and `target` are required,
|
||||
http(s), at most 2048 characters, and different from each other.
|
||||
Otherwise 400.
|
||||
3. The target's host and port must be this site's. The fragment is stripped
|
||||
before storing. The target must be a **published note**: anything else,
|
||||
including the home page, gets 400 `invalid_target`.
|
||||
4. The row is upserted into `webmentions` (unique on source + target) as
|
||||
`pending`, and **202** is returned. A repeat from the same source resets
|
||||
it to `pending` (an update, 3.2.4).
|
||||
|
||||
Verification is asynchronous (`WebmentionWorker`, poll
|
||||
`WEBMENTION_POLL_INTERVAL` 10 s, batch of 5):
|
||||
|
||||
- **Fetch** the source with `safe_get` (redirects followed and checked,
|
||||
1 MB, 10 s), with an `Accept` header preferring HTML.
|
||||
- **Exact-match verification** (3.2.2), per media type:
|
||||
- HTML: every link-bearing attribute (`lxml iterlinks`: `a`/`link` href,
|
||||
`img`/`video`/`audio` src, ...), resolved against the final URL.
|
||||
- JSON: any string value.
|
||||
- Other `text/*`: substring match.
|
||||
- Any other type: no match.
|
||||
- Fragments are ignored on both sides.
|
||||
- **Parse mf2** (HTML only) with mf2py:
|
||||
- `mf2_utils.responding_entry()` picks the h-entry that responds to the
|
||||
target.
|
||||
- `response_type()` classifies it as reply, like, repost, bookmark or
|
||||
mention.
|
||||
- `entry_author()` follows the authorship algorithm without extra
|
||||
fetches: author property, then the parent h-feed's author, then
|
||||
`rel=author`, then the representative h-card.
|
||||
- Also stored: a content snippet (e-content or p-summary, sanitized with
|
||||
nh3, at most 500 characters), the source's `u-url` and `dt-published`
|
||||
(migration 014).
|
||||
- The author URL and photo are kept only if they are absolute http(s)
|
||||
URLs (XSS, 4.1).
|
||||
- **Outcomes**:
|
||||
- Valid: `validation_status = 'valid'`. A first-time mention gets
|
||||
`moderation_status = 'approved'` if `WEBMENTION_AUTO_APPROVE` (default
|
||||
true), otherwise `pending`. An update keeps the admin's earlier
|
||||
moderation decision.
|
||||
- Source returns 4xx (including 410), or no longer links: `invalid`,
|
||||
which hides it (3.2.4 deletion).
|
||||
- Network error, timeout or 5xx: a mention that was valid before stays
|
||||
valid with its old data. A new one becomes `error`.
|
||||
- A source that is one of the note's own syndicated copies
|
||||
(`syndications.remote_url`) is `invalid` (v2.8.0).
|
||||
- **Re-send on display change** (3.1.4, v2.7.2): if what the note shows
|
||||
for this webmention changed, the note's own outgoing webmentions are
|
||||
re-queued. That covers a new response, a changed author or snippet, or a
|
||||
change of shown/hidden. Unchanged re-validations send nothing, so two
|
||||
sites doing the same don't ping-pong.
|
||||
|
||||
**Moderation and display.** `/admin/webmentions` lists, filters, and
|
||||
approves, hides or deletes mentions, one at a time or in bulk. Changes that
|
||||
alter what is shown also re-send the note's webmentions. Note pages show
|
||||
approved, valid mentions grouped as replies (`u-comment h-cite`), likes,
|
||||
reposts and bookmarks (facepiles), and mentions. See
|
||||
[Microformats2](#microformats2).
|
||||
|
||||
### Sending
|
||||
|
||||
Queued by `webmention_sender.queue_webmentions_for_note()` from `notes.py`.
|
||||
Sending isn't gated by `WEBMENTION_ENABLED`; it runs whenever the worker
|
||||
does (`WEBMENTION_WORKER_ENABLED`).
|
||||
|
||||
| Event | Targets |
|
||||
|---|---|
|
||||
| Publish | Reply / like / repost / bookmark target first, then `<a href>` links from the rendered HTML in document order (not inside `<pre>`/`<code>`), absolute http(s) only, de-duplicated, at most 10 |
|
||||
| Edit (published) | The same, plus every target notified before and links the edit removed (not capped) (3.1.4) |
|
||||
| Delete | Every previously notified target is re-queued (3.1.5); for a hard delete the queue rows are detached first so the cascade doesn't remove them |
|
||||
| Undelete | Re-queued again |
|
||||
| Displayed responses change | Re-queued (see above) |
|
||||
|
||||
Re-queuing resets the cached endpoint, so discovery runs again (3.1.4).
|
||||
|
||||
**Endpoint discovery** (3.1.2): GET the target with `safe_get` (redirects
|
||||
checked). The first `Link` header with a `rel` token `webmention` wins. The
|
||||
header is parsed per RFC 8288: several links per header, several headers,
|
||||
rel token lists, case-insensitive. Otherwise the first `<link>` or `<a>`
|
||||
with `rel~=webmention` and an `href`, in document order. Relative URLs
|
||||
resolve against the final URL, an empty `href` means the target itself, and
|
||||
the query string is kept.
|
||||
|
||||
**Send** (3.1.3): a form-encoded POST through `safe_post`, pinned and with
|
||||
no redirects followed (a 301/302 would turn the POST into a GET).
|
||||
|
||||
| Result | Outcome |
|
||||
|---|---|
|
||||
| Any 2xx | `sent` |
|
||||
| 429, 5xx, network error | Retried after 1 h, 6 h, 24 h; `failed` after the 4th attempt |
|
||||
| Other 3xx/4xx, SSRF-blocked endpoint | `failed` |
|
||||
| No endpoint | `no_endpoint` |
|
||||
|
||||
**Deleted posts**: `/note/<slug>` returns **410 Gone** with a tombstone page
|
||||
for soft-deleted notes, and for hard-deleted notes that had sent
|
||||
webmentions (3.1.5).
|
||||
|
||||
### Security (section 4)
|
||||
|
||||
- XSS: author URL and photo must be http(s). Snippets are sanitized with
|
||||
nh3 (and re-sanitized after truncation). The mentions list uses
|
||||
`striptags`.
|
||||
- SSRF: all fetches and POSTs go through the pinned fetcher described
|
||||
above, so the endpoint URL is checked before POSTing.
|
||||
- DoS: rate limits on receiving, a 1 MB / 10 s cap on fetches, and
|
||||
asynchronous processing.
|
||||
|
||||
### Known limits
|
||||
|
||||
From the v2.7.0 review, still current:
|
||||
|
||||
- Home-page mentions (and mentions of anything that isn't a published note)
|
||||
are refused rather than stored.
|
||||
- Discovery ignores HTTP cache headers (5.3). Targets are fetched only when
|
||||
a note is created, edited or deleted.
|
||||
- The 410 page is a generic tombstone without microformats.
|
||||
- `error` rows (transient failures of a new mention, or unexpected errors)
|
||||
are not retried automatically. They are re-checked only when the sender
|
||||
re-sends.
|
||||
- The 202 response has no `Location` status URL.
|
||||
|
||||
---
|
||||
|
||||
## Microformats2
|
||||
|
||||
Specs: microformats2 parsing, h-entry, h-card, h-feed, h-cite,
|
||||
representative h-card, authorship, Post Type Discovery. Decisions:
|
||||
[ADR-040](../decisions/ADR-040-microformats2-compliance.md),
|
||||
[ADR-061](../decisions/ADR-061-author-discovery.md). Review:
|
||||
[v2.8.0 microformats2 review](../design/v2.8.0/2026-11-03-microformats2-review.md).
|
||||
|
||||
### What pages publish
|
||||
|
||||
| Page | Markup |
|
||||
|---|---|
|
||||
| `/` (home) | `h-feed` with `p-name`, hidden `u-url`, hidden `p-author h-card` (photo, name, URL, note), feed `u-photo`; each note an `h-entry` preview with `u-uid` |
|
||||
| `/note/<slug>` | `h-entry`: `p-name` only when the note starts with a `# Heading` (so plain notes have no implied name); `u-photo` as direct children above `e-content`; `u-url u-uid` with `dt-published` (RFC 3339 UTC `Z`); `dt-updated`; `p-category` with `rel=tag`; `u-syndication` links; hidden `p-author h-card` |
|
||||
| Replies | `u-in-reply-to h-cite` holding the cached reply context (`p-content`, `u-url`) |
|
||||
| Likes / reposts / bookmarks | `u-like-of` / `u-repost-of` / `u-bookmark-of` link with "★ Liked" / "🔁 Reposted" / "🔖 Bookmarked" |
|
||||
| Received responses | `u-comment h-cite` (replies: `p-author h-card` with `u-url` on an inner link, `u-photo`, `p-name`; `e-content`; `u-url`; `dt-published`), and `u-like`, `u-repost`, `u-bookmark`, `u-mention` h-cites. Faces beyond 8 are `hidden`, not omitted |
|
||||
| `/likes`, `/tag/<tag>` | `h-feed` with `p-name`, `u-url`, `p-author h-card` |
|
||||
|
||||
Post Type Discovery gives note, article, reply, like, repost, bookmark and
|
||||
photo correctly.
|
||||
|
||||
### Identity links
|
||||
|
||||
`base.html` always emits `rel=me` and `rel=author` pointing at the author's
|
||||
home page (`author.url`, else `author.me`, else `ADMIN_ME`), even on a fresh
|
||||
install (v2.8.0). Further `rel=me` links come from the discovered profile.
|
||||
Because the `p-author h-card`'s `u-url` matches the `rel=me` URL, parsers
|
||||
treat it as the page's representative h-card. Without a discovered profile
|
||||
there is no h-card, only the `rel` links.
|
||||
|
||||
### Consuming
|
||||
|
||||
- **Author profile** (`author_discovery.py`, ADR-061): on login, StarPunk
|
||||
fetches `ADMIN_ME`, picks its representative h-card with
|
||||
`mf2_utils.representative_hcard()` (uid and url match the page, then
|
||||
url matches a `rel=me`, then a single h-card whose url is the page),
|
||||
parsed with the URL after redirects. It stores name, photo, URL, note and
|
||||
rel-me links, cached for 24 hours. If discovery fails, the stale cache is
|
||||
used.
|
||||
- **Webmention sources**: `responding_entry`, `response_type` and
|
||||
`entry_author` (see [Webmention](#webmention)).
|
||||
- **Reply context** (`reply_context.py`): title, snippet, author and site
|
||||
name of the post being replied to, liked, reposted or bookmarked, cached
|
||||
in `reply_contexts`.
|
||||
|
||||
### Known gaps
|
||||
|
||||
From the v2.8.0 review, "Not done":
|
||||
|
||||
- The reply-context `h-cite` doesn't carry the cached author h-card, name or
|
||||
published time.
|
||||
- Photo alt text falls back to "Image" when a photo-only post has no
|
||||
caption.
|
||||
- Person tags (`p-category h-card`) aren't supported.
|
||||
- Webmentions received before v2.8.0 keep NULL source details until they
|
||||
are re-validated.
|
||||
|
||||
---
|
||||
|
||||
## WebSub
|
||||
|
||||
Spec: https://www.w3.org/TR/websub/. Decision:
|
||||
[ADR-071](../decisions/ADR-071-websub.md), including its v2.10.1
|
||||
amendments, which replace the original text where they differ.
|
||||
|
||||
### Topics
|
||||
|
||||
All absolute URLs under `SITE_URL`:
|
||||
|
||||
| Topic | Content type | Updated when |
|
||||
|---|---|---|
|
||||
| `/` | text/html | A public non-like note changes |
|
||||
| `/feed.atom`, `/feed.rss`, `/feed.json` | feed types | A public non-like note changes |
|
||||
| `/likes` | text/html | A public like changes |
|
||||
| `/tag/<name>` | text/html | A note with that tag (before or after the change) changes |
|
||||
|
||||
`/feed.xml` (legacy) advertises `/feed.rss` as its `rel=self`. The
|
||||
negotiated `/feed` advertises the topic of the format it served.
|
||||
|
||||
### Discovery
|
||||
|
||||
Topic responses (200/304) carry `Link: <hub>; rel="hub"` and
|
||||
`Link: <topic>; rel="self"`. The same links are embedded: `<link rel="hub">`
|
||||
and `<link rel="self">` in HTML heads, a hub `<link>` in Atom, an
|
||||
`<atom:link rel="hub">` in RSS, and `"hubs"` in JSON Feed. All of this
|
||||
happens only while `WEBSUB_ENABLED` (default true).
|
||||
|
||||
### On change
|
||||
|
||||
`websub.note_changed()` runs on create, edit, publish/unpublish, delete and
|
||||
undelete, and once photos are attached. If the note is or was public, it:
|
||||
|
||||
1. bumps `content_version`, which is in the feed cache key and the ETag, so
|
||||
every process serves fresh feeds at once;
|
||||
2. queues one delivery per active subscription for each affected topic.
|
||||
Only the latest content is sent: a newer delivery replaces a waiting one.
|
||||
|
||||
Drafts that stay drafts change nothing.
|
||||
|
||||
### Hub
|
||||
|
||||
`POST /websub`, form-encoded only (case-insensitive check):
|
||||
|
||||
- `hub.mode` is `subscribe` or `unsubscribe`.
|
||||
- `hub.topic` must be one of the topics above, after decoding unreserved
|
||||
percent-encodings and comparing scheme and host case-insensitively. Tag
|
||||
topics must name an existing tag.
|
||||
- `hub.callback` must:
|
||||
- be printable, at most 2,048 characters, and absolute http(s);
|
||||
- not be on this site;
|
||||
- use port 80, 443 or 1024 and up;
|
||||
- pass the SSRF check.
|
||||
|
||||
It is stored with an IDNA host.
|
||||
- `hub.secret` must be non-empty and under 200 bytes. `hub.lease_seconds`
|
||||
is clamped to 1 to 30 days (default 10). Both are ignored for
|
||||
unsubscriptions.
|
||||
- The hub answers **202**, then verifies later. Errors are 4xx with a
|
||||
plain-text reason.
|
||||
|
||||
Limits (429 when hit):
|
||||
|
||||
- 30 requests per minute per client IP (shared by all processes; needs
|
||||
`TRUSTED_PROXY=true` behind a proxy);
|
||||
- at most 200 pending verifications;
|
||||
- `WEBSUB_MAX_SUBSCRIPTIONS` (1000) in total;
|
||||
- `WEBSUB_MAX_PER_HOST` (25) per callback IP address.
|
||||
|
||||
The limits count pending requests too. Renewals are always allowed.
|
||||
|
||||
`WebSubWorker` (file lock, poll `WEBSUB_POLL_INTERVAL` 5 s) handles each
|
||||
queued item on its own, so one bad item can't stop it:
|
||||
|
||||
- **Verification of intent** (5.3): `GET` the callback (existing query
|
||||
kept) with `hub.mode`, the topic exactly as requested, `hub.challenge`
|
||||
and `hub.lease_seconds`. Success is a 2xx whose body equals the challenge.
|
||||
Replies over 4 KB and redirects fail. Network/DNS errors and 5xx are
|
||||
retried after 1, 5 and 15 minutes. A request id per intent (migration
|
||||
017) stops an older verification from consuming a newer request. A
|
||||
subscription row exists only once verified.
|
||||
- **Distribution** (7): the topic is rendered inside the app, once per pass,
|
||||
and POSTed with its `Content-Type`, `Link` rel=hub/self, and
|
||||
`X-Hub-Signature: sha256=<HMAC>` when there is a secret.
|
||||
- 2xx: success.
|
||||
- 410: removes the subscription.
|
||||
- Other failures: retried after 1 min, 5 min, 30 min, 2 h and 6 h, then
|
||||
that delivery is dropped. The subscription stays until its lease ends.
|
||||
- A topic that renders 404/410 ends its subscriptions with a
|
||||
`hub.mode=denied` notice.
|
||||
- Expired subscriptions are pruned hourly.
|
||||
|
||||
Outbound requests use `safe_get` / `safe_post`: pinned, no redirects,
|
||||
10 s hard deadline.
|
||||
|
||||
**External hub**: with `WEBSUB_HUB_URL` set, StarPunk advertises that hub
|
||||
and sends it a `hub.mode=publish` ping per changed topic. Its own `/websub`
|
||||
refuses new subscriptions but still accepts unsubscriptions.
|
||||
|
||||
The admin dashboard shows subscribers per topic and deliveries being
|
||||
retried.
|
||||
|
||||
### Known limits
|
||||
|
||||
- The hub serves only this site's topics, by design. websub.rocks' hub
|
||||
tests use their own topics, so they can't be run against it.
|
||||
- Deliveries always carry the full topic content (no diffs). Fat pings
|
||||
aren't sent to external hubs.
|
||||
|
||||
---
|
||||
|
||||
## Feeds
|
||||
|
||||
Decisions: [ADR-014](../decisions/ADR-014-rss-feed-implementation.md),
|
||||
[ADR-038](../decisions/ADR-038-syndication-formats.md),
|
||||
[ADR-059](../decisions/ADR-059-full-feed-media-standardization.md).
|
||||
|
||||
| URL | Format |
|
||||
|---|---|
|
||||
| `/feed.rss` | RSS 2.0 (with `atom:link rel=self` and Media RSS) |
|
||||
| `/feed.xml` | Legacy alias of `/feed.rss` |
|
||||
| `/feed.atom` | Atom 1.0 |
|
||||
| `/feed.json` | JSON Feed 1.1 |
|
||||
| `/feed` | Content negotiation on `Accept` (RSS default; 406 with `X-Available-Formats` if nothing fits; `Vary: Accept`) |
|
||||
| `/opml.xml` | OPML 2.0 listing the three feeds |
|
||||
|
||||
- Feeds contain the newest `FEED_MAX_ITEMS` (50) published notes, with full
|
||||
HTML content and media. Likes are excluded, as on the home page.
|
||||
Reposts and bookmarks start with a "🔁 Reposted" / "🔖 Bookmarked" line.
|
||||
- Feeds are cached for `FEED_CACHE_SECONDS` (300). The cache key and ETag
|
||||
include `content_version`, so any change to a public note invalidates
|
||||
them in every process. `If-None-Match` gets 304.
|
||||
- `base.html` advertises the negotiated `/feed` (as RSS) and the OPML file
|
||||
with `rel=alternate`. Atom and JSON Feed are listed in the OPML but have
|
||||
no `<link rel=alternate>` of their own.
|
||||
- WebSub hub links are embedded in each feed (see above).
|
||||
|
||||
Gaps: there are no per-tag feeds (tag pages are h-feeds and WebSub topics,
|
||||
but have no RSS/Atom/JSON equivalent). The `/likes` page has no feed.
|
||||
+316
-1116
File diff suppressed because it is too large.
Load diff
@@ -1,428 +1,344 @@
|
||||
# StarPunk Simplified Authentication Architecture
|
||||
# Authentication and Authorization Architecture
|
||||
|
||||
Current as of v2.10.1. Decision record: [ADR-064](../decisions/ADR-064-oidc-admin-login-and-builtin-indieauth-server.md),
|
||||
which supersedes the "no self-hosted IndieAuth" decisions
|
||||
([ADR-050](../decisions/ADR-050-remove-custom-indieauth-server.md),
|
||||
[ADR-056](../decisions/ADR-056-no-selfhosted-indieauth.md)) and the external
|
||||
token verification of ADR-030 / ADR-036.
|
||||
|
||||
> The filename is historical: it described the v1.x "external provider only"
|
||||
> design. Other documents link to it, so it is kept.
|
||||
|
||||
## Overview
|
||||
|
||||
After removing the custom IndieAuth authorization server, StarPunk becomes a pure Micropub server that relies on external providers for all authentication and authorization.
|
||||
StarPunk is a single-user site with two kinds of client and two auth
|
||||
mechanisms:
|
||||
|
||||
## Architecture Diagrams
|
||||
| Who | Mechanism | Credential | Code |
|
||||
|---|---|---|---|
|
||||
| The site owner in a browser (admin UI, consent screen) | OpenID Connect login against an external provider (e.g. Pocket ID) | `starpunk_session` cookie, backed by a hashed row in `sessions` | `starpunk/auth.py`, `starpunk/routes/auth.py` |
|
||||
| Micropub apps (Quill, Indigenous, ...) | StarPunk's own IndieAuth authorization server | Bearer access token, backed by a hashed row in `indieauth_tokens` | `starpunk/indieauth.py`, `starpunk/routes/indieauth.py` |
|
||||
|
||||
The two are chained: a Micropub app is approved on a consent screen that
|
||||
requires an admin session, so every token ultimately rests on the OIDC
|
||||
login. StarPunk stores no passwords.
|
||||
|
||||
### Before: Complex Mixed-Mode Architecture
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ StarPunk Instance │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────┐ │
|
||||
│ │ Web Interface │ │
|
||||
│ │ ┌─────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
|
||||
│ │ │ Admin Login │ │ Authorization │ │ Token Issuer │ │ │
|
||||
│ │ └─────────────┘ └──────────────┘ └──────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────┐ │
|
||||
│ │ Auth Module │ │
|
||||
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
|
||||
│ │ │ Sessions │ │ PKCE │ │ Tokens │ │ Codes │ │ │
|
||||
│ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────┐ │
|
||||
│ │ Database │ │
|
||||
│ │ ┌────────┐ ┌──────────────────┐ ┌─────────────────┐ │ │
|
||||
│ │ │ Users │ │ authorization_codes│ │ tokens │ │ │
|
||||
│ │ └────────┘ └──────────────────┘ └─────────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
Browser (owner) StarPunk OIDC provider
|
||||
─────────────── ──────── ─────────────
|
||||
POST /auth/login ─────────────▶ initiate_login() ── redirect ──▶ authorize (PKCE, nonce)
|
||||
owner authenticates
|
||||
GET /auth/callback?code&state ◀────────────────────── redirect ───┘
|
||||
handle_callback(): code exchange, ID token checks,
|
||||
sub == OIDC_ADMIN_SUB → create_session(ADMIN_ME)
|
||||
◀── Set-Cookie: starpunk_session
|
||||
|
||||
Problems:
|
||||
- 500+ lines of security-critical code
|
||||
- Dual role: authorization server AND resource server
|
||||
- Complex token lifecycle management
|
||||
- Database bloat with token storage
|
||||
- Maintenance burden for security updates
|
||||
Micropub app StarPunk
|
||||
──────────── ────────
|
||||
discovers indieauth-metadata / authorization_endpoint / token_endpoint
|
||||
GET /auth/authorize?... ────────▶ validate request → (OIDC login if no session) → consent page
|
||||
owner approves ─────────────────▶ POST /auth/authorize/approve → redirect: code, state, iss
|
||||
POST /auth/token (code, verifier) ▶ access token (hash stored in indieauth_tokens)
|
||||
POST /micropub, /micropub/media ─▶ verify_access_token() + scope check
|
||||
```
|
||||
|
||||
### After: Clean Separation of Concerns
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ StarPunk Instance │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────┐ │
|
||||
│ │ Web Interface │ │
|
||||
│ │ ┌─────────────┐ ┌──────────────┐ │ │
|
||||
│ │ │ Admin Login │ │ Micropub │ │ │
|
||||
│ │ └─────────────┘ └──────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────┐ │
|
||||
│ │ Auth Module │ │
|
||||
│ │ ┌──────────────┐ ┌──────────────────────┐ │ │
|
||||
│ │ │ Sessions │ │ Token Verification │ │ │
|
||||
│ │ │ (Admin Only) │ │ (External Provider) │ │ │
|
||||
│ │ └──────────────┘ └──────────────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────┐ │
|
||||
│ │ Database │ │
|
||||
│ │ ┌────────┐ ┌──────────┐ ┌─────────┐ │ │
|
||||
│ │ │ Users │ │auth_state│ │ posts │ (No token tables)│ │
|
||||
│ │ └────────┘ └──────────┘ └─────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ API Calls
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ External IndieAuth Providers │
|
||||
│ ┌─────────────────────┐ ┌─────────────────────────┐ │
|
||||
│ │ indieauth.com │ │ tokens.indieauth.com │ │
|
||||
│ │ (Authorization) │ │ (Token Verification) │ │
|
||||
│ └─────────────────────┘ └─────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
## 1. Admin login (OIDC)
|
||||
|
||||
Benefits:
|
||||
- 500+ lines of code removed
|
||||
- Clear single responsibility
|
||||
- No security burden
|
||||
- Minimal database footprint
|
||||
- Zero maintenance for auth code
|
||||
```
|
||||
### Flow
|
||||
|
||||
## Authentication Flows
|
||||
1. `GET /auth/login` shows `templates/admin/login.html`, or redirects to the
|
||||
dashboard if the session cookie is already valid.
|
||||
2. `POST /auth/login` calls `auth.initiate_login(next)`:
|
||||
- Fetches `{OIDC_ISSUER}/.well-known/openid-configuration` (cached for an
|
||||
hour per process). The document's `issuer` must equal `OIDC_ISSUER`, and
|
||||
it must name `authorization_endpoint`, `token_endpoint` and `jwks_uri`.
|
||||
- Generates `state`, `nonce` and a PKCE `code_verifier`, and stores them in
|
||||
`oidc_login_state` with the sanitized `next` URL (same site only) and a
|
||||
10-minute expiry.
|
||||
- Redirects to the provider with `response_type=code`, `scope=openid` and
|
||||
`code_challenge_method=S256`.
|
||||
3. `GET /auth/callback`:
|
||||
- A provider `error` parameter, or a missing `code`/`state`, flashes an
|
||||
error and returns to the login page.
|
||||
- The `state` row is fetched and deleted in one step: it is single use and
|
||||
must not have expired.
|
||||
- If the provider sent `iss` (RFC 9207), it must match the issuer.
|
||||
- The code is exchanged at the token endpoint along with the
|
||||
`code_verifier`. StarPunk authenticates as a confidential client (HTTP
|
||||
Basic, `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET`).
|
||||
- The ID token is verified with PyJWT against the provider's JWKS. The
|
||||
checks are: the signature (algorithms from the provider metadata, never
|
||||
`none`), `iss`, `aud` equal to the client id, `exp`, required `iat` and
|
||||
`sub`, a 60 s leeway, and a `nonce` equal to the stored one.
|
||||
- `sub` must equal `OIDC_ADMIN_SUB` (constant-time comparison). Any other
|
||||
account gets "Not authorized as admin".
|
||||
- A session is created for `ADMIN_ME`, the owner's identity URL. It is also
|
||||
the `me` of every issued token. The author h-card is refreshed from
|
||||
`ADMIN_ME` on a best-effort basis that never blocks login (see
|
||||
[ADR-061](../decisions/ADR-061-author-discovery.md)).
|
||||
|
||||
### Flow 1: Admin Authentication (Unchanged)
|
||||
```
|
||||
Admin User StarPunk IndieLogin.com
|
||||
│ │ │
|
||||
├──── GET /admin/login ───→ │ │
|
||||
│ │ │
|
||||
│ ←── Login Form ─────────── │ │
|
||||
│ │ │
|
||||
├──── POST /auth/login ───→ │ │
|
||||
│ (me=admin.com) │ │
|
||||
│ ├──── Redirect ──────────────→ │
|
||||
│ │ (client_id=starpunk.com) │
|
||||
│ ←──────────── Authorization Request ───────────────────── │
|
||||
│ │ │
|
||||
├───────────── Authenticate with IndieLogin ──────────────→ │
|
||||
│ │ │
|
||||
│ │ ←── Callback ────────────────│
|
||||
│ │ (me=admin.com) │
|
||||
│ │ │
|
||||
│ ←── Session Cookie ─────── │ │
|
||||
│ │ │
|
||||
│ Admin Access │ │
|
||||
```
|
||||
### Sessions
|
||||
|
||||
### Flow 2: Micropub Client Authentication (Simplified)
|
||||
```
|
||||
Micropub Client StarPunk External Token Endpoint
|
||||
│ │ │
|
||||
├─── POST /micropub ───→ │ │
|
||||
│ Bearer: token123 │ │
|
||||
│ ├──── GET /token ─────────→ │
|
||||
│ │ Bearer: token123 │
|
||||
│ │ │
|
||||
│ │ ←── Token Info ──────────│
|
||||
│ │ {me, scope, client_id} │
|
||||
│ │ │
|
||||
│ │ [Validate me==ADMIN_ME] │
|
||||
│ │ [Check scope includes │
|
||||
│ │ "create"] │
|
||||
│ │ │
|
||||
│ ←── 201 Created ────────│ │
|
||||
│ Location: /post/123 │ │
|
||||
```
|
||||
- `create_session(me)` makes a random 32-byte URL-safe token. Only its
|
||||
SHA-256 hash is stored in `sessions`, with `me`, `expires_at`
|
||||
(`SESSION_LIFETIME` days, default 30), the user agent and the IP. Expired
|
||||
sessions and login states are cleaned up each time a session is created.
|
||||
- Cookie `starpunk_session`: `HttpOnly`, `SameSite=Lax`, `Secure` when
|
||||
`SITE_URL` is https, `max-age` equal to the session lifetime.
|
||||
- `verify_session(token)` looks up the hash, checks expiry and updates
|
||||
`last_used_at`.
|
||||
- `@require_auth` protects `/admin/*`. Without a valid session it remembers
|
||||
the URL and redirects to `/auth/login`. It sets `g.user` and `g.me`.
|
||||
- A `before_request` hook also resolves the session on public pages that
|
||||
carry the cookie, so the header can show "Admin" instead of "Log in".
|
||||
- `POST /auth/logout` deletes the session row and the cookie.
|
||||
|
||||
## Component Responsibilities
|
||||
### Configuration
|
||||
|
||||
### StarPunk Components
|
||||
Outside `DEV_MODE`, `OIDC_ISSUER`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`,
|
||||
`OIDC_ADMIN_SUB` and `ADMIN_ME` are required, and startup fails without them.
|
||||
The provider must have `{SITE_URL}/auth/callback` registered as a redirect
|
||||
URI.
|
||||
|
||||
#### 1. Admin Authentication (`/auth/*`)
|
||||
**Responsibility**: Manage admin sessions via IndieLogin.com
|
||||
**Does**:
|
||||
- Initiate OAuth flow with IndieLogin.com
|
||||
- Validate callback and create session
|
||||
- Manage session lifecycle
|
||||
### Development mode
|
||||
|
||||
**Does NOT**:
|
||||
- Issue tokens
|
||||
- Store passwords
|
||||
- Manage user identities
|
||||
With `DEV_MODE=true` (and `DEV_ADMIN_ME`), `/dev/login` creates a session
|
||||
without OIDC. The blueprint is registered only in dev mode, and its
|
||||
`before_request` guard returns 404 if `DEV_MODE` is off. Every page shows a
|
||||
warning banner while it is on. Never enable it in production.
|
||||
|
||||
#### 2. Micropub Endpoint (`/micropub`)
|
||||
**Responsibility**: Accept and process Micropub requests
|
||||
**Does**:
|
||||
- Extract Bearer tokens from requests
|
||||
- Verify tokens with external endpoint
|
||||
- Create/update/delete posts
|
||||
- Return proper Micropub responses
|
||||
## 2. Built-in IndieAuth authorization server
|
||||
|
||||
**Does NOT**:
|
||||
- Issue tokens
|
||||
- Manage authorization codes
|
||||
- Store token data
|
||||
StarPunk issues tokens for exactly one identity, `ADMIN_ME`. It implements
|
||||
the authorization code flow of the current IndieAuth spec
|
||||
(https://indieauth.spec.indieweb.org/), with PKCE required.
|
||||
|
||||
#### 3. Token Verification Module
|
||||
**Responsibility**: Validate tokens with external providers
|
||||
**Does**:
|
||||
- Call external token endpoint
|
||||
- Cache valid tokens (5 min TTL)
|
||||
- Validate scope and identity
|
||||
### Endpoints
|
||||
|
||||
**Does NOT**:
|
||||
- Generate tokens
|
||||
- Store tokens permanently
|
||||
- Manage token lifecycle
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| GET | `/.well-known/oauth-authorization-server` | Server metadata (the target of `rel=indieauth-metadata`) |
|
||||
| GET | `/auth/authorize` | Validate the request; show the consent screen (admin session required) |
|
||||
| POST | `/auth/authorize/approve` | Consent decision from the screen |
|
||||
| POST | `/auth/authorize` | Profile-only code redemption, returns `{"me": ...}` |
|
||||
| POST | `/auth/token` | Exchange a code for an access token; also legacy `action=revoke` |
|
||||
| GET | `/auth/token` | Legacy token verification (Bearer token → `me`, `client_id`, `scope`) |
|
||||
| POST | `/auth/revoke` | RFC 7009 revocation (always 200) |
|
||||
|
||||
### External Provider Responsibilities
|
||||
All JSON responses carry `Cache-Control: no-store`. Errors use the OAuth
|
||||
form `{"error", "error_description"}`.
|
||||
|
||||
#### indieauth.com
|
||||
- User authentication
|
||||
- Authorization consent
|
||||
- Authorization code generation
|
||||
- Profile discovery
|
||||
### Metadata
|
||||
|
||||
#### tokens.indieauth.com
|
||||
- Token issuance
|
||||
- Token verification
|
||||
- Token revocation
|
||||
- Scope management
|
||||
|
||||
## Configuration
|
||||
|
||||
### Required Settings
|
||||
```ini
|
||||
# Identity of the admin user
|
||||
ADMIN_ME=https://your-domain.com
|
||||
|
||||
# External token endpoint for verification
|
||||
TOKEN_ENDPOINT=https://tokens.indieauth.com/token
|
||||
|
||||
# Admin session secret (existing)
|
||||
SECRET_KEY=your-secret-key
|
||||
```
|
||||
|
||||
### HTML Discovery
|
||||
```html
|
||||
<!-- Added to all pages -->
|
||||
<link rel="authorization_endpoint" href="https://indieauth.com/auth">
|
||||
<link rel="token_endpoint" href="https://tokens.indieauth.com/token">
|
||||
<link rel="micropub" href="https://starpunk.example.com/micropub">
|
||||
```
|
||||
|
||||
## Security Model
|
||||
|
||||
### Trust Boundaries
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Trusted Zone │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ StarPunk Application │ │
|
||||
│ │ - Session management │ │
|
||||
│ │ - Post creation/management │ │
|
||||
│ │ - Admin interface │ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
Token Verification API
|
||||
│
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Semi-Trusted Zone │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ External IndieAuth Providers │ │
|
||||
│ │ - Token validation │ │
|
||||
│ │ - Identity verification │ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
User Authentication
|
||||
│
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Untrusted Zone │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ Micropub Clients │ │
|
||||
│ │ - Must provide valid Bearer tokens │ │
|
||||
│ │ - Tokens verified on every request │ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Security Benefits of Simplified Architecture
|
||||
|
||||
1. **Reduced Attack Surface**
|
||||
- No token generation = no cryptographic mistakes
|
||||
- No token storage = no database leaks
|
||||
- No PKCE = no implementation errors
|
||||
|
||||
2. **Specialized Security**
|
||||
- Auth providers focus solely on security
|
||||
- Regular updates from specialized teams
|
||||
- Community-vetted implementations
|
||||
|
||||
3. **Clear Boundaries**
|
||||
- StarPunk only verifies, never issues
|
||||
- Single source of truth (external provider)
|
||||
- No confused deputy problems
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
### Token Verification Performance
|
||||
```
|
||||
Without Cache:
|
||||
┌──────────┐ 200-500ms ┌─────────────┐
|
||||
│ Micropub ├───────────────────→│Token Endpoint│
|
||||
└──────────┘ └─────────────┘
|
||||
|
||||
With Cache (95% hit rate):
|
||||
┌──────────┐ <1ms ┌─────────────┐
|
||||
│ Micropub ├───────────────────→│ Memory Cache │
|
||||
└──────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
### Cache Strategy
|
||||
```python
|
||||
Cache Key: SHA256(token)
|
||||
Cache Value: {
|
||||
'me': 'https://user.com',
|
||||
'client_id': 'https://client.com',
|
||||
'scope': 'create update delete',
|
||||
'expires_at': timestamp + 300 # 5 minutes
|
||||
```json
|
||||
{
|
||||
"issuer": "<SITE_URL>/",
|
||||
"authorization_endpoint": "<SITE_URL>/auth/authorize",
|
||||
"token_endpoint": "<SITE_URL>/auth/token",
|
||||
"revocation_endpoint": "<SITE_URL>/auth/revoke",
|
||||
"revocation_endpoint_auth_methods_supported": ["none"],
|
||||
"scopes_supported": ["create", "update", "delete", "media", "profile"],
|
||||
"response_types_supported": ["code"],
|
||||
"grant_types_supported": ["authorization_code"],
|
||||
"code_challenge_methods_supported": ["S256"],
|
||||
"authorization_response_iss_parameter_supported": true
|
||||
}
|
||||
```
|
||||
|
||||
### Expected Latencies
|
||||
- First request: 200-500ms (external API)
|
||||
- Cached request: <1ms
|
||||
- Admin login: 1-2s (OAuth flow)
|
||||
- Post creation: <50ms (after auth)
|
||||
StarPunk has no introspection or userinfo endpoint. Only StarPunk itself
|
||||
checks its tokens.
|
||||
|
||||
## Migration Impact
|
||||
### Authorization request validation (`validate_authorization_request`)
|
||||
|
||||
### Breaking Changes
|
||||
1. **All existing tokens invalid**
|
||||
- Users must re-authenticate
|
||||
- No migration path for tokens
|
||||
- `client_id` must be an http(s) URL without a fragment.
|
||||
- `redirect_uri` must be present and have no fragment. It must either share
|
||||
the `client_id`'s scheme, host and port, or be published by the client. To
|
||||
check the second case, StarPunk fetches the `client_id` URL (5 s timeout,
|
||||
with an SSRF check on the URL unless `DEV_MODE` is on). It reads
|
||||
`redirect_uris` from a JSON client metadata document, or
|
||||
`rel="redirect_uri"` from the `Link` header and from `<link>`/`<a>`
|
||||
elements.
|
||||
- Problems with `client_id` or `redirect_uri` are shown to the user on an
|
||||
error page (HTTP 400). They are never redirected, because the redirect
|
||||
target can't be trusted.
|
||||
- Other problems are redirected to the client with `error`,
|
||||
`error_description`, `state` and `iss`. These are: a `response_type` other
|
||||
than `code` (`unsupported_response_type`), a missing `state`, a missing
|
||||
`code_challenge`, or a `code_challenge_method` other than `S256`.
|
||||
- `scope` is split on spaces. Requested scopes are not filtered against
|
||||
`scopes_supported`.
|
||||
|
||||
2. **Endpoint removal**
|
||||
- `/auth/authorization` → 404
|
||||
- `/auth/token` → 404
|
||||
### Consent screen
|
||||
|
||||
3. **Configuration required**
|
||||
- Must set `ADMIN_ME`
|
||||
- Must configure domain with IndieAuth links
|
||||
If `GET /auth/authorize` gets a valid request but there is no admin session,
|
||||
StarPunk stores the request URL and redirects to the OIDC login, which
|
||||
returns to that URL afterwards. With a session, it renders
|
||||
`templates/admin/authorize.html`. The page shows the client id, the identity
|
||||
the client will act as, the redirect URI, and one checkbox per requested
|
||||
scope (all checked). A request without scopes is described as "only wants to
|
||||
confirm your identity".
|
||||
|
||||
### Non-Breaking Preserved Functionality
|
||||
1. **Admin login unchanged**
|
||||
- Same URL (`/admin/login`)
|
||||
- Same provider (IndieLogin.com)
|
||||
- Sessions preserved
|
||||
`POST /auth/authorize/approve` checks the session again and re-validates the
|
||||
request from hidden form fields. "Deny" redirects with
|
||||
`error=access_denied`. "Approve" grants only the scopes that were both
|
||||
requested and left checked, issues a code, and redirects with `code`,
|
||||
`state` and `iss`.
|
||||
|
||||
2. **Micropub API unchanged**
|
||||
- Same endpoint (`/micropub`)
|
||||
- Same request format
|
||||
- Same response format
|
||||
### Codes and tokens
|
||||
|
||||
## Comparison with Other Systems
|
||||
| | Authorization code | Access token |
|
||||
|---|---|---|
|
||||
| Table | `indieauth_codes` | `indieauth_tokens` |
|
||||
| Value | 32 random bytes, URL-safe | 32 random bytes, URL-safe |
|
||||
| Stored as | SHA-256 hash | SHA-256 hash |
|
||||
| Lifetime | 10 minutes | `INDIEAUTH_TOKEN_LIFETIME_DAYS` (default 90) |
|
||||
| Bound to | `client_id`, `redirect_uri`, `scope`, `code_challenge` | `client_id`, `scope` |
|
||||
|
||||
### WordPress + IndieAuth Plugin
|
||||
- **Similarity**: External provider for auth
|
||||
- **Difference**: WP has user management, we don't
|
||||
`redeem_authorization_code` handles redemption for both `POST /auth/token`
|
||||
and `POST /auth/authorize`:
|
||||
|
||||
### Known IndieWeb Sites
|
||||
- **micro.blog**: Custom auth server (complex)
|
||||
- **Indigenous**: Client only, uses external auth
|
||||
- **StarPunk**: Micropub server only (simple)
|
||||
1. `code` and `code_verifier` are required.
|
||||
2. An atomic `UPDATE ... SET used_at = now WHERE code_hash = ? AND used_at IS
|
||||
NULL AND expires_at > now` makes the code single-use, even across
|
||||
gunicorn workers.
|
||||
3. `client_id` and `redirect_uri` must match the code exactly.
|
||||
4. `BASE64URL(SHA256(code_verifier))` must equal the stored challenge
|
||||
(constant-time comparison).
|
||||
|
||||
### Architecture Philosophy
|
||||
```
|
||||
"Do one thing well"
|
||||
│
|
||||
├── StarPunk: Publish notes
|
||||
├── IndieAuth.com: Authenticate users
|
||||
└── Tokens.indieauth.com: Manage tokens
|
||||
The token endpoint accepts only `grant_type=authorization_code`. It refuses
|
||||
a code issued with no scope (`invalid_grant`), because such codes are for
|
||||
the profile-only flow at the authorization endpoint. The response is
|
||||
`{access_token, token_type: "Bearer", scope, me, expires_in}`. There are no
|
||||
refresh tokens: a client re-authorizes when its token expires.
|
||||
|
||||
Expired codes are deleted whenever a new code is issued.
|
||||
|
||||
### Verification and revocation
|
||||
|
||||
- `verify_access_token(token)` hashes the token and accepts it if it is
|
||||
neither revoked nor expired, then updates `last_used_at`. It returns
|
||||
`{me: ADMIN_ME, client_id, scope}`.
|
||||
- `check_scope(required, scope)` matches whole words in the space-separated
|
||||
scope string.
|
||||
- Revocation sets `revoked_at`. It can happen by token value, through
|
||||
`POST /auth/revoke` or the legacy `POST /auth/token` with `action=revoke`
|
||||
(unknown tokens are ignored, per RFC 7009). It can also happen by id, from
|
||||
the admin page.
|
||||
- `/admin/tokens` ("Connected Apps") lists active tokens with their client,
|
||||
scope, dates and last use. Each has a Revoke button.
|
||||
|
||||
### Scopes
|
||||
|
||||
| Scope | Allows |
|
||||
|---|---|
|
||||
| `create` | Micropub create (published or draft), media upload, `q=source`, `q=category` |
|
||||
| `draft` | Micropub create, always saved as a draft (v2.9.0); `q=source`, `q=category` |
|
||||
| `update` | Micropub update; `q=source`, `q=category` |
|
||||
| `delete` | Micropub delete and undelete; `q=source`, `q=category` |
|
||||
| `media` | Media endpoint uploads |
|
||||
| `profile` | Nothing beyond identity (there is no profile or userinfo endpoint) |
|
||||
|
||||
`q=config` and `q=syndicate-to` work with any valid token.
|
||||
|
||||
## 3. Token use at the Micropub and media endpoints
|
||||
|
||||
`routes/micropub.py` handles `GET/POST /micropub` and
|
||||
`POST /micropub/media`. The old `POST /media` path is kept for existing
|
||||
clients.
|
||||
|
||||
### Extracting the token (`micropub.extract_bearer_token`)
|
||||
|
||||
- The token comes from `Authorization: Bearer <token>`, with the scheme
|
||||
matched case-insensitively (RFC 7235).
|
||||
- Or it comes from `access_token` in the form body (POST) or the query
|
||||
string (GET).
|
||||
- A token in both places gets **400 `invalid_request`** (RFC 6750 §2,
|
||||
v2.9.0).
|
||||
- No token gets **401 `unauthorized`**, "No access token provided".
|
||||
- An unknown, expired or revoked token gets **401 `unauthorized`**,
|
||||
"Invalid or expired access token".
|
||||
|
||||
Every 401 carries `WWW-Authenticate: Bearer realm="micropub"`. When a token
|
||||
was sent but isn't valid, `error="invalid_token"` is added (RFC 6750 §3,
|
||||
v2.9.0).
|
||||
|
||||
### Scope rules
|
||||
|
||||
| Operation | Required scope | Notes |
|
||||
|---|---|---|
|
||||
| create | `create` or `draft` | With only `draft`, the post is saved unpublished whatever `post-status` says |
|
||||
| update | `update` | |
|
||||
| delete / undelete | `delete` | |
|
||||
| `q=source`, `q=category` | any of `create`, `update`, `delete`, `draft` | These show drafts and tags (v2.9.0) |
|
||||
| `q=config`, `q=syndicate-to` | any valid token | |
|
||||
| media upload | `media` or `create` | `create` is accepted for older clients |
|
||||
|
||||
A missing scope gets **403 `insufficient_scope`** (RFC 6750). The Micropub
|
||||
Recommendation's error table lists 401 for this case. StarPunk keeps 403 on
|
||||
purpose (see the [v2.9.0 review](../design/v2.9.0/2026-11-03-micropub-review.md),
|
||||
"Kept as is").
|
||||
|
||||
Micropub request handling itself is described in
|
||||
[indieweb-protocols.md](indieweb-protocols.md#micropub).
|
||||
|
||||
## 4. Discovery
|
||||
|
||||
`templates/base.html` puts these links on every page, whether or not
|
||||
`WEBMENTION_ENABLED` is set (v2.8.0):
|
||||
|
||||
```html
|
||||
<link rel="micropub" href="<SITE_URL>/micropub">
|
||||
<link rel="indieauth-metadata" href="<SITE_URL>/.well-known/oauth-authorization-server">
|
||||
<link rel="authorization_endpoint" href="<SITE_URL>/auth/authorize">
|
||||
<link rel="token_endpoint" href="<SITE_URL>/auth/token">
|
||||
<link rel="me" href="<author home>">
|
||||
<link rel="author" href="<author home>">
|
||||
```
|
||||
|
||||
## Future Considerations
|
||||
Micropub clients start from the URL the user types, which is normally
|
||||
`ADMIN_ME`. If `ADMIN_ME` is the StarPunk site itself, the links above are
|
||||
enough. If it is another page, that page must advertise StarPunk's
|
||||
`indieauth-metadata` (or `authorization_endpoint` and `token_endpoint`) and
|
||||
`micropub` links. Otherwise clients won't find StarPunk
|
||||
([ADR-064](../decisions/ADR-064-oidc-admin-login-and-builtin-indieauth-server.md) §3).
|
||||
|
||||
### Potential V2 Enhancements (NOT for V1)
|
||||
1. **Multi-user support**
|
||||
- Would require user management
|
||||
- Still use external auth
|
||||
## 5. Data (migration 011)
|
||||
|
||||
2. **Multiple token endpoints**
|
||||
- Support different providers per user
|
||||
- Endpoint discovery from user domain
|
||||
| Table | Purpose | Key columns |
|
||||
|---|---|---|
|
||||
| `sessions` | Admin browser sessions | `session_token_hash`, `me`, `expires_at`, `last_used_at`, `user_agent`, `ip_address` |
|
||||
| `oidc_login_state` | One pending OIDC login | `state` (PK), `nonce`, `code_verifier`, `next_url`, `expires_at` |
|
||||
| `indieauth_codes` | Authorization codes | `code_hash`, `client_id`, `redirect_uri`, `scope`, `code_challenge`, `expires_at`, `used_at` |
|
||||
| `indieauth_tokens` | Access tokens | `token_hash`, `client_id`, `scope`, `expires_at`, `last_used_at`, `revoked_at` |
|
||||
|
||||
3. **Token caching layer**
|
||||
- Redis for distributed caching
|
||||
- Longer TTL with refresh
|
||||
The legacy v1 tables `auth_state`, `tokens` and `authorization_codes` are
|
||||
not used (see the migration's header comment).
|
||||
|
||||
### Explicitly NOT Implementing
|
||||
1. **Custom authorization server**
|
||||
- Violates simplicity principle
|
||||
- Maintenance burden
|
||||
## 6. Security properties and known gaps
|
||||
|
||||
2. **Password authentication**
|
||||
- Not IndieWeb compliant
|
||||
- Security burden
|
||||
In place:
|
||||
|
||||
3. **JWT validation**
|
||||
- Not part of IndieAuth spec
|
||||
- Unnecessary complexity
|
||||
- No secrets at rest: session tokens, codes and access tokens are stored only
|
||||
as SHA-256 hashes.
|
||||
- OIDC: PKCE S256, a nonce, single-use expiring state, issuer checks (in the
|
||||
metadata and the RFC 9207 `iss`), full ID-token validation, and a single
|
||||
allowed `sub`.
|
||||
- IndieAuth: PKCE S256 required, single-use 10-minute codes bound to the
|
||||
client and redirect URI, `iss` in every authorization response, and
|
||||
untrusted redirect URIs never followed.
|
||||
- Post-login redirects are limited to this site.
|
||||
|
||||
## Testing Strategy
|
||||
CSRF (v2.11.0): admin form POSTs, `/auth/logout` and the consent form's
|
||||
approve button need a `csrf_token` field (or `X-CSRF-Token` header) equal to
|
||||
an HMAC-SHA256 of the `starpunk_session` cookie under `SECRET_KEY`; anything
|
||||
else gets 400. The token changes with every login and needs no storage.
|
||||
Endpoints apps call with bearer tokens are not affected. The client metadata
|
||||
fetch uses `webmention_ssrf.safe_get` (every redirect hop checked, the
|
||||
connection pinned, a 5-second limit).
|
||||
|
||||
### Unit Tests
|
||||
```python
|
||||
# Test external verification
|
||||
@patch('httpx.get')
|
||||
def test_token_verification(mock_get):
|
||||
# Mock successful response
|
||||
mock_get.return_value.status_code = 200
|
||||
mock_get.return_value.json.return_value = {
|
||||
'me': 'https://example.com',
|
||||
'scope': 'create'
|
||||
}
|
||||
Known gaps:
|
||||
|
||||
result = verify_token('test-token')
|
||||
assert result is not None
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
```python
|
||||
# Test with real endpoint (in CI)
|
||||
def test_real_token_verification():
|
||||
# Use test token from tokens.indieauth.com
|
||||
token = get_test_token()
|
||||
result = verify_token(token)
|
||||
assert result['me'] == TEST_USER
|
||||
```
|
||||
|
||||
### Manual Testing
|
||||
1. Configure domain with IndieAuth links
|
||||
2. Use Quill or Indigenous
|
||||
3. Create test post
|
||||
4. Verify token caching
|
||||
|
||||
## Metrics for Success
|
||||
|
||||
### Quantitative Metrics
|
||||
- **Code removed**: >500 lines
|
||||
- **Database tables removed**: 2
|
||||
- **Complexity reduction**: ~40%
|
||||
- **Test coverage maintained**: >90%
|
||||
- **Performance**: <500ms token verification
|
||||
|
||||
### Qualitative Metrics
|
||||
- **Clarity**: Clear separation of concerns
|
||||
- **Maintainability**: No auth code to maintain
|
||||
- **Security**: Specialized providers
|
||||
- **Flexibility**: User choice of providers
|
||||
- **Simplicity**: Focus on core functionality
|
||||
|
||||
---
|
||||
|
||||
**Document Version**: 1.0
|
||||
**Created**: 2025-11-24
|
||||
**Author**: StarPunk Architecture Team
|
||||
**Purpose**: Document simplified authentication architecture after IndieAuth server removal
|
||||
- `scopes_supported` leaves out `draft`, although Micropub honours it and
|
||||
the consent screen grants it when requested. Unknown scopes are also shown
|
||||
and granted as requested; they just have no effect.
|
||||
- No token introspection endpoint and no refresh tokens.
|
||||
@@ -1,233 +1,294 @@
|
||||
# Syndication Architecture
|
||||
# Syndication (POSSE) Architecture
|
||||
|
||||
## Overview
|
||||
StarPunk's syndication architecture provides multiple feed formats for content distribution, ensuring broad compatibility with feed readers and IndieWeb tools while maintaining simplicity.
|
||||
Current as of v2.10.1. Code: `starpunk/syndication.py`.
|
||||
|
||||
## Current State (v1.1.0)
|
||||
```
|
||||
┌─────────────┐
|
||||
│ Database │
|
||||
│ (Notes) │
|
||||
└──────┬──────┘
|
||||
│
|
||||
┌──────▼──────┐
|
||||
│ feed.py │
|
||||
│ (RSS 2.0) │
|
||||
└──────┬──────┘
|
||||
│
|
||||
┌──────▼──────┐
|
||||
│ /feed.xml │
|
||||
│ endpoint │
|
||||
└─────────────┘
|
||||
```
|
||||
StarPunk publishes on its own site first and then copies each note to other
|
||||
networks (POSSE: Publish on your Own Site, Syndicate Elsewhere). Later
|
||||
edits, unpublishing and deletes follow the copies.
|
||||
|
||||
## Target Architecture (v1.1.2+)
|
||||
```
|
||||
┌─────────────┐
|
||||
│ Database │
|
||||
│ (Notes) │
|
||||
└──────┬──────┘
|
||||
│
|
||||
┌──────▼──────────────────┐
|
||||
│ Feed Generation Layer │
|
||||
├──────────┬───────────────┤
|
||||
│ feed.py │ json_feed.py │
|
||||
│ RSS/ATOM│ JSON │
|
||||
└──────────┴───────────────┘
|
||||
│
|
||||
┌──────▼──────────────────┐
|
||||
│ Feed Endpoints │
|
||||
├─────────┬───────────────┤
|
||||
│/feed.xml│ /feed.atom │
|
||||
│ (RSS) │ (ATOM) │
|
||||
├─────────┼───────────────┤
|
||||
│ /feed.json │
|
||||
│ (JSON Feed) │
|
||||
└─────────────────────────┘
|
||||
```
|
||||
> Feeds (RSS, Atom, JSON Feed, OPML) and WebSub are pull and push
|
||||
> distribution of the site itself, not crossposting. They are described in
|
||||
> [indieweb-protocols.md](indieweb-protocols.md#feeds). This document
|
||||
> covered feeds before v2.1.0.
|
||||
|
||||
## Design Principles
|
||||
Decisions: [ADR-065](../decisions/ADR-065-direct-syndication-posse.md)
|
||||
(GoToSocial, queue model), [ADR-066](../decisions/ADR-066-bluesky-syndication.md)
|
||||
(Bluesky), [ADR-068](../decisions/ADR-068-likes-and-reposts.md) (likes and
|
||||
reposts), [ADR-069](../decisions/ADR-069-threads-syndication.md) (Threads),
|
||||
[ADR-070](../decisions/ADR-070-crosspost-links-and-cards.md) (links and link
|
||||
cards), and the [v2.9.0 Micropub review](../design/v2.9.0/2026-11-03-micropub-review.md)
|
||||
(bookmarks, `mp-syndicate-to` default).
|
||||
|
||||
### 1. Format Independence
|
||||
Each syndication format operates independently:
|
||||
- No shared state between formats
|
||||
- Failures in one don't affect others
|
||||
- Can be enabled/disabled individually
|
||||
## Targets
|
||||
|
||||
### 2. Shared Data Access
|
||||
All formats read from the same data source:
|
||||
- Single query pattern for notes
|
||||
- Consistent ordering (newest first)
|
||||
- Same publication status filtering
|
||||
A target exists when its credentials are configured. `get_targets()` builds
|
||||
the list from config on each call and caches one client object per target
|
||||
in the process.
|
||||
|
||||
### 3. Library Leverage
|
||||
Maximize use of existing libraries:
|
||||
- `feedgen` for RSS and ATOM
|
||||
- Native Python `json` for JSON Feed
|
||||
- No custom XML generation
|
||||
| Target id | Client | Enabled by | Micropub `uid` | Text limit | Images | Edits follow? |
|
||||
|---|---|---|---|---|---|---|
|
||||
| `gotosocial` | `MastodonClient` (any Mastodon-API server) | `GOTOSOCIAL_URL` + `GOTOSOCIAL_ACCESS_TOKEN` | `GOTOSOCIAL_URL/` | `GOTOSOCIAL_MAX_CHARS` (5000) | `GOTOSOCIAL_MAX_MEDIA` (4) | Yes |
|
||||
| `bluesky` | `BlueskyClient` (AT Protocol, app password) | `BLUESKY_HANDLE` + `BLUESKY_APP_PASSWORD` (`BLUESKY_SERVICE`, default `https://bsky.social`) | `https://bsky.app/profile/<handle>` | `BLUESKY_MAX_CHARS` (300) | 4 | No |
|
||||
| `threads` | `ThreadsClient` (Meta Graph API) | `THREADS_ACCESS_TOKEN` (`THREADS_USERNAME` for links) | `https://www.threads.com/@<username>` | `THREADS_MAX_CHARS` (500) | `THREADS_MAX_MEDIA` (4) | No |
|
||||
|
||||
## Component Design
|
||||
Display names come from `GOTOSOCIAL_NAME`, `BLUESKY_NAME` and
|
||||
`THREADS_NAME`. GoToSocial statuses use `GOTOSOCIAL_VISIBILITY` (default
|
||||
`public`). The GoToSocial token needs `write:statuses` and `write:media`,
|
||||
plus `read:search` and `write:favourites` for likes and boosts.
|
||||
|
||||
### Feed Generation Module (`feed.py`)
|
||||
**Current Responsibility**: RSS 2.0 generation
|
||||
**Future Enhancement**: Add ATOM generation function
|
||||
Bluesky and Threads can't edit posts through their APIs. Once a copy exists
|
||||
there, edits to the note don't change it, but unpublish and delete still
|
||||
remove it.
|
||||
|
||||
```python
|
||||
# Pseudocode structure
|
||||
def generate_rss_feed(notes, config) -> str
|
||||
def generate_atom_feed(notes, config) -> str # New
|
||||
```
|
||||
## Choosing targets
|
||||
|
||||
### JSON Feed Module (`json_feed.py`)
|
||||
**New Component**: Dedicated JSON Feed generation
|
||||
`select_targets(note_id, target_ids)` records which targets a note goes to.
|
||||
Every configured target not in the list is disabled; if it already has a
|
||||
copy, that copy is deleted. Targets the note never went to get no row.
|
||||
|
||||
```python
|
||||
# Pseudocode structure
|
||||
def generate_json_feed(notes, config) -> str
|
||||
def format_json_item(note) -> dict
|
||||
```
|
||||
| Origin | Targets |
|
||||
|---|---|
|
||||
| Admin editor, new note | One checkbox per target, all checked by default |
|
||||
| Admin editor, edit | Checkboxes reflect the targets currently enabled; saving applies the selection |
|
||||
| Micropub create | Only the targets in `mp-syndicate-to` (form field, `mp-syndicate-to[]`, or inside JSON `properties`). Values may be a target's `uid` (trailing slash ignored) or its id. **No `mp-syndicate-to` means no crossposting** (v2.9.0; before, all targets) |
|
||||
| Micropub update / delete | Target selection is unchanged; the change follows existing copies |
|
||||
|
||||
### Route Handlers
|
||||
Simple pass-through to generation functions:
|
||||
```python
|
||||
@app.route('/feed.xml') # Existing
|
||||
@app.route('/feed.atom') # New
|
||||
@app.route('/feed.json') # New
|
||||
```
|
||||
Micropub clients learn the targets from `q=syndicate-to` and `q=config`
|
||||
(`[{"uid", "name"}]`). Selecting targets for a draft records them; nothing
|
||||
is sent until the note is published.
|
||||
|
||||
## Data Flow
|
||||
## Data: the `syndications` table (migration 012)
|
||||
|
||||
1. **Request**: Client requests feed at endpoint
|
||||
2. **Query**: Fetch published notes from database
|
||||
3. **Transform**: Convert notes to format-specific structure
|
||||
4. **Serialize**: Generate final output (XML/JSON)
|
||||
5. **Response**: Return with appropriate Content-Type
|
||||
One row per (note, target), unique on `(note_id, target)`:
|
||||
|
||||
## Microformats2 Architecture
|
||||
| Column | Meaning |
|
||||
|---|---|
|
||||
| `enabled` | Whether the note should have a copy on this target |
|
||||
| `remote_id`, `remote_url` | The copy: status id, `at://` URI, Threads media id, or a `favourite:<id>` / `reblog:<id>` marker for native likes and boosts |
|
||||
| `state` | `pending`, `synced` or `failed` |
|
||||
| `version` | Incremented on every change that needs a reconcile |
|
||||
| `attempts`, `last_error`, `next_attempt_at` | Retry bookkeeping |
|
||||
| `claimed_until` | Lease held by the worker processing the row |
|
||||
|
||||
### Template Layer Enhancement
|
||||
Microformats2 operates at the HTML template layer:
|
||||
`note_id` deliberately has no foreign key, so the row outlives a hard
|
||||
delete and the worker can still remove the copy.
|
||||
|
||||
## Change hooks
|
||||
|
||||
| Event | Call | Effect |
|
||||
|---|---|---|
|
||||
| Admin create/edit | `select_targets()` | Rows set to `pending`, version bumped |
|
||||
| Micropub create | `select_targets(resolve_targets(mp-syndicate-to))` | After photos are attached |
|
||||
| `update_note()` (admin or Micropub, including publish/unpublish) | `note_changed(id)` | Every row of the note set to `pending`, version bumped, attempts reset |
|
||||
| Micropub photo changes | `note_changed(id)` | Same |
|
||||
| `delete_note()` (soft or hard), `undelete_note()` | `note_changed(id)` | Same |
|
||||
| Retry button on the edit page | `retry(id)` | That row set to `pending` now, attempts reset |
|
||||
|
||||
## Worker
|
||||
|
||||
`SyndicationWorker` is a daemon thread started in every gunicorn process
|
||||
when at least one target is configured (not under `TESTING`). It runs
|
||||
`process_due()` every `SYNDICATION_POLL_INTERVAL` seconds (default 10), and
|
||||
loops again immediately while there is work.
|
||||
|
||||
`process_due()`:
|
||||
|
||||
1. Calls each client's `maintain()` (the Threads token refresh).
|
||||
2. Selects up to 5 rows that are `pending`, due, and not claimed.
|
||||
3. Claims each row with an atomic
|
||||
`UPDATE ... SET claimed_until = now + 5 min WHERE state = 'pending' AND
|
||||
(claimed_until IS NULL OR claimed_until < now)`. The other processes skip
|
||||
it. Unlike the Webmention and WebSub workers, there is no file lock: the
|
||||
row claim is the only coordination.
|
||||
4. Reconciles the row (below), then records the result.
|
||||
|
||||
### Reconcile
|
||||
|
||||
```
|
||||
┌──────────────┐
|
||||
│ Data Model │
|
||||
│ (Notes) │
|
||||
└──────┬───────┘
|
||||
│
|
||||
┌──────▼───────┐
|
||||
│ Templates │
|
||||
│ + mf2 markup│
|
||||
└──────┬───────┘
|
||||
│
|
||||
┌──────▼───────┐
|
||||
│ HTML Output │
|
||||
│ (Semantic) │
|
||||
└──────────────┘
|
||||
note published and row enabled?
|
||||
no → remote copy exists? delete it → (None, None)
|
||||
yes → copy exists and target can't edit? → keep it as is
|
||||
copy exists → update (if it's gone remotely: create again)
|
||||
no copy → create
|
||||
```
|
||||
|
||||
### Markup Strategy
|
||||
- **Progressive Enhancement**: Add classes without changing structure
|
||||
- **CSS Independence**: Use mf2-specific classes, not styling classes
|
||||
- **Validation First**: Test with parsers during development
|
||||
Deleted notes are not found by `get_note()`, so they count as "not
|
||||
published".
|
||||
|
||||
## Configuration Requirements
|
||||
### Versions
|
||||
|
||||
### New Configuration Variables
|
||||
```ini
|
||||
# Author information for h-card
|
||||
AUTHOR_NAME = "Site Author"
|
||||
AUTHOR_URL = "https://example.com"
|
||||
AUTHOR_PHOTO = "/static/avatar.jpg" # Optional
|
||||
The row's `version` is read when the row is picked up. `_finish()` sets
|
||||
`state = 'synced'` only if the version is unchanged. If the note was edited
|
||||
while the send was in flight, the row stays `pending` and is reconciled
|
||||
again with the new content. `_fail()` does the same.
|
||||
|
||||
# Feed settings
|
||||
FEED_LIMIT = 50
|
||||
FEED_FORMATS = "rss,atom,json" # Comma-separated
|
||||
```
|
||||
### Retries and idempotency
|
||||
|
||||
## Performance Considerations
|
||||
- `SyndicationError(permanent=True)` fails the row immediately. This covers
|
||||
HTTP 4xx other than 429, a missing target, images too large for Bluesky,
|
||||
a Threads container in `ERROR`/`EXPIRED`, and Graph API auth/permission
|
||||
errors (codes 10, 190, 200).
|
||||
- Other errors retry after `min(30 s × 2^attempts, 6 h)`. After 8 attempts
|
||||
the row becomes `failed`. The error is shown on the edit page with a
|
||||
Retry button.
|
||||
- `RemoteGone` (404, or Graph error 100/33) during an update means the copy
|
||||
was deleted remotely, so a new one is created. During a delete it counts as
|
||||
success.
|
||||
- Duplicate posts after a lost response are prevented as follows:
|
||||
- GoToSocial: an `Idempotency-Key: starpunk-<row id>-<version>` header.
|
||||
- Bluesky and Threads: on a retry (`attempts > 0`), the last 25 posts are
|
||||
searched for one containing the permalink (for Threads reposts, the
|
||||
reposted URL). A match is adopted instead of posting again.
|
||||
- Threads' publish step sometimes fails transiently ("Fatal"); it is
|
||||
retried unless the error is an auth error.
|
||||
|
||||
### Caching Strategy
|
||||
- Feed generation is read-heavy, write-light
|
||||
- Consider caching generated feeds (5-minute TTL)
|
||||
- Invalidate cache on note creation/update
|
||||
## Composing posts
|
||||
|
||||
### Resource Usage
|
||||
- RSS/ATOM: ~O(n) memory for n notes
|
||||
- JSON Feed: Similar memory profile
|
||||
- Microformats2: No additional server resources
|
||||
`Post` carries the note, permalink, tags, the target's character limit,
|
||||
media files, and for likes, reposts and bookmarks, the URL they point at and
|
||||
its fetched title (from the reply-context cache).
|
||||
|
||||
## Security Considerations
|
||||
### Text
|
||||
|
||||
### Content Sanitization
|
||||
- HTML in feeds must be properly escaped
|
||||
- CDATA wrapping for RSS/ATOM
|
||||
- JSON string encoding for JSON Feed
|
||||
- No script injection vectors
|
||||
`compose_text()` builds: body, a blank line, the permalink, a blank line,
|
||||
the hashtags. If that is too long, the body is cut at a word boundary with
|
||||
"…"; the permalink always survives. Hashtags (`#` plus the tag's display
|
||||
name without non-word characters) may take at most a third of the limit. A
|
||||
bookmark's body starts with `🔖 <bookmarked URL>` (v2.9.0). A photo-only
|
||||
note starts with the permalink.
|
||||
|
||||
### Rate Limiting
|
||||
- Apply same limits as HTML endpoints
|
||||
- Consider aggressive caching for feeds
|
||||
- Monitor for feed polling abuse
|
||||
| Target | Body format | Links | Hashtags |
|
||||
|---|---|---|---|
|
||||
| GoToSocial | The note's Markdown, sent with `content_type: text/markdown`; embedded images removed from the text (sent as attachments) | As written | In the text |
|
||||
| Bluesky | Plain text from the rendered Markdown | Markdown link text becomes a clickable link facet; bare URLs and `#tags` get link/tag facets (UTF-8 byte offsets) | In the text, as tag facets |
|
||||
| Threads | Plain text | Markdown links are written `text (url)`, up to 4 different URLs plus the permalink (Threads allows 5); the length counts non-ASCII characters as UTF-8 bytes | None in the text; the first usable tag becomes the post's `topic_tag` |
|
||||
|
||||
## Testing Architecture
|
||||
### Link cards (ADR-070)
|
||||
|
||||
### Unit Tests
|
||||
```
|
||||
tests/
|
||||
├── test_feed.py # Enhanced for ATOM
|
||||
├── test_json_feed.py # New test module
|
||||
└── test_microformats.py # Template parsing tests
|
||||
```
|
||||
- **Bluesky**: if the note has images, they are the embed. Otherwise, if
|
||||
the note links somewhere, StarPunk builds an `app.bsky.embed.external`
|
||||
card for the first link. `page_card()` fetches the page (at most 512 KB,
|
||||
every redirect hop checked against private addresses, 10 s per request,
|
||||
5 redirects) and reads the Open Graph, Twitter or HTML title, description
|
||||
and image. The image (at most 5 MB) is shrunk under 1 MB and uploaded as
|
||||
the thumbnail. If the page can't be fetched, the card has just the URL.
|
||||
A note without links gets a card for its own permalink, built from the
|
||||
note without fetching anything.
|
||||
- **Threads**: no explicit `link_attachment` (Meta rejects one for a page
|
||||
it hasn't crawled yet). Threads previews the first link in the text by
|
||||
itself.
|
||||
- **GoToSocial**: the server makes its own previews.
|
||||
|
||||
### Integration Tests
|
||||
- Validate against external validators
|
||||
- Test feed reader compatibility
|
||||
- Verify IndieWeb tool parsing
|
||||
### Media
|
||||
|
||||
## Backwards Compatibility
|
||||
`_media_files()` collects the note's attached photos first, then images
|
||||
embedded in the note text that are in this site's media library (some
|
||||
Micropub clients put uploaded photos in the content). It stops at the
|
||||
target's image limit. The `large` variant is used when one exists. Alt text
|
||||
comes from the caption, or from the embedded image's alt.
|
||||
|
||||
### URL Structure
|
||||
- `/feed.xml` remains RSS 2.0 (no breaking change)
|
||||
- New endpoints are additive only
|
||||
- Auto-discovery links updated in templates
|
||||
| Target | Handling |
|
||||
|---|---|
|
||||
| GoToSocial | Uploaded via `POST /api/v2/media` with description; re-uploaded on each edit |
|
||||
| Bluesky | `uploadBlob`; images over 1,000,000 bytes are re-encoded as JPEG at decreasing size and quality, and fail permanently if they still don't fit. Aspect ratio is sent |
|
||||
| Threads | Meta fetches each image from its public `/media/` URL. Only JPEG and PNG up to 8 MB are used; others are left off. 1 image → `IMAGE` container, 2 or more → `CAROUSEL` of child containers. Alt text up to 1000 characters |
|
||||
|
||||
### Database
|
||||
- No schema changes required
|
||||
- All features use existing Note model
|
||||
- No migration needed
|
||||
Only notes of kind `note` carry media; likes, reposts and bookmarks don't.
|
||||
|
||||
## Future Extensibility
|
||||
### Likes, reposts and bookmarks
|
||||
|
||||
### Potential Enhancements
|
||||
1. Content negotiation on `/feed`
|
||||
2. WebSub (PubSubHubbub) support
|
||||
3. Custom feed filtering (by tag, date)
|
||||
4. Feed pagination for large sites
|
||||
| Kind | GoToSocial | Bluesky | Threads |
|
||||
|---|---|---|---|
|
||||
| Like of a post on that network | Native favourite (`favourite:<id>`) | Native `app.bsky.feed.like` | Not crossposted (the API can't like) |
|
||||
| Like of anything else | Not crossposted | Not crossposted | Not crossposted |
|
||||
| Repost without comment, of a post on that network | Native boost (`reblog:<id>`) | Native `app.bsky.feed.repost` | Not native: posts `🔁 title` + URL |
|
||||
| Repost with comment, of a post on that network | Status: comment, `🔁 title`, URL | Quote post (`app.bsky.embed.record`) | Comment, `🔁 title`, URL |
|
||||
| Repost of another page | Status: (comment,) `🔁 title`, URL | Post with an external card for the URL | (Comment,) `🔁 title`, URL |
|
||||
| Bookmark | Ordinary status starting `🔖 <url>` | Ordinary post starting `🔖 <url>` | Ordinary post starting `🔖 <url>` |
|
||||
|
||||
### Format Support Matrix
|
||||
| Format | v1.1.0 | v1.1.2 | v1.2.0 |
|
||||
|--------|--------|--------|--------|
|
||||
| RSS 2.0 | ✅ | ✅ | ✅ |
|
||||
| ATOM | ❌ | ✅ | ✅ |
|
||||
| JSON Feed | ❌ | ✅ | ✅ |
|
||||
| Microformats2 | Partial | Partial | ✅ |
|
||||
"On that network" is decided by the client. GoToSocial resolves the URL
|
||||
through `/api/v2/search?resolve=true`. Bluesky matches
|
||||
`https://bsky.app/profile/<actor>/post/<rkey>` and resolves it to a URI and
|
||||
CID. Native likes and boosts have no URL of their own and nothing to edit.
|
||||
Deleting the note undoes them (`unfavourite` / `unreblog`, or deleting the
|
||||
like or repost record). A like that isn't crossposted is recorded as synced
|
||||
with no remote copy.
|
||||
|
||||
## Decision Rationale
|
||||
## u-syndication
|
||||
|
||||
### Why Multiple Formats?
|
||||
1. **No Universal Standard**: Different ecosystems prefer different formats
|
||||
2. **Low Maintenance**: Feed formats are stable, rarely change
|
||||
3. **User Choice**: Let users pick their preferred format
|
||||
4. **IndieWeb Philosophy**: Embrace plurality and interoperability
|
||||
`templates/note.html` lists every row that has a `remote_url` as "Also on
|
||||
<name>", using `<a class="u-syndication" rel="syndication">`. Micropub
|
||||
`q=source` returns the same URLs as the `syndication` property. The
|
||||
Webmention worker rejects a webmention whose source is one of the note's
|
||||
own copies (e.g. backfed by Bridgy), so a copy is never shown as a response
|
||||
to itself (v2.8.0).
|
||||
|
||||
### Why This Architecture?
|
||||
1. **Simplicity**: Each component has single responsibility
|
||||
2. **Testability**: Isolated components are easier to test
|
||||
3. **Maintainability**: Changes to one format don't affect others
|
||||
4. **Performance**: Can optimize each format independently
|
||||
## Backfeed (v2.13.0, ADR-073)
|
||||
|
||||
## References
|
||||
- [RSS 2.0 Specification](https://www.rssboard.org/rss-specification)
|
||||
- [ATOM RFC 4287](https://tools.ietf.org/html/rfc4287)
|
||||
- [JSON Feed Specification](https://www.jsonfeed.org/)
|
||||
- [Microformats2](https://microformats.org/wiki/microformats2)
|
||||
`starpunk/backfeed.py` brings responses to the copies back to the note.
|
||||
After each crosspost pass, the worker loop calls `backfeed.process_due()`,
|
||||
which checks up to three synced copies of published notes (not likes or
|
||||
reposts) that are due, each under a lease (`backfeed_claimed_until`):
|
||||
|
||||
- **Reading.** Each client has `responses(remote_id, remote_url)`, which
|
||||
returns a `Harvest`: GoToSocial `favourited_by`, `reblogged_by` and
|
||||
public direct replies from `context`; Bluesky `getLikes`,
|
||||
`getRepostedBy`, `getQuotes` and `getPostThread` (depth 1), plus
|
||||
`getProfiles` for reply and quote authors' bios; Threads `replies` and
|
||||
`insights` (like, repost and quote counts). Lists are paged up to 1,000
|
||||
items; a list cut short makes the harvest incomplete.
|
||||
- **Storing.** Responses are upserted into `webmentions` with `origin` (the
|
||||
target id) and `remote_id`, `validation_status = 'valid'`, keyed by
|
||||
`(source_url, target_url)`. Likes and reposts use the copy's URL plus
|
||||
`#liked-by-<actor>` / `#reposted-by-<actor>`. A new row gets the
|
||||
`WEBMENTION_AUTO_APPROVE` default; an existing row keeps its moderation.
|
||||
A complete harvest marks that silo's rows for the note that weren't
|
||||
found `invalid` ("No longer on <silo>"). Authors with `#nobridge` or
|
||||
`#nobot` in their bio, or Bluesky's `!no-unauthenticated` label, are
|
||||
skipped and their rows deleted. If the note's shown responses changed,
|
||||
its webmentions are re-sent (spec 3.1.4). Threads counts go in
|
||||
`syndications.likes_count`, `reposts_count`, `quotes_count`.
|
||||
- **Schedule.** `backfeed_next_at` follows the note's age: 15 minutes (day
|
||||
1), 1 hour (week 1), 1 day (to 30 days), 1 week (to
|
||||
`BACKFEED_MAX_AGE_DAYS`), with up to 10% jitter; none after. Failures back
|
||||
off like crossposts; after 8 they wait a month.
|
||||
- **Backfill.** `start_backfill()` flags every eligible copy
|
||||
(`backfeed_backfill = 1`). Flagged copies are checked only when no
|
||||
routine check is due, at most one every 5 seconds across workers
|
||||
(`site_state.backfeed_backfill_last`). Start and finish times are in
|
||||
`site_state`.
|
||||
- **Refused credentials.** A 401/403 (or Graph auth error) puts the target
|
||||
in `backfeed.denied` until restart; the admin Webmentions page says which
|
||||
scopes are needed. Crossposting is unaffected.
|
||||
|
||||
Avatars of all responses (received and backfed) are saved by
|
||||
`starpunk/avatars.py` and served from `/avatars/<sha256>.webp`; the
|
||||
webmention worker fills in missing ones every 10 minutes and removes unused
|
||||
ones weekly.
|
||||
|
||||
## Client sessions and tokens
|
||||
|
||||
- **Bluesky**: `createSession` with the app password, then
|
||||
`refreshSession` when the access JWT expires (a 401 or `ExpiredToken`
|
||||
triggers one re-login and retry). The PDS endpoint is taken from the DID
|
||||
document. Calls are serialized per client with a lock.
|
||||
- **Threads**: long-lived tokens expire after 60 days. `maintain()` tries a
|
||||
refresh (`refresh_access_token`, `th_refresh_token`) at most once a day.
|
||||
The current token is kept in `DATA_PATH/threads_token.json` (mode 0600,
|
||||
written atomically), together with a SHA-256 "seed" of the configured
|
||||
token. A different `THREADS_ACCESS_TOKEN` in the config therefore
|
||||
replaces the saved one. An exclusive `flock` on `threads_token.lock` makes
|
||||
one gunicorn process refresh while the others pick up its result. A failed
|
||||
refresh is retried an hour later (network error) or the next day (HTTP
|
||||
error, e.g. a token less than a day old). The httpx logger is raised to
|
||||
WARNING because Threads takes the token in the URL, and errors never echo
|
||||
the URL.
|
||||
- **GoToSocial**: a static bearer token.
|
||||
|
||||
## Known limits
|
||||
|
||||
- Bluesky and Threads copies don't follow edits (platform limitation).
|
||||
Unticking and re-ticking a target makes a new copy.
|
||||
- No per-target choice for Micropub updates: `mp-syndicate-to` is read only
|
||||
on create.
|
||||
- Likes of ordinary web pages, and all likes on Threads, stay on the site.
|
||||
@@ -1,5 +1,10 @@
|
||||
# StarPunk Technology Stack - Quick Reference
|
||||
|
||||
> **Historical document.** This describes StarPunk's original v0.x technology
|
||||
> choices (for example IndieLogin.com for admin login) and is kept for
|
||||
> context only. It does not describe the current system; see
|
||||
> [technology-stack.md](technology-stack.md) and [overview.md](overview.md).
|
||||
|
||||
## Project Understanding
|
||||
|
||||
StarPunk is a **minimal, single-user IndieWeb CMS** for publishing notes with RSS syndication. The core philosophy is radical simplicity: "Every line of code must justify its existence."
|
||||
|
||||
+120
-1080
File diff suppressed because it is too large.
Load diff
@@ -1,7 +1,9 @@
|
||||
# ADR-005: IndieLogin Authentication Integration
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
> Admin login stopped using IndieLogin.com during v1.x (it discovered the authorization endpoint from `ADMIN_ME`) and has used OIDC since v2.0.0 (ADR-064). (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
The user has explicitly required external IndieLogin authentication via indielogin.com for V1. This is different from implementing a full IndieAuth server (which CLAUDE.MD mentions). The distinction is important:
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-007: Slug Generation Algorithm
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-062 (was: Accepted)
|
||||
|
||||
> New notes get timestamp slugs by default since v1.5.0 (ADR-062). (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
@@ -466,7 +468,7 @@ def update_note_slug(note_id, new_slug):
|
||||
- [IndieWeb Permalink Design](https://indieweb.org/permalink)
|
||||
- [URL Slug Best Practices](https://moz.com/learn/seo/url)
|
||||
- [Python secrets Module](https://docs.python.org/3/library/secrets.html)
|
||||
- [ADR-004: File-Based Note Storage](/home/phil/Projects/starpunk/docs/decisions/ADR-004-file-based-note-storage.md)
|
||||
- [ADR-004: File-Based Note Storage](ADR-004-file-based-note-storage.md)
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
|
||||
@@ -444,9 +444,9 @@ When ready to release:
|
||||
|
||||
### Internal Documentation
|
||||
|
||||
- [Versioning Strategy](/home/phil/Projects/starpunk/docs/standards/versioning-strategy.md) - Complete versioning specification
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md) - System architecture
|
||||
- [Development Setup](/home/phil/Projects/starpunk/docs/standards/development-setup.md) - Development workflow
|
||||
- [Versioning Strategy](../standards/versioning-strategy.md) - Complete versioning specification
|
||||
- [Architecture Overview](../architecture/overview.md) - System architecture
|
||||
- [Development Setup](../standards/development-setup.md) - Development workflow
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -470,10 +470,10 @@ Created comprehensive documentation:
|
||||
|
||||
### Internal Documentation
|
||||
|
||||
- [ADR-008: Versioning Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-008-versioning-strategy.md) - Semantic versioning decision
|
||||
- [Versioning Strategy](/home/phil/Projects/starpunk/docs/standards/versioning-strategy.md) - Complete versioning spec
|
||||
- [Git Branching Strategy](/home/phil/Projects/starpunk/docs/standards/git-branching-strategy.md) - Complete branching spec
|
||||
- [Development Setup](/home/phil/Projects/starpunk/docs/standards/development-setup.md) - Development workflow
|
||||
- [ADR-008: Versioning Strategy](ADR-008-versioning-strategy.md) - Semantic versioning decision
|
||||
- [Versioning Strategy](../standards/versioning-strategy.md) - Complete versioning spec
|
||||
- [Git Branching Strategy](../standards/git-branching-strategy.md) - Complete branching spec
|
||||
- [Development Setup](../standards/development-setup.md) - Development workflow
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
> Partly superseded by ADR-064: the IndieLogin login flow was replaced by OIDC. The single `starpunk/auth.py` module and the session design are still in use. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
With the core utilities and notes management complete, StarPunk needs an authentication system for the admin interface. ADR-005 already decided on using IndieLogin.com as the authentication provider. This ADR defines the specific module architecture and implementation approach for the authentication system.
|
||||
@@ -204,7 +206,7 @@ Balance between security and usability:
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-005: IndieLogin Authentication](/home/phil/Projects/starpunk/docs/decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-005: IndieLogin Authentication](ADR-005-indielogin-authentication.md)
|
||||
- [IndieAuth Specification](https://www.w3.org/TR/indieauth/)
|
||||
- [OWASP Session Management](https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html)
|
||||
- [Flask Security Best Practices](https://flask.palletsprojects.com/en/3.0.x/security/)
|
||||
|
||||
@@ -507,8 +507,8 @@ Dev auth implementation is successful if:
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-005: IndieLogin Authentication](/home/phil/Projects/starpunk/docs/decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-010: Authentication Module Design](/home/phil/Projects/starpunk/docs/decisions/ADR-010-authentication-module-design.md)
|
||||
- [ADR-005: IndieLogin Authentication](ADR-005-indielogin-authentication.md)
|
||||
- [ADR-010: Authentication Module Design](ADR-010-authentication-module-design.md)
|
||||
- [OWASP Authentication Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html)
|
||||
- [The Twelve-Factor App - Dev/Prod Parity](https://12factor.net/dev-prod-parity)
|
||||
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
# ADR-012: Flaky Test Removal and Test Quality Standards
|
||||
|
||||
## Status
|
||||
Accepted (implemented in v1.5.0; was: Proposed)
|
||||
|
||||
## Context
|
||||
|
||||
The test suite contains several categories of flaky tests that pass inconsistently. These tests consume developer time without providing proportional value. Per the project philosophy ("Every line of code must justify its existence"), we must evaluate whether these tests should be kept, fixed, or removed.
|
||||
|
||||
## Analysis by Test Category
|
||||
|
||||
### 1. Migration Race Condition Tests (`test_migration_race_condition.py`)
|
||||
|
||||
**Failing Tests:**
|
||||
- `test_debug_level_for_early_retries` - Log message matching
|
||||
- `test_new_connection_per_retry` - Connection count assertions
|
||||
- `test_concurrent_workers_barrier_sync` - Multiprocessing pickle errors
|
||||
- `test_sequential_worker_startup` - Missing table errors
|
||||
- `test_worker_late_arrival` - Missing table errors
|
||||
- `test_single_worker_performance` - Missing table errors
|
||||
- `test_concurrent_workers_performance` - Pickle errors
|
||||
|
||||
**Value Analysis:**
|
||||
- The migration retry logic with exponential backoff is *critical* for production deployments with multiple Gunicorn workers
|
||||
- However, the flaky tests are testing implementation details (log levels, exact connection counts) rather than behavior
|
||||
- The multiprocessing tests fundamentally cannot work reliably because:
|
||||
1. `multiprocessing.Manager().Barrier()` objects cannot be pickled for `Pool.map()`
|
||||
2. The worker functions require Flask app context that doesn't transfer across processes
|
||||
3. SQLite database files in temp directories may not be accessible across process boundaries
|
||||
|
||||
**Root Cause:** Test design is flawed. These are attempting integration/stress tests using unit test infrastructure.
|
||||
|
||||
**Recommendation: REMOVE the multiprocessing tests entirely. KEEP and FIX the unit tests.**
|
||||
|
||||
Specifically:
|
||||
- **REMOVE:** `TestConcurrentExecution` class (all 3 tests) - fundamentally broken by design
|
||||
- **REMOVE:** `TestPerformance` class (both tests) - same multiprocessing issues
|
||||
- **KEEP:** `TestRetryLogic` - valuable, just needs mock fixes
|
||||
- **KEEP:** `TestGraduatedLogging` - valuable, needs logger configuration fixes
|
||||
- **KEEP:** `TestConnectionManagement` - valuable, needs assertion fixes
|
||||
- **KEEP:** `TestErrorHandling` - valuable, tests critical rollback behavior
|
||||
- **KEEP:** `TestBeginImmediateTransaction` - valuable, tests locking mechanism
|
||||
|
||||
**Rationale for removal:** If we need to test concurrent migration behavior, that requires:
|
||||
1. A proper integration test framework (not pytest unit tests)
|
||||
2. External process spawning (not multiprocessing.Pool)
|
||||
3. Real filesystem isolation
|
||||
4. This is out of scope for V1 - the code works; the tests are the problem
|
||||
|
||||
---
|
||||
|
||||
### 2. Feed Route Tests (`test_routes_feeds.py`)
|
||||
|
||||
**Failing Assertions:**
|
||||
- Tests checking for exact `<?xml version="1.0"` but code produces `<?xml version='1.0'` (single quotes)
|
||||
- Tests checking for exact Content-Type with charset but response may vary
|
||||
- Tests checking for exact `<rss version="2.0"` string
|
||||
|
||||
**Value Analysis:**
|
||||
- These tests ARE valuable - they verify feed output format
|
||||
- The tests are NOT flaky per se; they are *brittle* due to over-specific assertions
|
||||
|
||||
**Root Cause:** Tests are asserting implementation details (quote style) rather than semantics (valid XML).
|
||||
|
||||
**Recommendation: FIX by loosening assertions**
|
||||
|
||||
Current (brittle):
|
||||
```python
|
||||
assert b'<?xml version="1.0"' in response.data
|
||||
```
|
||||
|
||||
Better (semantic):
|
||||
```python
|
||||
assert b'<?xml version=' in response.data
|
||||
assert b"encoding=" in response.data
|
||||
```
|
||||
|
||||
The test file already has SOME tests using the correct pattern (lines 72, 103, 265). The ATOM test on line 84 is the outlier - it should match the RSS tests.
|
||||
|
||||
---
|
||||
|
||||
### 3. Feed Streaming Test (`test_routes_feed.py`)
|
||||
|
||||
**Failing Test:** `test_feed_route_streaming`
|
||||
|
||||
**Current assertion (line 124):**
|
||||
```python
|
||||
assert "ETag" in response.headers
|
||||
```
|
||||
|
||||
**But the test comment says:**
|
||||
```python
|
||||
# Cached responses include ETags for conditional requests
|
||||
# (Phase 3 caching was added, replacing streaming for better performance)
|
||||
```
|
||||
|
||||
**Value Analysis:**
|
||||
- The test title says "streaming" but the implementation uses caching
|
||||
- The test is actually correct (ETag SHOULD be present)
|
||||
- If ETag is NOT present, that's a bug in the feed caching implementation
|
||||
|
||||
**Root Cause:** This is not a flaky test - if it fails, there's an actual bug. The test name is misleading but the assertion is correct.
|
||||
|
||||
**Recommendation: KEEP and RENAME to `test_feed_route_caching`**
|
||||
|
||||
---
|
||||
|
||||
### 4. Search Security Tests (`test_search_security.py`)
|
||||
|
||||
**Analysis of the file:** After reviewing, I see no obviously flaky tests. All tests are:
|
||||
- Testing XSS prevention (correct)
|
||||
- Testing SQL injection prevention (correct)
|
||||
- Testing input validation (correct)
|
||||
- Testing pagination limits (correct)
|
||||
|
||||
**Possible flakiness sources:**
|
||||
- FTS5 special character handling varies by SQLite version
|
||||
- Tests that accept multiple status codes (200, 400, 500) are defensive, not flaky
|
||||
|
||||
**Recommendation: KEEP all tests**
|
||||
|
||||
If specific flakiness is identified, it's likely due to SQLite FTS5 version differences, which should be documented rather than the tests removed.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
### Remove Entirely
|
||||
1. `TestConcurrentExecution` class from `test_migration_race_condition.py`
|
||||
2. `TestPerformance` class from `test_migration_race_condition.py`
|
||||
|
||||
### Fix Tests (Developer Action Items)
|
||||
1. **`test_routes_feeds.py` line 84:** Change `assert b'<?xml version="1.0"'` to `assert b'<?xml version='`
|
||||
2. **`test_routes_feed.py` line 117:** Rename test from `test_feed_route_streaming` to `test_feed_route_caching`
|
||||
3. **`test_migration_race_condition.py`:** Fix logger configuration in `TestGraduatedLogging` tests to ensure DEBUG level is captured
|
||||
4. **`test_migration_race_condition.py`:** Fix mock setup in `test_new_connection_per_retry` to accurately count connection attempts
|
||||
|
||||
### Keep As-Is
|
||||
1. All tests in `test_search_security.py`
|
||||
2. All non-multiprocessing tests in `test_migration_race_condition.py` (after fixes)
|
||||
3. All other tests in `test_routes_feeds.py` and `test_routes_feed.py`
|
||||
|
||||
## Rationale
|
||||
|
||||
1. **Project philosophy alignment:** Tests that cannot reliably pass do not justify their existence. They waste developer time and erode confidence in the test suite.
|
||||
|
||||
2. **Pragmatic approach:** The migration concurrency code is tested in production by virtue of running with multiple Gunicorn workers. Manual testing during deployment is more reliable than broken multiprocessing tests.
|
||||
|
||||
3. **Test semantics over implementation:** Tests should verify behavior, not implementation details like quote styles in XML.
|
||||
|
||||
4. **Maintainability:** A smaller, reliable test suite is better than a larger, flaky one.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- Faster, more reliable CI/CD pipeline
|
||||
- Increased developer confidence in test results
|
||||
- Reduced time spent debugging test infrastructure
|
||||
- Tests that fail actually indicate bugs
|
||||
|
||||
### Negative
|
||||
- Reduced test coverage for concurrent migration scenarios
|
||||
- Manual testing required for multi-worker deployments
|
||||
|
||||
### Mitigations
|
||||
- Document the multi-worker testing procedure in deployment docs
|
||||
- Consider adding integration tests in a separate test category (not run in CI) for concurrent scenarios
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### 1. Fix the multiprocessing tests
|
||||
**Rejected:** Would require significant refactoring to use subprocess spawning instead of multiprocessing.Pool. The complexity is not justified for V1 given the code works correctly in production.
|
||||
|
||||
### 2. Mark tests as `@pytest.mark.skip`
|
||||
**Rejected:** Skipped tests are just noise. They either work and should run, or they don't work and should be removed. "Skip" is procrastination.
|
||||
|
||||
### 3. Use pytest-xdist for parallel testing
|
||||
**Rejected:** Does not solve the fundamental issue of needing to spawn external processes with proper app context.
|
||||
|
||||
### 4. Move to integration test framework (e.g., testcontainers)
|
||||
**Considered for future:** This is the correct long-term solution but is out of scope for V1. Should be considered for V2 if concurrent migration testing is deemed critical.
|
||||
@@ -4,6 +4,8 @@
|
||||
|
||||
Accepted
|
||||
|
||||
> Its deferral of WebSub is superseded by ADR-071 (v2.10.0), and "no feed caching" by the feed cache added in v1.1.2. RSS is now one of three feed formats (ADR-038) and is served at `/feed.rss` (`/feed.xml` still works). (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
Phase 5 requires implementing RSS feed generation for syndicating published notes. We need to decide on the implementation approach, feed format, caching strategy, and technical details for generating a standards-compliant RSS feed.
|
||||
@@ -365,8 +367,8 @@ The decision will be validated by:
|
||||
- [Feed Discovery](https://indieweb.org/feed_discovery)
|
||||
|
||||
### Internal Documentation
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
|
||||
- [Phase 5 Design](/home/phil/Projects/starpunk/docs/designs/phase-5-rss-and-container.md)
|
||||
- [Architecture Overview](../architecture/overview.md)
|
||||
- [Phase 5 Design](../design/v1.0.0/phase-5-rss-and-container.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -87,10 +87,10 @@ The developer should:
|
||||
6. Tag release after merge: `git tag -a v0.6.0 -m "Release 0.6.0: RSS feed and production container"`
|
||||
|
||||
## References
|
||||
- [Versioning Strategy](/home/phil/Projects/starpunk/docs/standards/versioning-strategy.md)
|
||||
- [Git Branching Strategy](/home/phil/Projects/starpunk/docs/standards/git-branching-strategy.md)
|
||||
- [Phase 5 Design](/home/phil/Projects/starpunk/docs/designs/phase-5-rss-and-container.md)
|
||||
- [Phase 5 Quick Reference](/home/phil/Projects/starpunk/docs/designs/phase-5-quick-reference.md)
|
||||
- [Versioning Strategy](../standards/versioning-strategy.md)
|
||||
- [Git Branching Strategy](../standards/git-branching-strategy.md)
|
||||
- [Phase 5 Design](../design/v1.0.0/phase-5-rss-and-container.md)
|
||||
- [Phase 5 Quick Reference](../design/v1.0.0/phase-5-quick-reference.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -4,6 +4,8 @@
|
||||
|
||||
**Superseded by ADR-019** - IndieLogin.com does not use h-app microformats for client discovery. PKCE implementation is the correct solution.
|
||||
|
||||
> ADR-019 was itself replaced by ADR-025, and the whole IndieLogin login flow by OIDC in ADR-064. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
StarPunk uses IndieLogin.com as a delegated IndieAuth provider for admin authentication. During the first production deployment to https://starpunk.thesatelliteoflove.com, authentication failed with the error:
|
||||
|
||||
@@ -4,6 +4,8 @@
|
||||
|
||||
**Superseded by ADR-019** - IndieLogin.com does not require OAuth metadata endpoint. PKCE implementation is the correct solution.
|
||||
|
||||
> ADR-019 was itself replaced by ADR-025, and the whole IndieLogin login flow by OIDC in ADR-064. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
StarPunk continues to experience "client_id is not registered" errors from IndieLogin.com despite implementing h-app microformats in ADR-016 and making them visible in ADR-006.
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
> The IndieAuth login flow whose HTTP traffic this logged was replaced by OIDC in v2.0.0. General logging is described in ADR-054. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
Superseded by ADR-025 (was: Proposed)
|
||||
|
||||
> ADR-025 is the accepted version of this decision; its login flow was in turn replaced by OIDC (ADR-064). (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-021: IndieAuth Provider Strategy
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
## Context
|
||||
|
||||
@@ -525,8 +525,8 @@ If there is user demand for a more integrated solution, V2 could add:
|
||||
- [IndieWeb Wiki: authorization-endpoint](https://indieweb.org/authorization-endpoint) - Endpoint details
|
||||
|
||||
### Related ADRs
|
||||
- [ADR-005: IndieLogin Authentication](/home/phil/Projects/starpunk/docs/decisions/ADR-005-indielogin-authentication.md) - Original auth decision
|
||||
- [ADR-010: Authentication Module Design](/home/phil/Projects/starpunk/docs/decisions/ADR-010-authentication-module-design.md) - Auth module structure
|
||||
- [ADR-005: IndieLogin Authentication](ADR-005-indielogin-authentication.md) - Original auth decision
|
||||
- [ADR-010: Authentication Module Design](ADR-010-authentication-module-design.md) - Auth module structure
|
||||
|
||||
### Community Examples
|
||||
- [Aaron Parecki's IndieAuth Notes](https://aaronparecki.com/2025/10/08/4/cimd) - Client ID metadata adoption
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-022: Fix IndieAuth Callback Route Mismatch
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Accepted (implemented in v0.9.2; was: Proposed)
|
||||
|
||||
> The `/auth` prefix remains; since v2.0.0 `/auth/callback` is the OIDC callback (ADR-064). (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
We have discovered a critical routing mismatch in our IndieAuth implementation that causes a 404 error when IndieAuth providers redirect back to our application.
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-023: IndieAuth Client Identification Strategy
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
> StarPunk is now a registered OIDC client for login and publishes no h-app. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-024: Static HTML Identity Pages for IndieAuth
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
> The `ADMIN_ME` page must now advertise StarPunk's own IndieAuth endpoints, not indieauth.com's (ADR-064, decision 3). (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-026: IndieAuth Token Exchange Compliance
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
## Context
|
||||
StarPunk's IndieAuth implementation is failing to authenticate with certain providers (specifically gondulf.thesatelliteoflove.com) during the token exchange phase. The provider is rejecting our token exchange requests with a "missing grant_type" error.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-027: IndieAuth Authentication Endpoint Correction
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
## Context
|
||||
|
||||
@@ -178,7 +178,7 @@ Continue current implementation, document incompatibility.
|
||||
- [IndieAuth Specification Section 5.4](https://www.w3.org/TR/indieauth/#authentication-response): Authorization Code Verification for authentication flows
|
||||
- [IndieAuth Specification Section 6.3](https://www.w3.org/TR/indieauth/#token-response): Token Endpoint for authorization flows
|
||||
- [IndieAuth Authentication vs Authorization](https://indieweb.org/IndieAuth#Authentication_vs_Authorization): Community documentation
|
||||
- [ADR-021: IndieAuth Provider Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-021-indieauth-provider-strategy.md): Related architectural decision
|
||||
- [ADR-021: IndieAuth Provider Strategy](ADR-021-indieauth-provider-strategy.md): Related architectural decision
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
Accepted (implemented in v1.0.0-rc.1; was: Proposed)
|
||||
|
||||
> Its token endpoint was removed by ADR-050; tokens are now issued and verified by StarPunk's own IndieAuth server (ADR-064). Update and delete were added by ADR-067. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-029: Micropub IndieAuth Integration Strategy
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-050, then ADR-064 (was: Accepted)
|
||||
|
||||
> The token endpoint designed here was removed in v1.0.0-rc.4 (ADR-050). v2.0.0 added a new built-in IndieAuth server (ADR-064). (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-030: External Token Verification Architecture
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-043, then ADR-064 (was: Accepted)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-031: Database Migration System Redesign
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Accepted (implemented through ADR-032 and ADR-033; was: Proposed)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-033: Database Migration System Redesign
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Accepted (implemented in v1.1.0; was: Proposed)
|
||||
|
||||
## Context
|
||||
The current migration system has a critical flaw: duplicate schema definitions exist between SCHEMA_SQL (used for fresh installs) and individual migration files. This violates the DRY principle and creates maintenance burden. When schema changes are made, developers must remember to update both locations, leading to potential inconsistencies.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-034: Full-Text Search with SQLite FTS5
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Accepted (implemented in v1.1.0; was: Proposed)
|
||||
|
||||
## Context
|
||||
Users need the ability to search through their notes efficiently. Currently, finding specific content requires manually browsing through notes or using external tools. A built-in search capability is essential for any content management system, especially as the number of notes grows.
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-035: Custom Slugs in Micropub
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Accepted (implemented in v1.1.0; was: Proposed)
|
||||
|
||||
> Default slugs later became timestamps (ADR-062); `mp-slug` is still honoured. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
Currently, StarPunk auto-generates slugs from note content (first 5 words). While this works well for most cases, users may want to specify custom slugs for:
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-036: IndieAuth Token Verification Method Diagnosis
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
> Micropub tokens are verified locally since v2.0.0. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
StarPunk is experiencing HTTP 405 Method Not Allowed errors when verifying tokens with the external IndieAuth provider (gondulf.thesatelliteoflove.com). The user questioned "why are we making GET requests to these endpoints?"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-022: Multiple Syndication Format Support
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Accepted (implemented in v1.1.2; was: Proposed)
|
||||
|
||||
## Context
|
||||
StarPunk currently provides RSS 2.0 feed generation using the feedgen library. The IndieWeb community and modern feed readers increasingly support additional syndication formats:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-023: Strict Microformats2 Compliance
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Accepted (implemented in v1.3.0, reviewed in v2.8.0; was: Proposed)
|
||||
|
||||
## Context
|
||||
StarPunk currently implements basic microformats2 markup:
|
||||
|
||||
@@ -158,7 +158,7 @@ The lesson learned: Release candidates are valuable for discovering not just bug
|
||||
- [Semantic Versioning 2.0.0](https://semver.org/)
|
||||
- [ADR-008: Versioning Strategy](../standards/versioning-strategy.md)
|
||||
- [ADR-021: IndieAuth Provider Strategy](./ADR-021-indieauth-provider-strategy.md)
|
||||
- [StarPunk Philosophy](../architecture/philosophy.md)
|
||||
- StarPunk Philosophy (`philosophy.md`, not preserved)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-043-CORRECTED: IndieAuth Endpoint Discovery Architecture
|
||||
|
||||
## Status
|
||||
Accepted (Replaces incorrect understanding in previous ADR-030)
|
||||
Superseded by ADR-064 (was: Accepted; it replaced ADR-030)
|
||||
|
||||
> Admin login uses OIDC and Micropub tokens are verified locally since v2.0.0; no endpoints are discovered from `ADMIN_ME`. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-031: IndieAuth Endpoint Discovery Implementation Details
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-064 (was: Accepted)
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-050: Remove Custom IndieAuth Server
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Superseded by ADR-064 (was: Proposed; implemented in v1.0.0-rc.4)
|
||||
|
||||
> v2.0.0 brought back a built-in IndieAuth server for Micropub clients (ADR-064). (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
@@ -363,7 +365,7 @@ TOKEN_ENDPOINT=https://tokens.indieauth.com/token
|
||||
|
||||
- [IndieAuth Spec](https://www.w3.org/TR/indieauth/)
|
||||
- [tokens.indieauth.com](https://tokens.indieauth.com/)
|
||||
- [ADR-021: IndieAuth Provider Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-021-indieauth-provider-strategy.md)
|
||||
- [ADR-021: IndieAuth Provider Strategy](ADR-021-indieauth-provider-strategy.md)
|
||||
- [Micropub Spec](https://www.w3.org/TR/micropub/)
|
||||
|
||||
---
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-053: Performance Monitoring Strategy
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
Superseded by ADR-072 (was: Accepted, v1.1.1)
|
||||
|
||||
> The metrics system was removed in v2.12.0. (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
StarPunk v1.1.1 introduces performance monitoring to help operators understand system behavior in production. Currently, we have no visibility into:
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-056: Use External IndieAuth Provider (Never Self-Host)
|
||||
|
||||
## Status
|
||||
**ACCEPTED** - This is a permanent, non-negotiable decision.
|
||||
Superseded by ADR-064 (was: Accepted, "permanent, non-negotiable")
|
||||
|
||||
> In v2.0.0 the operator moved login to OIDC and StarPunk became its own IndieAuth server for Micropub clients (ADR-064). (status updated 2026-11-03)
|
||||
|
||||
## Context
|
||||
StarPunk is a minimal IndieWeb CMS focused on **content creation and syndication**, not identity infrastructure. The project philosophy demands that every line of code must justify its existence.
|
||||
|
||||
@@ -271,8 +271,8 @@ Make media support pluggable for advanced features.
|
||||
- [Apple Podcast RSS Requirements](https://podcasters.apple.com/support/823-podcast-requirements)
|
||||
- [Podcast Index Namespace](https://github.com/Podcastindex-org/podcast-namespace)
|
||||
- [RSS 2.0 Enclosure Specification](https://www.rssboard.org/rss-specification#ltenclosuregtSubelementOfLtitemgt)
|
||||
- [ADR-057: Media Attachment Model](/home/phil/Projects/starpunk/docs/decisions/ADR-057-media-attachment-model.md)
|
||||
- [ADR-058: Image Optimization Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-058-image-optimization-strategy.md)
|
||||
- [ADR-057: Media Attachment Model](ADR-057-media-attachment-model.md)
|
||||
- [ADR-058: Image Optimization Strategy](ADR-058-image-optimization-strategy.md)
|
||||
|
||||
## Decision
|
||||
This ADR documents the scope of Full Standardization (Option 3) for the project backlog. Implementation should be scheduled for v1.3.0 and v1.4.0 releases according to the phased approach outlined above.
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
# ADR-060: Route Adapter for the Metrics Dashboard
|
||||
|
||||
## Status
|
||||
|
||||
Superseded by ADR-072 (was: Accepted, v1.1.1-rc.2, 2025-11-25)
|
||||
|
||||
> The metrics dashboard was removed in v2.12.0. (status updated 2026-11-03)
|
||||
|
||||
> Reconstructed on 2026-11-03 from code, changelog and related documents;
|
||||
> the original was lost. The original
|
||||
> (`ADR-060-production-hotfix-metrics-dashboard.md`) was deleted in the
|
||||
> v1.1.1-rc.2 architectural review as "a bug fix, not an architectural
|
||||
> decision", and its content was merged into
|
||||
> `docs/design/v1.1.1/hotfix-v1.1.1-rc2-consolidated.md`. Code
|
||||
> (`starpunk/routes/admin.py`) and CHANGELOG 1.1.1-rc.2 still cite it, so
|
||||
> this record restores the number.
|
||||
>
|
||||
> Not to be confused with the "Option 2 (ADR-060)" for Media RSS mentioned
|
||||
> in ADR-059: that ADR was planned but never written (see
|
||||
> `docs/design/v1.2.0/2025-12-10-documentation-audit-v1.2.0.md`).
|
||||
|
||||
## Context
|
||||
|
||||
v1.1.1-rc.1 added a metrics dashboard. In production it returned 500:
|
||||
|
||||
```
|
||||
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'database'
|
||||
```
|
||||
|
||||
Two problems were found:
|
||||
|
||||
1. **Route conflict.** The notes dashboard (`/admin/`, endpoint
|
||||
`admin.dashboard`) and the new metrics dashboard (`/admin/dashboard`)
|
||||
were easily confused. This was fixed first by moving the metrics
|
||||
dashboard to `/admin/metrics-dashboard` (the "ADR-022" route-naming fix
|
||||
cited in the CHANGELOG; see Numbering below).
|
||||
2. **Data contract mismatch** (the real cause). The template reads
|
||||
`metrics.database.count` and `.avg/.min/.max`, but
|
||||
`starpunk.monitoring.get_metrics_stats()` returns
|
||||
`by_type.database.count` and `avg_duration_ms`, `min_duration_ms`,
|
||||
`max_duration_ms`.
|
||||
|
||||
Either the monitoring module, the template (and its JavaScript), or the
|
||||
route could change.
|
||||
|
||||
## Decision
|
||||
|
||||
Adapt the data **in the route handler** (presentation layer) and leave
|
||||
both the monitoring module's API and the template unchanged.
|
||||
|
||||
- `transform_metrics_for_template()` in `starpunk/routes/admin.py`
|
||||
flattens `by_type.<type>` to `<type>` for `database`, `http` and
|
||||
`render`, renames `avg_duration_ms`/`min_duration_ms`/`max_duration_ms`
|
||||
to `avg`/`min`/`max`, and keeps `total_count`, `max_size` and
|
||||
`process_id`. Missing types get zero values.
|
||||
- If collecting metrics fails, the route flashes a warning and renders
|
||||
the same structure with zeros.
|
||||
- The monitoring imports are defensive: if `starpunk.monitoring` or the
|
||||
pool statistics are unavailable, stub functions return an error value
|
||||
and the dashboard still renders.
|
||||
- The metrics dashboard lives at `/admin/metrics-dashboard`
|
||||
(`admin.metrics_dashboard`); `/admin/` (`admin.dashboard`) stays the
|
||||
notes list, so existing `url_for("admin.dashboard")` redirects are
|
||||
unaffected.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Minimal, low-risk hotfix: one route changed, no template or monitoring
|
||||
changes.
|
||||
- The monitoring module's output stays the same for the JSON endpoint
|
||||
(`/admin/metrics`) and other consumers.
|
||||
- The dashboard degrades instead of failing when metrics are missing.
|
||||
|
||||
### Negative
|
||||
|
||||
- Treats the symptom: the template and the monitoring module still
|
||||
disagree about the data shape, and the adapter must be kept in step
|
||||
with both (noted in `docs/design/v1.1.1/hotfix-v1.1.1-rc2-review.md`).
|
||||
- Errors loading metrics are shown as zeros, which can hide a problem.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
- **Change the monitoring module to return the template's shape**: breaks
|
||||
its other consumers.
|
||||
- **Change the template and its JavaScript**: larger change in a hotfix,
|
||||
with more to test.
|
||||
|
||||
## Numbering
|
||||
|
||||
The CHANGELOG for 1.1.1-rc.2 says "See ADR-022 and ADR-060". The ADR-022
|
||||
it meant was `ADR-022-admin-dashboard-route-conflict-hotfix.md`, deleted in
|
||||
the same review; its number collided with the existing
|
||||
[ADR-022](ADR-022-auth-route-prefix-fix.md) (IndieAuth callback route) and
|
||||
it was not restored. Its decision (the route rename) is recorded above.
|
||||
|
||||
## References
|
||||
|
||||
- `starpunk/routes/admin.py`: `transform_metrics_for_template()`,
|
||||
`metrics_dashboard()`
|
||||
- `docs/design/v1.1.1/hotfix-v1.1.1-rc2-consolidated.md`
|
||||
- `docs/design/v1.1.1/2025-11-25-hotfix-v1.1.1-rc.2-implementation.md`
|
||||
- `docs/design/v1.1.1/hotfix-v1.1.1-rc2-review.md`
|
||||
- ADR-053 (performance monitoring)
|
||||
@@ -0,0 +1,197 @@
|
||||
# ADR-062: Timestamp-Based Slug Format
|
||||
|
||||
## Status
|
||||
Accepted (Supersedes ADR-007)
|
||||
|
||||
## Context
|
||||
|
||||
ADR-007 established a content-based slug generation algorithm that extracts the first 5 words from note content to create URL slugs. While this approach provides readable, SEO-friendly URLs, it has drawbacks for a personal note-taking system:
|
||||
|
||||
1. **Privacy Concerns**: The slug reveals note content in the URL. Private thoughts or draft content become visible in URLs that may be shared or logged.
|
||||
|
||||
2. **SEO Irrelevance**: StarPunk is designed for personal IndieWeb notes, not public blogs. Notes are typically short-form content (similar to tweets or status updates) where SEO optimization provides no meaningful benefit.
|
||||
|
||||
3. **Unpredictable Slugs**: Users cannot predict what slug will be generated without examining their content carefully.
|
||||
|
||||
4. **Edge Case Handling**: Content-based slugs require complex fallback logic for short content, unicode-only content, or special characters.
|
||||
|
||||
5. **Collision Complexity**: Similar notes require random suffix generation (e.g., `hello-world-a7c9`), which undermines the readability goal.
|
||||
|
||||
The user has explicitly stated: "Notes don't need SEO."
|
||||
|
||||
## Decision
|
||||
|
||||
Change the default slug format from content-based to timestamp-based:
|
||||
|
||||
**New Default Format**: `YYYYMMDDHHMMSS`
|
||||
- Example: `20251216143052`
|
||||
- Compact, sortable, predictable
|
||||
- 14 characters total
|
||||
|
||||
**Collision Handling**: Sequential numeric suffix
|
||||
- First collision: `20251216143052-1`
|
||||
- Second collision: `20251216143052-2`
|
||||
- Simple, predictable, no randomness
|
||||
|
||||
**Custom Slugs**: Continue to support user-specified slugs via `mp-slug` Micropub property and the web UI custom slug field. When provided, custom slugs take precedence.
|
||||
|
||||
### Algorithm Specification
|
||||
|
||||
```python
|
||||
def generate_slug(custom_slug: str = None, created_at: datetime = None) -> str:
|
||||
"""Generate a URL-safe slug for a note.
|
||||
|
||||
Args:
|
||||
custom_slug: User-provided custom slug (takes precedence if provided)
|
||||
created_at: Note creation timestamp (defaults to now)
|
||||
|
||||
Returns:
|
||||
URL-safe slug string
|
||||
"""
|
||||
if custom_slug:
|
||||
return sanitize_slug(custom_slug)
|
||||
|
||||
# Default: timestamp-based
|
||||
timestamp = (created_at or datetime.now()).strftime("%Y%m%d%H%M%S")
|
||||
return ensure_unique_slug(timestamp)
|
||||
|
||||
def ensure_unique_slug(base_slug: str) -> str:
|
||||
"""Ensure slug is unique, adding numeric suffix if needed."""
|
||||
if not slug_exists(base_slug):
|
||||
return base_slug
|
||||
|
||||
suffix = 1
|
||||
while slug_exists(f"{base_slug}-{suffix}"):
|
||||
suffix += 1
|
||||
return f"{base_slug}-{suffix}"
|
||||
```
|
||||
|
||||
### Examples
|
||||
|
||||
| Scenario | Generated Slug |
|
||||
|----------|----------------|
|
||||
| Normal note at 2:30:52 PM on Dec 16, 2025 | `20251216143052` |
|
||||
| Second note in same second | `20251216143052-1` |
|
||||
| Third note in same second | `20251216143052-2` |
|
||||
| Note with custom slug "my-custom-slug" | `my-custom-slug` |
|
||||
|
||||
## Rationale
|
||||
|
||||
### Timestamp Format (Score: 9/10)
|
||||
|
||||
**Pros**:
|
||||
- **Privacy**: URL reveals nothing about content
|
||||
- **Predictability**: User knows exactly what slug format to expect
|
||||
- **Sortability**: Chronological sorting by URL is possible
|
||||
- **Simplicity**: No complex word extraction or normalization
|
||||
- **Collision Rarity**: Same-second creation is rare; handled cleanly when it occurs
|
||||
- **Compact**: 14 characters vs potentially 50+ for content-based
|
||||
|
||||
**Cons**:
|
||||
- **Not Memorable**: `20251216143052` is harder to remember than `hello-world`
|
||||
- **No SEO Value**: Search engines prefer descriptive URLs
|
||||
|
||||
**Why Cons Don't Matter**:
|
||||
- StarPunk is for personal notes, not public SEO-optimized content
|
||||
- Notes are accessed via UI, feeds, or bookmarks, not memorized URLs
|
||||
- Users wanting memorable URLs can use custom slugs
|
||||
|
||||
### Sequential Suffix (Score: 10/10)
|
||||
|
||||
**Pros**:
|
||||
- **Deterministic**: No randomness; same collision always gets same suffix
|
||||
- **Simple**: No cryptographic random generation needed
|
||||
- **Readable**: `-1`, `-2` are clear and obvious
|
||||
- **Debuggable**: Easy to understand collision resolution
|
||||
|
||||
**Cons**:
|
||||
- **Enumerable**: Sequential numbers could be probed
|
||||
- Not a real security concern for note slugs
|
||||
|
||||
### Comparison with ADR-007 Approach
|
||||
|
||||
| Aspect | ADR-007 (Content-Based) | ADR-062 (Timestamp) |
|
||||
|--------|------------------------|---------------------|
|
||||
| Privacy | Reveals content | Reveals only time |
|
||||
| Complexity | High (word extraction, normalization, unicode handling) | Low (strftime) |
|
||||
| SEO | Good | None |
|
||||
| Predictability | Low | High |
|
||||
| Collision handling | Random suffix | Sequential suffix |
|
||||
| Fallback cases | Many (short content, unicode, etc.) | None |
|
||||
| Code lines | ~50 | ~15 |
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
1. **Simplified Code**: Remove complex word extraction, unicode normalization, and multiple fallback paths
|
||||
2. **Better Privacy**: Note content never appears in URLs
|
||||
3. **Predictable Output**: Users always know what slug format to expect
|
||||
4. **Fewer Edge Cases**: No special handling for short content, unicode, or special characters
|
||||
5. **Cleaner Collisions**: Sequential suffixes are more intuitive than random strings
|
||||
|
||||
### Negative
|
||||
|
||||
1. **Migration**: Existing notes keep their content-based slugs (no migration needed)
|
||||
2. **Not Human-Readable**: URLs don't describe content
|
||||
3. **No SEO**: Search engines won't benefit from descriptive URLs
|
||||
|
||||
### Mitigations
|
||||
|
||||
**Human-Readable URLs**:
|
||||
- Users wanting descriptive URLs can use custom slugs via `mp-slug`
|
||||
- The web UI custom slug field remains available
|
||||
- This is opt-in rather than default
|
||||
|
||||
**SEO**:
|
||||
- IndieWeb notes are typically not SEO targets
|
||||
- Content is in the page body where search engines can index it
|
||||
- Microformats2 markup provides semantic meaning
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
- Existing notes retain their slugs (no data migration)
|
||||
- New notes use timestamp format by default
|
||||
- Custom slug functionality unchanged
|
||||
- All existing URLs remain valid
|
||||
|
||||
## Testing Requirements
|
||||
|
||||
- Test default slug generation produces timestamp format
|
||||
- Test collision handling with sequential suffixes
|
||||
- Test custom slugs still take precedence
|
||||
- Test edge case: multiple notes in same second
|
||||
- Test reserved slug rejection still works
|
||||
- Verify existing tests for custom slugs pass
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
Changes required in:
|
||||
- `starpunk/slug_utils.py`: Update `generate_slug()` function
|
||||
- `starpunk/notes.py`: Remove content parameter from slug generation call
|
||||
- Tests: Update expected slug formats
|
||||
|
||||
Estimated effort: Small (1-2 hours implementation, 1 hour testing)
|
||||
|
||||
## References
|
||||
|
||||
- ADR-007: Slug Generation Algorithm (Superseded by this ADR)
|
||||
- ADR-035: Custom Slugs (Unchanged; complements this decision)
|
||||
- IndieWeb Permalink Best Practices: https://indieweb.org/permalink
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Default slugs use `YYYYMMDDHHMMSS` format
|
||||
- [ ] Collision handling uses sequential suffix (`-1`, `-2`, etc.)
|
||||
- [ ] Custom slugs via `mp-slug` continue to work
|
||||
- [ ] Custom slugs via web UI continue to work
|
||||
- [ ] Reserved slug validation unchanged
|
||||
- [ ] Existing notes unaffected
|
||||
- [ ] All tests pass
|
||||
- [ ] Code complexity reduced
|
||||
|
||||
---
|
||||
|
||||
**Approved**: 2025-12-16
|
||||
**Architect**: StarPunk Architect Agent
|
||||
**Supersedes**: ADR-007 (Slug Generation Algorithm)
|
||||
@@ -0,0 +1,293 @@
|
||||
# ADR-063: Webmention Implementation
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v1.6.0; revised v2.6.8, v2.6.9, v2.7.0, v2.7.2, v2.8.0, v2.10.1)
|
||||
|
||||
> Reconstructed on 2026-11-03 from code, changelog and related documents;
|
||||
> the original was lost. The v1.6.0 release was never committed: its code
|
||||
> was recovered from the production image (`v1.6.0-rc.2`), but its tests,
|
||||
> this ADR and the `docs/design/v1.6.0/` folder (`background-worker.md`,
|
||||
> `api-contract.md`, `display-moderation.md`, `rate-limiting.md`) were not
|
||||
> in the image. This document describes the design as it exists in
|
||||
> v2.10.1. The numbering of the decisions below is new, except that
|
||||
> Decision 9 (SSRF protection) keeps the number the code cites.
|
||||
|
||||
## Context
|
||||
|
||||
Webmention (W3C Recommendation, https://www.w3.org/TR/webmention/) is how
|
||||
IndieWeb sites tell each other that one page links to another. It is what
|
||||
makes replies, likes and reposts from other sites (and from Mastodon or
|
||||
Bluesky via Bridgy) show up on a note, and what tells the sites a note
|
||||
replies to or links to that it exists.
|
||||
|
||||
Both directions involve fetching URLs chosen by someone else, which is
|
||||
slow, can fail, and can be abused:
|
||||
|
||||
- A receiver must fetch the source page to check that it really links to
|
||||
the target. Doing that inside the request lets anyone make StarPunk
|
||||
fetch arbitrary URLs synchronously, and ties up a server process.
|
||||
- A sender must discover each target's endpoint and POST to it, with
|
||||
retries when the other site is down.
|
||||
- Any fetch can be pointed (directly, by redirect or by DNS) at private
|
||||
addresses on the server's own network (SSRF).
|
||||
- Content from other sites is shown on our pages (XSS).
|
||||
|
||||
StarPunk is a single-user site on SQLite, run under gunicorn with several
|
||||
processes, with no job queue or message broker.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Endpoint and discovery
|
||||
|
||||
- `POST /webmention` receives webmentions (form-encoded as the spec
|
||||
requires; JSON is accepted as an extension);
|
||||
`GET /webmention` returns a short description of the endpoint.
|
||||
- HTML responses advertise it with a `Link: </webmention>; rel="webmention"`
|
||||
header (added alongside other Link headers, not replacing them) and a
|
||||
`<link rel="webmention">` in `base.html`.
|
||||
- `WEBMENTION_ENABLED` (default true) turns the feature off.
|
||||
|
||||
### 2. Receiving is asynchronous
|
||||
|
||||
The endpoint does only cheap, local checks and then answers
|
||||
**202 Accepted**; verification happens later in the background worker.
|
||||
|
||||
Synchronous checks, each answered with 400 on failure: `source` and
|
||||
`target` are well-formed http(s) URLs of at most 2048 characters, they
|
||||
differ, and the target is a **published note** on this site (host compared
|
||||
case-insensitively and without default ports; `#fragment` dropped). Since
|
||||
v2.7.0 mentions of the home page or other pages are refused instead of
|
||||
being stored and never shown.
|
||||
|
||||
The mention is upserted into `webmentions` keyed by `(source_url,
|
||||
target_url)` with `validation_status = 'pending'`, so a re-sent mention is
|
||||
re-verified rather than duplicated.
|
||||
|
||||
### 3. Background worker
|
||||
|
||||
One daemon thread (`WebmentionWorker`) handles both incoming verification
|
||||
and outgoing sending, using two SQLite tables as queues (`webmentions`,
|
||||
`webmention_queue`). No external queue is introduced.
|
||||
|
||||
- Started by `create_app()` when `WEBMENTION_WORKER_ENABLED` is true (not
|
||||
in tests). It polls every `WEBMENTION_POLL_INTERVAL` seconds (10),
|
||||
handles up to `WEBMENTION_BATCH_SIZE` items (5) per pass, and backs off
|
||||
`WEBMENTION_ERROR_BACKOFF` seconds (60) after an unexpected error.
|
||||
- Each pass runs inside `app.app_context()` (v2.6.8: without it every
|
||||
item failed with "Working outside of application context"). The thread
|
||||
lives for the life of the process; it is not stopped on app-context
|
||||
teardown (v2.6.8: that stopped it after the first request).
|
||||
- **Only one process works the queues** (v2.6.9). Every gunicorn process
|
||||
starts a worker, but a worker only processes after taking a
|
||||
non-blocking `flock` on `<data>/webmention-worker.lock`. Others retry
|
||||
each poll, so one takes over if the holder dies.
|
||||
- On acquiring the lock it resets outgoing items stuck in `sending` for
|
||||
more than 5 minutes (orphans from a crash).
|
||||
- `/health?detailed=true` (authenticated) reports the worker's
|
||||
statistics.
|
||||
|
||||
### 4. Verifying a received webmention
|
||||
|
||||
Per Recommendation 3.2.2 and 3.2.4:
|
||||
|
||||
- The source is fetched with an `Accept` header, following redirects
|
||||
(Decision 9), and decoded in its declared character set.
|
||||
- The source must link to the **exact** target URL, by media type: for
|
||||
HTML any link attribute (resolved against the final URL after
|
||||
redirects); for JSON any string value; for `text/*` a substring; other
|
||||
types never match. Fragments are ignored (v2.7.0).
|
||||
- The source is parsed with mf2py. The responding entry and its author are
|
||||
found with the authorship algorithm (v2.8.0); the mention type is one of
|
||||
`reply`, `like`, `repost`, `bookmark` or `mention`. The source's own
|
||||
`u-url` and `dt-published` are stored (`source_post_url`,
|
||||
`source_published`, migration 014) so the response links to and is
|
||||
dated by the original post.
|
||||
- If the source is gone (410, other 4xx) or no longer links, the mention
|
||||
is removed from display. If it can't be reached right now (network
|
||||
error, 5xx, temporary DNS failure), a previously valid mention stays up
|
||||
(v2.7.0, v2.10.1).
|
||||
- A webmention from one of the note's own crossposts (e.g. its GoToSocial
|
||||
copy returned through Bridgy) is ignored (v2.8.0).
|
||||
|
||||
### 5. Moderation and display
|
||||
|
||||
- Valid mentions are approved automatically when `WEBMENTION_AUTO_APPROVE`
|
||||
is true (the default), otherwise they wait in `pending`. The admin can
|
||||
approve, hide or delete them at `/admin/webmentions` (single and bulk
|
||||
actions).
|
||||
- A re-sent mention keeps its moderation: hidden stays hidden, approved
|
||||
stays approved (v2.7.0).
|
||||
- Notes show only mentions that are both `valid` and `approved`, grouped
|
||||
by type. Replies are marked up as `u-comment h-cite`; likes, reposts,
|
||||
bookmarks and mentions as `u-like`, `u-repost`, `u-bookmark`,
|
||||
`u-mention` h-cites, each with the author's h-card (v2.8.0).
|
||||
|
||||
### 6. Sending
|
||||
|
||||
- After a note is created, edited, deleted or restored, links are
|
||||
extracted from its **rendered HTML**: `<a href>` only, excluding links
|
||||
inside `<pre>` and `<code>`, the `in_reply_to` target first, then
|
||||
document order, at most 10 current links per operation. Each target is
|
||||
queued in `webmention_queue` (unique per source and target).
|
||||
- On edit, every target sent to before is notified again, including links
|
||||
the edit removed and an old reply target; on delete and undelete too
|
||||
(Recommendation 3.1.4, 3.1.5; v2.7.0). Re-sends are not capped.
|
||||
- When the responses a note displays change (a mention arrives, changes
|
||||
or is removed, or the admin approves, hides or deletes one that was
|
||||
shown), the note's previous webmentions are re-queued, as 3.1.4
|
||||
recommends. An unchanged re-verification sends nothing, so two sites
|
||||
that both do this don't ping-pong (v2.7.2).
|
||||
- Endpoint discovery follows 3.1.2: the first `Link` header with a
|
||||
`webmention` rel token (RFC 8288 parsing, several headers, any case),
|
||||
otherwise the first `<link>` or `<a>` in document order; relative URLs
|
||||
resolve against the final URL. The endpoint is discovered afresh every
|
||||
time; there is no cache (v2.7.0).
|
||||
- The webmention is a form-encoded POST. Any 2xx is success; redirects
|
||||
are not followed (a redirected POST would become a GET). A target
|
||||
without an endpoint is marked `no_endpoint`.
|
||||
- Failures (network errors, 5xx, 429) are retried after 1 h, 6 h and
|
||||
24 h; after the fourth failed attempt the item is `failed`.
|
||||
- The User-Agent is `StarPunk/<version> Webmention (+site)`.
|
||||
|
||||
### 7. Deleted notes
|
||||
|
||||
A deleted note answers **410 Gone** with a "This post has been deleted"
|
||||
page (soft-deleted notes, and hard-deleted notes that sent webmentions),
|
||||
so receivers can remove the mention (3.1.5; v2.7.0).
|
||||
|
||||
### 8. Content safety (XSS)
|
||||
|
||||
- Snippets and reply content are sanitized with nh3, cut at 500
|
||||
characters and sanitized again so no tag is left open (v2.7.0); the
|
||||
admin "Mentions" list shows them with tags stripped.
|
||||
- Author URLs and photos are kept only if they are absolute http(s) URLs;
|
||||
otherwise the source URL is used as the author link and no photo is
|
||||
shown (v2.6.9). mf2 values with `alt` are reduced to their URL.
|
||||
|
||||
### 9. SSRF protection
|
||||
|
||||
All outbound requests for webmentions (source fetches, endpoint discovery
|
||||
and the POST) go through `starpunk/webmention_ssrf.py` (Recommendation
|
||||
4.2, 4.3, 4.5):
|
||||
|
||||
- Only http(s). **Every** address a hostname resolves to (IPv4 and IPv6)
|
||||
must be public: private, loopback, link-local, multicast, reserved,
|
||||
unspecified, carrier-grade NAT (`100.64.0.0/10`), `0.0.0.0/8` and
|
||||
IPv4-mapped forms are blocked.
|
||||
- Redirects (at most 10) are followed by hand and **each hop** is checked
|
||||
(v2.7.0).
|
||||
- **DNS pinning** (v2.7.2): each request connects to the exact address
|
||||
that passed the check, with the original host in the `Host` header and
|
||||
as TLS SNI, so certificates are still verified against the hostname. A
|
||||
rebinding DNS server can't answer differently the second time. IPv4 is
|
||||
preferred when a host has both.
|
||||
- Responses are limited to 1 MB, and each fetch has a total time limit of
|
||||
10 s that a slow, byte-by-byte response can't extend (v2.10.1).
|
||||
- `DEV_MODE` allows localhost for local testing.
|
||||
|
||||
The same guard is reused for IndieAuth client metadata fetches (ADR-064),
|
||||
Micropub remote photos (v2.9.0) and WebSub callbacks and deliveries
|
||||
(ADR-071).
|
||||
|
||||
### 10. Rate limiting
|
||||
|
||||
`POST /webmention` is rate-limited before any other work: by client IP
|
||||
(`WEBMENTION_RATE_LIMIT_IP`, 60) and by source domain
|
||||
(`WEBMENTION_RATE_LIMIT_DOMAIN`, 10) per `WEBMENTION_RATE_LIMIT_WINDOW`
|
||||
(60 s), answering 429 with `Retry-After`. Counters are an in-memory
|
||||
sliding-window log (`starpunk/rate_limiter.py`), so they are per server
|
||||
process. Behind a reverse proxy, `TRUSTED_PROXY=true` makes the first
|
||||
`X-Forwarded-For` address the client IP. `WEBMENTION_RATE_LIMIT_ENABLED`
|
||||
turns it off.
|
||||
|
||||
`starpunk/rate_limiter.py` and `routes/webmention.py` cite this as
|
||||
"ADR-064: Webmention Rate Limiting", a v1.6.0 ADR that was lost with this
|
||||
one. The number ADR-064 was later reused for OIDC login
|
||||
(ADR-064-oidc-admin-login-and-builtin-indieauth-server.md); the rate-limit
|
||||
decision is recorded here instead.
|
||||
|
||||
### 11. Data (migrations 010 and 014)
|
||||
|
||||
- `notes.in_reply_to`
|
||||
- `webmentions`: source, target, target note, type, author name/URL/photo,
|
||||
snippet, `validation_status` (`pending`, `valid`, `invalid`, `error`),
|
||||
`moderation_status` (`pending`, `approved`, `hidden`, `deleted`),
|
||||
timestamps, and (migration 014) `source_post_url`, `source_published`
|
||||
- `webmention_queue`: outgoing items with `status` (`pending`, `sending`,
|
||||
`sent`, `failed`, `no_endpoint`), retry count and next retry time
|
||||
- `reply_contexts`: cached title, snippet and author of pages notes reply
|
||||
to
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Receiving is cheap and can't be used to make StarPunk fetch URLs while
|
||||
the sender waits.
|
||||
- No new infrastructure: the queues are SQLite tables and the worker is a
|
||||
thread, as for crossposting and WebSub.
|
||||
- Outbound fetching is hardened once and reused by every other feature
|
||||
that fetches user-supplied URLs.
|
||||
- Follows every MUST and SHOULD of the Recommendation after v2.7.2.
|
||||
|
||||
### Negative
|
||||
|
||||
- Mentions appear only after the next worker pass (seconds), not
|
||||
immediately.
|
||||
- One worker per site: throughput is limited, which is fine for a
|
||||
personal site.
|
||||
- Rate limits are per process and reset on restart.
|
||||
- `error` rows (unexpected failures) are not retried automatically.
|
||||
- Endpoint discovery ignores HTTP cache headers (spec 5.3); targets are
|
||||
only fetched when a note changes.
|
||||
- The 410 page is a generic tombstone without microformats.
|
||||
|
||||
## Revisions
|
||||
|
||||
- **v2.6.8** — Worker runs inside an app context and is no longer stopped
|
||||
on teardown; before this, no webmention was ever processed
|
||||
(`docs/design/v2.6.8/2026-11-03-webmention-worker-app-context.md`).
|
||||
- **v2.6.9** — Author photos with alt text; endpoint discovery crash
|
||||
fixed and discovery errors retried; `flock` so only one process works
|
||||
the queues; http(s)-only author links
|
||||
(`docs/design/v2.6.9/2026-11-03-webmention-processing-fixes.md`).
|
||||
- **v2.7.0** — Full review against the Recommendation: published notes
|
||||
only, exact target match, RFC 8288 endpoint discovery, no discovery
|
||||
cache, re-sends on edit/delete/undelete, 410 for deleted notes,
|
||||
per-hop SSRF checks, size and time limits, moderation kept on re-send
|
||||
(`docs/design/v2.7.0/2026-11-03-webmention-spec-compliance.md`).
|
||||
- **v2.7.2** — DNS pinning; re-send when displayed responses change
|
||||
(`docs/design/v2.7.2/2026-11-03-webmention-shoulds.md`).
|
||||
- **v2.8.0** — Authorship algorithm, source URL and published time
|
||||
(migration 014), response microformats, own crossposts ignored
|
||||
(`docs/design/v2.8.0/2026-11-03-microformats2-review.md`).
|
||||
- **v2.10.1** — Hard total time limit on fetches; temporary DNS failures
|
||||
treated as transient.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
- **Synchronous verification in the request**: simpler, but slow, ties
|
||||
up a gunicorn process and turns the endpoint into a fetch-anything
|
||||
proxy. The Recommendation allows 202 for exactly this.
|
||||
- **External service (webmention.io)**: no code to run, but mentions
|
||||
would live outside the site's database and moderation, against the
|
||||
self-hosted approach.
|
||||
- **A job queue (Celery/RQ with Redis)**: new infrastructure for a few
|
||||
jobs a day.
|
||||
- **Resolving and validating the host, then letting the HTTP client
|
||||
connect normally**: open to DNS rebinding; replaced by pinning in
|
||||
v2.7.2.
|
||||
|
||||
## References
|
||||
|
||||
- W3C Webmention: https://www.w3.org/TR/webmention/
|
||||
- Code: `starpunk/routes/webmention.py`, `starpunk/webmention_worker.py`,
|
||||
`starpunk/webmention_processor.py`, `starpunk/webmention_sender.py`,
|
||||
`starpunk/webmention_ssrf.py`, `starpunk/webmention_queries.py`,
|
||||
`starpunk/rate_limiter.py`, `migrations/010_add_webmentions.sql`,
|
||||
`migrations/014_webmention_source_details.sql`
|
||||
- Tests: `tests/test_webmention_spec.py`, `tests/test_webmention_worker.py`
|
||||
- [IndieWeb protocols](../architecture/indieweb-protocols.md)
|
||||
- ADR-064 (OIDC login; reuses the SSRF guard), ADR-071 (WebSub; same
|
||||
guard and worker pattern)
|
||||
@@ -0,0 +1,70 @@
|
||||
# ADR-064: OIDC Admin Login and Built-in IndieAuth Server
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.0.0, 2026-10-31)
|
||||
|
||||
Supersedes the login parts of ADR-021, ADR-025, ADR-027 and ADR-043, and the
|
||||
external token verification of ADR-030 and ADR-036.
|
||||
|
||||
## Context
|
||||
|
||||
Up to v1.6.0 StarPunk was an IndieAuth *client*. Admin login discovered an
|
||||
authorization endpoint from `ADMIN_ME` and used the authentication-only
|
||||
flow; Micropub verified bearer tokens by calling the token endpoint
|
||||
discovered from `ADMIN_ME`. In practice both pointed at Gondulf, a separate
|
||||
self-hosted IndieAuth server.
|
||||
|
||||
The operator now runs Pocket ID (OIDC, passkeys) for every other
|
||||
self-hosted service and wants StarPunk to use it too. Pocket ID cannot act
|
||||
as an IndieAuth server: IndieAuth clients identify themselves by URL
|
||||
(`client_id` = their homepage) and are not pre-registered, while OIDC
|
||||
clients must be registered with the provider.
|
||||
|
||||
Micropub apps such as Quill and Indigenous only speak IndieAuth, so moving
|
||||
login to OIDC alone would end Micropub posting.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Admin login uses OIDC** (authorization code flow, PKCE S256, nonce).
|
||||
StarPunk is a confidential client. The ID token's signature (provider
|
||||
JWKS), `iss`, `aud`, `exp` and `nonce` are verified, and only the
|
||||
configured `OIDC_ADMIN_SUB` may log in. A successful login creates the
|
||||
same local session as before, for `ADMIN_ME`.
|
||||
2. **StarPunk becomes its own IndieAuth server** for Micropub clients:
|
||||
`/.well-known/oauth-authorization-server`, `/auth/authorize` (consent
|
||||
screen, admin session required, so approval is protected by OIDC),
|
||||
`/auth/token` and `/auth/revoke`. Only `response_type=code` with PKCE
|
||||
S256 is supported. Tokens are stored as SHA-256 hashes and verified
|
||||
locally by `/micropub` and `/media`.
|
||||
3. **`ADMIN_ME` stays** as the site owner's identity URL: it is the author
|
||||
h-card source and the `me` in issued tokens. That page must advertise
|
||||
StarPunk's IndieAuth metadata (or `authorization_endpoint` /
|
||||
`token_endpoint` links) so Micropub apps discover StarPunk.
|
||||
4. **New tables** (`oidc_login_state`, `indieauth_codes`,
|
||||
`indieauth_tokens`, migration 011). The legacy `auth_state`, `tokens`
|
||||
and `authorization_codes` tables are not reused because migration 004
|
||||
drops the last two after `INITIAL_SCHEMA_SQL` creates them.
|
||||
5. Client `redirect_uri` must share the `client_id` origin, or be published
|
||||
by the client (JSON `redirect_uris` or `rel="redirect_uri"`). The fetch
|
||||
reuses the webmention SSRF guard.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Gondulf is no longer needed and can be retired.
|
||||
- Existing admin sessions stay valid (sessions are keyed by `ADMIN_ME`, which
|
||||
does not change). Notes, media, tags and webmentions are untouched.
|
||||
- Micropub apps must be re-authorized once, since their old tokens were
|
||||
issued by Gondulf. Tokens are visible and revocable at `/admin/tokens`.
|
||||
- Production requires `OIDC_ISSUER`, `OIDC_CLIENT_ID`,
|
||||
`OIDC_CLIENT_SECRET`, `OIDC_ADMIN_SUB`. `INDIELOGIN_URL` and
|
||||
`TOKEN_ENDPOINT` are ignored.
|
||||
- New dependency: `PyJWT[crypto]`.
|
||||
- Major version bump (2.0.0): configuration and endpoint discovery change
|
||||
incompatibly.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
- **Keep Gondulf for Micropub only**: two auth systems to run and patch.
|
||||
- **Drop Micropub**: simplest, but loses posting from apps.
|
||||
- **Manually created API tokens**: Quill and most apps require IndieAuth.
|
||||
@@ -0,0 +1,45 @@
|
||||
# ADR-065: Direct Syndication (POSSE) to GoToSocial
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.1.0, 2026-10-31)
|
||||
|
||||
## Context
|
||||
|
||||
StarPunk posts were copied to other networks by Echofeed reading the feed.
|
||||
That gives no control per post, no record of where a post went, and edits
|
||||
and deletes never follow. The operator runs GoToSocial locally and wants
|
||||
StarPunk to post there directly, with Bluesky likely next.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Publish here, copy out (POSSE).** Each (note, target) has a row in
|
||||
`syndications` (migration 012). Creating, editing, unpublishing or
|
||||
deleting a note marks its rows pending; a background worker reconciles
|
||||
the remote post: create, update (`PUT /api/v1/statuses/:id`) or delete.
|
||||
2. **Defaults.** New notes go to every configured target unless unticked in
|
||||
the editor or a Micropub client sends `mp-syndicate-to` (an explicit
|
||||
list, possibly empty). Existing notes are never backfilled.
|
||||
3. **Content.** The full note as Markdown (`content_type: text/markdown`),
|
||||
then the permalink and the note's tags as hashtags. Longer than the
|
||||
target's limit: shortened at a word boundary with the link kept.
|
||||
Images are uploaded first (the `large` variant) with their captions as
|
||||
alt text.
|
||||
4. **Delivery.** Rows are claimed atomically so the four gunicorn workers'
|
||||
threads never send twice; an `Idempotency-Key` guards retries; a row
|
||||
`version` keeps edits made during a send. Network errors and 5xx/429
|
||||
retry with exponential backoff (8 attempts); other 4xx fail at once.
|
||||
A remote post deleted by hand is posted again on the next edit.
|
||||
5. **Backlinks.** Synced URLs are shown on the note page as `u-syndication`
|
||||
links, and in the editor with status, last error and a retry button.
|
||||
6. **Targets** are config-driven. One kind exists, a Mastodon-API client
|
||||
(`GOTOSOCIAL_URL`, `GOTOSOCIAL_ACCESS_TOKEN`); it also works for
|
||||
Mastodon. A Bluesky target would add a second client behind the same
|
||||
reconcile loop.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Echofeed must stop posting to GoToSocial or posts appear twice.
|
||||
- The token needs only `write:statuses write:media`.
|
||||
- `syndications.note_id` has no foreign key so a hard-deleted note's
|
||||
remote post can still be deleted.
|
||||
@@ -0,0 +1,37 @@
|
||||
# ADR-066: Crossposting to Bluesky
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.2.0, 2026-10-31)
|
||||
|
||||
## Context
|
||||
|
||||
ADR-065 added direct crossposting to GoToSocial. Bluesky was still fed by
|
||||
Echofeed, with the same problems: no per-post control, no record of the
|
||||
copy, deletes never follow.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Second client, same loop.** `BlueskyClient` sits behind the reconcile
|
||||
loop and `syndications` table from ADR-065, as target id `bluesky`.
|
||||
Clients share a small interface: `create`, `update`, `delete`, `can_edit`.
|
||||
2. **Auth.** An app password (`BLUESKY_HANDLE`, `BLUESKY_APP_PASSWORD`).
|
||||
`createSession` against `BLUESKY_SERVICE` (default `https://bsky.social`);
|
||||
writes go to the PDS named in the returned DID document. Expired access
|
||||
tokens are refreshed, falling back to a new session.
|
||||
3. **Content.** Plain text (Markdown rendered and stripped), then the
|
||||
permalink and hashtags (at most a third of the space), shortened to 300
|
||||
characters at a word boundary. Link and hashtag facets use UTF-8 byte
|
||||
offsets. Images (up to 4) are re-encoded as JPEG if over 1 MB, with alt
|
||||
text and aspect ratio; a note without images gets a link card.
|
||||
4. **Edits don't follow.** Bluesky has no edit; rewriting the record would
|
||||
break likes and replies, and delete-and-repost would lose them. The post
|
||||
stays as first sent. Unpublish, untick and delete remove it.
|
||||
5. **No double posts.** When a create is retried, recent posts are checked
|
||||
for one already linking to the note (a send whose reply was lost).
|
||||
|
||||
## Consequences
|
||||
|
||||
- Echofeed is no longer needed; it must stop posting to Bluesky or posts
|
||||
appear twice.
|
||||
- A post deleted by hand on Bluesky is not re-created.
|
||||
@@ -0,0 +1,34 @@
|
||||
# ADR-067: Micropub update, delete, drafts and replies
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.4.0, 2026-10-31)
|
||||
|
||||
## Context
|
||||
|
||||
The Micropub endpoint could only create published notes. Apps could not edit
|
||||
or delete posts, drafts and replies were dropped, and photos sent with the
|
||||
post or hosted elsewhere were lost.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Update** (JSON only, `update` scope). Supported properties are the ones a
|
||||
note can store: `content`, `name` (the `# Heading` first line),
|
||||
`category`, `photo`, `in-reply-to`, `post-status`. Anything else is a 400,
|
||||
not silently ignored. Order: delete, then replace, then add (`add` only for
|
||||
`category` and `photo`). Success is 204.
|
||||
2. **Delete is soft** (`delete` scope) so `undelete` can restore the note and
|
||||
its file from `.trash/`. Crossposts are removed on delete and posted again
|
||||
on undelete.
|
||||
3. **Drafts**: `post-status: draft` creates an unpublished note; crossposts
|
||||
wait until it is published.
|
||||
4. **Photos**: multipart uploads are saved like media-endpoint uploads; remote
|
||||
URLs are downloaded only from public IP addresses, up to 20 MB, max 4.
|
||||
5. **Queries**: `q=source` returns everything create accepts (round-trip);
|
||||
without `url` it lists recent posts, drafts included, for editing apps.
|
||||
`q=category` lists tags by use.
|
||||
|
||||
## Consequences
|
||||
|
||||
- `published` (backdating) can be set on create but not changed by update.
|
||||
- Likes, reposts and bookmarks remain unsupported (round 2).
|
||||
@@ -0,0 +1,34 @@
|
||||
# ADR-068: Likes and reposts
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.5.0, 2026-11-01). Bookmarks were considered and declined.
|
||||
|
||||
> Bookmarks were later added in v2.9.0 (Micropub `bookmark-of`, migration 015) without a separate ADR. (status updated 2026-11-03)
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Storage.** `notes.like_of` / `notes.repost_of` (migration 013). The note's
|
||||
content is an optional comment and may be empty.
|
||||
2. **Display.** Likes stay off the home page and feeds and are listed on
|
||||
`/likes`. Reposts appear in the stream and feeds. Both show the target's
|
||||
fetched title and author (the reply-context cache) and carry
|
||||
`u-like-of` / `u-repost-of`. The target is sent a webmention.
|
||||
3. **Crossposting, per target.**
|
||||
|
||||
| | Target is a post on that network | Anything else |
|
||||
|---|---|---|
|
||||
| Like | native favourite / like | nothing |
|
||||
| Repost, no comment | native boost / repost | "🔁 title url" post |
|
||||
| Repost, with comment | Bluesky quote post; GoToSocial status | comment + "🔁 title url" post |
|
||||
|
||||
GoToSocial finds fediverse posts with `/api/v2/search?resolve=true`
|
||||
(needs `read:search`; favourites need `write:favourites`). Bluesky
|
||||
recognises `bsky.app/profile/<actor>/post/<rkey>` URLs. Native actions
|
||||
are stored as `favourite:<id>` / `reblog:<id>` or the like/repost record
|
||||
URI, and deleting the note undoes them.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Editing a like/repost doesn't change a native favourite/boost.
|
||||
- A repost with a comment can't be a GoToSocial boost (no quote posts there).
|
||||
@@ -0,0 +1,57 @@
|
||||
# ADR-069: Crossposting to Threads
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.6.0, 2026-10-31)
|
||||
|
||||
## Context
|
||||
|
||||
ADR-065 and ADR-066 crosspost notes to GoToSocial and Bluesky. Threads
|
||||
(Meta) has a publishing API, so it can be a third target on the same loop.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Third client, same loop.** `ThreadsClient` is target id `threads`
|
||||
behind the `syndications` table and reconcile loop from ADR-065.
|
||||
2. **Auth.** A long-lived Threads user token (`THREADS_ACCESS_TOKEN`) with
|
||||
`threads_basic`, `threads_content_publish` and `threads_delete`. These
|
||||
tokens expire after 60 days, so StarPunk refreshes the token about once a
|
||||
day (`th_refresh_token`) from the worker loop, even when nothing is being
|
||||
posted. The current token is kept in `DATA_PATH/threads_token.json`
|
||||
(mode 0600) with a hash of the configured token. A different configured
|
||||
token replaces the saved one. A file lock stops gunicorn workers from
|
||||
refreshing at the same time. The file avoids a migration for one value.
|
||||
3. **Posting.** Create a container, poll its status until `FINISHED`, then
|
||||
`threads_publish`. Text-only notes are `TEXT` posts. Threads previews
|
||||
the first link in the text (the permalink) by itself. An explicit
|
||||
`link_attachment` is not sent, because Meta rejects one for a page it
|
||||
hasn't fetched yet ("Invalid Link Attachment", subcode 4279047), which
|
||||
is always the case for a new note (v2.6.2). One image is an `IMAGE` post and 2–4 images are a
|
||||
`CAROUSEL`. Meta fetches images from their public `/media/` URL. Only
|
||||
JPEG and PNG up to 8 MB are allowed, so GIF and WebP images are left off.
|
||||
Alt text goes in `alt_text`.
|
||||
4. **Content.** Plain text (as for Bluesky), then the permalink, shortened
|
||||
to 500 characters. Non-ASCII characters are counted as UTF-8 bytes,
|
||||
because Threads counts emoji that way. Threads allows one tag per post,
|
||||
so the note's first tag is sent as `topic_tag` and no hashtags are added
|
||||
to the text.
|
||||
5. **Edits don't follow.** The API has no way to edit a post. As with
|
||||
Bluesky, the post stays as first sent, and unpublishing, unticking the
|
||||
target or deleting the note removes it.
|
||||
6. **Likes and reposts (ADR-068).** The API can't like or repost outside
|
||||
content, so likes are not crossposted. Reposts are text posts with
|
||||
"🔁 title" and the URL, plus the comment if there is one, and the
|
||||
reposted page as the link preview.
|
||||
7. **No double posts.** A retried create first looks for a recent post that
|
||||
already contains the permalink.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Meta limits accounts to 250 posts and 100 deletes per day. Rate-limit
|
||||
errors (Graph codes 4, 17, 32, 613) are retried.
|
||||
- If StarPunk is down for more than 60 days, the token expires and a new
|
||||
one is needed.
|
||||
- Getting the first token requires a Meta developer app with the Threads
|
||||
use case. See the setup notes in `docs/design/v2.6.0/`.
|
||||
- Notes with more than five links fail on Threads (a Meta limit). The
|
||||
error is shown in the editor.
|
||||
@@ -0,0 +1,44 @@
|
||||
# ADR-070: Links and link cards in crossposts
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.6.3, 2026-11-01)
|
||||
|
||||
## Context
|
||||
|
||||
Every crosspost carries the note's permalink. Notes often link elsewhere
|
||||
too, usually to the article being discussed. Before this change:
|
||||
|
||||
- Bluesky and Threads get plain text, and a Markdown link `[text](url)`
|
||||
lost its URL, so readers there couldn't follow it.
|
||||
- The Bluesky card always showed the note, and the Threads preview showed
|
||||
whichever link came first.
|
||||
- Threads rejects posts with more than 5 different links.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **The card shows what the note links to.** If the note has its own link
|
||||
(a Markdown link, `<url>` or a bare URL), the first one gets the
|
||||
preview card. A note without links gets a card for itself, as before.
|
||||
Images still take the place of a card on Bluesky.
|
||||
2. **Bluesky.** The card is built by StarPunk: it fetches the page's Open
|
||||
Graph, Twitter or plain HTML title, description and image, and uploads
|
||||
the image (shrunk to under 1 MB) as the thumbnail. If the page can't
|
||||
be fetched, the card has just the URL. Markdown link text becomes a
|
||||
clickable link facet, so no characters are spent on the URL.
|
||||
3. **Threads.** It has no clickable link text, so Markdown links are
|
||||
written "text (url)". Threads previews the first link in the text, which
|
||||
is now the linked page. To stay within Meta's 5-link limit, after 4
|
||||
different links in the body a Markdown link keeps only its text.
|
||||
4. **GoToSocial** already gets Markdown, so its links are unchanged.
|
||||
Receiving servers build their own cards.
|
||||
5. **Fetching is careful.** Only http(s) URLs on public addresses are
|
||||
fetched, and this is checked again at every redirect. At most 512 KB of
|
||||
a page and 5 MB of an image are read. A failed fetch never fails the
|
||||
post.
|
||||
|
||||
## Consequences
|
||||
|
||||
- One or two extra requests to the linked site when a note is crossposted
|
||||
to Bluesky.
|
||||
- Reposts (ADR-068) are unchanged: their card is the reposted page.
|
||||
@@ -0,0 +1,206 @@
|
||||
# ADR-071: WebSub publishing with a built-in hub
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.10.0, 2026-11-03)
|
||||
|
||||
## Context
|
||||
|
||||
Readers poll StarPunk's feeds. WebSub (W3C Recommendation,
|
||||
https://www.w3.org/TR/websub/) lets readers that support it (Monocle,
|
||||
Aperture, Inoreader, NewsBlur, Miniflux and others) subscribe once and
|
||||
have new content pushed to them within seconds.
|
||||
|
||||
WebSub has three roles. StarPunk is the **publisher**. Someone has to run
|
||||
the **hub** that keeps subscriptions and delivers content. Public hubs
|
||||
exist (switchboard.p3k.io, pubsubhubbub.appspot.com), but they are a
|
||||
third-party dependency and some have shut down. The rest of this site is
|
||||
self-hosted.
|
||||
|
||||
ADR-014 deferred WebSub for v1 as "complexity, external dependency".
|
||||
|
||||
Two existing problems matter here:
|
||||
|
||||
- Feeds are cached for `FEED_CACHE_SECONDS` (5 minutes) and nothing clears
|
||||
the cache when a note changes. A hub fetching right after a change would
|
||||
get stale content. Regular readers are also up to 5 minutes behind.
|
||||
- The RSS feed is served at `/feed.rss` and `/feed.xml` (legacy alias), and
|
||||
calls itself `/feed.xml`.
|
||||
|
||||
## Decision
|
||||
|
||||
StarPunk runs its own hub, **for its own topics only**, at `/websub`.
|
||||
`WEBSUB_HUB_URL` can point at an external hub instead; then StarPunk
|
||||
advertises that hub, pings it on changes, and its own `/websub` is off.
|
||||
|
||||
### 1. Topics
|
||||
|
||||
Absolute URLs under `SITE_URL`:
|
||||
|
||||
| Topic | Content type |
|
||||
|---|---|
|
||||
| `/` (h-feed) | text/html |
|
||||
| `/feed.atom` | application/atom+xml |
|
||||
| `/feed.rss` | application/rss+xml |
|
||||
| `/feed.json` | application/feed+json |
|
||||
| `/likes` | text/html |
|
||||
| `/tag/<name>` | text/html |
|
||||
|
||||
`/feed.xml` and the negotiated `/feed` advertise the topic of the format
|
||||
they serve as `rel=self` (`/feed.rss`, `/feed.atom`, `/feed.json`). The
|
||||
RSS feed's own self link changes from `/feed.xml` to `/feed.rss`.
|
||||
|
||||
### 2. Discovery (publisher, spec 4)
|
||||
|
||||
Every topic response carries `Link: <hub>; rel="hub"` and exactly one
|
||||
`Link: <topic>; rel="self"`. Embedded links as well: `<link rel="hub">` and
|
||||
`<link rel="self">` in the HTML head of topic pages, `<link rel="hub">`
|
||||
next to the existing self link in Atom, `<atom:link rel="hub">` in RSS, and
|
||||
`"hubs": [{"type": "WebSub", "url": ...}]` in JSON Feed.
|
||||
|
||||
### 3. What changed, and when
|
||||
|
||||
`websub.note_changed(note, old_tags=())` is called wherever
|
||||
`syndication.note_changed` is, plus on create. It:
|
||||
|
||||
1. clears the feed cache (all formats), and
|
||||
2. if the note is or was public, queues delivery for `/`, the three
|
||||
feeds, `/likes` if it is a like, and each `/tag/<name>` for its tags
|
||||
before and after the change.
|
||||
|
||||
Drafts that stay drafts change nothing.
|
||||
|
||||
### 4. Hub endpoint (spec 5)
|
||||
|
||||
`POST /websub`, `application/x-www-form-urlencoded`:
|
||||
|
||||
- `hub.mode` subscribe or unsubscribe; `hub.topic` must be one of the
|
||||
topics above (after normalising), else 400 with a plain-text reason;
|
||||
`hub.callback` must be an absolute http(s) URL that passes the SSRF
|
||||
check; `hub.secret` under 200 bytes; `hub.lease_seconds` clamped to
|
||||
1 day to 30 days (default 10 days). Unknown parameters are ignored.
|
||||
- Response: 202 Accepted, independent of what verification later finds.
|
||||
- Limits: the Webmention rate limiter (per IP); at most 25 subscriptions
|
||||
per callback host and 1,000 in total; beyond that, 429.
|
||||
|
||||
### 5. Verification of intent (spec 5.3)
|
||||
|
||||
A background worker sends `GET <callback>` with the callback's query
|
||||
string kept and `hub.mode`, `hub.topic`, `hub.challenge` (random, 32
|
||||
bytes) and `hub.lease_seconds` appended. Success is a 2xx with the body
|
||||
equal to the challenge; anything else, including redirects, fails and
|
||||
leaves the subscription state unchanged. On success a subscribe becomes
|
||||
active with `expires_at = now + lease`, and an unsubscribe deletes it.
|
||||
A re-subscribe replaces the secret and lease only once verified.
|
||||
|
||||
### 6. Content distribution (spec 6, 7)
|
||||
|
||||
For each change, one delivery per active, unexpired subscription to the
|
||||
topic. The worker renders the topic inside the app (no HTTP round trip
|
||||
through Caddy), so subscribers get exactly what the public URL serves,
|
||||
then `POST`s it to the callback with:
|
||||
|
||||
- `Content-Type` of the topic,
|
||||
- `Link` headers with `rel=hub` and `rel=self`,
|
||||
- `X-Hub-Signature: sha256=<HMAC-SHA256(secret, body)>` if a secret was
|
||||
given.
|
||||
|
||||
A 2xx is success. 410 Gone removes the subscription. Anything else is
|
||||
retried after 1 min, 5 min, 30 min, 2 h and 6 h, then dropped; a
|
||||
subscription whose deliveries fail for 3 days is removed. Only the latest
|
||||
content is sent: a queued delivery for a topic is replaced by a newer one.
|
||||
|
||||
### 7. Outbound safety
|
||||
|
||||
Callbacks are arbitrary URLs, so verification and delivery go through
|
||||
`webmention_ssrf` (`safe_get`/`safe_post`): public addresses only, the
|
||||
connection pinned to the checked address, no redirects, 10 s time limit,
|
||||
1 MB response cap.
|
||||
|
||||
### 8. Data (migration 016)
|
||||
|
||||
- `websub_subscriptions(id, topic, callback, secret, lease_seconds,
|
||||
state ['pending','active'], expires_at, created_at, verified_at,
|
||||
last_success_at, last_error)`, unique on `(topic, callback)`.
|
||||
- `websub_intents(id, topic, callback, mode, secret, lease_seconds,
|
||||
attempts, next_attempt_at, created_at)`: verifications to do.
|
||||
- `websub_deliveries(id, subscription_id, attempts, next_attempt_at,
|
||||
last_error, created_at)`, unique on `subscription_id`.
|
||||
|
||||
### 9. Worker
|
||||
|
||||
The same pattern as the Webmention worker: one thread per gunicorn
|
||||
process, a file lock so only one process works the queues, app context
|
||||
around each poll. Expired subscriptions are pruned once a day.
|
||||
|
||||
### 10. Admin
|
||||
|
||||
The admin dashboard shows active subscribers per topic and failing
|
||||
deliveries.
|
||||
|
||||
### 11. Configuration
|
||||
|
||||
`WEBSUB_ENABLED` (default true), `WEBSUB_HUB_URL` (default: built-in),
|
||||
`WEBSUB_MAX_SUBSCRIPTIONS` (1000), `WEBSUB_MAX_PER_HOST` (25).
|
||||
|
||||
## Consequences
|
||||
|
||||
- Readers that support WebSub get posts and edits within seconds.
|
||||
- Feeds are fresh right after a change for every reader, because the cache
|
||||
is cleared.
|
||||
- StarPunk sends HTTP requests to subscriber-chosen URLs. The SSRF
|
||||
protections, per-host and total limits, and rate limiting bound that.
|
||||
- websub.rocks' hub tests use websub.rocks' own topics, which this hub
|
||||
refuses by design. Testing uses unit tests of each spec section and a
|
||||
real subscription from a reader after deploy.
|
||||
- The RSS self link changes to `/feed.rss`. Readers keep whichever URL
|
||||
they subscribed to; both keep working.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Public hub only.** Least code, but a third-party dependency for a
|
||||
self-hosted site. Still available through `WEBSUB_HUB_URL`.
|
||||
- **A separate hub service** (another container). More to deploy and
|
||||
keep updated, and few maintained hubs exist.
|
||||
- **Accepting any topic** (a general-purpose hub). Not needed, and it
|
||||
would let the hub be used to fetch and push arbitrary URLs.
|
||||
|
||||
## Amendments (v2.10.1, 2026-11-03)
|
||||
|
||||
An adversarial review of v2.10.0 against the Recommendation led to these
|
||||
changes; where they differ, they replace the text above.
|
||||
|
||||
- **Worker robustness.** Each queued item is handled on its own: an
|
||||
unexpected error drops (verification) or retries (delivery) that item
|
||||
only. Before, a callback answering with a large page or containing a
|
||||
control character stopped the worker for good.
|
||||
- **Time limits.** `safe_get` and `safe_post` close the connection when
|
||||
the whole exchange exceeds its time limit (10 s), not just a single read.
|
||||
Verification replies over 4 KB fail; delivery responses are read up to
|
||||
4 KB.
|
||||
- **Leases are kept (spec 7).** A failing subscriber is not removed; its
|
||||
deliveries are retried and then given up, and the next change is
|
||||
delivered again. Only lease expiry, 410 Gone or unsubscribing end a
|
||||
subscription. DNS failures are retried. A topic that renders 404/410
|
||||
ends its subscriptions with a `hub.mode=denied` notice; other render
|
||||
errors are retried.
|
||||
- **Photos.** A note with photos is announced once they are attached
|
||||
(`attach_media_to_note` notifies), and the feed cache key and ETag
|
||||
include `content_version`, so photo and caption changes reach feeds and
|
||||
subscribers. ETags no longer depend on the feed's build time.
|
||||
- **Subscription requests (spec 5.1).** Unreserved percent-encodings are
|
||||
decoded; the Content-Type check is case-insensitive; `hub.secret` and
|
||||
`hub.lease_seconds` are ignored for unsubscriptions; an empty secret is
|
||||
refused; verification echoes `hub.topic` exactly as requested
|
||||
(`requested_topic`); unsubscribing works while an external hub is
|
||||
configured. Callbacks must be printable, at most 2,048 characters, not on
|
||||
this site, on port 80, 443 or 1024+, and are stored with an IDNA host.
|
||||
- **Limits** count pending requests as well as subscriptions, per callback
|
||||
*address* (so subdomains on one server share a limit), with at most 200
|
||||
pending requests. Rate limiting needs `TRUSTED_PROXY=true` behind Caddy.
|
||||
- **Races.** Each request has a `request_id`; verifying an older request
|
||||
doesn't consume a newer one (migration 017).
|
||||
- **Less work.** A topic is rendered once per worker pass; likes only
|
||||
update `/likes` and tag pages.
|
||||
- Corrections: there is no `state` column (a subscription row exists only
|
||||
once verified); expired leases are pruned hourly.
|
||||
@@ -0,0 +1,51 @@
|
||||
# ADR-072: Remove the metrics dashboard
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.12.0, 2026-11-03). Supersedes ADR-053 and ADR-060.
|
||||
|
||||
## Context
|
||||
|
||||
v1.1.1 and v1.1.2 added built-in performance monitoring (ADR-053): HTTP
|
||||
request timing middleware, a wrapper on every database connection that
|
||||
timed queries, a memory-sampling thread, feed-generation counters, an
|
||||
in-memory buffer of the results, and an HTML dashboard at
|
||||
`/admin/metrics-dashboard` with a JSON twin at `/admin/metrics` (ADR-060
|
||||
covered adapting the data for the dashboard template).
|
||||
|
||||
In practice it told us little:
|
||||
|
||||
- The buffer lives in memory in each of the four gunicorn processes, so the
|
||||
dashboard showed whichever process served the request, and everything was
|
||||
lost on restart or deploy.
|
||||
- Nothing in it was ever used to find or fix a problem. Logs (with
|
||||
correlation IDs), `/health` and `/admin/health` were.
|
||||
- It cost something on every request and every query, kept a thread per
|
||||
process, and needed `psutil`, about 1,300 lines of code and three test
|
||||
files.
|
||||
|
||||
## Decision
|
||||
|
||||
Remove the dashboard and everything that only fed it: the `starpunk/monitoring`
|
||||
package, the HTTP metrics middleware (including its `X-Request-ID`
|
||||
response header), `MonitoredConnection`, the memory monitor, the
|
||||
feed-generation tracking calls, `/admin/metrics-dashboard`,
|
||||
`/admin/metrics`, the `METRICS_*` settings and the `psutil` dependency.
|
||||
|
||||
Keep:
|
||||
|
||||
- `/health` and `/health?detailed=true`;
|
||||
- `/admin/health`, without its `metrics` section (database, pool
|
||||
statistics, filesystem and disk checks stay);
|
||||
- connection pool statistics (`get_pool_stats`) and the feed cache, which
|
||||
do work of their own;
|
||||
- structured logs with correlation IDs (ADR-054).
|
||||
|
||||
## Consequences
|
||||
|
||||
- Less code and one fewer dependency; no timing work on each request or
|
||||
query; one fewer thread per process.
|
||||
- Slow database queries are no longer logged. If that is ever needed, a
|
||||
query timer can be added at the pool level, or an external tool used.
|
||||
- `METRICS_*` environment variables are ignored if still set.
|
||||
- Bookmarks to `/admin/metrics-dashboard` now get a 404.
|
||||
@@ -0,0 +1,281 @@
|
||||
# ADR-073: Backfeed of responses to crossposts
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.13.0, 2026-11-10)
|
||||
|
||||
Implementation notes (v2.13.0):
|
||||
|
||||
- Until an avatar has a local copy, pages show the initial and keep the
|
||||
original URL only as `<data class="u-photo">` for microformats parsers,
|
||||
so browsers don't load it.
|
||||
- Bluesky post views carry no bio, so reply and quote authors' profiles
|
||||
are fetched (`getProfiles`, 25 at a time) to honour `#nobridge`.
|
||||
- A Threads token without `threads_manage_insights` stops Threads
|
||||
backfeed as a whole, as for any refused credential.
|
||||
- Backfill checks take a shared slot in `site_state`
|
||||
(`backfeed_backfill_last`), so all gunicorn workers together do at most
|
||||
one every 5 seconds.
|
||||
|
||||
## Context
|
||||
|
||||
StarPunk crossposts notes to GoToSocial, Bluesky and Threads (ADR-065,
|
||||
ADR-066, ADR-069). People like, repost, quote and reply to those copies,
|
||||
and none of that reaches the note. [Backfeed](https://indieweb.org/backfeed)
|
||||
brings those responses back to the original post.
|
||||
|
||||
The usual tool is [Bridgy](https://brid.gy/about), which polls the silos and
|
||||
sends webmentions. It is a poor fit here:
|
||||
|
||||
- It supports Mastodon and Bluesky, not Threads.
|
||||
- It only checks the 30 most recent posts, about every 30 minutes, and
|
||||
says nothing when a like is withdrawn or a reply deleted.
|
||||
- Since v2.8.0 StarPunk rejects webmentions from a note's own crossposts
|
||||
(`_is_own_syndicated_copy`, `webmention_worker.py`). Bridgy's like and
|
||||
repost pages most likely carry the crosspost's URL plus a fragment as
|
||||
`u-url`, so they would be rejected. Replies have their own URL and get
|
||||
through (production webmention #1 was a Bridgy reply from Mastodon).
|
||||
- Every Bridgy source is on `brid.gy`, which shares one source-domain rate
|
||||
limit of 10 webmentions a minute (ADR-063, Decision 10).
|
||||
- Every response passes through a third party.
|
||||
|
||||
StarPunk already holds a credential for each silo and knows each copy
|
||||
(`syndications.remote_id`, `remote_url`). What the silos let it read:
|
||||
|
||||
| Silo | Likes | Reposts | Quotes | Replies | Credential change |
|
||||
|---|---|---|---|---|---|
|
||||
| Bluesky | `getLikes` | `getRepostedBy` | `getQuotes` | `getPostThread` | none (app password) |
|
||||
| GoToSocial | `favourited_by` | `reblogged_by` | not supported | `context` | read scopes |
|
||||
| Threads | count only | count only | count only | `replies` | `threads_read_replies`, `threads_manage_insights` |
|
||||
|
||||
The Threads API never says who liked, reposted or quoted a post; the
|
||||
Insights API gives lifetime counts only.
|
||||
|
||||
The production GoToSocial prunes remote posts older than 90 days, but never
|
||||
a thread that a local account started or took part in (GoToSocial's "Post
|
||||
Caching and Pruning" docs), so replies to StarPunk's copies are kept. On
|
||||
2026-11-10 the production database still had 106 remote replies to the
|
||||
account older than 90 days, back to August 2024.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Native polling, no Bridgy
|
||||
|
||||
StarPunk polls each silo itself for responses to its own copies. Bridgy is
|
||||
switched off for the Mastodon and Bluesky accounts once this ships. The
|
||||
own-copy rule stays as it is.
|
||||
|
||||
### 2. Backfed responses are webmentions
|
||||
|
||||
Responses are stored in the `webmentions` table, so moderation, the
|
||||
Responses section, and re-sending webmentions when a note's responses
|
||||
change (spec 3.1.4, `note_responses_changed`) work as they do now. A
|
||||
backfed row:
|
||||
|
||||
- `source_url`: the response's own URL where it has one (a reply or quote
|
||||
post). Likes and reposts have no page, so they get the copy's URL plus
|
||||
`#liked-by-<actor id>` or `#reposted-by-<actor id>`. The
|
||||
`(source_url, target_url)` key makes repeated checks update the same row.
|
||||
- `target_url`: the note's permalink. `target_note_id`: the note.
|
||||
- `origin`: new column. `webmention` for received webmentions (the
|
||||
default), else `gotosocial`, `bluesky` or `threads`.
|
||||
- `remote_id`: new column. The silo's id for the response (status id,
|
||||
AT-URI, Threads media id), or the actor's id for a like or repost.
|
||||
- `mention_type`: `like`, `repost`, `reply` or the new `quote`.
|
||||
- Author name, profile URL and avatar from the silo's account data;
|
||||
`content_snippet` from the reply or quote text, sanitised the same way as
|
||||
received webmentions (Mastodon HTML through nh3, Bluesky and Threads text
|
||||
escaped with links kept); `source_published` from the silo's timestamp.
|
||||
- `validation_status = 'valid'`: StarPunk read it from the silo's API
|
||||
itself, so there's nothing to fetch and verify. Moderation follows
|
||||
`WEBMENTION_AUTO_APPROVE` like any first-time webmention.
|
||||
|
||||
Only replies directly to the copy are taken, not deeper replies in the
|
||||
thread. Responses from the accounts StarPunk posts as are skipped.
|
||||
|
||||
### 3. Schedule
|
||||
|
||||
Each synced copy of a published note is checked on a schedule that slows
|
||||
as the note ages:
|
||||
|
||||
| Note age | Check every |
|
||||
|---|---|
|
||||
| under 1 day | 15 minutes |
|
||||
| 1 to 7 days | 1 hour |
|
||||
| 7 to 30 days | 1 day |
|
||||
| 30 to 90 days | 1 week |
|
||||
| over 90 days | not checked |
|
||||
|
||||
A random jitter of up to 10% spreads the checks. The cutoff is
|
||||
`BACKFEED_MAX_AGE_DAYS` (default 90). Copies of likes and reposts
|
||||
(ADR-068) aren't checked: they have no responses of their own. A note that
|
||||
is unpublished or deleted is not checked, and its rows stay as they are.
|
||||
|
||||
### 4. Withdrawn likes and deleted replies
|
||||
|
||||
A check that read the whole list (every page, under a cap of 1,000 items
|
||||
per list) marks that note's rows from that silo that are no longer there
|
||||
as `validation_status = 'invalid'` with the error "No longer on
|
||||
<silo>". They drop off the page but keep their moderation decision, and
|
||||
come back if they reappear. A check that stopped early (an error, or the
|
||||
cap) removes nothing.
|
||||
|
||||
### 5. Privacy
|
||||
|
||||
- Only public responses: GoToSocial `visibility` `public` (not unlisted,
|
||||
followers-only or direct); on Bluesky, nothing from accounts with the
|
||||
`!no-unauthenticated` label.
|
||||
- People who opt out are skipped: `#nobridge` or `#nobot` in their bio
|
||||
(Mastodon `note`, Bluesky `description`), the same signal Bridgy honours.
|
||||
A row already stored for them is removed at the next check.
|
||||
- Admin **Delete** stops being a real `DELETE` and sets
|
||||
`moderation_status = 'deleted'` (the value exists but is unused), so a
|
||||
later check or re-sent webmention can't bring the response back. This
|
||||
applies to all webmentions.
|
||||
|
||||
### 6. Avatars are cached locally
|
||||
|
||||
Received webmentions hotlink author photos today. That leaks visitors'
|
||||
addresses to other sites, and Threads and Bluesky avatar URLs change or
|
||||
expire. Avatars are fetched once with `safe_get` (1 MB limit), resized to
|
||||
96 px WebP with Pillow and saved as `DATA_PATH/avatars/<sha256 of URL>.webp`,
|
||||
served from `/avatars/`. They're refetched when the URL changes, and files
|
||||
no row uses are removed weekly. Received webmentions use the same cache
|
||||
from their validation step. A failed fetch falls back to the coloured
|
||||
initial.
|
||||
|
||||
### 7. Threads counts
|
||||
|
||||
Threads likes, reposts and quotes are shown as anonymous counts, not as
|
||||
responses. The worker reads
|
||||
`/{media-id}/insights?metric=likes,reposts,quotes` with each Threads check
|
||||
and stores the numbers on the copy's `syndications` row. The note shows a
|
||||
plain line such as "On Threads: 12 likes · 3 reposts" after the
|
||||
Responses, with no `u-like` or `h-cite` markup, because these aren't
|
||||
people. Threads `replies` counts aren't used: the replies themselves are
|
||||
shown. `views` and `shares` are left out (Meta marks them "in
|
||||
development").
|
||||
|
||||
### 8. Backfill
|
||||
|
||||
An admin action, **Backfill responses**, checks every synced copy once,
|
||||
whatever the note's age. Nothing runs by itself on deploy; the owner runs
|
||||
it after the first deploy (and may run it again later). It:
|
||||
|
||||
- marks every synced copy due now, flagged as a backfill;
|
||||
- is worked through by the same worker, at most one copy every 5 seconds,
|
||||
ahead of the routine schedule only when nothing routine is due;
|
||||
- records when it started and finished; the admin page shows this, and
|
||||
the button is disabled while one is running.
|
||||
|
||||
Upserts make it safe to repeat.
|
||||
|
||||
### 9. Worker
|
||||
|
||||
The checks run in the syndication worker's loop (ADR-065) after crossposts,
|
||||
reusing its clients, token refresh (`maintain()`) and the lease pattern:
|
||||
a copy is claimed with `backfeed_claimed_until` so only one gunicorn worker
|
||||
checks it. A failed check backs off like a failed crosspost
|
||||
(`min(30 s × 2^attempts, 6 h)`) without changing the copy's crosspost
|
||||
state. If a silo answers with a permission error, backfeed for that silo
|
||||
stops until restart and the admin page says which credential needs which
|
||||
scope; crossposting is unaffected.
|
||||
|
||||
Per check: Bluesky 4 calls (likes, reposts, quotes, thread), GoToSocial 3,
|
||||
Threads 2 (replies, insights), plus extra pages.
|
||||
|
||||
### 10. Data (migration 020)
|
||||
|
||||
- `webmentions`: rebuilt to add `quote` to the `mention_type` CHECK, with
|
||||
new columns `origin TEXT NOT NULL DEFAULT 'webmention'` and
|
||||
`remote_id TEXT`, and an index on `(target_note_id, origin)`.
|
||||
- `syndications`: new columns `backfeed_next_at`, `backfeed_checked_at`,
|
||||
`backfeed_claimed_until`, `backfeed_attempts`, `backfeed_error`,
|
||||
`backfeed_backfill` (flag), and `likes_count`, `reposts_count`,
|
||||
`quotes_count` (Threads only).
|
||||
- The backfill's start and finish times go in the existing `site_state`
|
||||
key/value table (migration 016): `backfeed_backfill_started`,
|
||||
`backfeed_backfill_finished`.
|
||||
|
||||
### 11. Display and admin
|
||||
|
||||
- Each backfed response shows a small "via Bluesky" (or GoToSocial,
|
||||
Threads) label. Quotes get their own group, rendered like mentions with
|
||||
the quote text, as `u-mention h-cite`.
|
||||
- The note editor shows when responses were last checked, with a
|
||||
**Check now** button for each copy.
|
||||
- The webmentions list can be filtered by origin.
|
||||
- The dashboard shows backfeed errors next to crosspost errors.
|
||||
|
||||
### 12. Configuration and credentials
|
||||
|
||||
- `BACKFEED_ENABLED` (default `false`; production sets `true`) and
|
||||
`BACKFEED_MAX_AGE_DAYS` (default 90).
|
||||
- GoToSocial: a new token with read access added to the current scopes
|
||||
(`read:statuses` and `read:accounts`, or plain `read`; checked against
|
||||
GoToSocial's scope list when the token is issued).
|
||||
- Threads: the Meta app is re-authorised with `threads_read_replies` and
|
||||
`threads_manage_insights` added. The new `THREADS_ACCESS_TOKEN` replaces
|
||||
the saved token file automatically (ADR-069, Decision 2).
|
||||
- Bluesky: no change.
|
||||
- Production tokens go in the Ansible vault (`bin/vault-run`).
|
||||
|
||||
While at it, the Threads client moves from `graph.threads.net` to
|
||||
`graph.threads.com`, the host Meta's current documentation uses.
|
||||
|
||||
## Amendments (v2.13.2, 2026-11-10)
|
||||
|
||||
- **Display.** Likes, reposts and bookmarks are a line of counts under the
|
||||
note instead of facepiles in the Responses section. Hovering or focusing
|
||||
a count lists the people (as `u-like` / `u-repost` / `u-bookmark`
|
||||
h-cites) and the silo each came from. Threads' anonymous counts are added
|
||||
to the totals ("+N on Threads") instead of a separate line (Decision 7).
|
||||
- **Web actions.** Each like, repost and reply count is an
|
||||
`<indie-action do="…" with="<permalink>">` whose fallback is a link to
|
||||
the same action on each crosspost. No web-action polyfill is loaded;
|
||||
handlers such as browser extensions can use the markup.
|
||||
|
||||
## Amendments (v2.13.3, 2026-11-10)
|
||||
|
||||
- **Opted-out people are counted.** Their likes, reposts and quotes are
|
||||
still not stored or named (Decision 5), but each check records how many
|
||||
there were on the copy's `syndications` row (`likes_count`,
|
||||
`reposts_count`, `quotes_count`, the columns Threads' counts use), and
|
||||
the counts line adds them to the totals as "N on <silo>". Totals then
|
||||
match the silo.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Responses on all three silos show on the note within 15 minutes for new
|
||||
notes, with no third party involved.
|
||||
- Withdrawn likes and deleted replies disappear by themselves for 90 days;
|
||||
after that a note's responses are frozen.
|
||||
- Threads shows replies in full but likes, reposts and quotes only as
|
||||
numbers.
|
||||
- Backfeed stores other people's public posts and avatars. The privacy
|
||||
rules (Decision 5) and moderation are the controls; there is no consent
|
||||
step.
|
||||
- More API calls. At about 50 new notes a month the routine schedule is
|
||||
roughly 5,000 calls a month across the three silos, well inside their
|
||||
limits. The backfill makes one burst of a few calls per copy.
|
||||
- Deleting a webmention in the admin now leaves a tombstone row.
|
||||
- A crosspost removed and posted again gets a new copy; responses to the
|
||||
old copy are marked gone at the next check.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Bridgy.** No Threads, 30-post window, no withdrawals, a third party in
|
||||
the path, and it needs the own-copy rule and rate limit loosened (see
|
||||
Context). Rejected.
|
||||
- **A separate responses table.** Would duplicate the display, moderation
|
||||
and 3.1.4 re-sending code that `webmentions` already has. Rejected.
|
||||
- **Notification feeds** (Mastodon `/api/v1/notifications`, Bluesky
|
||||
`listNotifications`): one call covers every post, but they only report
|
||||
new events, so withdrawals are never seen and a backfill can't use them.
|
||||
Could be added later as a fast path in front of per-post checks.
|
||||
- **Streaming** (Bluesky Jetstream, Mastodon streaming API): real time, but
|
||||
needs a long-lived connection per gunicorn worker. Not worth it for a
|
||||
personal site.
|
||||
- **Keep the Threads counts out.** They aren't IndieWeb responses, but they
|
||||
are the only signal Threads gives, and showing them anonymously costs one
|
||||
call and identifies no one. Kept.
|
||||
+165
-99
@@ -1,139 +1,205 @@
|
||||
# Architectural Decision Records (ADRs) Index
|
||||
|
||||
This directory contains all Architectural Decision Records for StarPunk CMS. ADRs document significant architectural decisions, their context, rationale, and consequences.
|
||||
This directory holds StarPunk's Architectural Decision Records. Each ADR
|
||||
records one decision: its context, what was decided, and the consequences.
|
||||
Superseded ADRs are kept as history; their Status line names the ADR that
|
||||
replaced them.
|
||||
|
||||
## ADR Format
|
||||
**Total ADR files**: 69 (68 numbers: ADR-001 to ADR-044 and ADR-050 to
|
||||
ADR-073, with two files numbered ADR-012). See [Numbering](#numbering).
|
||||
|
||||
Each ADR follows this structure:
|
||||
- **Title**: ADR-NNN-brief-descriptive-title.md
|
||||
- **Status**: Proposed, Accepted, Deprecated, Superseded
|
||||
- **Context**: Why we're making this decision
|
||||
- **Decision**: What we decided to do
|
||||
- **Consequences**: Impact of this decision
|
||||
Status values: **Accepted** (in force), **Superseded** (replaced by a later
|
||||
ADR), **Deprecated** (withdrawn, nothing replaces it; none at present),
|
||||
**Proposed** (not implemented). "Accepted*" means accepted, with later
|
||||
changes noted in the ADR's Status section.
|
||||
|
||||
## All ADRs (Chronological)
|
||||
## Current decisions by area
|
||||
|
||||
### Foundation & Technology Stack (ADR-001 to ADR-009)
|
||||
- **[ADR-001](ADR-001-python-web-framework.md)** - Python Web Framework Selection
|
||||
- **[ADR-002](ADR-002-flask-extensions.md)** - Flask Extensions Strategy
|
||||
- **[ADR-003](ADR-003-frontend-technology.md)** - Frontend Technology Stack
|
||||
- **[ADR-004](ADR-004-file-based-note-storage.md)** - File-Based Note Storage
|
||||
- **[ADR-005](ADR-005-indielogin-authentication.md)** - IndieLogin Authentication
|
||||
- **[ADR-006](ADR-006-python-virtual-environment-uv.md)** - Python Virtual Environment with uv
|
||||
- **[ADR-007](ADR-007-slug-generation-algorithm.md)** - Slug Generation Algorithm
|
||||
- **[ADR-008](ADR-008-versioning-strategy.md)** - Versioning Strategy
|
||||
- **[ADR-009](ADR-009-git-branching-strategy.md)** - Git Branching Strategy
|
||||
### Foundation and process
|
||||
|
||||
### Authentication & Authorization (ADR-010 to ADR-027)
|
||||
- **[ADR-010](ADR-010-authentication-module-design.md)** - Authentication Module Design
|
||||
- **[ADR-011](ADR-011-development-authentication-mechanism.md)** - Development Authentication Mechanism
|
||||
- **[ADR-016](ADR-016-indieauth-client-discovery.md)** - IndieAuth Client Discovery
|
||||
- **[ADR-017](ADR-017-oauth-client-metadata-document.md)** - OAuth Client Metadata Document
|
||||
- **[ADR-018](ADR-018-indieauth-detailed-logging.md)** - IndieAuth Detailed Logging
|
||||
- **[ADR-019](ADR-019-indieauth-correct-implementation.md)** - IndieAuth Correct Implementation
|
||||
- **[ADR-021](ADR-021-indieauth-provider-strategy.md)** - IndieAuth Provider Strategy
|
||||
- **[ADR-022](ADR-022-auth-route-prefix-fix.md)** - Auth Route Prefix Fix
|
||||
- **[ADR-023](ADR-023-indieauth-client-identification.md)** - IndieAuth Client Identification
|
||||
- **[ADR-024](ADR-024-static-identity-page.md)** - Static Identity Page
|
||||
- **[ADR-025](ADR-025-indieauth-pkce-authentication.md)** - IndieAuth PKCE Authentication
|
||||
- **[ADR-026](ADR-026-indieauth-token-exchange-compliance.md)** - IndieAuth Token Exchange Compliance
|
||||
- **[ADR-027](ADR-027-indieauth-authentication-endpoint-correction.md)** - IndieAuth Authentication Endpoint Correction
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [001](ADR-001-python-web-framework.md) | Python Web Framework Selection | Accepted | Flask as the web framework. |
|
||||
| [002](ADR-002-flask-extensions.md) | Flask Extensions and Dependencies | Accepted | A minimal dependency set; avoid Flask extensions where the standard library will do. |
|
||||
| [003](ADR-003-frontend-technology.md) | Front-end Technology Stack | Accepted | Server-rendered Jinja2 templates with minimal client-side code. |
|
||||
| [006](ADR-006-python-virtual-environment-uv.md) | Python Virtual Environment Management with uv | Accepted | uv manages the virtual environment and dependencies. |
|
||||
| [008](ADR-008-versioning-strategy.md) | Versioning Strategy | Accepted | Semantic Versioning, PEP 440 compatible. |
|
||||
| [009](ADR-009-git-branching-strategy.md) | Git Branching Strategy | Accepted | Simplified trunk-based development with feature branches and release tags. |
|
||||
| [012](ADR-012-flaky-test-removal.md) | Flaky Test Removal and Test Quality Standards | Accepted | Remove the multiprocessing migration tests and fix brittle assertions (done in v1.5.0). |
|
||||
| [015](ADR-015-phase-5-implementation-approach.md) | Phase 5 Implementation Approach | Accepted | Version jump 0.5.1 → 0.6.0 for Phase 5 (historical). |
|
||||
| [051](ADR-051-phase1-test-strategy.md) | Phase 1 Test Strategy and Implementation Review | Accepted | Review of the v1.1.1 Phase 1 test approach (historical). |
|
||||
|
||||
### Error Handling & Core Features (ADR-012 to ADR-015)
|
||||
- **[ADR-012](ADR-012-http-error-handling-policy.md)** - HTTP Error Handling Policy
|
||||
- **[ADR-013](ADR-013-expose-deleted-at-in-note-model.md)** - Expose Deleted-At in Note Model
|
||||
- **[ADR-014](ADR-014-rss-feed-implementation.md)** - RSS Feed Implementation
|
||||
- **[ADR-015](ADR-015-phase-5-implementation-approach.md)** - Phase 5 Implementation Approach
|
||||
### Storage and database
|
||||
|
||||
### Micropub & API (ADR-028 to ADR-029)
|
||||
- **[ADR-028](ADR-028-micropub-implementation.md)** - Micropub Implementation
|
||||
- **[ADR-029](ADR-029-micropub-indieauth-integration.md)** - Micropub IndieAuth Integration
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [004](ADR-004-file-based-note-storage.md) | File-Based Note Storage Architecture | Accepted | Note content in Markdown files under `data/notes/YYYY/MM/`, metadata in SQLite. |
|
||||
| [013](ADR-013-expose-deleted-at-in-note-model.md) | Expose deleted_at Field in Note Model | Accepted | `deleted_at` on the Note dataclass for soft deletes. |
|
||||
| [020](ADR-020-automatic-database-migrations.md) | Automatic Database Migration System | Accepted | Numbered SQL migrations applied at startup and tracked in a table. |
|
||||
| [031](ADR-031-database-migration-system-redesign.md) | Database Migration System Redesign | Accepted | Baseline schema first, then migrations, so existing databases upgrade (implemented via ADR-032/033). |
|
||||
| [032](ADR-032-initial-schema-sql-implementation.md) | Initial Schema SQL Implementation for Migration System | Accepted | `INITIAL_SCHEMA_SQL` holds the baseline schema. |
|
||||
| [033](ADR-033-database-migration-redesign.md) | Database Migration System Redesign | Accepted | Frozen baseline schema; every later change only in migration files (v1.1.0). |
|
||||
| [037](ADR-037-migration-race-condition-fix.md) | Database Migration Race Condition Resolution | Accepted | SQLite `BEGIN IMMEDIATE` lock with retries so several gunicorn workers don't migrate at once. |
|
||||
| [041](ADR-041-database-migration-conflict-resolution.md) | Database Migration Conflict Resolution | Accepted | Handling a production database that fell into a migration edge case. |
|
||||
|
||||
### Database & Migrations (ADR-020, ADR-031 to ADR-037)
|
||||
- **[ADR-020](ADR-020-automatic-database-migrations.md)** - Automatic Database Migrations
|
||||
- **[ADR-031](ADR-031-database-migration-system-redesign.md)** - Database Migration System Redesign
|
||||
- **[ADR-032](ADR-032-initial-schema-sql-implementation.md)** - Initial Schema SQL Implementation
|
||||
- **[ADR-033](ADR-033-database-migration-redesign.md)** - Database Migration Redesign
|
||||
- **[ADR-037](ADR-037-migration-race-condition-fix.md)** - Migration Race Condition Fix
|
||||
- **[ADR-041](ADR-041-database-migration-conflict-resolution.md)** - Database Migration Conflict Resolution
|
||||
See also [database architecture](../architecture/database.md).
|
||||
|
||||
### Search & Advanced Features (ADR-034 to ADR-036, ADR-038 to ADR-040)
|
||||
- **[ADR-034](ADR-034-full-text-search.md)** - Full-Text Search
|
||||
- **[ADR-035](ADR-035-custom-slugs.md)** - Custom Slugs
|
||||
- **[ADR-036](ADR-036-indieauth-token-verification-method.md)** - IndieAuth Token Verification Method
|
||||
- **[ADR-038](ADR-038-syndication-formats.md)** - Syndication Formats (ATOM, JSON Feed)
|
||||
- **[ADR-039](ADR-039-micropub-url-construction-fix.md)** - Micropub URL Construction Fix
|
||||
- **[ADR-040](ADR-040-microformats2-compliance.md)** - Microformats2 Compliance
|
||||
### Notes, slugs, media and search
|
||||
|
||||
### Architecture Refinements (ADR-042 to ADR-044)
|
||||
- **[ADR-042](ADR-042-versioning-strategy-for-authorization-removal.md)** - Versioning Strategy for Authorization Removal
|
||||
- **[ADR-043](ADR-043-CORRECTED-indieauth-endpoint-discovery.md)** - CORRECTED IndieAuth Endpoint Discovery
|
||||
- **[ADR-044](ADR-044-endpoint-discovery-implementation.md)** - Endpoint Discovery Implementation Details
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [062](ADR-062-timestamp-based-slug-format.md) | Timestamp-Based Slug Format | Accepted | Default slugs are `YYYYMMDDHHMMSS` timestamps (v1.5.0); replaces ADR-007. |
|
||||
| [035](ADR-035-custom-slugs.md) | Custom Slugs in Micropub | Accepted* | Honour `mp-slug`, sanitized, with reserved names protected (v1.1.0). |
|
||||
| [034](ADR-034-full-text-search.md) | Full-Text Search with SQLite FTS5 | Accepted | FTS5 search over note content (v1.1.0). |
|
||||
| [057](ADR-057-media-attachment-model.md) | Media Attachment Model | Accepted | Social-media-style image attachments: up to 4 per note, ordered, with captions. |
|
||||
| [058](ADR-058-image-optimization-strategy.md) | Image Optimization Strategy | Accepted | Upload limits, automatic resizing and size variants. |
|
||||
|
||||
### Major Architectural Changes (ADR-050 to ADR-051)
|
||||
- **[ADR-050](ADR-050-remove-custom-indieauth-server.md)** - Remove Custom IndieAuth Server
|
||||
- **[ADR-051](ADR-051-phase1-test-strategy.md)** - Phase 1 Test Strategy
|
||||
### Feeds
|
||||
|
||||
### v1.1.1 Quality & Production Readiness (ADR-052 to ADR-055)
|
||||
- **[ADR-052](ADR-052-configuration-system-architecture.md)** - Configuration System Architecture
|
||||
- **[ADR-053](ADR-053-performance-monitoring-strategy.md)** - Performance Monitoring Strategy
|
||||
- **[ADR-054](ADR-054-structured-logging-architecture.md)** - Structured Logging Architecture
|
||||
- **[ADR-055](ADR-055-error-handling-philosophy.md)** - Error Handling Philosophy
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [014](ADR-014-rss-feed-implementation.md) | RSS Feed Implementation Strategy | Accepted* | RSS 2.0 feed; its WebSub deferral is superseded by ADR-071. |
|
||||
| [038](ADR-038-syndication-formats.md) | Multiple Syndication Format Support | Accepted | RSS, Atom and JSON Feed side by side (v1.1.2). |
|
||||
| [071](ADR-071-websub.md) | WebSub publishing with a built-in hub | Accepted | Built-in WebSub hub at `/websub` for StarPunk's own feeds (v2.10.0, amended v2.10.1). |
|
||||
| [059](ADR-059-full-feed-media-standardization.md) | Full Feed Media Standardization (Option 3) | Proposed | Full Media RSS, podcasts, video and thumbnails in feeds; not scheduled. |
|
||||
|
||||
## ADRs by Topic
|
||||
### IndieWeb: microformats, Webmention, Micropub
|
||||
|
||||
### Authentication & IndieAuth
|
||||
ADR-005, ADR-010, ADR-011, ADR-016, ADR-017, ADR-018, ADR-019, ADR-021, ADR-022, ADR-023, ADR-024, ADR-025, ADR-026, ADR-027, ADR-036, ADR-043, ADR-044, ADR-050
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [040](ADR-040-microformats2-compliance.md) | Strict Microformats2 Compliance | Accepted | Full h-entry, h-card and h-feed markup (v1.3.0, reviewed v2.8.0). |
|
||||
| [061](ADR-061-author-discovery.md) | Author Profile Discovery from IndieAuth | Accepted | The author h-card is read from `ADMIN_ME` and cached. |
|
||||
| [063](ADR-063-webmention-implementation.md) | Webmention Implementation | Accepted | Async receiving (202) and sending via a background worker, SSRF guard, rate limits, moderation (v1.6.0, revised up to v2.10.1). *Reconstructed.* |
|
||||
| [028](ADR-028-micropub-implementation.md) | Micropub Implementation Strategy | Accepted* | `/micropub` endpoint for creating posts (v1.0.0-rc.1); tokens now per ADR-064. |
|
||||
| [039](ADR-039-micropub-url-construction-fix.md) | Micropub URL Construction Fix | Accepted | Fix the double slash in returned post URLs. |
|
||||
| [067](ADR-067-micropub-update-delete.md) | Micropub update, delete, drafts and replies | Accepted | Micropub update, delete, drafts and replies (v2.4.0). |
|
||||
| [068](ADR-068-likes-and-reposts.md) | Likes and reposts | Accepted* | `like-of` and `repost-of` posts (v2.5.0); bookmarks were added later in v2.9.0. |
|
||||
|
||||
### Database & Migrations
|
||||
ADR-004, ADR-020, ADR-031, ADR-032, ADR-033, ADR-037, ADR-041
|
||||
See also [IndieWeb protocols](../architecture/indieweb-protocols.md).
|
||||
|
||||
### API & Micropub
|
||||
ADR-028, ADR-029, ADR-039
|
||||
### Crossposting (POSSE)
|
||||
|
||||
### Content & Features
|
||||
ADR-007, ADR-013, ADR-014, ADR-034, ADR-035, ADR-038, ADR-040
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [065](ADR-065-direct-syndication-posse.md) | Direct Syndication (POSSE) to GoToSocial | Accepted | Crosspost to GoToSocial/Mastodon-API servers from a background worker (v2.1.0). |
|
||||
| [066](ADR-066-bluesky-syndication.md) | Crossposting to Bluesky | Accepted | Bluesky as a second crosspost target (v2.2.0). |
|
||||
| [069](ADR-069-threads-syndication.md) | Crossposting to Threads | Accepted | Threads as a third target, with automatic token refresh (v2.6.0). |
|
||||
| [070](ADR-070-crosspost-links-and-cards.md) | Links and link cards in crossposts | Accepted | Keep note links in crossposts and build link cards for the linked page (v2.6.3). |
|
||||
| [073](ADR-073-backfeed.md) | Backfeed of responses to crossposts | Accepted | Poll GoToSocial, Bluesky and Threads for responses to crossposts and store them as webmentions; admin backfill (v2.13.0). |
|
||||
|
||||
### Development & Operations
|
||||
ADR-001, ADR-002, ADR-003, ADR-006, ADR-008, ADR-009, ADR-012, ADR-015, ADR-042, ADR-051, ADR-052, ADR-053, ADR-054, ADR-055
|
||||
### Authentication (current)
|
||||
|
||||
## Superseded ADRs
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [064](ADR-064-oidc-admin-login-and-builtin-indieauth-server.md) | OIDC Admin Login and Built-in IndieAuth Server | Accepted | Admin login via OIDC; StarPunk issues and verifies Micropub tokens itself (v2.0.0). |
|
||||
| [010](ADR-010-authentication-module-design.md) | Authentication Module Design | Accepted* | Single `auth.py` module and server-side sessions; the login flow is now ADR-064. |
|
||||
| [011](ADR-011-development-authentication-mechanism.md) | Development Authentication Mechanism | Accepted | `DEV_MODE` login bypass for local development only. |
|
||||
| [022](ADR-022-auth-route-prefix-fix.md) | Fix IndieAuth Callback Route Mismatch | Accepted | Auth routes live under `/auth` (v0.9.2). |
|
||||
|
||||
These ADRs have been superseded by later decisions:
|
||||
- **ADR-030** (old) - Superseded by ADR-043 (CORRECTED IndieAuth Endpoint Discovery)
|
||||
### Operations, errors and monitoring
|
||||
|
||||
## How to Create a New ADR
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [012](ADR-012-http-error-handling-policy.md) | HTTP Error Handling Policy | Accepted | Correct HTTP status codes (e.g. 404 for missing notes) across routes. |
|
||||
| [052](ADR-052-configuration-system-architecture.md) | Configuration System Architecture | Accepted | Environment variables with defaults, loaded in one module. |
|
||||
| [054](ADR-054-structured-logging-architecture.md) | Structured Logging Architecture | Accepted | Rotating log files with correlation IDs (v1.1.1). |
|
||||
| [055](ADR-055-error-handling-philosophy.md) | Error Handling Philosophy | Accepted | Layered handling: graceful degradation, clear user messages, detailed logs. |
|
||||
| [072](ADR-072-remove-metrics-dashboard.md) | Remove the metrics dashboard | Accepted | Drop the in-memory metrics system and dashboard; keep `/health`, `/admin/health` and logs (v2.12.0). |
|
||||
|
||||
See also [deployment](../architecture/deployment.md).
|
||||
|
||||
## Superseded decisions
|
||||
|
||||
Kept for history. Most concern the IndieLogin.com and external-IndieAuth
|
||||
designs of v0.x–v1.6.0, all replaced by ADR-064 in v2.0.0.
|
||||
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [005](ADR-005-indielogin-authentication.md) | IndieLogin Authentication Integration | Superseded by 064 | Admin login through indielogin.com. |
|
||||
| [007](ADR-007-slug-generation-algorithm.md) | Slug Generation Algorithm | Superseded by 062 | Slugs from the first words of the note. |
|
||||
| [053](ADR-053-performance-monitoring-strategy.md) | Performance Monitoring Strategy | Superseded by 072 | Built-in, standard-library performance metrics (v1.1.1). |
|
||||
| [060](ADR-060-production-hotfix-metrics-dashboard.md) | Route Adapter for the Metrics Dashboard | Superseded by 072 | Adapt monitoring data to the dashboard template in the route (v1.1.1-rc.2). *Reconstructed.* |
|
||||
| [016](ADR-016-indieauth-client-discovery.md) | IndieAuth Client Discovery Mechanism | Superseded by 019 | h-app markup for client discovery. |
|
||||
| [017](ADR-017-oauth-client-metadata-document.md) | OAuth Client ID Metadata Document Implementation | Superseded by 019 | OAuth client metadata endpoint. |
|
||||
| [018](ADR-018-indieauth-detailed-logging.md) | IndieAuth Detailed Logging Strategy | Superseded by 064 | Redacted HTTP logging of the IndieAuth login flow. |
|
||||
| [019](ADR-019-indieauth-correct-implementation.md) | IndieAuth Correct Implementation Based on IndieLogin.com API | Superseded by 025 | Draft of the IndieLogin.com PKCE flow. |
|
||||
| [021](ADR-021-indieauth-provider-strategy.md) | IndieAuth Provider Strategy | Superseded by 064 | Keep IndieLogin.com for admin login. |
|
||||
| [023](ADR-023-indieauth-client-identification.md) | IndieAuth Client Identification Strategy | Superseded by 064 | Visible h-app in the footer. |
|
||||
| [024](ADR-024-static-identity-page.md) | Static HTML Identity Pages for IndieAuth | Superseded by 064 | Standalone identity page pointing at indieauth.com. |
|
||||
| [025](ADR-025-indieauth-pkce-authentication.md) | IndieAuth Correct Implementation Based on IndieLogin.com API | Superseded by 064 | IndieLogin.com authentication with PKCE. |
|
||||
| [026](ADR-026-indieauth-token-exchange-compliance.md) | IndieAuth Token Exchange Compliance | Superseded by 064 | Token exchange parameters for the IndieAuth login. |
|
||||
| [027](ADR-027-indieauth-authentication-endpoint-correction.md) | IndieAuth Authentication Endpoint Correction | Superseded by 064 | Use the authentication (not token) endpoint for login. |
|
||||
| [029](ADR-029-micropub-indieauth-integration.md) | Micropub IndieAuth Integration Strategy | Superseded by 050, 064 | Built-in token endpoint for Micropub (v1.0.0-rc.1). |
|
||||
| [030](ADR-030-external-token-verification-architecture.md) | External Token Verification Architecture | Superseded by 043, 064 | Verify Micropub tokens at a configured external endpoint. |
|
||||
| [036](ADR-036-indieauth-token-verification-method.md) | IndieAuth Token Verification Method Diagnosis | Superseded by 064 | How to call the external token endpoint. |
|
||||
| [042](ADR-042-versioning-strategy-for-authorization-removal.md) | Versioning Strategy for Authorization Server Removal | Accepted (historical) | Remove the authorization server in 1.0.0-rc.4 rather than in a later release. |
|
||||
| [043](ADR-043-CORRECTED-indieauth-endpoint-discovery.md) | IndieAuth Endpoint Discovery Architecture | Superseded by 064 | Discover the token endpoint from `ADMIN_ME` to verify tokens. |
|
||||
| [044](ADR-044-endpoint-discovery-implementation.md) | IndieAuth Endpoint Discovery Implementation Details | Superseded by 064 | Implementation details of ADR-043. |
|
||||
| [050](ADR-050-remove-custom-indieauth-server.md) | Remove Custom IndieAuth Server | Superseded by 064 | Remove the built-in authorization and token server (done in v1.0.0-rc.4). |
|
||||
| [056](ADR-056-no-selfhosted-indieauth.md) | Use External IndieAuth Provider (Never Self-Host) | Superseded by 064 | Never run an IndieAuth server in StarPunk. |
|
||||
|
||||
ADR-042 is listed here because it only matters together with ADR-050; it
|
||||
was carried out and is not itself superseded.
|
||||
|
||||
## Numbering
|
||||
|
||||
- **Gap ADR-045 to ADR-049**: never used; no document or code refers to
|
||||
them.
|
||||
- **Two ADR-012 files**: `ADR-012-http-error-handling-policy.md` and
|
||||
`ADR-012-flaky-test-removal.md` (added 2025-12-16). Both are cited as
|
||||
plain "ADR-012": `starpunk/routes/admin.py` and early CHANGELOG entries
|
||||
mean the error policy, `tests/test_migration_race_condition.py` means
|
||||
the flaky-test one. Not renumbered, since other documents link to them.
|
||||
- **Old numbers in headings**: several files were renumbered without
|
||||
updating their first heading: ADR-037 and ADR-038 say "ADR-022",
|
||||
ADR-040 says "ADR-023", ADR-042 says "ADR-027", ADR-044 says "ADR-031".
|
||||
Older CHANGELOG entries use those old numbers ("ADR-022: Migration Race
|
||||
Condition Fix" is ADR-037; "ADR-031: IndieAuth Endpoint Discovery" is
|
||||
ADR-044). The file name is authoritative.
|
||||
- **ADR-060** was deleted in the v1.1.1-rc.2 review and reconstructed on
|
||||
2026-11-03 because code still cites it. ADR-059 also mentions a planned
|
||||
"Option 2 (ADR-060)" for Media RSS that was never written.
|
||||
- **ADR-063** was lost with the uncommitted v1.6.0 docs and reconstructed
|
||||
on 2026-11-03.
|
||||
- **ADR-064 used twice**: v1.6.0 code (`starpunk/rate_limiter.py`,
|
||||
`starpunk/routes/webmention.py`) cites a lost "ADR-064: Webmention Rate
|
||||
Limiting"; the number was reused in v2.0.0 for OIDC login. The rate-limit
|
||||
decision is recorded in ADR-063, Decision 10.
|
||||
|
||||
## How to create a new ADR
|
||||
|
||||
1. Use the next number after the highest existing one (currently ADR-074).
|
||||
2. Name it `ADR-NNN-brief-descriptive-title.md`.
|
||||
3. Follow the format of recent ADRs (e.g. ADR-070, ADR-071):
|
||||
|
||||
1. **Find the next sequential number**: Check the highest existing ADR number
|
||||
2. **Use the naming format**: `ADR-NNN-brief-descriptive-title.md`
|
||||
3. **Follow the template**:
|
||||
```markdown
|
||||
# ADR-NNN: Title
|
||||
|
||||
## Status
|
||||
Proposed | Accepted | Deprecated | Superseded
|
||||
|
||||
Accepted (vX.Y.Z, YYYY-MM-DD)
|
||||
|
||||
## Context
|
||||
Why are we making this decision?
|
||||
|
||||
## Decision
|
||||
What have we decided to do?
|
||||
|
||||
## Consequences
|
||||
What are the positive and negative consequences?
|
||||
|
||||
## Alternatives Considered
|
||||
What other options did we evaluate?
|
||||
## Alternatives considered
|
||||
```
|
||||
4. **Update this index** with the new ADR
|
||||
|
||||
## Related Documentation
|
||||
- **[../architecture/](../architecture/)** - Architectural overviews and system design
|
||||
- **[../design/](../design/)** - Detailed design documents
|
||||
- **[../standards/](../standards/)** - Coding standards and conventions
|
||||
4. When a decision replaces an earlier one, change the earlier ADR's Status
|
||||
to "Superseded by ADR-NNN" with a one-line note; don't rewrite its body.
|
||||
5. Add the ADR to this index.
|
||||
|
||||
## Related documentation
|
||||
|
||||
- [../architecture/](../architecture/) — current system design
|
||||
- [../design/](../design/) — per-version implementation designs
|
||||
- [../standards/](../standards/) — coding and documentation standards
|
||||
- [../../CHANGELOG.md](../../CHANGELOG.md) — what changed in each release
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-11-25
|
||||
**Maintained By**: Documentation Manager Agent
|
||||
**Total ADRs**: 55
|
||||
**Last Updated**: 2026-11-03
|
||||
**Total ADR files**: 67
|
||||
+432
-96
@@ -1,128 +1,464 @@
|
||||
# Design Documentation Index
|
||||
|
||||
This directory contains detailed design documents, feature specifications, and phase implementation plans for StarPunk CMS.
|
||||
Implementation designs, developer questions and architect answers,
|
||||
implementation reports (`YYYY-MM-DD-description.md`) and reviews, one
|
||||
folder per release (see `docs/DOCUMENTATION.md`). Current system design
|
||||
is in `../architecture/`; decisions are in `../decisions/` (ADRs).
|
||||
|
||||
## Project Structure
|
||||
- **[project-structure.md](project-structure.md)** - Overall project structure and organization
|
||||
- **[initial-files.md](initial-files.md)** - Initial file structure for the project
|
||||
## Notes
|
||||
|
||||
## Phase Implementation Plans
|
||||
- v1.6.0 (webmentions) was deployed but never committed; its code was
|
||||
recovered from the production image on 2026-10-31 (commit `d3fa5b7`). Its
|
||||
documents and tests were not in the image. The five v1.6.0 documents the
|
||||
code cites were reconstructed from the implementation.
|
||||
- Releases from v2.0.1 on that had no folder have a short implementation
|
||||
report reconstructed from the changelog and git history.
|
||||
- Documents marked *(reconstructed)* were written on 2026-11-03 after the fact.
|
||||
- Every release from v2.0.0 on has a folder. Of the v1 releases, v1.0.1
|
||||
and v1.1.0 have none: their documents are in `v1.1.1/` (and the v1.0.1
|
||||
hotfix plan in `v1.0.0/`).
|
||||
- Description = the document's title.
|
||||
|
||||
### Phase 1: Foundation
|
||||
- **[phase-1.1-core-utilities.md](phase-1.1-core-utilities.md)** - Core utility functions and helpers
|
||||
- **[phase-1.1-quick-reference.md](phase-1.1-quick-reference.md)** - Quick reference for Phase 1.1
|
||||
- **[phase-1.2-data-models.md](phase-1.2-data-models.md)** - Data models and database schema
|
||||
- **[phase-1.2-quick-reference.md](phase-1.2-quick-reference.md)** - Quick reference for Phase 1.2
|
||||
## Contents
|
||||
|
||||
### Phase 2: Core Features
|
||||
- **[phase-2.1-notes-management.md](phase-2.1-notes-management.md)** - Notes CRUD functionality
|
||||
- **[phase-2.1-quick-reference.md](phase-2.1-quick-reference.md)** - Quick reference for Phase 2.1
|
||||
[v1.0.0](#v100) · [v1.1.1](#v111) · [v1.1.2](#v112) · [v1.2.0](#v120) · [v1.3.0](#v130) · [v1.3.1](#v131) · [v1.4.0](#v140) · [v1.4.1](#v141) · [v1.4.2](#v142) · [v1.5.0](#v150) · [v1.6.0](#v160) · [v2.0.0](#v200) · [v2.0.1](#v201) · [v2.0.2](#v202) · [v2.1.0](#v210) · [v2.2.0](#v220) · [v2.3.0](#v230) · [v2.3.1](#v231) · [v2.3.2](#v232) · [v2.3.3](#v233) · [v2.4.0](#v240) · [v2.5.0](#v250) · [v2.6.0](#v260) · [v2.6.1](#v261) · [v2.6.2](#v262) · [v2.6.3](#v263) · [v2.6.4](#v264) · [v2.6.5](#v265) · [v2.6.6](#v266) · [v2.6.7](#v267) · [v2.6.8](#v268) · [v2.6.9](#v269) · [v2.7.0](#v270) · [v2.7.1](#v271) · [v2.7.2](#v272) · [v2.8.0](#v280) · [v2.8.1](#v281) · [v2.9.0](#v290) · [v2.10.0](#v2100) · [v2.10.1](#v2101) · [v2.10.2](#v2102) · [v2.11.0](#v2110) · [v2.12.0](#v2120) · [v2.12.1](#v2121) · [v2.13.0](#v2130) · [v2.13.1](#v2131) · [v2.13.2](#v2132) · [v2.13.3](#v2133) · [v2.13.4](#v2134) · [hotfix](#hotfix)
|
||||
|
||||
### Phase 3: Authentication
|
||||
- **[phase-3-authentication.md](phase-3-authentication.md)** - Authentication system design
|
||||
- **[phase-3-authentication-implementation.md](phase-3-authentication-implementation.md)** - Implementation details
|
||||
- **[indieauth-pkce-authentication.md](indieauth-pkce-authentication.md)** - IndieAuth PKCE authentication design
|
||||
## v1.0.0
|
||||
|
||||
### Phase 4: Web Interface
|
||||
- **[phase-4-web-interface.md](phase-4-web-interface.md)** - Web interface design
|
||||
- **[phase-4-quick-reference.md](phase-4-quick-reference.md)** - Quick reference for Phase 4
|
||||
- **[phase-4-error-handling-fix.md](phase-4-error-handling-fix.md)** - Error handling improvements
|
||||
V1 foundation: phases 1–5 (utilities, models, notes, auth, web UI, RSS, container), Micropub, migrations, IndieAuth fixes and the release candidates.
|
||||
|
||||
### Phase 5: RSS & Deployment
|
||||
- **[phase-5-rss-and-container.md](phase-5-rss-and-container.md)** - RSS feed and container deployment
|
||||
- **[phase-5-executive-summary.md](phase-5-executive-summary.md)** - Executive summary of Phase 5
|
||||
- **[phase-5-quick-reference.md](phase-5-quick-reference.md)** - Quick reference for Phase 5
|
||||
- [2025-11-18-auth-redirect-loop-fix.md](v1.0.0/2025-11-18-auth-redirect-loop-fix.md) — Auth Redirect Loop Fix - Implementation Report
|
||||
- [2025-11-18-quickfix-auth-loop.md](v1.0.0/2025-11-18-quickfix-auth-loop.md) — QUICK FIX: Auth Redirect Loop
|
||||
- [2025-11-19-container-implementation-summary.md](v1.0.0/2025-11-19-container-implementation-summary.md) — Phase 5 Containerization - Implementation Complete
|
||||
- [2025-11-19-migration-implementation-quick-reference.md](v1.0.0/2025-11-19-migration-implementation-quick-reference.md) — Migration System - Quick Reference Card
|
||||
- [2025-11-19-migration-system-implementation-guidance.md](v1.0.0/2025-11-19-migration-system-implementation-guidance.md) — Migration System Implementation Guidance
|
||||
- [2025-11-19-migration-system-implementation-report.md](v1.0.0/2025-11-19-migration-system-implementation-report.md) — Migration System Implementation Report
|
||||
- [2025-11-19-todo-test-updates.md](v1.0.0/2025-11-19-todo-test-updates.md) — Test Updates Required for ADR-019 Implementation
|
||||
- [2025-11-22-auth-route-prefix-fix.md](v1.0.0/2025-11-22-auth-route-prefix-fix.md) — Auth Route Prefix Fix Implementation Report
|
||||
- [2025-11-22-authorization-endpoint-fix.md](v1.0.0/2025-11-22-authorization-endpoint-fix.md) — IndieAuth Authentication Endpoint Correction
|
||||
- [2025-11-22-grant-type-fix.md](v1.0.0/2025-11-22-grant-type-fix.md) — IndieAuth Token Exchange grant_type Fix
|
||||
- [2025-11-24-endpoint-discovery-analysis.md](v1.0.0/2025-11-24-endpoint-discovery-analysis.md) — IndieAuth Endpoint Discovery Implementation Analysis
|
||||
- [2025-11-24-indieauth-removal-complete.md](v1.0.0/2025-11-24-indieauth-removal-complete.md) — IndieAuth Server Removal - Complete Implementation Report
|
||||
- [2025-11-24-migration-detection-hotfix-rc3.md](v1.0.0/2025-11-24-migration-detection-hotfix-rc3.md) — Migration Detection Hotfix - v1.0.0-rc.3
|
||||
- [2025-11-24-migration-fix-v1.0.0-rc.2.md](v1.0.0/2025-11-24-migration-fix-v1.0.0-rc.2.md) — Implementation Report: Migration Fix for v1.0.0-rc.2
|
||||
- [2025-11-24-phase1-indieauth-server-removal.md](v1.0.0/2025-11-24-phase1-indieauth-server-removal.md) — Phase 1: IndieAuth Authorization Server Removal - Implementation Report
|
||||
- [2025-11-24-v1.0.0-rc.5-implementation.md](v1.0.0/2025-11-24-v1.0.0-rc.5-implementation.md) — v1.0.0-rc.5 Implementation Report
|
||||
- [ADR-025-implementation-report.md](v1.0.0/ADR-025-implementation-report.md) — ADR-025 Implementation Report
|
||||
- [ADR-025-implementation-summary.md](v1.0.0/ADR-025-implementation-summary.md) — ADR-025 Implementation Summary
|
||||
- [ADR-025-versioning-guidance.md](v1.0.0/ADR-025-versioning-guidance.md) — ADR-019 Implementation: Versioning Guidance
|
||||
- [ARCHITECT-FINAL-ANALYSIS.md](v1.0.0/ARCHITECT-FINAL-ANALYSIS.md) — Architect Final Analysis - Delete Route 404 Fix
|
||||
- [container-deployment.md](v1.0.0/container-deployment.md) — StarPunk Container Deployment Guide
|
||||
- [custom-slug-bug-diagnosis.md](v1.0.0/custom-slug-bug-diagnosis.md) — Custom Slug Bug Diagnosis Report
|
||||
- [custom-slug-bug-implementation.md](v1.0.0/custom-slug-bug-implementation.md) — Custom Slug Bug Fix - Implementation Report
|
||||
- [database-migration-architecture.md](v1.0.0/database-migration-architecture.md) — Database Migration Architecture
|
||||
- [database-migration-conflict-diagnosis.md](v1.0.0/database-migration-conflict-diagnosis.md) — Database Migration Conflict Diagnosis Report
|
||||
- [delete-nonexistent-note-error-analysis.md](v1.0.0/delete-nonexistent-note-error-analysis.md) — Delete Nonexistent Note Error Analysis
|
||||
- [delete-route-404-fix-implementation.md](v1.0.0/delete-route-404-fix-implementation.md) — Delete Route 404 Fix - Implementation Report
|
||||
- [delete-route-fix-summary.md](v1.0.0/delete-route-fix-summary.md) — Delete Route Fix - Developer Summary
|
||||
- [delete-route-implementation-spec.md](v1.0.0/delete-route-implementation-spec.md) — Delete Route Implementation Specification
|
||||
- [dependencies-diagram.md](v1.0.0/dependencies-diagram.md) — StarPunk V1 Dependencies Diagram
|
||||
- [endpoint-discovery-answers.md](v1.0.0/endpoint-discovery-answers.md) — IndieAuth Endpoint Discovery: Definitive Implementation Answers
|
||||
- [error-handling-rest-vs-web-patterns.md](v1.0.0/error-handling-rest-vs-web-patterns.md) — Architectural Review: Error Handling in Web Routes
|
||||
- [feature-scope.md](v1.0.0/feature-scope.md) — StarPunk V1 Feature Scope
|
||||
- [fix-hardcoded-endpoints.md](v1.0.0/fix-hardcoded-endpoints.md) — Migration Guide: Fixing Hardcoded IndieAuth Endpoints
|
||||
- [identity-domain-validation-2025-11-19.md](v1.0.0/identity-domain-validation-2025-11-19.md) — Identity Domain Validation Report
|
||||
- [identity-page-customization-guide.md](v1.0.0/identity-page-customization-guide.md) — IndieAuth Identity Page Customization Guide
|
||||
- [identity-page.html](v1.0.0/identity-page.html) — Example IndieAuth identity page (HTML template)
|
||||
- [implementation-guide-expose-deleted-at.md](v1.0.0/implementation-guide-expose-deleted-at.md) — Implementation Guide: Expose deleted_at in Note Model
|
||||
- [implementation-plan.md](v1.0.0/implementation-plan.md) — StarPunk V1 Implementation Plan
|
||||
- [indieauth-assessment.md](v1.0.0/indieauth-assessment.md) — IndieAuth Architecture Assessment
|
||||
- [indieauth-client-diagnosis.md](v1.0.0/indieauth-client-diagnosis.md) — IndieAuth Client Registration Issue - Diagnosis Report
|
||||
- [indieauth-client-discovery-analysis.md](v1.0.0/indieauth-client-discovery-analysis.md) — IndieAuth Client Discovery Error - Architectural Analysis
|
||||
- [indieauth-client-discovery-fix-implementation.md](v1.0.0/indieauth-client-discovery-fix-implementation.md) — IndieAuth Client Discovery Fix - Implementation Report
|
||||
- [indieauth-client-discovery-root-cause-analysis.md](v1.0.0/indieauth-client-discovery-root-cause-analysis.md) — IndieAuth Client Discovery Root Cause Analysis
|
||||
- [indieauth-detailed-logging-implementation.md](v1.0.0/indieauth-detailed-logging-implementation.md) — IndieAuth Detailed Logging Implementation Report
|
||||
- [indieauth-endpoint-discovery-security.md](v1.0.0/indieauth-endpoint-discovery-security.md) — IndieAuth Endpoint Discovery Security Analysis
|
||||
- [indieauth-endpoint-discovery.md](v1.0.0/indieauth-endpoint-discovery.md) — IndieAuth Endpoint Discovery Architecture
|
||||
- [indieauth-fix-summary.md](v1.0.0/indieauth-fix-summary.md) — IndieAuth Authentication Fix - Quick Summary
|
||||
- [indieauth-identity-page.md](v1.0.0/indieauth-identity-page.md) — IndieAuth Identity Page Architecture
|
||||
- [indieauth-questions-answered.md](v1.0.0/indieauth-questions-answered.md) — IndieAuth Implementation Questions - Answered
|
||||
- [indieauth-removal-analysis.md](v1.0.0/indieauth-removal-analysis.md) — IndieAuth Removal Implementation Analysis
|
||||
- [indieauth-removal-architectural-review.md](v1.0.0/indieauth-removal-architectural-review.md) — Architectural Review: IndieAuth Authorization Server Removal
|
||||
- [indieauth-removal-implementation-guide.md](v1.0.0/indieauth-removal-implementation-guide.md) — IndieAuth Provider Removal - Implementation Guide
|
||||
- [indieauth-removal-phases.md](v1.0.0/indieauth-removal-phases.md) — IndieAuth Removal: Phased Implementation Guide
|
||||
- [indieauth-removal-plan.md](v1.0.0/indieauth-removal-plan.md) — IndieAuth Server Removal Plan
|
||||
- [indieauth-removal-questions.md](v1.0.0/indieauth-removal-questions.md) — IndieAuth Removal - Questions for Architect
|
||||
- [indieauth-spec-url-standardization-2025-11-24.md](v1.0.0/indieauth-spec-url-standardization-2025-11-24.md) — IndieAuth Specification URL Standardization Report
|
||||
- [indieauth-token-verification-diagnosis.md](v1.0.0/indieauth-token-verification-diagnosis.md) — IndieAuth Token Verification Diagnosis
|
||||
- [initial-files.md](v1.0.0/initial-files.md) — Initial Project Files Design
|
||||
- [initial-schema-implementation-guide.md](v1.0.0/initial-schema-implementation-guide.md) — Initial Schema SQL Implementation Guide
|
||||
- [initial-schema-quick-reference.md](v1.0.0/initial-schema-quick-reference.md) — INITIAL_SCHEMA_SQL Quick Reference
|
||||
- [micropub-401-diagnosis.md](v1.0.0/micropub-401-diagnosis.md) — Micropub 401 Unauthorized Error - Architectural Diagnosis
|
||||
- [micropub-endpoint-design.md](v1.0.0/micropub-endpoint-design.md) — Micropub Endpoint Design for StarPunk V1
|
||||
- [micropub-phase1-architecture-review.md](v1.0.0/micropub-phase1-architecture-review.md) — Micropub V1 Implementation - Phase 1 Architecture Review
|
||||
- [micropub-phase3-architecture-review.md](v1.0.0/micropub-phase3-architecture-review.md) — Micropub Phase 3 Implementation Architecture Review
|
||||
- [micropub-v1-implementation-progress.md](v1.0.0/micropub-v1-implementation-progress.md) — Micropub V1 Implementation Progress Report
|
||||
- [migration-failure-diagnosis-v1.0.0-rc.1.md](v1.0.0/migration-failure-diagnosis-v1.0.0-rc.1.md) — Migration Failure Diagnosis - v1.0.0-rc.1
|
||||
- [migration-fix-quick-reference.md](v1.0.0/migration-fix-quick-reference.md) — Migration Race Condition Fix - Quick Implementation Reference
|
||||
- [migration-race-condition-answers.md](v1.0.0/migration-race-condition-answers.md) — Migration Race Condition Fix - Architectural Answers
|
||||
- [migration-race-condition-fix-implementation.md](v1.0.0/migration-race-condition-fix-implementation.md) — Migration Race Condition Fix - Implementation Guide
|
||||
- [oauth-metadata-implementation-2025-11-19.md](v1.0.0/oauth-metadata-implementation-2025-11-19.md) — OAuth Client ID Metadata Document Implementation Report
|
||||
- [phase-1.1-core-utilities.md](v1.0.0/phase-1.1-core-utilities.md) — Phase 1.1: Core Utilities Design
|
||||
- [phase-1.1-quick-reference.md](v1.0.0/phase-1.1-quick-reference.md) — Phase 1.1 Quick Reference: Core Utilities
|
||||
- [phase-1.2-data-models.md](v1.0.0/phase-1.2-data-models.md) — Phase 1.2: Data Models Design
|
||||
- [phase-1.2-quick-reference.md](v1.0.0/phase-1.2-quick-reference.md) — Phase 1.2 Quick Reference: Data Models
|
||||
- [phase-2-architectural-review.md](v1.0.0/phase-2-architectural-review.md) — Architectural Review: Phase 2 Implementation
|
||||
- [phase-2-implementation-report.md](v1.0.0/phase-2-implementation-report.md) — Phase 2 Implementation Report: Authorization and Token Endpoints
|
||||
- [phase-2.1-implementation-20251118.md](v1.0.0/phase-2.1-implementation-20251118.md) — Phase 2.1 Implementation Report: Notes Management
|
||||
- [phase-2.1-notes-management.md](v1.0.0/phase-2.1-notes-management.md) — Phase 2.1: Notes Management Design
|
||||
- [phase-2.1-quick-reference.md](v1.0.0/phase-2.1-quick-reference.md) — Phase 2.1: Notes Management - Quick Reference
|
||||
- [phase-3-authentication-20251118.md](v1.0.0/phase-3-authentication-20251118.md) — Phase 3: Authentication Implementation Report
|
||||
- [phase-3-authentication-architectural-review.md](v1.0.0/phase-3-authentication-architectural-review.md) — Phase 3: Authentication Implementation - Architectural Review
|
||||
- [phase-3-authentication-implementation.md](v1.0.0/phase-3-authentication-implementation.md) — Phase 3: Authentication Implementation Design
|
||||
- [phase-3-authentication.md](v1.0.0/phase-3-authentication.md) — Phase 3: Authentication Design
|
||||
- [phase-4-architectural-assessment-20251118.md](v1.0.0/phase-4-architectural-assessment-20251118.md) — Phase 4: Architectural Assessment and Design Report
|
||||
- [phase-4-error-handling-fix.md](v1.0.0/phase-4-error-handling-fix.md) — Phase 4: Error Handling Fix - Implementation Guide
|
||||
- [phase-4-quick-reference.md](v1.0.0/phase-4-quick-reference.md) — Phase 4: Quick Reference
|
||||
- [phase-4-test-fixes.md](v1.0.0/phase-4-test-fixes.md) — Phase 4 Test Fixes Report
|
||||
- [phase-4-web-interface.md](v1.0.0/phase-4-web-interface.md) — Phase 4: Web Interface - Implementation Design
|
||||
- [phase-5-approval-summary.md](v1.0.0/phase-5-approval-summary.md) — Phase 5 Containerization - Approval Summary
|
||||
- [phase-5-container-architectural-review.md](v1.0.0/phase-5-container-architectural-review.md) — Phase 5 Containerization - Architectural Review
|
||||
- [phase-5-container-implementation-report.md](v1.0.0/phase-5-container-implementation-report.md) — Phase 5 Container Implementation Report
|
||||
- [phase-5-executive-summary.md](v1.0.0/phase-5-executive-summary.md) — Phase 5 Executive Summary
|
||||
- [phase-5-pre-implementation-review.md](v1.0.0/phase-5-pre-implementation-review.md) — Phase 5 Pre-Implementation Review
|
||||
- [phase-5-quick-reference.md](v1.0.0/phase-5-quick-reference.md) — Phase 5 Quick Reference Guide
|
||||
- [phase-5-rss-and-container.md](v1.0.0/phase-5-rss-and-container.md) — Phase 5: RSS Feed & Production Container - Implementation Design
|
||||
- [phase-5-rss-implementation-20251119.md](v1.0.0/phase-5-rss-implementation-20251119.md) — Phase 5: RSS Feed Implementation Report
|
||||
- [phase-5-validation-report.md](v1.0.0/phase-5-validation-report.md) — Phase 5 RSS Feed Implementation - Architectural Validation Report
|
||||
- [phase1-completion-guide.md](v1.0.0/phase1-completion-guide.md) — Phase 1 Completion Guide: Test Cleanup and Commit
|
||||
- [project-structure.md](v1.0.0/project-structure.md) — Project Structure Design
|
||||
- [quick-reference.md](v1.0.0/quick-reference.md) — StarPunk V1 Quick Reference
|
||||
- [review-v1.0.0-rc.5.md](v1.0.0/review-v1.0.0-rc.5.md) — Architectural Review: v1.0.0-rc.5 Implementation
|
||||
- [setup-complete-2025-11-18.md](v1.0.0/setup-complete-2025-11-18.md) — StarPunk Project Setup - Complete
|
||||
- [test-failure-analysis-deleted-at-attribute.md](v1.0.0/test-failure-analysis-deleted-at-attribute.md) — Test Failure Analysis: Missing `deleted_at` Attribute on Note Model
|
||||
- [troubleshooting.md](v1.0.0/troubleshooting.md) — StarPunk Troubleshooting Guide
|
||||
- [v0.9.1-implementation-report.md](v1.0.0/v0.9.1-implementation-report.md) — StarPunk v0.9.1 Implementation Report
|
||||
- [v1-planning-overview.md](v1.0.0/v1-planning-overview.md) — StarPunk V1 Project Plan
|
||||
- [v1.0.0-rc.1-hotfix-instructions.md](v1.0.0/v1.0.0-rc.1-hotfix-instructions.md) — v1.0.0-rc.1 Production Hotfix Instructions
|
||||
- [v1.0.0-rc.5-migration-race-condition-implementation.md](v1.0.0/v1.0.0-rc.5-migration-race-condition-implementation.md) — v1.0.0-rc.5 Migration Race Condition Fix - Implementation Report
|
||||
- [v1.0.0-release-validation.md](v1.0.0/v1.0.0-release-validation.md) — StarPunk v1.0.0 Release Validation Report
|
||||
- [v1.0.1-hotfix-plan.md](v1.0.0/v1.0.1-hotfix-plan.md) — StarPunk v1.0.1 Hotfix Release Plan
|
||||
|
||||
## Feature-Specific Design
|
||||
## v1.1.1
|
||||
|
||||
### Micropub API
|
||||
- **[micropub-endpoint-design.md](micropub-endpoint-design.md)** - Micropub endpoint detailed design
|
||||
v1.1.0 "SearchLight" (search) and v1.1.1 "Polish" designs and reports; v1.0.1 hotfix.
|
||||
|
||||
### Authentication Fixes
|
||||
- **[auth-redirect-loop-diagnosis.md](auth-redirect-loop-diagnosis.md)** - Diagnosis of redirect loop issues
|
||||
- **[auth-redirect-loop-diagram.md](auth-redirect-loop-diagram.md)** - Visual diagrams of the problem
|
||||
- **[auth-redirect-loop-executive-summary.md](auth-redirect-loop-executive-summary.md)** - Executive summary
|
||||
- **[auth-redirect-loop-fix-implementation.md](auth-redirect-loop-fix-implementation.md)** - Implementation guide
|
||||
- [2025-11-25-hotfix-v1.1.1-rc.2-implementation.md](v1.1.1/2025-11-25-hotfix-v1.1.1-rc.2-implementation.md) — Implementation Report: Hotfix v1.1.1-rc.2 - Admin Dashboard Route Conflict
|
||||
- [2025-11-25-v1.0.1-micropub-url-fix.md](v1.1.1/2025-11-25-v1.0.1-micropub-url-fix.md) — v1.0.1 Hotfix Implementation Report
|
||||
- [auth-redirect-loop-diagnosis.md](v1.1.1/auth-redirect-loop-diagnosis.md) — Authentication Redirect Loop Diagnosis - Phase 4
|
||||
- [auth-redirect-loop-diagram.md](v1.1.1/auth-redirect-loop-diagram.md) — Auth Redirect Loop - Visual Diagram
|
||||
- [auth-redirect-loop-executive-summary.md](v1.1.1/auth-redirect-loop-executive-summary.md) — Auth Redirect Loop - Executive Summary
|
||||
- [auth-redirect-loop-fix-implementation.md](v1.1.1/auth-redirect-loop-fix-implementation.md) — Implementation Guide: Auth Redirect Loop Fix
|
||||
- [hotfix-v1.1.1-rc2-consolidated.md](v1.1.1/hotfix-v1.1.1-rc2-consolidated.md) — Hotfix Design: v1.1.1-rc.2 - Metrics Dashboard Template Data Mismatch
|
||||
- [hotfix-v1.1.1-rc2-review.md](v1.1.1/hotfix-v1.1.1-rc2-review.md) — Architectural Review: Hotfix v1.1.1-rc.2
|
||||
- [hotfix-v1.1.1-rc2-route-conflict.md](v1.1.1/hotfix-v1.1.1-rc2-route-conflict.md) — Hotfix Design: v1.1.1-rc.2 Route Conflict Resolution
|
||||
- [hotfix-validation-script.md](v1.1.1/hotfix-validation-script.md) — Hotfix Validation Script for v1.1.1-rc.2
|
||||
- [indieauth-pkce-authentication.md](v1.1.1/indieauth-pkce-authentication.md) — IndieAuth PKCE Authentication - Technical Design
|
||||
- [potential-features.md](v1.1.1/potential-features.md) — StarPunk V1.1: Potential Features
|
||||
- [priority-work.md](v1.1.1/priority-work.md) — StarPunk v1.1.0: Priority Work Items
|
||||
- [syndication-features.md](v1.1.1/syndication-features.md) — Syndication Features Specification
|
||||
- [token-security-migration.md](v1.1.1/token-security-migration.md) — Token Security Migration Strategy
|
||||
- [upgrade-to-v1.1.1.md](v1.1.1/upgrade-to-v1.1.1.md) — Upgrade Guide: StarPunk v1.1.1 "Polish"
|
||||
- [v1.1.0-feature-architecture.md](v1.1.1/v1.1.0-feature-architecture.md) — StarPunk v1.1.0 Feature Architecture
|
||||
- [v1.1.0-implementation-decisions.md](v1.1.1/v1.1.0-implementation-decisions.md) — V1.1.0 Implementation Decisions - Architectural Guidance
|
||||
- [v1.1.0-implementation-plan.md](v1.1.1/v1.1.0-implementation-plan.md) — StarPunk v1.1.0 Implementation Plan
|
||||
- [v1.1.0-implementation-report.md](v1.1.1/v1.1.0-implementation-report.md) — StarPunk v1.1.0 Implementation Report
|
||||
- [v1.1.0-release-status.md](v1.1.1/v1.1.0-release-status.md) — StarPunk v1.1.0 "SearchLight" Release Status
|
||||
- [v1.1.0-search-ui-validation.md](v1.1.1/v1.1.0-search-ui-validation.md) — StarPunk v1.1.0 Search UI Implementation Review
|
||||
- [v1.1.0-validation-report.md](v1.1.1/v1.1.0-validation-report.md) — StarPunk v1.1.0 Implementation Validation & Search UI Design
|
||||
- [v1.1.1-architecture-overview.md](v1.1.1/v1.1.1-architecture-overview.md) — v1.1.1 "Polish" Architecture Overview
|
||||
- [v1.1.1-final-release-review.md](v1.1.1/v1.1.1-final-release-review.md) — StarPunk v1.1.1 "Polish" - Final Architectural Release Review
|
||||
- [v1.1.1-instrumentation-assessment.md](v1.1.1/v1.1.1-instrumentation-assessment.md) — v1.1.1 Performance Monitoring Instrumentation Assessment
|
||||
- [v1.1.1-phase1-architectural-review.md](v1.1.1/v1.1.1-phase1-architectural-review.md) — StarPunk v1.1.1 Phase 1 - Architectural Review Report
|
||||
- [v1.1.1-phase1-implementation.md](v1.1.1/v1.1.1-phase1-implementation.md) — StarPunk v1.1.1 Phase 1 Implementation Report
|
||||
- [v1.1.1-phase2-architectural-review.md](v1.1.1/v1.1.1-phase2-architectural-review.md) — StarPunk v1.1.1 "Polish" - Phase 2 Architectural Review
|
||||
- [v1.1.1-phase2-implementation.md](v1.1.1/v1.1.1-phase2-implementation.md) — StarPunk v1.1.1 "Polish" - Phase 2 Implementation Report
|
||||
- [v1.1.1-phase3-implementation.md](v1.1.1/v1.1.1-phase3-implementation.md) — StarPunk v1.1.1 "Polish" - Phase 3 Implementation Report
|
||||
|
||||
### Database Schema
|
||||
- **[initial-schema-implementation-guide.md](initial-schema-implementation-guide.md)** - Schema implementation guide
|
||||
- **[initial-schema-quick-reference.md](initial-schema-quick-reference.md)** - Quick reference
|
||||
## v1.1.2
|
||||
|
||||
### Security
|
||||
- **[token-security-migration.md](token-security-migration.md)** - Token security improvements
|
||||
v1.1.2 "Syndicate": metrics, ATOM/JSON feeds, feed statistics, OPML.
|
||||
|
||||
## Version-Specific Design
|
||||
- [2025-11-26-phase2-architect-review.md](v1.1.2/2025-11-26-phase2-architect-review.md) — Architectural Review: StarPunk v1.1.2 Phase 2 "Syndicate" - Feed Formats
|
||||
- [2025-11-26-v1.1.2-phase1-review.md](v1.1.2/2025-11-26-v1.1.2-phase1-review.md) — StarPunk v1.1.2 Phase 1 Implementation Review
|
||||
- [2025-11-26-v1.1.2-phase2-complete.md](v1.1.2/2025-11-26-v1.1.2-phase2-complete.md) — StarPunk v1.1.2 Phase 2 Feed Formats - Implementation Report (COMPLETE)
|
||||
- [2025-11-26-v1.1.2-phase2-feed-formats-partial.md](v1.1.2/2025-11-26-v1.1.2-phase2-feed-formats-partial.md) — StarPunk v1.1.2 Phase 2 Feed Formats - Implementation Report (Partial)
|
||||
- [2025-11-27-phase3-architect-review.md](v1.1.2/2025-11-27-phase3-architect-review.md) — StarPunk v1.1.2 Phase 3 - Architectural Review
|
||||
- [2025-11-27-v1.1.2-phase3-complete.md](v1.1.2/2025-11-27-v1.1.2-phase3-complete.md) — v1.1.2 Phase 3 Implementation Report - Feed Statistics & OPML
|
||||
- [2025-11-28-v1.1.2-rc.1-architect-review.md](v1.1.2/2025-11-28-v1.1.2-rc.1-architect-review.md) — Architect Review: v1.1.2-rc.1 Production Issues
|
||||
- [2025-11-28-v1.1.2-rc.1-production-issues.md](v1.1.2/2025-11-28-v1.1.2-rc.1-production-issues.md) — v1.1.2-rc.1 Production Issues Investigation Report
|
||||
- [2025-11-28-v1.1.2-rc.2-fixes.md](v1.1.2/2025-11-28-v1.1.2-rc.2-fixes.md) — v1.1.2-rc.2 Production Bug Fixes - Implementation Report
|
||||
- [feed-media-handling-options.md](v1.1.2/feed-media-handling-options.md) — Feed Media Handling: Architecture Options Analysis
|
||||
- [feed-media-option2-design.md](v1.1.2/feed-media-option2-design.md) — Feed Media Enhancement Design: Option 2 (RSS + Media RSS Extension)
|
||||
- [upgrade-to-v1.1.2.md](v1.1.2/upgrade-to-v1.1.2.md) — Upgrade Guide: StarPunk v1.1.2 "Syndicate"
|
||||
- [v1.1.2-caption-alttext-update.md](v1.1.2/v1.1.2-caption-alttext-update.md) — Caption Display Update - Alt Text Only (v1.1.2)
|
||||
- [v1.1.2-options.md](v1.1.2/v1.1.2-options.md) — StarPunk v1.1.2 Release Plan Options
|
||||
- [v1.1.2-phase1-metrics-implementation.md](v1.1.2/v1.1.2-phase1-metrics-implementation.md) — StarPunk v1.1.2 Phase 1: Metrics Instrumentation - Implementation Report
|
||||
- [v1.1.2-syndicate-architecture.md](v1.1.2/v1.1.2-syndicate-architecture.md) — StarPunk v1.1.2 "Syndicate" - Architecture Overview
|
||||
|
||||
### v1.1.1
|
||||
- **[v1.1.1/](v1.1.1/)** - v1.1.1 specific design documents
|
||||
## v1.2.0
|
||||
|
||||
## Quick Reference Documents
|
||||
Custom slugs, author discovery and microformats2, media upload and display.
|
||||
|
||||
Quick reference documents provide condensed, actionable information for developers:
|
||||
- **phase-1.1-quick-reference.md** - Core utilities quick ref
|
||||
- **phase-1.2-quick-reference.md** - Data models quick ref
|
||||
- **phase-2.1-quick-reference.md** - Notes management quick ref
|
||||
- **phase-4-quick-reference.md** - Web interface quick ref
|
||||
- **phase-5-quick-reference.md** - RSS and deployment quick ref
|
||||
- **initial-schema-quick-reference.md** - Database schema quick ref
|
||||
- [2025-11-28-v1.2.0-design-complete.md](v1.2.0/2025-11-28-v1.2.0-design-complete.md) — v1.2.0 Design Review - Complete
|
||||
- [2025-11-28-v1.2.0-media-display-fixes.md](v1.2.0/2025-11-28-v1.2.0-media-display-fixes.md) — Media Display Fixes Implementation Report
|
||||
- [2025-11-28-v1.2.0-phase1-custom-slugs.md](v1.2.0/2025-11-28-v1.2.0-phase1-custom-slugs.md) — v1.2.0 Phase 1: Custom Slugs - Implementation Report
|
||||
- [2025-11-28-v1.2.0-phase1-review.md](v1.2.0/2025-11-28-v1.2.0-phase1-review.md) — v1.2.0 Phase 1: Custom Slugs - Architectural Review
|
||||
- [2025-11-28-v1.2.0-phase2-author-microformats.md](v1.2.0/2025-11-28-v1.2.0-phase2-author-microformats.md) — v1.2.0 Phase 2 Implementation Report: Author Discovery & Microformats2
|
||||
- [2025-11-28-v1.2.0-phase2-review.md](v1.2.0/2025-11-28-v1.2.0-phase2-review.md) — v1.2.0 Phase 2 Architectural Review: Author Discovery & Microformats2
|
||||
- [2025-11-28-v1.2.0-phase3-media-upload.md](v1.2.0/2025-11-28-v1.2.0-phase3-media-upload.md) — v1.2.0 Phase 3: Media Upload - Implementation Report
|
||||
- [2025-11-28-v1.2.0-phase3-review.md](v1.2.0/2025-11-28-v1.2.0-phase3-review.md) — v1.2.0 Phase 3 Architecture Review: Media Upload
|
||||
- [2025-12-09-feed-media-implementation.md](v1.2.0/2025-12-09-feed-media-implementation.md) — Feed Media Enhancement Implementation Report
|
||||
- [2025-12-09-media-display-validation.md](v1.2.0/2025-12-09-media-display-validation.md) — Media Display Implementation Validation Report
|
||||
- [2025-12-09-v1.2.0-release.md](v1.2.0/2025-12-09-v1.2.0-release.md) — v1.2.0 Release Report
|
||||
- [2025-12-10-documentation-audit-v1.2.0.md](v1.2.0/2025-12-10-documentation-audit-v1.2.0.md) — Documentation Audit Report - Post v1.2.0 Release
|
||||
- [media-display-fixes.md](v1.2.0/media-display-fixes.md) — Media Display Fixes - Architectural Design
|
||||
- [v1.2.0-media-css-design.md](v1.2.0/v1.2.0-media-css-design.md) — CSS Design for Media Display (v1.2.0)
|
||||
- [v1.X.X-indieweb-options.md](v1.2.0/v1.X.X-indieweb-options.md) — StarPunk v1.X.X IndieWeb-Focused Release Options
|
||||
- [v1.X.X-options.md](v1.2.0/v1.X.X-options.md) — StarPunk Next Release Options
|
||||
|
||||
## How to Use This Documentation
|
||||
## v1.3.0
|
||||
|
||||
### For Developers Implementing Features
|
||||
1. Start with the relevant **phase** document (e.g., phase-2.1-notes-management.md)
|
||||
2. Consult the **quick reference** for that phase
|
||||
3. Check **feature-specific design** docs for details
|
||||
4. Reference **ADRs** in ../decisions/ for architectural decisions
|
||||
Microformats2 compliance and tags.
|
||||
|
||||
### For Planning New Features
|
||||
1. Review similar **phase documents** for patterns
|
||||
2. Check **project-structure.md** for organization guidelines
|
||||
3. Create new design doc following existing format
|
||||
4. Update this index with the new document
|
||||
- [2025-12-10-phase1-implementation.md](v1.3.0/2025-12-10-phase1-implementation.md) — v1.3.0 Phase 1 Implementation Report
|
||||
- [2025-12-10-phase2-implementation.md](v1.3.0/2025-12-10-phase2-implementation.md) — v1.3.0 Phase 2: Templates Implementation Report
|
||||
- [2025-12-10-phase3-implementation.md](v1.3.0/2025-12-10-phase3-implementation.md) — v1.3.0 Phase 3 Implementation Report
|
||||
- [2025-12-10-phase4-implementation.md](v1.3.0/2025-12-10-phase4-implementation.md) — v1.3.0 Phase 4: Validation Implementation Report
|
||||
- [microformats-tags-design.md](v1.3.0/microformats-tags-design.md) — v1.3.0 Microformats2 Compliance and Tags Implementation Design
|
||||
|
||||
### For Understanding Existing Code
|
||||
1. Find the **phase** that implemented the feature
|
||||
2. Read the design document for context
|
||||
3. Check **ADRs** for decision rationale
|
||||
4. Review implementation reports in ../reports/
|
||||
## v1.3.1
|
||||
|
||||
## Document Types
|
||||
Tags in feeds.
|
||||
|
||||
### Phase Documents
|
||||
Comprehensive plans for each development phase, including:
|
||||
- Goals and scope
|
||||
- Implementation tasks
|
||||
- Dependencies
|
||||
- Testing requirements
|
||||
- [2025-12-10-feed-tags-implementation.md](v1.3.1/2025-12-10-feed-tags-implementation.md) — Feed Tags Implementation Report
|
||||
- [feed-tags-design.md](v1.3.1/feed-tags-design.md) — Feed Tags Implementation Design
|
||||
|
||||
### Quick Reference Documents
|
||||
Condensed information for rapid development:
|
||||
- Key decisions
|
||||
- Code patterns
|
||||
- Common operations
|
||||
- Gotchas and notes
|
||||
## v1.4.0
|
||||
|
||||
### Feature Design Documents
|
||||
Detailed specifications for specific features:
|
||||
- Requirements
|
||||
- API design
|
||||
- Data models
|
||||
- UI/UX considerations
|
||||
"Media" release: image variants and media pipeline.
|
||||
|
||||
### Diagnostic Documents
|
||||
Problem analysis and solutions:
|
||||
- Issue description
|
||||
- Root cause analysis
|
||||
- Solution design
|
||||
- Implementation plan
|
||||
- [2025-12-16-phase4-implementation-report.md](v1.4.0/2025-12-16-phase4-implementation-report.md) — v1.4.0 Phase 4 Implementation Report
|
||||
- [2025-12-16-v140-implementation-complete.md](v1.4.0/2025-12-16-v140-implementation-complete.md) — v1.4.0 "Media" Release - Implementation Complete
|
||||
- [media-implementation-design.md](v1.4.0/media-implementation-design.md) — v1.4.0 "Media" Release - Implementation Design
|
||||
|
||||
## v1.4.1
|
||||
|
||||
Media upload logging.
|
||||
|
||||
- [2025-12-16-implementation-report.md](v1.4.1/2025-12-16-implementation-report.md) — Implementation Report - v1.4.1 Media Upload Logging
|
||||
- [architect-responses.md](v1.4.1/architect-responses.md) — Architect Responses - v1.4.1 Media Upload Logging
|
||||
- [developer-questions.md](v1.4.1/developer-questions.md) — Developer Questions - v1.4.1 Media Upload Logging
|
||||
- [media-logging-design.md](v1.4.1/media-logging-design.md) — Media Upload Logging Design - v1.4.1
|
||||
|
||||
## v1.4.2
|
||||
|
||||
HEIC/MPO support and larger dimension limit.
|
||||
|
||||
- [2025-12-16-architect-review.md](v1.4.2/2025-12-16-architect-review.md) — Architect Review: v1.4.2 Implementation and End-to-End Media Pipeline
|
||||
- [2025-12-16-developer-review.md](v1.4.2/2025-12-16-developer-review.md) — v1.4.2 Developer Review - HEIC/MPO Support and Dimension Limit Increase
|
||||
- [2025-12-16-implementation-report.md](v1.4.2/2025-12-16-implementation-report.md) — v1.4.2 Implementation Report - HEIC Image Support
|
||||
- [heic-support-design.md](v1.4.2/heic-support-design.md) — HEIC Image Support Design - v1.4.2
|
||||
|
||||
## v1.5.0
|
||||
|
||||
Timestamp slugs, debug file management, N+1 fix, atomic variants, test coverage.
|
||||
|
||||
- [2025-12-16-architect-responses.md](v1.5.0/2025-12-16-architect-responses.md) — Architect Responses - v1.5.0 Developer Questions
|
||||
- [2025-12-16-developer-questions.md](v1.5.0/2025-12-16-developer-questions.md) — Developer Review Questions - v1.5.0 Release Plan
|
||||
- [2025-12-16-phase-0-implementation-report.md](v1.5.0/2025-12-16-phase-0-implementation-report.md) — Phase 0 Implementation Report - Test Fixes
|
||||
- [2025-12-16-phase2-architect-review.md](v1.5.0/2025-12-16-phase2-architect-review.md) — Phase 2 Architect Review: Debug File Management
|
||||
- [2025-12-17-indieauth-discovery-fix.md](v1.5.0/2025-12-17-indieauth-discovery-fix.md) — IndieAuth Endpoint Discovery Regression Analysis
|
||||
- [2025-12-17-phase-1-implementation-report.md](v1.5.0/2025-12-17-phase-1-implementation-report.md) — Phase 1 Implementation Report: Timestamp-Based Slugs
|
||||
- [2025-12-17-phase3-architect-review.md](v1.5.0/2025-12-17-phase3-architect-review.md) — Phase 3 Architect Review: N+1 Query Fix
|
||||
- [2025-12-17-phase3-implementation.md](v1.5.0/2025-12-17-phase3-implementation.md) — v1.5.0 Phase 3 Implementation Report
|
||||
- [2025-12-17-phase4-architect-review.md](v1.5.0/2025-12-17-phase4-architect-review.md) — Phase 4 Architect Review: Atomic Variant Generation
|
||||
- [2025-12-17-phase4-implementation.md](v1.5.0/2025-12-17-phase4-implementation.md) — v1.5.0 Phase 4: Atomic Variant Generation - Implementation Report
|
||||
- [2025-12-17-phase5-architect-review.md](v1.5.0/2025-12-17-phase5-architect-review.md) — Phase 5 Architect Review: Test Coverage Expansion (FINAL)
|
||||
- [2025-12-17-phase5-implementation.md](v1.5.0/2025-12-17-phase5-implementation.md) — v1.5.0 Phase 5: Test Coverage Expansion - Implementation Report
|
||||
- [phase-2-implementation-report.md](v1.5.0/phase-2-implementation-report.md) — v1.5.0 Phase 2 Implementation Report: Debug File Management
|
||||
|
||||
## v1.6.0
|
||||
|
||||
Webmentions (receive, send, moderate), reply contexts, rate limiting. Original documents lost; reconstructed from the code.
|
||||
|
||||
- [api-contract.md](v1.6.0/api-contract.md) — v1.6.0 Design: Webmention API Contract *(reconstructed)*
|
||||
- [background-worker.md](v1.6.0/background-worker.md) — v1.6.0 Design: Webmention Background Worker *(reconstructed)*
|
||||
- [database-schema.md](v1.6.0/database-schema.md) — v1.6.0 Design: Webmention Database Schema *(reconstructed)*
|
||||
- [display-moderation.md](v1.6.0/display-moderation.md) — v1.6.0 Design: Webmention Display and Moderation *(reconstructed)*
|
||||
- [rate-limiting.md](v1.6.0/rate-limiting.md) — v1.6.0 Design: Webmention Rate Limiting *(reconstructed)*
|
||||
|
||||
## v2.0.0
|
||||
|
||||
OIDC admin login and built-in IndieAuth server.
|
||||
|
||||
- [2026-10-31-oidc-migration.md](v2.0.0/2026-10-31-oidc-migration.md) — v2.0.0 Implementation Report: OIDC Login and Built-in IndieAuth Server
|
||||
|
||||
## v2.0.1
|
||||
|
||||
- [2026-10-31-absolute-urls-and-ci-buildkit.md](v2.0.1/2026-10-31-absolute-urls-and-ci-buildkit.md) — Absolute URLs from SITE_URL; CI builds with BuildKit *(reconstructed)*
|
||||
|
||||
## v2.0.2
|
||||
|
||||
- [2026-10-31-ci-registry-token.md](v2.0.2/2026-10-31-ci-registry-token.md) — CI registry login with REGISTRY_TOKEN *(reconstructed)*
|
||||
|
||||
## v2.1.0
|
||||
|
||||
- [2026-10-31-gotosocial-crossposting.md](v2.1.0/2026-10-31-gotosocial-crossposting.md) — Crossposting to GoToSocial *(reconstructed)*
|
||||
|
||||
## v2.2.0
|
||||
|
||||
- [2026-10-31-bluesky-crossposting.md](v2.2.0/2026-10-31-bluesky-crossposting.md) — Crossposting to Bluesky *(reconstructed)*
|
||||
|
||||
## v2.3.0
|
||||
|
||||
- [2026-10-31-public-page-restyle.md](v2.3.0/2026-10-31-public-page-restyle.md) — Public page restyle *(reconstructed)*
|
||||
|
||||
## v2.3.1
|
||||
|
||||
- [2026-10-31-login-link-in-header.md](v2.3.1/2026-10-31-login-link-in-header.md) — "Log in" back in the header *(reconstructed)*
|
||||
|
||||
## v2.3.2
|
||||
|
||||
- [2026-10-31-header-admin-link.md](v2.3.2/2026-10-31-header-admin-link.md) — Header shows "Admin" when logged in *(reconstructed)*
|
||||
|
||||
## v2.3.3
|
||||
|
||||
- [2026-10-31-micropub-title-and-source-url.md](v2.3.3/2026-10-31-micropub-title-and-source-url.md) — Micropub titles, q=source URL, single title display
|
||||
|
||||
## v2.4.0
|
||||
|
||||
- [2026-10-31-micropub-update-delete-drafts.md](v2.4.0/2026-10-31-micropub-update-delete-drafts.md) — Micropub update, delete/undelete, drafts, replies *(reconstructed)*
|
||||
|
||||
## v2.5.0
|
||||
|
||||
- [2026-10-31-likes-and-reposts.md](v2.5.0/2026-10-31-likes-and-reposts.md) — Likes and reposts *(reconstructed)*
|
||||
|
||||
## v2.6.0
|
||||
|
||||
- [2026-10-31-threads-crossposting.md](v2.6.0/2026-10-31-threads-crossposting.md) — Crossposting to Threads
|
||||
|
||||
## v2.6.1
|
||||
|
||||
- [2026-10-31-threads-publish-retry.md](v2.6.1/2026-10-31-threads-publish-retry.md) — Retry Threads publish errors *(reconstructed)*
|
||||
|
||||
## v2.6.2
|
||||
|
||||
- [2026-10-31-threads-link-preview.md](v2.6.2/2026-10-31-threads-link-preview.md) — Let Threads build the link preview *(reconstructed)*
|
||||
|
||||
## v2.6.3
|
||||
|
||||
- [2026-11-01-crosspost-links-and-cards.md](v2.6.3/2026-11-01-crosspost-links-and-cards.md) — Crosspost cards show the note's own link *(reconstructed)*
|
||||
|
||||
## v2.6.4
|
||||
|
||||
- [2026-11-01-tag-entry-order.md](v2.6.4/2026-11-01-tag-entry-order.md) — Tags keep the order they were entered in *(reconstructed)*
|
||||
|
||||
## v2.6.5
|
||||
|
||||
- [2026-11-01-micropub-media-fixes.md](v2.6.5/2026-11-01-micropub-media-fixes.md) — Micropub media fixes
|
||||
|
||||
## v2.6.6
|
||||
|
||||
- [2026-11-02-crosspost-embedded-images.md](v2.6.6/2026-11-02-crosspost-embedded-images.md) — Crossposts include photos embedded in the text *(reconstructed)*
|
||||
|
||||
## v2.6.7
|
||||
|
||||
- [2026-11-02-admin-layout-width.md](v2.6.7/2026-11-02-admin-layout-width.md) — Admin pages line up with the header *(reconstructed)*
|
||||
|
||||
## v2.6.8
|
||||
|
||||
- [2026-11-03-webmention-worker-app-context.md](v2.6.8/2026-11-03-webmention-worker-app-context.md) — Webmention worker app context
|
||||
|
||||
## v2.6.9
|
||||
|
||||
- [2026-11-03-webmention-processing-fixes.md](v2.6.9/2026-11-03-webmention-processing-fixes.md) — Webmention processing fixes
|
||||
|
||||
## v2.7.0
|
||||
|
||||
- [2026-11-03-webmention-spec-compliance.md](v2.7.0/2026-11-03-webmention-spec-compliance.md) — Webmention W3C compliance
|
||||
|
||||
## v2.7.1
|
||||
|
||||
- [2026-11-03-ci-test-stage.md](v2.7.1/2026-11-03-ci-test-stage.md) — Tests run before release images are built *(reconstructed)*
|
||||
|
||||
## v2.7.2
|
||||
|
||||
- [2026-11-03-webmention-shoulds.md](v2.7.2/2026-11-03-webmention-shoulds.md) — Remaining Webmention SHOULDs
|
||||
|
||||
## v2.8.0
|
||||
|
||||
- [2026-11-03-microformats2-review.md](v2.8.0/2026-11-03-microformats2-review.md) — Microformats2 review
|
||||
|
||||
## v2.8.1
|
||||
|
||||
- [2026-11-03-migration-lock.md](v2.8.1/2026-11-03-migration-lock.md) — Migration lock held for the whole run
|
||||
|
||||
## v2.9.0
|
||||
|
||||
- [2026-11-03-micropub-review.md](v2.9.0/2026-11-03-micropub-review.md) — Micropub review
|
||||
|
||||
## v2.10.0
|
||||
|
||||
- [2026-11-03-websub-implementation.md](v2.10.0/2026-11-03-websub-implementation.md) — WebSub with a built-in hub (implementation report) *(reconstructed)*
|
||||
|
||||
## v2.10.1
|
||||
|
||||
- [2026-11-03-websub-hardening.md](v2.10.1/2026-11-03-websub-hardening.md) — WebSub hardening (implementation report) *(reconstructed)*
|
||||
|
||||
## v2.10.2
|
||||
|
||||
- [2026-11-03-reply-context-fetch.md](v2.10.2/2026-11-03-reply-context-fetch.md) - Reply contexts fetched again (fetch_source returns three values since v2.7.0)
|
||||
|
||||
## v2.11.0
|
||||
|
||||
- [2026-11-03-maintenance-release.md](v2.11.0/2026-11-03-maintenance-release.md) - Dependency upgrades (Python 3.14), foreign keys, CSRF, fetchers, debug default, shared rate limits
|
||||
|
||||
## v2.12.0
|
||||
|
||||
- [2026-11-03-remove-metrics.md](v2.12.0/2026-11-03-remove-metrics.md) - Metrics dashboard and `starpunk/monitoring` removed (ADR-072)
|
||||
|
||||
## v2.12.1
|
||||
|
||||
- [2026-11-10-category-unused-tags.md](v2.12.1/2026-11-10-category-unused-tags.md) - q=category lists only tags on posts that exist
|
||||
|
||||
## v2.13.0
|
||||
|
||||
- [2026-11-10-backfeed.md](v2.13.0/2026-11-10-backfeed.md) - Backfeed of responses to crossposts (ADR-073), avatar cache, delete tombstones
|
||||
|
||||
## v2.13.1
|
||||
|
||||
- [2026-11-10-avatar-fetch.md](v2.13.1/2026-11-10-avatar-fetch.md) - Avatars from the own GoToSocial server and from Bluesky PDSes
|
||||
|
||||
## v2.13.2
|
||||
|
||||
- [2026-11-10-reaction-counts.md](v2.13.2/2026-11-10-reaction-counts.md) - Likes/reposts as a counts line with hover lists; web actions
|
||||
|
||||
## v2.13.3
|
||||
|
||||
- [2026-11-10-count-opted-out.md](v2.13.3/2026-11-10-count-opted-out.md) - Opted-out people's likes and reposts counted without names
|
||||
|
||||
## v2.13.4
|
||||
|
||||
- [2026-11-10-list-counts.md](v2.13.4/2026-11-10-list-counts.md) - Counts on lists of notes; whole reply card links to the reply; class clash fixed
|
||||
|
||||
## hotfix
|
||||
|
||||
Not a release folder.
|
||||
|
||||
IndieAuth PKCE endpoint discovery hotfix (2025-12-17, between v1.5.0 and v1.6.0; no version of its own).
|
||||
|
||||
- [2025-12-17-indieauth-pkce-endpoint-discovery-hotfix-implementation-report.md](hotfix/2025-12-17-indieauth-pkce-endpoint-discovery-hotfix-implementation-report.md) — IndieAuth Endpoint Discovery Hotfix - Implementation Report
|
||||
- [2025-12-17-indieauth-pkce-endpoint-discovery-hotfix.md](hotfix/2025-12-17-indieauth-pkce-endpoint-discovery-hotfix.md) — IndieAuth Endpoint Discovery Hotfix
|
||||
|
||||
## Related Documentation
|
||||
- **[../architecture/](../architecture/)** - System architecture and overviews
|
||||
- **[../decisions/](../decisions/)** - Architectural Decision Records (ADRs)
|
||||
- **[../reports/](../reports/)** - Implementation reports
|
||||
- **[../standards/](../standards/)** - Coding standards and conventions
|
||||
|
||||
- [../architecture/](../architecture/) - Current system architecture
|
||||
- [../decisions/](../decisions/) - Architectural Decision Records (ADRs)
|
||||
- [../projectplan/](../projectplan/) - Backlog and release plans
|
||||
- [../standards/](../standards/) - Coding standards and conventions
|
||||
- [../../CHANGELOG.md](../../CHANGELOG.md) - User-facing release notes
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-11-25
|
||||
**Maintained By**: Documentation Manager Agent
|
||||
**Last Updated**: 2026-11-03
|
||||
+173
@@ -0,0 +1,173 @@
|
||||
# IndieAuth Endpoint Discovery Hotfix - Implementation Report
|
||||
|
||||
**Date:** 2025-12-17
|
||||
**Type:** Production Hotfix
|
||||
**Priority:** Critical
|
||||
**Status:** Implementation Complete
|
||||
|
||||
## Summary
|
||||
|
||||
Successfully implemented the IndieAuth endpoint discovery hotfix as specified in the design document. The authentication flow now correctly discovers endpoints from the user's profile URL per the W3C IndieAuth specification, instead of hardcoding the indielogin.com service.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
All implementation steps were completed successfully:
|
||||
|
||||
### Step 1: Update starpunk/config.py - Remove INDIELOGIN_URL
|
||||
- Removed `INDIELOGIN_URL` config line (previously line 37)
|
||||
- Added deprecation warning for users still setting `INDIELOGIN_URL` in environment
|
||||
- Warning directs users to remove the deprecated config
|
||||
|
||||
### Step 2: Update starpunk/auth.py - Add imports and use endpoint discovery
|
||||
- Added imports: `discover_endpoints`, `DiscoveryError`, `normalize_url` from `starpunk.auth_external`
|
||||
- Rewrote `initiate_login()`:
|
||||
- Now discovers authorization_endpoint from the user's profile URL
|
||||
- Uses discovered endpoint instead of hardcoded INDIELOGIN_URL
|
||||
- Raises DiscoveryError if endpoint discovery fails or no authorization_endpoint found
|
||||
- Rewrote `handle_callback()`:
|
||||
- Discovers authorization_endpoint from ADMIN_ME profile
|
||||
- Uses authorization_endpoint for authentication-only flow (per IndieAuth spec)
|
||||
- Does NOT include `grant_type` parameter (not needed for auth-only flows)
|
||||
- Uses `normalize_url()` for URL comparison to handle trailing slashes and case differences
|
||||
|
||||
### Step 3: Update starpunk/auth_external.py - Relax endpoint validation
|
||||
- Changed endpoint validation in `_fetch_and_parse()`:
|
||||
- Now requires at least one endpoint (authorization_endpoint OR token_endpoint)
|
||||
- Previously required token_endpoint to be present
|
||||
- This allows profiles with only authorization_endpoint to work for login
|
||||
- Micropub will still require token_endpoint and fail gracefully with 401
|
||||
|
||||
### Step 4: Update starpunk/routes/auth.py - Import and handle DiscoveryError
|
||||
- Added import for `DiscoveryError` from `starpunk.auth_external`
|
||||
- Added exception handler in `login_initiate()`:
|
||||
- Catches DiscoveryError
|
||||
- Logs technical details at ERROR level
|
||||
- Shows user-friendly message: "Unable to verify your profile URL. Please check that it's correct and try again."
|
||||
- Redirects back to login form
|
||||
|
||||
### Step 5: Update tests/test_auth.py - Mock discover_endpoints()
|
||||
- Removed `INDIELOGIN_URL` from test app fixture
|
||||
- Updated all tests that call `initiate_login()` or `handle_callback()`:
|
||||
- Added `@patch("starpunk.auth.discover_endpoints")` decorator
|
||||
- Mock returns both authorization_endpoint and token_endpoint
|
||||
- Updated assertions to check for discovered endpoint instead of indielogin.com
|
||||
- Tests updated:
|
||||
- `TestInitiateLogin.test_initiate_login_success`
|
||||
- `TestInitiateLogin.test_initiate_login_stores_state`
|
||||
- `TestHandleCallback.test_handle_callback_success`
|
||||
- `TestHandleCallback.test_handle_callback_unauthorized_user`
|
||||
- `TestHandleCallback.test_handle_callback_indielogin_error`
|
||||
- `TestHandleCallback.test_handle_callback_no_identity`
|
||||
- `TestLoggingIntegration.test_initiate_login_logs_at_debug`
|
||||
- `TestLoggingIntegration.test_initiate_login_info_level`
|
||||
- `TestLoggingIntegration.test_handle_callback_logs_http_details`
|
||||
|
||||
### Step 6: Update tests/test_auth_external.py - Fix error message
|
||||
- Updated `test_discover_endpoints_no_token_endpoint`:
|
||||
- Changed assertion from "No token endpoint found" to "No IndieAuth endpoints found"
|
||||
- Matches new relaxed validation error message
|
||||
|
||||
### Step 7: Run tests to verify implementation
|
||||
- All 51 tests in `tests/test_auth.py` pass
|
||||
- All 35 tests in `tests/test_auth_external.py` pass
|
||||
- All 32 tests in `tests/test_routes_admin.py` pass
|
||||
- No regressions detected
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Lines Changed | Description |
|
||||
|------|--------------|-------------|
|
||||
| `starpunk/config.py` | 9 added, 1 removed | Removed INDIELOGIN_URL, added deprecation warning |
|
||||
| `starpunk/auth.py` | 1 added, 84 replaced | Added imports, rewrote initiate_login() and handle_callback() |
|
||||
| `starpunk/auth_external.py` | 6 replaced | Relaxed endpoint validation |
|
||||
| `starpunk/routes/auth.py` | 5 added | Added DiscoveryError import and exception handling |
|
||||
| `tests/test_auth.py` | 1 removed, 43 modified | Removed INDIELOGIN_URL from fixture, added mocks |
|
||||
| `tests/test_auth_external.py` | 2 modified | Updated error message assertion |
|
||||
|
||||
## Key Implementation Details
|
||||
|
||||
### Authorization Endpoint Usage
|
||||
Per IndieAuth spec and architect clarifications:
|
||||
- Authentication-only flows POST to the **authorization_endpoint** (not token_endpoint)
|
||||
- The `grant_type` parameter is NOT included (only for access token flows)
|
||||
- This differs from the previous implementation which incorrectly used indielogin.com's endpoints
|
||||
|
||||
### URL Normalization
|
||||
The implementation now uses `normalize_url()` when comparing the returned 'me' URL with ADMIN_ME:
|
||||
- Handles trailing slash differences (https://example.com vs https://example.com/)
|
||||
- Handles case differences (https://Example.com vs https://example.com)
|
||||
- This is spec-compliant behavior that was previously missing
|
||||
|
||||
### Error Handling
|
||||
- Discovery failures are caught and logged at ERROR level
|
||||
- User-facing error message is simplified and friendly
|
||||
- Technical details remain in logs for debugging
|
||||
|
||||
### Backward Compatibility
|
||||
- Deprecation warning added for INDIELOGIN_URL environment variable
|
||||
- Existing .env files with INDIELOGIN_URL will log a warning but continue to work
|
||||
- Users are instructed to remove the deprecated config
|
||||
|
||||
## Testing Results
|
||||
|
||||
### Unit Tests
|
||||
- `tests/test_auth.py`: 51/51 passed
|
||||
- `tests/test_auth_external.py`: 35/35 passed
|
||||
- `tests/test_routes_admin.py`: 32/32 passed
|
||||
|
||||
### Integration
|
||||
All modified tests now correctly mock endpoint discovery and validate the new behavior.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
No significant issues encountered during implementation. The design document was thorough and all architect clarifications were addressed:
|
||||
|
||||
1. Import placement - Moved to top-level as specified
|
||||
2. URL normalization - Included as intentional bugfix
|
||||
3. Endpoint selection - Used authorization_endpoint for auth-only flows
|
||||
4. Validation relaxation - Allowed profiles with only authorization_endpoint
|
||||
5. Test strategy - Mocked discover_endpoints() and removed INDIELOGIN_URL
|
||||
6. grant_type parameter - Correctly omitted for auth-only flows
|
||||
7. Error messages - Simplified user-facing messages
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Manual testing recommended:
|
||||
- Test login flow with actual IndieAuth profile
|
||||
- Verify endpoint discovery logs appear
|
||||
- Test with profiles that have custom endpoints
|
||||
- Verify error message appears for profiles without endpoints
|
||||
|
||||
2. Deployment:
|
||||
- Update production .env to remove INDIELOGIN_URL (optional - will show warning)
|
||||
- Deploy changes
|
||||
- Monitor logs for "Discovered authorization_endpoint" messages
|
||||
- Verify login works for admin user
|
||||
|
||||
3. Documentation:
|
||||
- Update CHANGELOG.md with hotfix entry
|
||||
- Consider adding migration guide if needed
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
- [x] All specified files modified
|
||||
- [x] All code changes follow architect's design exactly
|
||||
- [x] Tests updated and passing
|
||||
- [x] Error messages user-friendly
|
||||
- [x] Logging appropriate for debugging
|
||||
- [x] URL normalization implemented
|
||||
- [x] Endpoint validation relaxed correctly
|
||||
- [x] No regressions in existing tests
|
||||
- [x] Implementation report created
|
||||
|
||||
## Conclusion
|
||||
|
||||
The hotfix has been successfully implemented according to the architect's design. The authentication flow now correctly implements the W3C IndieAuth specification for endpoint discovery. All tests pass and no regressions were detected.
|
||||
|
||||
The critical production bug preventing user login should be resolved once this code is deployed.
|
||||
|
||||
---
|
||||
|
||||
**Developer:** Claude (Fullstack Developer Subagent)
|
||||
**Date Completed:** 2025-12-17
|
||||
**Ready for Review:** Yes
|
||||
@@ -0,0 +1,608 @@
|
||||
# IndieAuth Endpoint Discovery Hotfix
|
||||
|
||||
**Date:** 2025-12-17
|
||||
**Type:** Production Hotfix
|
||||
**Priority:** Critical
|
||||
**Status:** Ready for Implementation
|
||||
|
||||
## Problem Summary
|
||||
|
||||
Users cannot log in to StarPunk. The root cause is that the authentication code ignores endpoint discovery and hardcodes `INDIELOGIN_URL` instead of discovering the authorization and token endpoints from the user's profile URL.
|
||||
|
||||
**Root Cause:** The `starpunk/auth.py` module uses `INDIELOGIN_URL` config instead of discovering endpoints from the user's profile URL as required by the IndieAuth specification. This is a regression - the system used to respect discovered endpoints.
|
||||
|
||||
**Note:** The PKCE error message in the callback is a symptom, not the cause. Once we use the correct discovered endpoints, PKCE will not be required (since the user's actual IndieAuth server doesn't require it).
|
||||
|
||||
## Specification Requirements
|
||||
|
||||
### W3C IndieAuth Spec (https://www.w3.org/TR/indieauth/)
|
||||
|
||||
- Clients MUST discover `authorization_endpoint` from user's profile URL
|
||||
- Clients MUST discover `token_endpoint` from user's profile URL
|
||||
- Discovery via HTTP Link headers (highest priority) or HTML `<link>` elements
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Overview
|
||||
|
||||
The fix reuses the existing `discover_endpoints()` function from `auth_external.py` in the login flow. Changes are minimal and focused:
|
||||
|
||||
1. Use `discover_endpoints()` in `initiate_login()` to get the `authorization_endpoint`
|
||||
2. Use `discover_endpoints()` in `handle_callback()` to get the `token_endpoint`
|
||||
3. Remove `INDIELOGIN_URL` config (with deprecation warning)
|
||||
|
||||
---
|
||||
|
||||
### Step 1: Update config.py - Remove INDIELOGIN_URL
|
||||
|
||||
In `/home/phil/Projects/starpunk/starpunk/config.py`:
|
||||
|
||||
**Change 1:** Remove the INDIELOGIN_URL config line (line 37):
|
||||
|
||||
```python
|
||||
# DELETE THIS LINE:
|
||||
app.config["INDIELOGIN_URL"] = os.getenv("INDIELOGIN_URL", "https://indielogin.com")
|
||||
```
|
||||
|
||||
**Change 2:** Add deprecation warning for INDIELOGIN_URL (add after the TOKEN_ENDPOINT warning, around line 47):
|
||||
|
||||
```python
|
||||
# DEPRECATED: INDIELOGIN_URL no longer used (hotfix 2025-12-17)
|
||||
# Authorization endpoint is now discovered from ADMIN_ME profile per IndieAuth spec
|
||||
if 'INDIELOGIN_URL' in os.environ:
|
||||
app.logger.warning(
|
||||
"INDIELOGIN_URL is deprecated and will be ignored. "
|
||||
"Remove it from your configuration. "
|
||||
"The authorization endpoint is now discovered automatically from your ADMIN_ME profile."
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Update auth.py - Use Endpoint Discovery
|
||||
|
||||
In `/home/phil/Projects/starpunk/starpunk/auth.py`:
|
||||
|
||||
**Change 1:** Add import for endpoint discovery (after line 42):
|
||||
|
||||
```python
|
||||
from starpunk.auth_external import discover_endpoints, DiscoveryError, normalize_url
|
||||
```
|
||||
|
||||
> **Note:** The `normalize_url` import is at the top level (not inside `handle_callback()`) for consistency with the existing code style.
|
||||
|
||||
**Change 2:** Update `initiate_login()` to use discovered authorization_endpoint (replace lines 251-318):
|
||||
|
||||
```python
|
||||
def initiate_login(me_url: str) -> str:
|
||||
"""
|
||||
Initiate IndieAuth authentication flow with endpoint discovery.
|
||||
|
||||
Per W3C IndieAuth spec, discovers authorization_endpoint from user's
|
||||
profile URL rather than using a hardcoded service.
|
||||
|
||||
Args:
|
||||
me_url: User's IndieWeb identity URL
|
||||
|
||||
Returns:
|
||||
Redirect URL to discovered authorization endpoint
|
||||
|
||||
Raises:
|
||||
ValueError: Invalid me_url format
|
||||
DiscoveryError: Failed to discover endpoints from profile
|
||||
"""
|
||||
# Validate URL format
|
||||
if not is_valid_url(me_url):
|
||||
raise ValueError(f"Invalid URL format: {me_url}")
|
||||
|
||||
current_app.logger.debug(f"Auth: Validating me URL: {me_url}")
|
||||
|
||||
# Discover authorization endpoint from user's profile URL
|
||||
# Per IndieAuth spec: clients MUST discover endpoints, not hardcode them
|
||||
try:
|
||||
endpoints = discover_endpoints(me_url)
|
||||
except DiscoveryError as e:
|
||||
current_app.logger.error(f"Auth: Endpoint discovery failed for {me_url}: {e}")
|
||||
raise
|
||||
|
||||
auth_endpoint = endpoints.get('authorization_endpoint')
|
||||
if not auth_endpoint:
|
||||
raise DiscoveryError(
|
||||
f"No authorization_endpoint found at {me_url}. "
|
||||
"Ensure your profile has IndieAuth link elements or headers."
|
||||
)
|
||||
|
||||
current_app.logger.info(f"Auth: Discovered authorization_endpoint: {auth_endpoint}")
|
||||
|
||||
# Generate CSRF state token
|
||||
state = _generate_state_token()
|
||||
current_app.logger.debug(f"Auth: Generated state token: {_redact_token(state, 8)}")
|
||||
|
||||
# Store state in database (5-minute expiry)
|
||||
db = get_db(current_app)
|
||||
expires_at = datetime.utcnow() + timedelta(minutes=5)
|
||||
redirect_uri = f"{current_app.config['SITE_URL']}auth/callback"
|
||||
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO auth_state (state, expires_at, redirect_uri)
|
||||
VALUES (?, ?, ?)
|
||||
""",
|
||||
(state, expires_at, redirect_uri),
|
||||
)
|
||||
db.commit()
|
||||
|
||||
# Build authorization URL
|
||||
params = {
|
||||
"me": me_url,
|
||||
"client_id": current_app.config["SITE_URL"],
|
||||
"redirect_uri": redirect_uri,
|
||||
"state": state,
|
||||
"response_type": "code",
|
||||
}
|
||||
|
||||
current_app.logger.debug(
|
||||
f"Auth: Building authorization URL with params:\n"
|
||||
f" me: {me_url}\n"
|
||||
f" client_id: {current_app.config['SITE_URL']}\n"
|
||||
f" redirect_uri: {redirect_uri}\n"
|
||||
f" state: {_redact_token(state, 8)}\n"
|
||||
f" response_type: code"
|
||||
)
|
||||
|
||||
auth_url = f"{auth_endpoint}?{urlencode(params)}"
|
||||
|
||||
current_app.logger.debug(f"Auth: Complete authorization URL: {auth_url}")
|
||||
current_app.logger.info(f"Auth: Authentication initiated for {me_url}")
|
||||
|
||||
return auth_url
|
||||
```
|
||||
|
||||
**Change 3:** Update `handle_callback()` to use discovered authorization_endpoint (replace lines 321-474):
|
||||
|
||||
> **Important:** Per IndieAuth spec, authentication-only flows (identity verification without access tokens) POST to the **authorization_endpoint**, NOT the token_endpoint. The `grant_type` parameter is NOT included for authentication-only flows.
|
||||
|
||||
```python
|
||||
def handle_callback(code: str, state: str, iss: Optional[str] = None) -> Optional[str]:
|
||||
"""
|
||||
Handle IndieAuth callback with endpoint discovery.
|
||||
|
||||
Discovers authorization_endpoint from ADMIN_ME profile and exchanges
|
||||
authorization code for identity verification.
|
||||
|
||||
Per IndieAuth spec: Authentication-only flows POST to the authorization
|
||||
endpoint (not token endpoint) and do not include grant_type.
|
||||
|
||||
Args:
|
||||
code: Authorization code from authorization server
|
||||
state: CSRF state token
|
||||
iss: Issuer identifier (optional, for security validation)
|
||||
|
||||
Returns:
|
||||
Session token if successful, None otherwise
|
||||
|
||||
Raises:
|
||||
InvalidStateError: State token validation failed
|
||||
UnauthorizedError: User not authorized as admin
|
||||
IndieLoginError: Code exchange failed
|
||||
"""
|
||||
current_app.logger.debug(f"Auth: Verifying state token: {_redact_token(state, 8)}")
|
||||
|
||||
# Verify state token (CSRF protection)
|
||||
if not _verify_state_token(state):
|
||||
current_app.logger.warning(
|
||||
"Auth: Invalid state token received (possible CSRF or expired token)"
|
||||
)
|
||||
raise InvalidStateError("Invalid or expired state token")
|
||||
|
||||
current_app.logger.debug("Auth: State token valid")
|
||||
|
||||
# Discover authorization endpoint from ADMIN_ME profile
|
||||
admin_me = current_app.config.get("ADMIN_ME")
|
||||
if not admin_me:
|
||||
current_app.logger.error("Auth: ADMIN_ME not configured")
|
||||
raise IndieLoginError("ADMIN_ME not configured")
|
||||
|
||||
try:
|
||||
endpoints = discover_endpoints(admin_me)
|
||||
except DiscoveryError as e:
|
||||
current_app.logger.error(f"Auth: Endpoint discovery failed: {e}")
|
||||
raise IndieLoginError(f"Failed to discover endpoints: {e}")
|
||||
|
||||
# Use authorization_endpoint for authentication-only flow (identity verification)
|
||||
# Per IndieAuth spec: auth-only flows POST to authorization_endpoint, not token_endpoint
|
||||
auth_endpoint = endpoints.get('authorization_endpoint')
|
||||
if not auth_endpoint:
|
||||
raise IndieLoginError(
|
||||
f"No authorization_endpoint found at {admin_me}. "
|
||||
"Ensure your profile has IndieAuth endpoints configured."
|
||||
)
|
||||
|
||||
current_app.logger.debug(f"Auth: Using authorization_endpoint: {auth_endpoint}")
|
||||
|
||||
# Verify issuer if provided (security check - optional)
|
||||
if iss:
|
||||
current_app.logger.debug(f"Auth: Issuer provided: {iss}")
|
||||
|
||||
# Prepare code verification request
|
||||
# Note: grant_type is NOT included for authentication-only flows per IndieAuth spec
|
||||
token_exchange_data = {
|
||||
"code": code,
|
||||
"client_id": current_app.config["SITE_URL"],
|
||||
"redirect_uri": f"{current_app.config['SITE_URL']}auth/callback",
|
||||
}
|
||||
|
||||
# Log the request
|
||||
_log_http_request(
|
||||
method="POST",
|
||||
url=auth_endpoint,
|
||||
data=token_exchange_data,
|
||||
)
|
||||
|
||||
current_app.logger.debug(
|
||||
"Auth: Sending code verification request to authorization endpoint:\n"
|
||||
" Method: POST\n"
|
||||
" URL: %s\n"
|
||||
" Data: code=%s, client_id=%s, redirect_uri=%s",
|
||||
auth_endpoint,
|
||||
_redact_token(code),
|
||||
token_exchange_data["client_id"],
|
||||
token_exchange_data["redirect_uri"],
|
||||
)
|
||||
|
||||
# Exchange code for identity at authorization endpoint (authentication-only flow)
|
||||
try:
|
||||
response = httpx.post(
|
||||
auth_endpoint,
|
||||
data=token_exchange_data,
|
||||
timeout=10.0,
|
||||
)
|
||||
|
||||
current_app.logger.debug(
|
||||
"Auth: Received code verification response:\n"
|
||||
" Status: %d\n"
|
||||
" Headers: %s\n"
|
||||
" Body: %s",
|
||||
response.status_code,
|
||||
{k: v for k, v in dict(response.headers).items()
|
||||
if k.lower() not in ["set-cookie", "authorization"]},
|
||||
_redact_token(response.text) if response.text else "(empty)",
|
||||
)
|
||||
|
||||
_log_http_response(
|
||||
status_code=response.status_code,
|
||||
headers=dict(response.headers),
|
||||
body=response.text,
|
||||
)
|
||||
|
||||
response.raise_for_status()
|
||||
except httpx.RequestError as e:
|
||||
current_app.logger.error(f"Auth: Authorization endpoint request failed: {e}")
|
||||
raise IndieLoginError(f"Failed to verify code: {e}")
|
||||
except httpx.HTTPStatusError as e:
|
||||
current_app.logger.error(
|
||||
f"Auth: Authorization endpoint returned error: {e.response.status_code} - {e.response.text}"
|
||||
)
|
||||
raise IndieLoginError(
|
||||
f"Authorization endpoint returned error: {e.response.status_code}"
|
||||
)
|
||||
|
||||
# Parse response
|
||||
try:
|
||||
data = response.json()
|
||||
except Exception as e:
|
||||
current_app.logger.error(f"Auth: Failed to parse authorization endpoint response: {e}")
|
||||
raise IndieLoginError("Invalid JSON response from authorization endpoint")
|
||||
|
||||
me = data.get("me")
|
||||
|
||||
if not me:
|
||||
current_app.logger.error("Auth: No identity returned from authorization endpoint")
|
||||
raise IndieLoginError("No identity returned from authorization endpoint")
|
||||
|
||||
current_app.logger.debug(f"Auth: Received identity: {me}")
|
||||
|
||||
# Verify this is the admin user
|
||||
current_app.logger.info(f"Auth: Verifying admin authorization for me={me}")
|
||||
|
||||
# Normalize URLs for comparison (handles trailing slashes and case differences)
|
||||
# This is correct per IndieAuth spec - the returned 'me' is the canonical form
|
||||
if normalize_url(me) != normalize_url(admin_me):
|
||||
current_app.logger.warning(
|
||||
f"Auth: Unauthorized login attempt: {me} (expected {admin_me})"
|
||||
)
|
||||
raise UnauthorizedError(f"User {me} is not authorized")
|
||||
|
||||
current_app.logger.debug("Auth: Admin verification passed")
|
||||
|
||||
# Create session
|
||||
session_token = create_session(me)
|
||||
|
||||
# Trigger author profile discovery (v1.2.0 Phase 2)
|
||||
# Per Q14: Never block login, always allow fallback
|
||||
try:
|
||||
from starpunk.author_discovery import get_author_profile
|
||||
author_profile = get_author_profile(me, refresh=True)
|
||||
current_app.logger.info(f"Author profile refreshed for {me}")
|
||||
except Exception as e:
|
||||
current_app.logger.warning(f"Author discovery failed: {e}")
|
||||
# Continue login anyway - never block per Q14
|
||||
|
||||
return session_token
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Update auth_external.py - Relax Endpoint Validation
|
||||
|
||||
The existing `_fetch_and_parse()` function requires `token_endpoint` to be present. We need to relax this since some profiles may only have `authorization_endpoint` (for authentication-only flows).
|
||||
|
||||
In `/home/phil/Projects/starpunk/starpunk/auth_external.py`, update the validation in `_fetch_and_parse()` (around lines 302-307):
|
||||
|
||||
**Change:** Make token_endpoint not strictly required (allow authentication-only profiles):
|
||||
|
||||
```python
|
||||
# Validate we found at least one endpoint
|
||||
# - authorization_endpoint: Required for authentication-only flows (admin login)
|
||||
# - token_endpoint: Required for Micropub token verification
|
||||
# Having at least one allows the appropriate flow to work
|
||||
if 'token_endpoint' not in endpoints and 'authorization_endpoint' not in endpoints:
|
||||
raise DiscoveryError(
|
||||
f"No IndieAuth endpoints found at {profile_url}. "
|
||||
"Ensure your profile has authorization_endpoint or token_endpoint configured."
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 4: Update routes/auth.py - Handle DiscoveryError
|
||||
|
||||
In `/home/phil/Projects/starpunk/starpunk/routes/auth.py`:
|
||||
|
||||
**Change 1:** Add import for DiscoveryError (update lines 20-29):
|
||||
|
||||
```python
|
||||
from starpunk.auth import (
|
||||
IndieLoginError,
|
||||
InvalidStateError,
|
||||
UnauthorizedError,
|
||||
destroy_session,
|
||||
handle_callback,
|
||||
initiate_login,
|
||||
require_auth,
|
||||
verify_session,
|
||||
)
|
||||
from starpunk.auth_external import DiscoveryError
|
||||
```
|
||||
|
||||
**Change 2:** Handle DiscoveryError in login_initiate() (update lines 79-85):
|
||||
|
||||
> **Note:** The user-facing error message is kept simple. Technical details are logged but not shown to users.
|
||||
|
||||
```python
|
||||
try:
|
||||
# Initiate IndieAuth flow
|
||||
auth_url = initiate_login(me_url)
|
||||
return redirect(auth_url)
|
||||
except ValueError as e:
|
||||
flash(str(e), "error")
|
||||
return redirect(url_for("auth.login_form"))
|
||||
except DiscoveryError as e:
|
||||
current_app.logger.error(f"Endpoint discovery failed for {me_url}: {e}")
|
||||
flash("Unable to verify your profile URL. Please check that it's correct and try again.", "error")
|
||||
return redirect(url_for("auth.login_form"))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Summary
|
||||
|
||||
| File | Change Type | Description |
|
||||
|------|-------------|-------------|
|
||||
| `starpunk/config.py` | Edit | Remove INDIELOGIN_URL, add deprecation warning |
|
||||
| `starpunk/auth.py` | Edit | Use endpoint discovery instead of hardcoded URL |
|
||||
| `starpunk/auth_external.py` | Edit | Relax endpoint validation (allow auth-only flow) |
|
||||
| `starpunk/routes/auth.py` | Edit | Handle DiscoveryError exception |
|
||||
|
||||
---
|
||||
|
||||
## Testing Requirements
|
||||
|
||||
### Manual Testing
|
||||
|
||||
1. **Login Flow Test**
|
||||
- Navigate to `/auth/login`
|
||||
- Enter ADMIN_ME URL
|
||||
- Verify redirect goes to discovered authorization_endpoint (not hardcoded indielogin.com)
|
||||
- Complete login and verify session is created
|
||||
|
||||
2. **Endpoint Discovery Test**
|
||||
- Test with profile that declares custom endpoints
|
||||
- Verify discovered endpoints are used, not defaults
|
||||
|
||||
### Existing Test Updates
|
||||
|
||||
**Update test fixture in `tests/test_auth.py`:**
|
||||
|
||||
Remove `INDIELOGIN_URL` from the app fixture (line 51):
|
||||
|
||||
```python
|
||||
@pytest.fixture
|
||||
def app(tmp_path):
|
||||
"""Create Flask app for testing"""
|
||||
from starpunk import create_app
|
||||
|
||||
test_data_dir = tmp_path / "data"
|
||||
test_data_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
app = create_app(
|
||||
{
|
||||
"TESTING": True,
|
||||
"SITE_URL": "http://localhost:5000/",
|
||||
"ADMIN_ME": "https://example.com",
|
||||
"SESSION_SECRET": secrets.token_hex(32),
|
||||
"SESSION_LIFETIME": 30,
|
||||
# REMOVED: "INDIELOGIN_URL": "https://indielogin.com",
|
||||
"DATA_PATH": test_data_dir,
|
||||
"NOTES_PATH": test_data_dir / "notes",
|
||||
"DATABASE_PATH": test_data_dir / "starpunk.db",
|
||||
}
|
||||
)
|
||||
return app
|
||||
```
|
||||
|
||||
**Update existing tests that use `httpx.post` mock:**
|
||||
|
||||
Tests in `TestInitiateLogin` and `TestHandleCallback` need to mock `discover_endpoints()` in addition to `httpx.post`. Example pattern:
|
||||
|
||||
```python
|
||||
@patch("starpunk.auth.discover_endpoints")
|
||||
@patch("starpunk.auth.httpx.post")
|
||||
def test_handle_callback_success(self, mock_post, mock_discover, app, db, client):
|
||||
"""Test successful callback handling"""
|
||||
# Mock endpoint discovery
|
||||
mock_discover.return_value = {
|
||||
'authorization_endpoint': 'https://auth.example.com/authorize',
|
||||
'token_endpoint': 'https://auth.example.com/token'
|
||||
}
|
||||
|
||||
# Rest of test remains the same...
|
||||
```
|
||||
|
||||
**Update `TestInitiateLogin.test_initiate_login_success`:**
|
||||
|
||||
The assertion checking for `indielogin.com` needs to change to check for the mocked endpoint:
|
||||
|
||||
```python
|
||||
@patch("starpunk.auth.discover_endpoints")
|
||||
def test_initiate_login_success(self, mock_discover, app, db):
|
||||
"""Test successful login initiation"""
|
||||
mock_discover.return_value = {
|
||||
'authorization_endpoint': 'https://auth.example.com/authorize',
|
||||
'token_endpoint': 'https://auth.example.com/token'
|
||||
}
|
||||
|
||||
with app.app_context():
|
||||
me_url = "https://example.com"
|
||||
auth_url = initiate_login(me_url)
|
||||
|
||||
# Changed: Check for discovered endpoint instead of indielogin.com
|
||||
assert "auth.example.com/authorize" in auth_url
|
||||
assert "me=https%3A%2F%2Fexample.com" in auth_url
|
||||
# ... rest of assertions
|
||||
```
|
||||
|
||||
### New Automated Tests to Add
|
||||
|
||||
```python
|
||||
# tests/test_auth_endpoint_discovery.py
|
||||
|
||||
def test_initiate_login_uses_endpoint_discovery(client, mocker):
|
||||
"""Verify login uses discovered endpoint, not hardcoded"""
|
||||
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
|
||||
mock_discover.return_value = {
|
||||
'authorization_endpoint': 'https://custom-auth.example.com/authorize',
|
||||
'token_endpoint': 'https://custom-auth.example.com/token'
|
||||
}
|
||||
|
||||
response = client.post('/auth/login', data={'me': 'https://example.com'})
|
||||
|
||||
assert response.status_code == 302
|
||||
assert 'custom-auth.example.com' in response.headers['Location']
|
||||
|
||||
|
||||
def test_callback_uses_discovered_authorization_endpoint(client, mocker):
|
||||
"""Verify callback uses discovered authorization endpoint (not token endpoint)"""
|
||||
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
|
||||
mock_discover.return_value = {
|
||||
'authorization_endpoint': 'https://custom-auth.example.com/authorize',
|
||||
'token_endpoint': 'https://custom-auth.example.com/token'
|
||||
}
|
||||
mock_post = mocker.patch('starpunk.auth.httpx.post')
|
||||
# Setup state token and mock httpx response
|
||||
# Verify code exchange POSTs to authorization_endpoint, not token_endpoint
|
||||
pass
|
||||
|
||||
|
||||
def test_discovery_error_shows_user_friendly_message(client, mocker):
|
||||
"""Verify discovery failures show helpful error"""
|
||||
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
|
||||
mock_discover.side_effect = DiscoveryError("No endpoints found")
|
||||
|
||||
response = client.post('/auth/login', data={'me': 'https://example.com'})
|
||||
|
||||
assert response.status_code == 302
|
||||
# Should redirect back to login form with flash message
|
||||
|
||||
|
||||
def test_url_normalization_handles_trailing_slash(app, mocker):
|
||||
"""Verify URL normalization allows trailing slash differences"""
|
||||
# ADMIN_ME without trailing slash, auth server returns with trailing slash
|
||||
# Should still authenticate successfully
|
||||
pass
|
||||
|
||||
|
||||
def test_url_normalization_handles_case_differences(app, mocker):
|
||||
"""Verify URL normalization is case-insensitive"""
|
||||
# ADMIN_ME: https://Example.com, auth server returns: https://example.com
|
||||
# Should still authenticate successfully
|
||||
pass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If issues occur after deployment:
|
||||
|
||||
1. **Code:** Revert to previous commit
|
||||
2. **Config:** Re-add INDIELOGIN_URL to .env if needed
|
||||
|
||||
---
|
||||
|
||||
## Post-Deployment Verification
|
||||
|
||||
1. Verify login works with the user's actual profile URL
|
||||
2. Check logs for "Discovered authorization_endpoint" message
|
||||
3. Test logout and re-login cycle
|
||||
|
||||
---
|
||||
|
||||
## Architect Q&A (2025-12-17)
|
||||
|
||||
Developer questions answered by the architect prior to implementation:
|
||||
|
||||
### Q1: Import Placement
|
||||
**Q:** Should `normalize_url` import be inside the function or at top level?
|
||||
**A:** Move to top level with other imports for consistency. The design has been updated.
|
||||
|
||||
### Q2: URL Normalization Behavior Change
|
||||
**Q:** Is the URL normalization change intentional?
|
||||
**A:** Yes, this is an intentional bugfix. The current exact-match behavior is incorrect per IndieAuth spec. URLs differing only in trailing slashes or case should be considered equivalent for identity purposes. The `normalize_url()` function already exists in `auth_external.py` and is used by `verify_external_token()`.
|
||||
|
||||
### Q3: Which Endpoint for Authentication Flow?
|
||||
**Q:** Should we use token_endpoint or authorization_endpoint?
|
||||
**A:** Use **authorization_endpoint** for authentication-only flows. Per IndieAuth spec: "the client makes a POST request to the authorization endpoint to verify the authorization code and retrieve the final user profile URL." The design has been corrected.
|
||||
|
||||
### Q4: Endpoint Validation Relaxation
|
||||
**Q:** Is relaxed endpoint validation acceptable?
|
||||
**A:** Yes. Login requires `authorization_endpoint`, Micropub requires `token_endpoint`. Requiring at least one is correct. If only auth endpoint exists, login works but Micropub fails gracefully (401).
|
||||
|
||||
### Q5: Test Update Strategy
|
||||
**Q:** Remove INDIELOGIN_URL and/or mock discover_endpoints()?
|
||||
**A:** Both. Remove `INDIELOGIN_URL` from fixtures, add `discover_endpoints()` mocking to existing tests. Detailed guidance added to Testing Requirements section.
|
||||
|
||||
### Q6: grant_type Parameter
|
||||
**Q:** Should we include grant_type in the code exchange?
|
||||
**A:** No. Authentication-only flows do not include `grant_type`. This parameter is only required when POSTing to the token_endpoint for access tokens. The design has been corrected.
|
||||
|
||||
### Q7: Error Message Verbosity
|
||||
**Q:** Should we simplify the user-facing error message?
|
||||
**A:** Yes. User-facing message should be simple: "Unable to verify your profile URL. Please check that it's correct and try again." Technical details are logged at ERROR level. The design has been updated.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- W3C IndieAuth Specification: https://www.w3.org/TR/indieauth/
|
||||
- IndieAuth Endpoint Discovery: https://www.w3.org/TR/indieauth/#discovery-by-clients
|
||||
@@ -630,11 +630,11 @@ curl http://localhost:8000/health
|
||||
|
||||
### Documentation
|
||||
|
||||
- [Phase 5 Design](../designs/phase-5-rss-and-container.md)
|
||||
- [Containerfile](../../Containerfile)
|
||||
- [Compose Configuration](../../compose.yaml)
|
||||
- [Caddy Example](../../Caddyfile.example)
|
||||
- [Nginx Example](../../nginx.conf.example)
|
||||
- [Phase 5 Design](phase-5-rss-and-container.md)
|
||||
- [Containerfile](../../../Containerfile)
|
||||
- [Compose Configuration](../../../compose.yaml)
|
||||
- [Caddy Example](../../../Caddyfile.example)
|
||||
- [Nginx Example](../../../nginx.conf.example)
|
||||
|
||||
### External Resources
|
||||
|
||||
|
||||
@@ -21,12 +21,12 @@ This document provides a comprehensive, dependency-ordered implementation plan f
|
||||
|-------|--------|---------|---------------|--------|
|
||||
| 1.1 - Core Utilities | ✅ Complete | 0.1.0 | >90% | N/A |
|
||||
| 1.2 - Data Models | ✅ Complete | 0.1.0 | >90% | N/A |
|
||||
| 2.1 - Notes Management | ✅ Complete | 0.3.0 | 86% (85 tests) | [Phase 2.1 Report](/home/phil/Projects/starpunk/docs/reports/phase-2.1-implementation-20251118.md) |
|
||||
| 3.1 - Authentication | ✅ Complete | 0.8.0 | 96% (51 tests) | [Phase 3 Report](/home/phil/Projects/starpunk/docs/reports/phase-3-authentication-20251118.md) |
|
||||
| 2.1 - Notes Management | ✅ Complete | 0.3.0 | 86% (85 tests) | [Phase 2.1 Report](phase-2.1-implementation-20251118.md) |
|
||||
| 3.1 - Authentication | ✅ Complete | 0.8.0 | 96% (51 tests) | [Phase 3 Report](phase-3-authentication-20251118.md) |
|
||||
| 4.1-4.4 - Web Interface | ✅ Complete | 0.5.2 | 87% (405 tests) | Phase 4 implementation |
|
||||
| 5.1-5.2 - RSS Feed | ✅ Complete | 0.6.0 | 96% | ADR-014, ADR-015 |
|
||||
| 6 - Micropub | ✅ Complete | 1.0.0 | 95% | [v1.0.0 Release](/home/phil/Projects/starpunk/docs/reports/v1.0.0-implementation-report.md) |
|
||||
| V1.1 - Search & Enhancements | ✅ Complete | 1.1.0 | 598 tests | [v1.1.0 Report](/home/phil/Projects/starpunk/docs/reports/v1.1.0-implementation-report.md) |
|
||||
| 6 - Micropub | ✅ Complete | 1.0.0 | 95% | v1.0.0 Release (`v1.0.0-implementation-report.md`, not preserved) |
|
||||
| V1.1 - Search & Enhancements | ✅ Complete | 1.1.0 | 598 tests | [v1.1.0 Report](../v1.1.1/v1.1.0-implementation-report.md) |
|
||||
|
||||
### Current Status 🔵
|
||||
|
||||
@@ -1553,29 +1553,29 @@ Final steps before V1 release.
|
||||
## References
|
||||
|
||||
### Project Documentation
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
|
||||
- [Technology Stack](/home/phil/Projects/starpunk/docs/architecture/technology-stack.md)
|
||||
- [ADR-001: Python Web Framework](/home/phil/Projects/starpunk/docs/decisions/ADR-001-python-web-framework.md)
|
||||
- [ADR-002: Flask Extensions](/home/phil/Projects/starpunk/docs/decisions/ADR-002-flask-extensions.md)
|
||||
- [ADR-003: Frontend Technology](/home/phil/Projects/starpunk/docs/decisions/ADR-003-frontend-technology.md)
|
||||
- [ADR-004: File-Based Storage](/home/phil/Projects/starpunk/docs/decisions/ADR-004-file-based-note-storage.md)
|
||||
- [ADR-005: IndieLogin Authentication](/home/phil/Projects/starpunk/docs/decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-006: Python Virtual Environment](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md)
|
||||
- [ADR-007: Slug Generation Algorithm](/home/phil/Projects/starpunk/docs/decisions/ADR-007-slug-generation-algorithm.md)
|
||||
- [ADR-008: Versioning Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-008-versioning-strategy.md)
|
||||
- [ADR-009: Git Branching Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-009-git-branching-strategy.md)
|
||||
- [ADR-010: Authentication Module Design](/home/phil/Projects/starpunk/docs/decisions/ADR-010-authentication-module-design.md)
|
||||
- [ADR-011: Development Authentication Mechanism](/home/phil/Projects/starpunk/docs/decisions/ADR-011-development-authentication-mechanism.md)
|
||||
- [Project Structure](/home/phil/Projects/starpunk/docs/design/project-structure.md)
|
||||
- [Phase 4 Web Interface Design](/home/phil/Projects/starpunk/docs/design/phase-4-web-interface.md)
|
||||
- [Python Coding Standards](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
|
||||
- [Versioning Strategy](/home/phil/Projects/starpunk/docs/standards/versioning-strategy.md)
|
||||
- [Git Branching Strategy](/home/phil/Projects/starpunk/docs/standards/git-branching-strategy.md)
|
||||
- [Architecture Overview](../../architecture/overview.md)
|
||||
- [Technology Stack](../../architecture/technology-stack.md)
|
||||
- [ADR-001: Python Web Framework](../../decisions/ADR-001-python-web-framework.md)
|
||||
- [ADR-002: Flask Extensions](../../decisions/ADR-002-flask-extensions.md)
|
||||
- [ADR-003: Frontend Technology](../../decisions/ADR-003-frontend-technology.md)
|
||||
- [ADR-004: File-Based Storage](../../decisions/ADR-004-file-based-note-storage.md)
|
||||
- [ADR-005: IndieLogin Authentication](../../decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-006: Python Virtual Environment](../../decisions/ADR-006-python-virtual-environment-uv.md)
|
||||
- [ADR-007: Slug Generation Algorithm](../../decisions/ADR-007-slug-generation-algorithm.md)
|
||||
- [ADR-008: Versioning Strategy](../../decisions/ADR-008-versioning-strategy.md)
|
||||
- [ADR-009: Git Branching Strategy](../../decisions/ADR-009-git-branching-strategy.md)
|
||||
- [ADR-010: Authentication Module Design](../../decisions/ADR-010-authentication-module-design.md)
|
||||
- [ADR-011: Development Authentication Mechanism](../../decisions/ADR-011-development-authentication-mechanism.md)
|
||||
- [Project Structure](project-structure.md)
|
||||
- [Phase 4 Web Interface Design](phase-4-web-interface.md)
|
||||
- [Python Coding Standards](../../standards/python-coding-standards.md)
|
||||
- [Versioning Strategy](../../standards/versioning-strategy.md)
|
||||
- [Git Branching Strategy](../../standards/git-branching-strategy.md)
|
||||
|
||||
### Implementation Reports
|
||||
- [Phase 2.1 Implementation Report](/home/phil/Projects/starpunk/docs/reports/phase-2.1-implementation-20251118.md)
|
||||
- [Phase 3 Authentication Report](/home/phil/Projects/starpunk/docs/reports/phase-3-authentication-20251118.md)
|
||||
- [Phase 4 Architectural Assessment](/home/phil/Projects/starpunk/docs/reports/phase-4-architectural-assessment-20251118.md)
|
||||
- [Phase 2.1 Implementation Report](phase-2.1-implementation-20251118.md)
|
||||
- [Phase 3 Authentication Report](phase-3-authentication-20251118.md)
|
||||
- [Phase 4 Architectural Assessment](phase-4-architectural-assessment-20251118.md)
|
||||
|
||||
### External Standards
|
||||
- [Micropub Specification](https://micropub.spec.indieweb.org/)
|
||||
|
||||
@@ -333,8 +333,8 @@ This allows gradual migration without breaking existing integrations.
|
||||
## References
|
||||
|
||||
### Architect Documentation
|
||||
- [ADR-016: IndieAuth Client Discovery Mechanism](/home/phil/Projects/starpunk/docs/decisions/ADR-016-indieauth-client-discovery.md)
|
||||
- [IndieAuth Client Discovery Analysis Report](/home/phil/Projects/starpunk/docs/reports/indieauth-client-discovery-analysis.md)
|
||||
- [ADR-016: IndieAuth Client Discovery Mechanism](../../decisions/ADR-016-indieauth-client-discovery.md)
|
||||
- [IndieAuth Client Discovery Analysis Report](indieauth-client-discovery-analysis.md)
|
||||
|
||||
### IndieWeb Standards
|
||||
- [IndieAuth Specification](https://www.w3.org/TR/indieauth/)
|
||||
@@ -342,9 +342,9 @@ This allows gradual migration without breaking existing integrations.
|
||||
- [IndieLogin.com](https://indielogin.com/)
|
||||
|
||||
### Project Documentation
|
||||
- [ADR-008: Versioning Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-008-versioning-strategy.md)
|
||||
- [ADR-009: Git Branching Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-009-git-branching-strategy.md)
|
||||
- [Phase 3: Authentication Design](/home/phil/Projects/starpunk/docs/design/phase-3-authentication.md)
|
||||
- [ADR-008: Versioning Strategy](../../decisions/ADR-008-versioning-strategy.md)
|
||||
- [ADR-009: Git Branching Strategy](../../decisions/ADR-009-git-branching-strategy.md)
|
||||
- [Phase 3: Authentication Design](phase-3-authentication.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -376,6 +376,6 @@ The implementation exactly follows the architect's specification in ADR-018, use
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [ADR-018: IndieAuth Detailed Logging Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-018-indieauth-detailed-logging.md)
|
||||
- [Versioning Strategy](/home/phil/Projects/starpunk/docs/standards/versioning-strategy.md)
|
||||
- [CHANGELOG.md](/home/phil/Projects/starpunk/CHANGELOG.md)
|
||||
- [ADR-018: IndieAuth Detailed Logging Strategy](../../decisions/ADR-018-indieauth-detailed-logging.md)
|
||||
- [Versioning Strategy](../../standards/versioning-strategy.md)
|
||||
- [CHANGELOG.md](../../../CHANGELOG.md)
|
||||
@@ -1009,9 +1009,9 @@ echo "✓ Sensitive files are gitignored"
|
||||
|
||||
## References
|
||||
|
||||
- [Project Structure Design](/home/phil/Projects/starpunk/docs/design/project-structure.md)
|
||||
- [Development Setup Standards](/home/phil/Projects/starpunk/docs/standards/development-setup.md)
|
||||
- [ADR-002: Flask Extensions](/home/phil/Projects/starpunk/docs/decisions/ADR-002-flask-extensions.md)
|
||||
- [ADR-006: Python Virtual Environment](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md)
|
||||
- [Project Structure Design](project-structure.md)
|
||||
- [Development Setup Standards](../../standards/development-setup.md)
|
||||
- [ADR-002: Flask Extensions](../../decisions/ADR-002-flask-extensions.md)
|
||||
- [ADR-006: Python Virtual Environment](../../decisions/ADR-006-python-virtual-environment-uv.md)
|
||||
- [Python Packaging Guide](https://packaging.python.org/)
|
||||
- [Flask Configuration Handling](https://flask.palletsprojects.com/en/3.0.x/config/)
|
||||
@@ -328,7 +328,7 @@ This client_id is not registered (https://starpunk.thesatelliteoflove.com)
|
||||
- [IANA Well-Known URIs](https://www.iana.org/assignments/well-known-uris/)
|
||||
|
||||
### Project
|
||||
- [ADR-017: OAuth Client ID Metadata Document Implementation](../decisions/ADR-017-oauth-client-metadata-document.md)
|
||||
- [ADR-017: OAuth Client ID Metadata Document Implementation](../../decisions/ADR-017-oauth-client-metadata-document.md)
|
||||
- [IndieAuth Fix Summary](indieauth-fix-summary.md)
|
||||
- [Root Cause Analysis](indieauth-client-discovery-root-cause-analysis.md)
|
||||
|
||||
|
||||
@@ -1365,9 +1365,9 @@ def extract_title(content: str) -> Optional[str]:
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-004: File-Based Note Storage](/home/phil/Projects/starpunk/docs/decisions/ADR-004-file-based-note-storage.md)
|
||||
- [Python Coding Standards](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
|
||||
- [Project Structure](/home/phil/Projects/starpunk/docs/design/project-structure.md)
|
||||
- [ADR-004: File-Based Note Storage](../../decisions/ADR-004-file-based-note-storage.md)
|
||||
- [Python Coding Standards](../../standards/python-coding-standards.md)
|
||||
- [Project Structure](project-structure.md)
|
||||
- [CommonMark Specification](https://spec.commonmark.org/)
|
||||
- [RSS 2.0 Specification](https://www.rssboard.org/rss-specification)
|
||||
- [ISO 8601 Date Format](https://en.wikipedia.org/wiki/ISO_8601)
|
||||
|
||||
@@ -1900,10 +1900,10 @@ class Webmention:
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-004: File-Based Note Storage](/home/phil/Projects/starpunk/docs/decisions/ADR-004-file-based-note-storage.md)
|
||||
- [Database Schema](/home/phil/Projects/starpunk/starpunk/database.py)
|
||||
- [Phase 1.1: Core Utilities](/home/phil/Projects/starpunk/docs/design/phase-1.1-core-utilities.md)
|
||||
- [Python Coding Standards](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
|
||||
- [ADR-004: File-Based Note Storage](../../decisions/ADR-004-file-based-note-storage.md)
|
||||
- [Database Schema](../../../starpunk/monitoring/database.py)
|
||||
- [Phase 1.1: Core Utilities](phase-1.1-core-utilities.md)
|
||||
- [Python Coding Standards](../../standards/python-coding-standards.md)
|
||||
- [Python Dataclasses Documentation](https://docs.python.org/3/library/dataclasses.html)
|
||||
- [Python-Markdown Documentation](https://python-markdown.github.io/)
|
||||
|
||||
|
||||
@@ -2057,14 +2057,14 @@ Phase 2.1 is complete when:
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-004: File-Based Note Storage](/home/phil/Projects/starpunk/docs/decisions/ADR-004-file-based-note-storage.md)
|
||||
- [ADR-007: Slug Generation Algorithm](/home/phil/Projects/starpunk/docs/decisions/ADR-007-slug-generation-algorithm.md)
|
||||
- [Phase 1.1: Core Utilities](/home/phil/Projects/starpunk/docs/design/phase-1.1-core-utilities.md)
|
||||
- [Phase 1.2: Data Models](/home/phil/Projects/starpunk/docs/design/phase-1.2-data-models.md)
|
||||
- [Database Schema](/home/phil/Projects/starpunk/starpunk/database.py)
|
||||
- [Python Coding Standards](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
|
||||
- [Security Architecture](/home/phil/Projects/starpunk/docs/architecture/security.md)
|
||||
- [ADR-004: File-Based Note Storage](../../decisions/ADR-004-file-based-note-storage.md)
|
||||
- [ADR-007: Slug Generation Algorithm](../../decisions/ADR-007-slug-generation-algorithm.md)
|
||||
- [Phase 1.1: Core Utilities](phase-1.1-core-utilities.md)
|
||||
- [Phase 1.2: Data Models](phase-1.2-data-models.md)
|
||||
- [Database Schema](../../../starpunk/monitoring/database.py)
|
||||
- [Python Coding Standards](../../standards/python-coding-standards.md)
|
||||
- [Architecture Overview](../../architecture/overview.md)
|
||||
- Security Architecture (`security.md`, not preserved)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -607,10 +607,10 @@ After completing Phase 2.1:
|
||||
|
||||
## Help & Resources
|
||||
|
||||
- **Full Design Doc**: [phase-2.1-notes-management.md](/home/phil/Projects/starpunk/docs/design/phase-2.1-notes-management.md)
|
||||
- **Utilities Design**: [phase-1.1-core-utilities.md](/home/phil/Projects/starpunk/docs/design/phase-1.1-core-utilities.md)
|
||||
- **Models Design**: [phase-1.2-data-models.md](/home/phil/Projects/starpunk/docs/design/phase-1.2-data-models.md)
|
||||
- **Coding Standards**: [python-coding-standards.md](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
|
||||
- **Implementation Plan**: [implementation-plan.md](/home/phil/Projects/starpunk/docs/projectplan/v1/implementation-plan.md)
|
||||
- **Full Design Doc**: [phase-2.1-notes-management.md](phase-2.1-notes-management.md)
|
||||
- **Utilities Design**: [phase-1.1-core-utilities.md](phase-1.1-core-utilities.md)
|
||||
- **Models Design**: [phase-1.2-data-models.md](phase-1.2-data-models.md)
|
||||
- **Coding Standards**: [python-coding-standards.md](../../standards/python-coding-standards.md)
|
||||
- **Implementation Plan**: [implementation-plan.md](implementation-plan.md)
|
||||
|
||||
Remember: "Every line of code must justify its existence. When in doubt, leave it out."
|
||||
@@ -532,8 +532,8 @@ After Phase 3 completion:
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-005: IndieLogin Authentication](/home/phil/Projects/starpunk/docs/decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-010: Authentication Module Design](/home/phil/Projects/starpunk/docs/decisions/ADR-010-authentication-module-design.md)
|
||||
- [ADR-005: IndieLogin Authentication](../../decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-010: Authentication Module Design](../../decisions/ADR-010-authentication-module-design.md)
|
||||
- [IndieAuth Specification](https://www.w3.org/TR/indieauth/)
|
||||
- [IndieLogin API Documentation](https://indielogin.com/api)
|
||||
- [OWASP Authentication Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html)
|
||||
|
||||
@@ -1292,12 +1292,12 @@ Deferred features:
|
||||
## References
|
||||
|
||||
### Internal Documentation
|
||||
- [ADR-003: Frontend Technology](/home/phil/Projects/starpunk/docs/decisions/ADR-003-frontend-technology.md)
|
||||
- [ADR-005: IndieLogin Authentication](/home/phil/Projects/starpunk/docs/decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-010: Authentication Module Design](/home/phil/Projects/starpunk/docs/decisions/ADR-010-authentication-module-design.md)
|
||||
- [ADR-011: Development Authentication Mechanism](/home/phil/Projects/starpunk/docs/decisions/ADR-011-development-authentication-mechanism.md)
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
|
||||
- [Phase 3 Report](/home/phil/Projects/starpunk/docs/reports/phase-3-authentication-20251118.md)
|
||||
- [ADR-003: Frontend Technology](../../decisions/ADR-003-frontend-technology.md)
|
||||
- [ADR-005: IndieLogin Authentication](../../decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-010: Authentication Module Design](../../decisions/ADR-010-authentication-module-design.md)
|
||||
- [ADR-011: Development Authentication Mechanism](../../decisions/ADR-011-development-authentication-mechanism.md)
|
||||
- [Architecture Overview](../../architecture/overview.md)
|
||||
- [Phase 3 Report](phase-3-authentication-20251118.md)
|
||||
|
||||
### External Standards
|
||||
- [Microformats2 h-entry](http://microformats.org/wiki/h-entry)
|
||||
|
||||
@@ -382,10 +382,10 @@ The codebase is in excellent condition, all prerequisites are met, and comprehen
|
||||
## Quick Access Links
|
||||
|
||||
**Primary Documents**:
|
||||
- [Full Design Document](/home/phil/Projects/starpunk/docs/designs/phase-5-rss-and-container.md)
|
||||
- [Quick Reference Guide](/home/phil/Projects/starpunk/docs/designs/phase-5-quick-reference.md)
|
||||
- [ADR-014: RSS Implementation](/home/phil/Projects/starpunk/docs/decisions/ADR-014-rss-feed-implementation.md)
|
||||
- [Pre-Implementation Review](/home/phil/Projects/starpunk/docs/reports/phase-5-pre-implementation-review.md)
|
||||
- [Full Design Document](phase-5-rss-and-container.md)
|
||||
- [Quick Reference Guide](phase-5-quick-reference.md)
|
||||
- [ADR-014: RSS Implementation](../../decisions/ADR-014-rss-feed-implementation.md)
|
||||
- [Pre-Implementation Review](phase-5-pre-implementation-review.md)
|
||||
|
||||
**Standards References**:
|
||||
- [RSS 2.0 Specification](https://www.rssboard.org/rss-specification)
|
||||
@@ -393,8 +393,8 @@ The codebase is in excellent condition, all prerequisites are met, and comprehen
|
||||
- [Podman Documentation](https://docs.podman.io/)
|
||||
|
||||
**Project Standards**:
|
||||
- [Versioning Strategy](/home/phil/Projects/starpunk/docs/standards/versioning-strategy.md)
|
||||
- [Git Branching Strategy](/home/phil/Projects/starpunk/docs/standards/git-branching-strategy.md)
|
||||
- [Versioning Strategy](../../standards/versioning-strategy.md)
|
||||
- [Git Branching Strategy](../../standards/git-branching-strategy.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -421,10 +421,10 @@ Phase 5 complete when:
|
||||
|
||||
## Reference Documents
|
||||
|
||||
- [Phase 5 Full Design](/home/phil/Projects/starpunk/docs/designs/phase-5-rss-and-container.md)
|
||||
- [ADR-014: RSS Implementation](/home/phil/Projects/starpunk/docs/decisions/ADR-014-rss-feed-implementation.md)
|
||||
- [Versioning Strategy](/home/phil/Projects/starpunk/docs/standards/versioning-strategy.md)
|
||||
- [Git Branching Strategy](/home/phil/Projects/starpunk/docs/standards/git-branching-strategy.md)
|
||||
- [Phase 5 Full Design](phase-5-rss-and-container.md)
|
||||
- [ADR-014: RSS Implementation](../../decisions/ADR-014-rss-feed-implementation.md)
|
||||
- [Versioning Strategy](../../standards/versioning-strategy.md)
|
||||
- [Git Branching Strategy](../../standards/git-branching-strategy.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1236,10 +1236,10 @@ Deferred features:
|
||||
## References
|
||||
|
||||
### Internal Documentation
|
||||
- [Phase 4: Web Interface](/home/phil/Projects/starpunk/docs/design/phase-4-web-interface.md)
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
|
||||
- [ADR-002: Flask Extensions](/home/phil/Projects/starpunk/docs/decisions/ADR-002-flask-extensions.md)
|
||||
- [Versioning Strategy](/home/phil/Projects/starpunk/docs/standards/versioning-strategy.md)
|
||||
- [Phase 4: Web Interface](phase-4-web-interface.md)
|
||||
- [Architecture Overview](../../architecture/overview.md)
|
||||
- [ADR-002: Flask Extensions](../../decisions/ADR-002-flask-extensions.md)
|
||||
- [Versioning Strategy](../../standards/versioning-strategy.md)
|
||||
|
||||
### External Standards
|
||||
- [RSS 2.0 Specification](https://www.rssboard.org/rss-specification)
|
||||
|
||||
@@ -337,7 +337,7 @@ tests/
|
||||
|
||||
**Purpose**: Architecture, decisions, designs, and standards.
|
||||
|
||||
**Organization**: See [Documentation Organization Standard](/home/phil/Projects/starpunk/docs/standards/documentation-organization.md)
|
||||
**Organization**: See [Documentation Organization Standard](../../standards/documentation-organization.md)
|
||||
|
||||
```
|
||||
docs/
|
||||
@@ -790,6 +790,6 @@ After creating project structure, verify:
|
||||
- [Flask Project Layout](https://flask.palletsprojects.com/en/3.0.x/tutorial/layout/)
|
||||
- [Pytest Good Practices](https://docs.pytest.org/en/stable/goodpractices.html)
|
||||
- [PEP 8 Package Layout](https://peps.python.org/pep-0008/)
|
||||
- [ADR-004: File-Based Note Storage](/home/phil/Projects/starpunk/docs/decisions/ADR-004-file-based-note-storage.md)
|
||||
- [ADR-006: Python Virtual Environment](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md)
|
||||
- [Documentation Organization Standard](/home/phil/Projects/starpunk/docs/standards/documentation-organization.md)
|
||||
- [ADR-004: File-Based Note Storage](../../decisions/ADR-004-file-based-note-storage.md)
|
||||
- [ADR-006: Python Virtual Environment](../../decisions/ADR-006-python-virtual-environment-uv.md)
|
||||
- [Documentation Organization Standard](../../standards/documentation-organization.md)
|
||||
@@ -318,8 +318,8 @@ Quick lookup for architectural decisions:
|
||||
|
||||
### Documentation
|
||||
- [Full Implementation Plan](implementation-plan.md)
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
|
||||
- [Technology Stack](/home/phil/Projects/starpunk/docs/architecture/technology-stack.md)
|
||||
- [Architecture Overview](../../architecture/overview.md)
|
||||
- [Technology Stack](../../architecture/technology-stack.md)
|
||||
|
||||
### External Specs
|
||||
- [Micropub Spec](https://micropub.spec.indieweb.org/)
|
||||
|
||||
@@ -200,18 +200,18 @@ All validators must pass before V1 release.
|
||||
## Reference Documentation
|
||||
|
||||
### Project Documentation
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
|
||||
- [Technology Stack](/home/phil/Projects/starpunk/docs/architecture/technology-stack.md)
|
||||
- [Project Structure](/home/phil/Projects/starpunk/docs/design/project-structure.md)
|
||||
- [Python Coding Standards](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
|
||||
- [Architecture Overview](../../architecture/overview.md)
|
||||
- [Technology Stack](../../architecture/technology-stack.md)
|
||||
- [Project Structure](project-structure.md)
|
||||
- [Python Coding Standards](../../standards/python-coding-standards.md)
|
||||
|
||||
### Architecture Decision Records (ADRs)
|
||||
- [ADR-001: Python Web Framework](/home/phil/Projects/starpunk/docs/decisions/ADR-001-python-web-framework.md)
|
||||
- [ADR-002: Flask Extensions](/home/phil/Projects/starpunk/docs/decisions/ADR-002-flask-extensions.md)
|
||||
- [ADR-003: Frontend Technology](/home/phil/Projects/starpunk/docs/decisions/ADR-003-frontend-technology.md)
|
||||
- [ADR-004: File-Based Storage](/home/phil/Projects/starpunk/docs/decisions/ADR-004-file-based-note-storage.md)
|
||||
- [ADR-005: IndieLogin Authentication](/home/phil/Projects/starpunk/docs/decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-006: Python Virtual Environment](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md)
|
||||
- [ADR-001: Python Web Framework](../../decisions/ADR-001-python-web-framework.md)
|
||||
- [ADR-002: Flask Extensions](../../decisions/ADR-002-flask-extensions.md)
|
||||
- [ADR-003: Frontend Technology](../../decisions/ADR-003-frontend-technology.md)
|
||||
- [ADR-004: File-Based Storage](../../decisions/ADR-004-file-based-note-storage.md)
|
||||
- [ADR-005: IndieLogin Authentication](../../decisions/ADR-005-indielogin-authentication.md)
|
||||
- [ADR-006: Python Virtual Environment](../../decisions/ADR-006-python-virtual-environment-uv.md)
|
||||
|
||||
### External Standards
|
||||
- [Micropub Specification](https://micropub.spec.indieweb.org/)
|
||||
|
||||
@@ -6,8 +6,8 @@ This document tracks features that were considered for StarPunk V1.0 but have be
|
||||
|
||||
**Status**: Planning / Future Consideration
|
||||
**Related Documents**:
|
||||
- [V1.0 Implementation Plan](/home/phil/Projects/starpunk/docs/projectplan/v1/implementation-plan.md)
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
|
||||
- [V1.0 Implementation Plan](../v1.0.0/implementation-plan.md)
|
||||
- [Architecture Overview](../../architecture/overview.md)
|
||||
|
||||
## Feature Categories
|
||||
|
||||
@@ -424,8 +424,8 @@ When planning V1.1, features should be evaluated using:
|
||||
|
||||
## References
|
||||
|
||||
- [V1.0 Implementation Plan](/home/phil/Projects/starpunk/docs/projectplan/v1/implementation-plan.md)
|
||||
- [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
|
||||
- [V1.0 Implementation Plan](../v1.0.0/implementation-plan.md)
|
||||
- [Architecture Overview](../../architecture/overview.md)
|
||||
- [IndieWeb Principles](https://indieweb.org/principles)
|
||||
- [Micropub Specification](https://www.w3.org/TR/micropub/)
|
||||
- [Webmention Specification](https://www.w3.org/TR/webmention/)
|
||||
|
||||
@@ -208,10 +208,10 @@ Before releasing v1.1.0:
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-031: Database Migration System Redesign](/home/phil/Projects/starpunk/docs/decisions/ADR-031-database-migration-system-redesign.md)
|
||||
- [ADR-032: Initial Schema SQL Implementation](/home/phil/Projects/starpunk/docs/decisions/ADR-032-initial-schema-sql-implementation.md)
|
||||
- [v1.1 Potential Features](/home/phil/Projects/starpunk/docs/projectplan/v1.1/potential-features.md)
|
||||
- [Migration Implementation Reports](/home/phil/Projects/starpunk/docs/reports/)
|
||||
- [ADR-031: Database Migration System Redesign](../../decisions/ADR-031-database-migration-system-redesign.md)
|
||||
- [ADR-032: Initial Schema SQL Implementation](../../decisions/ADR-032-initial-schema-sql-implementation.md)
|
||||
- [v1.1 Potential Features](potential-features.md)
|
||||
- Migration Implementation Reports (`reports`, not preserved)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -368,11 +368,11 @@ Benchmarks
|
||||
|
||||
## Documentation References
|
||||
|
||||
- [Configuration System](/home/phil/Projects/starpunk/docs/decisions/ADR-052-configuration-system-architecture.md)
|
||||
- [Performance Monitoring](/home/phil/Projects/starpunk/docs/decisions/ADR-053-performance-monitoring-strategy.md)
|
||||
- [Structured Logging](/home/phil/Projects/starpunk/docs/decisions/ADR-054-structured-logging-architecture.md)
|
||||
- [Error Handling](/home/phil/Projects/starpunk/docs/decisions/ADR-055-error-handling-philosophy.md)
|
||||
- [Implementation Guide](/home/phil/Projects/starpunk/docs/design/v1.1.1/implementation-guide.md)
|
||||
- [Configuration System](../../decisions/ADR-052-configuration-system-architecture.md)
|
||||
- [Performance Monitoring](../../decisions/ADR-053-performance-monitoring-strategy.md)
|
||||
- [Structured Logging](../../decisions/ADR-054-structured-logging-architecture.md)
|
||||
- [Error Handling](../../decisions/ADR-055-error-handling-philosophy.md)
|
||||
- Implementation Guide (`implementation-guide.md`, not preserved)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1652,7 +1652,7 @@ This pattern is used in:
|
||||
- [W3C Micropub Specification](https://www.w3.org/TR/micropub/)
|
||||
- [Media RSS Specification](https://www.rssboard.org/media-rss)
|
||||
- [JSON Feed 1.1 Specification](https://jsonfeed.org/version/1.1)
|
||||
- [ADR-057: Media Attachment Model](/home/phil/Projects/starpunk/docs/decisions/ADR-057-media-attachment-model.md)
|
||||
- [ADR-058: Image Optimization Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-058-image-optimization-strategy.md)
|
||||
- [ADR-059: Full Feed Media Standardization](/home/phil/Projects/starpunk/docs/decisions/ADR-059-full-feed-media-standardization.md)
|
||||
- [v1.4.0 Release Definition](/home/phil/Projects/starpunk/docs/projectplan/v1.4.0/RELEASE.md)
|
||||
- [ADR-057: Media Attachment Model](../../decisions/ADR-057-media-attachment-model.md)
|
||||
- [ADR-058: Image Optimization Strategy](../../decisions/ADR-058-image-optimization-strategy.md)
|
||||
- [ADR-059: Full Feed Media Standardization](../../decisions/ADR-059-full-feed-media-standardization.md)
|
||||
- [v1.4.0 Release Definition](../../projectplan/v1.4.0/RELEASE.md)
|
||||
@@ -0,0 +1,527 @@
|
||||
# Architect Review: v1.4.2 Implementation and End-to-End Media Pipeline
|
||||
|
||||
**Date**: 2025-12-16
|
||||
**Architect**: Claude (StarPunk Architect Agent)
|
||||
**Scope**: v1.4.2 HEIC support implementation + comprehensive media pipeline review
|
||||
**Status**: Complete
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The v1.4.2 implementation is architecturally sound and follows the established design patterns. The HEIC support was implemented cleanly with minimal code changes. However, the comprehensive end-to-end media pipeline review reveals several architectural concerns that should be addressed in future releases, ranging from security hardening to consistency improvements.
|
||||
|
||||
**Overall Assessment**: Acceptable with recommendations for future improvement.
|
||||
|
||||
---
|
||||
|
||||
## Part 1: v1.4.2 Implementation Review
|
||||
|
||||
### 1.1 Design Decisions Assessment
|
||||
|
||||
| Decision | Assessment | Notes |
|
||||
|----------|------------|-------|
|
||||
| D1: Convert at validation time | **Acceptable** | Keeps change minimal; conversion in `validate_image()` is logical since it normalizes input formats |
|
||||
| D2: Convert to JPEG | **Acceptable** | JPEG has universal browser support; appropriate for photographic content |
|
||||
| D3: Graceful degradation | **Good** | Conditional import with `HEIC_SUPPORTED` flag enables runtime flexibility |
|
||||
| D4: Quality 95 for conversion | **Acceptable** | High quality preserved; subsequent optimization will reduce if needed |
|
||||
| D5: Return signature change | **Acceptable** | 4-tuple return is clean; API change is internal-only |
|
||||
|
||||
### 1.2 Issues Found in v1.4.2
|
||||
|
||||
#### Issue 1.2.1: Module-Level Documentation Stale (LOW)
|
||||
|
||||
**Location**: `/home/phil/Projects/starpunk/starpunk/media.py`, lines 1-12
|
||||
|
||||
**Problem**: The module docstring still states "4096x4096 max dimensions" but `MAX_DIMENSION` was updated to 12000 in v1.4.2.
|
||||
|
||||
**Impact**: Documentation mismatch causes confusion; could lead to incorrect assumptions by future developers.
|
||||
|
||||
**Recommendation**: Update docstring to reflect current limits:
|
||||
```
|
||||
- 12000x12000 max input dimensions (v1.4.2)
|
||||
```
|
||||
|
||||
#### Issue 1.2.2: Debug File Storage Without Cleanup (MEDIUM)
|
||||
|
||||
**Location**: `/home/phil/Projects/starpunk/starpunk/media.py`, lines 133-137
|
||||
|
||||
**Problem**: Failed uploads are saved to `data/debug/` directory for analysis, but there is no mechanism to clean up these files. Over time, this could consume significant disk space, especially if under attack.
|
||||
|
||||
```python
|
||||
debug_dir = Path(current_app.config.get('DATA_PATH', 'data')) / 'debug'
|
||||
debug_dir.mkdir(parents=True, exist_ok=True)
|
||||
debug_file = debug_dir / f"failed_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{filename}"
|
||||
debug_file.write_bytes(file_data)
|
||||
```
|
||||
|
||||
**Impact**:
|
||||
- Disk space exhaustion risk
|
||||
- Potential storage of malicious payloads
|
||||
- No visibility into debug file accumulation
|
||||
|
||||
**Recommendation**:
|
||||
1. Add a configuration option to enable/disable debug file saving
|
||||
2. Implement automatic cleanup (e.g., files older than 7 days)
|
||||
3. Add disk space check before saving
|
||||
4. Consider rate limiting debug file creation
|
||||
|
||||
#### Issue 1.2.3: Filename in Debug Path Not Sanitized (MEDIUM)
|
||||
|
||||
**Location**: `/home/phil/Projects/starpunk/starpunk/media.py`, line 135
|
||||
|
||||
**Problem**: The original filename is used directly in the debug file path without sanitization.
|
||||
|
||||
```python
|
||||
debug_file = debug_dir / f"failed_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{filename}"
|
||||
```
|
||||
|
||||
**Impact**: Path traversal or special character issues could occur if filename contains malicious patterns (though `pathlib` provides some protection).
|
||||
|
||||
**Recommendation**: Sanitize filename before use:
|
||||
```python
|
||||
safe_filename = "".join(c for c in filename if c.isalnum() or c in "._-")[:50]
|
||||
```
|
||||
|
||||
#### Issue 1.2.4: Explicit HEIC Read After Pillow Failure (LOW)
|
||||
|
||||
**Location**: `/home/phil/Projects/starpunk/starpunk/media.py`, lines 111-139
|
||||
|
||||
**Problem**: When Pillow fails to open a file, the code attempts to read it explicitly as HEIC. This is a workaround for iOS files with wrong extensions, but the error handling is complex and could be clearer.
|
||||
|
||||
**Impact**: Code complexity; potential for subtle bugs in error paths.
|
||||
|
||||
**Recommendation**: Consider refactoring to a cleaner pattern:
|
||||
```
|
||||
1. Try Pillow standard open
|
||||
2. If fails and HEIC_SUPPORTED, try explicit HEIC
|
||||
3. If both fail, provide clear diagnostic
|
||||
```
|
||||
|
||||
### 1.3 v1.4.2 Architecture Compliance
|
||||
|
||||
| Principle | Compliance | Notes |
|
||||
|-----------|------------|-------|
|
||||
| Minimal code | **Yes** | 41 lines added for significant functionality |
|
||||
| Standards first | **Yes** | HEIC conversion preserves IndieWeb compatibility |
|
||||
| No lock-in | **Yes** | JPEG output is universal |
|
||||
| Single responsibility | **Yes** | Validation handles input normalization |
|
||||
| Documentation | **Partial** | Design doc complete; module docstring stale |
|
||||
|
||||
---
|
||||
|
||||
## Part 2: End-to-End Media Pipeline Review
|
||||
|
||||
### 2.1 Architecture Diagram
|
||||
|
||||
```
|
||||
MEDIA PIPELINE ARCHITECTURE
|
||||
===========================
|
||||
|
||||
USER UPLOAD FEED DELIVERY
|
||||
=========== ============
|
||||
|
||||
+-------------------+
|
||||
Admin UI | |
|
||||
/admin/new ---------> | validate_image | ----+
|
||||
POST multipart | | |
|
||||
+-------------------+ |
|
||||
|
|
||||
+-------------------+ | +------------------+
|
||||
Micropub Endpoint | | | | |
|
||||
/media POST --------> | save_media |<----+---->| SQLite DB |
|
||||
multipart/form-data | | | - media |
|
||||
+-------------------+ | - note_media |
|
||||
| | - media_vars |
|
||||
v +------------------+
|
||||
+-------------------+ ^
|
||||
| | |
|
||||
| optimize_image | |
|
||||
| (resize, compress)| |
|
||||
+-------------------+ |
|
||||
| |
|
||||
v |
|
||||
+-------------------+ |
|
||||
| | |
|
||||
| generate_variants |--------------------+
|
||||
| (thumb/sm/md/lg) |
|
||||
+-------------------+
|
||||
|
|
||||
v
|
||||
+-------------------+
|
||||
| |
|
||||
| FILESYSTEM |
|
||||
| data/media/ |
|
||||
| YYYY/MM/uuid.ext|
|
||||
+-------------------+
|
||||
|
|
||||
|
|
||||
+----------------------------+----------------------------+
|
||||
| | |
|
||||
v v v
|
||||
+--------+ +--------+ +--------+
|
||||
| RSS | | ATOM | | JSON |
|
||||
| /feed | | /feed | | /feed |
|
||||
| .xml | | .atom | | .json |
|
||||
+--------+ +--------+ +--------+
|
||||
| | |
|
||||
+----------------------------+----------------------------+
|
||||
|
|
||||
v
|
||||
+-------------------+
|
||||
| |
|
||||
| /media/<path> |
|
||||
| Static serving |
|
||||
+-------------------+
|
||||
```
|
||||
|
||||
### 2.2 Stage-by-Stage Analysis
|
||||
|
||||
#### Stage 1: Upload Entry Points
|
||||
|
||||
**Admin Upload (`/home/phil/Projects/starpunk/starpunk/routes/admin.py`, lines 112-136)**
|
||||
|
||||
| Aspect | Assessment | Notes |
|
||||
|--------|------------|-------|
|
||||
| Authentication | **Good** | `@require_auth` decorator enforced |
|
||||
| Input validation | **Partial** | Relies on `save_media`; no pre-check on file count |
|
||||
| Error handling | **Good** | Per-file errors collected; partial success allowed |
|
||||
| Content-Type check | **Missing** | No verification of `multipart/form-data` |
|
||||
|
||||
**Issues**:
|
||||
- No maximum file count enforced at route level (relies on downstream check)
|
||||
- `request.files.getlist('media_files')` could be empty list with misleading behavior
|
||||
|
||||
**Micropub Upload (`/home/phil/Projects/starpunk/starpunk/routes/micropub.py`, lines 124-202)**
|
||||
|
||||
| Aspect | Assessment | Notes |
|
||||
|--------|------------|-------|
|
||||
| Authentication | **Good** | Token extraction and verification |
|
||||
| Scope check | **Good** | Requires `create` scope |
|
||||
| Content-Type check | **Good** | Explicit `multipart/form-data` verification |
|
||||
| Input validation | **Good** | Single file field validated |
|
||||
| Error handling | **Good** | OAuth 2.0 error format |
|
||||
|
||||
**Issues**:
|
||||
- None significant; well-implemented endpoint
|
||||
|
||||
#### Stage 2: Validation (`validate_image`)
|
||||
|
||||
**Location**: `/home/phil/Projects/starpunk/starpunk/media.py`, lines 79-219
|
||||
|
||||
| Check | Order | Assessment |
|
||||
|-------|-------|------------|
|
||||
| File size | 1st | **Good** - Early rejection before processing |
|
||||
| Pillow verify | 2nd | **Good** - Validates image integrity |
|
||||
| HEIC fallback | 3rd | **Acceptable** - Complex but necessary |
|
||||
| Format conversion | 4th | **Good** - HEIC/MPO to JPEG |
|
||||
| MIME type check | 5th | **Good** - Whitelist approach |
|
||||
| Dimension check | 6th | **Good** - Prevents memory issues |
|
||||
| Animated GIF check | 7th | **Good** - Special handling for animations |
|
||||
|
||||
**Issues Found**:
|
||||
|
||||
##### Issue 2.2.1: No Content-Type Header Validation (MEDIUM)
|
||||
|
||||
**Problem**: The upload content-type from the HTTP request is not validated against the detected image format.
|
||||
|
||||
**Impact**: Potential for MIME type confusion attacks. A file uploaded as `image/png` could actually be a JPEG.
|
||||
|
||||
**Recommendation**: Log warning when declared MIME type differs from detected format.
|
||||
|
||||
##### Issue 2.2.2: Missing WebP Animation Detection (LOW)
|
||||
|
||||
**Problem**: Animated GIF detection exists but animated WebP is not handled.
|
||||
|
||||
**Impact**: Large animated WebP files could bypass the animated image size check.
|
||||
|
||||
**Recommendation**: Add animated WebP detection similar to GIF handling.
|
||||
|
||||
##### Issue 2.2.3: Pillow Decompression Bomb Protection (LOW)
|
||||
|
||||
**Problem**: No explicit `Image.MAX_IMAGE_PIXELS` configuration.
|
||||
|
||||
**Impact**: Pillow has default protection, but explicit configuration documents intent.
|
||||
|
||||
**Recommendation**: Add explicit `Image.MAX_IMAGE_PIXELS` setting or document reliance on default.
|
||||
|
||||
#### Stage 3: Optimization (`optimize_image`)
|
||||
|
||||
**Location**: `/home/phil/Projects/starpunk/starpunk/media.py`, lines 222-304
|
||||
|
||||
| Aspect | Assessment | Notes |
|
||||
|--------|------------|-------|
|
||||
| Tiered strategy | **Good** | Size-aware quality/dimension selection |
|
||||
| EXIF handling | **Good** | Orientation correction applied |
|
||||
| GIF passthrough | **Good** | Animated GIFs preserved |
|
||||
| Iterative reduction | **Good** | Quality then dimension reduction |
|
||||
| Safety limits | **Good** | MIN_DIMENSION prevents infinite loop |
|
||||
|
||||
**Issues Found**:
|
||||
|
||||
##### Issue 2.2.4: PNG Optimization Limited (LOW)
|
||||
|
||||
**Problem**: PNG files only get `optimize=True` flag; no palette reduction or other optimizations.
|
||||
|
||||
**Impact**: Large PNG files may not compress well.
|
||||
|
||||
**Recommendation**: Consider PNG-specific optimization (pngquant integration) for future release.
|
||||
|
||||
##### Issue 2.2.5: No WebP Quality Handling (LOW)
|
||||
|
||||
**Problem**: WebP gets same quality treatment as JPEG but WebP quality values behave differently.
|
||||
|
||||
**Impact**: WebP files may not be optimized as expected.
|
||||
|
||||
**Recommendation**: Consider WebP-specific quality mapping.
|
||||
|
||||
#### Stage 4: Variant Generation (`generate_all_variants`)
|
||||
|
||||
**Location**: `/home/phil/Projects/starpunk/starpunk/media.py`, lines 306-452
|
||||
|
||||
| Aspect | Assessment | Notes |
|
||||
|--------|------------|-------|
|
||||
| Spec definitions | **Good** | Clear variant specifications |
|
||||
| Skip logic | **Good** | Smaller images skip larger variants |
|
||||
| Cleanup on failure | **Good** | Created files removed on error |
|
||||
| Database integration | **Good** | Variants recorded with dimensions |
|
||||
|
||||
**Issues Found**:
|
||||
|
||||
##### Issue 2.2.6: Transaction Not Atomic (MEDIUM)
|
||||
|
||||
**Problem**: Files are created on disk before database commit. If database fails, files remain orphaned (cleanup only happens on exception within the loop).
|
||||
|
||||
```python
|
||||
try:
|
||||
for variant_type in ['thumb', 'small', 'medium', 'large']:
|
||||
variant = generate_variant(...) # File created
|
||||
variants.append(variant)
|
||||
created_files.append(...)
|
||||
db.execute(...) # DB insert
|
||||
|
||||
db.execute(...) # Original variant
|
||||
db.commit() # <-- If this fails, files already exist
|
||||
```
|
||||
|
||||
**Impact**: Database failure after file creation could leave orphaned files.
|
||||
|
||||
**Recommendation**: Consider writing to temp location first, then moving after commit.
|
||||
|
||||
##### Issue 2.2.7: Variant Path Calculation Fragile (LOW)
|
||||
|
||||
**Problem**: Line 363 calculates relative path using parent traversal:
|
||||
```python
|
||||
'path': str(variant_path.relative_to(base_path.parent.parent))
|
||||
```
|
||||
|
||||
**Impact**: Dependent on directory structure assumptions.
|
||||
|
||||
**Recommendation**: Use explicit path construction as done for original (lines 428-429).
|
||||
|
||||
#### Stage 5: Storage and Serving
|
||||
|
||||
**Location**: `/home/phil/Projects/starpunk/starpunk/routes/public.py`, lines 174-221
|
||||
|
||||
| Aspect | Assessment | Notes |
|
||||
|--------|------------|-------|
|
||||
| Path traversal protection | **Good** | Resolve and prefix check |
|
||||
| Cache headers | **Good** | 1 year immutable cache |
|
||||
| File serving | **Good** | Uses Flask's `send_from_directory` |
|
||||
|
||||
**Issues Found**:
|
||||
|
||||
##### Issue 2.2.8: Symlink Following (LOW)
|
||||
|
||||
**Problem**: `resolve()` follows symlinks, which could potentially escape the media directory if symlinks exist within.
|
||||
|
||||
**Impact**: Low risk since symlinks would need to be created manually.
|
||||
|
||||
**Recommendation**: Add `strict=True` to resolve for Python 3.6+ or check `is_symlink()`.
|
||||
|
||||
#### Stage 6: Feed Integration
|
||||
|
||||
**Locations**:
|
||||
- `/home/phil/Projects/starpunk/starpunk/feeds/rss.py`
|
||||
- `/home/phil/Projects/starpunk/starpunk/routes/public.py` (`_get_cached_notes`)
|
||||
|
||||
| Aspect | Assessment | Notes |
|
||||
|--------|------------|-------|
|
||||
| Media attachment | **Good** | Media loaded and attached to notes |
|
||||
| URL construction | **Good** | Consistent absolute URLs |
|
||||
| Media RSS | **Good** | Proper namespace and elements |
|
||||
| Enclosure element | **Good** | First image for RSS 2.0 spec |
|
||||
| Variant selection | **Good** | Fallback order for default variant |
|
||||
|
||||
**Issues Found**:
|
||||
|
||||
##### Issue 2.2.9: N+1 Query Pattern in Feed Generation (MEDIUM)
|
||||
|
||||
**Problem**: In `_get_cached_notes()`, media and tags are loaded per-note in loops:
|
||||
|
||||
```python
|
||||
for note in notes:
|
||||
media = get_note_media(note.id) # DB query per note
|
||||
object.__setattr__(note, 'media', media)
|
||||
tags = get_note_tags(note.id) # DB query per note
|
||||
```
|
||||
|
||||
**Impact**: For 50 notes, this is 100 additional queries. Performance degrades with more notes.
|
||||
|
||||
**Recommendation**: Implement batch loading:
|
||||
```python
|
||||
def get_media_for_notes(note_ids: List[int]) -> Dict[int, List[Dict]]:
|
||||
# Single query with WHERE note_id IN (...)
|
||||
```
|
||||
|
||||
##### Issue 2.2.10: Caption Not Escaped in RSS (LOW)
|
||||
|
||||
**Problem**: In RSS generation, caption is used directly in alt attribute:
|
||||
```python
|
||||
html_content += f'<img src="{media_url}" alt="{caption}" />'
|
||||
```
|
||||
|
||||
**Impact**: If caption contains `"` or other HTML special characters, could break markup.
|
||||
|
||||
**Recommendation**: Use `html.escape()` for caption in HTML context.
|
||||
|
||||
### 2.3 Security Assessment
|
||||
|
||||
| Category | Status | Notes |
|
||||
|----------|--------|-------|
|
||||
| **Authentication** | **Good** | Admin routes protected; Micropub uses token auth |
|
||||
| **Authorization** | **Good** | Scope checking on Micropub |
|
||||
| **File Type Validation** | **Good** | Whitelist + Pillow verification |
|
||||
| **Path Traversal** | **Good** | Protected in media serving |
|
||||
| **File Size Limits** | **Good** | 50MB upload, 10MB output |
|
||||
| **Dimension Limits** | **Good** | 12000px max input |
|
||||
| **Filename Handling** | **Good** | UUID-based storage filenames |
|
||||
| **Debug File Exposure** | **Needs Attention** | Debug files may contain malicious content |
|
||||
| **DoS Protection** | **Partial** | Limits exist but no rate limiting |
|
||||
|
||||
**Security Recommendations**:
|
||||
|
||||
1. **Add rate limiting** on media upload endpoints (medium priority)
|
||||
2. **Disable debug file saving** in production or add access controls (medium priority)
|
||||
3. **Log MIME type mismatches** for security monitoring (low priority)
|
||||
4. **Consider Content-Security-Policy** headers for served media (low priority)
|
||||
|
||||
### 2.4 Performance Assessment
|
||||
|
||||
| Operation | Assessment | Notes |
|
||||
|-----------|------------|-------|
|
||||
| Upload processing | **Acceptable** | In-memory processing; synchronous variant generation |
|
||||
| Feed generation | **Needs Improvement** | N+1 query pattern |
|
||||
| Media serving | **Good** | Static file serving with long cache |
|
||||
| Caching | **Good** | Feed caching with ETag support |
|
||||
|
||||
**Performance Recommendations**:
|
||||
|
||||
1. **Implement batch media loading** for feeds (high priority)
|
||||
2. **Consider async variant generation** for large uploads (low priority)
|
||||
3. **Add database query logging** in development mode (low priority)
|
||||
|
||||
---
|
||||
|
||||
## Part 3: Recommendations Summary
|
||||
|
||||
### 3.1 Immediate Fixes (None Critical)
|
||||
|
||||
No blocking issues found. The implementation is production-ready.
|
||||
|
||||
### 3.2 Future Improvements
|
||||
|
||||
#### High Priority
|
||||
|
||||
| Item | Description | Effort |
|
||||
|------|-------------|--------|
|
||||
| N+1 Query Fix | Batch load media/tags for feeds | Small |
|
||||
| Debug File Controls | Config option + cleanup mechanism | Small |
|
||||
|
||||
#### Medium Priority
|
||||
|
||||
| Item | Description | Effort |
|
||||
|------|-------------|--------|
|
||||
| Rate Limiting | Add upload rate limits | Medium |
|
||||
| Caption Escaping | HTML escape in feed generation | Small |
|
||||
| Filename Sanitization | Sanitize debug filenames | Small |
|
||||
| Transaction Atomicity | Temp files before commit | Medium |
|
||||
|
||||
#### Low Priority
|
||||
|
||||
| Item | Description | Effort |
|
||||
|------|-------------|--------|
|
||||
| WebP Animation | Detect animated WebP | Small |
|
||||
| PNG Optimization | Enhanced PNG compression | Medium |
|
||||
| Decompression Bomb Config | Explicit pixel limit | Trivial |
|
||||
| Symlink Handling | Stricter path resolution | Small |
|
||||
| Module Docstring | Update documentation | Trivial |
|
||||
|
||||
### 3.3 Technical Debt Register
|
||||
|
||||
| Item | Location | Notes |
|
||||
|------|----------|-------|
|
||||
| Complex HEIC fallback logic | `media.py:111-139` | Works but could be cleaner |
|
||||
| Variant path calculation | `media.py:363` | Uses parent traversal |
|
||||
| Debug file accumulation | `media.py:133-137` | No cleanup mechanism |
|
||||
| N+1 queries in feed | `public.py:68-74` | Performance impact |
|
||||
|
||||
---
|
||||
|
||||
## Part 4: Architectural Observations
|
||||
|
||||
### 4.1 Strengths
|
||||
|
||||
1. **Clean separation of concerns**: Validation, optimization, variant generation are distinct functions
|
||||
2. **Defensive programming**: Extensive try/except blocks with proper cleanup
|
||||
3. **Standards compliance**: Good adherence to IndieWeb, RSS, and W3C specs
|
||||
4. **Logging**: Comprehensive logging at INFO and WARNING levels
|
||||
5. **Backward compatibility**: Variants are optional; pre-v1.4.0 media works
|
||||
|
||||
### 4.2 Design Pattern Compliance
|
||||
|
||||
| Pattern | Usage | Assessment |
|
||||
|---------|-------|------------|
|
||||
| Route Adapter | Feed generation | **Good** |
|
||||
| Graceful Degradation | HEIC support | **Good** |
|
||||
| Early Rejection | File size check | **Good** |
|
||||
| Iterative Optimization | Quality reduction | **Good** |
|
||||
| UUID-based Storage | Collision avoidance | **Good** |
|
||||
|
||||
### 4.3 Areas for Future ADRs
|
||||
|
||||
1. **Media Processing Strategy**: Consider documenting the full media pipeline as an ADR
|
||||
2. **Debug/Diagnostic Data Handling**: Policy for storing failed uploads
|
||||
3. **Performance Targets**: Document expected query counts and response times
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
The v1.4.2 implementation successfully addresses the iPhone HEIC upload issue with minimal, clean changes. The broader media pipeline is well-architected with appropriate security controls and error handling. The main areas for improvement are:
|
||||
|
||||
1. **Performance**: N+1 query pattern in feed generation should be addressed
|
||||
2. **Operations**: Debug file management needs cleanup mechanism
|
||||
3. **Security**: Rate limiting would harden upload endpoints
|
||||
|
||||
None of these issues block the v1.4.2 release. They should be tracked in the project backlog for future releases.
|
||||
|
||||
**Recommendation**: Accept v1.4.2 as implemented. Create backlog items for identified improvements.
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Files Reviewed
|
||||
|
||||
| File | Lines | Purpose |
|
||||
|------|-------|---------|
|
||||
| `starpunk/media.py` | 775 | Core media handling |
|
||||
| `starpunk/routes/admin.py` | 603 | Admin upload endpoint |
|
||||
| `starpunk/routes/micropub.py` | 203 | Micropub media endpoint |
|
||||
| `starpunk/routes/public.py` | 567 | Media serving, feeds |
|
||||
| `starpunk/feeds/rss.py` | 601 | RSS feed with media |
|
||||
| `migrations/007_add_media_support.sql` | 38 | Media schema |
|
||||
| `migrations/009_add_media_variants.sql` | 22 | Variant schema |
|
||||
| `docs/decisions/ADR-057-media-attachment-model.md` | 110 | Media architecture |
|
||||
| `docs/decisions/ADR-058-image-optimization-strategy.md` | 183 | Optimization strategy |
|
||||
| `docs/design/v1.4.2/heic-support-design.md` | 220 | HEIC design spec |
|
||||
| `docs/design/v1.4.2/2025-12-16-implementation-report.md` | 171 | Implementation report |
|
||||
@@ -0,0 +1,360 @@
|
||||
# v1.4.2 Developer Review - HEIC/MPO Support and Dimension Limit Increase
|
||||
|
||||
**Date**: 2025-12-16
|
||||
**Reviewer**: Claude (Fullstack Developer Agent - Review Mode)
|
||||
**Implementation Report**: `/home/phil/Projects/starpunk/docs/design/v1.4.2/2025-12-16-implementation-report.md`
|
||||
**Design Document**: `/home/phil/Projects/starpunk/docs/design/v1.4.2/heic-support-design.md`
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The v1.4.2 implementation successfully addresses iPhone photo upload issues through HEIC/MPO format support and increased dimension limits. The implementation is **production-ready** with **minor recommendations** for improvements. All core functionality works correctly, tests pass, and backward compatibility is maintained.
|
||||
|
||||
**Overall Assessment**: APPROVED with recommendations for follow-up enhancements
|
||||
|
||||
## Implementation Summary
|
||||
|
||||
### Changes Delivered
|
||||
|
||||
1. **HEIC/HEIF Support** - Automatic detection and conversion to JPEG
|
||||
2. **MPO Support** - Multi-Picture Object format handling for iPhone depth/portrait photos
|
||||
3. **Dimension Limit Increase** - From 4096px to 12000px (supports 48MP+ cameras)
|
||||
4. **Enhanced Error Handling** - Fallback HEIC detection, debug logging, and file saving
|
||||
5. **Graceful Degradation** - Works without pillow-heif library (with clear errors)
|
||||
|
||||
### Files Modified
|
||||
|
||||
- `starpunk/media.py` - Core media handling logic
|
||||
- `starpunk/config.py` - Added MAX_CONTENT_LENGTH
|
||||
- `requirements.txt` - Added pillow-heif dependency, bumped Pillow version
|
||||
- `tests/test_media_upload.py` - Added HEIC test coverage
|
||||
- `CHANGELOG.md` - Release notes
|
||||
- `starpunk/__init__.py` - Version bump to 1.4.2
|
||||
|
||||
## Code Quality Assessment
|
||||
|
||||
### Strengths
|
||||
|
||||
1. **Minimal Changes**: Implementation focused on specific problem without scope creep
|
||||
2. **Error Handling**: Comprehensive fallback strategy for HEIC detection
|
||||
3. **Debug Support**: Failed uploads saved to debug directory with magic bytes logged
|
||||
4. **Test Coverage**: 5 new HEIC-specific tests, all passing (33/33 total)
|
||||
5. **Documentation**: Clear inline comments explaining iPhone-specific behavior
|
||||
6. **Backward Compatibility**: No breaking changes, existing uploads unaffected
|
||||
|
||||
### Issues Found
|
||||
|
||||
#### Critical Issues
|
||||
|
||||
**None identified**
|
||||
|
||||
#### Major Issues
|
||||
|
||||
**None identified**
|
||||
|
||||
#### Minor Issues
|
||||
|
||||
**M1: MPO Format Has No Test Coverage**
|
||||
|
||||
- **Location**: `starpunk/media.py` lines 163-173
|
||||
- **Issue**: MPO conversion code exists but no tests verify it works
|
||||
- **Impact**: MPO handling is untested; potential for silent failures
|
||||
- **Evidence**: Grep search found zero MPO test cases in test suite
|
||||
- **Recommendation**: Add MPO test case to `TestHEICSupport` class:
|
||||
```python
|
||||
def test_mpo_detection_and_conversion(self):
|
||||
"""Test MPO file is detected and converted to JPEG"""
|
||||
# MPO format is used by iPhone for depth/portrait mode photos
|
||||
# Create an MPO test image (Pillow supports creating these)
|
||||
# ... test implementation
|
||||
```
|
||||
- **Severity**: Minor (MPO is relatively rare format, but advertised in CHANGELOG)
|
||||
|
||||
**M2: Debug File Saving Could Fill Disk**
|
||||
|
||||
- **Location**: `starpunk/media.py` lines 133-137
|
||||
- **Issue**: Failed uploads saved to `data/debug/` without size limits or cleanup
|
||||
- **Impact**: Repeated failed uploads could accumulate and consume disk space
|
||||
- **Scenario**: Malicious user repeatedly uploads invalid 50MB files
|
||||
- **Recommendation**: Add cleanup strategy:
|
||||
- Limit debug directory to 100MB total
|
||||
- Auto-delete files older than 7 days
|
||||
- Add config option `DEBUG_SAVE_FAILED_UPLOADS` (default: false in production)
|
||||
- **Severity**: Minor (requires deliberate abuse; easily monitored)
|
||||
|
||||
**M3: HEIC Conversion Quality Not Configurable**
|
||||
|
||||
- **Location**: `starpunk/media.py` line 157
|
||||
- **Issue**: Hardcoded `quality=95` for HEIC→JPEG conversion
|
||||
- **Impact**: Users with different quality/size tradeoff preferences cannot adjust
|
||||
- **Current Behavior**: 95% quality always used, then subsequent optimization may reduce further
|
||||
- **Recommendation**: Consider adding `HEIC_CONVERSION_QUALITY` config variable
|
||||
- **Severity**: Minor (95% is reasonable default; optimization handles size anyway)
|
||||
|
||||
**M4: Missing Dimension Limit in Config**
|
||||
|
||||
- **Location**: `starpunk/media.py` line 41
|
||||
- **Issue**: `MAX_DIMENSION = 12000` is hardcoded constant
|
||||
- **Impact**: Cannot adjust limit without code change
|
||||
- **Rationale**: Design doc shows this was intentional (comment says "v1.4.2 - supports modern phone cameras")
|
||||
- **Recommendation**: Consider making this configurable in future versions
|
||||
- **Severity**: Minor (12000px is generous; few use cases for larger)
|
||||
|
||||
**M5: Incomplete Documentation of MPO Format**
|
||||
|
||||
- **Location**: `starpunk/media.py` lines 163-165
|
||||
- **Issue**: Comment says "extract primary image" but doesn't explain multi-frame nature
|
||||
- **Missing Context**: MPO files contain multiple images (left/right for 3D, or depth map)
|
||||
- **Recommendation**: Enhance comment:
|
||||
```python
|
||||
# MPO (Multi-Picture Object) conversion (v1.4.2)
|
||||
# MPO is used by iPhones for depth/portrait photos - contains multiple frames
|
||||
# (e.g., original + depth map). We extract only the primary/first frame as JPEG.
|
||||
# Depth information is discarded, which is acceptable for web display.
|
||||
```
|
||||
- **Severity**: Minor (code works correctly; just documentation clarity)
|
||||
|
||||
## Security Review
|
||||
|
||||
### Findings
|
||||
|
||||
**S1: Debug File Naming Could Leak Timing Information**
|
||||
|
||||
- **Location**: `starpunk/media.py` line 135
|
||||
- **Issue**: Filename includes timestamp with second precision
|
||||
- **Pattern**: `failed_20251216_174532_photo.heic`
|
||||
- **Concern**: Timing could reveal upload attempt patterns
|
||||
- **Recommendation**: Use UUID instead of timestamp, or truncate to date only
|
||||
- **Severity**: Very Low (minimal actual risk; defense in depth)
|
||||
|
||||
**S2: Increased Attack Surface from pillow-heif**
|
||||
|
||||
- **Location**: `requirements.txt` line 38
|
||||
- **Issue**: New dependency increases vulnerability surface area
|
||||
- **Mitigation**: Version is pinned (`pillow-heif==0.18.*`)
|
||||
- **Recommendation**:
|
||||
- Monitor pillow-heif security advisories
|
||||
- Document in security policy that HEIC support requires this dependency
|
||||
- Consider making pillow-heif optional in future (already has graceful degradation)
|
||||
- **Severity**: Low (acceptable tradeoff for iPhone support)
|
||||
|
||||
**S3: No Magic Bytes Validation for HEIC**
|
||||
|
||||
- **Location**: `starpunk/media.py` lines 115-121
|
||||
- **Issue**: Fallback HEIC detection uses `pillow_heif.read_heif()` without verifying magic bytes first
|
||||
- **Risk**: Could attempt to parse arbitrary binary data as HEIC
|
||||
- **Current Mitigation**: Exception handling catches parse failures
|
||||
- **Recommendation**: Add magic bytes check before calling `read_heif()`:
|
||||
```python
|
||||
# Check for HEIC/HEIF magic bytes (ftyp followed by heic/heix/hevc/etc)
|
||||
if file_data[4:8] == b'ftyp' and file_data[8:12] in [b'heic', b'heix', b'hevc', b'heim']:
|
||||
# Proceed with read_heif()
|
||||
```
|
||||
- **Severity**: Very Low (existing exception handling prevents exploitation)
|
||||
|
||||
### Security Assessment: ACCEPTABLE
|
||||
|
||||
No critical or major security issues identified. The increased attack surface from pillow-heif is an acceptable tradeoff for iPhone compatibility.
|
||||
|
||||
## Performance Analysis
|
||||
|
||||
### Observations
|
||||
|
||||
1. **HEIC Conversion Overhead**: Implementation report notes ~100-300ms per image
|
||||
2. **In-Memory Processing**: Uses BytesIO (no temp files) - good for security and performance
|
||||
3. **Double Processing**: HEIC files processed twice (conversion + optimization)
|
||||
- Could be optimized by combining steps, but current approach is cleaner
|
||||
4. **Large File Handling**: 50MB files through HEIC conversion could spike memory
|
||||
- Current implementation acceptable (server-grade machines)
|
||||
|
||||
### Performance Assessment: ACCEPTABLE
|
||||
|
||||
No significant performance concerns. The double-processing of HEIC files is a reasonable tradeoff for code simplicity and maintainability.
|
||||
|
||||
## Test Coverage Analysis
|
||||
|
||||
### Coverage Breakdown
|
||||
|
||||
**HEIC Tests (5 cases):**
|
||||
- ✅ Basic detection and conversion
|
||||
- ✅ RGBA mode handling
|
||||
- ✅ Dimension preservation
|
||||
- ✅ Error without library
|
||||
- ✅ Full upload flow
|
||||
|
||||
**Missing Tests:**
|
||||
- ❌ MPO format handling (see M1)
|
||||
- ❌ Large HEIC file (40MB+) conversion
|
||||
- ❌ Corrupted HEIC file handling
|
||||
- ❌ Debug file saving behavior
|
||||
- ❌ Fallback HEIC detection path (lines 115-121)
|
||||
|
||||
### Test Coverage Assessment: GOOD
|
||||
|
||||
HEIC support is well-tested. MPO support is the main gap. Recommend adding tests for:
|
||||
1. MPO format (priority: high)
|
||||
2. Fallback HEIC detection path (priority: medium)
|
||||
3. Debug file saving (priority: low)
|
||||
|
||||
## Edge Cases Review
|
||||
|
||||
### Handled Correctly
|
||||
|
||||
1. ✅ HEIC with alpha channel → RGB conversion
|
||||
2. ✅ HEIC with P mode → RGB conversion
|
||||
3. ✅ Missing pillow-heif library → Clear error
|
||||
4. ✅ HEIC files >50MB → Rejected at validation
|
||||
5. ✅ iOS mislabeled files (HEIC with .jpeg extension) → Fallback detection
|
||||
|
||||
### Not Handled / Unclear
|
||||
|
||||
1. **Animated HEIC sequences**: Do they exist? If so, behavior undefined
|
||||
- Recommendation: Document that only first frame is used (like MPO)
|
||||
2. **HEIC with EXIF orientation**: Does conversion preserve EXIF?
|
||||
- Code re-opens image after conversion, so EXIF lost
|
||||
- Subsequent `optimize_image()` applies EXIF orientation, so this is OK
|
||||
3. **Very large dimension HEIC (8000x12000)**: Will conversion succeed?
|
||||
- Should work (within 12000px limit), but not explicitly tested
|
||||
|
||||
## Documentation Review
|
||||
|
||||
### CHANGELOG.md
|
||||
|
||||
**Assessment**: GOOD
|
||||
|
||||
- Clear feature descriptions
|
||||
- Mentions both HEIC and MPO
|
||||
- Documents dimension limit increase
|
||||
- Lists dependency changes
|
||||
|
||||
**Suggestion**: Add upgrade notes section:
|
||||
```markdown
|
||||
### Upgrade Notes
|
||||
- Run `pip install -r requirements.txt` to install pillow-heif
|
||||
- No configuration changes required
|
||||
- Existing media uploads unaffected
|
||||
```
|
||||
|
||||
### Inline Comments
|
||||
|
||||
**Assessment**: GOOD
|
||||
|
||||
- Clear version markers (v1.4.2)
|
||||
- Explains WHY (browsers can't display HEIC, iOS uses MPO for depth)
|
||||
- Links to relevant concepts (portrait mode, depth maps)
|
||||
|
||||
### Implementation Report
|
||||
|
||||
**Assessment**: EXCELLENT
|
||||
|
||||
- Comprehensive technical decisions documented
|
||||
- Clear before/after code examples
|
||||
- Deployment notes included
|
||||
- Performance considerations noted
|
||||
|
||||
## Technical Debt Assessment
|
||||
|
||||
### New Debt Introduced
|
||||
|
||||
1. **Fallback HEIC Detection Complexity**: Lines 111-142 are complex nested exception handling
|
||||
- Could be refactored into separate function for clarity
|
||||
- Not urgent; works correctly
|
||||
2. **Debug File Accumulation**: See M2 above
|
||||
3. **Hardcoded Constants**: See M3 and M4 above
|
||||
|
||||
### Debt Assessment: LOW
|
||||
|
||||
Minimal new technical debt. The complex error handling is justified by the real-world issue it solves (iOS mislabeling files).
|
||||
|
||||
## Dependencies Review
|
||||
|
||||
### New Dependencies
|
||||
|
||||
**pillow-heif==0.18.***
|
||||
- Purpose: HEIC/HEIF format support
|
||||
- License: BSD-3-Clause (compatible)
|
||||
- Maintenance: Active (last release 2024)
|
||||
- Security: No known CVEs as of 2025-12-16
|
||||
|
||||
**Pillow 10.0.* → 10.1.***
|
||||
- Reason: Required by pillow-heif
|
||||
- Risk: Minor version bump
|
||||
- Mitigation: Full test suite passed (879 tests)
|
||||
|
||||
### Dependencies Assessment: ACCEPTABLE
|
||||
|
||||
Dependencies are well-maintained, properly licensed, and version-pinned.
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Immediate (Before Production Deployment)
|
||||
|
||||
**None** - Implementation is production-ready as-is
|
||||
|
||||
### Short-Term (Next Patch/Minor Release)
|
||||
|
||||
1. **Add MPO test coverage** (M1) - Priority: HIGH
|
||||
- Advertised in CHANGELOG but untested
|
||||
- Create test case in `TestHEICSupport` class
|
||||
|
||||
2. **Add debug file cleanup** (M2) - Priority: MEDIUM
|
||||
- Prevent disk exhaustion from failed uploads
|
||||
- Add size limit and age-based cleanup
|
||||
|
||||
3. **Document edge cases in code** (M5) - Priority: LOW
|
||||
- Enhance MPO comment to explain multi-frame nature
|
||||
- Add comment about animated HEIC handling (if applicable)
|
||||
|
||||
### Long-Term (Future Versions)
|
||||
|
||||
1. **Make HEIC quality configurable** (M3)
|
||||
2. **Make MAX_DIMENSION configurable** (M4)
|
||||
3. **Add magic bytes validation** (S3)
|
||||
4. **Refactor fallback detection** (Technical Debt #1)
|
||||
5. **Consider WebP conversion target** (Better compression than JPEG)
|
||||
|
||||
## Deployment Checklist
|
||||
|
||||
- ✅ Dependencies updated in requirements.txt
|
||||
- ✅ Version number incremented
|
||||
- ✅ CHANGELOG.md updated
|
||||
- ✅ Tests pass (33/33 media tests, 879/879 total)
|
||||
- ✅ Backward compatibility maintained
|
||||
- ✅ No database migrations required
|
||||
- ✅ No configuration changes required
|
||||
- ⚠️ MPO format untested (see M1)
|
||||
|
||||
## Conclusion
|
||||
|
||||
The v1.4.2 implementation successfully solves the iPhone photo upload problem with a clean, well-documented solution. The code quality is high, security concerns are minimal, and backward compatibility is maintained.
|
||||
|
||||
**Recommendation**: **APPROVE for production deployment**
|
||||
|
||||
The identified minor issues (particularly MPO test coverage) should be addressed in a follow-up patch, but do not block this release. The implementation solves a real user problem (iPhone uploads failing) and does so safely.
|
||||
|
||||
### Confidence Level
|
||||
|
||||
- **Code Quality**: HIGH (clean implementation, follows project standards)
|
||||
- **Security**: MEDIUM-HIGH (acceptable risk from new dependency)
|
||||
- **Test Coverage**: MEDIUM (HEIC well-tested, MPO untested)
|
||||
- **Documentation**: HIGH (clear comments and design docs)
|
||||
- **Overall**: HIGH (ready for production use)
|
||||
|
||||
## Follow-Up Actions
|
||||
|
||||
### For Developer
|
||||
|
||||
1. Consider adding MPO test case before next release
|
||||
2. Monitor pillow-heif for security updates
|
||||
3. Add debug directory cleanup in v1.4.3 or v1.5.0
|
||||
|
||||
### For Architect
|
||||
|
||||
1. Review recommendation to make MAX_DIMENSION configurable
|
||||
2. Decide if WebP conversion should replace JPEG for HEIC (better compression)
|
||||
3. Consider documenting animated HEIC handling policy
|
||||
|
||||
---
|
||||
|
||||
**Review completed**: 2025-12-16
|
||||
**Reviewed by**: Claude (Fullstack Developer Agent)
|
||||
**Status**: Implementation APPROVED with minor recommendations
|
||||
@@ -0,0 +1,412 @@
|
||||
# Architect Responses - v1.5.0 Developer Questions
|
||||
|
||||
**Date**: 2025-12-16
|
||||
**Architect**: StarPunk Architect Agent
|
||||
**In Response To**: `docs/design/v1.5.0/2025-12-16-developer-questions.md`
|
||||
|
||||
---
|
||||
|
||||
## Response Summary
|
||||
|
||||
All 8 questions have been addressed. The implementation can proceed immediately after reading this document. Key decisions:
|
||||
|
||||
1. **Q1**: Keep existing pattern (pass `existing_slugs: Set[str]`). ADR-062 pseudocode is conceptual.
|
||||
2. **Q2**: First note gets base slug, first collision gets `-1`. The table in ADR-062 is correct.
|
||||
3. **Q3**: Create new function in `slug_utils.py`. Migration path provided.
|
||||
4. **Q4**: Keep reserved slug validation for custom slugs only. Safe to skip for timestamp slugs.
|
||||
5. **Q5**: Tests should use pattern matching for timestamps. Test strategy provided.
|
||||
6. **Q6**: Phase 0 is a hard dependency. Cannot proceed with Phase 1 until all tests pass.
|
||||
7. **Q7**: Slug format is opaque. No format-aware code needed.
|
||||
8. **Q8**: Phases 2-4 can be parallelized. Commit each phase separately.
|
||||
|
||||
---
|
||||
|
||||
## Detailed Responses
|
||||
|
||||
### Q1: Collision handling function location and signature
|
||||
|
||||
**Decision**: Keep the existing pattern of accepting `existing_slugs: Set[str]` as a parameter.
|
||||
|
||||
**Rationale**:
|
||||
The ADR-062 pseudocode (`slug_exists(base_slug)`) is conceptual illustration only. The actual implementation should:
|
||||
|
||||
1. Follow the existing pattern in the codebase (passing pre-fetched slug sets)
|
||||
2. Avoid introducing database coupling into utility functions
|
||||
3. Prevent N+1 queries when checking multiple candidates
|
||||
|
||||
**Implementation Specification**:
|
||||
|
||||
Create a new function in `/home/phil/Projects/starpunk/starpunk/slug_utils.py`:
|
||||
|
||||
```python
|
||||
def generate_timestamp_slug(
|
||||
created_at: datetime = None,
|
||||
existing_slugs: Set[str] = None
|
||||
) -> str:
|
||||
"""Generate a timestamp-based slug with collision handling.
|
||||
|
||||
Per ADR-062: Default format is YYYYMMDDHHMMSS with sequential
|
||||
suffix (-1, -2, etc.) for collisions.
|
||||
|
||||
Args:
|
||||
created_at: Note creation timestamp (defaults to now)
|
||||
existing_slugs: Set of existing slugs to check for collisions
|
||||
|
||||
Returns:
|
||||
Unique timestamp-based slug
|
||||
"""
|
||||
if created_at is None:
|
||||
created_at = datetime.utcnow()
|
||||
|
||||
if existing_slugs is None:
|
||||
existing_slugs = set()
|
||||
|
||||
base_slug = created_at.strftime("%Y%m%d%H%M%S")
|
||||
|
||||
# If no collision, return base slug
|
||||
if base_slug not in existing_slugs:
|
||||
return base_slug
|
||||
|
||||
# Sequential suffix for collisions
|
||||
suffix = 1
|
||||
while f"{base_slug}-{suffix}" in existing_slugs:
|
||||
suffix += 1
|
||||
|
||||
return f"{base_slug}-{suffix}"
|
||||
```
|
||||
|
||||
**Location**: `/home/phil/Projects/starpunk/starpunk/slug_utils.py`
|
||||
|
||||
**Do NOT modify**: The `ensure_unique_slug()` function signature in ADR-062 is pseudocode. Do not create a function with database coupling.
|
||||
|
||||
---
|
||||
|
||||
### Q2: Sequential suffix starting number
|
||||
|
||||
**Decision**: First note gets base slug (no suffix). First collision gets `-1`.
|
||||
|
||||
**Clarification**: The examples table in ADR-062 is correct. The text "first collision: `20251216143052-1`" means the first **duplicate** (collision) gets `-1`, not the first note overall.
|
||||
|
||||
**Example Scenario**:
|
||||
```
|
||||
Note A created at 14:30:52 -> slug = "20251216143052" (base, no suffix)
|
||||
Note B created at 14:30:52 -> slug = "20251216143052-1" (first collision)
|
||||
Note C created at 14:30:52 -> slug = "20251216143052-2" (second collision)
|
||||
```
|
||||
|
||||
**Implementation**: Sequential suffix loop should start at `1`, not `2`:
|
||||
|
||||
```python
|
||||
suffix = 1
|
||||
while f"{base_slug}-{suffix}" in existing_slugs:
|
||||
suffix += 1
|
||||
return f"{base_slug}-{suffix}"
|
||||
```
|
||||
|
||||
**Note**: This differs from the existing `make_slug_unique_with_suffix()` function in `slug_utils.py` which starts at `-2`. The new timestamp slug function should start at `-1` per ADR-062.
|
||||
|
||||
---
|
||||
|
||||
### Q3: Two `generate_slug()` functions with different signatures
|
||||
|
||||
**Decision**: Create a new function in `slug_utils.py`. Do not modify or remove the existing function in `utils.py`.
|
||||
|
||||
**Migration Path**:
|
||||
|
||||
1. **Create new function** in `/home/phil/Projects/starpunk/starpunk/slug_utils.py`:
|
||||
- Name: `generate_timestamp_slug()` (as specified in Q1 response)
|
||||
- Do not name it `generate_slug()` to avoid confusion during migration
|
||||
|
||||
2. **Update `/home/phil/Projects/starpunk/starpunk/notes.py`** (lines 228-234):
|
||||
|
||||
**Current code** (line 228-234):
|
||||
```python
|
||||
else:
|
||||
# Generate base slug from content
|
||||
base_slug = generate_slug(content, created_at)
|
||||
|
||||
# Make unique if collision
|
||||
slug = make_slug_unique(base_slug, existing_slugs)
|
||||
```
|
||||
|
||||
**Replace with**:
|
||||
```python
|
||||
else:
|
||||
# Generate timestamp-based slug (ADR-062)
|
||||
from starpunk.slug_utils import generate_timestamp_slug
|
||||
slug = generate_timestamp_slug(created_at, existing_slugs)
|
||||
```
|
||||
|
||||
3. **Keep the existing function** in `utils.py` unchanged. It may be used elsewhere or for future content-based slug options.
|
||||
|
||||
4. **Update imports** in `notes.py`:
|
||||
- Remove `generate_slug` from the `utils` import line (line 34) if no longer needed
|
||||
- Verify no other callers use `generate_slug()` for default note creation
|
||||
|
||||
**Rationale**:
|
||||
- Creating a distinctly-named function makes the transition explicit
|
||||
- Preserves backward compatibility if content-based slugs are ever needed
|
||||
- Allows gradual migration with clear audit trail
|
||||
|
||||
---
|
||||
|
||||
### Q4: Reserved slug handling in timestamp context
|
||||
|
||||
**Decision**:
|
||||
- **Default timestamp slugs**: Safe to skip reserved slug validation (timestamps cannot collide with reserved words)
|
||||
- **Custom slugs**: Must retain reserved slug validation
|
||||
|
||||
**Implementation**:
|
||||
|
||||
The new `generate_timestamp_slug()` function does NOT need to check reserved slugs because:
|
||||
- Reserved slugs are alphabetic (`api`, `admin`, `feed`, etc.)
|
||||
- Timestamp slugs are purely numeric (`20251216143052`)
|
||||
- These sets are mutually exclusive by construction
|
||||
|
||||
**However**, the existing custom slug validation in `validate_and_sanitize_custom_slug()` must retain reserved slug checking because user-provided slugs could be anything.
|
||||
|
||||
**Code Impact**:
|
||||
- `generate_timestamp_slug()`: No reserved slug check needed
|
||||
- `validate_and_sanitize_custom_slug()`: Unchanged (already handles reserved slugs)
|
||||
- `validate_slug()` in `utils.py`: Unchanged (still validates reserved slugs for custom slugs)
|
||||
|
||||
**Simplification**: You may remove the defensive `validate_slug()` call after timestamp generation in `notes.py` (line 236-237) since timestamp slugs are guaranteed valid by construction:
|
||||
```python
|
||||
# This check is no longer needed for timestamp slugs:
|
||||
# if not validate_slug(slug):
|
||||
# raise InvalidNoteDataError(...)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Q5: Test update strategy for slug format changes
|
||||
|
||||
**Decision**: Use pattern matching for timestamp assertions. Preserve content-based tests in a separate test file or clearly marked section.
|
||||
|
||||
**Test Strategy**:
|
||||
|
||||
1. **For new timestamp slug tests**, use regex pattern matching:
|
||||
```python
|
||||
import re
|
||||
|
||||
TIMESTAMP_SLUG_PATTERN = re.compile(r'^\d{14}(-\d+)?$')
|
||||
|
||||
def test_default_slug_is_timestamp():
|
||||
note = create_note("Some content here")
|
||||
assert TIMESTAMP_SLUG_PATTERN.match(note.slug)
|
||||
|
||||
def test_slug_collision_adds_suffix():
|
||||
# Create two notes at same timestamp
|
||||
fixed_time = datetime(2025, 12, 16, 14, 30, 52)
|
||||
note1 = create_note("First note", created_at=fixed_time)
|
||||
note2 = create_note("Second note", created_at=fixed_time)
|
||||
|
||||
assert note1.slug == "20251216143052"
|
||||
assert note2.slug == "20251216143052-1"
|
||||
```
|
||||
|
||||
2. **For timestamp-specific assertions**, use fixed timestamps in tests:
|
||||
```python
|
||||
def test_timestamp_slug_format():
|
||||
"""Test that timestamp slug matches exact format."""
|
||||
fixed_time = datetime(2025, 12, 16, 14, 30, 52)
|
||||
slug = generate_timestamp_slug(fixed_time, set())
|
||||
assert slug == "20251216143052"
|
||||
```
|
||||
|
||||
3. **Content-based tests**: Move to a separate section or file named `test_legacy_slug_generation.py` with a comment:
|
||||
```python
|
||||
"""
|
||||
Legacy tests for content-based slug generation.
|
||||
|
||||
These test the generate_slug() function in utils.py which is
|
||||
preserved for backward compatibility. New notes use timestamp-based
|
||||
slugs per ADR-062.
|
||||
"""
|
||||
```
|
||||
|
||||
4. **Update existing tests that assert specific slug formats**:
|
||||
- Replace exact string assertions (`assert slug == "hello-world"`) with pattern matching
|
||||
- OR use fixed timestamps and assert exact expected values
|
||||
|
||||
**Affected Test Files** (estimate from your question):
|
||||
- `/home/phil/Projects/starpunk/tests/test_utils.py`: ~20 tests
|
||||
- `/home/phil/Projects/starpunk/tests/test_notes.py`: ~10 tests
|
||||
- `/home/phil/Projects/starpunk/tests/test_micropub.py`: ~10 tests
|
||||
|
||||
**Helper Function** (add to test utilities or conftest.py):
|
||||
```python
|
||||
def assert_valid_timestamp_slug(slug: str) -> None:
|
||||
"""Assert slug matches timestamp format per ADR-062."""
|
||||
pattern = re.compile(r'^\d{14}(-\d+)?$')
|
||||
assert pattern.match(slug), f"Slug '{slug}' does not match timestamp format"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Q6: Phase 0 priority - are tests blocking?
|
||||
|
||||
**Decision**: Phase 0 is a **hard dependency**. Phase 1 cannot proceed until all tests pass.
|
||||
|
||||
**Rationale**:
|
||||
|
||||
1. **Failing tests mask regressions**: If you implement Phase 1 (slug changes) while 19 tests are failing, you cannot distinguish between:
|
||||
- Pre-existing failures
|
||||
- New failures caused by your changes
|
||||
|
||||
2. **CI/CD integrity**: The project should maintain a green build. Committing Phase 1 work with known failing tests violates this principle.
|
||||
|
||||
3. **Debugging complexity**: If slug changes interact unexpectedly with the failing test areas (e.g., feed generation, content negotiation), debugging becomes significantly harder.
|
||||
|
||||
**Implementation Order**:
|
||||
|
||||
```
|
||||
Phase 0 (Test Fixes) <-- MUST COMPLETE FIRST
|
||||
|
|
||||
v
|
||||
+-- Phase 1 (Timestamp Slugs)
|
||||
|
|
||||
+-- Phase 2 (Debug Files) <-- Can run in parallel
|
||||
|
|
||||
+-- Phase 3 (N+1 Query Fix) <-- after Phase 0
|
||||
|
|
||||
+-- Phase 4 (Atomic Variants)
|
||||
|
|
||||
v
|
||||
Phase 5 (Test Coverage) <-- LAST
|
||||
```
|
||||
|
||||
**Exception**: If fixing a Phase 0 test reveals it requires slug-related changes, document the circular dependency and bring to architect attention.
|
||||
|
||||
**Recommendation**: Start Phase 0 immediately. Many of those 19 failures may be simple test maintenance issues (timeouts, assertions that need updating, etc.).
|
||||
|
||||
---
|
||||
|
||||
### Q7: Backwards compatibility for existing notes
|
||||
|
||||
**Decision**: Slug format is **opaque**. No code needs to be format-aware.
|
||||
|
||||
**Clarification**:
|
||||
|
||||
1. **No format detection needed**: Do not create an `is_timestamp_slug()` helper function. The system should treat all slugs identically regardless of format.
|
||||
|
||||
2. **Code paths are format-agnostic**:
|
||||
- Database queries use slug as opaque string key
|
||||
- URL routing uses slug as path parameter
|
||||
- Feed generation includes slug verbatim
|
||||
- Templates display slug without interpretation
|
||||
|
||||
3. **Coexistence is automatic**: Content-based slugs (`my-first-post`) and timestamp slugs (`20251216143052`) are both valid slug patterns. They coexist naturally.
|
||||
|
||||
4. **Sorting behavior**:
|
||||
- Database ORDER BY on `created_at` is unaffected
|
||||
- Alphabetical slug sorting would interleave formats (numbers sort before letters)
|
||||
- This is acceptable; no UI sorts by slug alphabetically
|
||||
|
||||
5. **Display considerations**:
|
||||
- Timestamps are less readable in URLs, but URLs are rarely human-read
|
||||
- UI displays note content, not slugs
|
||||
- RSS feeds include full URLs; format doesn't affect feed readers
|
||||
|
||||
**No audit required**: You do not need to audit slug usages. The design ensures format-agnosticism.
|
||||
|
||||
---
|
||||
|
||||
### Q8: Phase ordering rationale
|
||||
|
||||
**Decision**:
|
||||
- Phases 2, 3, and 4 **can be parallelized** after Phase 0 completes
|
||||
- Phase 1 (Timestamp Slugs) should be done **before** Phases 2-4 for cleaner testing
|
||||
- Each phase should be **committed separately**
|
||||
|
||||
**Recommended Order**:
|
||||
|
||||
```
|
||||
Day 1: Phase 0 (Test Fixes) - Get to green build
|
||||
Day 2: Phase 1 (Timestamp Slugs) - Quick win, user-visible improvement
|
||||
Day 3: Phase 2 (Debug Files) or Phase 3 (N+1) or Phase 4 (Atomic) - Any order
|
||||
Day 4: Remaining phases from above
|
||||
Day 5: Phase 5 (Test Coverage) - Must be last
|
||||
```
|
||||
|
||||
**Commit Strategy**:
|
||||
|
||||
Each phase gets its own commit or small series of commits:
|
||||
```
|
||||
feat(slugs): Implement timestamp-based slugs per ADR-062
|
||||
fix(tests): Resolve 19 failing tests in Phase 0
|
||||
feat(media): Add debug file cleanup and configuration
|
||||
perf(feed): Batch load media and tags to fix N+1 query
|
||||
feat(media): Make variant generation atomic with database
|
||||
test: Expand coverage to 90% for v1.5.0
|
||||
```
|
||||
|
||||
**Why Not Batch**: Batching commits makes bisecting regressions harder and obscures the changelog.
|
||||
|
||||
**Parallelization Note**: If you have capacity to work on multiple phases simultaneously (e.g., waiting for test runs), Phases 2-4 have no code overlap and can be developed in parallel branches:
|
||||
- `feature/v1.5.0-debug-files`
|
||||
- `feature/v1.5.0-n+1-fix`
|
||||
- `feature/v1.5.0-atomic-variants`
|
||||
|
||||
Then merge all into main before Phase 5.
|
||||
|
||||
---
|
||||
|
||||
## Observations Responses
|
||||
|
||||
### O1: Code duplication between `utils.py` and `slug_utils.py`
|
||||
|
||||
**Acknowledged**. The duplicate `RESERVED_SLUGS` definitions should be consolidated in a future cleanup. For v1.5.0:
|
||||
|
||||
- Use the more comprehensive list in `slug_utils.py` (12 items) as the canonical source
|
||||
- Do not consolidate in this release (scope creep)
|
||||
- Add to BACKLOG.md for v1.5.1 or v1.6.0
|
||||
|
||||
### O2: Timestamp format consistency
|
||||
|
||||
**Confirmed**: The hyphen removal in ADR-062 (`YYYYMMDDHHMMSS`) vs existing fallback (`YYYYMMDD-HHMMSS`) is intentional.
|
||||
|
||||
- ADR-062 format: `20251216143052` (14 characters, no separator)
|
||||
- Old fallback format: `20251216-143052` (15 characters, with hyphen)
|
||||
|
||||
**Rationale**:
|
||||
- One fewer character
|
||||
- Consistent with ISO 8601 compact format
|
||||
- No functional difference; purely aesthetic
|
||||
|
||||
**Action**: Use the unhyphenated format per ADR-062.
|
||||
|
||||
### O3: Test coverage gap mentioned in Phase 5
|
||||
|
||||
**Clarification**: This is intentional duplication for visibility.
|
||||
|
||||
- BACKLOG.md lists the gap for tracking
|
||||
- Phase 5 in RELEASE.md lists it for implementation planning
|
||||
- Both point to the same work item
|
||||
|
||||
No action needed; this is not an error.
|
||||
|
||||
---
|
||||
|
||||
## ADR-062 Updates Required
|
||||
|
||||
No updates to ADR-062 are required based on these questions. The ADR's pseudocode is understood to be conceptual, and all implementation details have been clarified in this response document.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Checklist
|
||||
|
||||
Based on these responses, the developer can now proceed with:
|
||||
|
||||
- [ ] Phase 0: Fix 19 failing tests
|
||||
- [ ] Create `generate_timestamp_slug()` in `slug_utils.py`
|
||||
- [ ] Update `notes.py` to use new timestamp slug function
|
||||
- [ ] Update tests with pattern matching strategy
|
||||
- [ ] Proceed with Phases 2-4 (order flexible)
|
||||
- [ ] Complete Phase 5 test coverage last
|
||||
|
||||
---
|
||||
|
||||
**Architect**: StarPunk Architect Agent
|
||||
**Status**: Questions Answered - Ready for Implementation
|
||||
**Next Action**: Developer to begin Phase 0 (Test Fixes)
|
||||
@@ -0,0 +1,272 @@
|
||||
# Developer Review Questions - v1.5.0 Release Plan
|
||||
|
||||
**Date**: 2025-12-16
|
||||
**Reviewer**: StarPunk Developer Agent
|
||||
**Documents Reviewed**:
|
||||
- `docs/decisions/ADR-062-timestamp-based-slug-format.md`
|
||||
- `docs/projectplan/v1.5.0/RELEASE.md`
|
||||
- `docs/projectplan/BACKLOG.md`
|
||||
|
||||
## Executive Summary
|
||||
|
||||
After thorough review of the v1.5.0 release plan and ADR-062, I have identified **8 questions** that need clarification from the architect before implementation can begin. These questions fall into three categories:
|
||||
|
||||
1. **Slug implementation details** (Questions 1-4)
|
||||
2. **Test structure and expectations** (Questions 5-6)
|
||||
3. **Phase dependencies and ordering** (Questions 7-8)
|
||||
|
||||
The majority of questions concern Phase 1 (Timestamp-Based Slugs), which has some ambiguities around collision handling and interaction with existing code paths.
|
||||
|
||||
---
|
||||
|
||||
## Questions
|
||||
|
||||
### Phase 1: Timestamp-Based Slugs
|
||||
|
||||
#### Q1: Collision handling function location and signature
|
||||
|
||||
**Issue**: ADR-062 specifies an `ensure_unique_slug(base_slug: str) -> str` function that checks `slug_exists(base_slug)`, but the implementation location and database interaction pattern are unclear.
|
||||
|
||||
**Current State**:
|
||||
- `starpunk/slug_utils.py` has `make_slug_unique_with_suffix(base_slug, existing_slugs: Set[str])` that takes a pre-fetched set
|
||||
- `starpunk/utils.py` has `make_slug_unique(base_slug, existing_slugs: set[str])` with random suffix logic
|
||||
- `starpunk/notes.py` line 220 calls `_get_existing_slugs(db)` to fetch all slugs, then passes to uniqueness check
|
||||
|
||||
**Questions**:
|
||||
1. Should `ensure_unique_slug()` perform its own database queries (violates current pattern)?
|
||||
2. Or should it follow the existing pattern of accepting `existing_slugs: Set[str]` parameter?
|
||||
3. Should this function live in `slug_utils.py` or `utils.py`?
|
||||
4. The ADR shows `slug_exists()` as a separate function - should this be implemented, or is this pseudocode?
|
||||
|
||||
**Implementation Impact**: This affects the function signature and which module needs modification. The current codebase consistently passes pre-fetched slug sets to avoid N+1 queries.
|
||||
|
||||
**Suggested Clarification**: Confirm whether to:
|
||||
- Keep existing pattern: `ensure_unique_slug(base_slug: str, existing_slugs: Set[str]) -> str`
|
||||
- Or introduce database coupling: `ensure_unique_slug(base_slug: str, db: Connection) -> str`
|
||||
|
||||
---
|
||||
|
||||
#### Q2: Sequential suffix starting number
|
||||
|
||||
**Issue**: ADR-062 says "first collision: `20251216143052-1`" but the examples table shows the second note as `-1`, implying the first note has no suffix.
|
||||
|
||||
**Current State**:
|
||||
- Existing `make_slug_unique_with_suffix()` starts at `-2` for first collision (base slug is attempt #1)
|
||||
- This matches table in ADR-062 examples
|
||||
|
||||
**Questions**:
|
||||
1. Does "first collision" mean the first duplicate attempt (which should get `-1`)?
|
||||
2. Or does it mean the first note gets the base slug, and second note gets `-1`?
|
||||
|
||||
**Example Scenario**:
|
||||
```python
|
||||
# Two notes created at exactly 2025-12-16 14:30:52
|
||||
Note A created first: slug = ?
|
||||
Note B created second: slug = ?
|
||||
|
||||
# Is it:
|
||||
Option 1: A = 20251216143052, B = 20251216143052-1 (matches table)
|
||||
Option 2: A = 20251216143052-1, B = 20251216143052-2 (matches "first collision" text)
|
||||
```
|
||||
|
||||
**Implementation Impact**: Determines whether sequential suffix loop starts at 1 or 2.
|
||||
|
||||
**Suggested Clarification**: Confirm that first note gets base slug (no suffix), first collision gets `-1`.
|
||||
|
||||
---
|
||||
|
||||
#### Q3: Two `generate_slug()` functions with different signatures
|
||||
|
||||
**Issue**: The codebase has two `generate_slug()` functions:
|
||||
|
||||
**Current State**:
|
||||
- `starpunk/utils.py` line 173: `generate_slug(content: str, created_at: Optional[datetime]) -> str`
|
||||
- Content-based slug generation (extracts words)
|
||||
- Used by `notes.py` line 230
|
||||
- ADR-062 specifies: `generate_slug(custom_slug: str = None, created_at: datetime = None) -> str`
|
||||
- Timestamp-based, no content parameter
|
||||
|
||||
**Questions**:
|
||||
1. Should I replace the function in `utils.py`?
|
||||
2. Should I create a new function in `slug_utils.py` and leave `utils.py` as deprecated?
|
||||
3. Should both functions coexist with different names?
|
||||
4. The removal of the `content` parameter is a breaking change for `notes.py` - should `notes.py` be updated to not pass content at all?
|
||||
|
||||
**Implementation Impact**:
|
||||
- If replacing `utils.py` function: ~15 test files will need updates
|
||||
- If creating new function: need to update all callers to use new function
|
||||
- `notes.py` line 230 currently calls `generate_slug(content, created_at)` - this will break
|
||||
|
||||
**Suggested Clarification**: Specify exact function location and migration path for callers.
|
||||
|
||||
---
|
||||
|
||||
#### Q4: Reserved slug handling in timestamp context
|
||||
|
||||
**Issue**: ADR-062 doesn't address what happens if a timestamp-based slug conflicts with a reserved slug.
|
||||
|
||||
**Current State**:
|
||||
- Reserved slugs: `['api', 'admin', 'auth', 'feed', ...]` (all alphabetic)
|
||||
- Timestamp format: `20251216143052` (all numeric)
|
||||
- **These can never collide**
|
||||
|
||||
**Questions**:
|
||||
1. Should reserved slug validation be removed from the default slug path since timestamps can't conflict?
|
||||
2. Should it remain for safety/future-proofing?
|
||||
3. Custom slugs still need reserved validation - should this logic split?
|
||||
|
||||
**Implementation Impact**: May simplify code by removing unnecessary validation from timestamp path.
|
||||
|
||||
**Suggested Clarification**: Confirm whether reserved slug check is still needed for default slugs.
|
||||
|
||||
---
|
||||
|
||||
### Phase 0: Test Fixes
|
||||
|
||||
#### Q5: Test update strategy for slug format changes
|
||||
|
||||
**Issue**: Phase 1 will change default slug format from content-based to timestamp-based. Many tests currently assert content-based slugs.
|
||||
|
||||
**Current State**:
|
||||
- `tests/test_utils.py` has 20+ tests asserting content-based slug behavior (e.g., `assert slug == "hello-world-this-is-my"`)
|
||||
- Phase 1 acceptance criteria: "Update expected slug formats in test assertions"
|
||||
|
||||
**Questions**:
|
||||
1. Should these tests be updated to assert timestamp format (e.g., `assert slug.startswith("2025")`)?
|
||||
2. Should the old content-based tests be preserved but marked as testing legacy behavior?
|
||||
3. Should tests be split: "default slug generation" vs "content-based slug generation" (if we keep the old function)?
|
||||
4. Do you want all slug assertions to use pattern matching (timestamp format) or fixed timestamps in tests?
|
||||
|
||||
**Implementation Impact**:
|
||||
- Affects ~30 test assertions in `test_utils.py`
|
||||
- Affects ~10 integration tests in `test_notes.py`, `test_micropub.py`
|
||||
- May require test helper functions for deterministic timestamps
|
||||
|
||||
**Suggested Clarification**: Provide guidance on test update strategy.
|
||||
|
||||
---
|
||||
|
||||
#### Q6: Phase 0 priority - are tests blocking?
|
||||
|
||||
**Issue**: Phase 0 is marked "Must complete first - unblocks all other phases" but the 19 failing tests appear unrelated to slug changes.
|
||||
|
||||
**Current State**:
|
||||
- Failing tests: Migration performance, feed streaming, content negotiation, search security
|
||||
- None directly related to slug generation
|
||||
- Phase 1 (slugs) could theoretically proceed with these tests failing
|
||||
|
||||
**Questions**:
|
||||
1. Is Phase 0 truly blocking for Phase 1, or can Phase 1 proceed if slug-related tests pass?
|
||||
2. Should Phase 0 be completed before ANY other phase, or just before phases that depend on those specific test areas?
|
||||
3. If I discover that fixing some Phase 0 tests requires changes that conflict with Phase 1 work, what's the priority?
|
||||
|
||||
**Implementation Impact**: Affects work sequencing and potential merge conflicts.
|
||||
|
||||
**Suggested Clarification**: Confirm hard dependency of Phase 0 on all other phases, or allow parallel work.
|
||||
|
||||
---
|
||||
|
||||
### General Implementation Questions
|
||||
|
||||
#### Q7: Backwards compatibility for existing notes
|
||||
|
||||
**Issue**: ADR-062 states "Existing notes retain their slugs (no data migration)" but doesn't specify how the dual slug formats coexist.
|
||||
|
||||
**Current State**:
|
||||
- Database has notes with content-based slugs (e.g., `my-first-post`)
|
||||
- New notes will have timestamp slugs (e.g., `20251216143052`)
|
||||
- Both slug formats will exist simultaneously
|
||||
|
||||
**Questions**:
|
||||
1. Do any code paths need to detect/handle the two formats differently?
|
||||
2. Should there be a helper function `is_timestamp_slug(slug: str) -> bool` for future use?
|
||||
3. Are there any edge cases where code assumes slug format (e.g., sorting, display, URLs)?
|
||||
4. Does feed generation or UI need updates to handle mixed slug formats?
|
||||
|
||||
**Implementation Impact**: May require audit of all slug usages to ensure format-agnostic handling.
|
||||
|
||||
**Suggested Clarification**: Confirm if any code needs to be format-aware, or if slug format is truly opaque.
|
||||
|
||||
---
|
||||
|
||||
#### Q8: Phase ordering rationale
|
||||
|
||||
**Issue**: Phases 2-4 are all marked "Phase 0 complete" as dependency, but no interdependencies are noted between them.
|
||||
|
||||
**Current State**:
|
||||
- Phase 2: Debug file management
|
||||
- Phase 3: N+1 query fix (feed generation)
|
||||
- Phase 4: Atomic variant generation
|
||||
- All independent of each other
|
||||
|
||||
**Questions**:
|
||||
1. Can Phases 2-4 be implemented in parallel after Phase 0?
|
||||
2. Is there a preferred order (2→3→4) for any reason?
|
||||
3. Should each phase be committed separately or can they be batched?
|
||||
|
||||
**Implementation Impact**: Affects work planning and commit structure.
|
||||
|
||||
**Suggested Clarification**: Confirm whether Phases 2-4 can be parallelized or should be sequential.
|
||||
|
||||
---
|
||||
|
||||
## Additional Observations (Not Blocking)
|
||||
|
||||
These observations don't block implementation but may inform architectural decisions:
|
||||
|
||||
### O1: Code duplication between `utils.py` and `slug_utils.py`
|
||||
|
||||
**Observation**: Reserved slugs are defined in both:
|
||||
- `utils.py` line 24: `RESERVED_SLUGS = {"admin", "api", ...}` (5 items)
|
||||
- `slug_utils.py` line 24: `RESERVED_SLUGS = frozenset([...])` (12 items, more comprehensive)
|
||||
|
||||
**Impact**: Low - but may cause confusion. Consider consolidating in v1.5.1.
|
||||
|
||||
---
|
||||
|
||||
### O2: Timestamp format consistency
|
||||
|
||||
**Observation**: ADR-062 uses `YYYYMMDDHHMMSS` (no separator) but existing fallback in `utils.py` line 220 uses `YYYYMMDD-HHMMSS` (with hyphen).
|
||||
|
||||
**Questions**:
|
||||
- Is the hyphen removal intentional (shorter, more compact)?
|
||||
- Current: `20241118-143045` (15 chars)
|
||||
- ADR-062: `20241118143045` (14 chars)
|
||||
|
||||
**Impact**: Minor - affects slug length by 1 character.
|
||||
|
||||
---
|
||||
|
||||
### O3: Test coverage gap mentioned in Phase 5
|
||||
|
||||
**Observation**: Phase 5 mentions "MPO format handling (untested)" but this is already in BACKLOG as "High priority - Scheduled for v1.5.0".
|
||||
|
||||
**Question**: Is this intentionally listed in both BACKLOG and Phase 5, or is there duplication?
|
||||
|
||||
**Impact**: None - just a clarity question.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**Implementation Readiness**: 6/10
|
||||
|
||||
The release plan is well-structured with clear phases and acceptance criteria. However, **Phase 1 (Timestamp-Based Slugs)** has enough ambiguity in implementation details (Questions 1-4) that I cannot confidently proceed without architect clarification.
|
||||
|
||||
**Recommended Next Steps**:
|
||||
1. Architect addresses Questions 1-4 (Phase 1 blocking issues)
|
||||
2. Architect clarifies Questions 5-8 (helps inform implementation approach)
|
||||
3. Developer proceeds with Phase 0 (test fixes) while awaiting Q1-Q4 answers
|
||||
4. Developer implements Phases 1-4 after clarifications received
|
||||
|
||||
**Estimated Time to Implementation-Ready**: 1-2 hours for architect to address questions.
|
||||
|
||||
---
|
||||
|
||||
## Approval
|
||||
|
||||
Once these questions are addressed, I am confident the implementation can proceed smoothly with the 18-29 hour estimate provided in the release plan.
|
||||
|
||||
**Developer**: StarPunk Developer Agent
|
||||
**Status**: Awaiting Architect Response
|
||||
**Next Action**: Architect to review and answer questions 1-8
|
||||
@@ -0,0 +1,120 @@
|
||||
# Phase 0 Implementation Report - Test Fixes
|
||||
|
||||
**Date**: 2025-12-16
|
||||
**Developer**: Developer Agent
|
||||
**Phase**: v1.5.0 Phase 0 - Test Fixes
|
||||
**Status**: Complete
|
||||
|
||||
## Overview
|
||||
|
||||
Successfully implemented Phase 0 of v1.5.0 as specified in ADR-012 (Flaky Test Removal) and the v1.5.0 RELEASE.md. All test fixes completed and verified across 3 test runs with no flakiness detected.
|
||||
|
||||
## Changes Made
|
||||
|
||||
### 1. Removed 5 Broken Multiprocessing Tests
|
||||
|
||||
**File**: `tests/test_migration_race_condition.py`
|
||||
|
||||
Removed the following tests that fundamentally cannot work due to Python multiprocessing limitations:
|
||||
|
||||
- `test_concurrent_workers_barrier_sync` - Cannot pickle Barrier objects for Pool.map()
|
||||
- `test_sequential_worker_startup` - Missing Flask app context across processes
|
||||
- `test_worker_late_arrival` - Missing Flask app context across processes
|
||||
- `test_single_worker_performance` - Cannot pickle local functions
|
||||
- `test_concurrent_workers_performance` - Cannot pickle local functions
|
||||
|
||||
**Action Taken**:
|
||||
- Removed entire `TestConcurrentExecution` class (3 tests)
|
||||
- Removed entire `TestPerformance` class (2 tests)
|
||||
- Removed unused module-level worker functions (`_barrier_worker`, `_simple_worker`)
|
||||
- Removed unused imports (`time`, `multiprocessing`, `Barrier`)
|
||||
- Added explanatory comments documenting why tests were removed
|
||||
|
||||
**Justification**: Per ADR-012, these tests have architectural issues that make them unreliable. The migration retry logic they attempt to test is proven to work in production with multi-worker Gunicorn deployments. The tests are the problem, not the code.
|
||||
|
||||
### 2. Fixed Brittle Feed XML Assertions
|
||||
|
||||
**File**: `tests/test_routes_feeds.py`
|
||||
|
||||
Fixed assertions that were checking implementation details (quote style) rather than semantics (valid XML):
|
||||
|
||||
**Changes**:
|
||||
- `test_feed_atom_endpoint`: Changed from checking `<?xml version="1.0"` to checking `<?xml version=` and `encoding=` separately
|
||||
- `test_feed_json_endpoint`: Changed Content-Type assertion from exact match to `.startswith('application/feed+json')` to not require charset specification
|
||||
- `test_accept_json_feed`: Same Content-Type fix as above
|
||||
- `test_accept_json_generic`: Same Content-Type fix as above
|
||||
- `test_quality_factor_json_wins`: Same Content-Type fix as above
|
||||
|
||||
**Rationale**: XML generators may use single or double quotes. Tests should verify semantics (valid XML with correct encoding), not formatting details. Similarly, Content-Type may or may not include charset parameter depending on framework version.
|
||||
|
||||
### 3. Fixed test_debug_level_for_early_retries
|
||||
|
||||
**File**: `tests/test_migration_race_condition.py`
|
||||
|
||||
**Issue**: Logger not configured to capture DEBUG level messages.
|
||||
|
||||
**Fix**: Simplified logger configuration by:
|
||||
- Removing unnecessary `caplog.clear()` calls
|
||||
- Ensuring `caplog.at_level(logging.DEBUG, logger='starpunk.migrations')` wraps the actual test execution
|
||||
- Removing redundant clearing inside the context manager
|
||||
|
||||
**Result**: Test now reliably captures DEBUG level log messages from the migrations module.
|
||||
|
||||
### 4. Verified test_new_connection_per_retry
|
||||
|
||||
**File**: `tests/test_migration_race_condition.py`
|
||||
|
||||
**Finding**: This test is actually working correctly. It expects 10 connection attempts (retry_count 0-9) which matches the implementation (`while retry_count < max_retries` where `max_retries = 10`).
|
||||
|
||||
**Action**: No changes needed. Test runs successfully and correctly verifies that a new connection is created for each retry attempt.
|
||||
|
||||
### 5. Renamed Misleading Test
|
||||
|
||||
**File**: `tests/test_routes_feed.py`
|
||||
|
||||
**Change**: Renamed `test_feed_route_streaming` to `test_feed_route_caching`
|
||||
|
||||
**Rationale**: The test name said "streaming" but the implementation actually uses caching (Phase 3 feed optimization). The test correctly verifies ETag presence, which is a caching feature. The name was misleading but the test logic was correct.
|
||||
|
||||
## Test Results
|
||||
|
||||
Ran full test suite 3 times to verify no flakiness:
|
||||
|
||||
**Run 1**: 874 passed, 1 warning in 375.92s
|
||||
**Run 2**: 874 passed, 1 warning in 386.40s
|
||||
**Run 3**: 874 passed, 1 warning in 375.68s
|
||||
|
||||
**Test Count**: Reduced from 879 to 874 (5 tests removed as planned)
|
||||
**Flakiness**: None detected across 3 runs
|
||||
**Warnings**: 1 expected warning about DecompressionBombWarning (intentional test of large image handling)
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
| Criterion | Status | Evidence |
|
||||
|-----------|--------|----------|
|
||||
| All remaining tests pass consistently | ✓ | 3 successful test runs |
|
||||
| 5 broken tests removed | ✓ | Test count: 879 → 874 |
|
||||
| No new test skips added | ✓ | No `@pytest.mark.skip` added |
|
||||
| Test count reduced to 874 | ✓ | Verified in all 3 runs |
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `/home/phil/Projects/starpunk/tests/test_migration_race_condition.py`
|
||||
- `/home/phil/Projects/starpunk/tests/test_routes_feeds.py`
|
||||
- `/home/phil/Projects/starpunk/tests/test_routes_feed.py`
|
||||
|
||||
## Next Steps
|
||||
|
||||
Phase 0 is complete and ready for architect review. Once approved:
|
||||
- Commit changes with reference to ADR-012
|
||||
- Proceed to Phase 1 (Timestamp-Based Slugs)
|
||||
|
||||
## Notes
|
||||
|
||||
The test suite is now more reliable and maintainable:
|
||||
- Removed tests that cannot work reliably due to Python limitations
|
||||
- Fixed tests that checked implementation details instead of behavior
|
||||
- Improved test isolation and logger configuration
|
||||
- Clearer test names that reflect actual behavior being tested
|
||||
|
||||
All changes align with the project philosophy: "Every line of code must justify its existence." Tests that fail unreliably do not justify their existence.
|
||||
@@ -0,0 +1,242 @@
|
||||
# Phase 2 Architect Review: Debug File Management
|
||||
|
||||
**Date**: 2025-12-17
|
||||
**Reviewer**: Architect Agent
|
||||
**Phase**: 2 - Debug File Management
|
||||
**Status**: APPROVED
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Phase 2 implementation meets all acceptance criteria and demonstrates sound architectural decisions. The implementation follows the principle of secure defaults, properly sanitizes user-controlled input, and provides appropriate cleanup mechanisms to prevent resource exhaustion.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria Verification
|
||||
|
||||
| Criterion | Status | Notes |
|
||||
|-----------|--------|-------|
|
||||
| Debug files disabled by default | PASS | `DEBUG_SAVE_FAILED_UPLOADS` defaults to `False` |
|
||||
| Files older than 7 days auto-deleted | PASS | Configurable via `DEBUG_FILE_MAX_AGE_DAYS` |
|
||||
| Folder size limited to 100MB | PASS | Configurable via `DEBUG_FILE_MAX_SIZE_MB` |
|
||||
| Filenames sanitized (no path traversal) | PASS | Pattern: alphanumeric, `.`, `_`, `-` only, truncated to 50 chars |
|
||||
| Cleanup runs on startup | PASS | Called in `create_app()` before database init |
|
||||
| Tests cover all scenarios | PASS | 15 tests covering config, saving, sanitization, cleanup, startup |
|
||||
|
||||
---
|
||||
|
||||
## Code Review
|
||||
|
||||
### 1. Configuration (`starpunk/config.py`)
|
||||
|
||||
**Lines 100-103:**
|
||||
```python
|
||||
app.config["DEBUG_SAVE_FAILED_UPLOADS"] = os.getenv("DEBUG_SAVE_FAILED_UPLOADS", "false").lower() == "true"
|
||||
app.config["DEBUG_FILE_MAX_AGE_DAYS"] = int(os.getenv("DEBUG_FILE_MAX_AGE_DAYS", "7"))
|
||||
app.config["DEBUG_FILE_MAX_SIZE_MB"] = int(os.getenv("DEBUG_FILE_MAX_SIZE_MB", "100"))
|
||||
```
|
||||
|
||||
**Assessment**: Correct implementation.
|
||||
- Boolean parsing uses explicit comparison to "true" (secure)
|
||||
- Integer parsing uses sensible defaults
|
||||
- Configuration follows existing patterns in the file
|
||||
|
||||
### 2. Filename Sanitization (`starpunk/media.py`)
|
||||
|
||||
**Lines 141-143:**
|
||||
```python
|
||||
# Sanitize filename to prevent path traversal (v1.5.0 security fix)
|
||||
safe_filename = "".join(c for c in filename if c.isalnum() or c in "._-")[:50]
|
||||
debug_file = debug_dir / f"failed_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{safe_filename}"
|
||||
```
|
||||
|
||||
**Assessment**: Correct implementation of specified pattern.
|
||||
- Allowlist approach (only alphanumeric, `.`, `_`, `-`)
|
||||
- Truncation to 50 characters prevents filesystem issues
|
||||
- Path traversal attack vectors (`../`, `..\\`, etc.) are neutralized
|
||||
|
||||
**Security Note**: The pattern `"._-"` correctly allows:
|
||||
- `.` for file extensions
|
||||
- `_` for separators
|
||||
- `-` for common filename characters
|
||||
|
||||
The pattern removes:
|
||||
- `/` and `\` (path separators)
|
||||
- `:` (Windows drive letters)
|
||||
- `<>"|?*` (shell metacharacters)
|
||||
- Null bytes and other control characters
|
||||
|
||||
### 3. Cleanup Function (`starpunk/media.py`)
|
||||
|
||||
**Lines 785-857:**
|
||||
|
||||
**Assessment**: Well-designed two-phase cleanup algorithm.
|
||||
|
||||
**Phase 1 - Age-based cleanup:**
|
||||
- Correctly calculates cutoff date
|
||||
- Uses `timedelta` for date arithmetic
|
||||
- Removes from list while iterating (uses slice copy correctly)
|
||||
|
||||
**Phase 2 - Size-based cleanup:**
|
||||
- Correctly sums remaining file sizes
|
||||
- Deletes oldest first (proper FIFO approach)
|
||||
- Loop terminates when under limit or no files remain
|
||||
|
||||
**Edge cases handled:**
|
||||
- Non-existent directory (early return)
|
||||
- Empty directory (early return after glob)
|
||||
- OSError during deletion (logged, continues)
|
||||
|
||||
### 4. Startup Integration (`starpunk/__init__.py`)
|
||||
|
||||
**Lines 130-132:**
|
||||
```python
|
||||
# Clean up old debug files (v1.5.0 Phase 2)
|
||||
from starpunk.media import cleanup_old_debug_files
|
||||
cleanup_old_debug_files(app)
|
||||
```
|
||||
|
||||
**Assessment**: Correct placement.
|
||||
- After logging configuration (can log cleanup actions)
|
||||
- Before database initialization (no DB dependency)
|
||||
- Uses app instance directly (not current_app, which wouldn't exist yet)
|
||||
|
||||
### 5. Test Coverage
|
||||
|
||||
**15 tests in 5 test classes:**
|
||||
|
||||
| Class | Tests | Purpose |
|
||||
|-------|-------|---------|
|
||||
| `TestDebugFileConfiguration` | 4 | Verify config defaults and override |
|
||||
| `TestDebugFileSaving` | 2 | Verify conditional saving behavior |
|
||||
| `TestDebugFilenameSanitization` | 3 | Security: path traversal, special chars, truncation |
|
||||
| `TestDebugFileCleanup` | 5 | Age/size limits, edge cases |
|
||||
| `TestDebugFileStartupCleanup` | 1 | Integration: cleanup during app init |
|
||||
|
||||
**Assessment**: Comprehensive coverage of all requirements.
|
||||
|
||||
---
|
||||
|
||||
## Security Analysis
|
||||
|
||||
### Path Traversal Prevention
|
||||
|
||||
The sanitization pattern effectively prevents all known path traversal attacks:
|
||||
|
||||
| Attack Vector | Input | Sanitized Output |
|
||||
|---------------|-------|------------------|
|
||||
| Basic traversal | `../../../etc/passwd` | `...etcpasswd` |
|
||||
| Windows traversal | `..\\..\\..\system32` | `...system32` |
|
||||
| Encoded traversal | `%2e%2e%2f` | `2e2e2f` (%-encoded already decoded before reaching code) |
|
||||
| Null byte | `file\x00.jpg` | `file.jpg` |
|
||||
|
||||
### Default-Secure Configuration
|
||||
|
||||
The feature is disabled by default (`DEBUG_SAVE_FAILED_UPLOADS=false`), meaning:
|
||||
- Production deployments are protected without configuration
|
||||
- Debug files only appear when explicitly enabled
|
||||
- No accidental data retention in production
|
||||
|
||||
### Resource Exhaustion Prevention
|
||||
|
||||
Dual limits prevent disk exhaustion:
|
||||
- Age limit (7 days default): Prevents indefinite accumulation
|
||||
- Size limit (100MB default): Hard cap on disk usage
|
||||
- Both limits configurable for different environments
|
||||
|
||||
---
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Startup Overhead
|
||||
|
||||
The `cleanup_old_debug_files()` function runs on every app startup:
|
||||
|
||||
```python
|
||||
# Worst case: 1000 files in debug directory
|
||||
# - glob: O(n) directory listing
|
||||
# - sort: O(n log n)
|
||||
# - deletion: O(n) unlink calls
|
||||
|
||||
# Estimated overhead: ~100ms for 100 files, ~500ms for 1000 files
|
||||
```
|
||||
|
||||
**Assessment**: Acceptable. Startup is not a hot path, and the cleanup prevents greater issues.
|
||||
|
||||
**Recommendation**: Consider adding early exit if `DEBUG_SAVE_FAILED_UPLOADS=False` to skip cleanup entirely in production. However, this is a minor optimization and not required for approval.
|
||||
|
||||
---
|
||||
|
||||
## Architectural Alignment
|
||||
|
||||
### Project Philosophy
|
||||
|
||||
> "Every line of code must justify its existence."
|
||||
|
||||
This implementation follows the philosophy:
|
||||
- Minimal code (single sanitization pattern)
|
||||
- Single responsibility (cleanup function does one thing)
|
||||
- Secure defaults (disabled in production)
|
||||
- No unnecessary features
|
||||
|
||||
### Standards Compliance
|
||||
|
||||
The implementation follows established patterns:
|
||||
- Configuration uses existing `os.getenv()` pattern
|
||||
- Logging uses `app.logger`
|
||||
- File operations use `pathlib.Path`
|
||||
- Tests use pytest fixtures
|
||||
|
||||
---
|
||||
|
||||
## Minor Observations (Not Blocking)
|
||||
|
||||
1. **Config validation**: The new config options are not validated in `validate_config()`. Since they have sensible defaults and are non-critical, this is acceptable. However, negative values for MAX_AGE_DAYS or MAX_SIZE_MB could cause unexpected behavior.
|
||||
|
||||
2. **Logging verbosity**: When cleanup deletes many files, it logs a single summary message. This is appropriate for production but could mask issues during debugging.
|
||||
|
||||
3. **Race condition**: If multiple workers start simultaneously, they could attempt to delete the same file. The OSError handling covers this gracefully.
|
||||
|
||||
---
|
||||
|
||||
## Verdict
|
||||
|
||||
### APPROVED
|
||||
|
||||
Phase 2 is complete and meets all acceptance criteria. The implementation demonstrates:
|
||||
- Secure defaults
|
||||
- Proper input sanitization
|
||||
- Comprehensive test coverage
|
||||
- Good error handling
|
||||
- Alignment with project architecture
|
||||
|
||||
**The developer may proceed to Phase 3: N+1 Query Fix (Feed Generation).**
|
||||
|
||||
---
|
||||
|
||||
## Checklist for Project Plan Update
|
||||
|
||||
The following items from RELEASE.md Phase 2 are complete:
|
||||
|
||||
- [x] Configuration options added (DEBUG_SAVE_FAILED_UPLOADS, DEBUG_FILE_MAX_AGE_DAYS, DEBUG_FILE_MAX_SIZE_MB)
|
||||
- [x] Cleanup logic implemented in starpunk/media.py
|
||||
- [x] Config check before saving debug files
|
||||
- [x] `cleanup_old_debug_files()` function implemented
|
||||
- [x] Startup cleanup integrated in `__init__.py`
|
||||
- [x] Filename sanitization pattern implemented
|
||||
- [x] Tests cover all scenarios (15 tests)
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Test Verification
|
||||
|
||||
```
|
||||
$ uv run pytest tests/test_debug_file_management.py -v
|
||||
================================ 15 passed in 0.72s ============================
|
||||
|
||||
$ uv run pytest tests/test_media*.py -v
|
||||
================================ 33 passed in 6.16s ============================
|
||||
```
|
||||
|
||||
All tests pass consistently.
|
||||
Loaded 100 of 334 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user