Compare commits

...
12 Commits
Author SHA1 Message Date
Phil 88aea09e04 Merge branch 'fix/v2.14.1-tags-captions' for v2.14.1 release
Build Container / build (push) Successful in 9m6s
2026-11-10 15:37:43 -07:00
PhilandClaude Opus 5.5 7e2cdef150 fix: clear all tags from the editor; escape captions in feed HTML (v2.14.1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:37:35 -07:00
Phil 058b1959f9 Merge branch 'docs/v2.14.0-doc-review': documentation review for v2.14.0 2026-11-10 15:35:00 -07:00
PhilandClaude Opus 5.5 ceef048003 docs: documentation review for v2.14.0
Architecture, README, standards, CLAUDE.md, settings reference, indexes,
backlog, release plans and CHANGELOG checked against the code at v2.14.0:
backfeed, notifications, counts line and web actions documented; test
policy (changed area locally, full suite in CI); stale versions, statuses
and links fixed; legacy and superseded docs moved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 14:51:57 -07:00
Phil caa66f7bb1 Merge branch 'feat/v2.14.0-notifications' for v2.14.0 release
Build Container / build (push) Successful in 9m15s
2026-11-10 14:30:10 -07:00
PhilandClaude Opus 5.5 0465ff924b feat: notifications through Apprise (v2.14.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 14:30:01 -07:00
Phil 815aff5443 Merge branch 'docs/v2.14.0-notifications-adr': ADR-074 notifications 2026-11-10 14:25:58 -07:00
PhilandClaude Opus 5.5 4306633a42 docs: ADR-074 accepted
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 14:25:57 -07:00
PhilandClaude Opus 5.5 c5c2d9832e docs: ADR-074: one configurable priority for all messages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 14:24:43 -07:00
PhilandClaude Opus 5.5 483c24b565 docs: ADR-074 notifications through Apprise (proposed)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 14:17:30 -07:00
Phil d312fa0074 Merge branch 'fix/v2.13.4-list-counts' for v2.13.4 release
Build Container / build (push) Successful in 8m56s
2026-11-10 14:05:05 -07:00
PhilandClaude Opus 5.5 647f6c723b feat: counts on lists of notes, reply cards link to the reply, fix class clash (v2.13.4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 14:05:03 -07:00
73 changed files with 1616 additions and 269 deletions

No files matched your search

+24 -5
View File
@@ -1,4 +1,4 @@
# StarPunk configuration template (v2.10.1)
# StarPunk configuration template (v2.14.0)
#
# Copy to .env and fill in your values. DO NOT commit .env.
# Every variable here is read by starpunk/config.py. Uncommented lines are
@@ -124,10 +124,11 @@ CSRF_ENABLED=true
# WEBMENTION
# =============================================================================
# Receive webmentions at /webmention (default: true)
# Webmention endpoint at /webmention, its Link header, and showing responses
# (webmentions and backfeed) on notes; sending is not affected (default: true)
# WEBMENTION_ENABLED=true
# Background worker that verifies received webmentions and sends outgoing
# ones (default: true)
# Background worker that verifies received webmentions, sends outgoing ones,
# and sends response notifications (v2.14.0) (default: true)
# WEBMENTION_WORKER_ENABLED=true
# Seconds between queue checks (default: 10) and after an error (default: 60)
# WEBMENTION_POLL_INTERVAL=10
@@ -201,11 +202,29 @@ THREADS_USERNAME=
# SYNDICATION_POLL_INTERVAL=10
# Backfeed (v2.13.0, ADR-073): bring likes, reposts, quotes and replies to the
# crossposts back to the note as responses. Needs the read scopes above.
# crossposts back to the note as responses. Needs the read scopes above, and
# runs in the syndication worker, so at least one target must be configured.
# BACKFEED_ENABLED=false
# Stop checking a note's responses after this many days (default: 90)
# BACKFEED_MAX_AGE_DAYS=90
# =============================================================================
# NOTIFICATIONS (v2.14.0, ADR-074; optional)
# =============================================================================
# Apprise URLs to notify, space- or comma-separated
# (https://github.com/caronc/apprise/wiki). Empty: notifications off.
# Examples: gotifys://gotify.example.com/APPTOKEN ntfys://my-topic
# apprises://apprise.example.com/KEY (an apprise-api server)
# NOTIFY_URLS=
# What to notify: reply, mention, quote, like, repost, bookmark, problems
# (a crosspost that failed for good, a silo refusing backfeed). Names are
# case-insensitive; unknown ones are ignored.
# NOTIFY_EVENTS=reply,mention,quote,problems
# Priority of every message, added to URLs that don't set one:
# low, moderate, normal, high, emergency (Gotify also takes 0-10)
# NOTIFY_PRIORITY=normal
# =============================================================================
# DEBUGGING
# =============================================================================
+39 -5
View File
@@ -7,6 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [2.14.1] - 2026-11-10
### Fixed
- Clearing the Tags field in the note editor removes all of the note's tags. It used to leave them unchanged, so a note's last tags couldn't be removed from the editor
- Image captions are escaped in the HTML of RSS, Atom and JSON Feed items. A caption containing `"`, `<` or `&` broke the item's image markup (the Atom enclosure's title was already escaped)
## [2.14.0] - 2026-11-10
Notifications (ADR-074).
### Added
- StarPunk can notify you of new responses through [Apprise](https://github.com/caronc/apprise): Gotify, ntfy, Pushover, email, Matrix, Discord and about 100 more, or an apprise-api server. Set `NOTIFY_URLS` to one or more Apprise URLs (e.g. `gotifys://gotify.example.com/APPTOKEN`)
- `NOTIFY_EVENTS` chooses what you hear about: `reply`, `mention`, `quote`, `like`, `repost`, `bookmark`, and `problems` (a crosspost that failed for good, or a silo refusing backfeed). Default `reply,mention,quote,problems`. Replies, mentions and quotes come one at a time with their text; likes, reposts and bookmarks are grouped per note
- `NOTIFY_PRIORITY` (default `normal`) sets the priority of every message
- Received webmentions and backfed responses are notified alike, each once. Responses waiting for moderation say so and link to the admin
- Admin → Webmentions shows the notification settings and the last send error, with **Send test notification**
### Changed
- New dependency: `apprise` 2.2. Migration 021 marks every existing response as already notified, and responses found by a backfill (or on a crosspost's first check of an older note) are never notified
## [2.13.4] - 2026-11-10
### Added
- The home page, tag pages and the Likes page show each note's like, repost and reply counts next to its date (♥ 3 🔁 1 💬 2), linking to the note's responses. Notes with none show nothing
### Changed
- Clicking anywhere on a reply opens it where it was posted (the reply on Bluesky, GoToSocial or Threads, or the page that sent the webmention). The author link and links in the reply still go to their own pages
### Fixed
- The "Liked", "Reposted" and "In reply to" lines on notes picked up styles meant for v2.13.2's counts line (both used the class `reaction`). The counts line's classes are now `response-*`
## [2.13.3] - 2026-11-10
### Changed
@@ -604,7 +641,7 @@ Webmention now follows the [W3C Recommendation](https://www.w3.org/TR/webmention
- Successful uploads logged at INFO level with file details
- Removed duplicate logging in Micropub media endpoint
## [1.4.0rc1] - 2025-12-16
## [1.4.0-rc.1] - 2025-12-16
### Added
@@ -2147,7 +2184,4 @@ This is a release candidate for testing. Stable 1.0.0 will be released after tes
- ADR-007: Slug generation algorithm
- ADR-008: Versioning strategy
[Unreleased]: https://github.com/YOUR_USERNAME/starpunk/compare/v0.4.0...HEAD
[0.4.0]: https://github.com/YOUR_USERNAME/starpunk/compare/v0.3.0...v0.4.0
[0.3.0]: https://github.com/YOUR_USERNAME/starpunk/compare/v0.1.0...v0.3.0
[0.1.0]: https://github.com/YOUR_USERNAME/starpunk/releases/tag/v0.1.0
Releases are tagged `vX.Y.Z` in https://git.thesatelliteoflove.com/phil/StarPunk
+3 -3
View File
@@ -8,7 +8,7 @@ This file contains operational instructions for Claude agents working on this pr
- All Python commands must be run with `uv run` prefix
- Example: `uv run pytest`, `uv run flask --app app.py run --debug`
- There is no `pyproject.toml`: create the environment with `uv venv --python 3.14 && uv pip install -r requirements-dev.txt`
- Tests need `SESSION_SECRET` set: `SESSION_SECRET=test uv run pytest`. The full suite takes about 6 minutes, 5 of them in `tests/test_migration_race_condition.py`; add `--ignore=tests/test_migration_race_condition.py` while iterating, but run everything before a release
- Tests need `SESSION_SECRET` set. Locally, run only the tests for the area you changed (`SESSION_SECRET=test uv run pytest tests/test_<area>.py`); CI runs the full suite (about 6 minutes) on every `vX.Y.Z` tag, so don't run the whole suite yourself unless asked
- Local login without OIDC: `DEV_MODE=true` and `DEV_ADMIN_ME` in `.env`, then `/dev/login`
- Setup details: `docs/standards/development-setup.md`
@@ -50,12 +50,12 @@ When invoking the agent-developer, always remind it to:
5. **Follow git protocol**
- Adhere to git branching strategy in `docs/standards/git-branching-strategy.md`
- Create feature branches for non-trivial changes, in their own git worktree (`<prefix>/vX.Y.Z-description`, prefixes `feat/`, `fix/`, `ci/`, `docs/`)
- Create feature branches for non-trivial changes, in their own git worktree (`<prefix>/vX.Y.Z-description`, prefixes `feat/`, `fix/`, `ci/`, `chore/`, `docs/`)
- Write clear commit messages (`feat: ... (vX.Y.Z)`)
## Release Process
Follow "Release process" in `docs/standards/git-branching-strategy.md`. In short: work in a worktree on a `feat/`/`fix/` branch → bump `__version__` and `__version_info__` → `CHANGELOG.md` entry → design note in `docs/design/vX.Y.Z/` (or ADR amendment) → full test suite → commit → `git merge --no-ff` into `main` ("Merge branch '<branch>' for vX.Y.Z release") → annotated tag `vX.Y.Z` → push `main` and the tag. CI (`.gitea/workflows/build-container.yml`) runs the tests and publishes the image; production is then updated from the separate Ansible deployment repository. Back up the database before deploying a release with a migration.
Follow "Release process" in `docs/standards/git-branching-strategy.md`. In short: work in a worktree on a `feat/`/`fix/`/`chore/` branch → bump `__version__` and `__version_info__` → `CHANGELOG.md` entry → design note in `docs/design/vX.Y.Z/` (or ADR amendment) → tests for the changed area → commit → `git merge --no-ff` into `main` ("Merge branch '<branch>' for vX.Y.Z release") → annotated tag `vX.Y.Z` → push `main` and the tag. CI (`.gitea/workflows/build-container.yml`) runs the full test suite and publishes the image; production is then updated from the separate Ansible deployment repository. Back up the database before deploying a release with a migration.
Merging to `main`, tagging, pushing and deploying happen only when the project owner asks for a release.
+15 -8
View File
@@ -24,8 +24,9 @@ what changed in each release is in [CHANGELOG.md](CHANGELOG.md).
Micropub apps (consent screen, tokens, revocation, Connected Apps page)
- Admin login through any OpenID Connect provider (e.g. Pocket ID)
- [Webmention](https://www.w3.org/TR/webmention/): receiving (verified,
moderated, rate-limited) and sending; replies, likes and reposts shown on
each note
moderated, rate-limited) and sending; replies, quotes and mentions shown
on each note, with likes, reposts and replies as a line of counts with
hover cards and `<indie-action>` web actions
- [WebSub](https://www.w3.org/TR/websub/) with a built-in hub, so readers get
new posts within seconds
- [Microformats2](https://microformats.org/wiki/h-entry): h-entry, h-feed and
@@ -38,10 +39,15 @@ what changed in each release is in [CHANGELOG.md](CHANGELOG.md).
- POSSE crossposting to GoToSocial/Mastodon, Bluesky and Threads, with
`u-syndication` links back; edits and deletes follow, native likes and
boosts where the target supports them
- Backfeed: likes, reposts and replies to crossposts on GoToSocial and
Bluesky (and Bluesky quotes), and replies on Threads (with its like and
repost counts), come back as responses (`BACKFEED_ENABLED`, ADR-073)
**Operations**
- Migrations applied automatically at startup
- `/health` endpoint, `/admin/health` diagnostics, structured logs
- Notifications of new responses and crossposting problems through
Apprise (Gotify, ntfy, email…; `NOTIFY_URLS`, ADR-074)
- Fetches of user-supplied URLs are limited to public addresses, size and
time
@@ -54,7 +60,6 @@ git clone https://git.thesatelliteoflove.com/phil/StarPunk.git && cd StarPunk
cp .env.example .env # set SITE_URL, ADMIN_ME, SESSION_SECRET, OIDC_*
docker build -t starpunk -f Containerfile .
docker run -d --name starpunk -p 127.0.0.1:8000:8000 --env-file .env \
-e FLASK_ENV=production -e FLASK_DEBUG=0 \
-v ./starpunk-data:/data starpunk
```
@@ -72,8 +77,8 @@ redirect URI with your OIDC provider. Details:
uv venv --python 3.14
uv pip install -r requirements-dev.txt
cp .env.example .env # set SESSION_SECRET; add DEV_MODE=true and DEV_ADMIN_ME
uv run flask --app app.py run --debug # http://localhost:5000
SESSION_SECRET=test uv run pytest # tests (about 6 minutes)
uv run flask --app app.py run --debug # http://localhost:5000
SESSION_SECRET=test uv run pytest tests/test_<area>.py # tests for what you changed; CI runs the full suite
```
With `DEV_MODE=true`, `/dev/login` logs you in without OIDC. Never enable
@@ -86,13 +91,15 @@ All settings are environment variables (or a `.env` file), each listed with
its default in [`.env.example`](.env.example). Required in production:
`SESSION_SECRET`, `SITE_URL`, `ADMIN_ME`, `OIDC_ISSUER`, `OIDC_CLIENT_ID`,
`OIDC_CLIENT_SECRET` and `OIDC_ADMIN_SUB`. Crossposting is enabled by
setting a target's credentials.
setting a target's credentials. Backfeed is off unless
`BACKFEED_ENABLED=true`; notifications start when `NOTIFY_URLS` is set.
## Data and backups
Everything lives in the data directory (`/data` in the container): the
SQLite database, note files in `notes/`, uploaded media and logs. Back it
up regularly, and always before upgrading to a release with a migration.
SQLite database, note files in `notes/`, uploaded media, saved response
avatars in `avatars/`, the Threads token (`threads_token.json`) and logs.
Back it up regularly, and always before upgrading to a release with a migration.
## Documentation
+8 -7
View File
@@ -1,9 +1,11 @@
# Architecture Documentation Index
This directory describes the **current** design of StarPunk (v2.10.1). Per
This directory describes the **current** design of StarPunk (v2.14.0). Per
[../DOCUMENTATION.md](../DOCUMENTATION.md) it is updated at the end of each
release. Decisions and their history are in [../decisions/](../decisions/);
per-release designs and reports are in [../design/](../design/).
per-release designs and reports are in [../design/](../design/). The
original v0.x technology choices are kept in
[../design/v0.x/technology-stack-legacy.md](../design/v0.x/technology-stack-legacy.md).
## Documents
@@ -11,12 +13,11 @@ per-release designs and reports are in [../design/](../design/).
|----------|-------------|
| [overview.md](overview.md) | System overview: purpose, request flow, modules and routes, background workers, storage, auth model, URL surface, caching, security posture |
| [technology-stack.md](technology-stack.md) | Current runtime, Python dependencies, standards implemented, testing and CI |
| [database.md](database.md) | SQLite schema after migrations 001–019, migration runner and its lock, connection pool |
| [database.md](database.md) | SQLite schema after migrations 001–021, migration runner and its lock, connection pool |
| [deployment.md](deployment.md) | Container image stages, CI workflow, release process, `/data` layout, reverse proxy, all environment variables |
| [simplified-auth-architecture.md](simplified-auth-architecture.md) | Admin login via OpenID Connect and the built-in IndieAuth server that issues Micropub tokens (ADR-064) |
| [indieweb-protocols.md](indieweb-protocols.md) | How Micropub, IndieAuth, Webmention, microformats2, WebSub and the feeds are implemented, with compliance status and known gaps |
| [syndication-architecture.md](syndication-architecture.md) | Crossposting (POSSE) to GoToSocial, Bluesky and Threads: syndications table, worker, reconcile rules |
| [technology-stack-legacy.md](technology-stack-legacy.md) | Historical: the original v0.x technology choices (IndieLogin era); not current |
| [indieweb-protocols.md](indieweb-protocols.md) | How Micropub, IndieAuth, Webmention, microformats2, WebSub and the feeds are implemented (including backfeed responses, web actions and notifications), with compliance status and known gaps |
| [syndication-architecture.md](syndication-architecture.md) | Crossposting (POSSE) to GoToSocial, Bluesky and Threads: syndications table, worker, reconcile rules, backfeed of responses (ADR-073) |
## Where to start
@@ -37,4 +38,4 @@ per-release designs and reports are in [../design/](../design/).
---
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
+26 -12
View File
@@ -1,8 +1,8 @@
# Database Architecture
**Version**: 2.11.0 (2026-11-03)
**Version**: 2.14.0 (2026-11-10)
**Status**: Current. Verified against a database created by `create_app()`
with migrations 001–019.
with migrations 001–021.
StarPunk uses one SQLite file, `starpunk.db` under `DATA_PATH` (`/data` in
the container), in WAL mode. Note *content* is not in the database: it lives
@@ -73,6 +73,8 @@ Background: [ADR-031](../decisions/ADR-031-database-migration-system-redesign.md
| `017_websub_hardening.sql` | 2.10.1 | `websub_intents.request_id`, `requested_topic`, `callback_ip`; `websub_subscriptions.callback_ip` |
| `018_foreign_key_cleanup.sql` | 2.11.0 | Removes rows orphaned while foreign keys weren't enforced |
| `019_shared_rate_limits.sql` | 2.11.0 | `rate_limits` (counters shared by all processes) |
| `020_backfeed.sql` | 2.13.0 | Rebuilds `webmentions` adding `origin`, `remote_id`, the `quote` type and an index on (`target_note_id`, `origin`); `syndications.backfeed_next_at`, `backfeed_checked_at`, `backfeed_claimed_until`, `backfeed_attempts`, `backfeed_error`, `backfeed_backfill`, `likes_count`, `reposts_count`, `quotes_count` |
| `021_notifications.sql` | 2.14.0 | `webmentions.notified_at`; existing rows are marked as notified |
New migrations are numbered `NNN_description.sql`, must be safe to run inside
one transaction, and should use `IF NOT EXISTS` where SQLite allows it.
@@ -143,17 +145,20 @@ above.
### IndieWeb interactions
**`webmentions`**: received webmentions, one per (`source_url`,
`target_url`).
**`webmentions`**: received webmentions and backfed silo responses
(v2.13.0), one per (`source_url`, `target_url`).
| Column | Purpose |
|--------|---------|
| `target_note_id` | The note mentioned (SET NULL if the note is deleted) |
| `mention_type` | `reply`, `like`, `repost`, `bookmark`, `mention` |
| `mention_type` | `reply`, `like`, `repost`, `bookmark`, `mention`, `quote` |
| `author_name`, `author_url`, `author_photo`, `content_snippet` | From the source's mf2 (content sanitised) |
| `source_post_url`, `source_published` | The source entry's own `u-url` and `dt-published` |
| `validation_status` | `pending`, `valid`, `invalid`, `error` (set by the worker) |
| `moderation_status` | `pending`, `approved`, `hidden`, `deleted` (auto-approved when `WEBMENTION_AUTO_APPROVE`) |
| `moderation_status` | `pending`, `approved`, `hidden`, `deleted` (auto-approved when `WEBMENTION_AUTO_APPROVE`). `deleted` is a tombstone: deleting in the admin keeps the row, so a re-sent webmention or the next backfeed check doesn't bring it back; re-validation keeps `hidden` and `deleted` |
| `origin` | `webmention`, or the silo it was backfed from (`gotosocial`, `bluesky`, `threads`) |
| `remote_id` | Silo id of a backfed response (or of the person, for likes and reposts) |
| `notified_at` | When the owner was notified of it (v2.14.0); NULL means not yet |
| `validation_error`, `validated_at`, `received_at`, `updated_at` | Bookkeeping |
**`webmention_queue`**: outgoing webmentions, one per (`source_url`,
@@ -178,7 +183,12 @@ the note should be on that target; `remote_id` and `remote_url` identify the
remote post. `state` (`pending`, `synced`, `failed`), `version` (bumped on
every note change, so an edit made during a send is not lost), `attempts`,
`last_error`, `next_attempt_at` and `claimed_until` (a 5-minute lease that
lets any gunicorn process take the row exclusively) drive the worker. See
lets any gunicorn process take the row exclusively) drive the worker.
Backfeed (v2.13.0) adds `backfeed_next_at`, `backfeed_checked_at`,
`backfeed_claimed_until` (its own lease), `backfeed_attempts`,
`backfeed_error` and `backfeed_backfill` (set while a backfill is due), and
`likes_count`, `reposts_count`, `quotes_count`: silo-only counts (all of
Threads; elsewhere people who opted out). See
[syndication-architecture.md](syndication-architecture.md).
### WebSub
@@ -201,7 +211,10 @@ pending row of each kind. `attempts`, `next_attempt_at`, `last_error`.
**`site_state`**: key/value pairs. `content_version` is bumped whenever a
public note changes; every gunicorn process compares it to decide whether
its cached feed data is stale.
its cached feed data is stale. Backfeed keeps `backfeed_backfill_last` (spacing
backfill checks across processes), `backfeed_backfill_started` and
`backfeed_backfill_finished`; notifications keep `notify_last_error` (shown
in the admin, cleared by a successful send).
### Bookkeeping
@@ -223,7 +236,8 @@ server process enforces one shared limit. Old windows are pruned as it goes.
it goes back to the pool, rolled back first if a transaction was left
open.
- **Direct connections**: the migration runner, the FTS start-up check, the
webmention worker and sender, reply-context fetching and search helpers
webmention worker and sender, the notification sweep (`notifications.py`),
avatar upkeep (`avatars.py`), reply-context fetching and search helpers
open their own `sqlite3.connect(db_path)` connections.
- WAL mode lets readers carry on while one writer writes. The four gunicorn
processes and the worker threads share the one file; writes are short, so
@@ -233,9 +247,9 @@ server process enforces one shared limit. Old windows are pruned as it goes.
The application does not back itself up. A consistent copy needs the
database (use SQLite's online backup, e.g. `sqlite3 starpunk.db ".backup …"`,
because of WAL) together with `notes/` and `media/`; see
[deployment.md](deployment.md).
because of WAL) together with `notes/` and `media/`. `avatars/` is optional:
missing avatars are fetched again. See [deployment.md](deployment.md).
---
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
+31 -23
View File
@@ -1,6 +1,6 @@
# Deployment Architecture
**Version**: 2.10.1 (2026-11-03)
**Version**: 2.14.0 (2026-11-10)
**Status**: Current
How StarPunk is built, configured and run in production. The production site
@@ -70,21 +70,18 @@ authentication.
lower case.
4. Remove the builder and log out, even after a failure.
There is no CI on branches or pull requests; run the tests locally before
merging (see [technology-stack.md](technology-stack.md#testing)).
There is no CI on branches or pull requests. Locally, run the tests for the
area you changed; CI runs the full suite on the tag (see
[technology-stack.md](technology-stack.md#testing)).
## Release process
1. Finish the change on a branch, including the version bump in
`starpunk/__init__.py` (`__version__`, `__version_info__`) and a
`CHANGELOG.md` entry.
2. Merge to `main`.
3. Tag the merge commit `vX.Y.Z` and push the tag. CI tests, builds and
pushes the image.
4. In the Ansible-managed deployment, change the StarPunk image tag to the new
version and apply it. The container is recreated from the new image; new
migrations run automatically at start-up.
5. Check `/health` reports the new version.
Branching, version bump, changelog, merge and tagging are in
[git-branching-strategy.md](../standards/git-branching-strategy.md#release-process).
Pushing a `vX.Y.Z` tag makes CI test, build and push the image. Then, in the
Ansible-managed deployment, set `starpunk_version` to the new tag and deploy
the `starpunk` service: the container is recreated from the new image and new
migrations run at start-up. Check that `/health` reports the new version.
Rolling back means deploying the previous tag. Migrations are not reversed
automatically, so a release with a migration needs a database backup first
@@ -105,6 +102,7 @@ persistent is there; the image itself holds no state.
| `media/YYYY/MM/` | Uploaded images and their variants; `media/.tmp/` while processing |
| `logs/starpunk.log` | Application log, rotated at 10 MB, 10 files kept |
| `webmention-worker.lock`, `websub-worker.lock` | `flock` files choosing the process that runs each worker |
| `avatars/<sha256>.webp` | Saved copies of response authors' avatars (v2.13.0), served at `/avatars/`; unused ones removed weekly |
| `threads_token.json` (+ `.lock`) | Current Threads access token, refreshed daily (only with Threads configured) |
| `debug/` | Failed uploads, only with `DEBUG_SAVE_FAILED_UPLOADS=true`; cleaned up after `DEBUG_FILE_MAX_AGE_DAYS` |
@@ -161,6 +159,7 @@ unless noted.
| `INDIEAUTH_TOKEN_LIFETIME_DAYS` | `90` | Lifetime of Micropub access tokens |
| `DEV_MODE` | `false` | Enables `/dev/login`; never in production |
| `DEV_ADMIN_ME` | empty | Required when `DEV_MODE=true` |
| `CSRF_ENABLED` | `true` | CSRF tokens on admin forms, logout and IndieAuth consent; leave on |
`TOKEN_ENDPOINT` and `INDIELOGIN_URL` are v1 settings; if present they are
ignored with a warning.
@@ -190,8 +189,8 @@ ignored with a warning.
| Variable | Default | Notes |
|----------|---------|-------|
| `WEBMENTION_ENABLED` | `true` | Endpoint, `Link` header and sending |
| `WEBMENTION_WORKER_ENABLED` | `true` | |
| `WEBMENTION_ENABLED` | `true` | Endpoint, `Link` header, showing responses on notes (not sending) |
| `WEBMENTION_WORKER_ENABLED` | `true` | Verifies received and sends outgoing webmentions; saves avatars; sends response notifications |
| `WEBMENTION_POLL_INTERVAL` | `10` | Seconds |
| `WEBMENTION_ERROR_BACKOFF` | `60` | Seconds after a worker error |
| `WEBMENTION_BATCH_SIZE` | `5` | Items per pass |
@@ -219,7 +218,7 @@ starts only if at least one is.
| Variable | Default | Notes |
|----------|---------|-------|
| `GOTOSOCIAL_URL`, `GOTOSOCIAL_ACCESS_TOKEN` | empty | GoToSocial or any Mastodon-API server |
| `GOTOSOCIAL_URL`, `GOTOSOCIAL_ACCESS_TOKEN` | empty | GoToSocial or any Mastodon-API server. Token scopes: `write:statuses write:media read:search write:favourites`, plus `read:statuses read:accounts` (or `read`) for backfeed |
| `GOTOSOCIAL_NAME` | `GoToSocial` | Label in the UI and `syndicate-to` |
| `GOTOSOCIAL_VISIBILITY` | `public` | |
| `GOTOSOCIAL_MAX_CHARS` | `5000` | |
@@ -228,12 +227,22 @@ starts only if at least one is.
| `BLUESKY_SERVICE` | `https://bsky.social` | |
| `BLUESKY_NAME` | `Bluesky` | |
| `BLUESKY_MAX_CHARS` | `300` | |
| `THREADS_ACCESS_TOKEN` | empty | Long-lived token; refreshed automatically into `threads_token.json` |
| `THREADS_ACCESS_TOKEN` | empty | Long-lived token; refreshed automatically into `threads_token.json`. Backfeed also needs `threads_read_replies` and `threads_manage_insights` |
| `THREADS_USERNAME` | empty | |
| `THREADS_NAME` | `Threads` | |
| `THREADS_MAX_CHARS` | `500` | |
| `THREADS_MAX_MEDIA` | `4` | |
| `SYNDICATION_POLL_INTERVAL` | `10` | Seconds |
| `BACKFEED_ENABLED` | `false` | Bring responses to crossposts back to notes (v2.13.0); needs read scopes on the GoToSocial and Threads tokens |
| `BACKFEED_MAX_AGE_DAYS` | `90` | Days after posting to keep checking a crosspost's responses |
### Notifications
| Variable | Default | Notes |
|----------|---------|-------|
| `NOTIFY_URLS` | empty (off) | Apprise URLs, space- or comma-separated (v2.14.0) |
| `NOTIFY_EVENTS` | `reply,mention,quote,problems` | Also `like`, `repost`, `bookmark` |
| `NOTIFY_PRIORITY` | `normal` | Added to URLs that don't set a priority |
### Debugging
@@ -252,15 +261,14 @@ from the environment.
## Running locally
```bash
uv venv && uv pip install -r requirements-dev.txt
uv venv --python 3.14 && uv pip install -r requirements-dev.txt
cp .env.example .env # set SESSION_SECRET; DEV_MODE=true and DEV_ADMIN_ME for local login
flask --app app.py run # or: gunicorn app:app
uv run flask --app app.py run --debug # or: uv run gunicorn app:app
```
`compose.yaml` builds and runs the image with `./container-data` mounted at
`/data`; its image tag and `VERSION` values are out of date, so override them
or build with `podman build -t starpunk .`.
`compose.yaml` builds the image from this checkout (`starpunk:local`) and runs
it with `./container-data` mounted at `/data`.
---
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
+39 -14
View File
@@ -1,6 +1,6 @@
# IndieWeb and W3C Protocols
How StarPunk implements each protocol it speaks, as of v2.10.1, with its
How StarPunk implements each protocol it speaks, as of v2.14.0, with its
compliance status and the gaps that remain. Each section links to the
decision records and review documents behind it.
@@ -12,6 +12,8 @@ decision records and review documents behind it.
| [Microformats2](#microformats2) | Publisher and consumer | Reviewed in v2.8.0 | `templates/`, `starpunk/mf2_utils.py`, `starpunk/author_discovery.py` |
| [WebSub](#websub) | Publisher and built-in hub (own topics only) | Adversarial review in v2.10.1 | `starpunk/websub.py`, `starpunk/websub_worker.py`, `starpunk/routes/websub.py` |
| [Feeds](#feeds) | RSS 2.0, Atom 1.0, JSON Feed 1.1, OPML 2.0 | Stable | `starpunk/feeds/`, `starpunk/routes/public.py` |
| Backfeed | Responses to crossposts read from the silo APIs and stored as webmentions (v2.13.0); see [syndication-architecture.md](syndication-architecture.md#backfeed-v2130-adr-073) | ADR-073 | `starpunk/backfeed.py` |
| [Web actions](#moderation-and-display) | `<indie-action>` on the like, repost and reply counts (v2.13.2) | Markup only | `templates/partials/responses.html` |
Crossposting to other networks (POSSE) is covered in
[syndication-architecture.md](syndication-architecture.md).
@@ -19,8 +21,10 @@ Crossposting to other networks (POSSE) is covered in
## Shared infrastructure: the SSRF-safe fetcher
Webmention (fetching sources, discovering endpoints, sending), WebSub
(verification and delivery) and Micropub (downloading remote photos) all
make requests to URLs chosen by someone else. They use
(verification and delivery), Micropub (downloading remote photos), IndieAuth
(client pages), reply contexts, crosspost link cards (v2.11.0) and response
avatars (v2.13.0; except those on StarPunk's own GoToSocial) all make
requests to URLs chosen by someone else. They use
`starpunk/webmention_ssrf.py`:
- `validate_url_for_fetch()`: http(s) only. Every address from
@@ -128,7 +132,7 @@ repost, bookmark.
| `q=syndicate-to` | any token | `{"syndicate-to": [{"uid", "name"}]}` |
| `q=source&url=...[&properties[]=...]` | create/update/delete/draft | One post as mf2 JSON. Content comes back as `{"html": ...}` if it was authored as HTML. Includes `syndication` URLs and `post-status` |
| `q=source` without `url` | same | `{"items": [...]}`, drafts included, `limit` (default 10, max 50) and `offset` |
| `q=category[&filter=...]` | same | Tags in use, most used first |
| `q=category[&filter=...]` | same | Tags on notes that aren't deleted (drafts count), most used first (v2.12.1) |
Unknown or missing `q` gets 400.
@@ -171,8 +175,7 @@ Full description, scopes and gaps:
Decision: [ADR-064](../decisions/ADR-064-oidc-admin-login-and-builtin-indieauth-server.md).
Known gaps: `scopes_supported` doesn't list `draft`. There are no
introspection, userinfo or refresh tokens. The client metadata fetch
doesn't use the pinned SSRF fetcher.
introspection, userinfo or refresh tokens.
---
@@ -239,25 +242,46 @@ Verification is asynchronous (`WebmentionWorker`, poll
- Valid: `validation_status = 'valid'`. A first-time mention gets
`moderation_status = 'approved'` if `WEBMENTION_AUTO_APPROVE` (default
true), otherwise `pending`. An update keeps the admin's earlier
moderation decision.
moderation decision, including `hidden` and `deleted`.
- Source returns 4xx (including 410), or no longer links: `invalid`,
which hides it (3.2.4 deletion).
- Network error, timeout or 5xx: a mention that was valid before stays
valid with its old data. A new one becomes `error`.
- A source that is one of the note's own syndicated copies
(`syndications.remote_url`) is `invalid` (v2.8.0).
- A source that backfeed has already stored for the note is `invalid`
("Already backfed from the silo", v2.13.0).
- **Re-send on display change** (3.1.4, v2.7.2): if what the note shows
for this webmention changed, the note's own outgoing webmentions are
re-queued. That covers a new response, a changed author or snippet, or a
change of shown/hidden. Unchanged re-validations send nothing, so two
sites doing the same don't ping-pong.
**Moderation and display.** `/admin/webmentions` lists, filters, and
approves, hides or deletes mentions, one at a time or in bulk. Changes that
alter what is shown also re-send the note's webmentions. Note pages show
approved, valid mentions grouped as replies (`u-comment h-cite`), likes,
reposts and bookmarks (facepiles), and mentions. See
[Microformats2](#microformats2).
### Moderation and display
`/admin/webmentions` lists, filters (by status, including Deleted, and by
origin), and approves, hides or deletes responses, one at a time or in bulk;
it also has the Backfill button. Deleting keeps the row as a `deleted`
tombstone, so a re-sent webmention or backfeed doesn't bring it back.
Changes that alter what is shown also re-send the note's webmentions.
Note pages show approved, valid responses. Under the note, a line of counts
(♥ likes, 🔁 reposts, 💬 replies, bookmarks) opens a card on hover or focus
listing who responded and where from, including anonymous silo counts. Each
like, repost and reply count holds an `<indie-action do with>` web action
whose fallback links to the note's crossposts (v2.13.2). The Responses
section below holds replies (`u-comment h-cite`), quotes and mentions;
backfed ones say "via <silo>". See [Microformats2](#microformats2).
### Notifications (v2.14.0, ADR-074)
With `NOTIFY_URLS` set, new valid responses (received and backfed) are sent
to the owner once through Apprise by a sweep in the webmention worker, which
sets `webmentions.notified_at` (migration 021). Likes, reposts and bookmarks
are grouped per note; which kinds are sent is set by `NOTIFY_EVENTS`
(default replies, mentions, quotes and problems). Items still unsent after
24 hours are dropped. Responses found by a backfill, or by a copy's first
check on a note older than a day, are never notified.
### Sending
@@ -339,8 +363,9 @@ representative h-card, authorship, Post Type Discovery. Decisions:
| `/note/<slug>` | `h-entry`: `p-name` only when the note starts with a `# Heading` (so plain notes have no implied name); `u-photo` as direct children above `e-content`; `u-url u-uid` with `dt-published` (RFC 3339 UTC `Z`); `dt-updated`; `p-category` with `rel=tag`; `u-syndication` links; hidden `p-author h-card` |
| Replies | `u-in-reply-to h-cite` holding the cached reply context (`p-content`, `u-url`) |
| Likes / reposts / bookmarks | `u-like-of` / `u-repost-of` / `u-bookmark-of` link with "★ Liked" / "🔁 Reposted" / "🔖 Bookmarked" |
| Received responses | `u-comment h-cite` (replies: `p-author h-card` with `u-url` on an inner link, `u-photo`, `p-name`; `e-content`; `u-url`; `dt-published`), and `u-like`, `u-repost`, `u-bookmark`, `u-mention` h-cites. Faces beyond 8 are `hidden`, not omitted |
| Received responses | `u-comment h-cite` (replies: `p-author h-card` with `u-url` on an inner link, `u-photo`, `p-name`; `e-content`; `u-url`; `dt-published`), likes, reposts and bookmarks as `u-like`, `u-repost`, `u-bookmark` h-cites in the counts line card; mentions and quotes as `u-mention` h-cites |
| `/likes`, `/tag/<tag>` | `h-feed` with `p-name`, `u-url`, `p-author h-card` |
| Response counts | `/`, `/tag/<tag>` and `/likes` show each note's like, repost and reply counts, linking to its `#responses` (v2.13.4) |
Post Type Discovery gives note, article, reply, like, repost, bookmark and
photo correctly.
+55 -27
View File
@@ -1,6 +1,6 @@
# StarPunk Architecture Overview
**Version**: 2.10.1 (2026-11-03)
**Version**: 2.14.0 (2026-11-10)
**Status**: Current. Describes the system as it is in the code today.
## Purpose
@@ -10,7 +10,9 @@ owner) writes short notes, which are published as microformats2-marked HTML
pages and as RSS, Atom and JSON feeds. The site takes part in the IndieWeb:
it accepts posts from Micropub clients, issues its own IndieAuth tokens for
them, sends and receives Webmentions, pushes feed updates with WebSub, and
crossposts (POSSE) to GoToSocial, Bluesky and Threads.
crossposts (POSSE) to GoToSocial, Bluesky and Threads. Responses to those
crossposts are pulled back and shown on the note (backfeed), and the owner
can be notified of new responses.
Guiding principle: "every line of code must justify its existence". It is a
single Flask application with server-side rendering, SQLite and plain files.
@@ -39,6 +41,9 @@ Key decisions:
[ADR-067](../decisions/ADR-067-micropub-update-delete.md) (Micropub update/delete),
[ADR-068](../decisions/ADR-068-likes-and-reposts.md) (likes and reposts),
[ADR-071](../decisions/ADR-071-websub.md) (WebSub),
[ADR-072](../decisions/ADR-072-remove-metrics-dashboard.md) (metrics dashboard removed),
[ADR-073](../decisions/ADR-073-backfeed.md) (backfeed),
[ADR-074](../decisions/ADR-074-notifications.md) (notifications),
[ADR-057](../decisions/ADR-057-media-attachment-model.md) and
[ADR-058](../decisions/ADR-058-image-optimization-strategy.md) (media),
[ADR-034](../decisions/ADR-034-full-text-search.md) (search),
@@ -70,8 +75,9 @@ Key decisions:
│ starpunk.db │ │ OIDC provider, IndieAuth │
│ notes/*.md │ │ client pages, webmention │
│ media/ │ │ sources/targets, WebSub │
│ logs/, locks │ │ callbacks, GoToSocial, │
└────────────────┘ │ Bluesky, Threads │
│ avatars/ │ │ callbacks, GoToSocial, │
│ logs/, locks │ │ Bluesky, Threads, avatars, │
└────────────────┘ │ notifications (Apprise) │
└─────────────────────────────┘
```
@@ -127,12 +133,16 @@ wait for outbound network calls; the workers make them later.
| `reply_context.py` | Fetches and caches the post a reply responds to |
| `webmention_sender.py` | Extracts links from rendered notes and queues outgoing webmentions |
| `webmention_processor.py` | Validates incoming webmentions: fetch source, parse mf2, classify, sanitise content |
| `webmention_worker.py` | Background thread working the incoming and outgoing webmention queues |
| `webmention_worker.py` | Background thread working the incoming and outgoing webmention queues; also runs avatar upkeep and the notification sweep |
| `webmention_queries.py` | Webmention queries for display and moderation |
| `webmention_ssrf.py` | SSRF-safe fetcher: public-address checks, pinned connections, manual redirects, size and time limits |
| `websub.py` | WebSub publisher (discovery headers and links, `content_version`, queueing) and hub request handling |
| `websub_worker.py` | Background thread: verification of intent, content delivery, publish pings to an external hub |
| `syndication.py` | POSSE clients (Mastodon API/GoToSocial, Bluesky, Threads), reconcile logic, syndication worker |
| `syndication.py` | POSSE clients (Mastodon API/GoToSocial, Bluesky, Threads), reconcile logic, syndication worker (its loop also calls `backfeed.process_due()`) |
| `backfeed.py` | Backfeed: reads responses to crossposts from the silos, stores them as webmentions, schedules checks and backfills |
| `avatars.py` | Saves response authors' avatars as 96 px WebP files in `avatars/`; fills missing ones and removes unused ones |
| `notifications.py` | Notifications through Apprise: the sweep of new responses and immediate problem messages |
| `csrf.py` | CSRF tokens (HMAC of the session cookie) for admin forms, logout and the IndieAuth consent |
| `rate_limiter.py` | In-memory sliding-window rate limiter |
| `errors.py` | Central error handlers |
| `feeds/` | `rss.py` (feedgen), `atom.py`, `json_feed.py`, `opml.py`, `negotiation.py` (Accept header), `cache.py` (LRU + TTL feed cache with ETags) |
@@ -144,7 +154,7 @@ wait for outbound network calls; the workers make them later.
| Blueprint | File | Paths |
|-----------|------|-------|
| `public` | `public.py` | `/`, `/note/<slug>`, `/tag/<tag>`, `/likes`, `/media/<path>`, feeds, `/opml.xml` |
| `public` | `public.py` | `/`, `/note/<slug>`, `/tag/<tag>`, `/likes`, `/media/<path>`, `/avatars/<name>`, feeds, `/opml.xml` |
| `auth` | `auth.py` | `/auth/login` (GET, POST), `/auth/callback`, `/auth/logout` (POST) |
| `indieauth` | `indieauth.py` | `/.well-known/oauth-authorization-server`, `/auth/authorize` (GET, POST), `/auth/authorize/approve`, `/auth/token` (GET, POST), `/auth/revoke` |
| `micropub` | `micropub.py` | `/micropub` (GET, POST), `/micropub/media` (POST), `/media` (POST, legacy alias) |
@@ -163,20 +173,20 @@ Jinja2 templates live in `templates/`: public pages, `admin/`, `partials/`
reposts and bookmarks) and one page per HTTP error. `templates/base.html`
carries the discovery links (`rel=micropub`, `indieauth-metadata`,
`authorization_endpoint`, `token_endpoint`, `webmention`, `hub`/`self`, `me`,
feeds). Static assets are one stylesheet (`static/css/style.css`) and one
script (`static/js/preview.js`, the Markdown preview in the admin editor).
Everything works without JavaScript.
feeds). Static assets are one stylesheet (`static/css/style.css`); admin
pages have small inline scripts. Everything works without JavaScript.
## Public URL surface
| Path | Purpose |
|------|---------|
| `/` | Home page: recent published notes (h-feed); likes are kept off it |
| `/note/<slug>` | Note permalink (h-entry), with approved webmentions and reply context |
| `/note/<slug>` | Note permalink (h-entry), with approved responses (webmentions and backfed silo responses), a counts line with web actions, and reply context |
| `/tag/<tag>` | Notes with a tag |
| `/likes` | Likes |
| `/search`, `/api/search` | Full-text search page and JSON API |
| `/media/<path>` | Uploaded images and their variants |
| `/avatars/<name>` | Saved copies of response authors' avatars |
| `/feed` | Feed chosen by the `Accept` header (RSS by default; `Vary: Accept`) |
| `/feed.rss`, `/feed.atom`, `/feed.json` | RSS 2.0, Atom 1.0, JSON Feed 1.1 |
| `/feed.xml` | Legacy alias of `/feed.rss` |
@@ -191,8 +201,15 @@ Everything works without JavaScript.
| `/admin/webmentions`, `/admin/webmentions/<id>`, `…/action`, `…/bulk-action` | Webmention moderation |
| `/admin/tokens`, `/admin/tokens/<id>/revoke` | Issued IndieAuth tokens |
| `/admin/syndication/<id>/retry` | Retry a failed crosspost |
| `/admin/syndication/<id>/check` (POST) | Check one crosspost for responses now |
| `/admin/backfeed/backfill` (POST) | Check every crosspost for responses once (backfill) |
| `/admin/notifications/test` (POST) | Send a test notification |
| `/admin/health` | Full diagnostics (JSON): database, pool statistics, filesystem, disk |
| `/health` | Public liveness check (`{"status": "ok", "version": …}`); `?detailed=true` requires login |
| `/dev/login` | Development login; only when `DEV_MODE=true` |
`/`, `/tag/<tag>` and `/likes` show each note's response counts, linking to
its responses (v2.13.4).
## Background workers
@@ -202,9 +219,9 @@ coordinated through the data volume:
| Thread | Started when | Coordination | Work |
|--------|--------------|--------------|------|
| Webmention worker (`webmention_worker.py`) | `WEBMENTION_WORKER_ENABLED` and not testing | Non-blocking `fcntl.flock` on `webmention-worker.lock` next to the database. Only the holder works the queues; the others try again each poll interval. The OS releases the lock if the holder dies, and the new holder resets rows stuck in `sending`. | Validates received webmentions; discovers endpoints and sends queued outgoing webmentions, with retries |
| Webmention worker (`webmention_worker.py`) | `WEBMENTION_WORKER_ENABLED` and not testing | Non-blocking `fcntl.flock` on `webmention-worker.lock` next to the database. Only the holder works the queues; the others try again each poll interval. The OS releases the lock if the holder dies, and the new holder resets rows stuck in `sending`. | Validates received webmentions; discovers endpoints and sends queued outgoing webmentions, with retries; saves missing response avatars every 10 minutes and removes unused ones weekly (v2.13.0); sweeps new responses into notifications and sets `webmentions.notified_at` (v2.14.0, `notifications.py`, ADR-074) |
| WebSub worker (`websub_worker.py`) | `WEBSUB_ENABLED` and not testing | Same pattern with `websub-worker.lock` | Verifies subscription intents, delivers topic content (signed with `X-Hub-Signature` when the subscriber gave a secret), retries, expires leases; pings an external hub instead if `WEBSUB_HUB_URL` is set |
| Syndication worker (`syndication.py`) | At least one crosspost target configured, and not testing | No file lock. Rows in `syndications` are claimed with a 5-minute `claimed_until` lease by an atomic `UPDATE`, so any process may work a row but only one at a time; a `version` column keeps edits made during a send. Threads token refreshes are serialised with a lock file next to `threads_token.json`. | Creates, updates or deletes the remote post for each (note, target) pair; up to 8 attempts |
| Syndication worker (`syndication.py`) | At least one crosspost target configured, and not testing | No file lock. Rows in `syndications` are claimed with a 5-minute `claimed_until` lease by an atomic `UPDATE`, so any process may work a row but only one at a time; a `version` column keeps edits made during a send. Threads token refreshes are serialised with a lock file next to `threads_token.json`. | Creates, updates or deletes the remote post for each (note, target) pair; up to 8 attempts. Then, only when `BACKFEED_ENABLED`, backfeed: checks copies for responses, each claimed with a `backfeed_claimed_until` lease (v2.13.0, `backfeed.py`, ADR-073) |
All workers run inside an app context, are daemon threads (they end with the
process) and sleep for their poll interval when there is nothing to do. Under
@@ -225,9 +242,12 @@ Content is kept in files and everything else in SQLite, both under
- **Media**: `media/YYYY/MM/<uuid>.<ext>` plus `<uuid>_<variant>.<ext>`
variants, staged in `media/.tmp/` while being processed.
- **Database**: `starpunk.db` (WAL mode): note metadata, tags, media records,
sessions, IndieAuth codes and tokens, webmentions and the outgoing queue,
syndications, WebSub subscriptions and queues, caches (author h-card, reply
sessions, IndieAuth codes and tokens, webmentions (including backfed silo
responses, marked by `origin` and `remote_id`) and the outgoing queue,
syndications (with backfeed state and silo-only counts), WebSub subscriptions and queues, caches (author h-card, reply
contexts), the FTS5 index and `site_state`. See [database.md](database.md).
- **Avatars**: `avatars/<sha256>.webp`, saved copies of response authors'
avatars (v2.13.0); they can be re-fetched, so losing them is harmless.
- **Other files**: `logs/starpunk.log` (rotating), worker lock files,
`threads_token.json`, and `debug/` for failed uploads when
`DEBUG_SAVE_FAILED_UPLOADS=true`.
@@ -279,20 +299,28 @@ their credentials are set. The full list with defaults is in
- **Author h-card** (`author_profile`, 24 hours) and **reply contexts**
(`reply_contexts`) are cached in the database.
- The webmention worker caches endpoint discovery results in memory.
- **Response avatars** are saved once as files named by the SHA-256 of the
source URL and served from `/avatars/<name>` with
`Cache-Control: public, max-age=604800` (one week).
## Security posture
- **Outbound requests (SSRF)**: fetches of URLs that come from outside
(webmention sources and targets, endpoint discovery, reply contexts, WebSub
callbacks, remote Micropub photos, IndieAuth client pages, crosspost link
cards) are checked first: every address the host resolves to must be public
(private, loopback, link-local, CGNAT, multicast and reserved ranges are
refused). Every fetch of a user-supplied URL (webmentions, reply contexts,
WebSub, remote Micropub photos, IndieAuth client pages and, since v2.11.0,
crosspost link cards) uses the fetcher in `webmention_ssrf.py`: it follows
redirects by hand and re-checks each hop, connects to the address it
checked (no second DNS lookup) and limits response size (1 MB by default)
and total time. `DEV_MODE` relaxes the checks for local testing.
- **Outbound requests (SSRF)**: every fetch of a URL that comes from
outside (webmention sources and targets, endpoint discovery, reply
contexts, WebSub callbacks, remote Micropub photos, IndieAuth client pages,
crosspost link cards since v2.11.0, response avatars since v2.13.0) goes
through the fetcher in `webmention_ssrf.py`. Every address the host
resolves to must be public (private, loopback, link-local, CGNAT,
multicast and reserved ranges are refused); it follows redirects by hand
and re-checks each hop, connects to the address it checked (no second DNS
lookup) and limits response size (1 MB by default) and total time.
`DEV_MODE` relaxes the checks for local testing. Exceptions: avatars on
the site's own `GOTOSOCIAL_URL` host are fetched directly, as the
crossposting client does, since behind the same proxy it may resolve to an
internal address (v2.13.1); Bluesky CDN avatars are fetched from the
account's PDS with `com.atproto.sync.getBlob`, itself through the safe
fetcher. Apprise notifications go to the owner-configured `NOTIFY_URLS`,
outside the safe fetcher.
- **HTML sanitising**: rendered notes (Markdown or HTML sent by a Micropub
client) and webmention content are cleaned with nh3 allow-lists; search
highlighting escapes everything except `<mark>`.
@@ -327,4 +355,4 @@ their credentials are set. The full list with defaults is in
---
**Last Updated**: 2026-11-03 (v2.12.0)
**Last Updated**: 2026-11-10 (v2.14.0)
@@ -1,6 +1,6 @@
# Authentication and Authorization Architecture
Current as of v2.10.1. Decision record: [ADR-064](../decisions/ADR-064-oidc-admin-login-and-builtin-indieauth-server.md),
Current as of v2.14.0 (no auth changes since v2.11.0). Decision record: [ADR-064](../decisions/ADR-064-oidc-admin-login-and-builtin-indieauth-server.md),
which supersedes the "no self-hosted IndieAuth" decisions
([ADR-050](../decisions/ADR-050-remove-custom-indieauth-server.md),
[ADR-056](../decisions/ADR-056-no-selfhosted-indieauth.md)) and the external
+26 -10
View File
@@ -1,6 +1,6 @@
# Syndication (POSSE) Architecture
Current as of v2.10.1. Code: `starpunk/syndication.py`.
Current as of v2.14.0. Code: `starpunk/syndication.py`, `starpunk/backfeed.py`.
StarPunk publishes on its own site first and then copies each note to other
networks (POSSE: Publish on your Own Site, Syndicate Elsewhere). Later
@@ -16,7 +16,9 @@ Decisions: [ADR-065](../decisions/ADR-065-direct-syndication-posse.md)
(Bluesky), [ADR-068](../decisions/ADR-068-likes-and-reposts.md) (likes and
reposts), [ADR-069](../decisions/ADR-069-threads-syndication.md) (Threads),
[ADR-070](../decisions/ADR-070-crosspost-links-and-cards.md) (links and link
cards), and the [v2.9.0 Micropub review](../design/v2.9.0/2026-11-03-micropub-review.md)
cards), [ADR-073](../decisions/ADR-073-backfeed.md) (backfeed),
[ADR-074](../decisions/ADR-074-notifications.md) (notifications, including
crossposting problems), and the [v2.9.0 Micropub review](../design/v2.9.0/2026-11-03-micropub-review.md)
(bookmarks, `mp-syndicate-to` default).
## Targets
@@ -34,7 +36,10 @@ in the process.
Display names come from `GOTOSOCIAL_NAME`, `BLUESKY_NAME` and
`THREADS_NAME`. GoToSocial statuses use `GOTOSOCIAL_VISIBILITY` (default
`public`). The GoToSocial token needs `write:statuses` and `write:media`,
plus `read:search` and `write:favourites` for likes and boosts.
plus `read:search` and `write:favourites` for likes and boosts, and
`read:statuses` and `read:accounts` (or `read`) for backfeed. Backfeed on
Threads needs `threads_read_replies` and `threads_manage_insights`. The
Threads client uses the API host `graph.threads.com` (v2.13.0).
Bluesky and Threads can't edit posts through their APIs. Once a copy exists
there, edits to the note don't change it, but unpublish and delete still
@@ -59,6 +64,9 @@ is sent until the note is published.
## Data: the `syndications` table (migration 012)
Migration 020 adds the backfeed columns; see [Backfeed](#backfeed-v2130-adr-073)
and [database.md](database.md#crossposting).
One row per (note, target), unique on `(note_id, target)`:
| Column | Meaning |
@@ -83,6 +91,8 @@ delete and the worker can still remove the copy.
| Micropub photo changes | `note_changed(id)` | Same |
| `delete_note()` (soft or hard), `undelete_note()` | `note_changed(id)` | Same |
| Retry button on the edit page | `retry(id)` | That row set to `pending` now, attempts reset |
| Check now (`POST /admin/syndication/<id>/check`) | `backfeed.check_now(id)` | That copy's responses checked at the next worker pass |
| Backfill responses (`POST /admin/backfeed/backfill`) | `backfeed.start_backfill()` | Every eligible copy flagged for one check |
## Worker
@@ -130,7 +140,8 @@ again with the new content. `_fail()` does the same.
errors (codes 10, 190, 200).
- Other errors retry after `min(30 s × 2^attempts, 6 h)`. After 8 attempts
the row becomes `failed`. The error is shown on the edit page with a
Retry button.
Retry button. When a row finally fails, the owner also gets a `problems`
notification (ADR-074).
- `RemoteGone` (404, or Graph error 100/33) during an update means the copy
was deleted remotely, so a new one is created. During a delete it counts as
success.
@@ -227,7 +238,7 @@ to itself (v2.8.0).
`starpunk/backfeed.py` brings responses to the copies back to the note.
After each crosspost pass, the worker loop calls `backfeed.process_due()`,
which checks up to three synced copies of published notes (not likes or
which does nothing unless `BACKFEED_ENABLED`; otherwise it checks up to three synced copies of published notes (not likes or
reposts) that are due, each under a lease (`backfeed_claimed_until`):
- **Reading.** Each client has `responses(remote_id, remote_url)`, which
@@ -244,10 +255,14 @@ reposts) that are due, each under a lease (`backfeed_claimed_until`):
`WEBMENTION_AUTO_APPROVE` default; an existing row keeps its moderation.
A complete harvest marks that silo's rows for the note that weren't
found `invalid` ("No longer on <silo>"). Authors with `#nobridge` or
`#nobot` in their bio, or Bluesky's `!no-unauthenticated` label, are
skipped and their rows deleted. If the note's shown responses changed,
its webmentions are re-sent (spec 3.1.4). Threads counts go in
`syndications.likes_count`, `reposts_count`, `quotes_count`.
`#nobot` in their bio, or Bluesky's `!no-unauthenticated` label, are not
stored and their rows are deleted; since v2.13.3 their likes, reposts and
quotes are counted anonymously instead (`hidden_counts()`). If the note's
shown responses changed, its webmentions are re-sent (spec 3.1.4).
`syndications.likes_count`, `reposts_count` and `quotes_count` hold the
silo-only counts: all of Threads; elsewhere people who opted out. New
responses found by a backfill, or by a copy's first check on a note older
than a day, are stored as already notified (v2.14.0).
- **Schedule.** `backfeed_next_at` follows the note's age: 15 minutes (day
1), 1 hour (week 1), 1 day (to 30 days), 1 week (to
`BACKFEED_MAX_AGE_DAYS`), with up to 10% jitter; none after. Failures back
@@ -259,7 +274,8 @@ reposts) that are due, each under a lease (`backfeed_claimed_until`):
`site_state`.
- **Refused credentials.** A 401/403 (or Graph auth error) puts the target
in `backfeed.denied` until restart; the admin Webmentions page says which
scopes are needed. Crossposting is unaffected.
scopes are needed, and a `problems` notification is sent. Crossposting
is unaffected.
Avatars of all responses (received and backfed) are saved by
`starpunk/avatars.py` and served from `/avatars/<sha256>.webp`; the
+13 -10
View File
@@ -1,8 +1,8 @@
# StarPunk Technology Stack
**Version**: 2.10.1 (2026-11-03)
**Version**: 2.14.0 (2026-11-10)
**Status**: Current. The original v0.x stack write-up is kept as
[technology-stack-legacy.md](technology-stack-legacy.md) for history.
[technology-stack-legacy.md](../design/v0.x/technology-stack-legacy.md) for history.
StarPunk deliberately uses few dependencies: a WSGI web framework, SQLite
from the standard library, plain files, server-rendered templates and a
@@ -21,7 +21,7 @@ the pieces fit together.
| Content storage | Markdown files on disk |
| Background work | `threading` daemon threads; `fcntl.flock` file locks to pick one process |
| Templates | Jinja2, server-side rendered |
| Front end | One hand-written stylesheet; one small vanilla JS file for the editor's Markdown preview |
| Front end | One hand-written stylesheet; small inline scripts on admin pages; no JavaScript on public pages |
| Packaging | `uv` creates the virtual environment and installs requirements (locally and in the image) |
| Container | Multi-stage `Containerfile` (Podman/Docker), non-root user |
| Reverse proxy | Caddy in production (examples for Caddy and nginx in the repo) |
@@ -43,9 +43,10 @@ vulnerabilities (`pip-audit`); check again before each release.
| python-dateutil | `4.0.*` | Dates in reply contexts |
| feedgen | `1.0.*` | RSS 2.0 feed (`starpunk/feeds/rss.py`); Atom and JSON Feed are generated by StarPunk's own code |
| httpx | `0.28.*` | All outbound HTTP; requests to user-supplied URLs go through `webmention_ssrf.safe_get`/`safe_post` |
| apprise | `2.2.*` | Notifications to the owner's services (`starpunk/notifications.py`, ADR-074); sends to the URLs the owner configures, outside the SSRF-safe fetcher |
| PyJWT[crypto] | `2.15.*` | Verifying OIDC ID tokens (JWKS, RS/ES signatures via `cryptography`) |
| python-dotenv | `1.2.*` | Loading `.env` |
| Pillow | `13.0.*` | Image validation, resizing and variants |
| Pillow | `13.0.*` | Image validation, resizing and variants; converting response avatars to 96 px WebP (`avatars.py`) |
| pillow-heif | `1.8.*` | HEIC/HEIF uploads (iPhone photos) |
`markupsafe` comes with Flask/Jinja2.
@@ -68,10 +69,10 @@ front-end framework or bundler, no separate search server.
| Area | Standards |
|------|-----------|
| IndieWeb | Micropub (create, update, delete, undelete, queries, media endpoint), IndieAuth (authorization server with PKCE, metadata endpoint, token revocation), Webmention (send and receive), WebSub (publisher and built-in hub), microformats2 (h-entry, h-feed, h-card markup and parsing), rel=me |
| IndieWeb | Micropub (create, update, delete, undelete, queries, media endpoint), IndieAuth (authorization server with PKCE, metadata endpoint, token revocation), Webmention (send and receive), WebSub (publisher and built-in hub), microformats2 (h-entry, h-feed, h-card markup and parsing), rel=me, web actions (`<indie-action>`), backfeed of responses from the silos |
| Login | OpenID Connect authorization code flow with PKCE (provider: Pocket ID) |
| Feeds | RSS 2.0, Atom 1.0, JSON Feed 1.1, OPML 2.0, HTTP content negotiation |
| Crossposting | Mastodon client API (GoToSocial), AT Protocol (Bluesky XRPC), Threads Graph API |
| Crossposting | Mastodon client API (GoToSocial), AT Protocol (Bluesky XRPC), Threads Graph API; the same APIs are read for backfeed |
Protocol details: [indieweb-protocols.md](indieweb-protocols.md),
[simplified-auth-architecture.md](simplified-auth-architecture.md),
@@ -79,7 +80,7 @@ Protocol details: [indieweb-protocols.md](indieweb-protocols.md),
## Testing
- **Framework**: pytest with pytest-mock; about 1,250 tests in `tests/`
- **Framework**: pytest with pytest-mock; about 1,260 tests in `tests/`
(`test_*.py` per module or feature, shared fixtures in `tests/conftest.py`,
helpers in `tests/helpers/`).
- **Isolation**: the `app` fixture calls `create_app()` with `TESTING=True`
@@ -92,10 +93,12 @@ Protocol details: [indieweb-protocols.md](indieweb-protocols.md),
- **Running locally**:
```bash
uv venv && uv pip install -r requirements-dev.txt
.venv/bin/python -m pytest
uv venv --python 3.14 && uv pip install -r requirements-dev.txt
SESSION_SECRET=test uv run pytest tests/test_<area>.py
```
Locally, run the tests for the area you changed; CI runs the full suite.
- **In CI**: the `test` stage of the `Containerfile` installs the production
requirements plus `requirements-test.txt` and runs the whole suite. The
image is only built and pushed if it passes. Checklist:
@@ -119,4 +122,4 @@ standards.
---
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
+1 -1
View File
@@ -2,7 +2,7 @@
## Status
Accepted (v2.13.0, 2026-11-10)
Accepted (v2.13.0, 2026-11-10; amended v2.13.2, v2.13.3)
Implementation notes (v2.13.0):
+155
View File
@@ -0,0 +1,155 @@
# ADR-074: Notifications through Apprise
## Status
Accepted (v2.14.0, 2026-11-10)
## Context
Responses now reach a note from two places: received webmentions
(ADR-063) and backfeed from the crossposts (ADR-073). The owner only finds
out about them by opening the site or the admin. Crossposts that finally
fail, and silos that refuse a backfeed credential, are just as invisible.
Owners use different notification services. The production site uses
Gotify (read on iOS through iGotify). Others use ntfy, Pushover, email,
Matrix, Discord and so on. [Apprise](https://github.com/caronc/apprise)
covers about 100 of them behind one URL syntax (`gotifys://host/token`,
`ntfys://topic`, `mailtos://…`), both as a Python library and as a
separate HTTP server (apprise-api).
Options considered:
1. **Gotify only.** Simplest for this site, no use to anyone else.
2. **Post to an apprise-api server.** Standard, but every owner needs one
running, and messages take an extra hop.
3. **Embed the Apprise library.** Sends straight to each service; an owner
who prefers a central apprise-api can still use its `apprise://` /
`apprises://` URL.
4. **Gotify plus a generic JSON webhook, hand-written.** No dependency, but
every other service needs glue the owner has to write.
## Decision
### 1. Embed Apprise
StarPunk depends on the `apprise` package and sends with it directly.
`NOTIFY_URLS` holds one or more Apprise URLs (space- or comma-separated);
each message goes to all of them. With no URLs, notifications are off and
nothing else changes.
### 2. What can be notified
`NOTIFY_EVENTS` is a comma-separated list. Default:
`reply,mention,quote,problems`.
| Event | Sent when |
|---|---|
| `reply`, `mention`, `quote`, `like`, `repost`, `bookmark` | a response of that type is first shown on a note, or first awaits moderation (received webmention or backfeed alike) |
| `problems` | a crosspost fails for good (permanent error or last attempt), or a silo refuses the backfeed credential (ADR-073 Decision 9) |
### 3. Messages
Markdown, so services that render it (Gotify, iGotify, ntfy, Matrix…)
show links; others get the text.
- **Replies, mentions, quotes:** one message each. Title "↩ Reply from Ana
Lima via Bluesky" (or "💬 Mention from …", "❝ Quote from …"); body the
response text (plain, up to 300 characters), then a link to the note's
responses (`<permalink>#responses`).
- **Likes, reposts, bookmarks:** grouped per note and type in each pass:
"♥ 3 new likes on 'Starpunk now supports…'", with up to five names and a
link to the note.
- **Awaiting moderation:** when `WEBMENTION_AUTO_APPROVE` is off, the title
says so and the link goes to the admin webmention page instead.
- **Problems:** "⚠ Crosspost to Bluesky failed" / "⚠ Backfeed for Threads
stopped" with the error and a link to the note's edit page or the
Webmentions page.
Apprise's Gotify plugin has no click-through link, so the link is in the
text.
### 3a. Priority
Every message is sent with the same priority, `NOTIFY_PRIORITY` (default
`normal`). Apprise takes priority as a `priority` argument in each service
URL, with values that differ by service, so StarPunk adds
`priority=<NOTIFY_PRIORITY>` to each URL in `NOTIFY_URLS` that doesn't set
one itself; a URL can still set its own. Apprise's names (`low`,
`moderate`, `normal`, `high`, `emergency`) work for most services; Gotify
also takes 0–10 (it maps the names to 0, 3, 5, 8, 10). Different
priorities per event can come later.
### 4. Delivery
- New column `webmentions.notified_at`. A sweep finds responses with
`notified_at IS NULL`, `validation_status = 'valid'`, a moderation status
of `approved` or `pending`, and a type in `NOTIFY_EVENTS`, sends them,
and sets `notified_at`. Types not selected are marked without sending.
- The sweep runs in the webmention worker loop, which already runs in one
process at a time (its file lock), so nothing is sent twice by several
gunicorn workers. It handles received and backfed responses alike.
- A failed send leaves `notified_at` empty, so the next pass (about 10
seconds later) retries, with backoff up to 15 minutes. Responses older
than 24 hours that still haven't gone out are marked and dropped, with a
warning, so an outage doesn't end in a flood. The last error is shown on
the admin Webmentions page.
- Problems are sent at the moment they happen (from the syndication and
backfeed workers) and not retried: they are also in the log and admin.
- Sending never blocks or breaks validation, crossposting or backfeed.
### 5. No floods from history
- Migration 021 sets `notified_at` on every existing response.
- Responses stored by a backfill check (ADR-073 Decision 8) are stored with
`notified_at` set.
- So are responses found on a crosspost's first check when its note is more
than a day old (copies linked after the fact).
- Grouping (Decision 3) keeps a busy note to one like message per pass.
### 6. Admin
- **Send test notification** on the admin Webmentions page sends a test
message to every URL and reports each result.
- The page shows whether notifications are on, which events are selected,
and the last send error.
### 7. Configuration
| Setting | Default | Meaning |
|---|---|---|
| `NOTIFY_URLS` | empty (off) | Apprise URLs, space- or comma-separated |
| `NOTIFY_EVENTS` | `reply,mention,quote,problems` | which events are sent |
| `NOTIFY_PRIORITY` | `normal` | priority of every message, added to URLs that don't set one |
Settings stay in the environment, like the rest of StarPunk's (ADR-052).
URLs contain tokens and belong with the other secrets.
## Consequences
- One new dependency, `apprise`, with its own (requests, PyYAML, click,
Markdown, certifi). It must install on Python 3.14 in the container
image; this is checked before the release.
- Owners can send to almost any service without writing glue, directly or
through apprise-api.
- Each response is notified at most once, a few seconds after it appears,
and only if its type is selected.
- Apprise opens outbound connections to whatever URLs the owner configures,
outside the SSRF-safe fetcher. That is intended: the owner chooses these
URLs, as with the crosspost targets. A service behind the same reverse
proxy as StarPunk may need the deployment's internal-address entry
(`extra_hosts`), as GoToSocial does.
- Notification text includes other people's names and words, sent to the
owner's own services only.
## Alternatives considered
- **Gotify only / hand-written webhook.** See Context, options 1 and 4: no
dependency, but each service beyond Gotify needs glue the owner writes.
- **apprise-api only.** See option 2. Still possible through an
`apprises://` URL.
- **Sending from where responses are stored** (webmention worker,
backfeed) instead of a sweep. Two code paths, no retry, and backfeed
runs in every gunicorn worker. Rejected.
- **Choosing events in the admin UI.** Possible later; the environment is
where StarPunk's settings live today.
+7 -6
View File
@@ -5,8 +5,8 @@ records one decision: its context, what was decided, and the consequences.
Superseded ADRs are kept as history; their Status line names the ADR that
replaced them.
**Total ADR files**: 69 (68 numbers: ADR-001 to ADR-044 and ADR-050 to
ADR-073, with two files numbered ADR-012). See [Numbering](#numbering).
**Total ADR files**: 70 (69 numbers: ADR-001 to ADR-044 and ADR-050 to
ADR-074, with two files numbered ADR-012). See [Numbering](#numbering).
Status values: **Accepted** (in force), **Superseded** (replaced by a later
ADR), **Deprecated** (withdrawn, nothing replaces it; none at present),
@@ -85,7 +85,7 @@ See also [IndieWeb protocols](../architecture/indieweb-protocols.md).
| [066](ADR-066-bluesky-syndication.md) | Crossposting to Bluesky | Accepted | Bluesky as a second crosspost target (v2.2.0). |
| [069](ADR-069-threads-syndication.md) | Crossposting to Threads | Accepted | Threads as a third target, with automatic token refresh (v2.6.0). |
| [070](ADR-070-crosspost-links-and-cards.md) | Links and link cards in crossposts | Accepted | Keep note links in crossposts and build link cards for the linked page (v2.6.3). |
| [073](ADR-073-backfeed.md) | Backfeed of responses to crossposts | Accepted | Poll GoToSocial, Bluesky and Threads for responses to crossposts and store them as webmentions; admin backfill (v2.13.0). |
| [073](ADR-073-backfeed.md) | Backfeed of responses to crossposts | Accepted | Poll GoToSocial, Bluesky and Threads for responses to crossposts and store them as webmentions; admin backfill (v2.13.0, amended v2.13.2–v2.13.3). |
### Authentication (current)
@@ -105,6 +105,7 @@ See also [IndieWeb protocols](../architecture/indieweb-protocols.md).
| [054](ADR-054-structured-logging-architecture.md) | Structured Logging Architecture | Accepted | Rotating log files with correlation IDs (v1.1.1). |
| [055](ADR-055-error-handling-philosophy.md) | Error Handling Philosophy | Accepted | Layered handling: graceful degradation, clear user messages, detailed logs. |
| [072](ADR-072-remove-metrics-dashboard.md) | Remove the metrics dashboard | Accepted | Drop the in-memory metrics system and dashboard; keep `/health`, `/admin/health` and logs (v2.12.0). |
| [074](ADR-074-notifications.md) | Notifications through Apprise | Accepted | Notify the owner of new responses and crosspost/backfeed problems via the embedded Apprise library (v2.14.0). |
See also [deployment](../architecture/deployment.md).
@@ -168,7 +169,7 @@ was carried out and is not itself superseded.
## How to create a new ADR
1. Use the next number after the highest existing one (currently ADR-074).
1. Use the next number after the highest existing one (currently ADR-075).
2. Name it `ADR-NNN-brief-descriptive-title.md`.
3. Follow the format of recent ADRs (e.g. ADR-070, ADR-071):
@@ -201,5 +202,5 @@ was carried out and is not itself superseded.
---
**Last Updated**: 2026-11-03
**Total ADR files**: 67
**Last Updated**: 2026-11-10
**Total ADR files**: 70
+28 -10
View File
@@ -21,7 +21,13 @@ is in `../architecture/`; decisions are in `../decisions/` (ADRs).
## Contents
[v1.0.0](#v100) · [v1.1.1](#v111) · [v1.1.2](#v112) · [v1.2.0](#v120) · [v1.3.0](#v130) · [v1.3.1](#v131) · [v1.4.0](#v140) · [v1.4.1](#v141) · [v1.4.2](#v142) · [v1.5.0](#v150) · [v1.6.0](#v160) · [v2.0.0](#v200) · [v2.0.1](#v201) · [v2.0.2](#v202) · [v2.1.0](#v210) · [v2.2.0](#v220) · [v2.3.0](#v230) · [v2.3.1](#v231) · [v2.3.2](#v232) · [v2.3.3](#v233) · [v2.4.0](#v240) · [v2.5.0](#v250) · [v2.6.0](#v260) · [v2.6.1](#v261) · [v2.6.2](#v262) · [v2.6.3](#v263) · [v2.6.4](#v264) · [v2.6.5](#v265) · [v2.6.6](#v266) · [v2.6.7](#v267) · [v2.6.8](#v268) · [v2.6.9](#v269) · [v2.7.0](#v270) · [v2.7.1](#v271) · [v2.7.2](#v272) · [v2.8.0](#v280) · [v2.8.1](#v281) · [v2.9.0](#v290) · [v2.10.0](#v2100) · [v2.10.1](#v2101) · [v2.10.2](#v2102) · [v2.11.0](#v2110) · [v2.12.0](#v2120) · [v2.12.1](#v2121) · [v2.13.0](#v2130) · [v2.13.1](#v2131) · [v2.13.2](#v2132) · [v2.13.3](#v2133) · [hotfix](#hotfix)
[v0.x](#v0x) · [v1.0.0](#v100) · [v1.1.1](#v111) · [v1.1.2](#v112) · [v1.2.0](#v120) · [v1.3.0](#v130) · [v1.3.1](#v131) · [v1.4.0](#v140) · [v1.4.1](#v141) · [v1.4.2](#v142) · [v1.5.0](#v150) · [v1.6.0](#v160) · [v2.0.0](#v200) · [v2.0.1](#v201) · [v2.0.2](#v202) · [v2.1.0](#v210) · [v2.2.0](#v220) · [v2.3.0](#v230) · [v2.3.1](#v231) · [v2.3.2](#v232) · [v2.3.3](#v233) · [v2.4.0](#v240) · [v2.5.0](#v250) · [v2.6.0](#v260) · [v2.6.1](#v261) · [v2.6.2](#v262) · [v2.6.3](#v263) · [v2.6.4](#v264) · [v2.6.5](#v265) · [v2.6.6](#v266) · [v2.6.7](#v267) · [v2.6.8](#v268) · [v2.6.9](#v269) · [v2.7.0](#v270) · [v2.7.1](#v271) · [v2.7.2](#v272) · [v2.8.0](#v280) · [v2.8.1](#v281) · [v2.9.0](#v290) · [v2.10.0](#v2100) · [v2.10.1](#v2101) · [v2.10.2](#v2102) · [v2.11.0](#v2110) · [v2.12.0](#v2120) · [v2.12.1](#v2121) · [v2.13.0](#v2130) · [v2.13.1](#v2131) · [v2.13.2](#v2132) · [v2.13.3](#v2133) · [v2.13.4](#v2134) · [v2.14.0](#v2140) · [v2.14.1](#v2141) · [hotfix](#hotfix)
## v0.x
Not a release folder: documents from before v1.0.0 kept for history.
- [technology-stack-legacy.md](v0.x/technology-stack-legacy.md) — StarPunk Technology Stack - Quick Reference
## v1.0.0
@@ -408,35 +414,47 @@ OIDC admin login and built-in IndieAuth server.
## v2.10.2
- [2026-11-03-reply-context-fetch.md](v2.10.2/2026-11-03-reply-context-fetch.md) - Reply contexts fetched again (fetch_source returns three values since v2.7.0)
- [2026-11-03-reply-context-fetch.md](v2.10.2/2026-11-03-reply-context-fetch.md) — Reply context fetch
## v2.11.0
- [2026-11-03-maintenance-release.md](v2.11.0/2026-11-03-maintenance-release.md) - Dependency upgrades (Python 3.14), foreign keys, CSRF, fetchers, debug default, shared rate limits
- [2026-11-03-maintenance-release.md](v2.11.0/2026-11-03-maintenance-release.md) — Maintenance release
## v2.12.0
- [2026-11-03-remove-metrics.md](v2.12.0/2026-11-03-remove-metrics.md) - Metrics dashboard and `starpunk/monitoring` removed (ADR-072)
- [2026-11-03-remove-metrics.md](v2.12.0/2026-11-03-remove-metrics.md) — Remove the metrics dashboard
## v2.12.1
- [2026-11-10-category-unused-tags.md](v2.12.1/2026-11-10-category-unused-tags.md) - q=category lists only tags on posts that exist
- [2026-11-10-category-unused-tags.md](v2.12.1/2026-11-10-category-unused-tags.md) — q=category lists only tags in use
## v2.13.0
- [2026-11-10-backfeed.md](v2.13.0/2026-11-10-backfeed.md) - Backfeed of responses to crossposts (ADR-073), avatar cache, delete tombstones
- [2026-11-10-backfeed.md](v2.13.0/2026-11-10-backfeed.md) — Backfeed (implementation report)
## v2.13.1
- [2026-11-10-avatar-fetch.md](v2.13.1/2026-11-10-avatar-fetch.md) - Avatars from the own GoToSocial server and from Bluesky PDSes
- [2026-11-10-avatar-fetch.md](v2.13.1/2026-11-10-avatar-fetch.md) — Avatar fetching in production
## v2.13.2
- [2026-11-10-reaction-counts.md](v2.13.2/2026-11-10-reaction-counts.md) - Likes/reposts as a counts line with hover lists; web actions
- [2026-11-10-reaction-counts.md](v2.13.2/2026-11-10-reaction-counts.md) — Reaction counts and web actions
## v2.13.3
- [2026-11-10-count-opted-out.md](v2.13.3/2026-11-10-count-opted-out.md) - Opted-out people's likes and reposts counted without names
- [2026-11-10-count-opted-out.md](v2.13.3/2026-11-10-count-opted-out.md) — Count opted-out people without naming them
## v2.13.4
- [2026-11-10-list-counts.md](v2.13.4/2026-11-10-list-counts.md) — Counts on lists of notes, reply cards as links
## v2.14.0
- [2026-11-10-notifications.md](v2.14.0/2026-11-10-notifications.md) — Notifications (implementation report)
## v2.14.1
- [2026-11-10-tags-and-captions.md](v2.14.1/2026-11-10-tags-and-captions.md) — Clearing tags in the editor; captions escaped in feeds
## hotfix
@@ -457,4 +475,4 @@ IndieAuth PKCE endpoint discovery hotfix (2025-12-17, between v1.5.0 and v1.6.0;
---
**Last Updated**: 2026-11-03
**Last Updated**: 2026-11-10
@@ -3,7 +3,7 @@
> **Historical document.** This describes StarPunk's original v0.x technology
> choices (for example IndieLogin.com for admin login) and is kept for
> context only. It does not describe the current system; see
> [technology-stack.md](technology-stack.md) and [overview.md](overview.md).
> [technology-stack.md](../../architecture/technology-stack.md) and [overview.md](../../architecture/overview.md).
## Project Understanding
+1 -1
View File
@@ -1901,7 +1901,7 @@ class Webmention:
## References
- [ADR-004: File-Based Note Storage](../../decisions/ADR-004-file-based-note-storage.md)
- [Database Schema](../../../starpunk/monitoring/database.py)
- [Database Schema](../../../starpunk/database/schema.py)
- [Phase 1.1: Core Utilities](phase-1.1-core-utilities.md)
- [Python Coding Standards](../../standards/python-coding-standards.md)
- [Python Dataclasses Documentation](https://docs.python.org/3/library/dataclasses.html)
@@ -2061,7 +2061,7 @@ Phase 2.1 is complete when:
- [ADR-007: Slug Generation Algorithm](../../decisions/ADR-007-slug-generation-algorithm.md)
- [Phase 1.1: Core Utilities](phase-1.1-core-utilities.md)
- [Phase 1.2: Data Models](phase-1.2-data-models.md)
- [Database Schema](../../../starpunk/monitoring/database.py)
- [Database Schema](../../../starpunk/database/schema.py)
- [Python Coding Standards](../../standards/python-coding-standards.md)
- [Architecture Overview](../../architecture/overview.md)
- Security Architecture (`security.md`, not preserved)
@@ -0,0 +1,24 @@
# v2.13.4: Counts on lists of notes, reply cards as links
## What changed
- `webmention_queries.response_counts(note_ids)`: likes, reposts and
replies per note in two grouped queries (shown webmentions, plus the
anonymous counts on synced `syndications` rows). `routes/public.py`
`attach_response_counts()` runs after `attach_contexts()` on the home,
tag and Likes pages.
- `partials/responses.html` `list_counts`: icons and numbers in the note's
meta line, linking to `<permalink>#responses` (an anchor on the note page).
- Reply cards: the date link (already the reply's own URL) stretches over
the card with `::after`; author and content links sit above it.
- v2.13.2's `.reaction*` classes clashed with the older `.reaction` lines
(Liked / Reposted / In reply to, `partials/reaction.html`): the counts
line now uses `.response-count*` / `.response-card*` / `.response-*`.
## Verification
- `tests/test_backfeed.py::test_home_page_shows_counts`; template, route,
microformats and likes/reposts tests pass.
- Rendered the home page, a note page and the Likes page in headless
Chromium: counts beside the date, quiet notes unchanged, the Liked line
laid out as before v2.13.2.
@@ -0,0 +1,38 @@
# v2.14.0: Notifications (implementation report)
Implements ADR-074.
## What changed
- `starpunk/notifications.py` (new): settings (`urls`, `events`,
`with_priority`), `send` (Apprise, Markdown, 15 s timeout, never raises,
errors captured with `apprise.LogCapture`), `problem`, and `Sweeper`,
which sends due responses, groups likes/reposts/bookmarks per note,
marks `notified_at`, backs off after a failure (10 s doubling to 15 min),
drops responses older than 24 hours, and records the last error in
`site_state.notify_last_error`.
- `migrations/021_notifications.sql`: `webmentions.notified_at`, set on all
existing rows.
- `webmention_worker.py`: runs the sweep each pass (one process holds the
worker lock).
- `backfeed.py`: `store(..., quiet=True)` inserts new rows as notified;
`check()` is quiet for backfill checks and a copy's first check of a note
older than a day. A refused credential sends a `problems` message.
- `syndication.py`: `_fail` sends a `problems` message when it gives up.
- Admin Webmentions page: status (events, services, priority, last error)
and **Send test notification** (`POST /admin/notifications/test`).
- Settings `NOTIFY_URLS`, `NOTIFY_EVENTS`, `NOTIFY_PRIORITY`; dependency
`apprise==2.2.*`.
## Verification
- `tests/test_notifications.py` (11 tests): settings parsing and priority
injection; one message per reply/mention, likes grouped, unselected types
marked, pending moderation linked to the admin; nothing sent twice;
failure backoff and retry, last error set and cleared; 24-hour drop; off
without URLs; quiet backfill and late first checks; problems (selected or
not) and the final crosspost failure; admin status and test button; a
real Apprise send (`json://`) to a local server; migration 021.
- Webmention, syndication, backfeed, admin and migration tests pass.
- Apprise 2.2.0 installs and imports on Python 3.14.
- Production Gotify (3.1.1) accepted a message with the StarPunk app token.
@@ -0,0 +1,27 @@
# v2.14.1: Clearing tags in the editor; captions escaped in feeds
Two bugs from the backlog (High), confirmed against the code at v2.14.0.
## Clearing tags
`routes/admin.py` (edit POST) passed `tags=tags if tags else None`, and
`notes.update_note` treats `None` as "unchanged", so an empty Tags field
kept the old tags although the form says "Leave blank to remove all tags".
It now passes the parsed list (possibly empty) whenever the form has a
`tags` field, and `None` only when it doesn't.
## Captions in feed HTML
The media `<img>` built for item HTML put the caption into `alt="…"`
unescaped in four places: `feeds/rss.py` (two), `feeds/atom.py` and
`feeds/json_feed.py`. A caption with `"`, `<` or `&` broke the markup. The
caption and the media URL are now `html.escape(..., quote=True)`d. The
Atom enclosure's `title` attribute was already escaped with `_escape_xml`
and is unchanged (escaping it again would double it).
## Verification
`tests/test_tags_and_captions.py` (4 tests, all failing before the fix):
caption escaped once in RSS, Atom (and the enclosure title not doubled) and
JSON Feed; the editor leaves tags alone without the field and clears them
with an empty one. Feed, admin, tag, public-route and media tests pass.
+22 -15
View File
@@ -1,9 +1,9 @@
# StarPunk Backlog
**Last Updated**: 2026-11-03 (current release: v2.11.0)
**Last Updated**: 2026-11-10 (current release: v2.14.0)
These are the open features and bugs, sorted by priority. Each item was checked against the code
at v2.11.0. Work that has shipped is listed per release in `vX.Y.Z/RELEASE.md` and in
at v2.14.0. Work that has shipped is listed per release in `vX.Y.Z/RELEASE.md` and in
`CHANGELOG.md`.
## Priority Levels
@@ -29,16 +29,6 @@ None open.
## Medium
### Admin editor cannot clear all tags
- **Description**: When a note is saved with an empty tag field, the route passes `tags=None`. `update_note` treats that as "unchanged", so the last tags can't be removed from the editor.
- **Where**: `starpunk/routes/admin.py` (edit POST: `tags=tags if tags else None`), `starpunk/notes.py` (`update_note`)
- **Source**: Old backlog (v1.3.0 follow-up); still present
### Media captions not escaped in RSS HTML
- **Description**: The RSS description embeds `<img alt="{caption}">` without escaping the caption. A caption containing `"` or `<` breaks the item's HTML.
- **Where**: `starpunk/feeds/rss.py` (two places, ~lines 136 and 277)
- **Source**: Old backlog (Architect Review 2.2.10); still present
### No rate limiting on upload endpoints
- **Description**: Media uploads (`/admin/new` and the Micropub media endpoint) have no rate limit. Both require authentication, so the risk comes from a leaked token or session.
- **Where**: `starpunk/routes/admin.py`, `starpunk/routes/micropub.py`
@@ -75,7 +65,7 @@ None open.
- **Source**: Documentation refresh 2026-11-03
### Tag-filtered feeds
- **Description**: Tag pages are h-feeds and WebSub topics, but there are no RSS, Atom or JSON feeds per tag (for example `/tag/python/feed.atom`). Tag pages also have no pagination, and the editor has no tag autocomplete.
- **Description**: Tag pages are h-feeds and WebSub topics, but there are no RSS, Atom or JSON feeds per tag (for example `/tag/python/feed.atom`). Tag pages also have no pagination, and the editor has no tag autocomplete (Micropub `q=category`, v2.12.1, already gives the list a `<datalist>` could use).
- **Where**: `starpunk/routes/public.py`, `starpunk/feeds/`, `templates/admin/`
- **Source**: Old backlog (Tag enhancements)
@@ -83,6 +73,16 @@ None open.
## Low
### Quote counts without names are not shown
- **Description**: Backfeed stores anonymous quote counts (`syndications.quotes_count`: Threads quotes, and quotes by people who opted out elsewhere), but the counts line only adds likes and reposts, so these quotes are counted nowhere on the page.
- **Where**: `templates/partials/responses.html` (`reactions`), `starpunk/webmention_queries.py` (`response_counts`)
- **Source**: Documentation review 2026-11-10
### Dead files and settings
- **Description**: `static/js/preview.js` is empty, yet the editor pages load it and `marked` from jsDelivr for nothing. `templates/feed.xml` is empty and `static/indielogin-test.html` is a v1 leftover. `syndication.get_targets()` reads a `THREADS_API` setting that `config.py` never loads, so it can't be set.
- **Where**: `templates/admin/new.html`, `templates/admin/edit.html`, `static/`, `templates/feed.xml`, `starpunk/syndication.py`
- **Source**: Documentation review 2026-11-10
### Micropub: `location` / `checkin` ignored
- **Description**: These properties are accepted but dropped, which spec 3.3.5 allows.
- **Where**: `starpunk/micropub.py`
@@ -154,7 +154,7 @@ None open.
- **Source**: Old backlog (v1.5.0 scoping)
### Unused settings and legacy tables
- **Description**: `NOTES_PATH` is read and its folder created, but notes are always written under `DATA_PATH/notes`. `SITE_AUTHOR` is required but shown nowhere. `INITIAL_SCHEMA_SQL` recreates the unused `tokens` and `authorization_codes` tables (dropped by migration 004) on every start, and `auth_state` is unused. Version fallbacks are stale (`author_discovery.py` "1.2.0", `base.html` "0.5.0").
- **Description**: `NOTES_PATH` is read and its folder created, but notes are always written under `DATA_PATH/notes`. `SITE_AUTHOR` is checked as required but defaults to "Unknown" and is shown nowhere. `INITIAL_SCHEMA_SQL` recreates the unused `tokens` and `authorization_codes` tables (dropped by migration 004) on every start, and `auth_state` is unused. Version fallbacks are stale (`author_discovery.py` "1.2.0", `base.html` "0.5.0").
- **Where**: `starpunk/config.py`, `starpunk/utils.py`, `starpunk/database/schema.py`, `starpunk/author_discovery.py`, `templates/base.html`
- **Source**: Documentation refresh 2026-11-03 (docs/architecture/database.md, deployment.md)
@@ -171,10 +171,17 @@ None open.
---
## Done since the last update (2025-12-17)
## Done (recent releases)
These items were removed from the backlog because they have shipped:
- v2.14.1: the editor can clear all tags; image captions escaped in RSS, Atom and JSON Feed HTML
- v2.14.0: notifications of new responses and crosspost/backfeed problems through Apprise (ADR-074)
- v2.13.0–v2.13.4: backfeed of likes, reposts, quotes and replies from GoToSocial, Bluesky and Threads (ADR-073); counts line with web actions; counts on lists of notes; local avatar copies
- v2.12.1: Micropub `q=category` lists only tags in use
- v2.12.0: metrics dashboard and monitoring package removed (ADR-072)
- LICENSE: MIT, copyright Phil Skentelbery 2025-2026 (2026-11-03)
- v2.11.0: SQLite foreign keys enforced (orphans cleaned up by migration 018); CSRF tokens on admin, logout and consent forms; IndieAuth client and crosspost link-card fetches moved onto the SSRF-safe fetcher; `FLASK_DEBUG`/`FLASK_ENV` default to off/production; rate limits shared by all processes (and old windows pruned, so the webmention limiter no longer grows); dependencies upgraded (Python 3.14, no known vulnerabilities); stale `compose.yaml`, duplicate `dev_auth.py` and `test.ini` removed or fixed; `lxml`/`python-dateutil` pinned, `pytest` out of the runtime requirements
+15 -7
View File
@@ -4,19 +4,19 @@ Future work and per-release plans. See `../DOCUMENTATION.md` (PROJECTPLAN).
## Backlog
- **[BACKLOG.md](BACKLOG.md)** - Open features and bugs by priority (updated for v2.11.0)
- **[BACKLOG.md](BACKLOG.md)** - Open features and bugs by priority (updated for v2.14.0)
## Releases
- **[v1.0.0](v1.0.0/RELEASE.md)** (Released 2025-11-24)
- **[v1.1.0](v1.1.0/RELEASE.md)** (Released 2025-11-25)
- **[v1.1.1](v1.1.1/RELEASE.md)** (Released 2025-11-26)
- **[v1.1.2](v1.1.2/RELEASE.md)** (Released 2025-11-27)
- **[v1.1.1](v1.1.1/RELEASE.md)** (Released 2025-11-25)
- **[v1.1.2](v1.1.2/RELEASE.md)** (Released 2025-11-28)
- **[v1.2.0](v1.2.0/RELEASE.md)** (Released 2025-12-09)
- **[v1.3.0](v1.3.0/RELEASE.md)** (Complete)
- **[v1.3.1](v1.3.1/RELEASE.md)** - Feed Categories/Tags Support (Planning)
- **[v1.4.0](v1.4.0/RELEASE.md)** - Micropub Media Endpoint, Large Image Support, Enhanced Feed Media (Planning)
- **[v1.5.0](v1.5.0/RELEASE.md)** - Stability, Test Coverage, and Technical Debt Reduction (Approved)
- **[v1.3.0](v1.3.0/RELEASE.md)** (Released 2025-12-10)
- **[v1.3.1](v1.3.1/RELEASE.md)** - Feed Categories/Tags Support (Released 2025-12-10)
- **[v1.4.0](v1.4.0/RELEASE.md)** - Micropub Media Endpoint, Large Image Support, Enhanced Feed Media (Released 2025-12-16 as 1.4.0rc1, then 1.4.1 and 1.4.2)
- **[v1.5.0](v1.5.0/RELEASE.md)** - Stability, Test Coverage, and Technical Debt Reduction (Released 2025-12-17)
- **[v2.0.0](v2.0.0/RELEASE.md)** - OIDC admin login and built-in IndieAuth server (Released 2026-10-31)
- **[v2.0.1](v2.0.1/RELEASE.md)** - Absolute URLs behind HTTPS proxy (Released 2026-10-31)
- **[v2.0.2](v2.0.2/RELEASE.md)** - CI registry login (Released 2026-10-31)
@@ -49,3 +49,11 @@ Future work and per-release plans. See `../DOCUMENTATION.md` (PROJECTPLAN).
- **[v2.10.2](v2.10.2/RELEASE.md)** - Reply context fetch fix (Released 2026-11-03)
- **[v2.11.0](v2.11.0/RELEASE.md)** - Maintenance: dependency upgrades, High backlog items (Released 2026-11-03)
- **[v2.12.0](v2.12.0/RELEASE.md)** - Metrics dashboard removed (Released 2026-11-03)
- **[v2.12.1](v2.12.1/RELEASE.md)** - `q=category` lists only tags in use (Released 2026-11-10)
- **[v2.13.0](v2.13.0/RELEASE.md)** - Backfeed of responses to crossposts (Released 2026-11-10)
- **[v2.13.1](v2.13.1/RELEASE.md)** - Avatar fetching fixes (Released 2026-11-10)
- **[v2.13.2](v2.13.2/RELEASE.md)** - Counts line and web actions (Released 2026-11-10)
- **[v2.13.3](v2.13.3/RELEASE.md)** - Opted-out people counted without names (Released 2026-11-10)
- **[v2.13.4](v2.13.4/RELEASE.md)** - Counts on lists of notes, reply cards link out (Released 2026-11-10)
- **[v2.14.0](v2.14.0/RELEASE.md)** - Notifications through Apprise (Released 2026-11-10)
- **[v2.14.1](v2.14.1/RELEASE.md)** - Clearing tags in the editor; captions escaped in feeds (Released 2026-11-10)
+1 -1
View File
@@ -1,6 +1,6 @@
# StarPunk v1.1.1 Release
**Status**: Released 2025-11-26
**Status**: Released 2025-11-25
## Features
+1 -1
View File
@@ -1,6 +1,6 @@
# StarPunk v1.1.2 Release
**Status**: Released 2025-11-27
**Status**: Released 2025-11-28
**Codename**: Syndicate
## Features
+1 -1
View File
@@ -1,6 +1,6 @@
# StarPunk v1.3.0 Release
**Status**: Complete
**Status**: Released 2025-12-10
**Codename**: "Categories"
**Branch**: `feature/v1.3.0-tags-microformats`
**Approved**: 2025-12-10
+1 -1
View File
@@ -1,6 +1,6 @@
# StarPunk v1.3.1 Release
**Status**: Planning
**Status**: Released 2025-12-10
**Codename**: "Syndicate Tags"
**Focus**: Feed Categories/Tags Support
+1 -1
View File
@@ -1,6 +1,6 @@
# StarPunk v1.4.0 Release
**Status**: Planning
**Status**: Released 2025-12-16 (as 1.4.0rc1; then 1.4.1 and 1.4.2)
**Codename**: "Media"
**Focus**: Micropub Media Endpoint, Large Image Support, Enhanced Feed Media
+1 -1
View File
@@ -1,6 +1,6 @@
# StarPunk v1.5.0 Release Plan
**Status**: Approved
**Status**: Released 2025-12-17
**Codename**: "Trigger"
**Focus**: Stability, Test Coverage, and Technical Debt Reduction
**Last Updated**: 2025-12-17
+1 -1
View File
@@ -1,7 +1,7 @@
# v2.10.2
- **Date**: 2026-11-03
- **Status**: Released
- **Status**: Released 2026-11-03
## Bugs fixed
+1 -1
View File
@@ -1,7 +1,7 @@
# v2.11.0
- **Date**: 2026-11-03
- **Status**: Released
- **Status**: Released 2026-11-03
- **Focus**: Maintenance: dependency upgrades and the High backlog items
## Security
+1 -1
View File
@@ -1,7 +1,7 @@
# v2.12.0
- **Date**: 2026-11-03
- **Status**: Released
- **Status**: Released 2026-11-03
- **Focus**: Remove the metrics dashboard
## Removed
+14
View File
@@ -0,0 +1,14 @@
# v2.12.1
- **Date**: 2026-11-10
- **Status**: Released
- **Focus**: Micropub `q=category` lists only tags in use
## Fixed
- `q=category` returns only tags on posts that exist (not ones an edit removed, nor ones only on deleted posts)
## Related
- [CHANGELOG](../../../CHANGELOG.md)
- [Design note](../../design/v2.12.1/2026-11-10-category-unused-tags.md)
+25
View File
@@ -0,0 +1,25 @@
# v2.13.0
- **Date**: 2026-11-10
- **Status**: Released
- **Focus**: Backfeed of responses to crossposts
## Added
- Likes, reposts, quotes and replies to GoToSocial and Bluesky crossposts, and replies on Threads, shown on the note
- Age-based checking schedule up to `BACKFEED_MAX_AGE_DAYS` (90); admin backfill and per-crosspost Check now
- Threads like/repost/quote counts; a Quotes group; origin filter and Deleted view in the admin
- Settings `BACKFEED_ENABLED`, `BACKFEED_MAX_AGE_DAYS`; migration 020
## Changed
- Response avatars saved locally (`/avatars/`)
- Admin delete keeps a tombstone
- `#nobridge` / `#nobot` and Bluesky's logged-out label honoured; already backfed responses not shown twice from Bridgy
- Threads client on `graph.threads.com`
## Related
- [CHANGELOG](../../../CHANGELOG.md)
- [ADR-073](../../decisions/ADR-073-backfeed.md)
- [Design note](../../design/v2.13.0/2026-11-10-backfeed.md)
+15
View File
@@ -0,0 +1,15 @@
# v2.13.1
- **Date**: 2026-11-10
- **Status**: Released
- **Focus**: Avatar fetching in production
## Fixed
- Avatars from the site's own GoToSocial (resolves to an internal address) and from Bluesky (CDN refuses some server IPv4 addresses; fetched from the PDS instead)
## Related
- [CHANGELOG](../../../CHANGELOG.md)
- [ADR-073](../../decisions/ADR-073-backfeed.md)
- [Design note](../../design/v2.13.1/2026-11-10-avatar-fetch.md)
+20
View File
@@ -0,0 +1,20 @@
# v2.13.2
- **Date**: 2026-11-10
- **Status**: Released
- **Focus**: Counts line for likes and reposts; web actions
## Changed
- Likes, reposts and bookmarks shown as one line of counts under the note, with who on hover; Threads counts folded in
- `<indie-action>` web actions with links to the note's crossposts
## Fixed
- Avatars GoToSocial hasn't cached (redirects to the original server)
## Related
- [CHANGELOG](../../../CHANGELOG.md)
- [ADR-073](../../decisions/ADR-073-backfeed.md)
- [Design note](../../design/v2.13.2/2026-11-10-reaction-counts.md)
+15
View File
@@ -0,0 +1,15 @@
# v2.13.3
- **Date**: 2026-11-10
- **Status**: Released
- **Focus**: Opted-out people counted without names
## Changed
- Likes and reposts by people who opted out of backfeed count in the totals, without name or avatar
## Related
- [CHANGELOG](../../../CHANGELOG.md)
- [ADR-073](../../decisions/ADR-073-backfeed.md)
- [Design note](../../design/v2.13.3/2026-11-10-count-opted-out.md)
+22
View File
@@ -0,0 +1,22 @@
# v2.13.4
- **Date**: 2026-11-10
- **Status**: Released
- **Focus**: Counts on lists of notes; reply cards
## Added
- Like, repost and reply counts on the home, tag and Likes pages
## Changed
- A whole reply card opens the reply where it was posted
## Fixed
- The counts line's CSS clashed with the Liked / Reposted / In reply to lines (`.reaction`)
## Related
- [CHANGELOG](../../../CHANGELOG.md)
- [Design note](../../design/v2.13.4/2026-11-10-list-counts.md)
+16
View File
@@ -0,0 +1,16 @@
# v2.14.0
- **Date**: 2026-11-10
- **Status**: Released
- **Focus**: Notifications through Apprise
## Added
- Notifications of new responses and crosspost/backfeed problems via the embedded Apprise library
- Settings `NOTIFY_URLS`, `NOTIFY_EVENTS`, `NOTIFY_PRIORITY`; admin status and Send test notification; migration 021
## Related
- [CHANGELOG](../../../CHANGELOG.md)
- [ADR-074](../../decisions/ADR-074-notifications.md)
- [Design note](../../design/v2.14.0/2026-11-10-notifications.md)
+15
View File
@@ -0,0 +1,15 @@
# v2.14.1
- **Date**: 2026-11-10
- **Status**: Released
- **Focus**: The two High backlog bugs
## Fixed
- The editor can clear all of a note's tags
- Image captions escaped in RSS, Atom and JSON Feed item HTML
## Related
- [CHANGELOG](../../../CHANGELOG.md)
- [Design note](../../design/v2.14.1/2026-11-10-tags-and-captions.md)
@@ -1,11 +1,11 @@
# Utility Function Patterns
**Status**: Active
**Last Updated**: 2026-11-03 (v2.10.1; reviewed, content from 2025-11-18)
**Status**: Deprecated (2026-11-10). Superseded by [python-coding-standards.md](../python-coding-standards.md), which this document contradicts on type hints (`X | None`, built-in generics, hints on new and changed public functions).
**Last Updated**: 2026-11-03 (v2.10.1; content from 2025-11-18)
## Purpose
This document establishes coding patterns and conventions specifically for utility functions in StarPunk (`starpunk/utils.py`, `starpunk/slug_utils.py`, `starpunk/mf2_utils.py` and helpers in other modules). These patterns ensure utilities are consistent, testable, and maintainable. General rules, including project-wide patterns for configuration, database access and outbound HTTP, are in [python-coding-standards.md](python-coding-standards.md); where the two differ, that document wins.
This document establishes coding patterns and conventions specifically for utility functions in StarPunk (`starpunk/utils.py`, `starpunk/slug_utils.py`, `starpunk/mf2_utils.py` and helpers in other modules). These patterns ensure utilities are consistent, testable, and maintainable. General rules, including project-wide patterns for configuration, database access and outbound HTTP, are in [python-coding-standards.md](../python-coding-standards.md); where the two differ, that document wins.
The examples are illustrative. Several mirror real functions in `starpunk/utils.py` (`calculate_content_hash`, `write_note_file`, `validate_slug`, `generate_slug`, `validate_note_path`); others (`require_valid_slug`, `iter_note_files`) only show the pattern.
@@ -731,7 +731,7 @@ def generate_slug(content: str) -> str:
## References
- [Python Coding Standards](python-coding-standards.md)
- [Python Coding Standards](../python-coding-standards.md)
- [PEP 8 - Style Guide](https://peps.python.org/pep-0008/)
- [PEP 257 - Docstring Conventions](https://peps.python.org/pep-0257/)
- [PEP 484 - Type Hints](https://peps.python.org/pep-0484/)
+7 -7
View File
@@ -1,10 +1,10 @@
# Standards Index
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
Standards for developing and releasing StarPunk. Folder rules are in
[../DOCUMENTATION.md](../DOCUMENTATION.md). All standards below were checked
against the code at v2.10.1.
against the code at v2.14.0.
## Active standards
@@ -16,7 +16,6 @@ against the code at v2.10.1.
| [versioning-strategy.md](versioning-strategy.md) | Choosing MAJOR / MINOR / PATCH |
| [version-implementation-guide.md](version-implementation-guide.md) | Where the version lives, changelog entry and design note format |
| [python-coding-standards.md](python-coding-standards.md) | Style, naming, and project patterns (config, database, outbound HTTP, errors, routes, workers) |
| [utility-function-patterns.md](utility-function-patterns.md) | Patterns for small helper functions |
| [cookie-naming-convention.md](cookie-naming-convention.md) | `starpunk_` cookie prefix and the cookies in use |
| [documentation-organization.md](documentation-organization.md) | Templates and conventions for docs (folder rules: DOCUMENTATION.md) |
@@ -31,8 +30,8 @@ against the code at v2.10.1.
## Key rules in brief
- `uv` for environments, `uv run` for every command.
- Tests need `SESSION_SECRET`; the full suite takes about 6 minutes, 5 of
them in `tests/test_migration_race_condition.py`.
- Tests need `SESSION_SECRET`. Locally, run only the tests for the area you
changed; CI runs the full suite (about 6 minutes) on every `vX.Y.Z` tag.
- Every release: branch in a worktree, bump `__version__` and
`__version_info__`, changelog entry, design note (or ADR amendment),
`git merge --no-ff`, annotated `vX.Y.Z` tag; CI tests and publishes the
@@ -43,8 +42,9 @@ against the code at v2.10.1.
## Deprecated standards
Standards that no longer apply move to `DEPRECATED/` (none at
present). Earlier versions of the standards above are in git history.
Standards that no longer apply move to `DEPRECATED/`:
[utility-function-patterns.md](DEPRECATED/utility-function-patterns.md)
(superseded by python-coding-standards.md, 2026-11-10). Earlier versions of the standards above are in git history.
## Related
+1 -1
View File
@@ -2,7 +2,7 @@
**Status**: Active
**Date**: 2025-11-18
**Last Updated**: 2026-11-03 (v2.10.1; cookie list checked against the code)
**Last Updated**: 2026-11-10 (v2.14.0; cookie list checked against the code)
**Version**: 1.1
## Purpose
+9 -8
View File
@@ -1,7 +1,7 @@
# Development Setup
**Status**: Active
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
How to set up a local StarPunk environment, run the app and run the tests.
Each configuration variable is described in [`.env.example`](../../.env.example);
@@ -17,8 +17,9 @@ deployment is in [architecture/deployment.md](../architecture/deployment.md).
## Environment
The repository has no `pyproject.toml`; dependencies are in
`requirements.txt` (runtime) and `requirements-dev.txt` (adds pytest-cov,
pytest-mock, black, flake8, mypy).
`requirements.txt` (runtime) and `requirements-dev.txt` (adds
`requirements-test.txt` (pytest, pytest-mock), plus pytest-cov, black, flake8,
mypy and pip-audit).
```bash
git clone https://git.thesatelliteoflove.com/phil/StarPunk.git
@@ -100,9 +101,9 @@ docker run --rm -p 8000:8000 --env-file .env -v ./container-data:/data starpunk:
## Running the tests
```bash
SESSION_SECRET=test uv run pytest # full suite
SESSION_SECRET=test uv run pytest tests/test_websub.py # one file
SESSION_SECRET=test uv run pytest tests/test_websub.py # the area you changed
SESSION_SECRET=test uv run pytest tests/test_micropub.py -k delete -x
SESSION_SECRET=test uv run pytest # full suite (CI runs this; rarely needed locally)
```
- `SESSION_SECRET` must be set (in `.env` or the environment) because
@@ -113,9 +114,9 @@ SESSION_SECRET=test uv run pytest tests/test_micropub.py -k delete -x
`DATA_PATH` and database, so tests never touch `./data`. Under
`TESTING=True` the background workers don't start and
the production OIDC check is skipped.
- `tests/test_migration_race_condition.py` is slow (about 5 minutes). While
iterating, add `--ignore=tests/test_migration_race_condition.py`; run the
full suite before releasing.
- Locally, run the tests for the area you changed; the full suite, including
the slow `tests/test_migration_race_condition.py` (about 5 minutes), runs in
CI on the release tag.
- Coverage: `SESSION_SECRET=test uv run pytest --cov=starpunk --cov-report=term-missing`.
- The CI test stage can be reproduced exactly with
`docker build --target test -f Containerfile .` (see
+2 -2
View File
@@ -1,7 +1,7 @@
# Documentation Organization
**Status**: Active
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
**[docs/DOCUMENTATION.md](../DOCUMENTATION.md) is authoritative** for what
goes in which folder and supersedes this document. This standard only adds
@@ -94,5 +94,5 @@ top saying what replaced it, and removed from the active list in
- Architecture documents describing a design that no longer exists are
either rewritten or renamed with a `-legacy` suffix and a note at the top
(e.g. `architecture/technology-stack-legacy.md`).
(e.g. `design/v0.x/technology-stack-legacy.md`).
- Design notes and ADRs are historical records and stay where they are.
+5 -3
View File
@@ -1,7 +1,7 @@
# Git Branching and Release Process
**Status**: Active
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
How changes reach `main`, how releases are cut, and what happens after a
tag is pushed. This is the flow used for the v2 releases (see
@@ -27,6 +27,7 @@ differ, this document describes current practice.
| `feat/` | new behaviour (MINOR) | `feat/v2.10.0-websub` |
| `fix/` | bug and security fixes (PATCH) | `fix/v2.10.1-websub-hardening` |
| `ci/` | build and CI changes | `ci/v2.7.1-pytest` |
| `chore/` | maintenance, removals, licensing | `chore/remove-metrics` |
| `docs/` | documentation only, no release | `docs/refresh-v2.10.1` |
Release branches are named `<prefix>/vX.Y.Z-<short-description>`, so the
@@ -83,8 +84,9 @@ Code, tests and docs together:
- [ ] Add a design note `docs/design/vX.Y.Z/YYYY-MM-DD-<topic>.md` (what
changed and why, how it was verified). Small releases fully covered by
an ADR amendment and the changelog may skip it
- [ ] Full test suite passes:
`SESSION_SECRET=test uv run pytest`
- [ ] Tests for the changed area pass locally, e.g.
`SESSION_SECRET=test uv run pytest tests/test_notifications.py`;
CI runs the full suite on the tag (step 4)
Commit (usually one commit):
+2 -4
View File
@@ -1,12 +1,11 @@
# Python Coding Standards
**Status**: Active
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
How Python code in StarPunk is written. The project rule comes first:
"Every line of code must justify its existence. When in doubt, leave it
out." Patterns specific to small helper functions are in
[utility-function-patterns.md](utility-function-patterns.md).
out."
## Base style
@@ -147,5 +146,4 @@ functions or `Test...` classes with descriptive names
## References
- [PEP 8](https://peps.python.org/pep-0008/), [PEP 257](https://peps.python.org/pep-0257/), [PEP 484](https://peps.python.org/pep-0484/)
- [utility-function-patterns.md](utility-function-patterns.md)
- [ADR-055: Error handling philosophy](../decisions/ADR-055-error-handling-philosophy.md)
+12 -9
View File
@@ -1,7 +1,7 @@
# Testing Checklist
**Status**: Active
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
What to test, how to run the tests, and what CI does with them. Setup and
the environment variables the tests need are in
@@ -12,12 +12,14 @@ the environment variables the tests need are in
**Locally**
```bash
SESSION_SECRET=test uv run pytest # everything
SESSION_SECRET=test uv run pytest --ignore=tests/test_migration_race_condition.py # fast loop
SESSION_SECRET=test uv run pytest tests/test_websub.py tests/test_websub_hardening.py # the area you changed
SESSION_SECRET=test uv run pytest # everything (CI does this)
```
`tests/test_migration_race_condition.py` takes about 5 minutes; skip it
while iterating, never before a release.
Locally, run the tests for the area you changed. The full suite, including
the 5-minute `tests/test_migration_race_condition.py`, runs in CI. For wide
changes (migrations, `config.py`, `create_app`, `tests/conftest.py`) you can
build the CI test stage first: `docker build --target test -f Containerfile .`
**In CI** (`.gitea/workflows/build-container.yml`), on every pushed
`vX.Y.Z` tag (or a manual run):
@@ -30,16 +32,17 @@ while iterating, never before a release.
2. Only if that succeeds is the runtime image built and pushed as
`:vX.Y.Z` and `:latest`.
So a failing test blocks the release image; it does not block the merge or
the tag. Run the full suite before merging (see
[git-branching-strategy.md](git-branching-strategy.md#release-process)).
So a failing test blocks the release image, not the merge or the tag; CI is
where the full suite runs. If it fails, fix it in the next PATCH (see
[git-branching-strategy.md](git-branching-strategy.md#release-process),
step 4).
To reproduce CI locally: `docker build --target test -f Containerfile .`
(pytest-cov is not installed in that stage, so don't depend on it in
tests).
## Every change
- [ ] Full suite passes locally
- [ ] Tests for the changed area pass locally (CI runs the full suite)
- [ ] New behaviour has tests; a bug fix has a test that failed before the fix
- [ ] Tests use the `app` / `client` fixtures from `tests/conftest.py`
(temporary data directory, `TESTING=True`); nothing touches `./data`
@@ -1,7 +1,7 @@
# Version Implementation Guide
**Status**: Active
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
Where the version number lives and what to write for each release. Which
number to pick: [versioning-strategy.md](versioning-strategy.md). Git steps:
@@ -14,8 +14,8 @@ There is one source: `starpunk/__init__.py`, at the end of the file.
```python
# Package version (Semantic Versioning 2.0.0)
# See docs/standards/versioning-strategy.md for details
__version__ = "2.10.1"
__version_info__ = (2, 10, 1)
__version__ = "X.Y.Z"
__version_info__ = (X, Y, Z)
```
Update both lines. Nothing else holds the number: there is no
+2 -2
View File
@@ -1,10 +1,10 @@
# Versioning Strategy
**Status**: Active
**Last Updated**: 2026-11-03 (v2.10.1)
**Last Updated**: 2026-11-10 (v2.14.0)
StarPunk uses [Semantic Versioning 2.0.0](https://semver.org/):
`MAJOR.MINOR.PATCH`, currently **2.13.3**. Every release is a git tag
`MAJOR.MINOR.PATCH` (the current version is in `starpunk/__init__.py`). Every release is a git tag
`vX.Y.Z` and a `CHANGELOG.md` entry. Where the number is stored and how to
write the changelog entry is in
[version-implementation-guide.md](version-implementation-guide.md); the
+6
View File
@@ -0,0 +1,6 @@
-- Migration 021: Notifications (v2.14.0, ADR-074)
-- When the owner was notified of a response. Everything already stored
-- counts as notified, so turning notifications on doesn't send history.
ALTER TABLE webmentions ADD COLUMN notified_at TIMESTAMP;
UPDATE webmentions SET notified_at = CURRENT_TIMESTAMP;
+3
View File
@@ -19,6 +19,9 @@ httpx==0.28.*
PyJWT[crypto]==2.15.*
python-dotenv==1.2.*
# Notifications (v2.14.0, ADR-074)
apprise==2.2.*
# Images
Pillow==13.0.*
pillow-heif==1.8.*
+2 -2
View File
@@ -465,5 +465,5 @@ def create_app(config=None):
# Package version (Semantic Versioning 2.0.0)
# See docs/standards/versioning-strategy.md for details
__version__ = "2.13.3"
__version_info__ = (2, 13, 3)
__version__ = "2.14.1"
__version_info__ = (2, 14, 1)
+19 -6
View File
@@ -258,12 +258,14 @@ def hidden_counts(harvest: Harvest) -> Dict[str, int]:
(("likes", "like"), ("reposts", "repost"), ("quotes", "quote"))}
def store(db, note_id: int, target_url: str, origin: str, harvest: Harvest) -> None:
def store(db, note_id: int, target_url: str, origin: str, harvest: Harvest,
quiet: bool = False) -> None:
"""
Upsert a check's responses for one note and silo. With a complete
harvest, that silo's responses for the note that weren't found are
marked invalid. Responses from people who opted out are removed (they
are counted instead, see `hidden_counts`).
are counted instead, see `hidden_counts`). With `quiet`, new responses
are stored as already notified (ADR-074 Decision 5).
"""
from starpunk.webmention_processor import _snippet
@@ -289,8 +291,9 @@ def store(db, note_id: int, target_url: str, origin: str, harvest: Harvest) -> N
INSERT INTO webmentions (
source_url, target_url, target_note_id, mention_type, author_name, author_url,
author_photo, content_snippet, validation_status, moderation_status,
validated_at, source_published, origin, remote_id)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'valid', ?, CURRENT_TIMESTAMP, ?, ?, ?)
validated_at, source_published, origin, remote_id, notified_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'valid', ?, CURRENT_TIMESTAMP, ?, ?, ?,
CASE WHEN ? THEN CURRENT_TIMESTAMP END)
ON CONFLICT(source_url, target_url) DO UPDATE SET
target_note_id = excluded.target_note_id,
mention_type = excluded.mention_type,
@@ -307,7 +310,8 @@ def store(db, note_id: int, target_url: str, origin: str, harvest: Harvest) -> N
updated_at = CURRENT_TIMESTAMP
""",
(r.source_url, target_url, note_id, r.kind, r.author_name or None, r.author_url or None,
r.author_photo or None, snippet, moderation, r.published, origin, r.remote_id),
r.author_photo or None, snippet, moderation, r.published, origin, r.remote_id,
1 if quiet else 0),
)
if dropped:
marks = ",".join("?" * len(dropped))
@@ -373,7 +377,11 @@ def check(row, target) -> None:
db = get_db(current_app)
site = current_app.config["SITE_URL"].rstrip("/")
harvest = target.client.responses(row["remote_id"], row["remote_url"] or "")
store(db, row["note_id"], f"{site}/note/{row['note_slug']}", row["target"], harvest)
# No notifications for history: a backfill, or a copy's first check on an
# older note (e.g. one linked after the fact). ADR-074 Decision 5.
quiet = bool(row["backfeed_backfill"]) or (
row["backfeed_checked_at"] is None and _age_seconds(row["note_created"]) > DAY)
store(db, row["note_id"], f"{site}/note/{row['note_slug']}", row["target"], harvest, quiet=quiet)
_store_counts(db, row["id"], hidden_counts(harvest))
@@ -398,6 +406,11 @@ def process_due(limit: int = 3) -> int:
if _is_denied(e):
denied[row["target"]] = f"Needs {SCOPES.get(row['target'], 'read access')}: {e}"
current_app.logger.error(f"Backfeed for {row['target']} stopped: {denied[row['target']]}")
from starpunk import notifications
site = current_app.config["SITE_URL"].rstrip("/")
notifications.problem(current_app.config, current_app.config["DATABASE_PATH"],
f"Backfeed for {SILO_NAMES.get(row['target'], row['target'])} stopped",
denied[row["target"]], f"{site}/admin/webmentions")
db.execute("UPDATE syndications SET backfeed_claimed_until = NULL WHERE id = ?", (row["id"],))
db.commit()
continue
+4
View File
@@ -149,6 +149,10 @@ def load_config(app, config_override=None):
# Backfeed of responses to crossposts (v2.13.0, ADR-073)
app.config["BACKFEED_ENABLED"] = os.getenv("BACKFEED_ENABLED", "false").lower() == "true"
app.config["BACKFEED_MAX_AGE_DAYS"] = int(os.getenv("BACKFEED_MAX_AGE_DAYS", "90"))
# Notifications through Apprise (v2.14.0, ADR-074)
app.config["NOTIFY_URLS"] = os.getenv("NOTIFY_URLS", "")
app.config["NOTIFY_EVENTS"] = os.getenv("NOTIFY_EVENTS", "reply,mention,quote,problems")
app.config["NOTIFY_PRIORITY"] = os.getenv("NOTIFY_PRIORITY", "normal")
# Apply overrides if provided
# CSRF protection for admin and consent forms (v2.11.0); on by default.
+3 -2
View File
@@ -16,6 +16,7 @@ Standards:
"""
# Standard library imports
from html import escape
from datetime import datetime, timezone
from typing import Optional
import xml.etree.ElementTree as ET
@@ -199,8 +200,8 @@ def generate_atom_streaming(
html_content += '<div class="media">'
for item in note.media:
media_url = f"{site_url}/media/{item['path']}"
caption = item.get('caption', '')
html_content += f'<img src="{media_url}" alt="{caption}" />'
caption = escape(item.get('caption') or '', quote=True) # v2.14.1
html_content += f'<img src="{escape(media_url, quote=True)}" alt="{caption}" />'
html_content += '</div>'
# Add text content below media
+3 -2
View File
@@ -16,6 +16,7 @@ Standards:
"""
# Standard library imports
from html import escape
from datetime import datetime, timezone
from typing import Optional, Dict, Any
import json
@@ -247,8 +248,8 @@ def _build_item_object(site_url: str, note: Note) -> Dict[str, Any]:
content_html += '<div class="media">'
for media_item in note.media:
media_url = f"{site_url}/media/{media_item['path']}"
caption = media_item.get('caption', '')
content_html += f'<img src="{media_url}" alt="{caption}" />'
caption = escape(media_item.get('caption') or '', quote=True) # v2.14.1
content_html += f'<img src="{escape(media_url, quote=True)}" alt="{caption}" />'
content_html += '</div>'
# Add text content below media
+5 -4
View File
@@ -20,6 +20,7 @@ Standards:
"""
# Standard library imports
from html import escape
from datetime import datetime, timezone
from typing import Optional
@@ -127,8 +128,8 @@ def generate_rss(
html_content += '<div class="media">'
for item in note.media:
media_url = f"{site_url}/media/{item['path']}"
caption = item.get('caption', '')
html_content += f'<img src="{media_url}" alt="{caption}" />'
caption = escape(item.get('caption') or '', quote=True) # v2.14.1
html_content += f'<img src="{escape(media_url, quote=True)}" alt="{caption}" />'
html_content += '</div>'
# Add text content below media
@@ -254,8 +255,8 @@ def generate_rss_streaming(
html_content += '<div class="media">'
for item in note.media:
media_url = f"{site_url}/media/{item['path']}"
caption = item.get('caption', '')
html_content += f'<img src="{media_url}" alt="{caption}" />'
caption = escape(item.get('caption') or '', quote=True) # v2.14.1
html_content += f'<img src="{escape(media_url, quote=True)}" alt="{caption}" />'
html_content += '</div>'
# Add text content below media
+236
View File
@@ -0,0 +1,236 @@
"""
Notifications through Apprise (v2.14.0, ADR-074)
The owner hears about new responses (received webmentions and backfeed
alike) and about crossposting problems, on whatever service the Apprise
URLs in NOTIFY_URLS point at (Gotify, ntfy, Pushover, email, an
apprise-api server…).
Responses are sent by a sweep (`sweep`) in the webmention worker, which
runs in one process at a time: each response with `notified_at` empty is
sent once and marked. Likes, reposts and bookmarks are grouped per note.
Failed sends are retried with backoff; responses still unsent after 24
hours are dropped. Problems (`problem`) are sent at once, without retry.
Nothing here may break validation, crossposting or backfeed: every send is
wrapped, and failures are only logged and shown in the admin.
"""
import html
import logging
import re
import sqlite3
import time
from typing import Dict, List, Optional, Tuple
from urllib.parse import parse_qsl, urlsplit
logger = logging.getLogger(__name__)
RESPONSE_EVENTS = ("reply", "mention", "quote", "like", "repost", "bookmark")
EVENTS = RESPONSE_EVENTS + ("problems",)
DEFAULT_EVENTS = "reply,mention,quote,problems"
GROUPED = ("like", "repost", "bookmark")
DROP_AFTER_HOURS = 24
MAX_BACKOFF = 15 * 60
SEND_TIMEOUT = 15
HEADINGS = {
"reply": "↩ Reply from {who}",
"mention": "💬 Mention from {who}",
"quote": "❝ Quote from {who}",
}
GROUP_HEADINGS = {
"like": ("♥", "like", "likes"),
"repost": ("🔁", "repost", "reposts"),
"bookmark": ("🔖", "bookmark", "bookmarks"),
}
# --- Settings ---------------------------------------------------------------------
def urls(config) -> List[str]:
"""NOTIFY_URLS as a list (space- or comma-separated)"""
return [u for u in re.split(r"[\s,]+", config.get("NOTIFY_URLS") or "") if u]
def events(config) -> set:
"""NOTIFY_EVENTS as a set of known event names"""
chosen = {e.strip().lower() for e in (config.get("NOTIFY_EVENTS") or DEFAULT_EVENTS).split(",")}
return chosen & set(EVENTS)
def enabled(config) -> bool:
return bool(urls(config))
def with_priority(url: str, priority: str) -> str:
"""The URL with `priority=` added, unless it sets one itself"""
if not priority or any(k == "priority" for k, _ in parse_qsl(urlsplit(url).query)):
return url
return url + ("&" if "?" in url else "?") + "priority=" + priority
# --- Sending ----------------------------------------------------------------------------
def _deliver(targets: List[str], title: str, body: str) -> Tuple[bool, str]:
"""Send one message to every URL. Returns (all sent, error text)."""
import apprise
sender = apprise.Apprise()
bad = [u.split("://", 1)[0] + "://…" for u in targets if not sender.add(u)]
if bad:
return False, f"Not a usable Apprise URL: {', '.join(bad)}"
with apprise.LogCapture(level=logging.WARNING, fmt="%(message)s") as log:
result = sender.notify(body, title=title, body_format=apprise.NotifyFormat.MARKDOWN,
timeout=SEND_TIMEOUT)
return bool(result), log.getvalue().strip()[:500]
def send(config, title: str, body: str) -> Tuple[bool, str]:
"""Send to all NOTIFY_URLS at NOTIFY_PRIORITY. Never raises."""
targets = [with_priority(u, config.get("NOTIFY_PRIORITY") or "normal") for u in urls(config)]
if not targets:
return False, "No NOTIFY_URLS set"
try:
ok, error = _deliver(targets, title, body)
except Exception as e: # a broken service mustn't break the caller
ok, error = False, f"{type(e).__name__}: {e}"
if not ok:
logger.warning(f"Notification '{title}' not sent: {error or 'unknown error'}")
return ok, error
def problem(config, db_path, title: str, detail: str, link: str) -> None:
"""A crosspost or backfeed problem, sent at once if 'problems' is selected"""
if not enabled(config) or "problems" not in events(config):
return
ok, error = send(config, f"⚠ {title}", f"{detail}\n\n[Open in StarPunk]({link})")
_record(db_path, ok, error)
def _record(db_path, ok: bool, error: str) -> None:
"""Remember the last failure for the admin page (cleared by a success)"""
try:
conn = sqlite3.connect(db_path)
try:
conn.execute("INSERT INTO site_state (key, value) VALUES ('notify_last_error', ?) "
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
("" if ok else time.strftime("%Y-%m-%d %H:%M:%S UTC ", time.gmtime())
+ (error or "unknown error"),))
conn.commit()
finally:
conn.close()
except sqlite3.Error as e:
logger.warning(f"Could not record notification result: {e}")
def last_error(db) -> str:
row = db.execute("SELECT value FROM site_state WHERE key = 'notify_last_error'").fetchone()
return row[0] if row else ""
# --- Messages -----------------------------------------------------------------------------
def _plain(snippet: Optional[str], limit: int = 300) -> str:
text = html.unescape(re.sub(r"<[^>]+>", " ", snippet or ""))
text = re.sub(r"\s+", " ", text).strip()
return text if len(text) <= limit else text[:limit - 1].rsplit(" ", 1)[0] + "…"
def _note_label(conn, note_id: int) -> str:
from starpunk.notes import get_note
try:
note = get_note(id=note_id, load_content=True)
label = " ".join((note.title or "").split()) if note else ""
except Exception:
label = ""
return (label[:57] + "…") if len(label) > 60 else (label or "a note")
def _messages(rows, site_url: str, conn) -> List[Tuple[str, str, List[int]]]:
"""(title, body, webmention ids) for a batch of responses"""
site = site_url.rstrip("/")
names = {"gotosocial": "GoToSocial", "bluesky": "Bluesky", "threads": "Threads"}
messages, groups = [], {}
for row in rows:
via = names.get(row["origin"]) if row["origin"] != "webmention" else None
who = (row["author_name"] or "someone") + (f" via {via}" if via else "")
pending = row["moderation_status"] == "pending"
link = (f"{site}/admin/webmentions/{row['id']}" if pending
else f"{site}/note/{row['slug']}#responses")
if row["mention_type"] in GROUPED:
groups.setdefault((row["target_note_id"], row["mention_type"], row["slug"], pending), []).append(
(row["id"], row["author_name"] or "someone", link))
continue
title = HEADINGS.get(row["mention_type"], "Response from {who}").format(who=who)
if pending:
title += " (awaiting moderation)"
text = _plain(row["content_snippet"])
body = (f"{text}\n\n" if text else "") + f"On “{_note_label(conn, row['target_note_id'])}”: [open]({link})"
messages.append((title, body, [row["id"]]))
for (note_id, kind, slug, pending), items in groups.items():
icon, one, many = GROUP_HEADINGS[kind]
n = len(items)
title = f"{icon} {n} new {one if n == 1 else many} on “{_note_label(conn, note_id)}”"
if pending:
title += " (awaiting moderation)"
people = ", ".join(name for _, name, _ in items[:5]) + (f" and {n - 5} more" if n > 5 else "")
messages.append((title, f"{people}\n\n[open]({items[0][2]})", [i for i, _, _ in items]))
return messages
# --- Sweep ----------------------------------------------------------------------------------
class Sweeper:
"""Sends new responses; one instance per webmention worker"""
def __init__(self):
self._failures = 0
self._next_try = 0.0
def run(self, config, db_path, site_url: str) -> int:
"""Send what's due. Returns how many responses were sent."""
if not enabled(config) or time.time() < self._next_try:
return 0
chosen = events(config)
conn = sqlite3.connect(db_path)
conn.row_factory = sqlite3.Row
try:
# Too old to be news (an outage, say): drop quietly
dropped = conn.execute(
f"""UPDATE webmentions SET notified_at = CURRENT_TIMESTAMP
WHERE notified_at IS NULL
AND received_at < datetime('now', '-{DROP_AFTER_HOURS} hours')""").rowcount
if dropped:
logger.warning(f"Dropped {dropped} notification(s) older than {DROP_AFTER_HOURS} hours")
rows = conn.execute(
"""
SELECT w.*, n.slug FROM webmentions w JOIN notes n ON n.id = w.target_note_id
WHERE w.notified_at IS NULL AND w.validation_status = 'valid'
AND w.moderation_status IN ('approved', 'pending')
ORDER BY w.id LIMIT 100
"""
).fetchall()
skip = [r["id"] for r in rows if r["mention_type"] not in chosen]
if skip:
conn.execute(f"UPDATE webmentions SET notified_at = CURRENT_TIMESTAMP "
f"WHERE id IN ({','.join('?' * len(skip))})", skip)
conn.commit()
due = [r for r in rows if r["mention_type"] in chosen]
sent = 0
for title, body, ids in _messages(due, site_url, conn):
ok, error = send(config, title, body)
_record(db_path, ok, error)
if not ok:
self._failures += 1
self._next_try = time.time() + min(10 * 2 ** self._failures, MAX_BACKOFF)
break
self._failures = 0
conn.execute(f"UPDATE webmentions SET notified_at = CURRENT_TIMESTAMP "
f"WHERE id IN ({','.join('?' * len(ids))})", ids)
conn.commit()
sent += len(ids)
return sent
finally:
conn.close()
+32 -1
View File
@@ -309,7 +309,9 @@ def update_note_submit(note_id: int):
id=note_id,
content=content,
published=published,
tags=tags if tags else None
# An empty Tags field removes all tags (v2.14.1); a form without
# the field leaves them alone
tags=tags if "tags" in request.form else None
)
if "syndication_form" in request.form:
from starpunk.syndication import select_targets
@@ -515,6 +517,7 @@ def webmentions():
status_filter=status_filter,
origin_filter=origin_filter,
backfeed=backfeed.status(),
notify=_notify_status(),
page=page,
total_pages=total_pages,
user_me=g.me
@@ -681,6 +684,34 @@ def revoke_token(token_id):
return redirect(url_for("admin.tokens"))
def _notify_status() -> dict:
"""Notification settings and last error for the Webmentions page (v2.14.0)"""
from starpunk import notifications
from starpunk.database import get_db
return {
"enabled": notifications.enabled(current_app.config),
"events": sorted(notifications.events(current_app.config)),
"priority": current_app.config.get("NOTIFY_PRIORITY") or "normal",
"services": [u.split("://", 1)[0] for u in notifications.urls(current_app.config)],
"last_error": notifications.last_error(get_db(current_app)),
}
@bp.route("/notifications/test", methods=["POST"])
@require_auth
def notifications_test():
"""Send a test notification to every NOTIFY_URLS service (v2.14.0)"""
from starpunk import notifications
ok, error = notifications.send(current_app.config, "StarPunk test notification",
"Notifications from StarPunk are working.")
notifications._record(current_app.config["DATABASE_PATH"], ok, error)
flash("Test notification sent" if ok else f"Test notification failed: {error or 'unknown error'}",
"success" if ok else "error")
return redirect(url_for("admin.webmentions"))
@bp.route("/backfeed/backfill", methods=["POST"])
@require_auth
def backfeed_backfill():
+16
View File
@@ -292,9 +292,23 @@ def index():
object.__setattr__(note, '_cached_tags', tags)
attach_contexts(notes)
attach_response_counts(notes)
return render_template("index.html", notes=notes)
def attach_response_counts(notes) -> None:
"""Give listed notes their like, repost and reply counts (v2.13.4)"""
counts = {}
if current_app.config.get('WEBMENTION_ENABLED', False) and notes:
try:
from starpunk.webmention_queries import response_counts
counts = response_counts([n.id for n in notes])
except Exception as e:
current_app.logger.warning(f"Could not count responses: {e}")
for note in notes:
object.__setattr__(note, 'response_counts', counts.get(note.id))
def attach_contexts(notes) -> None:
"""Give likes and reposts the fetched context of the post they point at"""
from starpunk.webmention_queries import get_reply_context
@@ -320,6 +334,7 @@ def likes():
object.__setattr__(note, 'media', get_note_media(note.id))
object.__setattr__(note, '_cached_tags', get_note_tags(note.id))
attach_contexts(notes)
attach_response_counts(notes)
return render_template("likes.html", notes=notes)
@@ -467,6 +482,7 @@ def tag(tag: str):
media = get_note_media(note.id)
object.__setattr__(note, 'media', media)
attach_contexts(notes)
attach_response_counts(notes)
return render_template(
"tag.html",
+7
View File
@@ -1464,6 +1464,13 @@ def _fail(db, row, error: SyndicationError) -> None:
f"Syndication {row['target']} for note {row['note_id']} failed "
f"(attempt {attempts}{', giving up' if failed else ''}): {error}"
)
if failed: # ADR-074: tell the owner, once, when we give up
from starpunk import notifications
target = get_targets().get(row["target"])
site = current_app.config["SITE_URL"].rstrip("/")
notifications.problem(current_app.config, current_app.config["DATABASE_PATH"],
f"Crosspost to {target.name if target else row['target']} failed",
str(error)[:300], f"{site}/admin/edit/{row['note_id']}")
def _target_title(url: Optional[str]) -> Optional[str]:
+41
View File
@@ -110,6 +110,47 @@ def get_webmentions_for_note(note_id: int) -> Dict[str, List[Dict[str, Any]]]:
return grouped
def response_counts(note_ids: List[int]) -> Dict[int, Dict[str, int]]:
"""
Likes, reposts and replies per note, for lists of notes (v2.13.4).
Shown responses (valid and approved) plus the counts silos give without
names (Threads; people who opted out of backfeed) on synced crossposts.
Notes with none are left out.
"""
if not note_ids:
return {}
db = get_db()
marks = ','.join('?' * len(note_ids))
counts: Dict[int, Dict[str, int]] = {}
def add(note_id, key, n):
if n:
counts.setdefault(note_id, {'likes': 0, 'reposts': 0, 'replies': 0})[key] += n
for row in db.execute(
f"""
SELECT target_note_id, mention_type, COUNT(*) FROM webmentions
WHERE target_note_id IN ({marks}) AND validation_status = 'valid'
AND moderation_status = 'approved' AND mention_type IN ('like', 'repost', 'reply')
GROUP BY target_note_id, mention_type
""",
note_ids,
):
add(row[0], {'like': 'likes', 'repost': 'reposts', 'reply': 'replies'}[row[1]], row[2])
for row in db.execute(
f"""
SELECT note_id, SUM(COALESCE(likes_count, 0)), SUM(COALESCE(reposts_count, 0))
FROM syndications WHERE note_id IN ({marks}) AND state = 'synced'
GROUP BY note_id
""",
note_ids,
):
add(row[0], 'likes', row[1])
add(row[0], 'reposts', row[2])
return counts
def list_webmentions(
status_filter: Optional[str] = None,
page: int = 1,
+7
View File
@@ -83,7 +83,9 @@ class WebmentionWorker(threading.Thread):
self._last_processed = None
self._lock_file = None
from starpunk.avatars import Upkeep
from starpunk.notifications import Sweeper
self._avatar_upkeep = Upkeep()
self._notifier = Sweeper()
def run(self):
"""
@@ -118,6 +120,11 @@ class WebmentionWorker(threading.Thread):
# Local copies of response avatars (v2.13.0)
self._avatar_upkeep.run(self.db_path, self.app.config['DATA_PATH'],
self.site_url)
# New responses to the owner (v2.14.0, ADR-074)
try:
self._notifier.run(self.app.config, self.db_path, self.site_url)
except Exception as e:
logger.error(f"Notification sweep failed: {e}", exc_info=True)
# Update last processed time
if processed > 0:
+33 -20
View File
@@ -350,31 +350,44 @@ small { display: block; margin-top: var(--spacing-xs); color: var(--color-text-l
}
/* Likes, reposts and bookmarks: counts with a hover list (v2.13.2) */
.reactions { display: flex; flex-wrap: wrap; gap: var(--spacing-md); margin-top: var(--spacing-sm); color: var(--color-text-light); font-size: 0.875rem; }
.reaction { position: relative; display: inline-flex; align-items: center; gap: 0.3rem; padding-bottom: 0.4rem; cursor: default; }
.reaction > svg { width: 1rem; height: 1rem; flex: none; }
.reaction:hover, .reaction:focus-within { color: var(--color-text); }
.reaction-like:hover > svg, .reaction-like:focus-within > svg { color: #e0245e; }
.reaction-repost:hover > svg, .reaction-repost:focus-within > svg { color: #17bf63; }
.reaction-who {
.response-counts { display: flex; flex-wrap: wrap; gap: var(--spacing-md); margin-top: var(--spacing-sm); color: var(--color-text-light); font-size: 0.875rem; }
.response-count { position: relative; display: inline-flex; align-items: center; gap: 0.3rem; padding-bottom: 0.4rem; cursor: default; }
.response-count > svg { width: 1rem; height: 1rem; flex: none; }
.response-count:hover, .response-count:focus-within { color: var(--color-text); }
.response-count-like:hover > svg, .response-count-like:focus-within > svg { color: #e0245e; }
.response-count-repost:hover > svg, .response-count-repost:focus-within > svg { color: #17bf63; }
.response-card {
display: none; position: absolute; left: 0; top: 100%; z-index: 10;
flex-direction: column; gap: 0.4rem; min-width: 13rem; max-width: 20rem;
padding: var(--spacing-sm); background: var(--color-bg-alt); border: 1px solid var(--color-border);
border-radius: var(--border-radius); box-shadow: 0 6px 20px rgba(0, 0, 0, 0.25);
}
.reaction:hover .reaction-who, .reaction:focus-within .reaction-who { display: flex; }
.reaction-title { font-weight: 600; color: var(--color-text); }
.reaction-people { display: flex; flex-direction: column; gap: 0.35rem; max-height: 14rem; overflow-y: auto; }
.reaction-person { display: flex; align-items: center; gap: 0.4rem; min-width: 0; line-height: 1.3; }
.reaction-person .h-card { display: flex; min-width: 0; }
.reaction-person .h-card a { display: inline-flex; align-items: center; gap: 0.4rem; color: var(--color-text); min-width: 0; }
.reaction-person .p-name { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.reaction-person img { width: 20px; height: 20px; border-radius: 50%; object-fit: cover; flex: none; }
.reaction-person .webmention-via { margin-left: auto; padding-left: 0.5rem; white-space: nowrap; }
.reaction-anon { color: var(--color-text-light); }
.reaction-elsewhere { display: block; margin-top: 0.15rem; padding-top: 0.4rem; border-top: 1px solid var(--color-border); color: var(--color-text-light); }
.reaction-elsewhere:first-child { margin-top: 0; padding-top: 0; border-top: 0; }
.reaction-elsewhere a { color: var(--color-link); }
.response-count:hover .response-card, .response-count:focus-within .response-card { display: flex; }
.response-card-title { font-weight: 600; color: var(--color-text); }
.response-people { display: flex; flex-direction: column; gap: 0.35rem; max-height: 14rem; overflow-y: auto; }
.response-person { display: flex; align-items: center; gap: 0.4rem; min-width: 0; line-height: 1.3; }
.response-person .h-card { display: flex; min-width: 0; }
.response-person .h-card a { display: inline-flex; align-items: center; gap: 0.4rem; color: var(--color-text); min-width: 0; }
.response-person .p-name { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.response-person img { width: 20px; height: 20px; border-radius: 50%; object-fit: cover; flex: none; }
.response-person .webmention-via { margin-left: auto; padding-left: 0.5rem; white-space: nowrap; }
.response-anon { color: var(--color-text-light); }
.response-elsewhere { display: block; margin-top: 0.15rem; padding-top: 0.4rem; border-top: 1px solid var(--color-border); color: var(--color-text-light); }
.response-elsewhere:first-child { margin-top: 0; padding-top: 0; border-top: 0; }
.response-elsewhere a { color: var(--color-link); }
/* A reply card opens the reply where it was posted (v2.13.4): its date link
stretches over the card; the author link and links in the text stay on top */
.webmention-reply { position: relative; }
.webmention-reply .webmention-date::after { content: ""; position: absolute; inset: 0; }
.webmention-reply .webmention-author a, .webmention-reply .e-content a { position: relative; z-index: 1; }
.webmention-reply:hover { cursor: pointer; }
/* Counts in lists of notes (v2.13.4) */
.list-counts { display: inline-flex; gap: 0.6rem; color: var(--color-text-light); }
.note-meta a.list-counts:hover { color: var(--color-text); text-decoration: none; }
.list-count { display: inline-flex; align-items: center; gap: 0.2rem; }
.list-count > svg { width: 0.85rem; height: 0.85rem; flex: none; }
/* Generic Mentions */
.webmention-mention {
+16
View File
@@ -12,6 +12,22 @@
</a>
</div>
{# Notifications (v2.14.0, ADR-074) #}
{% if notify %}
<section class="notify-status">
<h3>Notifications</h3>
{% if notify.enabled %}
<p>Sending {{ notify.events|join(', ') }} to {{ notify.services|join(', ') }} at {{ notify.priority }} priority.</p>
{% if notify.last_error %}<p class="flash flash-error">Last send failed: {{ notify.last_error }}</p>{% endif %}
<form method="POST" action="{{ url_for('admin.notifications_test') }}"><input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
<button type="submit" class="button">Send test notification</button>
</form>
{% else %}
<p>Off. Set <code>NOTIFY_URLS</code> to one or more Apprise URLs to be notified of new responses.</p>
{% endif %}
</section>
{% endif %}
{# Backfeed (v2.13.0, ADR-073) #}
{% if backfeed and backfeed.enabled %}
<section class="backfeed-status">
+1
View File
@@ -78,6 +78,7 @@
{% endif %}
</footer>
<div id="responses"></div>
{{ responses.reactions(likes, reposts, replies, bookmarks, syndications, absolute_url_for('public.note', slug=note.slug)) }}
<nav class="note-nav">
+2
View File
@@ -1,6 +1,7 @@
{# One note in a list (home, tag pages). v2.3.0 #}
{% from "partials/media.html" import display_media %}
{% from "partials/reaction.html" import reaction %}
{% from "partials/responses.html" import list_counts %}
{% macro note_preview(note, author) %}
<article class="h-entry note-preview">
{# Notes starting with a '# Heading' have a title (Q22; Note.has_title) #}
@@ -37,6 +38,7 @@
{% for tag in note.tags %}<a class="p-category" rel="tag" href="{{ url_for('public.tag', tag=tag.name) }}">{{ tag.display_name }}</a>{% endfor %}
</span>
{% endif %}
{{ list_counts(note.response_counts if note.response_counts is defined else None, absolute_url_for('public.note', slug=note.slug)) }}
{% if author %}
{# Single-author site: keep the h-card for parsers, hide it from readers #}
<span class="p-author h-card visually-hidden"><a class="p-name u-url" href="{{ author.url or author.me }}">{{ author.name or author.url or author.me }}</a></span>
+21 -9
View File
@@ -73,22 +73,22 @@
{% for name, n in anon %}{% set ns.total = ns.total + n %}{% endfor %}
{% set verb = VERBS.get(kind) %}
{% if ns.total or (verb and copies) %}
<span class="reaction reaction-{{ kind }}" tabindex="0">
{{ ICONS[kind]|safe }}{% if ns.total %}<span class="reaction-count">{{ ns.total }}</span>{% endif %}<span class="visually-hidden"> {{ ns.total }} {{ one if ns.total == 1 else many }}</span>
<span class="reaction-who" role="tooltip">
{% if ns.total %}<span class="reaction-title">{{ ns.total }} {{ one if ns.total == 1 else many }}</span>{% endif %}
<span class="response-count response-count-{{ kind }}" tabindex="0">
{{ ICONS[kind]|safe }}{% if ns.total %}<span class="response-count-n">{{ ns.total }}</span>{% endif %}<span class="visually-hidden"> {{ ns.total }} {{ one if ns.total == 1 else many }}</span>
<span class="response-card" role="tooltip">
{% if ns.total %}<span class="response-card-title">{{ ns.total }} {{ one if ns.total == 1 else many }}</span>{% endif %}
{% if (people and items) or anon %}
<span class="reaction-people">
<span class="response-people">
{% if people %}
{% for wm in items %}
<span class="reaction-person {{ prop }} h-cite">{{ author_card(wm, photo_size=20) }}{% if wm.via %}<span class="webmention-via">{{ wm.via }}</span>{% endif %}<a class="u-url" href="{{ response_url(wm) }}" hidden>{{ response_time(wm) }}</a></span>
<span class="response-person {{ prop }} h-cite">{{ author_card(wm, photo_size=20) }}{% if wm.via %}<span class="webmention-via">{{ wm.via }}</span>{% endif %}<a class="u-url" href="{{ response_url(wm) }}" hidden>{{ response_time(wm) }}</a></span>
{% endfor %}
{% endif %}
{% for name, n in anon %}<span class="reaction-anon">{{ '+' if items and people else '' }}{{ n }} on {{ name }}</span>{% endfor %}
{% for name, n in anon %}<span class="response-anon">{{ '+' if items and people else '' }}{{ n }} on {{ name }}</span>{% endfor %}
</span>
{% endif %}
{% if verb and copies %}
<indie-action class="reaction-elsewhere" do="{{ kind }}" with="{{ permalink }}">{{ verb }} on {% for s in copies %}<a href="{{ s.remote_url }}" rel="noopener">{{ s.name }}</a>{% if not loop.last %} · {% endif %}{% endfor %}</indie-action>
<indie-action class="response-elsewhere" do="{{ kind }}" with="{{ permalink }}">{{ verb }} on {% for s in copies %}<a href="{{ s.remote_url }}" rel="noopener">{{ s.name }}</a>{% if not loop.last %} · {% endif %}{% endfor %}</indie-action>
{% endif %}
</span>
</span>
@@ -103,7 +103,7 @@
{% endfor %}
{% set copies = (syndications or [])|selectattr('remote_url')|list %}
{% if copies or likes or reposts or replies or bookmarks or anon_likes or anon_reposts %}
<div class="reactions">
<div class="response-counts">
{{ reaction('like', likes or [], 'u-like', anon_likes, 'like', 'likes', copies, permalink) }}
{{ reaction('repost', reposts or [], 'u-repost', anon_reposts, 'repost', 'reposts', copies, permalink) }}
{# Replies are listed in full below; here just the count and where to reply #}
@@ -112,3 +112,15 @@
</div>
{% endif %}
{%- endmacro %}
{# Compact like / repost / reply counts for lists of notes (v2.13.4); links
to the note's responses. `counts` is from webmention_queries.response_counts. #}
{% macro list_counts(counts, permalink) %}
{% if counts %}
<a class="list-counts" href="{{ permalink }}#responses">
{%- for key, kind, one, many in [('likes', 'like', 'like', 'likes'), ('reposts', 'repost', 'repost', 'reposts'), ('replies', 'reply', 'reply', 'replies')] if counts[key] -%}
<span class="list-count list-count-{{ kind }}" title="{{ counts[key] }} {{ one if counts[key] == 1 else many }}">{{ ICONS[kind]|safe }}{{ counts[key] }}<span class="visually-hidden"> {{ one if counts[key] == 1 else many }}</span></span>
{%- endfor -%}
</a>
{% endif %}
{%- endmacro %}
+21 -4
View File
@@ -335,9 +335,9 @@ def test_note_page_shows_backfed_responses_quotes_and_counts(app, client, web):
assert "via GoToSocial" in page
assert "1 Quote" in page and "Look at this" in page
# Likes and reposts are a counts line (v2.13.2): named likes plus the silo's anonymous count
assert '<span class="reaction-count">13</span>' in page and "+12 on GoToSocial" in page
assert '<span class="reaction-count">1</span>' in page
assert f'<indie-action class="reaction-elsewhere" do="like" with="{SITE}/note/{note.slug}">' in page
assert '<span class="response-count-n">13</span>' in page and "+12 on GoToSocial" in page
assert '<span class="response-count-n">1</span>' in page
assert f'<indie-action class="response-elsewhere" do="like" with="{SITE}/note/{note.slug}">' in page
assert f'Like on <a href="{COPY}"' in page
assert 'src="/avatars/' in page and 'src="https://cdn.example/a.png"' not in page
@@ -620,5 +620,22 @@ def test_people_who_opt_out_are_counted_not_named(app, client):
assert (s["likes_count"], s["reposts_count"]) == (2, 1)
assert [r["source_url"] for r in rows(app, note.id)] == [like(1).source_url]
page = app.test_client().get(f"/note/{note.slug}").get_data(as_text=True)
assert '<span class="reaction-count">3</span>' in page and "+2 on GoToSocial" in page
assert '<span class="response-count-n">3</span>' in page and "+2 on GoToSocial" in page
assert "Liker 5" not in page and "Quiet" not in page
def test_home_page_shows_counts(app, client):
"""v2.13.4: lists of notes show like, repost and reply counts, linking to the responses"""
note, sid = crossposted(app)
with app.app_context():
quiet = create_note(content="Nobody saw this", published=True)
client.harvest = Harvest([like(1), like(2), reply(3)])
run(app)
with app.app_context():
get_db(app).execute("UPDATE syndications SET likes_count = 1, reposts_count = 2 WHERE id = ?", (sid,))
get_db(app).commit()
page = app.test_client().get("/").get_data(as_text=True)
assert f'href="{SITE}/note/{note.slug}#responses"' in page
assert 'title="3 likes"' in page and 'title="2 reposts"' in page and 'title="1 reply"' in page
assert f"/note/{quiet.slug}#responses" not in page
assert 'id="responses"' in app.test_client().get(f"/note/{note.slug}").get_data(as_text=True)
+251
View File
@@ -0,0 +1,251 @@
"""
Tests for notifications through Apprise (v2.14.0, ADR-074)
Most tests replace the Apprise send with a recorder. One sends for real
through Apprise to a local HTTP server (json://), to check the wiring.
"""
import json
import sqlite3
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer
import pytest
from starpunk import backfeed, create_app, notifications, syndication
from starpunk.backfeed import Harvest, Response
from starpunk.database import get_db
from starpunk.notes import create_note
SITE = "https://blog.example.com"
@pytest.fixture
def sent(monkeypatch):
messages = []
def deliver(targets, title, body):
messages.append({"targets": targets, "title": title, "body": body})
return True, ""
monkeypatch.setattr(notifications, "_deliver", deliver)
return messages
@pytest.fixture
def app(tmp_path):
data = tmp_path / "data"
app = create_app({
"TESTING": True, "SITE_URL": SITE, "ADMIN_ME": "https://example.com",
"SESSION_SECRET": "x", "DATA_PATH": data, "NOTES_PATH": data / "notes",
"DATABASE_PATH": data / "starpunk.db",
"NOTIFY_URLS": "gotifys://gotify.example.com/TOKEN, ntfys://topic?priority=max",
"NOTIFY_EVENTS": "reply,mention,like,problems",
"NOTIFY_PRIORITY": "high",
"CSRF_ENABLED": False,
})
yield app
syndication._client_cache.clear()
backfeed.denied.clear()
def note(app, content="Hello world", age_days=0):
with app.app_context():
n = create_note(content=content, published=True)
if age_days:
get_db(app).execute("UPDATE notes SET created_at = datetime('now', ?) WHERE id = ?",
(f"-{age_days} days", n.id))
get_db(app).commit()
return n
def response(app, n, kind, name, source, moderation="approved", origin="webmention",
snippet=None, hours_ago=0):
with app.app_context():
get_db(app).execute(
"INSERT INTO webmentions (source_url, target_url, target_note_id, mention_type, author_name, "
"validation_status, moderation_status, origin, content_snippet, received_at) "
"VALUES (?, ?, ?, ?, ?, 'valid', ?, ?, ?, datetime('now', ?))",
(source, f"{SITE}/note/{n.slug}", n.id, kind, name, moderation, origin, snippet,
f"-{hours_ago} hours"))
get_db(app).commit()
def sweep(app, sweeper=None):
sweeper = sweeper or notifications.Sweeper()
with app.app_context():
return sweeper.run(app.config, app.config["DATABASE_PATH"], SITE), sweeper
def unsent(app):
with app.app_context():
return [r[0] for r in get_db(app).execute(
"SELECT source_url FROM webmentions WHERE notified_at IS NULL ORDER BY id")]
# --- Settings --------------------------------------------------------------------------
def test_settings():
config = {"NOTIFY_URLS": " a://x,b://y c://z ", "NOTIFY_EVENTS": "Reply, likes, problems"}
assert notifications.urls(config) == ["a://x", "b://y", "c://z"]
assert notifications.events(config) == {"reply", "problems"}, "unknown names are ignored"
assert notifications.events({}) == {"reply", "mention", "quote", "problems"}
assert notifications.with_priority("gotifys://h/T", "high") == "gotifys://h/T?priority=high"
assert notifications.with_priority("ntfys://t?tags=x", "5") == "ntfys://t?tags=x&priority=5"
assert notifications.with_priority("ntfys://t?priority=max", "low") == "ntfys://t?priority=max"
# --- Sweep -------------------------------------------------------------------------------------
def test_new_responses_are_sent_once_and_likes_grouped(app, sent):
n = note(app, "Starpunk now supports backfeeds")
response(app, n, "reply", "Ana", "https://a.example/r", origin="bluesky",
snippet="<p>Great &amp; useful!</p>")
response(app, n, "like", "Bob", "https://b.example/l")
response(app, n, "like", "Cy", "https://c.example/l")
response(app, n, "repost", "Dee", "https://d.example/rp") # not selected
response(app, n, "mention", "Eve", "https://e.example/m", moderation="pending")
count, _ = sweep(app)
assert count == 4
titles = [m["title"] for m in sent]
assert "↩ Reply from Ana via Bluesky" in titles
assert "💬 Mention from Eve (awaiting moderation)" in titles
assert "♥ 2 new likes on “Starpunk now supports backfeeds”" in titles
reply = next(m for m in sent if m["title"].startswith("↩"))
assert reply["body"].startswith("Great & useful!")
assert f"({SITE}/note/{n.slug}#responses)" in reply["body"]
mention = next(m for m in sent if m["title"].startswith("💬"))
assert f"{SITE}/admin/webmentions/" in mention["body"], "pending ones link to moderation"
assert all(t.endswith("priority=high") or t.endswith("priority=max") for t in sent[0]["targets"])
assert unsent(app) == []
sent.clear()
assert sweep(app)[0] == 0 and sent == [], "nothing is sent twice"
def test_failed_send_is_retried_with_backoff(app, monkeypatch):
calls = []
monkeypatch.setattr(notifications, "_deliver", lambda t, ti, b: (calls.append(ti), (False, "boom"))[1])
n = note(app)
response(app, n, "reply", "Ana", "https://a.example/r")
count, sweeper = sweep(app)
assert count == 0 and unsent(app) == ["https://a.example/r"]
assert sweep(app, sweeper)[0] == 0 and len(calls) == 1, "waits before retrying"
with app.app_context():
assert "boom" in notifications.last_error(get_db(app))
sweeper._next_try = 0
monkeypatch.setattr(notifications, "_deliver", lambda t, ti, b: (True, ""))
assert sweep(app, sweeper)[0] == 1
with app.app_context():
assert notifications.last_error(get_db(app)) == ""
def test_old_unsent_responses_are_dropped(app, sent):
n = note(app)
response(app, n, "reply", "Ana", "https://a.example/r", hours_ago=30)
assert sweep(app)[0] == 0 and sent == [] and unsent(app) == []
def test_off_without_urls(app, sent):
app.config["NOTIFY_URLS"] = ""
n = note(app)
response(app, n, "reply", "Ana", "https://a.example/r")
assert sweep(app)[0] == 0 and unsent(app) == ["https://a.example/r"]
def test_backfill_and_late_first_checks_are_quiet(app, sent):
class Client:
config_token = "t"
def responses(self, remote_id, remote_url, cap=1000):
return Harvest([Response(kind="reply", source_url=f"{remote_url}/reply", remote_id="r",
author_name="Ana", content_html="<p>hi</p>")])
def maintain(self):
pass
app.config.update(GOTOSOCIAL_URL="https://social.example", GOTOSOCIAL_ACCESS_TOKEN="t",
BACKFEED_ENABLED=True)
syndication._client_cache["https://social.example"] = Client()
fresh, old = note(app, "New"), note(app, "Old", age_days=10)
with app.app_context():
for n, url in ((fresh, "https://social.example/1"), (old, "https://social.example/2")):
get_db(app).execute("INSERT INTO syndications (note_id, target, enabled, remote_id, remote_url, "
"state) VALUES (?, 'gotosocial', 1, 'x', ?, 'synced')", (n.id, url))
get_db(app).commit()
backfeed.process_due()
assert unsent(app) == ["https://social.example/1/reply"], "only the new note's reply is news"
# --- Problems and admin ----------------------------------------------------------------------------
def test_problems(app, sent):
with app.app_context():
notifications.problem(app.config, app.config["DATABASE_PATH"], "Crosspost to Bluesky failed",
"HTTP 400", f"{SITE}/admin/edit/1")
assert sent[0]["title"] == "⚠ Crosspost to Bluesky failed"
assert "HTTP 400" in sent[0]["body"] and "/admin/edit/1" in sent[0]["body"]
app.config["NOTIFY_EVENTS"] = "reply"
with app.app_context():
notifications.problem(app.config, app.config["DATABASE_PATH"], "x", "y", "z")
assert len(sent) == 1, "not selected: not sent"
def test_final_crosspost_failure_notifies(app, sent):
n = note(app)
app.config.update(GOTOSOCIAL_URL="https://social.example", GOTOSOCIAL_ACCESS_TOKEN="t")
with app.app_context():
cur = get_db(app).execute("INSERT INTO syndications (note_id, target, enabled, state, attempts) "
"VALUES (?, 'gotosocial', 1, 'pending', 0)", (n.id,))
get_db(app).commit()
row = get_db(app).execute("SELECT * FROM syndications WHERE id = ?", (cur.lastrowid,)).fetchone()
syndication._fail(get_db(app), row, syndication.SyndicationError("temporary", permanent=False))
assert sent == [], "a retry is coming: no message"
syndication._fail(get_db(app), row, syndication.SyndicationError("HTTP 422 bad", permanent=True))
assert sent and sent[0]["title"] == "⚠ Crosspost to GoToSocial failed"
def test_admin_status_and_test_button(app, sent):
from starpunk.auth import create_session
with app.test_request_context():
token = create_session("https://example.com")
browser = app.test_client()
browser.set_cookie("starpunk_session", token)
page = browser.get("/admin/webmentions").get_data(as_text=True)
assert "Sending like, mention, problems, reply to gotifys, ntfys at high priority" in page
page = browser.post("/admin/notifications/test", follow_redirects=True).get_data(as_text=True)
assert "Test notification sent" in page and sent[-1]["title"] == "StarPunk test notification"
# --- Real Apprise -------------------------------------------------------------------------------------
def test_apprise_delivers_markdown_to_a_service(app):
received = []
class Handler(BaseHTTPRequestHandler):
def do_POST(self):
received.append(json.loads(self.rfile.read(int(self.headers["Content-Length"]))))
self.send_response(200)
self.end_headers()
def log_message(self, *args):
pass
server = HTTPServer(("127.0.0.1", 0), Handler)
threading.Thread(target=server.handle_request, daemon=True).start()
app.config["NOTIFY_URLS"] = f"json://127.0.0.1:{server.server_port}/"
ok, error = notifications.send(app.config, "Title", "Body with [a link](https://x.example)")
server.server_close()
assert ok, error
assert received[0]["title"] == "Title" and "a link" in received[0]["message"]
app.config["NOTIFY_URLS"] = "notaservice://nope"
ok, error = notifications.send(app.config, "t", "b")
assert not ok and "Not a usable Apprise URL" in error
def test_migration_021_marks_history(tmp_path):
from pathlib import Path
conn = sqlite3.connect(tmp_path / "db")
conn.execute("CREATE TABLE webmentions (id INTEGER PRIMARY KEY, source_url TEXT)")
conn.execute("INSERT INTO webmentions (source_url) VALUES ('https://old.example')")
conn.executescript((Path(__file__).parent.parent / "migrations" / "021_notifications.sql").read_text())
assert conn.execute("SELECT notified_at IS NOT NULL FROM webmentions").fetchone()[0] == 1
+89
View File
@@ -0,0 +1,89 @@
"""
v2.14.1 fixes: clearing all tags in the editor, and media captions escaped
in the HTML of RSS, Atom and JSON Feed items.
"""
import io
import json
from xml.etree import ElementTree as ET
import pytest
from PIL import Image
from starpunk import create_app
from starpunk.media import attach_media_to_note, save_media
from starpunk.notes import create_note
from starpunk.tags import get_note_tags
CAPTION = 'Say "hi" <b>bold</b> & more'
ESCAPED = 'alt="Say &quot;hi&quot; &lt;b&gt;bold&lt;/b&gt; &amp; more"'
@pytest.fixture
def app(tmp_path):
data = tmp_path / "data"
app = create_app({
"TESTING": True, "DATABASE_PATH": data / "starpunk.db", "DATA_PATH": data,
"NOTES_PATH": data / "notes", "SESSION_SECRET": "x", "ADMIN_ME": "https://example.com",
"SITE_URL": "https://blog.example.com", "SITE_NAME": "Test", "CSRF_ENABLED": False,
})
yield app
def png():
buf = io.BytesIO()
Image.new("RGB", (800, 600), "red").save(buf, "PNG")
return buf.getvalue()
@pytest.fixture
def captioned(app):
with app.app_context():
note = create_note(content="A photo", published=True)
media = save_media(png(), "photo.png")
attach_media_to_note(note.id, [media["id"]], [CAPTION])
return note
def test_rss_caption_escaped(app, captioned):
body = app.test_client().get("/feed.rss").get_data(as_text=True)
description = ET.fromstring(body).find("./channel/item/description").text
assert ESCAPED in description
assert "<b>bold</b>" not in description
def test_atom_caption_escaped_once(app, captioned):
body = app.test_client().get("/feed.atom").get_data(as_text=True)
ns = {"a": "http://www.w3.org/2005/Atom"}
entry = ET.fromstring(body).find("a:entry", ns)
content = entry.find("a:content", ns).text
assert ESCAPED in content and "<b>bold</b>" not in content
# The enclosure's title attribute was already escaped; not twice
enclosure = next(l for l in entry.findall("a:link", ns) if l.get("rel") == "enclosure")
assert enclosure.get("title") == CAPTION
def test_json_feed_caption_escaped(app, captioned):
item = json.loads(app.test_client().get("/feed.json").get_data(as_text=True))["items"][0]
assert ESCAPED in item["content_html"] and "<b>bold</b>" not in item["content_html"]
def test_editor_can_clear_all_tags(app):
from starpunk.auth import create_session
with app.app_context():
note = create_note(content="Tagged", published=True, tags=["one", "two"])
with app.test_request_context():
token = create_session("https://example.com")
client = app.test_client()
client.set_cookie("starpunk_session", token)
# A form without the Tags field leaves the tags alone
client.post(f"/admin/edit/{note.id}", data={"content": "Tagged", "published": "on"})
with app.app_context():
assert {t["name"] for t in get_note_tags(note.id)} == {"one", "two"}
# An empty Tags field removes them all
client.post(f"/admin/edit/{note.id}", data={"content": "Tagged", "published": "on", "tags": ""})
with app.app_context():
assert get_note_tags(note.id) == []