Compare commits

..
14 Commits
Author SHA1 Message Date
PhilandClaude Opus 5.5 0dd59fbc18 Merge branch 'fix/v2.6.1-threads-publish-retry' for v2.6.1 release
Build Container / build (push) Successful in 6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 21:18:11 -06:00
PhilandClaude Opus 5.5 11e09a0d02 fix: Retry Threads publish errors, show full error detail (v2.6.1)
Meta's threads_publish returned a one-off HTTP 400 "Fatal" for a plain
text post; the same post published fine on retry. Publish-step errors
are now retryable unless they are auth/permission errors (codes 10, 190,
200); the retry's existing-post check prevents doubles. Errors carry the
Graph code, subcode, user message and fbtrace id. A short pause before
the first container status poll.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 21:18:11 -06:00
PhilandClaude Opus 5.5 565adaeecf Merge branch 'feature/v2.6.0-threads' for v2.6.0 release
Build Container / build (push) Successful in 7s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:54:15 -06:00
PhilandClaude Opus 5.5 53d10ed271 feat: Crosspost notes to Threads (v2.6.0)
Third crossposting target on the ADR-065 loop (ADR-069). Text posts get a
link preview, 1 image posts as IMAGE and 2-4 as a CAROUSEL (Meta fetches
them from /media/), with alt text and the first tag as topic_tag. Posts are
shortened to 500 characters, counting non-ASCII as UTF-8 bytes. Edits don't
follow (the API has no edit); unpublish and delete remove the post. Retried
creates check for an existing post first.

The 60-day token is refreshed daily from the worker loop and kept in
DATA_PATH/threads_token.json under a file lock; a new configured token
replaces it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:46:56 -06:00
PhilandClaude Opus 5.5 e8c1b03980 feat: Likes and reposts (v2.5.0)
Build Container / build (push) Successful in 6s
Micropub like-of/repost-of with optional comment; likes on /likes, off the
home page and feeds; reposts in the stream. Native favourites/boosts on
GoToSocial and likes/reposts/quotes on Bluesky when the target is on that
network, link posts for other reposts. Fix double slash in outgoing
webmention source URLs. ADR-068.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 20:16:21 -06:00
PhilandClaude Opus 5.5 0d3a5cd519 feat: Micropub update, delete/undelete, drafts, replies (v2.4.0)
Build Container / build (push) Successful in 6s
Update (replace/add/delete) for content, name, category, photo,
in-reply-to, post-status; soft delete with undelete; post-status drafts;
in-reply-to; multipart and remote photos; q=source lists and properties[];
q=category; media scope. Fix media endpoint 404 at the advertised
/micropub/media path. ADR-067.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 19:53:18 -06:00
Phil 88088e10e1 Merge branch 'fix/v2.3.3-micropub-title-source-url' for v2.3.3 release
Build Container / build (push) Successful in 7s
2026-10-31 19:34:24 -06:00
PhilandClaude Opus 5.5 391bdb9db3 fix: Micropub titles, q=source URL, single title display (v2.3.3)
Micropub `name` was dropped; store it as the note's '# Heading' first
line, which is how StarPunk titles work. q=source returned /notes/<slug>
(a 404) and a name for every note; it now returns the real permalink and
name only for titled notes, without the heading in content. Titled notes
no longer show their title twice (Note.body_html, excerpt).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 19:34:23 -06:00
PhilandClaude Opus 5.5 c51cf726b0 fix: Header shows Admin when logged in (v2.3.2)
Build Container / build (push) Successful in 9s
Public pages never checked the session, so they always showed Log in.
Set g.me in before_request when a session cookie is present.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:57:12 -06:00
PhilandClaude Opus 5.5 2616ef5060 fix: Put Log in back in the header nav (v2.3.1)
Build Container / build (push) Successful in 7s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:46:33 -06:00
Phil 0ae16bd1fc Merge branch 'feature/v2.3.0-style-pass' for v2.3.0 release
Build Container / build (push) Successful in 8s
2026-10-31 18:43:20 -06:00
PhilandClaude Opus 5.5 cc09098741 style: Public page pass (v2.3.0)
Fix note metadata inheriting page-footer styles, show full notes instead of
HTML cut at 300 chars, auto-link bare URLs, one-line meta, SITE_NAME in the
header and titles, serif reading font, dark mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 18:24:29 -06:00
Phil 16925663a3 Merge branch 'feature/v2.2.0-bluesky' for v2.2.0 release
Build Container / build (push) Successful in 7s
2026-10-31 14:19:17 -06:00
PhilandClaude Opus 5.5 e92c27a84c feat: Crosspost notes to Bluesky (v2.2.0)
BlueskyClient behind the ADR-065 reconcile loop: plain text with link and
hashtag facets, images under 1 MB with alt text, link cards, app-password
sessions with refresh. Deletes follow; edits don't (ADR-066).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 14:19:17 -06:00
40 changed files with 3369 additions and 477 deletions

No files matched your search

+8
View File
@@ -63,6 +63,14 @@ GOTOSOCIAL_NAME=GoToSocial
# public, unlisted, private (default: public)
GOTOSOCIAL_VISIBILITY=public
# Threads account (v2.6.0). A long-lived user token with threads_basic,
# threads_content_publish and threads_delete; StarPunk refreshes it daily and
# keeps the current one in DATA_PATH/threads_token.json.
# How to get one: docs/design/v2.6.0/2026-10-31-threads-crossposting.md
THREADS_ACCESS_TOKEN=
# Your Threads username (for the Micropub syndicate-to uid)
THREADS_USERNAME=
# =============================================================================
# DATA STORAGE
# =============================================================================
+104
View File
@@ -7,6 +7,110 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [2.6.1] - 2026-11-01
### Fixed
- Threads crossposts gave up after Meta's publish step returned a one-off `HTTP 400 Fatal`; publish errors are now retried (a retry checks for an existing post first, so nothing is posted twice). Only a bad token or missing permission stops retries
- Threads errors now include Meta's error code, subcode, user message and trace id, so the editor shows why a post failed
- Wait a moment before the first readiness check on a Threads post, since Meta can report it ready too early
## [2.6.0] - 2026-11-01
### Added
- **Crossposting to Threads** (ADR-069): set `THREADS_ACCESS_TOKEN` (and optionally `THREADS_USERNAME`) to add a Threads target. Text posts get a link preview; 1 image posts as an image, 2–4 as a carousel, with alt text; the note's first tag becomes the post's topic. Shortened to 500 characters. Edits don't follow (the API can't edit); unpublish and delete remove the post. Likes aren't crossposted to Threads; reposts post "🔁 title url" (with the comment, if any)
- The Threads token (60-day lifetime) is refreshed automatically once a day and kept in `data/threads_token.json`, so it survives restarts; setting a new token in the config replaces it. Setup steps: `docs/design/v2.6.0/`
## [2.5.0] - 2026-11-01
### Added
- **Likes and reposts** via Micropub `like-of` / `repost-of` (ADR-068); a comment is optional
- Likes are kept off the home page and feeds and listed on a new `/likes` page (linked in the header)
- Reposts appear in the stream and feeds with a "🔁 Reposted" line
- The liked/reposted post's title and author are fetched and shown; it is sent a webmention
- GoToSocial: likes/reposts of fediverse posts become native favourites/boosts; reposts of other pages (or with a comment) post "🔁 title url"
- Bluesky: likes/reposts of Bluesky posts become native likes/reposts; a repost with a comment is a quote post; reposts of other pages post with a link card
- Likes of ordinary web pages are not crossposted; deleting a like/repost undoes it everywhere
- Migration 013: `notes.like_of`, `notes.repost_of`
### Fixed
- Outgoing webmentions named posts `https://site//note/<slug>` (double slash)
### Notes
- The GoToSocial token needs the extra scopes `read:search` and `write:favourites`
## [2.4.0] - 2026-10-31
### Added
- **Micropub update** (`action=update`, JSON): replace/add/delete for `content`, `name`, `category`, `photo`, `in-reply-to`, `post-status`; edits follow to crossposts (ADR-067)
- **Micropub delete and undelete**: soft delete, restorable; crossposts are removed and re-posted
- **Drafts** via `post-status: draft`; **replies** via `in-reply-to`
- Photos uploaded in a multipart create request; photos hosted elsewhere are downloaded (public addresses only, 20 MB cap)
- `q=source` without `url` lists recent posts (drafts included, `limit`/`offset`); `properties[]` filter; returns photos, syndication links, reply and post status
- `q=category` (tag suggestions, optional `filter`); `q=config` lists supported queries and post types
- Media endpoint accepts the `media` scope
### Fixed
- Media endpoint was advertised at `/micropub/media` but served at `/media` (404 for clients); now both work
- Unsupported Micropub types (e.g. `h=event`) were silently stored as notes; now rejected
## [2.3.3] - 2026-10-31
### Fixed
- Micropub `name` was ignored, so titles sent by Micropub clients never appeared. It is now stored as the note's `# Heading` first line, which is how StarPunk titles work
- Micropub `q=source` returned `/notes/<slug>` links (a 404); it now returns the real permalink `/note/<slug>`
- Micropub `q=source` reported every note's first line as its `name`; it now returns `name` only for titled notes, with the heading left out of `content`, matching what was posted
- Titled notes showed their title twice: as the page heading and again at the top of the body (and of the home-page summary)
## [2.3.2] - 2026-10-31
### Changed
- Header link reads "Admin" (was "Dashboard") when logged in
### Fixed
- Public pages never knew you were logged in (only `/admin` routes checked the session), so the header always said "Log in"
## [2.3.1] - 2026-10-31
### Fixed
- "Log in" is back in the header nav (2.3.0 had moved it to the footer, where it was easy to miss)
## [2.3.0] - 2026-10-31
### Changed
- **Public pages restyled**: compact header with the site name from `SITE_NAME`, serif reading font for notes, one-line metadata (date · #tags · Also on …), dark mode via `prefers-color-scheme`, footer pinned to the bottom, "Log in" moved to the footer
- Home and tag pages show short notes in full; titled notes show a summary and "Read more"
- Author h-card kept for parsers but visually hidden on this single-author site
- Page titles use the note text and `SITE_NAME` instead of the slug and "StarPunk"
### Fixed
- Every note's metadata was styled as the page footer (centered, ruled, padded)
- Previews cut the note's HTML at 300 characters, which could break markup
- Bare URLs in notes are now links
## [2.2.0] - 2026-10-31
### Added
- **Crossposting to Bluesky**, ADR-066
- Plain-text note (Markdown stripped) with clickable link and hashtag facets, shortened to 300 characters with the permalink kept
- Images (up to 4, shrunk under Bluesky's 1 MB limit) with alt text; notes without images get a link card
- Unpublishing, unticking and deleting remove the Bluesky post; edits don't follow (Bluesky has no edits)
- Logs in with an app password: `BLUESKY_HANDLE`, `BLUESKY_APP_PASSWORD` (optional `BLUESKY_SERVICE`, default `https://bsky.social`)
- Shown in the editor and Micropub `syndicate-to` next to GoToSocial
## [2.1.0] - 2026-10-31
### Added
@@ -0,0 +1,37 @@
# ADR-066: Crossposting to Bluesky
## Status
Accepted (v2.2.0, 2026-10-31)
## Context
ADR-065 added direct crossposting to GoToSocial. Bluesky was still fed by
Echofeed, with the same problems: no per-post control, no record of the
copy, deletes never follow.
## Decision
1. **Second client, same loop.** `BlueskyClient` sits behind the reconcile
loop and `syndications` table from ADR-065, as target id `bluesky`.
Clients share a small interface: `create`, `update`, `delete`, `can_edit`.
2. **Auth.** An app password (`BLUESKY_HANDLE`, `BLUESKY_APP_PASSWORD`).
`createSession` against `BLUESKY_SERVICE` (default `https://bsky.social`);
writes go to the PDS named in the returned DID document. Expired access
tokens are refreshed, falling back to a new session.
3. **Content.** Plain text (Markdown rendered and stripped), then the
permalink and hashtags (at most a third of the space), shortened to 300
characters at a word boundary. Link and hashtag facets use UTF-8 byte
offsets. Images (up to 4) are re-encoded as JPEG if over 1 MB, with alt
text and aspect ratio; a note without images gets a link card.
4. **Edits don't follow.** Bluesky has no edit; rewriting the record would
break likes and replies, and delete-and-repost would lose them. The post
stays as first sent. Unpublish, untick and delete remove it.
5. **No double posts.** When a create is retried, recent posts are checked
for one already linking to the note (a send whose reply was lost).
## Consequences
- Echofeed is no longer needed; it must stop posting to Bluesky or posts
appear twice.
- A post deleted by hand on Bluesky is not re-created.
@@ -0,0 +1,34 @@
# ADR-067: Micropub update, delete, drafts and replies
## Status
Accepted (v2.4.0, 2026-10-31)
## Context
The Micropub endpoint could only create published notes. Apps could not edit
or delete posts, drafts and replies were dropped, and photos sent with the
post or hosted elsewhere were lost.
## Decision
1. **Update** (JSON only, `update` scope). Supported properties are the ones a
note can store: `content`, `name` (the `# Heading` first line),
`category`, `photo`, `in-reply-to`, `post-status`. Anything else is a 400,
not silently ignored. Order: delete, then replace, then add (`add` only for
`category` and `photo`). Success is 204.
2. **Delete is soft** (`delete` scope) so `undelete` can restore the note and
its file from `.trash/`. Crossposts are removed on delete and posted again
on undelete.
3. **Drafts**: `post-status: draft` creates an unpublished note; crossposts
wait until it is published.
4. **Photos**: multipart uploads are saved like media-endpoint uploads; remote
URLs are downloaded only from public IP addresses, up to 20 MB, max 4.
5. **Queries**: `q=source` returns everything create accepts (round-trip);
without `url` it lists recent posts, drafts included, for editing apps.
`q=category` lists tags by use.
## Consequences
- `published` (backdating) can be set on create but not changed by update.
- Likes, reposts and bookmarks remain unsupported (round 2).
@@ -0,0 +1,32 @@
# ADR-068: Likes and reposts
## Status
Accepted (v2.5.0, 2026-11-01). Bookmarks were considered and declined.
## Decision
1. **Storage.** `notes.like_of` / `notes.repost_of` (migration 013). The note's
content is an optional comment and may be empty.
2. **Display.** Likes stay off the home page and feeds and are listed on
`/likes`. Reposts appear in the stream and feeds. Both show the target's
fetched title and author (the reply-context cache) and carry
`u-like-of` / `u-repost-of`. The target is sent a webmention.
3. **Crossposting, per target.**
| | Target is a post on that network | Anything else |
|---|---|---|
| Like | native favourite / like | nothing |
| Repost, no comment | native boost / repost | "🔁 title url" post |
| Repost, with comment | Bluesky quote post; GoToSocial status | comment + "🔁 title url" post |
GoToSocial finds fediverse posts with `/api/v2/search?resolve=true`
(needs `read:search`; favourites need `write:favourites`). Bluesky
recognises `bsky.app/profile/<actor>/post/<rkey>` URLs. Native actions
are stored as `favourite:<id>` / `reblog:<id>` or the like/repost record
URI, and deleting the note undoes them.
## Consequences
- Editing a like/repost doesn't change a native favourite/boost.
- A repost with a comment can't be a GoToSocial boost (no quote posts there).
@@ -0,0 +1,54 @@
# ADR-069: Crossposting to Threads
## Status
Accepted (v2.6.0, 2026-10-31)
## Context
ADR-065 and ADR-066 crosspost notes to GoToSocial and Bluesky. Threads
(Meta) has a publishing API, so it can be a third target on the same loop.
## Decision
1. **Third client, same loop.** `ThreadsClient` is target id `threads`
behind the `syndications` table and reconcile loop from ADR-065.
2. **Auth.** A long-lived Threads user token (`THREADS_ACCESS_TOKEN`) with
`threads_basic`, `threads_content_publish` and `threads_delete`. These
tokens expire after 60 days, so StarPunk refreshes the token about once a
day (`th_refresh_token`) from the worker loop, even when nothing is being
posted. The current token is kept in `DATA_PATH/threads_token.json`
(mode 0600) with a hash of the configured token. A different configured
token replaces the saved one. A file lock stops gunicorn workers from
refreshing at the same time. The file avoids a migration for one value.
3. **Posting.** Create a container, poll its status until `FINISHED`, then
`threads_publish`. Text-only notes are `TEXT` posts with the permalink as
`link_attachment`. One image is an `IMAGE` post and 2–4 images are a
`CAROUSEL`. Meta fetches images from their public `/media/` URL. Only
JPEG and PNG up to 8 MB are allowed, so GIF and WebP images are left off.
Alt text goes in `alt_text`.
4. **Content.** Plain text (as for Bluesky), then the permalink, shortened
to 500 characters. Non-ASCII characters are counted as UTF-8 bytes,
because Threads counts emoji that way. Threads allows one tag per post,
so the note's first tag is sent as `topic_tag` and no hashtags are added
to the text.
5. **Edits don't follow.** The API has no way to edit a post. As with
Bluesky, the post stays as first sent, and unpublishing, unticking the
target or deleting the note removes it.
6. **Likes and reposts (ADR-068).** The API can't like or repost outside
content, so likes are not crossposted. Reposts are text posts with
"🔁 title" and the URL, plus the comment if there is one, and the
reposted page as the link preview.
7. **No double posts.** A retried create first looks for a recent post that
already contains the permalink.
## Consequences
- Meta limits accounts to 250 posts and 100 deletes per day. Rate-limit
errors (Graph codes 4, 17, 32, 613) are retried.
- If StarPunk is down for more than 60 days, the token expires and a new
one is needed.
- Getting the first token requires a Meta developer app with the Threads
use case. See the setup notes in `docs/design/v2.6.0/`.
- Notes with more than five links fail on Threads (a Meta limit). The
error is shown in the editor.
@@ -0,0 +1,23 @@
# v2.3.3: Micropub titles, q=source URL, single title display
Found while building tuipub, a terminal Micropub client, against StarPunk.
## Problems
1. **Micropub `name` dropped.** `handle_create` called `extract_title()` and threw the result away. StarPunk has no title column: `Note.title` is derived from the content's first line, and a `# Heading` first line marks a titled note (Q22). So a `name` sent by a client never showed up.
2. **`q=source` URL wrong.** It built `{SITE_URL}notes/{slug}`, but the route is `/note/<slug>`, so the URL was a 404.
3. **`q=source` name wrong.** It returned `note.title` for every note. For untitled notes that is just the first line of the content, so clients saw a `name` they never sent.
4. **Title shown twice.** `note.html` renders the title as `<h1 class="p-name">` and then the whole content, which includes the `# Heading` again. Home-page previews did the same with the excerpt.
## Decisions
- Store `name` as a `# {name}` first line (`micropub.add_title_heading`) rather than adding a title column. This matches how titled notes already work everywhere (pages, feeds, search, admin editor), and needs no migration. If the content already starts with that exact heading, it isn't added twice.
- `models.split_title()` is the single definition of "title heading": the first non-blank line starting with `#` (the same rule as the templates' `has_explicit_title`). `q=source` uses it to return `name` + `content` the way they were posted.
- `Note.body_html` renders the content without the title heading; `note.html` uses it. `Note.excerpt` also leaves the title out. `Note.html` is unchanged, so feeds and syndication behave as before (feed items carry a separate title anyway).
- `extract_title()` no longer derives a fallback title from content: that value was never used.
## Tests
- `tests/test_micropub.py`: name stored as heading, no duplicate heading, `q=source` url equals the Location permalink, `name` only for titled notes, round trip.
- `tests/test_models.py`: `split_title`, `body_html`, excerpt without title.
- Checked end to end with tuipub against a local dev server: titled post → page shows the title once, `q=source` returns `name`, `content` and a working url.
@@ -0,0 +1,62 @@
# v2.6.0: Crossposting to Threads
Implements ADR-069.
## What changed
- `starpunk/syndication.py`: `ThreadsClient` (target `threads`); media
entries now carry their public `url`; `process_due()` calls each client's
optional `maintain()` hook, which Threads uses to refresh its token.
- `starpunk/config.py`: `THREADS_ACCESS_TOKEN`, `THREADS_USERNAME`,
`THREADS_NAME`, `THREADS_MAX_CHARS` (500), `THREADS_MAX_MEDIA` (4).
- `tests/test_syndication_threads.py`: a fake Graph API covering text,
image and carousel posts; container polling; edits not following; deletes
(including posts already deleted by hand); a lost publish reply;
permanent and rate-limit errors; and token refresh, persistence and
replacement.
## Configuration
| Variable | Meaning |
|---|---|
| `THREADS_ACCESS_TOKEN` | Long-lived Threads user token. Enables the target. |
| `THREADS_USERNAME` | Optional. Used for the Micropub `syndicate-to` uid (`https://www.threads.com/@name`) and for fallback links. |
| `THREADS_NAME` | Label in the editor (default `Threads`) |
After the first daily refresh, the working token lives in
`DATA_PATH/threads_token.json`. The configured value only seeds it.
Changing `THREADS_ACCESS_TOKEN` switches to the new token.
## Getting a token (one time)
1. At <https://developers.facebook.com/apps>, create an app with the
**Access the Threads API** use case.
2. Under the use case's permissions, add `threads_basic`,
`threads_content_publish` and `threads_delete`.
3. Under **App roles → Roles**, add your Threads account as a Threads
tester. Accept the invite in Threads (Settings → Account → Website
permissions → Invites). The app can stay in development mode.
4. In the use case settings, add `https://<site>/` as a redirect URI. Then
open:
https://threads.net/oauth/authorize?client_id=APP_ID&redirect_uri=https://<site>/&scope=threads_basic,threads_content_publish,threads_delete&response_type=code
After you approve, the browser lands on `https://<site>/?code=...`. Copy
the code (drop a trailing `#_`).
5. Exchange the code for a short-lived token, then exchange that for a
long-lived one:
curl -s -X POST https://graph.threads.net/oauth/access_token \
-F client_id=APP_ID -F client_secret=APP_SECRET \
-F grant_type=authorization_code -F redirect_uri=https://<site>/ -F code=CODE
curl -s "https://graph.threads.net/access_token?grant_type=th_exchange_token&client_secret=APP_SECRET&access_token=SHORT_TOKEN"
6. Set `THREADS_ACCESS_TOKEN` to the long-lived `access_token`.
The app secret is only needed for these steps. StarPunk doesn't use it.
## Not done
- Video, GIF and WebP images (Threads takes video by URL. StarPunk only
stores still images, and Threads rejects GIF and WebP).
- Edits (no API for them).
+8
View File
@@ -0,0 +1,8 @@
-- Migration 013: Likes and reposts (v2.5.0, ADR-068)
-- A note with like_of is a like; with repost_of, a repost. Content is the
-- optional comment and may be empty.
ALTER TABLE notes ADD COLUMN like_of TEXT;
ALTER TABLE notes ADD COLUMN repost_of TEXT;
CREATE INDEX IF NOT EXISTS idx_notes_like_of ON notes(like_of);
+19 -5
View File
@@ -280,6 +280,21 @@ def create_app(config=None):
"""Add correlation ID to request context for tracing"""
add_correlation_id()
# Know who is logged in on every page, not just /admin (v2.3.2), so
# the header can say "Admin" instead of "Log in". Only requests that
# carry a session cookie touch the database.
from flask import g, request
token = request.cookies.get("starpunk_session")
if token:
from starpunk.auth import verify_session
try:
info = verify_session(token)
except Exception:
info = None
if info:
g.user = info
g.me = info["me"]
# Response middleware - Add webmention Link header (v1.6.0)
@app.after_request
def add_webmention_headers(response):
@@ -337,9 +352,8 @@ def create_app(config=None):
app.webmention_worker.stop()
# Start syndication worker when a target is configured (v2.1.0)
if (app.config.get("GOTOSOCIAL_URL") and app.config.get("GOTOSOCIAL_ACCESS_TOKEN")
and not app.config.get("TESTING", False)):
from starpunk.syndication import SyndicationWorker
from starpunk.syndication import SyndicationWorker, targets_configured
if targets_configured(app.config) and not app.config.get("TESTING", False):
app.syndication_worker = SyndicationWorker(
app, poll_interval=app.config["SYNDICATION_POLL_INTERVAL"]
)
@@ -450,5 +464,5 @@ def create_app(config=None):
# Package version (Semantic Versioning 2.0.0)
# See docs/standards/versioning-strategy.md for details
__version__ = "2.1.0"
__version_info__ = (2, 1, 0)
__version__ = "2.6.1"
__version_info__ = (2, 6, 1)
+11
View File
@@ -132,6 +132,17 @@ def load_config(app, config_override=None):
app.config["GOTOSOCIAL_VISIBILITY"] = os.getenv("GOTOSOCIAL_VISIBILITY", "public")
app.config["GOTOSOCIAL_MAX_CHARS"] = int(os.getenv("GOTOSOCIAL_MAX_CHARS", "5000"))
app.config["GOTOSOCIAL_MAX_MEDIA"] = int(os.getenv("GOTOSOCIAL_MAX_MEDIA", "4"))
app.config["BLUESKY_HANDLE"] = os.getenv("BLUESKY_HANDLE", "")
app.config["BLUESKY_APP_PASSWORD"] = os.getenv("BLUESKY_APP_PASSWORD", "")
app.config["BLUESKY_SERVICE"] = os.getenv("BLUESKY_SERVICE", "https://bsky.social")
app.config["BLUESKY_NAME"] = os.getenv("BLUESKY_NAME", "Bluesky")
app.config["BLUESKY_MAX_CHARS"] = int(os.getenv("BLUESKY_MAX_CHARS", "300"))
# Crossposting to Threads (v2.6.0, ADR-069): long-lived token, refreshed automatically
app.config["THREADS_ACCESS_TOKEN"] = os.getenv("THREADS_ACCESS_TOKEN", "")
app.config["THREADS_USERNAME"] = os.getenv("THREADS_USERNAME", "")
app.config["THREADS_NAME"] = os.getenv("THREADS_NAME", "Threads")
app.config["THREADS_MAX_CHARS"] = int(os.getenv("THREADS_MAX_CHARS", "500"))
app.config["THREADS_MAX_MEDIA"] = int(os.getenv("THREADS_MAX_MEDIA", "4"))
app.config["SYNDICATION_POLL_INTERVAL"] = int(os.getenv("SYNDICATION_POLL_INTERVAL", "10"))
# Apply overrides if provided
+2 -2
View File
@@ -198,7 +198,7 @@ def generate_atom_streaming(
yield f' <link rel="enclosure" type="{_escape_xml(mime_type)}" href="{_escape_xml(media_url)}" length="{size}"{title_attr}/>\n'
# Content - include media as HTML (per Q24)
if note.html:
if note.feed_html:
# Build HTML content with media at top
html_content = ""
@@ -212,7 +212,7 @@ def generate_atom_streaming(
html_content += '</div>'
# Add text content below media
html_content += note.html
html_content += note.feed_html
# HTML content - escaped
yield ' <content type="html">'
+2 -2
View File
@@ -267,7 +267,7 @@ def _build_item_object(site_url: str, note: Note) -> Dict[str, Any]:
# Add content (HTML or text)
# Per Q24: Include media as HTML in content_html
if note.html:
if note.feed_html:
content_html = ""
# Add media at top if present (v1.2.0 Phase 3)
@@ -280,7 +280,7 @@ def _build_item_object(site_url: str, note: Note) -> Dict[str, Any]:
content_html += '</div>'
# Add text content below media
content_html += note.html
content_html += note.feed_html
item["content_html"] = content_html
else:
item["content_text"] = note.content
+2 -2
View File
@@ -137,7 +137,7 @@ def generate_rss(
html_content += '</div>'
# Add text content below media
html_content += clean_html_for_rss(note.html)
html_content += clean_html_for_rss(note.feed_html)
# feedgen automatically wraps content in CDATA for RSS
fe.description(html_content)
@@ -278,7 +278,7 @@ def generate_rss_streaming(
html_content += '</div>'
# Add text content below media
html_content += clean_html_for_rss(note.html)
html_content += clean_html_for_rss(note.feed_html)
# Build item XML
item_xml = f""" <item>
+42
View File
@@ -0,0 +1,42 @@
"""
Markdown extension: turn bare URLs into links (v2.3.0)
Python-Markdown only links <https://...>; notes often contain pasted URLs.
"""
import re
import xml.etree.ElementTree as etree
from markdown.extensions import Extension
from markdown.inlinepatterns import InlineProcessor
from markdown.treeprocessors import Treeprocessor
from markdown.util import AtomicString
# Stop at whitespace and brackets; trailing punctuation is not part of the URL
BARE_URL_RE = r"(?<![\w/\"'=])https?://[^\s<>()\[\]\"']*[^\s<>()\[\]\"'.,;:!?…]"
class _BareUrl(InlineProcessor):
def handleMatch(self, m, data):
url = m.group(0)
el = etree.Element("a")
el.set("href", url)
el.text = AtomicString(url)
return el, m.start(0), m.end(0)
class _UnnestLinks(Treeprocessor):
"""[https://x](https://x) would put a link inside a link; keep the outer one"""
def run(self, root):
for outer in root.iter("a"):
for inner in list(outer.iter("a"))[1:]:
inner.tag = "span"
inner.attrib.clear()
class BareUrlExtension(Extension):
def extendMarkdown(self, md):
# Below links (160) and <autolinks> (120), so those win
md.inlinePatterns.register(_BareUrl(BARE_URL_RE, md), "bare_url", 105)
md.treeprocessors.register(_UnnestLinks(md), "unnest_links", 0)
+459 -187
View File
@@ -7,9 +7,11 @@ interface for creating posts via external clients.
Functions:
normalize_properties: Convert form/JSON data to Micropub properties format
extract_content: Get content from Micropub properties
extract_title: Get or generate title from Micropub properties
extract_title: Get the explicit title (name) from Micropub properties
extract_tags: Get category tags from Micropub properties
handle_create: Process Micropub create action
handle_update: Process Micropub update action (v2.4.0)
handle_delete: Process Micropub delete / undelete actions (v2.4.0)
handle_query: Process Micropub query endpoints
extract_bearer_token: Get token from Authorization header or form
@@ -28,7 +30,11 @@ from typing import Optional
from flask import Request, current_app, jsonify
from starpunk.notes import create_note, get_note, InvalidNoteDataError, NoteNotFoundError
from starpunk.models import split_title
from starpunk.notes import (
create_note, delete_note, get_note, list_notes, undelete_note, update_note,
InvalidNoteDataError, NoteNotFoundError,
)
from starpunk.indieauth import check_scope
from starpunk.utils import absolute_url_for
@@ -136,6 +142,8 @@ def normalize_properties(data: dict) -> dict:
# Form format - convert to properties dict
properties = {}
for key, value in data.items():
if hasattr(value, "filename") or (isinstance(value, list) and value and hasattr(value[0], "filename")):
continue # uploaded files are handled separately
# Skip reserved Micropub parameters
if key.startswith("mp-") or key in ["action", "url", "access_token", "h"]:
continue
@@ -189,38 +197,31 @@ def extract_content(properties: dict) -> str:
def extract_title(properties: dict) -> Optional[str]:
"""
Extract or generate title from Micropub properties
Extract the explicit title (Micropub 'name' property)
Per ADR-029 mapping rules:
1. Use 'name' property if provided
2. If no name, extract from content (first line, max 50 chars)
StarPunk has no title field: a note's title is a '# Heading' first line
(see Note.title), so the caller stores the name as that heading.
Args:
properties: Normalized Micropub properties dict
Returns:
Title string or None
Title on a single line, or None if no name was sent
"""
# Try explicit name property first
name = properties.get("name", [""])[0]
if name:
return name.strip()
name = (properties.get("name") or [""])[0]
if not isinstance(name, str):
return None
return " ".join(name.split()) or None
# Generate from content (first line, max 50 chars)
content_list = properties.get("content", [])
if content_list:
content = content_list[0]
# Handle structured content
if isinstance(content, dict):
content = content.get("text") or content.get("html", "")
if content:
first_line = content.split("\n")[0].strip()
if len(first_line) > 50:
return first_line[:50] + "..."
return first_line
return None
def add_title_heading(content: str, title: Optional[str]) -> str:
"""Prefix content with '# title' unless it already starts with that heading"""
if not title:
return content
first_line = content.lstrip().split("\n", 1)[0]
if first_line.startswith("#") and first_line.lstrip("#").strip() == title:
return content
return f"# {title}\n\n{content}"
def extract_tags(properties: dict) -> list[str]:
@@ -264,6 +265,51 @@ def extract_published_date(properties: dict) -> Optional[datetime]:
# Action Handlers
MAX_PHOTOS = 4 # ADR-057
MAX_REMOTE_PHOTO_BYTES = 20 * 1024 * 1024
# Properties we can store, and so can be updated (v2.4.0)
UPDATABLE = {"content", "name", "category", "in-reply-to", "post-status", "photo"}
def _first(value):
if isinstance(value, list):
return value[0] if value else None
return value
def _require_scope(token_info: dict, *scopes: str) -> None:
granted = token_info.get("scope", "")
if not any(check_scope(s, granted) for s in scopes):
raise MicropubError(
"insufficient_scope", f"Token lacks {' or '.join(scopes)} scope", status_code=403
)
def extract_post_status(properties: dict) -> bool:
"""post-status: 'draft' -> unpublished; 'published' or absent -> published"""
status = _first(properties.get("post-status"))
if status in (None, "", "published"):
return True
if status == "draft":
return False
raise MicropubValidationError(f"Unsupported post-status: {status}")
def extract_url_property(properties: dict, name: str) -> Optional[str]:
"""A URL property (in-reply-to, like-of, repost-of): string or h-cite"""
value = _first(properties.get(name))
if isinstance(value, dict): # h-cite
value = _first((value.get("properties") or {}).get("url")) or value.get("value")
value = (value or "").strip() if isinstance(value, str) else ""
if value and not value.startswith(("http://", "https://")):
raise MicropubValidationError(f"{name} must be an http(s) URL")
return value or None
def extract_in_reply_to(properties: dict) -> Optional[str]:
return extract_url_property(properties, "in-reply-to")
def extract_photos(properties: dict) -> list[dict[str, str]]:
"""
@@ -271,101 +317,172 @@ def extract_photos(properties: dict) -> list[dict[str, str]]:
Handles both simple URL strings and structured photo objects with alt text.
Args:
properties: Normalized Micropub properties dict
Returns:
List of dicts with 'url' and optional 'alt' keys
Examples:
>>> # Simple URL
>>> extract_photos({'photo': ['https://example.com/photo.jpg']})
[{'url': 'https://example.com/photo.jpg', 'alt': ''}]
>>> # With alt text
>>> extract_photos({'photo': [{'value': 'https://example.com/photo.jpg', 'alt': 'Sunset'}]})
[{'url': 'https://example.com/photo.jpg', 'alt': 'Sunset'}]
"""
photos = properties.get("photo", [])
result = []
for photo in photos:
for photo in properties.get("photo", []):
if isinstance(photo, str):
# Simple URL string
result.append({'url': photo, 'alt': ''})
elif isinstance(photo, dict):
# Structured object with value and alt
url = photo.get('value') or photo.get('url', '')
alt = photo.get('alt', '')
if url:
result.append({'url': url, 'alt': alt})
result.append({'url': url, 'alt': photo.get('alt', '')})
return result
def _attach_photos_to_note(note_id: int, photos: list[dict[str, str]]) -> None:
"""
Attach photos to a note by URL
def _public_address(host: str) -> bool:
"""True if every address the host resolves to is public (no SSRF into the LAN)"""
import ipaddress
import socket
Photos must already exist on this server (uploaded via media endpoint).
External URLs are accepted but stored as-is (no download).
try:
infos = socket.getaddrinfo(host, None)
except socket.gaierror:
return False
for info in infos:
ip = ipaddress.ip_address(info[4][0])
if not ip.is_global:
return False
return bool(infos)
Args:
note_id: ID of the note to attach to
photos: List of dicts with 'url' and 'alt' keys
"""
def _fetch_remote_photo(url: str) -> Optional[int]:
"""Download a photo hosted elsewhere into the media library. Returns media id."""
from urllib.parse import urlparse
import httpx
from starpunk.media import save_media
parsed = urlparse(url)
if parsed.scheme not in ("http", "https") or not parsed.hostname:
return None
if not current_app.config.get("TESTING") and not _public_address(parsed.hostname):
current_app.logger.warning(f"Refusing to fetch photo from non-public address: {url}")
return None
try:
with httpx.stream("GET", url, timeout=20.0, follow_redirects=True) as response:
response.raise_for_status()
data = bytearray()
for chunk in response.iter_bytes():
data += chunk
if len(data) > MAX_REMOTE_PHOTO_BYTES:
raise ValueError("photo too large")
name = parsed.path.rsplit("/", 1)[-1] or "photo.jpg"
return save_media(bytes(data), name)["id"]
except Exception as e:
current_app.logger.warning(f"Could not fetch photo {url}: {e}")
return None
def _media_id_for_url(url: str) -> Optional[int]:
"""Media id for a photo URL: ours by path, or downloaded if hosted elsewhere"""
from starpunk.database import get_db
site_url = current_app.config.get("SITE_URL", "http://localhost:5000").rstrip('/')
if url.startswith(site_url) or url.startswith('/media/'):
path = url[len(site_url):] if url.startswith(site_url) else url
if path.startswith('/media/'):
path = path[len('/media/'):]
row = get_db(current_app).execute("SELECT id FROM media WHERE path = ?", (path,)).fetchone()
if row:
return row[0]
current_app.logger.warning(f"Photo URL not found in media: {url}")
return None
return _fetch_remote_photo(url)
def _save_uploads(files) -> list[dict]:
"""Photos uploaded in a multipart create request -> [{'media_id', 'alt'}]"""
from starpunk.media import save_media
result = []
for key in ("photo", "photo[]"):
for upload in (files.getlist(key) if files else []):
if upload and upload.filename:
try:
result.append({"media_id": save_media(upload.read(), upload.filename)["id"], "alt": ""})
except ValueError as e:
raise MicropubValidationError(f"Invalid photo {upload.filename}: {e}")
return result
def _resolve_photos(photos: list[dict]) -> tuple[list[int], list[str]]:
"""Photo dicts ({'url'|'media_id', 'alt'}) -> (media ids, captions), max 4"""
if len(photos) > MAX_PHOTOS:
current_app.logger.warning(
f"Micropub received {len(photos)} photos, truncating to {MAX_PHOTOS} per ADR-057"
)
media_ids, captions = [], []
for photo in photos:
if len(media_ids) == MAX_PHOTOS:
break
media_id = photo.get("media_id") or _media_id_for_url(photo["url"])
if media_id:
media_ids.append(media_id)
captions.append(photo.get("alt", ""))
return media_ids, captions
def _attach_photos_to_note(note_id: int, photos: list[dict]) -> None:
"""Attach photos (by URL or uploaded media id) to a note, replacing any others"""
from starpunk.media import attach_media_to_note
# Normalize SITE_URL by stripping trailing slash for consistent comparison
site_url = current_app.config.get("SITE_URL", "http://localhost:5000").rstrip('/')
db = get_db(current_app)
media_ids = []
captions = []
# Log warning if photos are being truncated
if len(photos) > 4:
current_app.logger.warning(
f"Micropub create received {len(photos)} photos, truncating to 4 per ADR-057"
)
for photo in photos[:4]: # Max 4 photos per ADR-057
url = photo['url']
alt = photo.get('alt', '')
# Check if URL is on our server
if url.startswith(site_url) or url.startswith('/media/'):
# Extract path from URL
if url.startswith(site_url):
path = url[len(site_url):]
else:
path = url
# Remove leading /media/ if present
if path.startswith('/media/'):
path = path[7:]
# Look up media by path
row = db.execute(
"SELECT id FROM media WHERE path = ?",
(path,)
).fetchone()
if row:
media_ids.append(row[0])
captions.append(alt)
else:
current_app.logger.warning(f"Photo URL not found in media: {url}")
else:
# External URL - log but don't fail
current_app.logger.info(f"External photo URL ignored: {url}")
if media_ids:
media_ids, captions = _resolve_photos(photos)
if media_ids or not photos:
attach_media_to_note(note_id, media_ids, captions)
def handle_create(data: dict, token_info: dict):
def _note_photos(note) -> list[dict]:
"""A note's current photos as Micropub photo dicts"""
from starpunk.media import get_note_media
site_url = current_app.config.get("SITE_URL", "http://localhost:5000").rstrip('/')
return [
{"url": f"{site_url}/media/{m['path']}", "alt": m.get("caption") or "", "media_id": m["id"]}
for m in get_note_media(note.id)
]
def _check_type(data: dict) -> None:
"""Only h-entry is supported"""
h = _first(data.get("h"))
kind = _first(data.get("type"))
if (h and h != "entry") or (kind and kind != "h-entry"):
raise MicropubValidationError(f"Only h-entry posts are supported (got {h or kind})")
def _note_from_url(url: Optional[str], deleted: bool = False):
"""Find the note a URL points at (/note/<slug> on this site)"""
from urllib.parse import urlparse
from starpunk.database import get_db
if not url:
raise MicropubValidationError("No url provided")
site = urlparse(current_app.config.get("SITE_URL", ""))
parsed = urlparse(url)
if parsed.netloc and site.netloc and parsed.netloc != site.netloc:
raise MicropubValidationError("URL is not on this site")
parts = [p for p in parsed.path.split("/") if p]
if len(parts) != 2 or parts[0] not in ("note", "notes"):
raise MicropubValidationError("URL is not a post on this site")
slug = parts[1]
if deleted:
row = get_db(current_app).execute(
"SELECT id FROM notes WHERE slug = ? AND deleted_at IS NOT NULL", (slug,)
).fetchone()
if row is None:
raise MicropubError("not_found", "No deleted post at that URL", 404)
return slug
note = get_note(slug=slug)
if note is None:
raise MicropubError("not_found", "Post not found", 404)
return note
def handle_create(data: dict, token_info: dict, files=None):
"""
Handle Micropub create action
@@ -375,32 +492,24 @@ def handle_create(data: dict, token_info: dict):
Args:
data: Raw request data (form or JSON)
token_info: Authenticated token information (me, client_id, scope)
files: Uploaded files for multipart requests (v2.4.0)
Returns:
Tuple of (response_body, status_code, headers)
Raises:
MicropubError: If scope insufficient or creation fails
"""
# Check scope
if not check_scope("create", token_info.get("scope", "")):
raise MicropubError(
"insufficient_scope", "Token lacks create scope", status_code=403
)
_require_scope(token_info, "create")
_check_type(data)
# Extract mp-slug BEFORE normalizing properties (it's not a property!)
# mp-slug is a Micropub server extension parameter that gets filtered during normalization
# mp-slug and mp-syndicate-to are server commands, not properties
custom_slug = None
if isinstance(data, dict) and 'mp-slug' in data:
# Handle both form-encoded (list) and JSON (could be string or list)
slug_value = data.get('mp-slug')
if isinstance(slug_value, list) and slug_value:
custom_slug = slug_value[0]
elif isinstance(slug_value, str):
custom_slug = slug_value
# mp-syndicate-to is also a server command, not a property (v2.1.0).
# Absent means "all targets"; present means exactly those.
# Absent means "all targets"; present means exactly those (v2.1.0)
requested_targets = None
if isinstance(data, dict):
for key in ("mp-syndicate-to", "mp-syndicate-to[]"):
@@ -410,54 +519,178 @@ def handle_create(data: dict, token_info: dict):
value if isinstance(value, list) else [value]
)
# Normalize and extract properties
try:
properties = normalize_properties(data)
content = extract_content(properties)
like_of = extract_url_property(properties, "like-of")
repost_of = extract_url_property(properties, "repost-of")
if like_of and repost_of:
raise MicropubValidationError("A post can't be both a like and a repost")
# Likes and reposts may come without a comment (v2.5.0)
has_content = any(
(c.strip() if isinstance(c, str) else (c.get("text") or c.get("html") or "").strip())
for c in properties.get("content", []) if c
)
content = extract_content(properties) if has_content or not (like_of or repost_of) else ""
title = extract_title(properties)
tags = extract_tags(properties)
published_date = extract_published_date(properties)
photos = extract_photos(properties) # v1.4.0
except MicropubValidationError as e:
raise e
published = extract_post_status(properties)
in_reply_to = extract_in_reply_to(properties)
photos = extract_photos(properties) + _save_uploads(files)
except MicropubError:
raise
except Exception as e:
current_app.logger.error(f"Property extraction failed: {e}")
raise MicropubValidationError(f"Failed to parse request: {str(e)}")
# Create note using existing CRUD
try:
note = create_note(
content=content,
published=True, # Micropub posts are published by default
content=add_title_heading(content, title),
published=published,
created_at=published_date,
custom_slug=custom_slug,
tags=tags if tags else None # Pass tags to create_note (v1.3.0)
tags=tags if tags else None,
in_reply_to=in_reply_to,
like_of=like_of,
repost_of=repost_of,
)
# Attach photos if present (v1.4.0)
if photos:
_attach_photos_to_note(note.id, photos)
# Crosspost after photos are attached (v2.1.0)
# Crosspost after photos are attached (v2.1.0); drafts wait until published
from starpunk.syndication import get_targets, resolve_targets, select_targets
if requested_targets is None:
select_targets(note.id, list(get_targets()))
else:
select_targets(note.id, resolve_targets(requested_targets))
permalink = absolute_url_for("public.note", slug=note.slug)
# Return 201 Created with Location header
return "", 201, {"Location": permalink}
return "", 201, {"Location": absolute_url_for("public.note", slug=note.slug)}
except InvalidNoteDataError as e:
raise MicropubValidationError(str(e))
except MicropubError:
raise
except Exception as e:
current_app.logger.error(f"Failed to create note via Micropub: {e}")
raise MicropubError(
"server_error", "Failed to create post", status_code=500
)
raise MicropubError("server_error", "Failed to create post", status_code=500)
def handle_update(data: dict, token_info: dict):
"""
Handle Micropub update action (v2.4.0)
JSON body: {"action": "update", "url": ..., "replace": {...},
"add": {...}, "delete": [...] or {...}}. Edits follow to crossposts.
"""
from starpunk.syndication import note_changed
_require_scope(token_info, "update")
note = _note_from_url(_first(data.get("url")))
replace = data.get("replace") or {}
add = data.get("add") or {}
delete = data.get("delete") or {}
if not isinstance(replace, dict) or not isinstance(add, dict) or not isinstance(delete, (dict, list)):
raise MicropubValidationError("replace and add must be objects; delete an object or list")
for prop, values in list(replace.items()) + list(add.items()):
if not isinstance(values, list):
raise MicropubValidationError(f"Values for {prop} must be an array")
touched = set(replace) | set(add) | set(delete)
unsupported = sorted(touched - UPDATABLE)
if unsupported:
raise MicropubValidationError(f"Updating {', '.join(unsupported)} is not supported")
if set(add) - {"category", "photo"}:
raise MicropubValidationError("Only category and photo can be added to")
title, body = split_title(note.content)
tags = [t["display_name"] for t in note.tags]
photos = _note_photos(note)
changes = {}
# Removals first, then replacements, then additions
if isinstance(delete, list):
for prop in delete:
if prop == "name":
title = None
elif prop == "content":
raise MicropubValidationError("content can't be deleted")
elif prop == "category":
tags = []
elif prop == "photo":
photos = []
elif prop == "in-reply-to":
changes["in_reply_to"] = ""
elif prop == "post-status":
changes["published"] = True
else:
for prop, values in delete.items():
if prop == "category":
drop = {v.lower() for v in values}
tags = [t for t in tags if t.lower() not in drop]
elif prop == "photo":
drop = {p["url"] for p in extract_photos({"photo": values})}
photos = [p for p in photos if p["url"] not in drop]
else:
raise MicropubValidationError(f"Can't delete individual values of {prop}")
if "content" in replace:
body = extract_content(replace)
if "name" in replace:
title = extract_title(replace)
if "category" in replace:
tags = extract_tags(replace)
if "photo" in replace:
photos = extract_photos(replace)
if "in-reply-to" in replace:
changes["in_reply_to"] = extract_in_reply_to(replace) or ""
if "post-status" in replace:
changes["published"] = extract_post_status(replace)
if "category" in add:
known = {t.lower() for t in tags}
tags += [t for t in extract_tags(add) if t.lower() not in known]
if "photo" in add:
photos += extract_photos(add)
new_content = add_title_heading(body, title)
if new_content != note.content:
changes["content"] = new_content
if {"category"} & touched:
changes["tags"] = tags
try:
if changes:
update_note(id=note.id, **changes)
if "photo" in touched:
_attach_photos_to_note(note.id, photos)
note_changed(note.id)
except InvalidNoteDataError as e:
raise MicropubValidationError(str(e))
except MicropubError:
raise
except Exception as e:
current_app.logger.error(f"Failed to update note via Micropub: {e}")
raise MicropubError("server_error", "Failed to update post", status_code=500)
return "", 204
def handle_delete(data: dict, token_info: dict, undelete: bool = False):
"""Handle Micropub delete / undelete (v2.4.0). Deletes are soft so they can be undone."""
_require_scope(token_info, "delete")
url = _first(data.get("url"))
try:
if undelete:
undelete_note(_note_from_url(url, deleted=True))
else:
delete_note(id=_note_from_url(url).id, soft=True)
except MicropubError:
raise
except Exception as e:
current_app.logger.error(f"Micropub {'undelete' if undelete else 'delete'} failed: {e}")
raise MicropubError("server_error", "Failed to change post", status_code=500)
return "", 204
def _syndicate_to() -> list:
@@ -467,83 +700,122 @@ def _syndicate_to() -> list:
return [{"uid": t.uid, "name": t.name} for t in get_targets().values()]
def handle_query(args: dict, token_info: dict):
def note_to_mf2(note, wanted: Optional[list] = None) -> dict:
"""
A note as Microformats2 JSON, the inverse of create
A '# Heading' first line is the title, returned as name.
"""
from starpunk.syndication import get_note_syndications
title, content = split_title(note.content)
props = {
"content": [content],
"published": [note.created_at.isoformat()],
"url": [absolute_url_for("public.note", slug=note.slug)],
"post-status": ["published" if note.published else "draft"],
}
if title:
props["name"] = [title]
if note.tags:
props["category"] = [tag["display_name"] for tag in note.tags]
if getattr(note, "in_reply_to", None):
props["in-reply-to"] = [note.in_reply_to]
if getattr(note, "like_of", None):
props["like-of"] = [note.like_of]
if getattr(note, "repost_of", None):
props["repost-of"] = [note.repost_of]
photos = _note_photos(note)
if photos:
props["photo"] = [{"value": p["url"], "alt": p["alt"]} for p in photos]
synced = [s["remote_url"] for s in get_note_syndications(note.id) if s.get("remote_url")]
if synced:
props["syndication"] = synced
if wanted:
return {"properties": {k: v for k, v in props.items() if k in wanted}}
return {"type": ["h-entry"], "properties": props}
def _all_categories(prefix: str = "") -> list:
"""Tags in use, most used first (q=category)"""
from starpunk.database import get_db
rows = get_db(current_app).execute(
"""
SELECT t.display_name, COUNT(nt.note_id) AS uses
FROM tags t LEFT JOIN note_tags nt ON nt.tag_id = t.id
GROUP BY t.id ORDER BY uses DESC, t.display_name
"""
).fetchall()
prefix = prefix.lower()
return [r["display_name"] for r in rows if not prefix or prefix in r["display_name"].lower()]
QUERIES = ["config", "source", "syndicate-to", "category"]
def handle_query(args, token_info: dict):
"""
Handle Micropub query endpoints
Supports:
- q=config: Return server configuration
- q=source: Return post source in Microformats2 JSON
- q=syndicate-to: Return syndication targets
- q=config: Server configuration
- q=source&url=...[&properties[]=...]: One post in Microformats2 JSON
- q=source (no url) [&limit=N&offset=N]: Recent posts, drafts included
- q=syndicate-to: Crosspost targets
- q=category[&filter=...]: Tags in use
Args:
args: Query string arguments
token_info: Authenticated token information
Returns:
Tuple of (response, status_code)
args: Query arguments (werkzeug MultiDict or plain dict)
"""
def get_list(key):
if hasattr(args, "getlist"):
return args.getlist(key) + args.getlist(f"{key}[]")
value = args.get(key) or args.get(f"{key}[]")
return value if isinstance(value, list) else ([value] if value else [])
q = args.get("q")
if q == "config":
# Return server configuration with media endpoint (v1.4.0)
site_url = current_app.config.get("SITE_URL", "http://localhost:5000").rstrip('/')
config = {
return jsonify({
"media-endpoint": f"{site_url}/micropub/media",
"syndicate-to": _syndicate_to(),
"q": QUERIES,
"post-types": [
{"type": "note", "name": "Note", "properties": ["content"]},
{"type": "photo", "name": "Photo", "properties": ["photo"]}
{"type": "article", "name": "Article", "properties": ["name", "content"]},
{"type": "photo", "name": "Photo", "properties": ["photo"]},
{"type": "reply", "name": "Reply", "properties": ["in-reply-to", "content"]},
{"type": "like", "name": "Like", "properties": ["like-of"]},
{"type": "repost", "name": "Repost", "properties": ["repost-of", "content"]},
],
}
return jsonify(config), 200
}), 200
elif q == "source":
# Return source of a specific post
if q == "source":
url = args.get("url")
if not url:
return error_response("invalid_request", "No URL provided")
# Extract slug from URL
wanted = get_list("properties")
if url:
try:
note = _note_from_url(url)
except MicropubError as e:
return error_response("invalid_request" if e.status_code == 400 else e.error,
e.error_description, e.status_code)
return jsonify(note_to_mf2(note, wanted)), 200
try:
# URL format: https://example.com/note/{slug}
slug = url.rstrip("/").split("/")[-1]
note = get_note(slug)
limit = max(1, min(int(args.get("limit", 10)), 50))
offset = max(0, int(args.get("offset", 0)))
except ValueError:
return error_response("invalid_request", "limit and offset must be numbers")
notes = list_notes(published_only=False, limit=limit, offset=offset)
items = [note_to_mf2(n, wanted) for n in notes]
return jsonify({"items": items}), 200
# Check if note exists
if note is None:
return error_response("invalid_request", "Post not found")
except NoteNotFoundError:
return error_response("invalid_request", "Post not found")
except Exception as e:
current_app.logger.error(f"Failed to get note source: {e}")
return error_response("server_error", "Failed to retrieve post")
# Convert note to Micropub Microformats2 format
# Note: SITE_URL is normalized to include trailing slash (for IndieAuth spec compliance)
site_url = current_app.config.get("SITE_URL", "http://localhost:5000")
mf2 = {
"type": ["h-entry"],
"properties": {
"content": [note.content],
"published": [note.created_at.isoformat()],
"url": [f"{site_url}notes/{note.slug}"],
},
}
# Add optional properties
if note.title:
mf2["properties"]["name"] = [note.title]
# Add tags if present (v1.3.0)
if note.tags:
mf2["properties"]["category"] = [tag["display_name"] for tag in note.tags]
return jsonify(mf2), 200
elif q == "syndicate-to":
if q == "syndicate-to":
return jsonify({"syndicate-to": _syndicate_to()}), 200
else:
return error_response("invalid_request", f"Unknown query: {q}")
if q == "category":
return jsonify({"categories": _all_categories(args.get("filter") or "")}), 200
return error_response("invalid_request", f"Unknown query: {q}")
+71 -4
View File
@@ -34,13 +34,32 @@ DEFAULT_AUTH_STATE_EXPIRY_MINUTES = 5
DEFAULT_TOKEN_EXPIRY_DAYS = 90
# Constants - Markdown rendering
MARKDOWN_EXTENSIONS = ["extra", "codehilite", "nl2br"]
from starpunk.markdown_ext import BareUrlExtension
MARKDOWN_EXTENSIONS = ["extra", "codehilite", "nl2br", BareUrlExtension()]
# Constants - Content limits
MAX_TITLE_LENGTH = 200
EXCERPT_LENGTH = 200
def split_title(content: str) -> tuple[Optional[str], str]:
"""
Split a note's '# Heading' first line (its title, per Q22) off its content
Returns:
(title, remaining content); title is None if the note has no heading
"""
stripped = content.lstrip()
if not stripped.startswith("#"):
return None, content
first_line, _, rest = stripped.partition("\n")
title = first_line.lstrip("#").strip()
if not title:
return None, content
return title, rest.strip()
@dataclass(frozen=True)
class Note:
"""
@@ -115,6 +134,8 @@ class Note:
deleted_at: Optional[datetime] = None
content_hash: Optional[str] = None
in_reply_to: Optional[str] = None # v1.6.0: URL being replied to
like_of: Optional[str] = None # v2.5.0: URL liked (this note is a like)
repost_of: Optional[str] = None # v2.5.0: URL reposted (this note is a repost)
_cached_content: Optional[str] = field(
default=None, repr=False, compare=False, init=False
)
@@ -173,6 +194,8 @@ class Note:
_data_dir=data_dir,
content_hash=data.get("content_hash"),
in_reply_to=data.get("in_reply_to"),
like_of=data.get("like_of"),
repost_of=data.get("repost_of"),
)
@property
@@ -242,6 +265,19 @@ class Note:
return self._cached_html
@property
def body_html(self) -> str:
"""
Rendered HTML without the title heading
Pages show a titled note's title as their own heading, so the body
leaves it out to avoid showing it twice. Untitled notes: same as html.
"""
title, body = split_title(self.content)
if title is None:
return self.html
return markdown.markdown(body, extensions=MARKDOWN_EXTENSIONS)
@property
def title(self) -> str:
"""
@@ -309,10 +345,10 @@ class Note:
'This is my note content. It has some interesting points...'
"""
try:
# Get content
content = self.content
# Leave out the title; previews show it separately
_, content = split_title(self.content)
# Remove markdown heading syntax from first line
# Remove markdown heading syntax
lines = content.split("\n")
cleaned_lines = []
for line in lines:
@@ -335,12 +371,43 @@ class Note:
excerpt = excerpt[:last_space]
excerpt += "..."
if not excerpt and self.reacts_to:
# A like or repost with no comment
verb = "Liked" if self.kind == "like" else "Reposted"
return f"{verb} {self.reacts_to}"
return excerpt
except (FileNotFoundError, OSError):
# If file doesn't exist, return slug
return self.slug
@property
def kind(self) -> str:
"""'like', 'repost' or 'note' (v2.5.0)"""
if self.like_of:
return "like"
if self.repost_of:
return "repost"
return "note"
@property
def reacts_to(self) -> Optional[str]:
"""The URL a like or repost points at"""
return self.like_of or self.repost_of
@property
def feed_html(self) -> str:
"""HTML for feeds: a repost/like line, then the note's own HTML"""
from html import escape
if not self.reacts_to:
return self.html
verb = "Liked" if self.kind == "like" else "Reposted"
url = escape(self.reacts_to, quote=True)
line = f'<p>{"★" if self.kind == "like" else "🔁"} {verb} <a href="{url}">{url}</a></p>'
return line + (self.html or "")
@property
def permalink(self) -> str:
"""
+87 -20
View File
@@ -139,7 +139,9 @@ def create_note(
created_at: Optional[datetime] = None,
custom_slug: Optional[str] = None,
tags: Optional[list[str]] = None,
in_reply_to: Optional[str] = None
in_reply_to: Optional[str] = None,
like_of: Optional[str] = None,
repost_of: Optional[str] = None,
) -> Note:
"""
Create a new note
@@ -203,7 +205,11 @@ def create_note(
- created_at and updated_at set to same value initially
"""
# 1. VALIDATION (before any changes)
if not content or not content.strip():
# Likes and reposts may have no comment (v2.5.0)
if like_of and repost_of:
raise InvalidNoteDataError("like_of", like_of, "A note can't be both a like and a repost")
content = content or ""
if not content.strip() and not (like_of or repost_of):
raise InvalidNoteDataError(
"content", content, "Content cannot be empty or whitespace-only"
)
@@ -264,10 +270,10 @@ def create_note(
try:
db.execute(
"""
INSERT INTO notes (slug, file_path, published, created_at, updated_at, content_hash, in_reply_to)
VALUES (?, ?, ?, ?, ?, ?, ?)
INSERT INTO notes (slug, file_path, published, created_at, updated_at, content_hash, in_reply_to, like_of, repost_of)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(slug, file_path_rel, published, created_at, updated_at, content_hash, in_reply_to),
(slug, file_path_rel, published, created_at, updated_at, content_hash, in_reply_to, like_of, repost_of),
)
db.commit()
except Exception as e:
@@ -322,33 +328,36 @@ def create_note(
from starpunk.webmention_sender import queue_webmentions_for_note
db_path = Path(current_app.config["DATABASE_PATH"])
site_url = current_app.config["SITE_URL"]
note_url = f"{site_url}/note/{slug}"
from starpunk.utils import absolute_url_for
note_url = absolute_url_for("public.note", slug=slug)
# Get rendered HTML from note
html_content = note.html
# Queue webmentions (best-effort, failures logged but don't fail note creation)
# The reply/like/repost target is notified first (v2.5.0)
queue_webmentions_for_note(
db_path=db_path,
note_id=note_id,
note_url=note_url,
content_html=html_content,
in_reply_to=in_reply_to
in_reply_to=in_reply_to or like_of or repost_of
)
except Exception as e:
# Webmention queueing failure should not prevent note creation
current_app.logger.warning(f"Failed to queue webmentions for note {slug}: {e}")
# 12. FETCH REPLY CONTEXT (v1.6.0 Phase 3)
# Fetch context if this is a reply
if in_reply_to:
# Fetch context for the post being replied to, liked or reposted
context_url = in_reply_to or like_of or repost_of
if context_url:
try:
from starpunk.reply_context import fetch_reply_context
db_path = Path(current_app.config["DATABASE_PATH"])
site_url = current_app.config["SITE_URL"]
# Fetch context (best-effort, failures logged but don't fail note creation)
fetch_reply_context(db_path, in_reply_to, site_url)
fetch_reply_context(db_path, context_url, site_url)
except Exception as e:
# Reply context fetch failure should not prevent note creation
current_app.logger.warning(f"Failed to fetch reply context for {in_reply_to}: {e}")
@@ -472,6 +481,7 @@ def list_notes(
offset: int = 0,
order_by: str = "created_at",
order_dir: str = "DESC",
likes: Optional[bool] = None,
) -> list[Note]:
"""
List notes with filtering and pagination
@@ -564,6 +574,11 @@ def list_notes(
params = []
if published_only:
query += " AND published = 1"
# likes: None = all notes, False = leave likes out, True = only likes (v2.5.0)
if likes is True:
query += " AND like_of IS NOT NULL"
elif likes is False:
query += " AND like_of IS NULL"
# Add ordering (safe because order_by validated above)
query += f" ORDER BY {order_by} {order_dir}"
@@ -663,12 +678,9 @@ def update_note(
if content is None and published is None and tags is None and in_reply_to is None:
raise ValueError("Must provide at least one of content, published, tags, or in_reply_to to update")
# Validate content if provided
if content is not None:
if not content or not content.strip():
raise InvalidNoteDataError(
"content", content, "Content cannot be empty or whitespace-only"
)
# Validate content if provided (checked against the note below for
# likes/reposts, whose comment may be empty)
empty_content = content is not None and not content.strip()
# 2. GET EXISTING NOTE
# Phase 6: Load content to get old HTML for webmention URL diffing
@@ -678,6 +690,11 @@ def update_note(
identifier = slug if slug is not None else id
raise NoteNotFoundError(identifier)
if empty_content and not (existing_note.like_of or existing_note.repost_of):
raise InvalidNoteDataError(
"content", content, "Content cannot be empty or whitespace-only"
)
# 3. SETUP
updated_at = datetime.utcnow()
data_dir = Path(current_app.config["DATA_PATH"])
@@ -815,13 +832,14 @@ def update_note(
from starpunk.webmention_sender import queue_webmentions_for_note
db_path = Path(current_app.config["DATABASE_PATH"])
site_url = current_app.config["SITE_URL"]
note_url = f"{site_url}/note/{updated_note.slug}"
from starpunk.utils import absolute_url_for
note_url = absolute_url_for("public.note", slug=updated_note.slug)
# Get rendered HTML from updated note (if published)
html_content = updated_note.html if updated_note.published else None
# Determine in_reply_to (use new value if provided, otherwise existing)
reply_to = in_reply_to if in_reply_to is not None else updated_note.in_reply_to
reply_to = (in_reply_to if in_reply_to is not None else updated_note.in_reply_to) or updated_note.like_of or updated_note.repost_of
# Queue webmentions (best-effort)
# Pass old_html for URL diffing (Phase 6)
@@ -838,8 +856,8 @@ def update_note(
current_app.logger.warning(f"Failed to queue webmentions for note {updated_note.slug}: {e}")
# 9. FETCH REPLY CONTEXT (v1.6.0 Phase 3)
# Fetch context if in_reply_to changed and is not None
if in_reply_to is not None:
# Fetch context if in_reply_to changed and is set ("" clears it)
if in_reply_to:
try:
from starpunk.reply_context import fetch_reply_context
db_path = Path(current_app.config["DATABASE_PATH"])
@@ -1034,3 +1052,52 @@ def delete_note(
# 6. RETURN (no value)
return None
def undelete_note(slug: str) -> Note:
"""
Restore a soft-deleted note (v2.4.0, Micropub undelete)
Moves the file back from .trash/ and clears deleted_at. Crossposts are
queued again, so a note deleted from them is posted again.
Raises:
NoteNotFoundError: If there is no soft-deleted note with this slug
"""
import shutil
from starpunk.utils import TRASH_DIR_NAME
db = get_db(current_app)
row = db.execute(
"SELECT id, file_path, created_at FROM notes WHERE slug = ? AND deleted_at IS NOT NULL",
(slug,),
).fetchone()
if row is None:
raise NoteNotFoundError(slug)
data_dir = Path(current_app.config["DATA_PATH"])
note_path = data_dir / row["file_path"]
if not note_path.exists():
# delete_note_file put it in .trash/<year>/<month>/<name>
parts = Path(row["file_path"]).parts
candidates = []
if "notes" in parts and len(parts) > parts.index("notes") + 2:
i = parts.index("notes")
candidates.append(data_dir / TRASH_DIR_NAME / parts[i + 1] / parts[i + 2] / note_path.name)
candidates += sorted((data_dir / TRASH_DIR_NAME).glob(f"*/*/{note_path.name}"))
source = next((c for c in candidates if c.exists()), None)
if source is None:
raise NoteSyncError("undelete", f"Trashed file not found for {slug}", "Cannot restore note file")
note_path.parent.mkdir(parents=True, exist_ok=True)
shutil.move(str(source), str(note_path))
db.execute("UPDATE notes SET deleted_at = NULL WHERE id = ?", (row["id"],))
db.commit()
try:
from starpunk.syndication import note_changed
note_changed(row["id"])
except Exception as e:
current_app.logger.warning(f"Failed to queue syndication after undelete: {e}")
return get_note(slug=slug)
+1 -1
View File
@@ -277,7 +277,7 @@ def update_note_submit(note_id: int):
published = "published" in request.form
tags_input = request.form.get("tags", "")
if not content:
if not content and not existing_note.reacts_to: # likes/reposts may have no comment
flash("Content cannot be empty", "error")
return redirect(url_for("admin.edit_note_form", note_id=note_id))
+18 -18
View File
@@ -27,6 +27,8 @@ from starpunk.micropub import (
extract_bearer_token,
error_response,
handle_create,
handle_delete,
handle_update,
handle_query,
)
from starpunk.indieauth import verify_access_token, check_scope
@@ -49,8 +51,8 @@ def micropub_endpoint():
POST requests:
Handle action operations (form-encoded or JSON):
- action=create (or no action): Create new post
- action=update: Update existing post (not supported in V1)
- action=delete: Delete post (not supported in V1)
- action=update: Update existing post (JSON; replace/add/delete)
- action=delete / action=undelete: Delete or restore a post
Authentication:
Requires valid bearer token in Authorization header or
@@ -79,7 +81,7 @@ def micropub_endpoint():
# Handle query endpoints (GET requests)
if request.method == "GET":
try:
return handle_query(request.args.to_dict(), token_info)
return handle_query(request.args, token_info)
except MicropubError as e:
return error_response(e.error, e.error_description, e.status_code)
except Exception as e:
@@ -95,23 +97,19 @@ def micropub_endpoint():
data = request.get_json() or {}
action = data.get("action", "create")
else:
# Form-encoded or multipart (V1 only supports form-encoded)
# Form-encoded or multipart (files handled separately)
data = request.form.to_dict(flat=False)
action = data.get("action", ["create"])[0]
# Route to appropriate handler
if action == "create":
return handle_create(data, token_info)
return handle_create(data, token_info, files=request.files)
elif action == "update":
# V1: Update not supported
return error_response(
"invalid_request", "Update action not supported in V1", 400
)
elif action == "delete":
# V1: Delete not supported
return error_response(
"invalid_request", "Delete action not supported in V1", 400
)
if "application/json" not in content_type:
return error_response("invalid_request", "Updates must be sent as JSON", 400)
return handle_update(data, token_info)
elif action in ("delete", "undelete"):
return handle_delete(data, token_info, undelete=(action == "undelete"))
else:
return error_response("invalid_request", f"Unknown action: {action}", 400)
@@ -122,7 +120,8 @@ def micropub_endpoint():
return error_response("server_error", "An unexpected error occurred", 500)
@bp.route('/media', methods=['POST'])
@bp.route('/micropub/media', methods=['POST'])
@bp.route('/media', methods=['POST']) # old path, kept for existing clients
def media_endpoint():
"""
Micropub media endpoint for file uploads
@@ -145,11 +144,12 @@ def media_endpoint():
if not token_info:
return error_response("unauthorized", "Invalid or expired access token", 401)
# Check scope (create scope allows media upload)
if not check_scope("create", token_info.get("scope", "")):
# Check scope: "media", or "create" for older clients
scope = token_info.get("scope", "")
if not (check_scope("media", scope) or check_scope("create", scope)):
return error_response(
"insufficient_scope",
"Token lacks create scope",
"Token lacks media or create scope",
403
)
+34 -3
View File
@@ -64,7 +64,7 @@ def _get_cached_notes():
# Cache expired or empty, fetch fresh notes
max_items = current_app.config.get("FEED_MAX_ITEMS", 50)
notes = list_notes(published_only=True, limit=max_items)
notes = list_notes(published_only=True, limit=max_items, likes=False)
# Batch load media and tags for all notes (v1.5.0 Phase 3)
# Reduces query count from O(n) to O(1) for both media and tags
@@ -249,8 +249,8 @@ def index():
from starpunk.media import get_note_media
from starpunk.tags import get_note_tags
# Get recent published notes (limit 20)
notes = list_notes(published_only=True, limit=20)
# Get recent published notes (limit 20); likes have their own page (v2.5.0)
notes = list_notes(published_only=True, limit=20, likes=False)
# Attach media and tags to each note for display
for note in notes:
@@ -262,9 +262,38 @@ def index():
tags = get_note_tags(note.id)
object.__setattr__(note, '_cached_tags', tags)
attach_contexts(notes)
return render_template("index.html", notes=notes)
def attach_contexts(notes) -> None:
"""Give likes and reposts the fetched context of the post they point at"""
from starpunk.webmention_queries import get_reply_context
for note in notes:
context = None
if note.reacts_to:
try:
context = get_reply_context(note.reacts_to)
except Exception:
context = None
object.__setattr__(note, 'context', context)
@bp.route("/likes")
def likes():
"""Likes, kept off the home page and feeds (v2.5.0)"""
from starpunk.media import get_note_media
from starpunk.tags import get_note_tags
notes = list_notes(published_only=True, limit=50, likes=True)
for note in notes:
object.__setattr__(note, 'media', get_note_media(note.id))
object.__setattr__(note, '_cached_tags', get_note_tags(note.id))
attach_contexts(notes)
return render_template("likes.html", notes=notes)
@bp.route("/note/<slug>")
def note(slug: str):
"""
@@ -334,6 +363,7 @@ def note(slug: str):
from starpunk.syndication import get_note_syndications
syndications = [s for s in get_note_syndications(note_obj.id) if s["state"] == "synced"]
attach_contexts([note_obj])
return render_template(
"note.html",
@@ -384,6 +414,7 @@ def tag(tag: str):
for note in notes:
media = get_note_media(note.id)
object.__setattr__(note, 'media', media)
attach_contexts(notes)
return render_template(
"tag.html",
+745 -20
View File
@@ -1,9 +1,10 @@
"""
Crossposting (POSSE) for StarPunk (v2.1.0, ADR-065)
Crossposting (POSSE) for StarPunk (v2.1.0 ADR-065, v2.2.0 ADR-066, v2.6.0 ADR-069)
Published notes are copied to syndication targets, and later edits and
deletes follow them. Currently one target kind exists: a GoToSocial (or any
Mastodon-API) account.
deletes follow them. Three target kinds exist: a GoToSocial (or any
Mastodon-API) account, a Bluesky account and a Threads account. Bluesky and
Threads posts can't be edited, so there only deletes follow.
Each (note, target) pair has a row in `syndications`. Changing a note marks
its rows pending; a background worker then reconciles the remote post with
@@ -16,15 +17,26 @@ the note:
Rows carry a version so an edit made while a send is in flight is not lost.
"""
import fcntl
import hashlib
import io
import json
import logging
import os
import re
import threading
from dataclasses import dataclass
import time
from dataclasses import dataclass, field
from datetime import datetime, timezone
from pathlib import Path
from typing import Dict, List, Optional
import httpx
from flask import current_app
# httpx logs request URLs at INFO, and Threads takes its token in the URL
logging.getLogger("httpx").setLevel(logging.WARNING)
from starpunk.database import get_db
MAX_ATTEMPTS = 8
@@ -108,18 +120,607 @@ class MastodonClient:
except RemoteGone:
pass # already gone
# Interface used by the reconcile loop
can_edit = True
NATIVE = ("favourite", "reblog") # remote_id prefixes for likes/boosts (v2.5.0)
def find_status(self, url: str) -> Optional[str]:
"""Local id of a fediverse post, fetching it if needed; None if the URL isn't one"""
try:
response = self._call("GET", "/api/v2/search", params={
"q": url, "resolve": "true", "type": "statuses", "limit": 1,
})
except RemoteGone:
return None
statuses = response.json().get("statuses") or []
return statuses[0]["id"] if statuses else None
def _prepare(self, post: "Post") -> tuple:
if post.kind == "repost":
return compose_repost_text(post, post.note.content, post.max_chars), []
text = compose_text(post.note, post.permalink, post.tags, post.max_chars)
media_ids = [self.upload_media(f["path"], f["mime_type"], f["description"]) for f in post.media]
return text, media_ids
def create(self, post: "Post", key: str, retrying: bool = False) -> dict:
if post.kind in ("like", "repost") and post.target_url:
# Native favourite/boost when the target is a fediverse post; a
# repost with a comment is posted as a status so the comment stays
commented = bool(post.note.content.strip())
if post.kind == "like" or not commented:
status_id = self.find_status(post.target_url)
if status_id:
action = "favourite" if post.kind == "like" else "reblog"
self._call("POST", f"/api/v1/statuses/{status_id}/{action}")
return {"id": f"{action}:{status_id}", "url": None}
if post.kind == "like":
return {"id": None, "url": None} # nothing to like here
text, media_ids = self._prepare(post)
return self.create_status(text, media_ids, idempotency_key=key)
def update(self, remote_id: str, post: "Post") -> dict:
if remote_id.split(":", 1)[0] in self.NATIVE:
return {"id": remote_id, "url": None} # a favourite/boost has nothing to edit
text, media_ids = self._prepare(post)
return self.update_status(remote_id, text, media_ids)
def delete(self, remote_id: str) -> None:
action, _, status_id = remote_id.partition(":")
if action in self.NATIVE and status_id:
try:
self._call("POST", f"/api/v1/statuses/{status_id}/un{action}")
except RemoteGone:
pass
return
self.delete_status(remote_id)
BLUESKY_BLOB_LIMIT = 1_000_000 # app.bsky.embed.images maxSize
_TID_CHARS = "234567abcdefghijklmnopqrstuvwxyz"
class BlueskyClient:
"""
Minimal AT Protocol client for posting to Bluesky with an app password
Posts are plain text with link and hashtag facets. Images go in an
images embed; a note without images gets a link card instead.
"""
can_edit = False # Bluesky has no edits; only deletes follow the note
POST = "app.bsky.feed.post"
def __init__(self, service: str, identifier: str, password: str,
http: Optional[httpx.Client] = None):
self.service = service.rstrip("/")
self.identifier = identifier
self.password = password
self.http = http or httpx.Client(timeout=30.0)
self.pds = self.service
self.did: Optional[str] = None
self.handle = identifier
self._access: Optional[str] = None
self._refresh: Optional[str] = None
self._lock = threading.Lock()
# --- XRPC plumbing ---
def _raw(self, method: str, nsid: str, base: str, token: Optional[str], **kwargs) -> httpx.Response:
headers = kwargs.pop("headers", {})
if token:
headers["Authorization"] = f"Bearer {token}"
try:
return self.http.request(method, f"{base}/xrpc/{nsid}", headers=headers, **kwargs)
except httpx.HTTPError as e:
raise SyndicationError(f"{nsid}: {e}")
@staticmethod
def _error_name(response: httpx.Response) -> str:
try:
return response.json().get("error") or ""
except ValueError:
return ""
def _raise_for(self, nsid: str, response: httpx.Response) -> None:
if response.status_code < 400:
return
permanent = 400 <= response.status_code < 500 and response.status_code != 429
raise SyndicationError(
f"{nsid}: HTTP {response.status_code} {response.text[:200]}", permanent=permanent
)
def _login(self) -> None:
response = None
if self._refresh:
response = self._raw("POST", "com.atproto.server.refreshSession", self.service, self._refresh)
if response.status_code >= 400:
response = None
if response is None:
response = self._raw("POST", "com.atproto.server.createSession", self.service, None,
json={"identifier": self.identifier, "password": self.password})
self._raise_for("com.atproto.server.createSession", response)
data = response.json()
self._access, self._refresh = data["accessJwt"], data["refreshJwt"]
self.did = data["did"]
self.handle = data.get("handle") or self.handle
for service in (data.get("didDoc") or {}).get("service", []):
if service.get("id", "").endswith("#atproto_pds") and service.get("serviceEndpoint"):
self.pds = service["serviceEndpoint"].rstrip("/")
def _call(self, method: str, nsid: str, **kwargs) -> httpx.Response:
with self._lock:
if not self._access:
self._login()
response = self._raw(method, nsid, self.pds, self._access, **kwargs)
if response.status_code == 401 or self._error_name(response) == "ExpiredToken":
self._access = None
self._login()
response = self._raw(method, nsid, self.pds, self._access, **kwargs)
self._raise_for(nsid, response)
return response
# --- Posts ---
def upload_blob(self, data: bytes, mime_type: str) -> dict:
response = self._call("POST", "com.atproto.repo.uploadBlob", content=data,
headers={"Content-Type": mime_type})
return response.json()["blob"]
def _image(self, media: dict) -> dict:
data, mime_type, width, height = _fit_image(media["path"], media["mime_type"], BLUESKY_BLOB_LIMIT)
image = {"image": self.upload_blob(data, mime_type), "alt": media["description"] or ""}
if width and height:
image["aspectRatio"] = {"width": width, "height": height}
return image
@staticmethod
def _now() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z"
def _record(self, post: "Post", text: Optional[str] = None, embed: Optional[dict] = None) -> dict:
if text is None:
text = compose_text(post.note, post.permalink, post.tags, post.max_chars,
body=plain_text(post.note.content))
record = {"$type": self.POST, "text": text, "createdAt": self._now()}
facets = bluesky_facets(text)
if facets:
record["facets"] = facets
if embed is not None:
record["embed"] = embed
elif post.media:
record["embed"] = {"$type": "app.bsky.embed.images",
"images": [self._image(m) for m in post.media]}
else:
record["embed"] = {"$type": "app.bsky.embed.external", "external": {
"uri": post.permalink,
"title": post.note.title[:300],
"description": plain_text(post.note.content)[:300],
}}
return record
_POST_URL = re.compile(r"^https://bsky\.app/profile/([^/]+)/post/([^/?#]+)")
def resolve_post(self, url: str) -> Optional[dict]:
"""{'uri', 'cid'} of a Bluesky post URL; None if the URL isn't one"""
m = self._POST_URL.match(url or "")
if not m:
return None
actor, rkey = m.groups()
if not actor.startswith("did:"):
actor = self._call("GET", "com.atproto.identity.resolveHandle",
params={"handle": actor}).json()["did"]
uri = f"at://{actor}/{self.POST}/{rkey}"
posts = self._call("GET", "app.bsky.feed.getPosts", params={"uris": uri}).json().get("posts") or []
return {"uri": uri, "cid": posts[0]["cid"]} if posts else None
def _create(self, collection: str, record: dict) -> str:
response = self._call("POST", "com.atproto.repo.createRecord",
json={"repo": self.did or self.identifier, "collection": collection,
"record": record})
return response.json()["uri"]
def _react(self, post: "Post") -> dict:
"""Like, repost or quote a Bluesky post; fall back to a link post for reposts"""
comment = plain_text(post.note.content) if post.note.content.strip() else ""
subject = self.resolve_post(post.target_url)
if subject and post.kind == "like":
uri = self._create("app.bsky.feed.like", {
"$type": "app.bsky.feed.like", "subject": subject, "createdAt": self._now()})
return {"id": uri, "url": None}
if subject and not comment:
uri = self._create("app.bsky.feed.repost", {
"$type": "app.bsky.feed.repost", "subject": subject, "createdAt": self._now()})
return {"id": uri, "url": None}
if subject: # repost with a comment -> quote post
text = comment if len(comment) <= post.max_chars else \
comment[:post.max_chars - 1].rsplit(" ", 1)[0].rstrip() + "…"
uri = self._create(self.POST, self._record(
post, text=text, embed={"$type": "app.bsky.embed.record", "record": subject}))
return {"id": uri, "url": self._web_url(uri)}
if post.kind == "like":
return {"id": None, "url": None} # not a Bluesky post: nothing to like
text = compose_repost_text(post, comment, post.max_chars)
embed = {"$type": "app.bsky.embed.external", "external": {
"uri": post.target_url, "title": (post.target_title or post.target_url)[:300],
"description": ""}}
uri = self._create(self.POST, self._record(post, text=text, embed=embed))
return {"id": uri, "url": self._web_url(uri)}
def _web_url(self, uri: str) -> str:
rkey = uri.rsplit("/", 1)[1]
return f"https://bsky.app/profile/{self.handle or self.did}/post/{rkey}"
def _find_existing(self, permalink: str) -> Optional[dict]:
"""A recent post that already links to this note (an earlier send whose reply was lost)"""
response = self._call("GET", "com.atproto.repo.listRecords",
params={"repo": self.did, "collection": self.POST, "limit": 25})
for item in response.json().get("records", []):
if permalink in (item.get("value") or {}).get("text", ""):
return {"id": item["uri"], "url": self._web_url(item["uri"])}
return None
def create(self, post: "Post", key: str, retrying: bool = False) -> dict:
if post.kind in ("like", "repost") and post.target_url:
if not self.did:
with self._lock:
self._login()
return self._react(post)
if retrying:
if not self.did:
with self._lock:
self._login()
existing = self._find_existing(post.permalink)
if existing:
return existing
record = self._record(post)
response = self._call("POST", "com.atproto.repo.createRecord",
json={"repo": self.did or self.identifier, "collection": self.POST,
"record": record})
uri = response.json()["uri"]
return {"id": uri, "url": self._web_url(uri)}
def update(self, remote_id: str, post: "Post") -> dict:
raise NotImplementedError("Bluesky posts can't be edited")
def delete(self, remote_id: str) -> None:
# at://<did>/app.bsky.feed.post/<rkey>; deleting a missing record succeeds
repo, collection, rkey = remote_id[len("at://"):].split("/", 2)
self._call("POST", "com.atproto.repo.deleteRecord",
json={"repo": repo, "collection": collection, "rkey": rkey})
def _fit_image(path: Path, mime_type: str, limit: int) -> tuple:
"""Image bytes no bigger than `limit`, re-encoded as JPEG if needed. Returns (data, mime, w, h)."""
from PIL import Image
data = Path(path).read_bytes()
with Image.open(io.BytesIO(data)) as img:
width, height = img.size
if len(data) <= limit:
return data, mime_type, width, height
img = img.convert("RGB")
for max_side, quality in ((2000, 85), (2000, 75), (1600, 70), (1200, 65), (1000, 60)):
copy = img.copy()
copy.thumbnail((max_side, max_side))
buf = io.BytesIO()
copy.save(buf, format="JPEG", quality=quality, optimize=True)
if buf.tell() <= limit:
return buf.getvalue(), "image/jpeg", copy.size[0], copy.size[1]
raise SyndicationError(f"{Path(path).name}: image too large for Bluesky", permanent=True)
THREADS_API = "https://graph.threads.net/v1.0"
THREADS_IMAGE_TYPES = {"image/jpeg", "image/png"}
THREADS_IMAGE_LIMIT = 8_000_000
THREADS_REFRESH_EVERY = 24 * 3600 # tokens last 60 days; Meta allows a refresh once one is a day old
# Graph API error codes that are temporary (rate limits, service hiccups)
_GRAPH_TRANSIENT = {1, 2, 4, 17, 32, 341, 613}
_GRAPH_AUTH = {10, 190, 200} # bad token or missing permission: retrying won't help
def threads_length(text: str) -> int:
"""Length as Threads counts it: emoji (and, to be safe, all non-ASCII) by UTF-8 bytes"""
return sum(1 if ord(ch) < 128 else len(ch.encode()) for ch in text)
def threads_topic(tags: List[str]) -> str:
"""The first tag usable as a Threads topic (one per post; no '.' or '&', 1-50 chars)"""
for tag in tags:
topic = re.sub(r"[.&]", "", tag).strip()[:50]
if topic:
return topic
return ""
class ThreadsClient:
"""
Minimal Threads API client (Meta Graph API)
Posting is two steps: create a container, wait for it to be ready, then
publish it. Meta fetches images from their public URL on this site.
Posts can't be edited through the API, so only deletes follow.
Long-lived tokens expire after 60 days unless refreshed. `maintain()`
refreshes the token once a day and keeps it in `token_file`, so it
survives restarts. A different token in the config replaces it.
"""
can_edit = False
def __init__(self, token: str, token_file: Optional[Path] = None, api: str = THREADS_API,
http: Optional[httpx.Client] = None, sleep=time.sleep):
self.config_token = token
self.token_file = Path(token_file) if token_file else None
self.api = api.rstrip("/")
self.http = http or httpx.Client(timeout=30.0)
self.sleep = sleep
self.user_id: Optional[str] = None
self.username: Optional[str] = None
self._lock = threading.Lock()
self._token = token
self._checked_at = 0.0
saved = self._read_file()
if saved:
self._token = saved["token"]
# --- Token ---
def _seed(self) -> str:
return hashlib.sha256(self.config_token.encode()).hexdigest()
def _read_file(self) -> Optional[dict]:
"""The saved token, if it descends from the configured one"""
if not self.token_file or not self.token_file.exists():
return None
try:
data = json.loads(self.token_file.read_text())
except (OSError, ValueError):
return None
return data if data.get("seed") == self._seed() and data.get("token") else None
def _write_file(self, token: str, refreshed_at: float) -> None:
tmp = self.token_file.with_suffix(".tmp")
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
with os.fdopen(fd, "w") as f:
json.dump({"seed": self._seed(), "token": token, "refreshed_at": refreshed_at}, f)
os.replace(tmp, self.token_file)
def maintain(self) -> None:
"""Refresh the token if a day has passed since the last try (cheap to call often)"""
now = time.time()
if now - self._checked_at < THREADS_REFRESH_EVERY:
return
self._checked_at = now
if not self.token_file:
return
# Gunicorn workers share the file: one refreshes, the others pick up its result
self.token_file.parent.mkdir(parents=True, exist_ok=True)
with open(self.token_file.with_suffix(".lock"), "w") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
saved = self._read_file()
if saved:
self._token = saved["token"]
if now - saved.get("refreshed_at", 0) < THREADS_REFRESH_EVERY:
self._checked_at = saved["refreshed_at"]
return
base = self.api.rsplit("/", 1)[0] # refresh lives outside the versioned path
try:
response = self.http.get(f"{base}/refresh_access_token", params={
"grant_type": "th_refresh_token", "access_token": self._token})
except httpx.HTTPError as e:
current_app.logger.warning(f"Threads token refresh failed: {type(e).__name__}")
self._checked_at = now - THREADS_REFRESH_EVERY + 3600 # try again in an hour
return
if response.status_code < 400:
self._token = response.json()["access_token"]
current_app.logger.info("Threads access token refreshed")
else:
# e.g. a token less than a day old: keep it, try again tomorrow
current_app.logger.warning(
f"Threads token refresh failed: HTTP {response.status_code} {response.text[:200]}")
self._write_file(self._token, now)
# --- Graph plumbing ---
def _call(self, method: str, path: str, **params) -> dict:
params["access_token"] = self._token
kwargs = {"params": params} if method in ("GET", "DELETE") else {"data": params}
try:
response = self.http.request(method, f"{self.api}/{path}", **kwargs)
except httpx.HTTPError as e:
raise SyndicationError(f"{method} {path}: {type(e).__name__}") # never echo the URL/token
try:
body = response.json()
except ValueError:
body = {}
error = body.get("error") if isinstance(body, dict) else None
if response.status_code >= 400 or error:
error = error if isinstance(error, dict) else {}
code = error.get("code")
detail = " / ".join(str(error[k]) for k in ("error_user_title", "error_user_msg")
if error.get(k))
message = (f"{method} {path}: HTTP {response.status_code} "
f"{error.get('message') or response.text[:200]}"
f" (code {code}, subcode {error.get('error_subcode')}"
f"{', ' + detail if detail else ''}, fbtrace {error.get('fbtrace_id')})")
if response.status_code == 404 or (code == 100 and error.get("error_subcode") == 33):
raise RemoteGone(message, permanent=True)
permanent = (400 <= response.status_code < 500 and response.status_code != 429
and code not in _GRAPH_TRANSIENT)
exc = SyndicationError(message, permanent=permanent)
exc.code = code
raise exc
return body
def _me(self) -> str:
with self._lock:
if not self.user_id:
me = self._call("GET", "me", fields="id,username")
self.user_id, self.username = me["id"], me.get("username")
return self.user_id
def _wait(self, container_id: str) -> None:
"""Wait until Meta has processed a container (images take a few seconds)"""
delay = 1
self.sleep(2) # Meta can report a container ready a moment too early
for _ in range(8):
status = self._call("GET", container_id, fields="status,error_message")
state = status.get("status")
if state in ("FINISHED", None):
return
if state in ("ERROR", "EXPIRED"):
raise SyndicationError(
f"Threads container {state}: {status.get('error_message') or 'no detail'}",
permanent=True)
self.sleep(delay)
delay = min(delay * 2, 15)
raise SyndicationError("Threads container still processing")
# --- Posts ---
def _text(self, post: "Post") -> str:
body = plain_text(post.note.content)
limit = post.max_chars
while True:
text = compose_text(post.note, post.permalink, [], limit, body=body)
over = threads_length(text) - post.max_chars
if over <= 0:
return text
limit -= over
def _image_container(self, uid: str, media: dict, **extra) -> str:
params = {"media_type": "IMAGE", "image_url": media["url"], **extra}
if media["description"]:
params["alt_text"] = media["description"][:1000]
return self._call("POST", f"{uid}/threads", **params)["id"]
def _repost_text(self, post: "Post") -> str:
comment = plain_text(post.note.content)
limit = post.max_chars
while True:
text = compose_repost_text(post, comment, limit)
over = threads_length(text) - post.max_chars
if over <= 0:
return text
limit -= over
def _container(self, uid: str, post: "Post") -> str:
if post.kind == "repost":
# No native repost of outside content: post "🔁 title url" with a link preview
return self._call("POST", f"{uid}/threads", media_type="TEXT",
text=self._repost_text(post), link_attachment=post.target_url)["id"]
common = {"text": self._text(post)}
topic = threads_topic(post.tags)
if topic:
common["topic_tag"] = topic
# Threads takes JPEG/PNG up to 8 MB; others (GIF, WebP) are left off
images = [m for m in post.media if m["mime_type"] in THREADS_IMAGE_TYPES
and Path(m["path"]).stat().st_size <= THREADS_IMAGE_LIMIT]
if not images:
return self._call("POST", f"{uid}/threads", media_type="TEXT",
link_attachment=post.permalink, **common)["id"]
if len(images) == 1:
return self._image_container(uid, images[0], **common)
children = [self._image_container(uid, m, is_carousel_item="true") for m in images]
for child in children:
self._wait(child)
return self._call("POST", f"{uid}/threads", media_type="CAROUSEL",
children=",".join(children), **common)["id"]
def _profile_url(self) -> str:
return f"https://www.threads.com/@{self.username}" if self.username else "https://www.threads.com/"
def _find_existing(self, uid: str, link: str) -> Optional[dict]:
"""A recent post that already has this link (an earlier send whose reply was lost)"""
listing = self._call("GET", f"{uid}/threads", fields="id,text,permalink", limit=25)
for item in listing.get("data", []):
if link in (item.get("text") or ""):
return {"id": item["id"], "url": item.get("permalink") or self._profile_url()}
return None
def create(self, post: "Post", key: str, retrying: bool = False) -> dict:
if post.kind == "like" and post.target_url:
return {"id": None, "url": None} # the API can't like things; likes stay on the site
uid = self._me()
if retrying:
marker = post.target_url if post.kind == "repost" and post.target_url else post.permalink
existing = self._find_existing(uid, marker)
if existing:
return existing
container = self._container(uid, post)
self._wait(container)
try:
media_id = self._call("POST", f"{uid}/threads_publish", creation_id=container)["id"]
except SyndicationError as e:
if getattr(e, "code", None) in _GRAPH_AUTH:
raise
# Meta's publish step fails now and then ("Fatal") and works on a
# later try; the retry checks for an existing post first
raise SyndicationError(str(e))
try:
url = self._call("GET", media_id, fields="permalink").get("permalink")
except SyndicationError:
url = None # it's published; don't fail (and post twice) over the link
return {"id": media_id, "url": url or self._profile_url()}
def update(self, remote_id: str, post: "Post") -> dict:
raise NotImplementedError("Threads posts can't be edited through the API")
def delete(self, remote_id: str) -> None:
try:
self._call("DELETE", remote_id)
except RemoteGone:
pass # already gone
@dataclass
class Target:
id: str # stored in syndications.target
uid: str # Micropub syndicate-to uid
name: str # shown in the editor and Micropub clients
client: MastodonClient
client: object # MastodonClient or BlueskyClient
max_chars: int = 5000
max_media: int = 4
_client_cache: Dict[str, MastodonClient] = {}
@dataclass
class Post:
"""What a client needs to publish a note"""
note: object
permalink: str
tags: List[str]
max_chars: int
media: List[dict] = field(default_factory=list)
kind: str = "note" # note, like or repost (v2.5.0)
target_url: Optional[str] = None # what a like/repost points at
target_title: Optional[str] = None # its fetched title, if any
def compose_repost_text(post: "Post", comment: str, max_chars: int) -> str:
"""'comment' then '🔁 title' and the URL; the comment is shortened to fit"""
title = post.target_title or "Reposted"
tail = f"🔁 {title}\n{post.target_url}"
if len(tail) > max_chars:
tail = f"🔁 {post.target_url}"
comment = (comment or "").strip()
if not comment:
return tail
room = max_chars - len(tail) - 2
if len(comment) > room:
comment = comment[:max(room - 1, 0)].rsplit(" ", 1)[0].rstrip() + "…"
return f"{comment}\n\n{tail}" if room > 1 else tail
_client_cache: Dict[str, object] = {}
def targets_configured(config) -> bool:
"""Whether any crossposting target has credentials (used to start the worker)"""
return bool(
(config.get("GOTOSOCIAL_URL") and config.get("GOTOSOCIAL_ACCESS_TOKEN"))
or (config.get("BLUESKY_HANDLE") and config.get("BLUESKY_APP_PASSWORD"))
or config.get("THREADS_ACCESS_TOKEN")
)
def get_targets() -> Dict[str, Target]:
@@ -142,6 +743,41 @@ def get_targets() -> Dict[str, Target]:
max_chars=int(config.get("GOTOSOCIAL_MAX_CHARS", 5000)),
max_media=int(config.get("GOTOSOCIAL_MAX_MEDIA", 4)),
)
handle = (config.get("BLUESKY_HANDLE") or "").lstrip("@")
password = config.get("BLUESKY_APP_PASSWORD") or ""
if handle and password:
service = (config.get("BLUESKY_SERVICE") or "https://bsky.social").rstrip("/")
key = f"bluesky:{service}:{handle}"
client = _client_cache.get(key)
if client is None:
client = BlueskyClient(service, handle, password)
_client_cache[key] = client
targets["bluesky"] = Target(
id="bluesky",
uid=f"https://bsky.app/profile/{handle}",
name=config.get("BLUESKY_NAME", "Bluesky"),
client=client,
max_chars=int(config.get("BLUESKY_MAX_CHARS", 300)),
max_media=4,
)
token = config.get("THREADS_ACCESS_TOKEN") or ""
if token:
client = _client_cache.get("threads")
if client is None or client.config_token != token:
client = ThreadsClient(token, Path(config["DATA_PATH"]) / "threads_token.json",
api=config.get("THREADS_API") or THREADS_API)
_client_cache["threads"] = client
username = (config.get("THREADS_USERNAME") or "").lstrip("@")
targets["threads"] = Target(
id="threads",
uid=f"https://www.threads.com/@{username}" if username else "https://www.threads.com/",
name=config.get("THREADS_NAME", "Threads"),
client=client,
max_chars=int(config.get("THREADS_MAX_CHARS", 500)),
max_media=int(config.get("THREADS_MAX_MEDIA", 4)),
)
return targets
@@ -242,28 +878,88 @@ def _hashtag(display_name: str) -> str:
return f"#{word}" if word else ""
def compose_text(note, permalink: str, tags: List[str], max_chars: int) -> str:
"""Full note as Markdown, then the permalink and hashtags; shortened if too long"""
hashtags = " ".join(h for h in (_hashtag(t) for t in tags) if h)
def compose_text(note, permalink: str, tags: List[str], max_chars: int,
body: Optional[str] = None) -> str:
"""
Full note (Markdown unless `body` is given), then the permalink and
hashtags; shortened at a word boundary if too long. On short targets the
hashtags may use at most a third of the space.
"""
hashtags = ""
for h in (_hashtag(t) for t in tags):
if h and len(hashtags) + len(h) + 1 <= max(max_chars // 3, 1):
hashtags = f"{hashtags} {h}".strip()
tail = f"\n\n{permalink}" + (f"\n\n{hashtags}" if hashtags else "")
body = note.content.strip()
body = (note.content if body is None else body).strip()
if len(body) + len(tail) > max_chars:
room = max_chars - len(tail) - 1
body = body[:room].rsplit(" ", 1)[0].rstrip() + "…"
return body + tail
_BLOCKS = ["p", "h1", "h2", "h3", "h4", "h5", "h6", "pre", "blockquote", "ul", "ol", "hr", "table"]
def plain_text(markdown_text: str) -> str:
"""Markdown as readable plain text (for targets without Markdown)"""
import markdown
from bs4 import BeautifulSoup
soup = BeautifulSoup(markdown.markdown(markdown_text, extensions=["fenced_code"]), "html.parser")
for img in soup.find_all("img"):
img.decompose()
for node in soup.find_all(string=True):
if not node.strip() and "\n" in node and node.parent.name != "pre":
node.extract() # layout newlines between tags; we add our own
for br in soup.find_all("br"):
br.replace_with("\n")
for li in soup.find_all("li"):
li.insert(0, "• ")
li.append("\n")
for block in soup.find_all(_BLOCKS):
block.append("\n\n")
text = soup.get_text()
text = re.sub(r"[ \t]+\n", "\n", text)
return re.sub(r"\n{3,}", "\n\n", text).strip()
_URL_RE = re.compile(r"https?://[^\s<>\"]+")
_TAG_RE = re.compile(r"(?<![\w#])#(\w*[^\W\d]\w*)")
def bluesky_facets(text: str) -> List[dict]:
"""Link and hashtag facets (UTF-8 byte ranges) for a Bluesky post"""
def span(start: int, end: int) -> dict:
return {"byteStart": len(text[:start].encode()), "byteEnd": len(text[:end].encode())}
facets = []
for m in _URL_RE.finditer(text):
url = m.group(0).rstrip(".,;:!?)\"'…")
facets.append({"index": span(m.start(), m.start() + len(url)),
"features": [{"$type": "app.bsky.richtext.facet#link", "uri": url}]})
urls = [(f["index"]["byteStart"], f["index"]["byteEnd"]) for f in facets]
for m in _TAG_RE.finditer(text):
index = span(m.start(), m.end())
if any(a <= index["byteStart"] < b for a, b in urls):
continue # a #fragment inside a URL
facets.append({"index": index,
"features": [{"$type": "app.bsky.richtext.facet#tag", "tag": m.group(1)}]})
return facets
def _media_files(note_id: int, limit: int) -> List[dict]:
"""Image files for a note: the 'large' variant when there is one"""
from starpunk.media import get_note_media
media_dir = Path(current_app.config["DATA_PATH"]) / "media"
site = current_app.config["SITE_URL"].rstrip("/")
files = []
for media in get_note_media(note_id)[:limit]:
variant = (media.get("variants") or {}).get("large")
rel = variant["path"] if variant else media["path"]
files.append({
"path": media_dir / rel,
"url": f"{site}/media/{rel}", # for targets that fetch images themselves
"mime_type": media.get("mime_type") or "image/jpeg",
"description": media.get("caption") or "",
})
@@ -321,6 +1017,21 @@ def _fail(db, row, error: SyndicationError) -> None:
)
def _target_title(url: Optional[str]) -> Optional[str]:
"""Fetched title of a liked/reposted post, if we have one"""
if not url:
return None
try:
from starpunk.webmention_queries import get_reply_context
context = get_reply_context(url) or {}
except Exception:
return None
title = context.get("title") or ""
if not title and context.get("author_name"):
title = f"Post by {context['author_name']}"
return " ".join(title.split())[:200] or None
def _reconcile(row, target: Target) -> tuple:
"""Bring the remote post in line with the note. Returns (remote_id, remote_url)."""
from starpunk.notes import get_note
@@ -331,26 +1042,33 @@ def _reconcile(row, target: Target) -> tuple:
if not wanted:
if row["remote_id"]:
target.client.delete_status(row["remote_id"])
target.client.delete(row["remote_id"])
return None, None
if row["remote_id"] and not target.client.can_edit:
return row["remote_id"], row["remote_url"] # posted already; edits don't follow
site = current_app.config["SITE_URL"].rstrip("/")
tags = [t["display_name"] for t in get_note_tags(note.id)]
text = compose_text(note, f"{site}/note/{note.slug}", tags, target.max_chars)
media_ids = [
target.client.upload_media(f["path"], f["mime_type"], f["description"])
for f in _media_files(note.id, target.max_media)
]
post = Post(
note=note,
permalink=f"{site}/note/{note.slug}",
tags=[t["display_name"] for t in get_note_tags(note.id)],
max_chars=target.max_chars,
media=_media_files(note.id, target.max_media) if note.kind == "note" else [],
kind=note.kind,
target_url=note.reacts_to,
target_title=_target_title(note.reacts_to),
)
if row["remote_id"]:
try:
result = target.client.update_status(row["remote_id"], text, media_ids)
result = target.client.update(row["remote_id"], post)
return result["id"], result["url"]
except RemoteGone:
pass # deleted on the remote side; post it again
result = target.client.create_status(
text, media_ids, idempotency_key=f"starpunk-{row['id']}-{row['version']}"
result = target.client.create(
post, key=f"starpunk-{row['id']}-{row['version']}", retrying=row["attempts"] > 0
)
return result["id"], result["url"]
@@ -360,6 +1078,13 @@ def process_due(limit: int = 5) -> int:
targets = get_targets()
if not targets:
return 0
for target in targets.values():
maintain = getattr(target.client, "maintain", None)
if maintain:
try:
maintain()
except Exception as e:
current_app.logger.error(f"Syndication {target.id} upkeep failed: {e}")
db = get_db(current_app)
rows = db.execute(
"""
+96 -25
View File
@@ -1,36 +1,75 @@
/* StarPunk CSS - Minimal responsive stylesheet */
:root {
--color-text: #333; --color-text-light: #666; --color-bg: #fff; --color-bg-alt: #f5f5f5;
--color-link: #0066cc; --color-link-hover: #004499; --color-border: #ddd;
--color-text: #1f2328; --color-text-light: #59636e; --color-bg: #fff; --color-bg-alt: #f4f5f7;
--color-link: #0a5cc2; --color-link-hover: #083f85; --color-border: #e3e6ea; --color-accent-soft: #e8f0fb;
--color-success: #28a745; --color-error: #dc3545; --color-warning: #ffc107; --color-info: #17a2b8;
--font-body: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Arial, sans-serif;
--font-mono: 'SF Mono', Monaco, Consolas, 'Courier New', monospace;
--font-body: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
--font-reading: Charter, 'Bitstream Charter', 'Sitka Text', Cambria, 'Noto Serif', Georgia, serif;
--font-mono: ui-monospace, 'SF Mono', Monaco, Consolas, 'Liberation Mono', monospace;
--spacing-xs: 0.25rem; --spacing-sm: 0.5rem; --spacing-md: 1rem; --spacing-lg: 2rem; --spacing-xl: 4rem;
--max-width: 42rem; --border-radius: 4px;
--max-width: 40rem; --border-radius: 6px;
}
@media (prefers-color-scheme: dark) {
:root {
--color-text: #e4e6e9; --color-text-light: #9aa3ad; --color-bg: #121417; --color-bg-alt: #1c1f24;
--color-link: #74b0ff; --color-link-hover: #a8ceff; --color-border: #2b3038; --color-accent-soft: #1a2738;
}
}
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: var(--font-body); font-size: 1rem; line-height: 1.6; color: var(--color-text); background: var(--color-bg); padding: var(--spacing-md); }
body {
font-family: var(--font-body); font-size: 1rem; line-height: 1.6; color: var(--color-text); background: var(--color-bg);
padding: var(--spacing-md); min-height: 100vh; display: flex; flex-direction: column;
-webkit-font-smoothing: antialiased;
}
h1, h2, h3 { margin-bottom: var(--spacing-md); line-height: 1.2; font-weight: 600; }
h1 { font-size: 2rem; } h2 { font-size: 1.5rem; } h3 { font-size: 1.25rem; }
h1, h2, h3 { margin-bottom: var(--spacing-md); line-height: 1.25; font-weight: 650; letter-spacing: -0.01em; }
h1 { font-size: 1.75rem; } h2 { font-size: 1.375rem; } h3 { font-size: 1.125rem; }
p { margin-bottom: var(--spacing-md); }
a { color: var(--color-link); text-decoration: none; }
a:hover { color: var(--color-link-hover); text-decoration: underline; }
:focus-visible { outline: 2px solid var(--color-link); outline-offset: 2px; border-radius: 2px; }
code, pre { font-family: var(--font-mono); background: var(--color-bg-alt); padding: 0.125rem 0.25rem; border-radius: var(--border-radius); font-size: 0.875rem; }
code, pre { font-family: var(--font-mono); background: var(--color-bg-alt); padding: 0.125rem 0.3rem; border-radius: var(--border-radius); font-size: 0.875em; }
pre { padding: var(--spacing-md); overflow-x: auto; margin-bottom: var(--spacing-md); }
pre code { padding: 0; background: none; }
header, main, footer { max-width: var(--max-width); margin: 0 auto; }
header { margin-bottom: var(--spacing-lg); padding-bottom: var(--spacing-md); border-bottom: 2px solid var(--color-border); }
header h1 { margin-bottom: var(--spacing-sm); }
header h1 a { color: var(--color-text); text-decoration: none; }
nav { display: flex; gap: var(--spacing-md); flex-wrap: wrap; }
nav a { padding: var(--spacing-sm) var(--spacing-md); border-radius: var(--border-radius); transition: background 0.2s; }
nav a:hover { background: var(--color-bg-alt); text-decoration: none; }
main { min-height: 60vh; margin-bottom: var(--spacing-lg); }
footer { padding-top: var(--spacing-lg); border-top: 1px solid var(--color-border); text-align: center; color: var(--color-text-light); font-size: 0.875rem; }
.visually-hidden { position: absolute !important; width: 1px; height: 1px; overflow: hidden; clip: rect(0 0 0 0); clip-path: inset(50%); white-space: nowrap; }
/* Page frame */
.site-header, main, .site-footer { width: 100%; max-width: var(--max-width); margin-left: auto; margin-right: auto; }
main { flex: 1; margin-bottom: var(--spacing-xl); }
.site-header {
display: flex; flex-wrap: wrap; align-items: baseline; justify-content: space-between; gap: var(--spacing-sm) var(--spacing-lg);
padding: var(--spacing-md) 0 var(--spacing-md); margin-bottom: var(--spacing-lg); border-bottom: 1px solid var(--color-border);
}
.site-title { font-size: 1.375rem; font-weight: 700; letter-spacing: -0.02em; color: var(--color-text); }
.site-title:hover { color: var(--color-link); text-decoration: none; }
.site-nav { display: flex; align-items: center; flex-wrap: wrap; gap: var(--spacing-md); font-size: 0.9375rem; }
.site-nav > a { color: var(--color-text-light); }
.site-nav > a:hover { color: var(--color-link); text-decoration: none; }
.site-search { display: flex; }
.site-search input[type="search"] {
width: 10rem; padding: 0.3rem 0.6rem; font: inherit; font-size: 0.875rem; color: var(--color-text);
background: var(--color-bg-alt); border: 1px solid var(--color-border); border-right: 0; border-radius: var(--border-radius) 0 0 var(--border-radius);
}
.site-search button {
padding: 0.3rem 0.6rem; font: inherit; font-size: 0.875rem; cursor: pointer; color: var(--color-text-light);
background: var(--color-bg-alt); border: 1px solid var(--color-border); border-radius: 0 var(--border-radius) var(--border-radius) 0;
}
.site-search button:hover { color: var(--color-link); }
.site-footer { padding: var(--spacing-md) 0; border-top: 1px solid var(--color-border); color: var(--color-text-light); font-size: 0.8125rem; }
.site-footer p { margin: 0; }
.site-footer a { color: var(--color-text-light); text-decoration: underline; text-decoration-color: var(--color-border); }
.site-footer a:hover { color: var(--color-link); }
.page-title { font-size: 1.375rem; margin-bottom: var(--spacing-lg); }
.tag-name { color: var(--color-link); }
.tag-name::before { content: "#"; }
.dev-mode-warning { background: var(--color-error); color: white; padding: var(--spacing-md); text-align: center; font-weight: bold; margin: calc(-1 * var(--spacing-md)); margin-bottom: var(--spacing-lg); }
@@ -40,17 +79,46 @@ footer { padding-top: var(--spacing-lg); border-top: 1px solid var(--color-borde
.flash-warning { background: #fff3cd; border-color: var(--color-warning); color: #856404; }
.flash-info { background: #d1ecf1; border-color: var(--color-info); color: #0c5460; }
/* Notes */
.h-entry { margin-bottom: var(--spacing-lg); padding-bottom: var(--spacing-lg); border-bottom: 1px solid var(--color-border); }
.h-entry:last-child { border-bottom: none; }
.note-preview .e-content { margin-bottom: var(--spacing-md); }
.note-meta { color: var(--color-text-light); font-size: 0.875rem; }
.h-entry > .p-name a { color: var(--color-text); }
.h-entry > .p-name a:hover { color: var(--color-link); }
.e-content, .p-summary { font-family: var(--font-reading); font-size: 1.125rem; line-height: 1.65; overflow-wrap: anywhere; }
.e-content > :last-child, .p-summary > :last-child { margin-bottom: 0; }
.e-content a { text-decoration: underline; text-decoration-thickness: 1px; text-underline-offset: 2px; text-decoration-color: color-mix(in srgb, currentColor 40%, transparent); }
.e-content a:hover { text-decoration-color: currentColor; }
.e-content blockquote { margin: 0 0 var(--spacing-md); padding-left: var(--spacing-md); border-left: 3px solid var(--color-border); color: var(--color-text-light); }
.e-content ul, .e-content ol { margin: 0 0 var(--spacing-md) 1.25rem; }
.e-content img { max-width: 100%; height: auto; }
.e-content h1, .e-content h2, .e-content h3 { font-family: var(--font-body); }
.reaction { margin-bottom: var(--spacing-sm); font-size: 0.9375rem; color: var(--color-text-light); overflow-wrap: anywhere; }
.reaction-verb { font-weight: 600; margin-right: 0.25rem; }
.reaction-like .reaction-verb { color: #c2410c; }
.reaction-repost .reaction-verb { color: #15803d; }
.reaction a { color: var(--color-text); font-weight: 500; }
.reaction a:hover { color: var(--color-link); }
.reaction-by { margin-left: 0.25rem; }
@media (prefers-color-scheme: dark) {
.reaction-like .reaction-verb { color: #fb923c; }
.reaction-repost .reaction-verb { color: #4ade80; }
}
.read-more { margin: var(--spacing-sm) 0 0; font-size: 0.9375rem; }
.note-meta { display: flex; flex-wrap: wrap; align-items: baseline; gap: 0 0.4rem; margin-top: var(--spacing-sm); color: var(--color-text-light); font-size: 0.8125rem; }
.note-meta a { color: var(--color-text-light); }
.note-nav { margin-top: var(--spacing-md); }
.note-meta a:hover { color: var(--color-link); }
.note-meta > :not(.visually-hidden) + :not(.visually-hidden)::before { content: "·"; margin-right: 0.4rem; color: var(--color-text-light); }
.note-tags { display: inline-flex; flex-wrap: wrap; gap: 0 0.4rem; }
.note-meta > .note-tags::before { margin-right: 0; }
.note-tags .p-category::before { content: "#"; opacity: 0.7; }
.h-entry:not(.note-preview) .note-meta { margin-top: var(--spacing-md); }
.note-nav { margin-top: var(--spacing-lg); font-size: 0.9375rem; }
.empty-state { text-align: center; padding: var(--spacing-xl); color: var(--color-text-light); }
.form-group { margin-bottom: var(--spacing-md); }
label { display: block; margin-bottom: var(--spacing-sm); font-weight: 600; }
input[type="text"], input[type="url"], input[type="email"], textarea { width: 100%; padding: var(--spacing-sm); border: 1px solid var(--color-border); border-radius: var(--border-radius); font-family: inherit; font-size: 1rem; }
input[type="text"], input[type="url"], input[type="email"], input[type="search"], textarea { color: var(--color-text); background: var(--color-bg); width: 100%; padding: var(--spacing-sm); border: 1px solid var(--color-border); border-radius: var(--border-radius); font-family: inherit; font-size: 1rem; }
textarea { font-family: var(--font-mono); resize: vertical; min-height: 10rem; }
small { display: block; margin-top: var(--spacing-xs); color: var(--color-text-light); font-size: 0.875rem; }
.form-checkbox { display: flex; align-items: center; gap: var(--spacing-sm); }
@@ -171,11 +239,14 @@ small { display: block; margin-top: var(--spacing-xs); color: var(--color-text-l
}
@media (min-width: 768px) {
body { padding: var(--spacing-lg); }
h1 { font-size: 2.5rem; } h2 { font-size: 2rem; } h3 { font-size: 1.5rem; }
body { padding: var(--spacing-md) var(--spacing-lg); }
h1 { font-size: 2rem; } h2 { font-size: 1.5rem; } h3 { font-size: 1.25rem; }
}
@media (max-width: 767px) {
.site-nav { width: 100%; }
.site-search { flex: 1; justify-content: flex-end; }
.site-search input[type="search"] { width: 100%; max-width: 12rem; }
.note-table { font-size: 0.875rem; }
.note-table th, .note-table td { padding: var(--spacing-sm); }
.note-actions .button { font-size: 0.75rem; padding: 0.25rem 0.5rem; }
@@ -426,7 +497,7 @@ small { display: block; margin-top: var(--spacing-xs); color: var(--color-text-l
.content-preview {
padding: var(--spacing-sm);
background: white;
background: var(--color-bg);
border: 1px solid var(--color-border);
border-radius: 4px;
margin-top: var(--spacing-xs);
+6 -3
View File
@@ -12,12 +12,15 @@
<form action="{{ url_for('admin.update_note_submit', note_id=note.id) }}" method="POST" class="note-form">
<div class="form-group">
<label for="content">Content (Markdown)</label>
{% if note.reacts_to %}
<p class="note-meta">{{ '★ Like of' if note.kind == 'like' else '🔁 Repost of' }} <a href="{{ note.reacts_to }}">{{ note.reacts_to }}</a></p>
{% endif %}
<label for="content">{% if note.reacts_to %}Comment (optional, Markdown){% else %}Content (Markdown){% endif %}</label>
<textarea
id="content"
name="content"
rows="20"
required
rows="{{ 6 if note.reacts_to else 20 }}"
{% if not note.reacts_to %}required{% endif %}
autofocus
>{{ note.content }}</textarea>
<small>Use Markdown syntax for formatting</small>
+16 -21
View File
@@ -3,7 +3,8 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{% block title %}StarPunk{% endblock %}</title>
<title>{% block title %}{{ config.SITE_NAME }}{% endblock %}</title>
<meta name="color-scheme" content="light dark">
<link rel="stylesheet" href="{{ url_for('static', filename='css/style.css') }}">
<link rel="alternate" type="application/rss+xml" title="{{ config.SITE_NAME }} RSS Feed" href="{{ absolute_url_for('public.feed') }}">
<link rel="alternate" type="application/xml+opml" title="{{ config.SITE_NAME }} Feed Subscription List" href="{{ absolute_url_for('public.opml') }}">
@@ -33,29 +34,21 @@
</div>
{% endif %}
<header>
<h1><a href="/">StarPunk</a></h1>
<nav>
<header class="site-header">
<a class="site-title" href="/">{{ config.SITE_NAME }}</a>
<nav class="site-nav">
<a href="/">Home</a>
<a href="{{ url_for('public.likes') }}">Likes</a>
<a href="{{ url_for('public.feed') }}">RSS</a>
{% if g.me %}
<a href="{{ url_for('admin.dashboard') }}">Dashboard</a>
<a href="{{ url_for('admin.dashboard') }}">Admin</a>
{% else %}
<a href="{{ url_for('auth.login_form') }}">Login</a>
<a href="{{ url_for('auth.login_form') }}">Log in</a>
{% endif %}
<form action="/search" method="get" role="search" style="margin-left: auto; display: flex; gap: var(--spacing-sm);">
<input
type="search"
name="q"
placeholder="Search..."
aria-label="Search"
value="{{ request.args.get('q', '') }}"
minlength="2"
maxlength="100"
required
style="width: 200px; padding: var(--spacing-xs) var(--spacing-sm);"
>
<button type="submit" class="button button-small" style="padding: var(--spacing-xs) var(--spacing-sm);">🔍</button>
<form class="site-search" action="/search" method="get" role="search">
<input type="search" name="q" placeholder="Search" aria-label="Search"
value="{{ request.args.get('q', '') }}" minlength="2" maxlength="100" required>
<button type="submit" aria-label="Search">Search</button>
</form>
</nav>
</header>
@@ -72,8 +65,10 @@
{% block content %}{% endblock %}
</main>
<footer>
<p>StarPunk v{{ config.get('VERSION', '0.5.0') }}</p>
<footer class="site-footer">
<p>
Powered by <a href="https://git.thesatelliteoflove.com/phil/StarPunk">StarPunk</a> v{{ config.get('VERSION', '0.5.0') }}
</p>
</footer>
</body>
</html>
+4 -47
View File
@@ -1,12 +1,12 @@
{% extends "base.html" %}
{% from "partials/media.html" import display_media %}
{% from "partials/note_preview.html" import note_preview %}
{% block title %}StarPunk - Home{% endblock %}
{% block title %}{{ config.SITE_NAME }}{% endblock %}
{% block content %}
<div class="h-feed">
{# h-feed required properties per microformats.org/wiki/h-feed #}
<h2 class="p-name">{{ config.SITE_NAME or 'Recent Notes' }}</h2>
<h1 class="p-name visually-hidden">{{ config.SITE_NAME or 'Recent Notes' }}</h1>
{# u-url for feed (self-reference) #}
<a class="u-url" href="{{ absolute_url_for('public.index') }}" style="display: none;">Feed URL</a>
@@ -34,50 +34,7 @@
{% if notes %}
{% for note in notes %}
<article class="h-entry note-preview">
{# Detect if note has explicit title (starts with # heading) - per Q22 #}
{% set has_explicit_title = note.content.strip().startswith('#') %}
{# p-name only if note has explicit title (per Q22) #}
{% if has_explicit_title %}
<h3 class="p-name">{{ note.title }}</h3>
{% endif %}
{# Media preview (if available) #}
{{ display_media(note.media) }}
{# e-content: note content (preview) #}
<div class="e-content">
{{ note.html[:300]|safe }}{% if note.html|length > 300 %}...{% endif %}
</div>
<footer class="note-meta">
{# u-url for permalink #}
<a class="u-url" href="{{ absolute_url_for('public.note', slug=note.slug) }}">
<time class="dt-published" datetime="{{ note.created_at.isoformat() }}">
{{ note.created_at.strftime('%B %d, %Y') }}
</time>
</a>
{# Tags in preview #}
{% if note.tags %}
<span class="note-tags">
{% for tag in note.tags %}
<a class="p-category" rel="tag" href="{{ url_for('public.tag', tag=tag.name) }}">{{ tag.display_name }}</a>
{% endfor %}
</span>
{% endif %}
{# Author h-card nested in each h-entry (per Q20) #}
{% if author %}
<div class="p-author h-card">
<a class="p-name u-url" href="{{ author.url or author.me }}">
{{ author.name or author.url or author.me }}
</a>
</div>
{% endif %}
</footer>
</article>
{{ note_preview(note, author) }}
{% endfor %}
{% else %}
<p class="empty-state">No notes published yet. Check back soon!</p>
+22
View File
@@ -0,0 +1,22 @@
{% extends "base.html" %}
{% from "partials/note_preview.html" import note_preview %}
{% block title %}Likes · {{ config.SITE_NAME }}{% endblock %}
{% block content %}
<div class="h-feed">
<h1 class="p-name page-title">Likes</h1>
{% if notes %}
{% for note in notes %}
{{ note_preview(note, author) }}
{% endfor %}
{% else %}
<p class="empty-state">Nothing liked yet.</p>
{% endif %}
<nav class="note-nav">
<a href="/">← All notes</a>
</nav>
</div>
{% endblock %}
+22 -39
View File
@@ -1,7 +1,8 @@
{% extends "base.html" %}
{% from "partials/media.html" import display_media %}
{% from "partials/reaction.html" import reaction %}
{% block title %}{{ note.slug }} - StarPunk{% endblock %}
{% block title %}{% if note.content.strip().startswith('#') %}{{ note.title }}{% else %}{{ note.excerpt|truncate(60) }}{% endif %} · {{ config.SITE_NAME }}{% endblock %}
{% block content %}
<article class="h-entry">
@@ -38,63 +39,45 @@
</div>
{% endif %}
{{ reaction(note) }}
{# u-photo placement: Per draft spec, u-photo must be direct child of h-entry, #}
{# NOT inside e-content. Media is rendered ABOVE e-content to meet this requirement. #}
{{ display_media(note.media) }}
{# e-content: note content BELOW media (per ADR-057) #}
<div class="e-content">
{{ note.html|safe }}
{{ note.body_html|safe }}
</div>
<footer class="note-meta">
{# u-url and u-uid same for notes (per Q23) #}
{% for s in syndications if s.remote_url %}
<a class="u-syndication" href="{{ s.remote_url }}" rel="syndication">Also on {{ s.name }}</a>
{% endfor %}
<a class="u-url u-uid" href="{{ absolute_url_for('public.note', slug=note.slug) }}">
<time class="dt-published" datetime="{{ note.created_at.isoformat() }}">
{{ note.created_at.strftime('%B %d, %Y at %I:%M %p') }}
</time>
</a>
{# dt-updated if note was modified #}
<a class="u-url u-uid" href="{{ absolute_url_for('public.note', slug=note.slug) }}"><time class="dt-published" datetime="{{ note.created_at.isoformat() }}">{{ note.created_at.strftime('%B %-d, %Y · %-I:%M %p') }}</time></a>
{% if note.updated_at and note.updated_at != note.created_at %}
<span class="updated">
(Updated: <time class="dt-updated" datetime="{{ note.updated_at.isoformat() }}">{{ note.updated_at.strftime('%B %d, %Y') }}</time>)
<span class="updated">Updated <time class="dt-updated" datetime="{{ note.updated_at.isoformat() }}">{{ note.updated_at.strftime('%b %-d, %Y') }}</time></span>
{% endif %}
{# rel="tag" per microformats2 p-category #}
{% if note.tags %}
<span class="note-tags">
{% for tag in note.tags %}<a class="p-category" rel="tag" href="{{ url_for('public.tag', tag=tag.name) }}">{{ tag.display_name }}</a>{% endfor %}
</span>
{% endif %}
{# Tags / Categories #}
{# rel="tag" per microformats2 p-category specification #}
{% if note.tags %}
<div class="note-tags">
{% for tag in note.tags %}
<a class="p-category" rel="tag" href="{{ url_for('public.tag', tag=tag.name) }}">
{{ tag.display_name }}
</a>
{% endfor %}
</div>
{% set synced = (syndications or [])|selectattr('remote_url')|list %}
{% if synced %}
<span class="syndication">Also on {% for s in synced %}<a class="u-syndication" href="{{ s.remote_url }}" rel="syndication">{{ s.name }}</a>{% if not loop.last %}, {% endif %}{% endfor %}</span>
{% endif %}
{# Author h-card (nested within h-entry per Q20) #}
{# Author h-card (nested within h-entry per Q20); hidden on a single-author site #}
{% if author %}
<div class="p-author h-card">
{% if author.photo %}
<img class="u-photo" src="{{ author.photo }}" alt="{{ author.name or 'Author' }}" width="48" height="48">
{% endif %}
<a class="p-name u-url" href="{{ author.url or author.me }}">
{{ author.name or author.url or author.me }}
</a>
{% if author.note %}
<span class="p-note" style="display: none;">{{ author.note }}</span>
{% endif %}
</div>
<span class="p-author h-card visually-hidden">
{% if author.photo %}<img class="u-photo" src="{{ author.photo }}" alt="">{% endif %}
<a class="p-name u-url" href="{{ author.url or author.me }}">{{ author.name or author.url or author.me }}</a>
{% if author.note %}<span class="p-note">{{ author.note }}</span>{% endif %}
</span>
{% endif %}
</footer>
<nav class="note-nav">
<a href="/">Back to all notes</a>
<a href="/">← All notes</a>
</nav>
{# Webmentions section (v1.6.0 Phase 5) - only shown if approved webmentions exist #}
+41
View File
@@ -0,0 +1,41 @@
{# One note in a list (home, tag pages). v2.3.0 #}
{% from "partials/media.html" import display_media %}
{% from "partials/reaction.html" import reaction %}
{% macro note_preview(note, author) %}
<article class="h-entry note-preview">
{# Notes starting with a # heading have a title (Q22) #}
{% set has_explicit_title = note.content.strip().startswith('#') %}
{% if has_explicit_title %}
<h2 class="p-name"><a href="{{ absolute_url_for('public.note', slug=note.slug) }}">{{ note.title }}</a></h2>
{% endif %}
{{ reaction(note) }}
{{ display_media(note.media) }}
{% if has_explicit_title %}
{# Titled notes are articles: show a summary and link through #}
<div class="p-summary">
<p>{{ note.excerpt }}</p>
</div>
<p class="read-more"><a href="{{ absolute_url_for('public.note', slug=note.slug) }}">Read more →</a></p>
{% elif note.html %}
<div class="e-content">
{{ note.html|safe }}
</div>
{% endif %}
<footer class="note-meta">
<a class="u-url" href="{{ absolute_url_for('public.note', slug=note.slug) }}"><time class="dt-published" datetime="{{ note.created_at.isoformat() }}">{{ note.created_at.strftime('%b %-d, %Y') }}</time></a>
{% if note.tags %}
<span class="note-tags">
{% for tag in note.tags %}<a class="p-category" rel="tag" href="{{ url_for('public.tag', tag=tag.name) }}">{{ tag.display_name }}</a>{% endfor %}
</span>
{% endif %}
{% if author %}
{# Single-author site: keep the h-card for parsers, hide it from readers #}
<span class="p-author h-card visually-hidden"><a class="p-name u-url" href="{{ author.url or author.me }}">{{ author.name or author.url or author.me }}</a></span>
{% endif %}
</footer>
</article>
{% endmacro %}
+12
View File
@@ -0,0 +1,12 @@
{# "Liked" / "Reposted" line for likes and reposts (v2.5.0) #}
{% macro reaction(note) %}
{% if note.reacts_to %}
{% set ctx = note.context if note.context is defined else None %}
{% if ctx and ctx.title %}{% set label = ctx.title %}{% elif ctx and ctx.content_snippet %}{% set label = ctx.content_snippet|striptags|truncate(90) %}{% elif ctx and (ctx.author_name or ctx.site_name) %}{% set label = 'a post' %}{% else %}{% set label = note.reacts_to %}{% endif %}
<p class="reaction reaction-{{ note.kind }}">
<span class="reaction-verb">{% if note.kind == 'like' %}★ Liked{% else %}🔁 Reposted{% endif %}</span>
<a class="{{ 'u-like-of' if note.kind == 'like' else 'u-repost-of' }}" href="{{ note.reacts_to }}">{{ label }}</a>
{% if ctx and (ctx.author_name or ctx.site_name) %}<span class="reaction-by">by {{ ctx.author_name or ctx.site_name }}</span>{% endif %}
</p>
{% endif %}
{% endmacro %}
+1 -1
View File
@@ -1,6 +1,6 @@
{% extends "base.html" %}
{% block title %}{% if query %}Search: {{ query }}{% else %}Search{% endif %} - StarPunk{% endblock %}
{% block title %}{% if query %}Search: {{ query }}{% else %}Search{% endif %} · {{ config.SITE_NAME }}{% endblock %}
{% block content %}
<div class="search-container">
+5 -48
View File
@@ -1,65 +1,22 @@
{% extends "base.html" %}
{% from "partials/media.html" import display_media %}
{% from "partials/note_preview.html" import note_preview %}
{% block title %}{{ tag.display_name }} - StarPunk{% endblock %}
{% block title %}{{ tag.display_name }} · {{ config.SITE_NAME }}{% endblock %}
{% block content %}
<div class="h-feed">
<h1 class="p-name">Notes tagged "{{ tag.display_name }}"</h1>
<h1 class="p-name page-title">Notes tagged <span class="tag-name">{{ tag.display_name }}</span></h1>
{% if notes %}
{% for note in notes %}
<article class="h-entry note-preview">
{# Detect if note has explicit title (starts with # heading) - per Q22 #}
{% set has_explicit_title = note.content.strip().startswith('#') %}
{# p-name only if note has explicit title (per Q22) #}
{% if has_explicit_title %}
<h3 class="p-name">{{ note.title }}</h3>
{% endif %}
{# Media preview (if available) #}
{{ display_media(note.media) }}
{# e-content: note content (preview) #}
<div class="e-content">
{{ note.html[:300]|safe }}{% if note.html|length > 300 %}...{% endif %}
</div>
<footer class="note-meta">
{# u-url for permalink #}
<a class="u-url" href="{{ absolute_url_for('public.note', slug=note.slug) }}">
<time class="dt-published" datetime="{{ note.created_at.isoformat() }}">
{{ note.created_at.strftime('%B %d, %Y') }}
</time>
</a>
{# Tags in preview #}
{% if note.tags %}
<span class="note-tags">
{% for tag in note.tags %}
<a class="p-category" rel="tag" href="{{ url_for('public.tag', tag=tag.name) }}">{{ tag.display_name }}</a>
{% endfor %}
</span>
{% endif %}
{# Author h-card nested in each h-entry (per Q20) #}
{% if author %}
<div class="p-author h-card">
<a class="p-name u-url" href="{{ author.url or author.me }}">
{{ author.name or author.url or author.me }}
</a>
</div>
{% endif %}
</footer>
</article>
{{ note_preview(note, author) }}
{% endfor %}
{% else %}
<p class="empty-state">No notes with this tag.</p>
{% endif %}
<nav class="note-nav">
<a href="/">Back to all notes</a>
<a href="/">← All notes</a>
</nav>
</div>
{% endblock %}
+223
View File
@@ -0,0 +1,223 @@
"""
Likes and reposts (v2.5.0, ADR-068)
Fake GoToSocial and Bluesky servers check that likes/reposts of their own
posts become native favourites/boosts/likes/reposts, and everything else
falls back (reposts) or is skipped (likes).
"""
import json
import httpx
import pytest
from starpunk import create_app, syndication
from starpunk.indieauth import create_access_token
from starpunk.notes import delete_note, get_note
from starpunk.syndication import BlueskyClient, MastodonClient, get_note_syndications, process_due
SITE = "https://blog.example.com"
GTS = "https://social.example.com"
FEDI_POST = "https://mastodon.example/@alice/111"
BSKY_POST = "https://bsky.app/profile/alice.example/post/3abc"
WEB_PAGE = "https://someblog.example/2026/a-post"
class FakeGTS:
def __init__(self):
self.calls = []
self.statuses = {}
def __call__(self, request):
path = request.url.path
self.calls.append((request.method, path))
if path == "/api/v2/search":
q = request.url.params["q"]
return httpx.Response(200, json={"statuses": [{"id": "s1"}] if q == FEDI_POST else []})
if path in ("/api/v1/statuses/s1/favourite", "/api/v1/statuses/s1/unfavourite",
"/api/v1/statuses/s1/reblog", "/api/v1/statuses/s1/unreblog"):
return httpx.Response(200, json={"id": "s1"})
if path == "/api/v1/statuses" and request.method == "POST":
sid = f"p{len(self.statuses) + 1}"
self.statuses[sid] = json.loads(request.content)
return httpx.Response(200, json={"id": sid, "url": f"{GTS}/@phil/statuses/{sid}"})
if path.startswith("/api/v1/statuses/") and request.method == "DELETE":
self.statuses.pop(path.rsplit("/", 1)[1], None)
return httpx.Response(200, json={})
return httpx.Response(500)
class FakePDS:
def __init__(self):
self.records = {}
self.calls = []
def __call__(self, request):
nsid = request.url.path.removeprefix("/xrpc/")
self.calls.append(nsid)
if nsid == "com.atproto.server.createSession":
return httpx.Response(200, json={"did": "did:plc:me", "handle": "me.example",
"accessJwt": "a", "refreshJwt": "r"})
if nsid == "com.atproto.identity.resolveHandle":
return httpx.Response(200, json={"did": "did:plc:alice"})
if nsid == "app.bsky.feed.getPosts":
return httpx.Response(200, json={"posts": [{"uri": request.url.params["uris"], "cid": "cid1"}]})
if nsid == "com.atproto.repo.createRecord":
body = json.loads(request.content)
uri = f"at://did:plc:me/{body['collection']}/r{len(self.records) + 1}"
self.records[uri] = body["record"]
return httpx.Response(200, json={"uri": uri, "cid": "c"})
if nsid == "com.atproto.repo.deleteRecord":
body = json.loads(request.content)
self.records.pop(f"at://{body['repo']}/{body['collection']}/{body['rkey']}", None)
return httpx.Response(200, json={})
return httpx.Response(500, json={"error": nsid})
@pytest.fixture
def gts():
return FakeGTS()
@pytest.fixture
def pds():
return FakePDS()
@pytest.fixture
def app(tmp_path, gts, pds):
data = tmp_path / "data"
app = create_app({
"TESTING": True, "SITE_URL": SITE, "ADMIN_ME": "https://example.com", "SESSION_SECRET": "x",
"DATA_PATH": data, "NOTES_PATH": data / "notes", "DATABASE_PATH": data / "starpunk.db",
"GOTOSOCIAL_URL": GTS, "GOTOSOCIAL_ACCESS_TOKEN": "t",
"BLUESKY_HANDLE": "me.example", "BLUESKY_APP_PASSWORD": "p", "BLUESKY_SERVICE": "https://pds.example",
"FEED_CACHE_ENABLED": False,
})
# The feed's note list is cached at module level across apps
from starpunk.routes import public
public._feed_cache["notes"] = None
public._feed_cache["timestamp"] = None
syndication._client_cache[GTS] = MastodonClient(GTS, "t", http=httpx.Client(transport=httpx.MockTransport(gts)))
syndication._client_cache["bluesky:https://pds.example:me.example"] = BlueskyClient(
"https://pds.example", "me.example", "p", http=httpx.Client(transport=httpx.MockTransport(pds)))
yield app
syndication._client_cache.clear()
@pytest.fixture
def post(app, monkeypatch):
# Don't fetch contexts from the network
monkeypatch.setattr("starpunk.reply_context.fetch_reply_context", lambda *a, **k: None)
with app.test_request_context():
token = create_access_token("https://quill.example/", "create update delete")["access_token"]
client = app.test_client()
def _post(**props):
response = client.post("/micropub", json={"type": ["h-entry"], "properties": props},
headers={"Authorization": f"Bearer {token}"})
assert response.status_code == 201, response.get_json()
location = response.headers["Location"]
with app.app_context():
note = get_note(slug=location.rsplit("/", 1)[1])
process_due(limit=10)
rows = {r["target"]: r for r in get_note_syndications(note.id)}
return note, rows
return _post
def page(app, path):
return app.test_client().get(path).data.decode()
class TestSite:
def test_like_without_comment(self, app, post):
note, _ = post(**{"like-of": [WEB_PAGE]})
assert note.kind == "like" and note.content == ""
assert note.slug not in page(app, "/")
likes = page(app, "/likes")
assert f'class="u-like-of" href="{WEB_PAGE}"' in likes and "★ Liked" in likes
assert note.slug not in page(app, "/feed.json")
def test_repost_in_stream_and_feed(self, app, post):
note, _ = post(**{"repost-of": [WEB_PAGE], "content": ["Worth a read"]})
home = page(app, "/")
assert f'class="u-repost-of" href="{WEB_PAGE}"' in home and "Worth a read" in home
feed = json.loads(page(app, "/feed.json"))
item = next(i for i in feed["items"] if note.slug in i["url"])
assert "Reposted" in item["content_html"] and WEB_PAGE in item["content_html"]
def test_webmention_goes_to_target_with_clean_url(self, app, post, monkeypatch):
sent = {}
monkeypatch.setattr("starpunk.webmention_sender.queue_webmentions_for_note",
lambda **kw: sent.update(kw) or 0)
app.config["WEBMENTION_ENABLED"] = True
note, _ = post(**{"like-of": [WEB_PAGE]})
assert sent["in_reply_to"] == WEB_PAGE
assert sent["note_url"] == f"{SITE}/note/{note.slug}" # no double slash
def test_both_kinds_rejected(self, app, post):
with pytest.raises(AssertionError):
post(**{"like-of": [WEB_PAGE], "repost-of": [WEB_PAGE]})
class TestGoToSocial:
def test_like_of_fediverse_post_favourites(self, app, post, gts):
note, rows = post(**{"like-of": [FEDI_POST]})
assert rows["gotosocial"]["remote_id"] == "favourite:s1"
assert rows["gotosocial"]["remote_url"] is None
with app.test_request_context():
delete_note(id=note.id)
process_due(limit=10)
assert ("POST", "/api/v1/statuses/s1/unfavourite") in gts.calls
def test_like_of_web_page_is_not_crossposted(self, app, post, gts):
_, rows = post(**{"like-of": [WEB_PAGE]})
assert rows["gotosocial"]["state"] == "synced" and rows["gotosocial"]["remote_id"] is None
assert gts.statuses == {}
def test_repost_boosts(self, app, post, gts):
_, rows = post(**{"repost-of": [FEDI_POST]})
assert rows["gotosocial"]["remote_id"] == "reblog:s1"
def test_repost_with_comment_posts_status(self, app, post, gts):
_, rows = post(**{"repost-of": [FEDI_POST], "content": ["So true"]})
(status,) = gts.statuses.values()
assert status["status"].startswith("So true") and FEDI_POST in status["status"]
assert ("POST", "/api/v1/statuses/s1/reblog") not in gts.calls
def test_repost_of_web_page_posts_link(self, app, post, gts):
post(**{"repost-of": [WEB_PAGE]})
(status,) = gts.statuses.values()
assert status["status"].startswith("🔁") and WEB_PAGE in status["status"]
class TestBluesky:
def test_like_of_bluesky_post(self, app, post, pds):
note, rows = post(**{"like-of": [BSKY_POST]})
uri = rows["bluesky"]["remote_id"]
assert "/app.bsky.feed.like/" in uri
assert pds.records[uri]["subject"] == {"uri": "at://did:plc:alice/app.bsky.feed.post/3abc", "cid": "cid1"}
with app.test_request_context():
delete_note(id=note.id)
process_due(limit=10)
assert uri not in pds.records
def test_like_of_web_page_skipped(self, app, post, pds):
_, rows = post(**{"like-of": [WEB_PAGE]})
assert rows["bluesky"]["remote_id"] is None and pds.records == {}
def test_repost_and_quote(self, app, post, pds):
_, rows = post(**{"repost-of": [BSKY_POST]})
assert "/app.bsky.feed.repost/" in rows["bluesky"]["remote_id"]
_, rows = post(**{"repost-of": [BSKY_POST], "content": ["Great thread"]})
record = pds.records[rows["bluesky"]["remote_id"]]
assert record["text"] == "Great thread"
assert record["embed"] == {"$type": "app.bsky.embed.record",
"record": {"uri": "at://did:plc:alice/app.bsky.feed.post/3abc", "cid": "cid1"}}
def test_repost_of_web_page_posts_link_card(self, app, post, pds):
_, rows = post(**{"repost-of": [WEB_PAGE]})
record = pds.records[rows["bluesky"]["remote_id"]]
assert record["embed"]["external"]["uri"] == WEB_PAGE
assert record["text"].startswith("🔁")
+75 -27
View File
@@ -167,6 +167,34 @@ def test_micropub_create_with_name(client, app, mock_valid_token):
assert response.status_code == 201
assert 'Location' in response.headers
# StarPunk titles are a '# Heading' first line, so name must land there
slug = response.headers['Location'].rstrip('/').split('/')[-1]
with app.app_context():
note = get_note(slug)
assert note.content == '# My Test Title\n\nContent goes here'
assert note.title == 'My Test Title'
def test_micropub_create_with_name_already_heading(client, app, mock_valid_token):
"""A name that matches the content's heading is not added twice"""
with patch('starpunk.routes.micropub.verify_access_token', mock_valid_token):
response = client.post(
'/micropub',
json={
'type': ['h-entry'],
'properties': {
'name': ['Same Title'],
'content': ['# Same Title\n\nBody']
}
},
headers={'Authorization': 'Bearer valid_token'}
)
assert response.status_code == 201
slug = response.headers['Location'].rstrip('/').split('/')[-1]
with app.app_context():
assert get_note(slug).content == '# Same Title\n\nBody'
def test_micropub_create_with_categories(client, app, mock_valid_token):
"""Test creating a note with tags (category property)"""
@@ -290,6 +318,38 @@ def test_micropub_query_source(client, mock_valid_token):
data = response.get_json()
assert 'properties' in data
assert data['properties']['content'][0] == 'Source test'
# url must be the real permalink (was /notes/<slug>, a 404)
assert data['properties']['url'] == [location]
# A note without a heading has no name
assert 'name' not in data['properties']
def test_micropub_query_source_with_name(client, mock_valid_token):
"""q=source returns name and content the way they were posted"""
with patch('starpunk.routes.micropub.verify_access_token', mock_valid_token):
create_response = client.post(
'/micropub',
json={
'type': ['h-entry'],
'properties': {
'name': ['Round Trip'],
'content': ['Body text']
}
},
headers={'Authorization': 'Bearer valid_token'}
)
location = create_response.headers['Location']
response = client.get(
f'/micropub?q=source&url={location}',
headers={'Authorization': 'Bearer valid_token'}
)
assert response.status_code == 200
properties = response.get_json()['properties']
assert properties['name'] == ['Round Trip']
assert properties['content'] == ['Body text']
assert properties['url'] == [location]
# Error Handling Tests
@@ -312,33 +372,21 @@ def test_micropub_missing_content(client, mock_valid_token):
assert data['error'] == 'invalid_request'
def test_micropub_unsupported_action(client, mock_valid_token):
"""Test unsupported actions (update, delete) return error"""
def test_micropub_update_delete_need_scope(client, mock_valid_token):
"""A create-only token can't update or delete (v2.4.0)"""
with patch('starpunk.routes.micropub.verify_access_token', mock_valid_token):
# Test update
response = client.post(
'/micropub',
json={
'action': 'update',
'url': 'https://example.com/note/123'
},
headers={'Authorization': 'Bearer valid_token'}
)
for action in ('update', 'delete', 'undelete'):
response = client.post(
'/micropub',
json={'action': action, 'url': 'https://example.com/note/123', 'replace': {}},
headers={'Authorization': 'Bearer valid_token'}
)
assert response.status_code == 403, action
assert response.get_json()['error'] == 'insufficient_scope'
def test_micropub_unknown_action(client, mock_valid_token):
with patch('starpunk.routes.micropub.verify_access_token', mock_valid_token):
response = client.post('/micropub', json={'action': 'frobnicate'},
headers={'Authorization': 'Bearer valid_token'})
assert response.status_code == 400
data = response.get_json()
assert 'not supported' in data['error_description']
# Test delete
response = client.post(
'/micropub',
json={
'action': 'delete',
'url': 'https://example.com/note/123'
},
headers={'Authorization': 'Bearer valid_token'}
)
assert response.status_code == 400
data = response.get_json()
assert 'not supported' in data['error_description']
+206
View File
@@ -0,0 +1,206 @@
"""
Micropub round 1 (v2.4.0): update, delete/undelete, drafts, replies,
multipart and remote photos, q=source lists and filters, q=category.
Uses real tokens from StarPunk's own IndieAuth server.
"""
import io
import httpx
import pytest
from PIL import Image
from starpunk import create_app
from starpunk.indieauth import create_access_token
from starpunk.notes import get_note
SITE = "https://blog.example.com"
@pytest.fixture
def app(tmp_path):
data = tmp_path / "data"
return create_app({
"TESTING": True, "SITE_URL": SITE, "ADMIN_ME": "https://example.com",
"SESSION_SECRET": "x", "DATA_PATH": data, "NOTES_PATH": data / "notes",
"DATABASE_PATH": data / "starpunk.db",
})
def auth(app, scope="create update delete media"):
with app.test_request_context():
token = create_access_token("https://quill.example/", scope)["access_token"]
return {"Authorization": f"Bearer {token}"}
@pytest.fixture
def client(app):
return app.test_client()
@pytest.fixture
def h(app):
return auth(app)
def jpeg(color="red"):
buf = io.BytesIO()
Image.new("RGB", (64, 48), color).save(buf, format="JPEG")
return buf.getvalue()
def create(client, h, **props):
body = {"type": ["h-entry"], "properties": {"content": ["Hello"], **props}}
response = client.post("/micropub", json=body, headers=h)
assert response.status_code == 201, response.get_json()
return response.headers["Location"]
def note_at(app, url):
with app.app_context():
return get_note(slug=url.rstrip("/").rsplit("/", 1)[1])
def source(client, h, url, *props):
query = f"/micropub?q=source&url={url}" + "".join(f"&properties[]={p}" for p in props)
return client.get(query, headers=h).get_json()
class TestCreate:
def test_draft(self, app, client, h):
url = create(client, h, **{"post-status": ["draft"]})
assert note_at(app, url).published is False
assert client.get(url.replace(SITE, "")).status_code == 404
def test_reply(self, app, client, h):
url = create(client, h, **{"in-reply-to": ["https://other.example/post/1"]})
assert note_at(app, url).in_reply_to == "https://other.example/post/1"
def test_multipart_photo(self, app, client, h):
response = client.post(
"/micropub",
data={"h": "entry", "content": "With a file", "photo": (io.BytesIO(jpeg()), "pic.jpg")},
headers=h, content_type="multipart/form-data",
)
assert response.status_code == 201
assert "photo" in source(client, h, response.headers["Location"])["properties"]
def test_remote_photo_is_downloaded(self, app, client, h, monkeypatch):
transport = httpx.MockTransport(lambda req: httpx.Response(200, content=jpeg("blue")))
monkeypatch.setattr(httpx, "stream", lambda method, url, **kw: httpx.Client(transport=transport).stream(method, url))
url = create(client, h, photo=[{"value": "https://photos.example/cat.jpg", "alt": "A cat"}])
(photo,) = source(client, h, url)["properties"]["photo"]
assert photo["value"].startswith(f"{SITE}/media/") and photo["alt"] == "A cat"
def test_only_h_entry(self, client, h):
response = client.post("/micropub", data={"h": "event", "content": "x"}, headers=h)
assert response.status_code == 400
class TestUpdate:
def update(self, client, h, url, **ops):
return client.post("/micropub", json={"action": "update", "url": url, **ops}, headers=h)
def test_replace_content_keeps_title(self, app, client, h):
url = create(client, h, name=["My Title"])
assert self.update(client, h, url, replace={"content": ["New body"]}).status_code == 204
props = source(client, h, url)["properties"]
assert props["name"] == ["My Title"] and props["content"] == ["New body"]
def test_replace_and_delete_name(self, app, client, h):
url = create(client, h)
self.update(client, h, url, replace={"name": ["Added"]})
assert note_at(app, url).content.startswith("# Added")
self.update(client, h, url, delete=["name"])
assert "name" not in source(client, h, url)["properties"]
def test_categories(self, app, client, h):
url = create(client, h, category=["one", "two"])
self.update(client, h, url, add={"category": ["three", "One"]})
assert source(client, h, url)["properties"]["category"] == ["one", "three", "two"]
self.update(client, h, url, delete={"category": ["two"]})
assert source(client, h, url)["properties"]["category"] == ["one", "three"]
self.update(client, h, url, delete=["category"])
assert "category" not in source(client, h, url)["properties"]
def test_post_status_and_reply(self, app, client, h):
url = create(client, h)
self.update(client, h, url, replace={"post-status": ["draft"], "in-reply-to": ["https://x.example/a"]})
note = note_at(app, url)
assert note.published is False and note.in_reply_to == "https://x.example/a"
self.update(client, h, url, delete=["in-reply-to", "post-status"])
note = note_at(app, url)
assert note.published is True and not note.in_reply_to
def test_photos(self, app, client, h):
uploaded = client.post("/micropub/media", data={"file": (io.BytesIO(jpeg()), "a.jpg")},
headers=h, content_type="multipart/form-data").headers["Location"]
url = create(client, h)
self.update(client, h, url, add={"photo": [{"value": uploaded, "alt": "Red"}]})
assert source(client, h, url)["properties"]["photo"] == [{"value": uploaded, "alt": "Red"}]
self.update(client, h, url, delete={"photo": [uploaded]})
assert "photo" not in source(client, h, url)["properties"]
def test_unsupported_property(self, client, h):
url = create(client, h)
response = self.update(client, h, url, replace={"published": ["2020-01-01T00:00:00Z"]})
assert response.status_code == 400 and "published" in response.get_json()["error_description"]
def test_unknown_post(self, client, h):
response = self.update(client, h, f"{SITE}/note/nope", replace={"content": ["x"]})
assert response.status_code == 404
def test_form_update_rejected(self, client, h):
url = create(client, h)
assert client.post("/micropub", data={"action": "update", "url": url}, headers=h).status_code == 400
class TestDelete:
def test_delete_and_undelete(self, app, client, h):
url = create(client, h, category=["kept"])
path = url.replace(SITE, "")
assert client.post("/micropub", data={"action": "delete", "url": url}, headers=h).status_code == 204
assert client.get(path).status_code == 404
response = client.post("/micropub", json={"action": "undelete", "url": url}, headers=h)
assert response.status_code == 204
assert client.get(path).status_code == 200
assert source(client, h, url)["properties"]["category"] == ["kept"]
def test_undelete_needs_a_deleted_post(self, client, h):
url = create(client, h)
assert client.post("/micropub", json={"action": "undelete", "url": url}, headers=h).status_code == 404
def test_delete_scope(self, app, client):
h = auth(app, "create")
url = create(client, h)
assert client.post("/micropub", data={"action": "delete", "url": url}, headers=h).status_code == 403
class TestQueries:
def test_source_properties_filter(self, client, h):
url = create(client, h, name=["T"], category=["x"])
data = source(client, h, url, "name", "category")
assert data == {"properties": {"name": ["T"], "category": ["x"]}}
def test_source_list_includes_drafts(self, client, h):
create(client, h, content=["first"])
create(client, h, content=["second"], **{"post-status": ["draft"]})
items = client.get("/micropub?q=source&limit=5", headers=h).get_json()["items"]
assert {i["properties"]["post-status"][0] for i in items} == {"published", "draft"}
def test_category(self, client, h):
create(client, h, category=["IndieWeb", "python"])
create(client, h, category=["IndieWeb"])
assert client.get("/micropub?q=category", headers=h).get_json()["categories"] == ["IndieWeb", "python"]
assert client.get("/micropub?q=category&filter=py", headers=h).get_json()["categories"] == ["python"]
def test_config_lists_queries(self, client, h):
config = client.get("/micropub?q=config", headers=h).get_json()
assert "category" in config["q"] and "media-endpoint" in config
def test_media_scope_alone_can_upload(app, client):
response = client.post("/micropub/media", data={"file": (io.BytesIO(jpeg()), "a.jpg")},
headers=auth(app, "media"), content_type="multipart/form-data")
assert response.status_code == 201
+38 -1
View File
@@ -11,7 +11,7 @@ Organized by model:
import pytest
from datetime import datetime, timedelta
from pathlib import Path
from starpunk.models import Note, Session, Token, AuthState
from starpunk.models import Note, Session, Token, AuthState, split_title
from starpunk.utils import calculate_content_hash
@@ -245,6 +245,43 @@ class TestNoteModel:
assert note.excerpt == "my-note"
def _note_with(self, tmp_path, content):
note_file = tmp_path / "notes" / "2024" / "11" / "test.md"
note_file.parent.mkdir(parents=True, exist_ok=True)
note_file.write_text(content)
return Note(
id=1,
slug="test",
file_path="notes/2024/11/test.md",
published=True,
created_at=datetime.utcnow(),
updated_at=datetime.utcnow(),
_data_dir=tmp_path,
)
def test_split_title(self):
"""A '# Heading' first line is split off as the title"""
assert split_title("# My Title\n\nBody") == ("My Title", "Body")
assert split_title("\n## Sub\nBody") == ("Sub", "Body")
assert split_title("No heading\n# later") == (None, "No heading\n# later")
assert split_title("#\nBody") == (None, "#\nBody")
def test_body_html_leaves_out_title(self, tmp_path):
"""Pages show the title themselves, so the body must not repeat it"""
note = self._note_with(tmp_path, "# My Title\n\nBody text.")
assert "My Title" not in note.body_html
assert "<p>Body text.</p>" in note.body_html
assert "<h1>My Title</h1>" in note.html # html is unchanged
def test_body_html_untitled_matches_html(self, tmp_path):
note = self._note_with(tmp_path, "Plain note.")
assert note.body_html == note.html
def test_excerpt_leaves_out_title(self, tmp_path):
"""Previews show the title above the excerpt, so it is not repeated"""
note = self._note_with(tmp_path, "# Title\n\nShort content.")
assert note.excerpt == "Short content."
def test_permalink(self):
"""Test permalink generation"""
note = Note(
+69
View File
@@ -0,0 +1,69 @@
"""Public page markup from the v2.3.0 style pass"""
import markdown
import pytest
from starpunk import create_app
from starpunk.models import MARKDOWN_EXTENSIONS
from starpunk.notes import create_note
def md(text):
return markdown.markdown(text, extensions=MARKDOWN_EXTENSIONS)
@pytest.mark.parametrize("text, expected", [
("see https://sqlite.org/x.html.", '<a href="https://sqlite.org/x.html">https://sqlite.org/x.html</a>.'),
("(https://a.example/p)", '(<a href="https://a.example/p">https://a.example/p</a>)'),
("[label](https://a.example)", '<a href="https://a.example">label</a>'),
("`https://code.example`", "<code>https://code.example</code>"),
])
def test_bare_urls_are_linked(text, expected):
assert expected in md(text)
def test_url_as_link_label_is_not_nested():
assert md("[https://b.example](https://b.example)").count("<a ") == 1
@pytest.fixture
def app(tmp_path):
data = tmp_path / "data"
return create_app({
"TESTING": True, "SITE_URL": "https://blog.example.com", "SITE_NAME": "My Notes",
"ADMIN_ME": "https://example.com", "SESSION_SECRET": "x",
"DATA_PATH": data, "NOTES_PATH": data / "notes", "DATABASE_PATH": data / "starpunk.db",
})
def test_home_shows_full_notes_and_site_name(app):
long_text = "word " * 120 + "THE-END"
with app.app_context():
create_note(content=long_text, published=True, tags=["thoughts"])
html = app.test_client().get("/").data.decode()
assert "THE-END" in html # not cut at 300 characters
assert '<a class="site-title" href="/">My Notes</a>' in html
assert 'class="p-category" rel="tag"' in html
# Notes' metadata is not styled as the page footer
assert html.count('<footer class="site-footer">') == 1
assert '<footer class="note-meta">' in html
nav = html.split('class="site-nav"', 1)[1].split("</nav>", 1)[0]
assert 'href="/auth/login"' in nav # login is in the top bar
def test_note_page_meta_line(app):
with app.app_context():
note = create_note(content="Hello", published=True)
html = app.test_client().get(f"/note/{note.slug}").data.decode()
assert "<title>Hello · My Notes</title>" in html
assert 'class="u-url u-uid"' in html
def test_nav_shows_admin_when_logged_in(app):
from starpunk.auth import create_session
client = app.test_client()
with app.test_request_context():
client.set_cookie("starpunk_session", create_session("https://example.com"))
nav = client.get("/").data.decode().split('class="site-nav"', 1)[1].split("</nav>", 1)[0]
assert ">Admin</a>" in nav
assert "/auth/login" not in nav
+2 -1
View File
@@ -274,7 +274,8 @@ class TestVersionDisplay:
"""Test version number appears in footer"""
response = client.get("/")
assert response.status_code == 200
assert b"0.5.0" in response.data or b"StarPunk v" in response.data
from starpunk import __version__
assert f"v{__version__}".encode() in response.data
# OAuth metadata endpoint tests removed in Phase 1 of IndieAuth server removal
+274
View File
@@ -0,0 +1,274 @@
"""
Tests for crossposting to Bluesky (starpunk/syndication.py, v2.2.0)
A fake PDS (httpx.MockTransport) records what StarPunk sends.
"""
import io
import json
import httpx
import pytest
from PIL import Image
from starpunk import create_app, syndication
from starpunk.notes import create_note, delete_note, update_note
from starpunk.syndication import (
BlueskyClient,
bluesky_facets,
get_note_syndications,
plain_text,
process_due,
select_targets,
)
SERVICE = "https://bsky.example"
PDS = "https://pds.example"
DID = "did:plc:abc123"
class FakePDS:
def __init__(self):
self.records = {}
self.blobs = []
self.calls = []
self.logins = 0
self.expire_next = False
self.lose_next_reply = False
self.fail_next = None
def _json(self, code, body):
return httpx.Response(code, json=body)
def __call__(self, request: httpx.Request) -> httpx.Response:
nsid = request.url.path.removeprefix("/xrpc/")
self.calls.append((request.url.host, nsid))
if nsid == "com.atproto.server.createSession":
body = json.loads(request.content)
if body["password"] != "app-pass":
return self._json(401, {"error": "AuthenticationRequired"})
self.logins += 1
return self._json(200, {
"did": DID, "handle": "phil.example", "accessJwt": f"access{self.logins}",
"refreshJwt": "refresh",
"didDoc": {"service": [{"id": "#atproto_pds", "serviceEndpoint": PDS}]},
})
if nsid == "com.atproto.server.refreshSession":
self.logins += 1
return self._json(200, {"did": DID, "handle": "phil.example",
"accessJwt": f"access{self.logins}", "refreshJwt": "refresh"})
assert request.url.host == "pds.example"
if request.headers.get("Authorization") != f"Bearer access{self.logins}" or self.expire_next:
self.expire_next = False
return self._json(400, {"error": "ExpiredToken"})
if self.fail_next:
code, self.fail_next = self.fail_next, None
return self._json(code, {"error": "Nope"})
if nsid == "com.atproto.repo.uploadBlob":
self.blobs.append(len(request.content))
assert len(request.content) <= 1_000_000
return self._json(200, {"blob": {"$type": "blob", "ref": {"$link": f"b{len(self.blobs)}"},
"mimeType": request.headers["Content-Type"],
"size": len(request.content)}})
if nsid == "com.atproto.repo.createRecord":
body = json.loads(request.content)
uri = f"at://{DID}/app.bsky.feed.post/r{len(self.records) + 1}"
self.records[uri] = body["record"]
if self.lose_next_reply:
self.lose_next_reply = False
raise httpx.ReadTimeout("lost reply")
return self._json(200, {"uri": uri, "cid": "c"})
if nsid == "com.atproto.repo.listRecords":
return self._json(200, {"records": [{"uri": u, "value": v} for u, v in self.records.items()]})
if nsid == "com.atproto.repo.deleteRecord":
body = json.loads(request.content)
self.records.pop(f"at://{body['repo']}/{body['collection']}/{body['rkey']}", None)
return self._json(200, {})
return self._json(500, {"error": "unknown"})
@pytest.fixture
def pds():
return FakePDS()
@pytest.fixture
def app(tmp_path, pds):
data = tmp_path / "data"
app = create_app({
"TESTING": True,
"SITE_URL": "https://blog.example.com",
"ADMIN_ME": "https://example.com",
"SESSION_SECRET": "test-secret",
"DATA_PATH": data,
"NOTES_PATH": data / "notes",
"DATABASE_PATH": data / "starpunk.db",
"BLUESKY_HANDLE": "phil.example",
"BLUESKY_APP_PASSWORD": "app-pass",
"BLUESKY_SERVICE": SERVICE,
})
syndication._client_cache[f"bluesky:{SERVICE}:phil.example"] = BlueskyClient(
SERVICE, "phil.example", "app-pass", http=httpx.Client(transport=httpx.MockTransport(pds))
)
yield app
syndication._client_cache.clear()
def publish(app, content="Hello Bluesky", tags=None):
with app.test_request_context():
note = create_note(content=content, published=True, tags=tags)
select_targets(note.id, ["bluesky"])
process_due()
return note
def only_record(pds):
(record,) = pds.records.values()
return record
def row(app, note):
with app.app_context():
(r,) = get_note_syndications(note.id)
return r
class TestText:
def test_markdown_becomes_plain_text(self):
md = "# Title\n\nSome *emphasis* and a [link](https://x.example).\n\n- one\n- two\n\n![pic](/m.jpg)"
assert plain_text(md) == "Title\n\nSome emphasis and a link.\n\n• one\n• two"
def test_facets_use_utf8_byte_offsets(self):
text = "Café ☕ https://blog.example.com/note/x #IndieWeb"
link, tag = bluesky_facets(text)
raw = text.encode()
assert raw[link["index"]["byteStart"]:link["index"]["byteEnd"]] == b"https://blog.example.com/note/x"
assert raw[tag["index"]["byteStart"]:tag["index"]["byteEnd"]] == b"#IndieWeb"
assert tag["features"][0]["tag"] == "IndieWeb"
def test_fragment_in_url_is_not_a_tag(self):
facets = bluesky_facets("see https://x.example/page#section and #42 too")
assert [f["features"][0]["$type"] for f in facets] == ["app.bsky.richtext.facet#link"]
class TestLifecycle:
def test_publish_posts_with_link_card(self, app, pds):
note = publish(app, content="Hello **Bluesky**", tags=["IndieWeb"])
record = only_record(pds)
permalink = f"https://blog.example.com/note/{note.slug}"
assert record["text"] == f"Hello Bluesky\n\n{permalink}\n\n#IndieWeb"
assert record["embed"]["external"]["uri"] == permalink
kinds = {f["features"][0]["$type"] for f in record["facets"]}
assert kinds == {"app.bsky.richtext.facet#link", "app.bsky.richtext.facet#tag"}
r = row(app, note)
assert r["state"] == "synced"
assert r["remote_id"] == f"at://{DID}/app.bsky.feed.post/r1"
assert r["remote_url"] == "https://bsky.app/profile/phil.example/post/r1"
def test_long_note_fits_300(self, app, pds):
note = publish(app, content="word " * 500)
text = only_record(pds)["text"]
assert len(text) <= 300
assert text.endswith(f"https://blog.example.com/note/{note.slug}")
def test_edit_leaves_post_alone(self, app, pds):
note = publish(app, content="first")
with app.test_request_context():
update_note(id=note.id, content="second")
process_due()
assert only_record(pds)["text"].startswith("first")
assert row(app, note)["state"] == "synced"
def test_delete_and_unpublish_remove_post(self, app, pds):
note = publish(app)
with app.test_request_context():
update_note(id=note.id, published=False)
process_due()
assert pds.records == {}
update_note(id=note.id, published=True)
process_due()
assert len(pds.records) == 1
delete_note(id=note.id)
process_due()
assert pds.records == {}
class TestSession:
def test_logs_in_once_and_uses_pds(self, app, pds):
publish(app, content="one")
publish(app, content="two")
assert pds.logins == 1
assert ("bsky.example", "com.atproto.server.createSession") in pds.calls
assert ("pds.example", "com.atproto.repo.createRecord") in pds.calls
def test_expired_token_is_refreshed(self, app, pds):
publish(app, content="one")
pds.expire_next = True
publish(app, content="two")
assert len(pds.records) == 2
assert ("bsky.example", "com.atproto.server.refreshSession") in pds.calls
def test_bad_password_fails_permanently(self, app, pds):
syndication._client_cache[f"bluesky:{SERVICE}:phil.example"].password = "wrong"
note = publish(app)
r = row(app, note)
assert r["state"] == "failed" and "401" in r["last_error"]
def test_lost_reply_does_not_double_post(self, app, pds):
pds.lose_next_reply = True
note = publish(app)
assert row(app, note)["state"] == "pending"
with app.test_request_context():
from starpunk.syndication import retry
retry(row(app, note)["id"])
# retry() resets attempts; simulate the scheduled retry instead
from starpunk.database import get_db
get_db(app).execute("UPDATE syndications SET attempts = 1")
get_db(app).commit()
process_due()
assert len(pds.records) == 1
assert row(app, note)["state"] == "synced"
def test_server_error_retries(self, app, pds):
publish(app, content="warm up") # log in first
pds.fail_next = 502
note = publish(app, content="later")
assert row(app, note)["state"] == "pending"
class TestMedia:
def test_large_image_is_shrunk_with_alt_and_aspect_ratio(self, app, pds):
from starpunk.media import attach_media_to_note, save_media
import os
img = Image.frombytes("RGB", (2400, 1800), os.urandom(2400 * 1800 * 3))
buf = io.BytesIO()
img.save(buf, format="PNG")
assert buf.tell() > 1_000_000
with app.test_request_context():
note = create_note(content="noisy", published=True)
media = save_media(buf.getvalue(), "noise.png")
attach_media_to_note(note.id, [media["id"]], ["Static"])
select_targets(note.id, ["bluesky"])
process_due()
record = only_record(pds)
(image,) = record["embed"]["images"]
assert record["embed"]["$type"] == "app.bsky.embed.images"
assert image["alt"] == "Static"
assert image["aspectRatio"]["width"] > 0
def test_both_targets_listed_for_micropub(tmp_path):
data = tmp_path / "d"
app = create_app({
"TESTING": True, "SITE_URL": "https://b.example", "ADMIN_ME": "https://e.example",
"SESSION_SECRET": "x", "DATA_PATH": data, "NOTES_PATH": data / "notes",
"DATABASE_PATH": data / "s.db",
"GOTOSOCIAL_URL": "https://social.example", "GOTOSOCIAL_ACCESS_TOKEN": "t",
"BLUESKY_HANDLE": "phil.example", "BLUESKY_APP_PASSWORD": "p",
})
with app.app_context():
from starpunk.syndication import get_targets, resolve_targets
assert set(get_targets()) == {"gotosocial", "bluesky"}
assert resolve_targets(["https://bsky.app/profile/phil.example"]) == ["bluesky"]
syndication._client_cache.clear()
+402
View File
@@ -0,0 +1,402 @@
"""
Tests for crossposting to Threads (starpunk/syndication.py, v2.6.0)
A fake Graph API (httpx.MockTransport) records what StarPunk sends.
"""
import io
import json
import time
from urllib.parse import parse_qsl
import httpx
import pytest
from PIL import Image
from starpunk import create_app, syndication
from starpunk.notes import create_note, delete_note, update_note
from starpunk.syndication import (
THREADS_REFRESH_EVERY,
ThreadsClient,
get_note_syndications,
process_due,
select_targets,
threads_length,
threads_topic,
)
API = "https://graph.threads.example/v1.0"
UID = "1789"
class FakeThreads:
def __init__(self):
self.containers = {}
self.posts = {}
self.calls = []
self.token = "tok1"
self.refreshes = 0
self.refresh_fails = False
self.pending_polls = 0
self.lose_next_publish = False
self.fail_next = None
self.fail_next_publish = False
def _json(self, code, body):
return httpx.Response(code, json=body)
def _error(self, code, message, err_code, subcode=None):
error = {"message": message, "type": "OAuthException", "code": err_code}
if subcode:
error["error_subcode"] = subcode
return self._json(code, {"error": error})
def __call__(self, request: httpx.Request) -> httpx.Response:
params = dict(request.url.params)
if request.content:
params.update(parse_qsl(request.content.decode()))
path = request.url.path
self.calls.append((request.method, path, params))
if path == "/refresh_access_token":
assert params["grant_type"] == "th_refresh_token"
if self.refresh_fails or params.get("access_token") != self.token:
return self._error(400, "token too new", 190)
self.refreshes += 1
self.token = f"tok{self.refreshes + 1}"
return self._json(200, {"access_token": self.token, "token_type": "bearer",
"expires_in": 5184000})
if params.get("access_token") != self.token:
return self._error(400, "Invalid OAuth access token", 190)
if self.fail_next:
code, err, self.fail_next = *self.fail_next, None
return self._error(code, "nope", err)
path = path.removeprefix("/v1.0/")
if path == "me":
return self._json(200, {"id": UID, "username": "phil"})
if path == f"{UID}/threads" and request.method == "POST":
cid = f"c{len(self.containers) + 1}"
self.containers[cid] = params
return self._json(200, {"id": cid})
if path == f"{UID}/threads" and request.method == "GET":
return self._json(200, {"data": [
{"id": pid, "text": p["text"], "permalink": f"https://www.threads.com/@phil/post/{pid}"}
for pid, p in self.posts.items()]})
if path == f"{UID}/threads_publish" and self.fail_next_publish:
self.fail_next_publish = False
return self._error(400, "Fatal", 24, 4279009)
if path == f"{UID}/threads_publish":
container = self.containers[params["creation_id"]]
pid = f"p{len(self.posts) + 1}"
self.posts[pid] = container
if self.lose_next_publish:
self.lose_next_publish = False
raise httpx.ReadTimeout("lost reply")
return self._json(200, {"id": pid})
if path in self.containers and request.method == "GET":
if self.pending_polls:
self.pending_polls -= 1
return self._json(200, {"status": "IN_PROGRESS"})
return self._json(200, {"status": "FINISHED"})
if path in self.posts and request.method == "GET":
return self._json(200, {"id": path, "permalink": f"https://www.threads.com/@phil/post/{path}"})
if request.method == "DELETE":
if self.posts.pop(path, None) is None:
return self._error(400, "Object does not exist", 100, 33)
return self._json(200, {"success": True, "deleted_id": path})
return self._json(500, {"error": {"message": "unknown", "code": 2}})
@pytest.fixture
def api():
return FakeThreads()
def make_client(app, api, token="tok1"):
return ThreadsClient(token, app.config["DATA_PATH"] / "threads_token.json", api=API,
http=httpx.Client(transport=httpx.MockTransport(api)), sleep=lambda s: None)
@pytest.fixture
def app(tmp_path, api):
data = tmp_path / "data"
app = create_app({
"TESTING": True,
"SITE_URL": "https://blog.example.com",
"ADMIN_ME": "https://example.com",
"SESSION_SECRET": "test-secret",
"DATA_PATH": data,
"NOTES_PATH": data / "notes",
"DATABASE_PATH": data / "starpunk.db",
"THREADS_ACCESS_TOKEN": "tok1",
"THREADS_USERNAME": "phil",
})
syndication._client_cache["threads"] = make_client(app, api)
yield app
syndication._client_cache.clear()
def publish(app, content="Hello Threads", tags=None):
with app.test_request_context():
note = create_note(content=content, published=True, tags=tags)
select_targets(note.id, ["threads"])
process_due()
return note
def only_post(api):
(post,) = api.posts.values()
return post
def row(app, note):
with app.app_context():
(r,) = get_note_syndications(note.id)
return r
def image_bytes(fmt="JPEG", size=(800, 600)):
buf = io.BytesIO()
Image.new("RGB", size, (200, 30, 30)).save(buf, format=fmt)
return buf.getvalue()
class TestText:
def test_emoji_count_as_bytes(self):
assert threads_length("abc") == 3
assert threads_length("☕") == 3
def test_topic_is_first_usable_tag(self):
assert threads_topic(["...", "Indie.Web & Co", "x"]) == "IndieWeb Co"
assert threads_topic([]) == ""
class TestLifecycle:
def test_publish_text_post(self, app, api):
note = publish(app, content="Hello **Threads**", tags=["IndieWeb", "Second"])
post = only_post(api)
permalink = f"https://blog.example.com/note/{note.slug}"
assert post["media_type"] == "TEXT"
assert post["text"] == f"Hello Threads\n\n{permalink}"
assert post["link_attachment"] == permalink
assert post["topic_tag"] == "IndieWeb"
r = row(app, note)
assert r["state"] == "synced"
assert r["remote_id"] == "p1"
assert r["remote_url"] == "https://www.threads.com/@phil/post/p1"
def test_long_note_fits_500(self, app, api):
note = publish(app, content="☕ word " * 300)
text = only_post(api)["text"]
assert threads_length(text) <= 500
assert text.endswith(f"https://blog.example.com/note/{note.slug}")
def test_waits_for_container(self, app, api):
api.pending_polls = 2
note = publish(app)
assert row(app, note)["state"] == "synced"
def test_edit_leaves_post_alone(self, app, api):
note = publish(app, content="first")
with app.test_request_context():
update_note(id=note.id, content="second")
process_due()
assert only_post(api)["text"].startswith("first")
assert row(app, note)["state"] == "synced"
def test_delete_and_unpublish_remove_post(self, app, api):
note = publish(app)
with app.test_request_context():
update_note(id=note.id, published=False)
process_due()
assert api.posts == {}
update_note(id=note.id, published=True)
process_due()
assert len(api.posts) == 1
delete_note(id=note.id)
process_due()
assert api.posts == {}
def test_post_deleted_on_threads_is_fine(self, app, api):
note = publish(app)
api.posts.clear()
with app.test_request_context():
delete_note(id=note.id)
process_due()
assert row(app, note)["state"] == "synced"
class TestErrors:
def test_lost_reply_does_not_double_post(self, app, api):
api.lose_next_publish = True
note = publish(app)
assert row(app, note)["state"] == "pending"
with app.test_request_context():
from starpunk.database import get_db
get_db(app).execute("UPDATE syndications SET next_attempt_at = datetime('now')")
get_db(app).commit()
process_due()
assert len(api.posts) == 1
assert row(app, note)["state"] == "synced"
def test_publish_fatal_is_retried(self, app, api):
api.fail_next_publish = True
note = publish(app)
r = row(app, note)
assert r["state"] == "pending" and "Fatal" in r["last_error"] and "subcode 4279009" in r["last_error"]
with app.test_request_context():
from starpunk.database import get_db
get_db(app).execute("UPDATE syndications SET next_attempt_at = datetime('now')")
get_db(app).commit()
process_due()
assert len(api.posts) == 1
assert row(app, note)["state"] == "synced"
def test_bad_token_fails_permanently(self, app, api):
api.token = "other"
note = publish(app)
r = row(app, note)
assert r["state"] == "failed" and "Invalid OAuth" in r["last_error"]
assert "tok1" not in r["last_error"]
def test_rate_limit_retries(self, app, api):
publish(app, content="warm up")
api.fail_next = (400, 4) # Graph "application request limit reached"
note = publish(app, content="later")
assert row(app, note)["state"] == "pending"
class TestMedia:
def _note_with(self, app, images):
from starpunk.media import attach_media_to_note, save_media
with app.test_request_context():
note = create_note(content="pics", published=True)
ids = [save_media(data, name)["id"] for data, name in images]
attach_media_to_note(note.id, ids, [f"alt {i}" for i in range(len(ids))])
select_targets(note.id, ["threads"])
process_due()
return note
def test_single_image(self, app, api):
self._note_with(app, [(image_bytes(), "a.jpg")])
post = only_post(api)
assert post["media_type"] == "IMAGE"
assert post["image_url"].startswith("https://blog.example.com/media/")
assert post["alt_text"] == "alt 0"
assert "link_attachment" not in post
def test_carousel(self, app, api):
self._note_with(app, [(image_bytes(), "a.jpg"), (image_bytes("PNG"), "b.png")])
post = only_post(api)
assert post["media_type"] == "CAROUSEL"
children = [api.containers[c] for c in post["children"].split(",")]
assert [c["is_carousel_item"] for c in children] == ["true", "true"]
assert [c["alt_text"] for c in children] == ["alt 0", "alt 1"]
def test_unsupported_format_is_left_off(self, app, api):
self._note_with(app, [(image_bytes("WEBP"), "a.webp")])
assert only_post(api)["media_type"] == "TEXT"
class TestToken:
def test_refreshes_daily_and_saves(self, app, api):
client = syndication._client_cache["threads"]
with app.app_context():
client.maintain()
assert api.refreshes == 1
client.maintain() # same day: nothing
assert api.refreshes == 1
saved = json.loads((app.config["DATA_PATH"] / "threads_token.json").read_text())
assert saved["token"] == "tok2"
# A restart picks up the saved token
assert make_client(app, api)._token == "tok2"
publish(app)
assert len(api.posts) == 1
def test_other_worker_refresh_is_reused(self, app, api):
first, second = make_client(app, api), make_client(app, api)
with app.app_context():
first.maintain()
second.maintain()
assert api.refreshes == 1
assert second._token == "tok2"
def test_refresh_due_after_a_day(self, app, api, monkeypatch):
client = syndication._client_cache["threads"]
with app.app_context():
client.maintain()
later = time.time() + THREADS_REFRESH_EVERY + 1
monkeypatch.setattr(syndication.time, "time", lambda: later)
client.maintain()
assert api.refreshes == 2
def test_failed_refresh_keeps_token(self, app, api):
api.refresh_fails = True
client = syndication._client_cache["threads"]
with app.app_context():
client.maintain()
assert client._token == "tok1"
publish(app)
assert len(api.posts) == 1
def test_new_config_token_replaces_saved_one(self, app, api):
with app.app_context():
syndication._client_cache["threads"].maintain()
api.token = "fresh"
assert make_client(app, api, token="fresh")._token == "fresh"
def test_worker_runs_upkeep(self, app, api):
with app.app_context():
process_due()
assert api.refreshes == 1
def test_all_targets_listed_for_micropub(tmp_path):
data = tmp_path / "d"
app = create_app({
"TESTING": True, "SITE_URL": "https://b.example", "ADMIN_ME": "https://e.example",
"SESSION_SECRET": "x", "DATA_PATH": data, "NOTES_PATH": data / "notes",
"DATABASE_PATH": data / "s.db",
"GOTOSOCIAL_URL": "https://social.example", "GOTOSOCIAL_ACCESS_TOKEN": "t",
"BLUESKY_HANDLE": "phil.example", "BLUESKY_APP_PASSWORD": "p",
"THREADS_ACCESS_TOKEN": "x", "THREADS_USERNAME": "@phil",
})
with app.app_context():
from starpunk.syndication import get_targets, resolve_targets
assert set(get_targets()) == {"gotosocial", "bluesky", "threads"}
assert resolve_targets(["https://www.threads.com/@phil"]) == ["threads"]
syndication._client_cache.clear()
class TestLikesReposts:
PAGE = "https://other.example/post/1"
def _post(self, app, monkeypatch, content="", **kind):
monkeypatch.setattr("starpunk.reply_context.fetch_reply_context", lambda *a, **k: None)
with app.test_request_context():
note = create_note(content=content, published=True, **kind)
select_targets(note.id, ["threads"])
process_due()
return note
def test_like_is_not_crossposted(self, app, api, monkeypatch):
note = self._post(app, monkeypatch, like_of=self.PAGE)
assert api.posts == {}
r = row(app, note)
assert r["state"] == "synced" and r["remote_id"] is None
def test_repost_posts_link(self, app, api, monkeypatch):
self._post(app, monkeypatch, repost_of=self.PAGE)
post = only_post(api)
assert post["text"].startswith("🔁") and self.PAGE in post["text"]
assert post["link_attachment"] == self.PAGE
def test_repost_with_comment_keeps_comment(self, app, api, monkeypatch):
note = self._post(app, monkeypatch, content="So **true**", repost_of=self.PAGE)
assert only_post(api)["text"].startswith("So true")
with app.test_request_context():
delete_note(id=note.id)
process_due()
assert api.posts == {}