Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d3148063e3 | ||
|
|
39c8259f3f | ||
|
|
1ce88b38a5 | ||
|
|
4ffe36f9d3 | ||
|
|
a551770c76 | ||
|
|
d0bfc80c2e | ||
|
|
d16463ef8a | ||
|
|
4b56f7cd4b | ||
|
|
2474628646 | ||
|
|
0fe341161e | ||
|
|
a62077ab30 | ||
|
|
fade75fd2d | ||
|
|
0dd59fbc18 | ||
|
|
11e09a0d02 | ||
|
|
565adaeecf | ||
|
|
53d10ed271 | ||
|
|
e8c1b03980 |
No files matched your search
@@ -63,6 +63,14 @@ GOTOSOCIAL_NAME=GoToSocial
|
||||
# public, unlisted, private (default: public)
|
||||
GOTOSOCIAL_VISIBILITY=public
|
||||
|
||||
# Threads account (v2.6.0). A long-lived user token with threads_basic,
|
||||
# threads_content_publish and threads_delete; StarPunk refreshes it daily and
|
||||
# keeps the current one in DATA_PATH/threads_token.json.
|
||||
# How to get one: docs/design/v2.6.0/2026-10-31-threads-crossposting.md
|
||||
THREADS_ACCESS_TOKEN=
|
||||
# Your Threads username (for the Micropub syndicate-to uid)
|
||||
THREADS_USERNAME=
|
||||
|
||||
# =============================================================================
|
||||
# DATA STORAGE
|
||||
# =============================================================================
|
||||
|
||||
@@ -7,6 +7,87 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.6.6] - 2026-11-02
|
||||
|
||||
### Fixed
|
||||
|
||||
- Crossposts left out photos that were embedded in the post text instead of attached. Some Micropub clients (OutPost, for one) upload a photo to the media endpoint and then put it in `content` as an image rather than sending it as `photo`; both are valid Micropub. Images in the text that are in this site's media library are now sent to GoToSocial, Bluesky and Threads like attached photos, with their alt text, after any attached ones and within each target's photo limit. GoToSocial's text no longer includes the image Markdown. Images hosted on other sites are still left out
|
||||
|
||||
## [2.6.5] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- A Micropub post that failed partway (for example, the same photo listed twice) answered "Failed to create post" but stayed published without its photos, so a client retrying would post it twice. Photos are now looked up before the post is made, and a post that fails after that is removed again
|
||||
- After such a failure, every request failed with "database is locked" for about 10 seconds: database connections now roll back unfinished writes when they're returned to the pool
|
||||
- The same photo listed twice in one post is shown once instead of failing the post
|
||||
- Replies didn't read as replies to IndieWeb readers and webmention receivers: on note pages `u-in-reply-to` sat inside an unrelated `h-cite`, and the home page didn't mark replies at all, so [Post Type Discovery](https://www.w3.org/TR/post-type-discovery/) saw notes (or photo posts). The reply context is now the entry's `u-in-reply-to h-cite`, and home page entries show "↩ In reply to"
|
||||
- Photos without alt text used "Image" as their description, which IndieWeb readers passed on as if you'd written it. They now use the post's text (up to 150 characters), and fall back to "Image" only when the post has no text
|
||||
- Home page entries always have an `e-content`, so a photo post without text never gets an implied name (which readers would show as its caption)
|
||||
|
||||
### Changed
|
||||
|
||||
- Photo posts no longer need text. Micropub doesn't require `content`, and `q=config` already offered a Photo post type with only `photo`. A Micropub update may also clear the text of a post that keeps its photos. Crossposts of a photo post without text start with its link, and its page is titled "Photo"
|
||||
|
||||
## [2.6.4] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- Tags keep the order you entered them in. The first tag is the note's main tag: it's the Threads topic, and Bluesky keeps the first tags when space runs out. Before, both used the alphabetically first tag. Public pages still list tags alphabetically
|
||||
- Opening a note in the editor or updating it via Micropub reordered its tags alphabetically on save; the editor and Micropub `q=source` now show, and keep, the entered order
|
||||
|
||||
## [2.6.3] - 2026-11-01
|
||||
|
||||
### Changed
|
||||
|
||||
- **A note's own link gets the preview card** (ADR-070). When a note links somewhere (say, an article), the crosspost's card shows that page instead of the note. Bluesky cards are built from the page's title, description and image; Threads previews it from the text. Notes without links keep their own card
|
||||
|
||||
### Fixed
|
||||
|
||||
- Markdown links (`[text](url)`) lost their URL on Bluesky and Threads. On Bluesky the link text is now clickable; on Threads it's written "text (url)"
|
||||
- Notes with many links failed on Threads (Meta allows 5 per post); extra links now keep just their text
|
||||
|
||||
## [2.6.2] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- New notes failed their first Threads crosspost with "Invalid Link Attachment" (Meta rejects an explicit link preview for a page it hasn't fetched yet) and only went out on the retry a minute later. StarPunk no longer sends the explicit link preview; Threads still shows one, built from the note's link in the text
|
||||
|
||||
## [2.6.1] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- Threads crossposts gave up after Meta's publish step returned a one-off `HTTP 400 Fatal`; publish errors are now retried (a retry checks for an existing post first, so nothing is posted twice). Only a bad token or missing permission stops retries
|
||||
- Threads errors now include Meta's error code, subcode, user message and trace id, so the editor shows why a post failed
|
||||
- Wait a moment before the first readiness check on a Threads post, since Meta can report it ready too early
|
||||
|
||||
## [2.6.0] - 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
- **Crossposting to Threads** (ADR-069): set `THREADS_ACCESS_TOKEN` (and optionally `THREADS_USERNAME`) to add a Threads target. Text posts get a link preview; 1 image posts as an image, 2–4 as a carousel, with alt text; the note's first tag becomes the post's topic. Shortened to 500 characters. Edits don't follow (the API can't edit); unpublish and delete remove the post. Likes aren't crossposted to Threads; reposts post "🔁 title url" (with the comment, if any)
|
||||
- The Threads token (60-day lifetime) is refreshed automatically once a day and kept in `data/threads_token.json`, so it survives restarts; setting a new token in the config replaces it. Setup steps: `docs/design/v2.6.0/`
|
||||
|
||||
## [2.5.0] - 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
- **Likes and reposts** via Micropub `like-of` / `repost-of` (ADR-068); a comment is optional
|
||||
- Likes are kept off the home page and feeds and listed on a new `/likes` page (linked in the header)
|
||||
- Reposts appear in the stream and feeds with a "🔁 Reposted" line
|
||||
- The liked/reposted post's title and author are fetched and shown; it is sent a webmention
|
||||
- GoToSocial: likes/reposts of fediverse posts become native favourites/boosts; reposts of other pages (or with a comment) post "🔁 title url"
|
||||
- Bluesky: likes/reposts of Bluesky posts become native likes/reposts; a repost with a comment is a quote post; reposts of other pages post with a link card
|
||||
- Likes of ordinary web pages are not crossposted; deleting a like/repost undoes it everywhere
|
||||
- Migration 013: `notes.like_of`, `notes.repost_of`
|
||||
|
||||
### Fixed
|
||||
|
||||
- Outgoing webmentions named posts `https://site//note/<slug>` (double slash)
|
||||
|
||||
### Notes
|
||||
|
||||
- The GoToSocial token needs the extra scopes `read:search` and `write:favourites`
|
||||
|
||||
## [2.4.0] - 2026-10-31
|
||||
|
||||
### Added
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
# ADR-068: Likes and reposts
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.5.0, 2026-11-01). Bookmarks were considered and declined.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Storage.** `notes.like_of` / `notes.repost_of` (migration 013). The note's
|
||||
content is an optional comment and may be empty.
|
||||
2. **Display.** Likes stay off the home page and feeds and are listed on
|
||||
`/likes`. Reposts appear in the stream and feeds. Both show the target's
|
||||
fetched title and author (the reply-context cache) and carry
|
||||
`u-like-of` / `u-repost-of`. The target is sent a webmention.
|
||||
3. **Crossposting, per target.**
|
||||
|
||||
| | Target is a post on that network | Anything else |
|
||||
|---|---|---|
|
||||
| Like | native favourite / like | nothing |
|
||||
| Repost, no comment | native boost / repost | "🔁 title url" post |
|
||||
| Repost, with comment | Bluesky quote post; GoToSocial status | comment + "🔁 title url" post |
|
||||
|
||||
GoToSocial finds fediverse posts with `/api/v2/search?resolve=true`
|
||||
(needs `read:search`; favourites need `write:favourites`). Bluesky
|
||||
recognises `bsky.app/profile/<actor>/post/<rkey>` URLs. Native actions
|
||||
are stored as `favourite:<id>` / `reblog:<id>` or the like/repost record
|
||||
URI, and deleting the note undoes them.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Editing a like/repost doesn't change a native favourite/boost.
|
||||
- A repost with a comment can't be a GoToSocial boost (no quote posts there).
|
||||
@@ -0,0 +1,57 @@
|
||||
# ADR-069: Crossposting to Threads
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.6.0, 2026-10-31)
|
||||
|
||||
## Context
|
||||
|
||||
ADR-065 and ADR-066 crosspost notes to GoToSocial and Bluesky. Threads
|
||||
(Meta) has a publishing API, so it can be a third target on the same loop.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Third client, same loop.** `ThreadsClient` is target id `threads`
|
||||
behind the `syndications` table and reconcile loop from ADR-065.
|
||||
2. **Auth.** A long-lived Threads user token (`THREADS_ACCESS_TOKEN`) with
|
||||
`threads_basic`, `threads_content_publish` and `threads_delete`. These
|
||||
tokens expire after 60 days, so StarPunk refreshes the token about once a
|
||||
day (`th_refresh_token`) from the worker loop, even when nothing is being
|
||||
posted. The current token is kept in `DATA_PATH/threads_token.json`
|
||||
(mode 0600) with a hash of the configured token. A different configured
|
||||
token replaces the saved one. A file lock stops gunicorn workers from
|
||||
refreshing at the same time. The file avoids a migration for one value.
|
||||
3. **Posting.** Create a container, poll its status until `FINISHED`, then
|
||||
`threads_publish`. Text-only notes are `TEXT` posts. Threads previews
|
||||
the first link in the text (the permalink) by itself. An explicit
|
||||
`link_attachment` is not sent, because Meta rejects one for a page it
|
||||
hasn't fetched yet ("Invalid Link Attachment", subcode 4279047), which
|
||||
is always the case for a new note (v2.6.2). One image is an `IMAGE` post and 2–4 images are a
|
||||
`CAROUSEL`. Meta fetches images from their public `/media/` URL. Only
|
||||
JPEG and PNG up to 8 MB are allowed, so GIF and WebP images are left off.
|
||||
Alt text goes in `alt_text`.
|
||||
4. **Content.** Plain text (as for Bluesky), then the permalink, shortened
|
||||
to 500 characters. Non-ASCII characters are counted as UTF-8 bytes,
|
||||
because Threads counts emoji that way. Threads allows one tag per post,
|
||||
so the note's first tag is sent as `topic_tag` and no hashtags are added
|
||||
to the text.
|
||||
5. **Edits don't follow.** The API has no way to edit a post. As with
|
||||
Bluesky, the post stays as first sent, and unpublishing, unticking the
|
||||
target or deleting the note removes it.
|
||||
6. **Likes and reposts (ADR-068).** The API can't like or repost outside
|
||||
content, so likes are not crossposted. Reposts are text posts with
|
||||
"🔁 title" and the URL, plus the comment if there is one, and the
|
||||
reposted page as the link preview.
|
||||
7. **No double posts.** A retried create first looks for a recent post that
|
||||
already contains the permalink.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Meta limits accounts to 250 posts and 100 deletes per day. Rate-limit
|
||||
errors (Graph codes 4, 17, 32, 613) are retried.
|
||||
- If StarPunk is down for more than 60 days, the token expires and a new
|
||||
one is needed.
|
||||
- Getting the first token requires a Meta developer app with the Threads
|
||||
use case. See the setup notes in `docs/design/v2.6.0/`.
|
||||
- Notes with more than five links fail on Threads (a Meta limit). The
|
||||
error is shown in the editor.
|
||||
@@ -0,0 +1,44 @@
|
||||
# ADR-070: Links and link cards in crossposts
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.6.3, 2026-11-01)
|
||||
|
||||
## Context
|
||||
|
||||
Every crosspost carries the note's permalink. Notes often link elsewhere
|
||||
too, usually to the article being discussed. Before this change:
|
||||
|
||||
- Bluesky and Threads get plain text, and a Markdown link `[text](url)`
|
||||
lost its URL, so readers there couldn't follow it.
|
||||
- The Bluesky card always showed the note, and the Threads preview showed
|
||||
whichever link came first.
|
||||
- Threads rejects posts with more than 5 different links.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **The card shows what the note links to.** If the note has its own link
|
||||
(a Markdown link, `<url>` or a bare URL), the first one gets the
|
||||
preview card. A note without links gets a card for itself, as before.
|
||||
Images still take the place of a card on Bluesky.
|
||||
2. **Bluesky.** The card is built by StarPunk: it fetches the page's Open
|
||||
Graph, Twitter or plain HTML title, description and image, and uploads
|
||||
the image (shrunk to under 1 MB) as the thumbnail. If the page can't
|
||||
be fetched, the card has just the URL. Markdown link text becomes a
|
||||
clickable link facet, so no characters are spent on the URL.
|
||||
3. **Threads.** It has no clickable link text, so Markdown links are
|
||||
written "text (url)". Threads previews the first link in the text, which
|
||||
is now the linked page. To stay within Meta's 5-link limit, after 4
|
||||
different links in the body a Markdown link keeps only its text.
|
||||
4. **GoToSocial** already gets Markdown, so its links are unchanged.
|
||||
Receiving servers build their own cards.
|
||||
5. **Fetching is careful.** Only http(s) URLs on public addresses are
|
||||
fetched, and this is checked again at every redirect. At most 512 KB of
|
||||
a page and 5 MB of an image are read. A failed fetch never fails the
|
||||
post.
|
||||
|
||||
## Consequences
|
||||
|
||||
- One or two extra requests to the linked site when a note is crossposted
|
||||
to Bluesky.
|
||||
- Reposts (ADR-068) are unchanged: their card is the reposted page.
|
||||
@@ -0,0 +1,62 @@
|
||||
# v2.6.0: Crossposting to Threads
|
||||
|
||||
Implements ADR-069.
|
||||
|
||||
## What changed
|
||||
|
||||
- `starpunk/syndication.py`: `ThreadsClient` (target `threads`); media
|
||||
entries now carry their public `url`; `process_due()` calls each client's
|
||||
optional `maintain()` hook, which Threads uses to refresh its token.
|
||||
- `starpunk/config.py`: `THREADS_ACCESS_TOKEN`, `THREADS_USERNAME`,
|
||||
`THREADS_NAME`, `THREADS_MAX_CHARS` (500), `THREADS_MAX_MEDIA` (4).
|
||||
- `tests/test_syndication_threads.py`: a fake Graph API covering text,
|
||||
image and carousel posts; container polling; edits not following; deletes
|
||||
(including posts already deleted by hand); a lost publish reply;
|
||||
permanent and rate-limit errors; and token refresh, persistence and
|
||||
replacement.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Variable | Meaning |
|
||||
|---|---|
|
||||
| `THREADS_ACCESS_TOKEN` | Long-lived Threads user token. Enables the target. |
|
||||
| `THREADS_USERNAME` | Optional. Used for the Micropub `syndicate-to` uid (`https://www.threads.com/@name`) and for fallback links. |
|
||||
| `THREADS_NAME` | Label in the editor (default `Threads`) |
|
||||
|
||||
After the first daily refresh, the working token lives in
|
||||
`DATA_PATH/threads_token.json`. The configured value only seeds it.
|
||||
Changing `THREADS_ACCESS_TOKEN` switches to the new token.
|
||||
|
||||
## Getting a token (one time)
|
||||
|
||||
1. At <https://developers.facebook.com/apps>, create an app with the
|
||||
**Access the Threads API** use case.
|
||||
2. Under the use case's permissions, add `threads_basic`,
|
||||
`threads_content_publish` and `threads_delete`.
|
||||
3. Under **App roles → Roles**, add your Threads account as a Threads
|
||||
tester. Accept the invite in Threads (Settings → Account → Website
|
||||
permissions → Invites). The app can stay in development mode.
|
||||
4. In the use case settings, add `https://<site>/` as a redirect URI. Then
|
||||
open:
|
||||
|
||||
https://threads.net/oauth/authorize?client_id=APP_ID&redirect_uri=https://<site>/&scope=threads_basic,threads_content_publish,threads_delete&response_type=code
|
||||
|
||||
After you approve, the browser lands on `https://<site>/?code=...`. Copy
|
||||
the code (drop a trailing `#_`).
|
||||
5. Exchange the code for a short-lived token, then exchange that for a
|
||||
long-lived one:
|
||||
|
||||
curl -s -X POST https://graph.threads.net/oauth/access_token \
|
||||
-F client_id=APP_ID -F client_secret=APP_SECRET \
|
||||
-F grant_type=authorization_code -F redirect_uri=https://<site>/ -F code=CODE
|
||||
curl -s "https://graph.threads.net/access_token?grant_type=th_exchange_token&client_secret=APP_SECRET&access_token=SHORT_TOKEN"
|
||||
|
||||
6. Set `THREADS_ACCESS_TOKEN` to the long-lived `access_token`.
|
||||
|
||||
The app secret is only needed for these steps. StarPunk doesn't use it.
|
||||
|
||||
## Not done
|
||||
|
||||
- Video, GIF and WebP images (Threads takes video by URL. StarPunk only
|
||||
stores still images, and Threads rejects GIF and WebP).
|
||||
- Edits (no API for them).
|
||||
@@ -0,0 +1,89 @@
|
||||
# v2.6.5: Micropub media fixes
|
||||
|
||||
Found while adding media uploads to tuipub, a Micropub client written to the
|
||||
spec rather than to StarPunk.
|
||||
|
||||
## Problems
|
||||
|
||||
1. **Failed creates left a post behind.** `handle_create` made the note, then
|
||||
attached photos. Listing one photo twice broke the `note_media` unique key,
|
||||
so the client got a 500 while the note stayed published without photos.
|
||||
A retry would post it twice.
|
||||
2. **Then the database stayed locked.** `attach_media_to_note` didn't roll
|
||||
back, and `ConnectionPool.return_connection` pooled the connection with
|
||||
its write transaction still open. Other requests got "database is locked"
|
||||
until the 10 s timeout.
|
||||
3. **Photo posts needed text.** The Micropub spec doesn't require `content`,
|
||||
and `q=config` advertised a `photo` post type with only `photo`, yet
|
||||
creates without content got 400 "Content is required".
|
||||
|
||||
## Changes
|
||||
|
||||
- `database/pool.py`: `return_connection` rolls back an open transaction
|
||||
(or closes the connection if that fails).
|
||||
- `media.py`: `attach_media_to_note` rolls back on error.
|
||||
- `micropub.py`:
|
||||
- `handle_create` resolves photos (by media URL, downloading remote ones)
|
||||
before `create_note`. If attaching or crosspost selection fails, it
|
||||
hard-deletes the note before answering 500.
|
||||
- `_resolve_photos` skips repeated media.
|
||||
- Content is optional when photos resolve. Content-less posts whose photos
|
||||
all fail still get a 400 from `create_note`.
|
||||
- `handle_update` allows `replace: {content: [""]}` and `delete: ["content"]`
|
||||
while the note keeps photos (or is a like or repost). Otherwise it answers
|
||||
400 "A post needs text or a photo".
|
||||
- `notes.py`: `create_note` and `update_note` take `has_media`, so
|
||||
empty content is allowed for photo posts.
|
||||
- `syndication.py`: `compose_text` drops the leading blank line when there's
|
||||
no body.
|
||||
|
||||
## Post types (IndieWeb standards check)
|
||||
|
||||
Post Type Discovery (W3C Note) has no separate "photo" type to store. A post
|
||||
with a valid `photo` is a photo post, unless it's first a repost, like or
|
||||
reply (step order: repost, like, reply, video, photo). Otherwise it's a note,
|
||||
or an article when its `name` isn't a prefix of its content. h-entry treats
|
||||
`u-photo` as the primary content, with `name` as a caption. mf2 implies a
|
||||
`name` only when an entry has no other `p-*`/`e-*` properties and no nested
|
||||
microformats.
|
||||
|
||||
StarPunk already stores photo posts as notes with media and renders
|
||||
`u-photo` outside `e-content`, which is what the standards ask for. Checking
|
||||
the rendered pages with mf2py found:
|
||||
|
||||
- **Replies weren't replies.** In `note.html`, `u-in-reply-to` was inside a
|
||||
bare `h-cite`, so it was the h-cite's property, not the entry's.
|
||||
`note_preview.html` had no reply markup at all. Fixed: the blockquote is
|
||||
`u-in-reply-to h-cite` (with `u-url` inside), and previews show
|
||||
"↩ In reply to" with `u-in-reply-to`.
|
||||
- **Implied names were possible.** Previews left out `e-content` for posts
|
||||
without text. They worked only because the author h-card blocked an
|
||||
implied name. Previews now always emit `e-content`.
|
||||
- Page title for text-free posts follows PTD order: Repost, Like, Reply,
|
||||
Photo.
|
||||
|
||||
Photos without a caption take the post's text as alt (`partials/media.html`
|
||||
`fallback_alt`) instead of the made-up "Image". The Micropub `q=source`
|
||||
still reports the stored caption (`""`).
|
||||
|
||||
`tests/test_post_types.py` parses note pages and the home page with mf2py and
|
||||
runs PTD for notes, articles, photo posts (no text, caption, title), replies
|
||||
(with and without a photo) and reposts.
|
||||
|
||||
## Not changed (noted for later)
|
||||
|
||||
- Text-free photos without alt text still render `alt="Image"`. Captioned
|
||||
photos use their caption, and others now use the post's text (`display_media`
|
||||
`fallback_alt`, the excerpt cut to 150 characters).
|
||||
|
||||
- More than 4 photos are still cut to 4 without an error (ADR-057).
|
||||
- `video` and `audio` are ignored. This is allowed by the spec, and they
|
||||
aren't in `q=config` post-types.
|
||||
- Media endpoint errors for non-images expose Python text
|
||||
("cannot identify image file <_io.BytesIO …>").
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/test_micropub_v265.py`. The full suite runs with
|
||||
`SESSION_SECRET=test`. 8 tests in `test_media_upload.py` and
|
||||
`test_migration_race_condition.py` already fail on main, and still fail here.
|
||||
@@ -0,0 +1,8 @@
|
||||
-- Migration 013: Likes and reposts (v2.5.0, ADR-068)
|
||||
-- A note with like_of is a like; with repost_of, a repost. Content is the
|
||||
-- optional comment and may be empty.
|
||||
|
||||
ALTER TABLE notes ADD COLUMN like_of TEXT;
|
||||
ALTER TABLE notes ADD COLUMN repost_of TEXT;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_notes_like_of ON notes(like_of);
|
||||
@@ -464,5 +464,5 @@ def create_app(config=None):
|
||||
|
||||
# Package version (Semantic Versioning 2.0.0)
|
||||
# See docs/standards/versioning-strategy.md for details
|
||||
__version__ = "2.4.0"
|
||||
__version_info__ = (2, 1, 0)
|
||||
__version__ = "2.6.6"
|
||||
__version_info__ = (2, 6, 4)
|
||||
@@ -137,6 +137,12 @@ def load_config(app, config_override=None):
|
||||
app.config["BLUESKY_SERVICE"] = os.getenv("BLUESKY_SERVICE", "https://bsky.social")
|
||||
app.config["BLUESKY_NAME"] = os.getenv("BLUESKY_NAME", "Bluesky")
|
||||
app.config["BLUESKY_MAX_CHARS"] = int(os.getenv("BLUESKY_MAX_CHARS", "300"))
|
||||
# Crossposting to Threads (v2.6.0, ADR-069): long-lived token, refreshed automatically
|
||||
app.config["THREADS_ACCESS_TOKEN"] = os.getenv("THREADS_ACCESS_TOKEN", "")
|
||||
app.config["THREADS_USERNAME"] = os.getenv("THREADS_USERNAME", "")
|
||||
app.config["THREADS_NAME"] = os.getenv("THREADS_NAME", "Threads")
|
||||
app.config["THREADS_MAX_CHARS"] = int(os.getenv("THREADS_MAX_CHARS", "500"))
|
||||
app.config["THREADS_MAX_MEDIA"] = int(os.getenv("THREADS_MAX_MEDIA", "4"))
|
||||
app.config["SYNDICATION_POLL_INTERVAL"] = int(os.getenv("SYNDICATION_POLL_INTERVAL", "10"))
|
||||
|
||||
# Apply overrides if provided
|
||||
|
||||
@@ -107,6 +107,16 @@ class ConnectionPool:
|
||||
if not conn:
|
||||
return
|
||||
|
||||
# A request that failed mid-write must not hand an open transaction
|
||||
# (and its write lock) to the next request (v2.6.5)
|
||||
if conn.in_transaction:
|
||||
try:
|
||||
conn.rollback()
|
||||
except Exception:
|
||||
conn.close()
|
||||
self._stats['connections_closed'] += 1
|
||||
return
|
||||
|
||||
with self._lock:
|
||||
if len(self._pool) < self.pool_size:
|
||||
# Return to pool
|
||||
|
||||
@@ -198,7 +198,7 @@ def generate_atom_streaming(
|
||||
yield f' <link rel="enclosure" type="{_escape_xml(mime_type)}" href="{_escape_xml(media_url)}" length="{size}"{title_attr}/>\n'
|
||||
|
||||
# Content - include media as HTML (per Q24)
|
||||
if note.html:
|
||||
if note.feed_html:
|
||||
# Build HTML content with media at top
|
||||
html_content = ""
|
||||
|
||||
@@ -212,7 +212,7 @@ def generate_atom_streaming(
|
||||
html_content += '</div>'
|
||||
|
||||
# Add text content below media
|
||||
html_content += note.html
|
||||
html_content += note.feed_html
|
||||
|
||||
# HTML content - escaped
|
||||
yield ' <content type="html">'
|
||||
|
||||
@@ -267,7 +267,7 @@ def _build_item_object(site_url: str, note: Note) -> Dict[str, Any]:
|
||||
|
||||
# Add content (HTML or text)
|
||||
# Per Q24: Include media as HTML in content_html
|
||||
if note.html:
|
||||
if note.feed_html:
|
||||
content_html = ""
|
||||
|
||||
# Add media at top if present (v1.2.0 Phase 3)
|
||||
@@ -280,7 +280,7 @@ def _build_item_object(site_url: str, note: Note) -> Dict[str, Any]:
|
||||
content_html += '</div>'
|
||||
|
||||
# Add text content below media
|
||||
content_html += note.html
|
||||
content_html += note.feed_html
|
||||
item["content_html"] = content_html
|
||||
else:
|
||||
item["content_text"] = note.content
|
||||
|
||||
@@ -137,7 +137,7 @@ def generate_rss(
|
||||
html_content += '</div>'
|
||||
|
||||
# Add text content below media
|
||||
html_content += clean_html_for_rss(note.html)
|
||||
html_content += clean_html_for_rss(note.feed_html)
|
||||
|
||||
# feedgen automatically wraps content in CDATA for RSS
|
||||
fe.description(html_content)
|
||||
@@ -278,7 +278,7 @@ def generate_rss_streaming(
|
||||
html_content += '</div>'
|
||||
|
||||
# Add text content below media
|
||||
html_content += clean_html_for_rss(note.html)
|
||||
html_content += clean_html_for_rss(note.feed_html)
|
||||
|
||||
# Build item XML
|
||||
item_xml = f""" <item>
|
||||
|
||||
+16
-12
@@ -868,20 +868,24 @@ def attach_media_to_note(note_id: int, media_ids: List[int], captions: List[str]
|
||||
|
||||
db = get_db(current_app)
|
||||
|
||||
# Delete existing associations (for edit case)
|
||||
db.execute("DELETE FROM note_media WHERE note_id = ?", (note_id,))
|
||||
try:
|
||||
# Delete existing associations (for edit case)
|
||||
db.execute("DELETE FROM note_media WHERE note_id = ?", (note_id,))
|
||||
|
||||
# Insert new associations
|
||||
for i, (media_id, caption) in enumerate(zip(media_ids, captions)):
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO note_media (note_id, media_id, display_order, caption)
|
||||
VALUES (?, ?, ?, ?)
|
||||
""",
|
||||
(note_id, media_id, i, caption or None)
|
||||
)
|
||||
# Insert new associations
|
||||
for i, (media_id, caption) in enumerate(zip(media_ids, captions)):
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO note_media (note_id, media_id, display_order, caption)
|
||||
VALUES (?, ?, ?, ?)
|
||||
""",
|
||||
(note_id, media_id, i, caption or None)
|
||||
)
|
||||
|
||||
db.commit()
|
||||
db.commit()
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
|
||||
|
||||
def get_note_media(note_id: int) -> List[Dict]:
|
||||
|
||||
+71
-14
@@ -195,6 +195,16 @@ def extract_content(properties: dict) -> str:
|
||||
return content.strip()
|
||||
|
||||
|
||||
def _has_text(content) -> bool:
|
||||
"""Whether a content property has any text. Likes, reposts (v2.5.0) and
|
||||
photo posts (v2.6.5) may come without."""
|
||||
values = content if isinstance(content, list) else [content]
|
||||
return any(
|
||||
(c.strip() if isinstance(c, str) else (c.get("text") or c.get("html") or "").strip())
|
||||
for c in values if c
|
||||
)
|
||||
|
||||
|
||||
def extract_title(properties: dict) -> Optional[str]:
|
||||
"""
|
||||
Extract the explicit title (Micropub 'name' property)
|
||||
@@ -296,16 +306,21 @@ def extract_post_status(properties: dict) -> bool:
|
||||
raise MicropubValidationError(f"Unsupported post-status: {status}")
|
||||
|
||||
|
||||
def extract_in_reply_to(properties: dict) -> Optional[str]:
|
||||
value = _first(properties.get("in-reply-to"))
|
||||
def extract_url_property(properties: dict, name: str) -> Optional[str]:
|
||||
"""A URL property (in-reply-to, like-of, repost-of): string or h-cite"""
|
||||
value = _first(properties.get(name))
|
||||
if isinstance(value, dict): # h-cite
|
||||
value = _first((value.get("properties") or {}).get("url")) or value.get("value")
|
||||
value = (value or "").strip() if isinstance(value, str) else ""
|
||||
if value and not value.startswith(("http://", "https://")):
|
||||
raise MicropubValidationError("in-reply-to must be an http(s) URL")
|
||||
raise MicropubValidationError(f"{name} must be an http(s) URL")
|
||||
return value or None
|
||||
|
||||
|
||||
def extract_in_reply_to(properties: dict) -> Optional[str]:
|
||||
return extract_url_property(properties, "in-reply-to")
|
||||
|
||||
|
||||
def extract_photos(properties: dict) -> list[dict[str, str]]:
|
||||
"""
|
||||
Extract photo URLs and alt text from Micropub properties
|
||||
@@ -415,7 +430,8 @@ def _resolve_photos(photos: list[dict]) -> tuple[list[int], list[str]]:
|
||||
if len(media_ids) == MAX_PHOTOS:
|
||||
break
|
||||
media_id = photo.get("media_id") or _media_id_for_url(photo["url"])
|
||||
if media_id:
|
||||
# The same photo twice is shown once (v2.6.5)
|
||||
if media_id and media_id not in media_ids:
|
||||
media_ids.append(media_id)
|
||||
captions.append(photo.get("alt", ""))
|
||||
return media_ids, captions
|
||||
@@ -516,19 +532,33 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
|
||||
try:
|
||||
properties = normalize_properties(data)
|
||||
content = extract_content(properties)
|
||||
like_of = extract_url_property(properties, "like-of")
|
||||
repost_of = extract_url_property(properties, "repost-of")
|
||||
if like_of and repost_of:
|
||||
raise MicropubValidationError("A post can't be both a like and a repost")
|
||||
has_content = _has_text(properties.get("content"))
|
||||
photos = extract_photos(properties) + _save_uploads(files)
|
||||
# Photo posts may have no text (v2.6.5): Micropub doesn't require content
|
||||
content = (
|
||||
extract_content(properties)
|
||||
if has_content or not (like_of or repost_of or photos)
|
||||
else ""
|
||||
)
|
||||
title = extract_title(properties)
|
||||
tags = extract_tags(properties)
|
||||
published_date = extract_published_date(properties)
|
||||
published = extract_post_status(properties)
|
||||
in_reply_to = extract_in_reply_to(properties)
|
||||
photos = extract_photos(properties) + _save_uploads(files)
|
||||
# Look up and download photos before the note exists, so a bad photo
|
||||
# can't leave a half-made post behind (v2.6.5)
|
||||
media_ids, captions = _resolve_photos(photos)
|
||||
except MicropubError:
|
||||
raise
|
||||
except Exception as e:
|
||||
current_app.logger.error(f"Property extraction failed: {e}")
|
||||
raise MicropubValidationError(f"Failed to parse request: {str(e)}")
|
||||
|
||||
note = None
|
||||
try:
|
||||
note = create_note(
|
||||
content=add_title_heading(content, title),
|
||||
@@ -537,10 +567,14 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
custom_slug=custom_slug,
|
||||
tags=tags if tags else None,
|
||||
in_reply_to=in_reply_to,
|
||||
like_of=like_of,
|
||||
repost_of=repost_of,
|
||||
has_media=bool(media_ids),
|
||||
)
|
||||
|
||||
if photos:
|
||||
_attach_photos_to_note(note.id, photos)
|
||||
if media_ids:
|
||||
from starpunk.media import attach_media_to_note
|
||||
attach_media_to_note(note.id, media_ids, captions)
|
||||
|
||||
# Crosspost after photos are attached (v2.1.0); drafts wait until published
|
||||
from starpunk.syndication import get_targets, resolve_targets, select_targets
|
||||
@@ -557,6 +591,13 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
raise
|
||||
except Exception as e:
|
||||
current_app.logger.error(f"Failed to create note via Micropub: {e}")
|
||||
# The client is told it failed, so don't leave the post up (v2.6.5)
|
||||
if note is not None:
|
||||
try:
|
||||
from starpunk.notes import delete_note
|
||||
delete_note(id=note.id, soft=False)
|
||||
except Exception as cleanup:
|
||||
current_app.logger.error(f"Could not remove failed note {note.slug}: {cleanup}")
|
||||
raise MicropubError("server_error", "Failed to create post", status_code=500)
|
||||
|
||||
|
||||
@@ -587,8 +628,10 @@ def handle_update(data: dict, token_info: dict):
|
||||
if set(add) - {"category", "photo"}:
|
||||
raise MicropubValidationError("Only category and photo can be added to")
|
||||
|
||||
from starpunk.tags import get_note_tags
|
||||
|
||||
title, body = split_title(note.content)
|
||||
tags = [t["display_name"] for t in note.tags]
|
||||
tags = [t["display_name"] for t in get_note_tags(note.id, entered_order=True)]
|
||||
photos = _note_photos(note)
|
||||
changes = {}
|
||||
|
||||
@@ -598,7 +641,7 @@ def handle_update(data: dict, token_info: dict):
|
||||
if prop == "name":
|
||||
title = None
|
||||
elif prop == "content":
|
||||
raise MicropubValidationError("content can't be deleted")
|
||||
body = ""
|
||||
elif prop == "category":
|
||||
tags = []
|
||||
elif prop == "photo":
|
||||
@@ -619,7 +662,8 @@ def handle_update(data: dict, token_info: dict):
|
||||
raise MicropubValidationError(f"Can't delete individual values of {prop}")
|
||||
|
||||
if "content" in replace:
|
||||
body = extract_content(replace)
|
||||
# Empty text is fine if the note keeps its photos; update_note checks (v2.6.5)
|
||||
body = extract_content(replace) if _has_text(replace["content"]) else ""
|
||||
if "name" in replace:
|
||||
title = extract_title(replace)
|
||||
if "category" in replace:
|
||||
@@ -637,6 +681,10 @@ def handle_update(data: dict, token_info: dict):
|
||||
if "photo" in add:
|
||||
photos += extract_photos(add)
|
||||
|
||||
# Text may go only if photos stay (or it's a like or repost) (v2.6.5)
|
||||
if not (body or "").strip() and not (photos or note.like_of or note.repost_of):
|
||||
raise MicropubValidationError("A post needs text or a photo")
|
||||
|
||||
new_content = add_title_heading(body, title)
|
||||
if new_content != note.content:
|
||||
changes["content"] = new_content
|
||||
@@ -645,7 +693,7 @@ def handle_update(data: dict, token_info: dict):
|
||||
|
||||
try:
|
||||
if changes:
|
||||
update_note(id=note.id, **changes)
|
||||
update_note(id=note.id, has_media=bool(photos), **changes)
|
||||
if "photo" in touched:
|
||||
_attach_photos_to_note(note.id, photos)
|
||||
note_changed(note.id)
|
||||
@@ -701,10 +749,17 @@ def note_to_mf2(note, wanted: Optional[list] = None) -> dict:
|
||||
}
|
||||
if title:
|
||||
props["name"] = [title]
|
||||
if note.tags:
|
||||
props["category"] = [tag["display_name"] for tag in note.tags]
|
||||
from starpunk.tags import get_note_tags
|
||||
|
||||
tags = get_note_tags(note.id, entered_order=True)
|
||||
if tags:
|
||||
props["category"] = [tag["display_name"] for tag in tags]
|
||||
if getattr(note, "in_reply_to", None):
|
||||
props["in-reply-to"] = [note.in_reply_to]
|
||||
if getattr(note, "like_of", None):
|
||||
props["like-of"] = [note.like_of]
|
||||
if getattr(note, "repost_of", None):
|
||||
props["repost-of"] = [note.repost_of]
|
||||
photos = _note_photos(note)
|
||||
if photos:
|
||||
props["photo"] = [{"value": p["url"], "alt": p["alt"]} for p in photos]
|
||||
@@ -768,6 +823,8 @@ def handle_query(args, token_info: dict):
|
||||
{"type": "article", "name": "Article", "properties": ["name", "content"]},
|
||||
{"type": "photo", "name": "Photo", "properties": ["photo"]},
|
||||
{"type": "reply", "name": "Reply", "properties": ["in-reply-to", "content"]},
|
||||
{"type": "like", "name": "Like", "properties": ["like-of"]},
|
||||
{"type": "repost", "name": "Repost", "properties": ["repost-of", "content"]},
|
||||
],
|
||||
}), 200
|
||||
|
||||
|
||||
@@ -134,6 +134,8 @@ class Note:
|
||||
deleted_at: Optional[datetime] = None
|
||||
content_hash: Optional[str] = None
|
||||
in_reply_to: Optional[str] = None # v1.6.0: URL being replied to
|
||||
like_of: Optional[str] = None # v2.5.0: URL liked (this note is a like)
|
||||
repost_of: Optional[str] = None # v2.5.0: URL reposted (this note is a repost)
|
||||
_cached_content: Optional[str] = field(
|
||||
default=None, repr=False, compare=False, init=False
|
||||
)
|
||||
@@ -192,6 +194,8 @@ class Note:
|
||||
_data_dir=data_dir,
|
||||
content_hash=data.get("content_hash"),
|
||||
in_reply_to=data.get("in_reply_to"),
|
||||
like_of=data.get("like_of"),
|
||||
repost_of=data.get("repost_of"),
|
||||
)
|
||||
|
||||
@property
|
||||
@@ -367,12 +371,43 @@ class Note:
|
||||
excerpt = excerpt[:last_space]
|
||||
excerpt += "..."
|
||||
|
||||
if not excerpt and self.reacts_to:
|
||||
# A like or repost with no comment
|
||||
verb = "Liked" if self.kind == "like" else "Reposted"
|
||||
return f"{verb} {self.reacts_to}"
|
||||
|
||||
return excerpt
|
||||
|
||||
except (FileNotFoundError, OSError):
|
||||
# If file doesn't exist, return slug
|
||||
return self.slug
|
||||
|
||||
@property
|
||||
def kind(self) -> str:
|
||||
"""'like', 'repost' or 'note' (v2.5.0)"""
|
||||
if self.like_of:
|
||||
return "like"
|
||||
if self.repost_of:
|
||||
return "repost"
|
||||
return "note"
|
||||
|
||||
@property
|
||||
def reacts_to(self) -> Optional[str]:
|
||||
"""The URL a like or repost points at"""
|
||||
return self.like_of or self.repost_of
|
||||
|
||||
@property
|
||||
def feed_html(self) -> str:
|
||||
"""HTML for feeds: a repost/like line, then the note's own HTML"""
|
||||
from html import escape
|
||||
|
||||
if not self.reacts_to:
|
||||
return self.html
|
||||
verb = "Liked" if self.kind == "like" else "Reposted"
|
||||
url = escape(self.reacts_to, quote=True)
|
||||
line = f'<p>{"★" if self.kind == "like" else "🔁"} {verb} <a href="{url}">{url}</a></p>'
|
||||
return line + (self.html or "")
|
||||
|
||||
@property
|
||||
def permalink(self) -> str:
|
||||
"""
|
||||
|
||||
+41
-21
@@ -139,7 +139,10 @@ def create_note(
|
||||
created_at: Optional[datetime] = None,
|
||||
custom_slug: Optional[str] = None,
|
||||
tags: Optional[list[str]] = None,
|
||||
in_reply_to: Optional[str] = None
|
||||
in_reply_to: Optional[str] = None,
|
||||
like_of: Optional[str] = None,
|
||||
repost_of: Optional[str] = None,
|
||||
has_media: bool = False,
|
||||
) -> Note:
|
||||
"""
|
||||
Create a new note
|
||||
@@ -203,7 +206,11 @@ def create_note(
|
||||
- created_at and updated_at set to same value initially
|
||||
"""
|
||||
# 1. VALIDATION (before any changes)
|
||||
if not content or not content.strip():
|
||||
# Likes and reposts may have no comment (v2.5.0), photo posts no text (v2.6.5)
|
||||
if like_of and repost_of:
|
||||
raise InvalidNoteDataError("like_of", like_of, "A note can't be both a like and a repost")
|
||||
content = content or ""
|
||||
if not content.strip() and not (like_of or repost_of or has_media):
|
||||
raise InvalidNoteDataError(
|
||||
"content", content, "Content cannot be empty or whitespace-only"
|
||||
)
|
||||
@@ -264,10 +271,10 @@ def create_note(
|
||||
try:
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO notes (slug, file_path, published, created_at, updated_at, content_hash, in_reply_to)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
INSERT INTO notes (slug, file_path, published, created_at, updated_at, content_hash, in_reply_to, like_of, repost_of)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(slug, file_path_rel, published, created_at, updated_at, content_hash, in_reply_to),
|
||||
(slug, file_path_rel, published, created_at, updated_at, content_hash, in_reply_to, like_of, repost_of),
|
||||
)
|
||||
db.commit()
|
||||
except Exception as e:
|
||||
@@ -322,33 +329,36 @@ def create_note(
|
||||
from starpunk.webmention_sender import queue_webmentions_for_note
|
||||
db_path = Path(current_app.config["DATABASE_PATH"])
|
||||
site_url = current_app.config["SITE_URL"]
|
||||
note_url = f"{site_url}/note/{slug}"
|
||||
from starpunk.utils import absolute_url_for
|
||||
note_url = absolute_url_for("public.note", slug=slug)
|
||||
|
||||
# Get rendered HTML from note
|
||||
html_content = note.html
|
||||
|
||||
# Queue webmentions (best-effort, failures logged but don't fail note creation)
|
||||
# The reply/like/repost target is notified first (v2.5.0)
|
||||
queue_webmentions_for_note(
|
||||
db_path=db_path,
|
||||
note_id=note_id,
|
||||
note_url=note_url,
|
||||
content_html=html_content,
|
||||
in_reply_to=in_reply_to
|
||||
in_reply_to=in_reply_to or like_of or repost_of
|
||||
)
|
||||
except Exception as e:
|
||||
# Webmention queueing failure should not prevent note creation
|
||||
current_app.logger.warning(f"Failed to queue webmentions for note {slug}: {e}")
|
||||
|
||||
# 12. FETCH REPLY CONTEXT (v1.6.0 Phase 3)
|
||||
# Fetch context if this is a reply
|
||||
if in_reply_to:
|
||||
# Fetch context for the post being replied to, liked or reposted
|
||||
context_url = in_reply_to or like_of or repost_of
|
||||
if context_url:
|
||||
try:
|
||||
from starpunk.reply_context import fetch_reply_context
|
||||
db_path = Path(current_app.config["DATABASE_PATH"])
|
||||
site_url = current_app.config["SITE_URL"]
|
||||
|
||||
# Fetch context (best-effort, failures logged but don't fail note creation)
|
||||
fetch_reply_context(db_path, in_reply_to, site_url)
|
||||
fetch_reply_context(db_path, context_url, site_url)
|
||||
except Exception as e:
|
||||
# Reply context fetch failure should not prevent note creation
|
||||
current_app.logger.warning(f"Failed to fetch reply context for {in_reply_to}: {e}")
|
||||
@@ -472,6 +482,7 @@ def list_notes(
|
||||
offset: int = 0,
|
||||
order_by: str = "created_at",
|
||||
order_dir: str = "DESC",
|
||||
likes: Optional[bool] = None,
|
||||
) -> list[Note]:
|
||||
"""
|
||||
List notes with filtering and pagination
|
||||
@@ -564,6 +575,11 @@ def list_notes(
|
||||
params = []
|
||||
if published_only:
|
||||
query += " AND published = 1"
|
||||
# likes: None = all notes, False = leave likes out, True = only likes (v2.5.0)
|
||||
if likes is True:
|
||||
query += " AND like_of IS NOT NULL"
|
||||
elif likes is False:
|
||||
query += " AND like_of IS NULL"
|
||||
|
||||
# Add ordering (safe because order_by validated above)
|
||||
query += f" ORDER BY {order_by} {order_dir}"
|
||||
@@ -589,7 +605,8 @@ def update_note(
|
||||
content: Optional[str] = None,
|
||||
published: Optional[bool] = None,
|
||||
tags: Optional[list[str]] = None,
|
||||
in_reply_to: Optional[str] = None
|
||||
in_reply_to: Optional[str] = None,
|
||||
has_media: bool = False,
|
||||
) -> Note:
|
||||
"""
|
||||
Update a note's content and/or published status
|
||||
@@ -663,12 +680,9 @@ def update_note(
|
||||
if content is None and published is None and tags is None and in_reply_to is None:
|
||||
raise ValueError("Must provide at least one of content, published, tags, or in_reply_to to update")
|
||||
|
||||
# Validate content if provided
|
||||
if content is not None:
|
||||
if not content or not content.strip():
|
||||
raise InvalidNoteDataError(
|
||||
"content", content, "Content cannot be empty or whitespace-only"
|
||||
)
|
||||
# Validate content if provided (checked against the note below for
|
||||
# likes/reposts, whose comment may be empty)
|
||||
empty_content = content is not None and not content.strip()
|
||||
|
||||
# 2. GET EXISTING NOTE
|
||||
# Phase 6: Load content to get old HTML for webmention URL diffing
|
||||
@@ -678,6 +692,11 @@ def update_note(
|
||||
identifier = slug if slug is not None else id
|
||||
raise NoteNotFoundError(identifier)
|
||||
|
||||
if empty_content and not (existing_note.like_of or existing_note.repost_of or has_media):
|
||||
raise InvalidNoteDataError(
|
||||
"content", content, "Content cannot be empty or whitespace-only"
|
||||
)
|
||||
|
||||
# 3. SETUP
|
||||
updated_at = datetime.utcnow()
|
||||
data_dir = Path(current_app.config["DATA_PATH"])
|
||||
@@ -815,13 +834,14 @@ def update_note(
|
||||
from starpunk.webmention_sender import queue_webmentions_for_note
|
||||
db_path = Path(current_app.config["DATABASE_PATH"])
|
||||
site_url = current_app.config["SITE_URL"]
|
||||
note_url = f"{site_url}/note/{updated_note.slug}"
|
||||
from starpunk.utils import absolute_url_for
|
||||
note_url = absolute_url_for("public.note", slug=updated_note.slug)
|
||||
|
||||
# Get rendered HTML from updated note (if published)
|
||||
html_content = updated_note.html if updated_note.published else None
|
||||
|
||||
# Determine in_reply_to (use new value if provided, otherwise existing)
|
||||
reply_to = in_reply_to if in_reply_to is not None else updated_note.in_reply_to
|
||||
reply_to = (in_reply_to if in_reply_to is not None else updated_note.in_reply_to) or updated_note.like_of or updated_note.repost_of
|
||||
|
||||
# Queue webmentions (best-effort)
|
||||
# Pass old_html for URL diffing (Phase 6)
|
||||
@@ -838,8 +858,8 @@ def update_note(
|
||||
current_app.logger.warning(f"Failed to queue webmentions for note {updated_note.slug}: {e}")
|
||||
|
||||
# 9. FETCH REPLY CONTEXT (v1.6.0 Phase 3)
|
||||
# Fetch context if in_reply_to changed and is not None
|
||||
if in_reply_to is not None:
|
||||
# Fetch context if in_reply_to changed and is set ("" clears it)
|
||||
if in_reply_to:
|
||||
try:
|
||||
from starpunk.reply_context import fetch_reply_context
|
||||
db_path = Path(current_app.config["DATABASE_PATH"])
|
||||
|
||||
@@ -226,8 +226,8 @@ def edit_note_form(note_id: int):
|
||||
flash("Note not found", "error")
|
||||
return redirect(url_for("admin.dashboard")), 404
|
||||
|
||||
# Pre-load tags for the edit form (v1.3.0 Phase 3)
|
||||
tags = get_note_tags(note.id)
|
||||
# Pre-load tags for the edit form (v1.3.0 Phase 3), as entered so saving keeps the order
|
||||
tags = get_note_tags(note.id, entered_order=True)
|
||||
object.__setattr__(note, '_cached_tags', tags)
|
||||
|
||||
from starpunk.syndication import get_note_syndications, get_targets
|
||||
@@ -277,7 +277,7 @@ def update_note_submit(note_id: int):
|
||||
published = "published" in request.form
|
||||
tags_input = request.form.get("tags", "")
|
||||
|
||||
if not content:
|
||||
if not content and not existing_note.reacts_to: # likes/reposts may have no comment
|
||||
flash("Content cannot be empty", "error")
|
||||
return redirect(url_for("admin.edit_note_form", note_id=note_id))
|
||||
|
||||
|
||||
@@ -64,7 +64,7 @@ def _get_cached_notes():
|
||||
|
||||
# Cache expired or empty, fetch fresh notes
|
||||
max_items = current_app.config.get("FEED_MAX_ITEMS", 50)
|
||||
notes = list_notes(published_only=True, limit=max_items)
|
||||
notes = list_notes(published_only=True, limit=max_items, likes=False)
|
||||
|
||||
# Batch load media and tags for all notes (v1.5.0 Phase 3)
|
||||
# Reduces query count from O(n) to O(1) for both media and tags
|
||||
@@ -249,8 +249,8 @@ def index():
|
||||
from starpunk.media import get_note_media
|
||||
from starpunk.tags import get_note_tags
|
||||
|
||||
# Get recent published notes (limit 20)
|
||||
notes = list_notes(published_only=True, limit=20)
|
||||
# Get recent published notes (limit 20); likes have their own page (v2.5.0)
|
||||
notes = list_notes(published_only=True, limit=20, likes=False)
|
||||
|
||||
# Attach media and tags to each note for display
|
||||
for note in notes:
|
||||
@@ -262,9 +262,38 @@ def index():
|
||||
tags = get_note_tags(note.id)
|
||||
object.__setattr__(note, '_cached_tags', tags)
|
||||
|
||||
attach_contexts(notes)
|
||||
return render_template("index.html", notes=notes)
|
||||
|
||||
|
||||
def attach_contexts(notes) -> None:
|
||||
"""Give likes and reposts the fetched context of the post they point at"""
|
||||
from starpunk.webmention_queries import get_reply_context
|
||||
|
||||
for note in notes:
|
||||
context = None
|
||||
if note.reacts_to:
|
||||
try:
|
||||
context = get_reply_context(note.reacts_to)
|
||||
except Exception:
|
||||
context = None
|
||||
object.__setattr__(note, 'context', context)
|
||||
|
||||
|
||||
@bp.route("/likes")
|
||||
def likes():
|
||||
"""Likes, kept off the home page and feeds (v2.5.0)"""
|
||||
from starpunk.media import get_note_media
|
||||
from starpunk.tags import get_note_tags
|
||||
|
||||
notes = list_notes(published_only=True, limit=50, likes=True)
|
||||
for note in notes:
|
||||
object.__setattr__(note, 'media', get_note_media(note.id))
|
||||
object.__setattr__(note, '_cached_tags', get_note_tags(note.id))
|
||||
attach_contexts(notes)
|
||||
return render_template("likes.html", notes=notes)
|
||||
|
||||
|
||||
@bp.route("/note/<slug>")
|
||||
def note(slug: str):
|
||||
"""
|
||||
@@ -334,6 +363,7 @@ def note(slug: str):
|
||||
from starpunk.syndication import get_note_syndications
|
||||
|
||||
syndications = [s for s in get_note_syndications(note_obj.id) if s["state"] == "synced"]
|
||||
attach_contexts([note_obj])
|
||||
|
||||
return render_template(
|
||||
"note.html",
|
||||
@@ -384,6 +414,7 @@ def tag(tag: str):
|
||||
for note in notes:
|
||||
media = get_note_media(note.id)
|
||||
object.__setattr__(note, 'media', media)
|
||||
attach_contexts(notes)
|
||||
|
||||
return render_template(
|
||||
"tag.html",
|
||||
|
||||
+710
-33
@@ -1,10 +1,10 @@
|
||||
"""
|
||||
Crossposting (POSSE) for StarPunk (v2.1.0 ADR-065, v2.2.0 ADR-066)
|
||||
Crossposting (POSSE) for StarPunk (v2.1.0 ADR-065, v2.2.0 ADR-066, v2.6.0 ADR-069, v2.6.3 ADR-070)
|
||||
|
||||
Published notes are copied to syndication targets, and later edits and
|
||||
deletes follow them. Two target kinds exist: a GoToSocial (or any
|
||||
Mastodon-API) account, and a Bluesky account. Bluesky posts can't be edited,
|
||||
so there only deletes follow.
|
||||
deletes follow them. Three target kinds exist: a GoToSocial (or any
|
||||
Mastodon-API) account, a Bluesky account and a Threads account. Bluesky and
|
||||
Threads posts can't be edited, so there only deletes follow.
|
||||
|
||||
Each (note, target) pair has a row in `syndications`. Changing a note marks
|
||||
its rows pending; a background worker then reconciles the remote post with
|
||||
@@ -17,9 +17,15 @@ the note:
|
||||
Rows carry a version so an edit made while a send is in flight is not lost.
|
||||
"""
|
||||
|
||||
import fcntl
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
@@ -28,6 +34,9 @@ from typing import Dict, List, Optional
|
||||
import httpx
|
||||
from flask import current_app
|
||||
|
||||
# httpx logs request URLs at INFO, and Threads takes its token in the URL
|
||||
logging.getLogger("httpx").setLevel(logging.WARNING)
|
||||
|
||||
from starpunk.database import get_db
|
||||
|
||||
MAX_ATTEMPTS = 8
|
||||
@@ -113,21 +122,58 @@ class MastodonClient:
|
||||
|
||||
# Interface used by the reconcile loop
|
||||
can_edit = True
|
||||
NATIVE = ("favourite", "reblog") # remote_id prefixes for likes/boosts (v2.5.0)
|
||||
|
||||
def find_status(self, url: str) -> Optional[str]:
|
||||
"""Local id of a fediverse post, fetching it if needed; None if the URL isn't one"""
|
||||
try:
|
||||
response = self._call("GET", "/api/v2/search", params={
|
||||
"q": url, "resolve": "true", "type": "statuses", "limit": 1,
|
||||
})
|
||||
except RemoteGone:
|
||||
return None
|
||||
statuses = response.json().get("statuses") or []
|
||||
return statuses[0]["id"] if statuses else None
|
||||
|
||||
def _prepare(self, post: "Post") -> tuple:
|
||||
text = compose_text(post.note, post.permalink, post.tags, post.max_chars)
|
||||
if post.kind == "repost":
|
||||
return compose_repost_text(post, post.note.content, post.max_chars), []
|
||||
# Images in the text are sent as attachments instead (v2.6.6)
|
||||
text = compose_text(post.note, post.permalink, post.tags, post.max_chars,
|
||||
body=strip_images(post.note.content))
|
||||
media_ids = [self.upload_media(f["path"], f["mime_type"], f["description"]) for f in post.media]
|
||||
return text, media_ids
|
||||
|
||||
def create(self, post: "Post", key: str, retrying: bool = False) -> dict:
|
||||
if post.kind in ("like", "repost") and post.target_url:
|
||||
# Native favourite/boost when the target is a fediverse post; a
|
||||
# repost with a comment is posted as a status so the comment stays
|
||||
commented = bool(post.note.content.strip())
|
||||
if post.kind == "like" or not commented:
|
||||
status_id = self.find_status(post.target_url)
|
||||
if status_id:
|
||||
action = "favourite" if post.kind == "like" else "reblog"
|
||||
self._call("POST", f"/api/v1/statuses/{status_id}/{action}")
|
||||
return {"id": f"{action}:{status_id}", "url": None}
|
||||
if post.kind == "like":
|
||||
return {"id": None, "url": None} # nothing to like here
|
||||
text, media_ids = self._prepare(post)
|
||||
return self.create_status(text, media_ids, idempotency_key=key)
|
||||
|
||||
def update(self, remote_id: str, post: "Post") -> dict:
|
||||
if remote_id.split(":", 1)[0] in self.NATIVE:
|
||||
return {"id": remote_id, "url": None} # a favourite/boost has nothing to edit
|
||||
text, media_ids = self._prepare(post)
|
||||
return self.update_status(remote_id, text, media_ids)
|
||||
|
||||
def delete(self, remote_id: str) -> None:
|
||||
action, _, status_id = remote_id.partition(":")
|
||||
if action in self.NATIVE and status_id:
|
||||
try:
|
||||
self._call("POST", f"/api/v1/statuses/{status_id}/un{action}")
|
||||
except RemoteGone:
|
||||
pass
|
||||
return
|
||||
self.delete_status(remote_id)
|
||||
|
||||
|
||||
@@ -229,20 +275,42 @@ class BlueskyClient:
|
||||
image["aspectRatio"] = {"width": width, "height": height}
|
||||
return image
|
||||
|
||||
def _record(self, post: "Post") -> dict:
|
||||
text = compose_text(post.note, post.permalink, post.tags, post.max_chars,
|
||||
body=plain_text(post.note.content))
|
||||
record = {
|
||||
"$type": self.POST,
|
||||
"text": text,
|
||||
"createdAt": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z",
|
||||
}
|
||||
facets = bluesky_facets(text)
|
||||
@staticmethod
|
||||
def _now() -> str:
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z"
|
||||
|
||||
def _link_card(self, url: str) -> dict:
|
||||
"""External embed for a page, with its title, description and image when we can get them"""
|
||||
card = page_card(self.http, url)
|
||||
external = {"uri": url, "title": card["title"][:300], "description": card["description"][:1000]}
|
||||
if card["image"]:
|
||||
try:
|
||||
data, mime_type, _ = _fetch_public(self.http, card["image"], CARD_IMAGE_LIMIT,
|
||||
truncate=False)
|
||||
data, mime_type, _, _ = _fit_image(data, mime_type.split(";")[0] or "image/jpeg",
|
||||
BLUESKY_BLOB_LIMIT)
|
||||
external["thumb"] = self.upload_blob(data, mime_type)
|
||||
except Exception as e: # a card without a picture is fine
|
||||
current_app.logger.info(f"No link card image for {url}: {e}")
|
||||
return {"$type": "app.bsky.embed.external", "external": external}
|
||||
|
||||
def _record(self, post: "Post", text: Optional[str] = None, embed: Optional[dict] = None,
|
||||
anchors: List[tuple] = ()) -> dict:
|
||||
if text is None:
|
||||
body, anchors = plain_text_links(post.note.content, links="anchor")
|
||||
text = compose_text(post.note, post.permalink, post.tags, post.max_chars, body=body)
|
||||
record = {"$type": self.POST, "text": text, "createdAt": self._now()}
|
||||
facets = bluesky_facets(text, anchors)
|
||||
if facets:
|
||||
record["facets"] = facets
|
||||
if post.media:
|
||||
if embed is not None:
|
||||
record["embed"] = embed
|
||||
elif post.media:
|
||||
record["embed"] = {"$type": "app.bsky.embed.images",
|
||||
"images": [self._image(m) for m in post.media]}
|
||||
elif note_links(post.note.content):
|
||||
# The note links somewhere: the card shows that page, not the note
|
||||
record["embed"] = self._link_card(note_links(post.note.content)[0])
|
||||
else:
|
||||
record["embed"] = {"$type": "app.bsky.embed.external", "external": {
|
||||
"uri": post.permalink,
|
||||
@@ -251,6 +319,56 @@ class BlueskyClient:
|
||||
}}
|
||||
return record
|
||||
|
||||
_POST_URL = re.compile(r"^https://bsky\.app/profile/([^/]+)/post/([^/?#]+)")
|
||||
|
||||
def resolve_post(self, url: str) -> Optional[dict]:
|
||||
"""{'uri', 'cid'} of a Bluesky post URL; None if the URL isn't one"""
|
||||
m = self._POST_URL.match(url or "")
|
||||
if not m:
|
||||
return None
|
||||
actor, rkey = m.groups()
|
||||
if not actor.startswith("did:"):
|
||||
actor = self._call("GET", "com.atproto.identity.resolveHandle",
|
||||
params={"handle": actor}).json()["did"]
|
||||
uri = f"at://{actor}/{self.POST}/{rkey}"
|
||||
posts = self._call("GET", "app.bsky.feed.getPosts", params={"uris": uri}).json().get("posts") or []
|
||||
return {"uri": uri, "cid": posts[0]["cid"]} if posts else None
|
||||
|
||||
def _create(self, collection: str, record: dict) -> str:
|
||||
response = self._call("POST", "com.atproto.repo.createRecord",
|
||||
json={"repo": self.did or self.identifier, "collection": collection,
|
||||
"record": record})
|
||||
return response.json()["uri"]
|
||||
|
||||
def _react(self, post: "Post") -> dict:
|
||||
"""Like, repost or quote a Bluesky post; fall back to a link post for reposts"""
|
||||
comment, anchors = (plain_text_links(post.note.content, links="anchor")
|
||||
if post.note.content.strip() else ("", []))
|
||||
subject = self.resolve_post(post.target_url)
|
||||
if subject and post.kind == "like":
|
||||
uri = self._create("app.bsky.feed.like", {
|
||||
"$type": "app.bsky.feed.like", "subject": subject, "createdAt": self._now()})
|
||||
return {"id": uri, "url": None}
|
||||
if subject and not comment:
|
||||
uri = self._create("app.bsky.feed.repost", {
|
||||
"$type": "app.bsky.feed.repost", "subject": subject, "createdAt": self._now()})
|
||||
return {"id": uri, "url": None}
|
||||
if subject: # repost with a comment -> quote post
|
||||
text = comment if len(comment) <= post.max_chars else \
|
||||
comment[:post.max_chars - 1].rsplit(" ", 1)[0].rstrip() + "…"
|
||||
uri = self._create(self.POST, self._record(
|
||||
post, text=text, embed={"$type": "app.bsky.embed.record", "record": subject},
|
||||
anchors=anchors))
|
||||
return {"id": uri, "url": self._web_url(uri)}
|
||||
if post.kind == "like":
|
||||
return {"id": None, "url": None} # not a Bluesky post: nothing to like
|
||||
text = compose_repost_text(post, comment, post.max_chars)
|
||||
embed = {"$type": "app.bsky.embed.external", "external": {
|
||||
"uri": post.target_url, "title": (post.target_title or post.target_url)[:300],
|
||||
"description": ""}}
|
||||
uri = self._create(self.POST, self._record(post, text=text, embed=embed))
|
||||
return {"id": uri, "url": self._web_url(uri)}
|
||||
|
||||
def _web_url(self, uri: str) -> str:
|
||||
rkey = uri.rsplit("/", 1)[1]
|
||||
return f"https://bsky.app/profile/{self.handle or self.did}/post/{rkey}"
|
||||
@@ -265,6 +383,11 @@ class BlueskyClient:
|
||||
return None
|
||||
|
||||
def create(self, post: "Post", key: str, retrying: bool = False) -> dict:
|
||||
if post.kind in ("like", "repost") and post.target_url:
|
||||
if not self.did:
|
||||
with self._lock:
|
||||
self._login()
|
||||
return self._react(post)
|
||||
if retrying:
|
||||
if not self.did:
|
||||
with self._lock:
|
||||
@@ -289,11 +412,15 @@ class BlueskyClient:
|
||||
json={"repo": repo, "collection": collection, "rkey": rkey})
|
||||
|
||||
|
||||
def _fit_image(path: Path, mime_type: str, limit: int) -> tuple:
|
||||
"""Image bytes no bigger than `limit`, re-encoded as JPEG if needed. Returns (data, mime, w, h)."""
|
||||
def _fit_image(path, mime_type: str, limit: int) -> tuple:
|
||||
"""
|
||||
Image (a path, or bytes) no bigger than `limit`, re-encoded as JPEG if
|
||||
needed. Returns (data, mime, w, h).
|
||||
"""
|
||||
from PIL import Image
|
||||
|
||||
data = Path(path).read_bytes()
|
||||
data = path if isinstance(path, bytes) else Path(path).read_bytes()
|
||||
name = "image" if isinstance(path, bytes) else Path(path).name
|
||||
with Image.open(io.BytesIO(data)) as img:
|
||||
width, height = img.size
|
||||
if len(data) <= limit:
|
||||
@@ -306,7 +433,355 @@ def _fit_image(path: Path, mime_type: str, limit: int) -> tuple:
|
||||
copy.save(buf, format="JPEG", quality=quality, optimize=True)
|
||||
if buf.tell() <= limit:
|
||||
return buf.getvalue(), "image/jpeg", copy.size[0], copy.size[1]
|
||||
raise SyndicationError(f"{Path(path).name}: image too large for Bluesky", permanent=True)
|
||||
raise SyndicationError(f"{name}: image too large for Bluesky", permanent=True)
|
||||
|
||||
|
||||
# --- Link cards ---------------------------------------------------------------
|
||||
|
||||
CARD_PAGE_LIMIT = 512 * 1024 # the <head> is near the start; read no more
|
||||
CARD_IMAGE_LIMIT = 5 * 1024 * 1024
|
||||
|
||||
|
||||
def _fetch_public(http: httpx.Client, url: str, limit: int, truncate: bool) -> tuple:
|
||||
"""
|
||||
GET a public web URL, checking every redirect hop (no requests into the
|
||||
LAN). Reads at most `limit` bytes: stops there if `truncate`, else fails.
|
||||
Returns (data, content_type, final_url).
|
||||
"""
|
||||
from urllib.parse import urljoin, urlparse
|
||||
from starpunk.micropub import _public_address
|
||||
|
||||
site = current_app.config.get("SITE_URL", "")
|
||||
for _ in range(5):
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in ("http", "https") or not parsed.hostname:
|
||||
raise SyndicationError(f"not a web URL: {url}", permanent=True)
|
||||
if not current_app.config.get("TESTING") and not _public_address(parsed.hostname):
|
||||
raise SyndicationError(f"not a public address: {url}", permanent=True)
|
||||
try:
|
||||
with http.stream("GET", url, follow_redirects=False, timeout=10.0,
|
||||
headers={"User-Agent": f"StarPunk (+{site})"}) as response:
|
||||
if response.is_redirect and "location" in response.headers:
|
||||
url = urljoin(url, response.headers["location"])
|
||||
continue
|
||||
if response.status_code >= 400:
|
||||
raise SyndicationError(f"{url}: HTTP {response.status_code}")
|
||||
data = bytearray()
|
||||
for chunk in response.iter_bytes():
|
||||
data += chunk
|
||||
if len(data) > limit:
|
||||
if truncate:
|
||||
break
|
||||
raise SyndicationError(f"{url}: too large")
|
||||
return bytes(data), response.headers.get("content-type", ""), url
|
||||
except httpx.HTTPError as e:
|
||||
raise SyndicationError(f"{url}: {type(e).__name__}")
|
||||
raise SyndicationError(f"{url}: too many redirects")
|
||||
|
||||
|
||||
def page_card(http: httpx.Client, url: str) -> dict:
|
||||
"""
|
||||
A page's title, description and image URL (Open Graph, Twitter or plain
|
||||
HTML tags), for a link card. Missing parts come back empty; a page that
|
||||
can't be fetched gives just the URL as the title.
|
||||
"""
|
||||
from urllib.parse import urljoin
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
card = {"title": url, "description": "", "image": ""}
|
||||
try:
|
||||
data, content_type, final = _fetch_public(http, url, CARD_PAGE_LIMIT, truncate=True)
|
||||
except SyndicationError as e:
|
||||
current_app.logger.info(f"No link card details for {url}: {e}")
|
||||
return card
|
||||
if "html" not in content_type.lower():
|
||||
return card
|
||||
soup = BeautifulSoup(data, "html.parser")
|
||||
|
||||
def meta(*names):
|
||||
for name in names:
|
||||
tag = soup.find("meta", attrs={"property": name}) or soup.find("meta", attrs={"name": name})
|
||||
if tag and (tag.get("content") or "").strip():
|
||||
return " ".join(tag["content"].split())
|
||||
return ""
|
||||
|
||||
title = meta("og:title", "twitter:title")
|
||||
if not title and soup.title:
|
||||
title = " ".join(soup.title.get_text(" ").split())
|
||||
image = meta("og:image:secure_url", "og:image", "og:image:url", "twitter:image")
|
||||
card.update(title=title or url,
|
||||
description=meta("og:description", "twitter:description", "description"),
|
||||
image=urljoin(final, image) if image else "")
|
||||
return card
|
||||
|
||||
|
||||
THREADS_API = "https://graph.threads.net/v1.0"
|
||||
THREADS_IMAGE_TYPES = {"image/jpeg", "image/png"}
|
||||
THREADS_IMAGE_LIMIT = 8_000_000
|
||||
THREADS_MAX_LINKS = 5
|
||||
THREADS_REFRESH_EVERY = 24 * 3600 # tokens last 60 days; Meta allows a refresh once one is a day old
|
||||
# Graph API error codes that are temporary (rate limits, service hiccups)
|
||||
_GRAPH_TRANSIENT = {1, 2, 4, 17, 32, 341, 613}
|
||||
_GRAPH_AUTH = {10, 190, 200} # bad token or missing permission: retrying won't help
|
||||
|
||||
|
||||
def threads_length(text: str) -> int:
|
||||
"""Length as Threads counts it: emoji (and, to be safe, all non-ASCII) by UTF-8 bytes"""
|
||||
return sum(1 if ord(ch) < 128 else len(ch.encode()) for ch in text)
|
||||
|
||||
|
||||
def threads_topic(tags: List[str]) -> str:
|
||||
"""The first tag usable as a Threads topic (one per post; no '.' or '&', 1-50 chars)"""
|
||||
for tag in tags:
|
||||
topic = re.sub(r"[.&]", "", tag).strip()[:50]
|
||||
if topic:
|
||||
return topic
|
||||
return ""
|
||||
|
||||
|
||||
class ThreadsClient:
|
||||
"""
|
||||
Minimal Threads API client (Meta Graph API)
|
||||
|
||||
Posting is two steps: create a container, wait for it to be ready, then
|
||||
publish it. Meta fetches images from their public URL on this site.
|
||||
Posts can't be edited through the API, so only deletes follow.
|
||||
|
||||
Long-lived tokens expire after 60 days unless refreshed. `maintain()`
|
||||
refreshes the token once a day and keeps it in `token_file`, so it
|
||||
survives restarts. A different token in the config replaces it.
|
||||
"""
|
||||
|
||||
can_edit = False
|
||||
|
||||
def __init__(self, token: str, token_file: Optional[Path] = None, api: str = THREADS_API,
|
||||
http: Optional[httpx.Client] = None, sleep=time.sleep):
|
||||
self.config_token = token
|
||||
self.token_file = Path(token_file) if token_file else None
|
||||
self.api = api.rstrip("/")
|
||||
self.http = http or httpx.Client(timeout=30.0)
|
||||
self.sleep = sleep
|
||||
self.user_id: Optional[str] = None
|
||||
self.username: Optional[str] = None
|
||||
self._lock = threading.Lock()
|
||||
self._token = token
|
||||
self._checked_at = 0.0
|
||||
saved = self._read_file()
|
||||
if saved:
|
||||
self._token = saved["token"]
|
||||
|
||||
# --- Token ---
|
||||
|
||||
def _seed(self) -> str:
|
||||
return hashlib.sha256(self.config_token.encode()).hexdigest()
|
||||
|
||||
def _read_file(self) -> Optional[dict]:
|
||||
"""The saved token, if it descends from the configured one"""
|
||||
if not self.token_file or not self.token_file.exists():
|
||||
return None
|
||||
try:
|
||||
data = json.loads(self.token_file.read_text())
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
return data if data.get("seed") == self._seed() and data.get("token") else None
|
||||
|
||||
def _write_file(self, token: str, refreshed_at: float) -> None:
|
||||
tmp = self.token_file.with_suffix(".tmp")
|
||||
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||
with os.fdopen(fd, "w") as f:
|
||||
json.dump({"seed": self._seed(), "token": token, "refreshed_at": refreshed_at}, f)
|
||||
os.replace(tmp, self.token_file)
|
||||
|
||||
def maintain(self) -> None:
|
||||
"""Refresh the token if a day has passed since the last try (cheap to call often)"""
|
||||
now = time.time()
|
||||
if now - self._checked_at < THREADS_REFRESH_EVERY:
|
||||
return
|
||||
self._checked_at = now
|
||||
if not self.token_file:
|
||||
return
|
||||
# Gunicorn workers share the file: one refreshes, the others pick up its result
|
||||
self.token_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(self.token_file.with_suffix(".lock"), "w") as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX)
|
||||
saved = self._read_file()
|
||||
if saved:
|
||||
self._token = saved["token"]
|
||||
if now - saved.get("refreshed_at", 0) < THREADS_REFRESH_EVERY:
|
||||
self._checked_at = saved["refreshed_at"]
|
||||
return
|
||||
base = self.api.rsplit("/", 1)[0] # refresh lives outside the versioned path
|
||||
try:
|
||||
response = self.http.get(f"{base}/refresh_access_token", params={
|
||||
"grant_type": "th_refresh_token", "access_token": self._token})
|
||||
except httpx.HTTPError as e:
|
||||
current_app.logger.warning(f"Threads token refresh failed: {type(e).__name__}")
|
||||
self._checked_at = now - THREADS_REFRESH_EVERY + 3600 # try again in an hour
|
||||
return
|
||||
if response.status_code < 400:
|
||||
self._token = response.json()["access_token"]
|
||||
current_app.logger.info("Threads access token refreshed")
|
||||
else:
|
||||
# e.g. a token less than a day old: keep it, try again tomorrow
|
||||
current_app.logger.warning(
|
||||
f"Threads token refresh failed: HTTP {response.status_code} {response.text[:200]}")
|
||||
self._write_file(self._token, now)
|
||||
|
||||
# --- Graph plumbing ---
|
||||
|
||||
def _call(self, method: str, path: str, **params) -> dict:
|
||||
params["access_token"] = self._token
|
||||
kwargs = {"params": params} if method in ("GET", "DELETE") else {"data": params}
|
||||
try:
|
||||
response = self.http.request(method, f"{self.api}/{path}", **kwargs)
|
||||
except httpx.HTTPError as e:
|
||||
raise SyndicationError(f"{method} {path}: {type(e).__name__}") # never echo the URL/token
|
||||
try:
|
||||
body = response.json()
|
||||
except ValueError:
|
||||
body = {}
|
||||
error = body.get("error") if isinstance(body, dict) else None
|
||||
if response.status_code >= 400 or error:
|
||||
error = error if isinstance(error, dict) else {}
|
||||
code = error.get("code")
|
||||
detail = " / ".join(str(error[k]) for k in ("error_user_title", "error_user_msg")
|
||||
if error.get(k))
|
||||
message = (f"{method} {path}: HTTP {response.status_code} "
|
||||
f"{error.get('message') or response.text[:200]}"
|
||||
f" (code {code}, subcode {error.get('error_subcode')}"
|
||||
f"{', ' + detail if detail else ''}, fbtrace {error.get('fbtrace_id')})")
|
||||
if response.status_code == 404 or (code == 100 and error.get("error_subcode") == 33):
|
||||
raise RemoteGone(message, permanent=True)
|
||||
permanent = (400 <= response.status_code < 500 and response.status_code != 429
|
||||
and code not in _GRAPH_TRANSIENT)
|
||||
exc = SyndicationError(message, permanent=permanent)
|
||||
exc.code = code
|
||||
raise exc
|
||||
return body
|
||||
|
||||
def _me(self) -> str:
|
||||
with self._lock:
|
||||
if not self.user_id:
|
||||
me = self._call("GET", "me", fields="id,username")
|
||||
self.user_id, self.username = me["id"], me.get("username")
|
||||
return self.user_id
|
||||
|
||||
def _wait(self, container_id: str) -> None:
|
||||
"""Wait until Meta has processed a container (images take a few seconds)"""
|
||||
delay = 1
|
||||
self.sleep(2) # Meta can report a container ready a moment too early
|
||||
for _ in range(8):
|
||||
status = self._call("GET", container_id, fields="status,error_message")
|
||||
state = status.get("status")
|
||||
if state in ("FINISHED", None):
|
||||
return
|
||||
if state in ("ERROR", "EXPIRED"):
|
||||
raise SyndicationError(
|
||||
f"Threads container {state}: {status.get('error_message') or 'no detail'}",
|
||||
permanent=True)
|
||||
self.sleep(delay)
|
||||
delay = min(delay * 2, 15)
|
||||
raise SyndicationError("Threads container still processing")
|
||||
|
||||
# --- Posts ---
|
||||
|
||||
def _text(self, post: "Post") -> str:
|
||||
# Links written "text (url)"; the first one gets Threads' preview card.
|
||||
# Meta allows 5 different links per post and the permalink is one.
|
||||
body = plain_text(post.note.content, links="inline", max_urls=THREADS_MAX_LINKS - 1)
|
||||
limit = post.max_chars
|
||||
while True:
|
||||
text = compose_text(post.note, post.permalink, [], limit, body=body)
|
||||
over = threads_length(text) - post.max_chars
|
||||
if over <= 0:
|
||||
return text
|
||||
limit -= over
|
||||
|
||||
def _image_container(self, uid: str, media: dict, **extra) -> str:
|
||||
params = {"media_type": "IMAGE", "image_url": media["url"], **extra}
|
||||
if media["description"]:
|
||||
params["alt_text"] = media["description"][:1000]
|
||||
return self._call("POST", f"{uid}/threads", **params)["id"]
|
||||
|
||||
def _repost_text(self, post: "Post") -> str:
|
||||
comment = plain_text(post.note.content, links="inline", max_urls=THREADS_MAX_LINKS - 1)
|
||||
limit = post.max_chars
|
||||
while True:
|
||||
text = compose_repost_text(post, comment, limit)
|
||||
over = threads_length(text) - post.max_chars
|
||||
if over <= 0:
|
||||
return text
|
||||
limit -= over
|
||||
|
||||
def _container(self, uid: str, post: "Post") -> str:
|
||||
if post.kind == "repost":
|
||||
# No native repost of outside content: post "🔁 title url"
|
||||
return self._call("POST", f"{uid}/threads", media_type="TEXT",
|
||||
text=self._repost_text(post))["id"]
|
||||
common = {"text": self._text(post)}
|
||||
topic = threads_topic(post.tags)
|
||||
if topic:
|
||||
common["topic_tag"] = topic
|
||||
# Threads takes JPEG/PNG up to 8 MB; others (GIF, WebP) are left off
|
||||
images = [m for m in post.media if m["mime_type"] in THREADS_IMAGE_TYPES
|
||||
and Path(m["path"]).stat().st_size <= THREADS_IMAGE_LIMIT]
|
||||
if not images:
|
||||
# No explicit link_attachment: Meta rejects one for a page it hasn't
|
||||
# seen yet ("Invalid Link Attachment"), but previews the text's
|
||||
# first link by itself without failing
|
||||
return self._call("POST", f"{uid}/threads", media_type="TEXT", **common)["id"]
|
||||
if len(images) == 1:
|
||||
return self._image_container(uid, images[0], **common)
|
||||
children = [self._image_container(uid, m, is_carousel_item="true") for m in images]
|
||||
for child in children:
|
||||
self._wait(child)
|
||||
return self._call("POST", f"{uid}/threads", media_type="CAROUSEL",
|
||||
children=",".join(children), **common)["id"]
|
||||
|
||||
def _profile_url(self) -> str:
|
||||
return f"https://www.threads.com/@{self.username}" if self.username else "https://www.threads.com/"
|
||||
|
||||
def _find_existing(self, uid: str, link: str) -> Optional[dict]:
|
||||
"""A recent post that already has this link (an earlier send whose reply was lost)"""
|
||||
listing = self._call("GET", f"{uid}/threads", fields="id,text,permalink", limit=25)
|
||||
for item in listing.get("data", []):
|
||||
if link in (item.get("text") or ""):
|
||||
return {"id": item["id"], "url": item.get("permalink") or self._profile_url()}
|
||||
return None
|
||||
|
||||
def create(self, post: "Post", key: str, retrying: bool = False) -> dict:
|
||||
if post.kind == "like" and post.target_url:
|
||||
return {"id": None, "url": None} # the API can't like things; likes stay on the site
|
||||
uid = self._me()
|
||||
if retrying:
|
||||
marker = post.target_url if post.kind == "repost" and post.target_url else post.permalink
|
||||
existing = self._find_existing(uid, marker)
|
||||
if existing:
|
||||
return existing
|
||||
container = self._container(uid, post)
|
||||
self._wait(container)
|
||||
try:
|
||||
media_id = self._call("POST", f"{uid}/threads_publish", creation_id=container)["id"]
|
||||
except SyndicationError as e:
|
||||
if getattr(e, "code", None) in _GRAPH_AUTH:
|
||||
raise
|
||||
# Meta's publish step fails now and then ("Fatal") and works on a
|
||||
# later try; the retry checks for an existing post first
|
||||
raise SyndicationError(str(e))
|
||||
try:
|
||||
url = self._call("GET", media_id, fields="permalink").get("permalink")
|
||||
except SyndicationError:
|
||||
url = None # it's published; don't fail (and post twice) over the link
|
||||
return {"id": media_id, "url": url or self._profile_url()}
|
||||
|
||||
def update(self, remote_id: str, post: "Post") -> dict:
|
||||
raise NotImplementedError("Threads posts can't be edited through the API")
|
||||
|
||||
def delete(self, remote_id: str) -> None:
|
||||
try:
|
||||
self._call("DELETE", remote_id)
|
||||
except RemoteGone:
|
||||
pass # already gone
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -327,6 +802,24 @@ class Post:
|
||||
tags: List[str]
|
||||
max_chars: int
|
||||
media: List[dict] = field(default_factory=list)
|
||||
kind: str = "note" # note, like or repost (v2.5.0)
|
||||
target_url: Optional[str] = None # what a like/repost points at
|
||||
target_title: Optional[str] = None # its fetched title, if any
|
||||
|
||||
|
||||
def compose_repost_text(post: "Post", comment: str, max_chars: int) -> str:
|
||||
"""'comment' then '🔁 title' and the URL; the comment is shortened to fit"""
|
||||
title = post.target_title or "Reposted"
|
||||
tail = f"🔁 {title}\n{post.target_url}"
|
||||
if len(tail) > max_chars:
|
||||
tail = f"🔁 {post.target_url}"
|
||||
comment = (comment or "").strip()
|
||||
if not comment:
|
||||
return tail
|
||||
room = max_chars - len(tail) - 2
|
||||
if len(comment) > room:
|
||||
comment = comment[:max(room - 1, 0)].rsplit(" ", 1)[0].rstrip() + "…"
|
||||
return f"{comment}\n\n{tail}" if room > 1 else tail
|
||||
|
||||
|
||||
_client_cache: Dict[str, object] = {}
|
||||
@@ -337,6 +830,7 @@ def targets_configured(config) -> bool:
|
||||
return bool(
|
||||
(config.get("GOTOSOCIAL_URL") and config.get("GOTOSOCIAL_ACCESS_TOKEN"))
|
||||
or (config.get("BLUESKY_HANDLE") and config.get("BLUESKY_APP_PASSWORD"))
|
||||
or config.get("THREADS_ACCESS_TOKEN")
|
||||
)
|
||||
|
||||
|
||||
@@ -378,6 +872,23 @@ def get_targets() -> Dict[str, Target]:
|
||||
max_chars=int(config.get("BLUESKY_MAX_CHARS", 300)),
|
||||
max_media=4,
|
||||
)
|
||||
|
||||
token = config.get("THREADS_ACCESS_TOKEN") or ""
|
||||
if token:
|
||||
client = _client_cache.get("threads")
|
||||
if client is None or client.config_token != token:
|
||||
client = ThreadsClient(token, Path(config["DATA_PATH"]) / "threads_token.json",
|
||||
api=config.get("THREADS_API") or THREADS_API)
|
||||
_client_cache["threads"] = client
|
||||
username = (config.get("THREADS_USERNAME") or "").lstrip("@")
|
||||
targets["threads"] = Target(
|
||||
id="threads",
|
||||
uid=f"https://www.threads.com/@{username}" if username else "https://www.threads.com/",
|
||||
name=config.get("THREADS_NAME", "Threads"),
|
||||
client=client,
|
||||
max_chars=int(config.get("THREADS_MAX_CHARS", 500)),
|
||||
max_media=int(config.get("THREADS_MAX_MEDIA", 4)),
|
||||
)
|
||||
return targets
|
||||
|
||||
|
||||
@@ -494,20 +1005,82 @@ def compose_text(note, permalink: str, tags: List[str], max_chars: int,
|
||||
if len(body) + len(tail) > max_chars:
|
||||
room = max_chars - len(tail) - 1
|
||||
body = body[:room].rsplit(" ", 1)[0].rstrip() + "…"
|
||||
return body + tail
|
||||
# A photo post may have no text: start with the link (v2.6.5)
|
||||
return (body + tail).lstrip()
|
||||
|
||||
|
||||
_BLOCKS = ["p", "h1", "h2", "h3", "h4", "h5", "h6", "pre", "blockquote", "ul", "ol", "hr", "table"]
|
||||
|
||||
|
||||
def plain_text(markdown_text: str) -> str:
|
||||
"""Markdown as readable plain text (for targets without Markdown)"""
|
||||
def _note_soup(markdown_text: str):
|
||||
import markdown
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
soup = BeautifulSoup(markdown.markdown(markdown_text, extensions=["fenced_code"]), "html.parser")
|
||||
return BeautifulSoup(markdown.markdown(markdown_text, extensions=["fenced_code"]), "html.parser")
|
||||
|
||||
|
||||
def _clean_url(url: str) -> str:
|
||||
return url.rstrip(".,;:!?)\"'…")
|
||||
|
||||
|
||||
def _is_web(url: str) -> bool:
|
||||
return url.startswith(("http://", "https://"))
|
||||
|
||||
|
||||
def note_links(markdown_text: str) -> List[str]:
|
||||
"""Web links in a note, in reading order: Markdown links, <autolinks> and bare URLs"""
|
||||
from bs4 import NavigableString
|
||||
|
||||
links = []
|
||||
for node in _note_soup(markdown_text).descendants:
|
||||
if getattr(node, "name", None) == "a" and _is_web(node.get("href", "")):
|
||||
links.append(node["href"])
|
||||
elif isinstance(node, NavigableString) and not node.find_parent("a"):
|
||||
links.extend(_clean_url(m.group(0)) for m in _URL_RE.finditer(str(node)))
|
||||
return list(dict.fromkeys(links))
|
||||
|
||||
|
||||
def plain_text(markdown_text: str, links: str = "text", max_urls: Optional[int] = None) -> str:
|
||||
"""Markdown as readable plain text (for targets without Markdown). See `plain_text_links`."""
|
||||
return plain_text_links(markdown_text, links, max_urls)[0]
|
||||
|
||||
|
||||
def plain_text_links(markdown_text: str, links: str = "text",
|
||||
max_urls: Optional[int] = None) -> tuple:
|
||||
"""
|
||||
Markdown as plain text, and the links whose text stands in for a URL.
|
||||
|
||||
A Markdown link `[text](url)` becomes:
|
||||
links="text" just its text
|
||||
links="inline" "text (url)", for targets with no clickable link text;
|
||||
after `max_urls` different URLs, just the text
|
||||
links="anchor" its text, listed in the returned anchors as (text, url)
|
||||
so the target can make that text clickable
|
||||
A link whose text is the URL itself stays a bare URL.
|
||||
"""
|
||||
from bs4 import NavigableString
|
||||
|
||||
soup = _note_soup(markdown_text)
|
||||
for img in soup.find_all("img"):
|
||||
img.decompose()
|
||||
anchors = []
|
||||
seen = set()
|
||||
for node in list(soup.descendants): # bare URLs count towards max_urls first
|
||||
if isinstance(node, NavigableString) and not node.find_parent("a"):
|
||||
seen.update(_clean_url(m.group(0)) for m in _URL_RE.finditer(str(node)))
|
||||
for a in soup.find_all("a"):
|
||||
href, label = a.get("href", ""), a.get_text()
|
||||
if not _is_web(href):
|
||||
continue
|
||||
bare = re.sub(r"^https?://", "", href).rstrip("/")
|
||||
if label.strip().rstrip("/") in (href.rstrip("/"), bare):
|
||||
a.replace_with(href)
|
||||
seen.add(href)
|
||||
elif links == "inline" and (href in seen or max_urls is None or len(seen) < max_urls):
|
||||
a.replace_with(f"{label} ({href})")
|
||||
seen.add(href)
|
||||
elif links == "anchor":
|
||||
anchors.append((label, href))
|
||||
for node in soup.find_all(string=True):
|
||||
if not node.strip() and "\n" in node and node.parent.name != "pre":
|
||||
node.extract() # layout newlines between tags; we add our own
|
||||
@@ -520,24 +1093,41 @@ def plain_text(markdown_text: str) -> str:
|
||||
block.append("\n\n")
|
||||
text = soup.get_text()
|
||||
text = re.sub(r"[ \t]+\n", "\n", text)
|
||||
return re.sub(r"\n{3,}", "\n\n", text).strip()
|
||||
return re.sub(r"\n{3,}", "\n\n", text).strip(), anchors
|
||||
|
||||
|
||||
_URL_RE = re.compile(r"https?://[^\s<>\"]+")
|
||||
_TAG_RE = re.compile(r"(?<![\w#])#(\w*[^\W\d]\w*)")
|
||||
|
||||
|
||||
def bluesky_facets(text: str) -> List[dict]:
|
||||
"""Link and hashtag facets (UTF-8 byte ranges) for a Bluesky post"""
|
||||
def bluesky_facets(text: str, anchors: List[tuple] = ()) -> List[dict]:
|
||||
"""
|
||||
Link and hashtag facets (UTF-8 byte ranges) for a Bluesky post.
|
||||
`anchors` are (text, url) pairs: that text, where it appears, links to url.
|
||||
"""
|
||||
def span(start: int, end: int) -> dict:
|
||||
return {"byteStart": len(text[:start].encode()), "byteEnd": len(text[:end].encode())}
|
||||
|
||||
facets = []
|
||||
for m in _URL_RE.finditer(text):
|
||||
url = m.group(0).rstrip(".,;:!?)\"'…")
|
||||
url = _clean_url(m.group(0))
|
||||
facets.append({"index": span(m.start(), m.start() + len(url)),
|
||||
"features": [{"$type": "app.bsky.richtext.facet#link", "uri": url}]})
|
||||
taken = [(m.start(), m.end()) for m in _URL_RE.finditer(text)]
|
||||
pos = 0
|
||||
for label, url in anchors:
|
||||
start = text.find(label, pos) if label.strip() else -1
|
||||
if start < 0:
|
||||
continue # shortened away
|
||||
end = start + len(label)
|
||||
if any(a < end and start < b for a, b in taken):
|
||||
continue
|
||||
taken.append((start, end))
|
||||
pos = end
|
||||
facets.append({"index": span(start, end),
|
||||
"features": [{"$type": "app.bsky.richtext.facet#link", "uri": url}]})
|
||||
urls = [(f["index"]["byteStart"], f["index"]["byteEnd"]) for f in facets]
|
||||
facets.sort(key=lambda f: f["index"]["byteStart"])
|
||||
for m in _TAG_RE.finditer(text):
|
||||
index = span(m.start(), m.end())
|
||||
if any(a <= index["byteStart"] < b for a, b in urls):
|
||||
@@ -547,23 +1137,84 @@ def bluesky_facets(text: str) -> List[dict]:
|
||||
return facets
|
||||
|
||||
|
||||
def _media_files(note_id: int, limit: int) -> List[dict]:
|
||||
"""Image files for a note: the 'large' variant when there is one"""
|
||||
_MD_IMAGE_RE = re.compile(r"!\[[^\]]*\]\([^)]*\)")
|
||||
_HTML_IMAGE_RE = re.compile(r"<img\b[^>]*>", re.IGNORECASE)
|
||||
|
||||
|
||||
def inline_images(markdown_text: str) -> List[tuple]:
|
||||
"""(url, alt) for each image embedded in a note's text, in reading order"""
|
||||
images = []
|
||||
for img in _note_soup(markdown_text).find_all("img"):
|
||||
if img.get("src"):
|
||||
images.append((img["src"], img.get("alt") or ""))
|
||||
return images
|
||||
|
||||
|
||||
def strip_images(markdown_text: str) -> str:
|
||||
"""A note's Markdown without its embedded images"""
|
||||
text = _HTML_IMAGE_RE.sub("", _MD_IMAGE_RE.sub("", markdown_text))
|
||||
return re.sub(r"\n{3,}", "\n\n", text).strip()
|
||||
|
||||
|
||||
def _own_media_path(url: str) -> Optional[str]:
|
||||
"""Path in the media library for one of this site's /media/ URLs"""
|
||||
site = current_app.config["SITE_URL"].rstrip("/")
|
||||
if url.startswith(site):
|
||||
url = url[len(site):]
|
||||
return url[len("/media/"):] if url.startswith("/media/") else None
|
||||
|
||||
|
||||
def _media_files(note_id: int, limit: int, content: str = "") -> List[dict]:
|
||||
"""
|
||||
Image files for a note: the 'large' variant when there is one.
|
||||
|
||||
Attached photos come first, then images embedded in the text that are in
|
||||
this site's media library. Some Micropub clients (OutPost, for one)
|
||||
upload a photo and put it in the content rather than in `photo` (v2.6.6).
|
||||
"""
|
||||
from starpunk.media import get_note_media
|
||||
|
||||
media_dir = Path(current_app.config["DATA_PATH"]) / "media"
|
||||
site = current_app.config["SITE_URL"].rstrip("/")
|
||||
items = [(m, m.get("caption") or "") for m in get_note_media(note_id)]
|
||||
seen = {m["path"] for m, _ in items}
|
||||
for url, alt in inline_images(content) if content else []:
|
||||
path = _own_media_path(url)
|
||||
if not path or path in seen:
|
||||
continue
|
||||
media = _media_by_path(path)
|
||||
if media:
|
||||
seen.add(path)
|
||||
items.append((media, alt or media.get("caption") or ""))
|
||||
|
||||
files = []
|
||||
for media in get_note_media(note_id)[:limit]:
|
||||
for media, description in items[:limit]:
|
||||
variant = (media.get("variants") or {}).get("large")
|
||||
rel = variant["path"] if variant else media["path"]
|
||||
files.append({
|
||||
"path": media_dir / rel,
|
||||
"url": f"{site}/media/{rel}", # for targets that fetch images themselves
|
||||
"mime_type": media.get("mime_type") or "image/jpeg",
|
||||
"description": media.get("caption") or "",
|
||||
"description": description,
|
||||
})
|
||||
return files
|
||||
|
||||
|
||||
def _media_by_path(path: str) -> Optional[dict]:
|
||||
"""A media row with its variants, found by its path in the library"""
|
||||
db = get_db(current_app)
|
||||
row = db.execute("SELECT id, path, mime_type FROM media WHERE path = ?", (path,)).fetchone()
|
||||
if not row:
|
||||
return None
|
||||
variants = {
|
||||
v["variant_type"]: {"path": v["path"]}
|
||||
for v in db.execute(
|
||||
"SELECT variant_type, path FROM media_variants WHERE media_id = ?", (row["id"],)
|
||||
)
|
||||
}
|
||||
return {"path": row["path"], "mime_type": row["mime_type"], "variants": variants, "caption": ""}
|
||||
|
||||
|
||||
# --- Worker -----------------------------------------------------------------
|
||||
|
||||
def _claim(db, row_id: int) -> bool:
|
||||
@@ -615,6 +1266,21 @@ def _fail(db, row, error: SyndicationError) -> None:
|
||||
)
|
||||
|
||||
|
||||
def _target_title(url: Optional[str]) -> Optional[str]:
|
||||
"""Fetched title of a liked/reposted post, if we have one"""
|
||||
if not url:
|
||||
return None
|
||||
try:
|
||||
from starpunk.webmention_queries import get_reply_context
|
||||
context = get_reply_context(url) or {}
|
||||
except Exception:
|
||||
return None
|
||||
title = context.get("title") or ""
|
||||
if not title and context.get("author_name"):
|
||||
title = f"Post by {context['author_name']}"
|
||||
return " ".join(title.split())[:200] or None
|
||||
|
||||
|
||||
def _reconcile(row, target: Target) -> tuple:
|
||||
"""Bring the remote post in line with the note. Returns (remote_id, remote_url)."""
|
||||
from starpunk.notes import get_note
|
||||
@@ -635,9 +1301,13 @@ def _reconcile(row, target: Target) -> tuple:
|
||||
post = Post(
|
||||
note=note,
|
||||
permalink=f"{site}/note/{note.slug}",
|
||||
tags=[t["display_name"] for t in get_note_tags(note.id)],
|
||||
tags=[t["display_name"] for t in get_note_tags(note.id, entered_order=True)],
|
||||
max_chars=target.max_chars,
|
||||
media=_media_files(note.id, target.max_media),
|
||||
media=_media_files(note.id, target.max_media, note.content)
|
||||
if note.kind == "note" else [],
|
||||
kind=note.kind,
|
||||
target_url=note.reacts_to,
|
||||
target_title=_target_title(note.reacts_to),
|
||||
)
|
||||
|
||||
if row["remote_id"]:
|
||||
@@ -658,6 +1328,13 @@ def process_due(limit: int = 5) -> int:
|
||||
targets = get_targets()
|
||||
if not targets:
|
||||
return 0
|
||||
for target in targets.values():
|
||||
maintain = getattr(target.client, "maintain", None)
|
||||
if maintain:
|
||||
try:
|
||||
maintain()
|
||||
except Exception as e:
|
||||
current_app.logger.error(f"Syndication {target.id} upkeep failed: {e}")
|
||||
db = get_db(current_app)
|
||||
rows = db.execute(
|
||||
"""
|
||||
|
||||
+11
-6
@@ -115,27 +115,32 @@ def add_tags_to_note(note_id: int, tags: list[str]) -> None:
|
||||
db.commit()
|
||||
|
||||
|
||||
def get_note_tags(note_id: int) -> list[dict]:
|
||||
def get_note_tags(note_id: int, entered_order: bool = False) -> list[dict]:
|
||||
"""
|
||||
Get all tags for a note
|
||||
|
||||
Args:
|
||||
note_id: Note database ID
|
||||
entered_order: In the order they were entered (the first is the
|
||||
note's main tag, e.g. its Threads topic) instead of alphabetical.
|
||||
add_tags_to_note() rewrites the links in entry order on every
|
||||
save, so note_tags.id records it. Use this wherever the tags are
|
||||
saved back (editor, Micropub update) so the order survives.
|
||||
|
||||
Returns:
|
||||
List of tag dicts with 'name' and 'display_name'
|
||||
Ordered alphabetically by display_name (case-insensitive)
|
||||
|
||||
Query should use: ORDER BY LOWER(tags.display_name) ASC
|
||||
Ordered alphabetically by display_name (case-insensitive), unless
|
||||
entered_order
|
||||
"""
|
||||
db = get_db(current_app)
|
||||
order = "note_tags.id" if entered_order else "LOWER(tags.display_name)"
|
||||
tags = db.execute(
|
||||
"""
|
||||
f"""
|
||||
SELECT tags.name, tags.display_name
|
||||
FROM tags
|
||||
JOIN note_tags ON tags.id = note_tags.tag_id
|
||||
WHERE note_tags.note_id = ?
|
||||
ORDER BY LOWER(tags.display_name) ASC
|
||||
ORDER BY {order} ASC
|
||||
""",
|
||||
(note_id,)
|
||||
).fetchall()
|
||||
|
||||
@@ -92,6 +92,17 @@ main { flex: 1; margin-bottom: var(--spacing-xl); }
|
||||
.e-content ul, .e-content ol { margin: 0 0 var(--spacing-md) 1.25rem; }
|
||||
.e-content img { max-width: 100%; height: auto; }
|
||||
.e-content h1, .e-content h2, .e-content h3 { font-family: var(--font-body); }
|
||||
.reaction { margin-bottom: var(--spacing-sm); font-size: 0.9375rem; color: var(--color-text-light); overflow-wrap: anywhere; }
|
||||
.reaction-verb { font-weight: 600; margin-right: 0.25rem; }
|
||||
.reaction-like .reaction-verb { color: #c2410c; }
|
||||
.reaction-repost .reaction-verb { color: #15803d; }
|
||||
.reaction a { color: var(--color-text); font-weight: 500; }
|
||||
.reaction a:hover { color: var(--color-link); }
|
||||
.reaction-by { margin-left: 0.25rem; }
|
||||
@media (prefers-color-scheme: dark) {
|
||||
.reaction-like .reaction-verb { color: #fb923c; }
|
||||
.reaction-repost .reaction-verb { color: #4ade80; }
|
||||
}
|
||||
.read-more { margin: var(--spacing-sm) 0 0; font-size: 0.9375rem; }
|
||||
|
||||
.note-meta { display: flex; flex-wrap: wrap; align-items: baseline; gap: 0 0.4rem; margin-top: var(--spacing-sm); color: var(--color-text-light); font-size: 0.8125rem; }
|
||||
|
||||
@@ -12,12 +12,15 @@
|
||||
|
||||
<form action="{{ url_for('admin.update_note_submit', note_id=note.id) }}" method="POST" class="note-form">
|
||||
<div class="form-group">
|
||||
<label for="content">Content (Markdown)</label>
|
||||
{% if note.reacts_to %}
|
||||
<p class="note-meta">{{ '★ Like of' if note.kind == 'like' else '🔁 Repost of' }} <a href="{{ note.reacts_to }}">{{ note.reacts_to }}</a></p>
|
||||
{% endif %}
|
||||
<label for="content">{% if note.reacts_to %}Comment (optional, Markdown){% else %}Content (Markdown){% endif %}</label>
|
||||
<textarea
|
||||
id="content"
|
||||
name="content"
|
||||
rows="20"
|
||||
required
|
||||
rows="{{ 6 if note.reacts_to else 20 }}"
|
||||
{% if not note.reacts_to %}required{% endif %}
|
||||
autofocus
|
||||
>{{ note.content }}</textarea>
|
||||
<small>Use Markdown syntax for formatting</small>
|
||||
|
||||
@@ -38,6 +38,7 @@
|
||||
<a class="site-title" href="/">{{ config.SITE_NAME }}</a>
|
||||
<nav class="site-nav">
|
||||
<a href="/">Home</a>
|
||||
<a href="{{ url_for('public.likes') }}">Likes</a>
|
||||
<a href="{{ url_for('public.feed') }}">RSS</a>
|
||||
{% if g.me %}
|
||||
<a href="{{ url_for('admin.dashboard') }}">Admin</a>
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
{% extends "base.html" %}
|
||||
{% from "partials/note_preview.html" import note_preview %}
|
||||
|
||||
{% block title %}Likes · {{ config.SITE_NAME }}{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="h-feed">
|
||||
<h1 class="p-name page-title">Likes</h1>
|
||||
|
||||
{% if notes %}
|
||||
{% for note in notes %}
|
||||
{{ note_preview(note, author) }}
|
||||
{% endfor %}
|
||||
{% else %}
|
||||
<p class="empty-state">Nothing liked yet.</p>
|
||||
{% endif %}
|
||||
|
||||
<nav class="note-nav">
|
||||
<a href="/">← All notes</a>
|
||||
</nav>
|
||||
</div>
|
||||
{% endblock %}
|
||||
+9
-5
@@ -1,7 +1,8 @@
|
||||
{% extends "base.html" %}
|
||||
{% from "partials/media.html" import display_media %}
|
||||
{% from "partials/reaction.html" import reaction %}
|
||||
|
||||
{% block title %}{% if note.content.strip().startswith('#') %}{{ note.title }}{% else %}{{ note.excerpt|truncate(60) }}{% endif %} · {{ config.SITE_NAME }}{% endblock %}
|
||||
{% block title %}{% if note.content.strip().startswith('#') %}{{ note.title }}{% elif note.excerpt.strip() %}{{ note.excerpt|truncate(60) }}{% elif note.repost_of %}Repost{% elif note.like_of %}Like{% elif note.in_reply_to %}Reply{% else %}Photo{% endif %} · {{ config.SITE_NAME }}{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<article class="h-entry">
|
||||
@@ -17,13 +18,14 @@
|
||||
{% if note.in_reply_to %}
|
||||
<div class="reply-context">
|
||||
<span class="reply-label">In reply to:</span>
|
||||
<blockquote class="h-cite">
|
||||
{# u-in-reply-to on the h-cite itself makes it the entry's property (Post Type Discovery) #}
|
||||
<blockquote class="u-in-reply-to h-cite">
|
||||
{% if reply_context and reply_context.fetch_status == 'fetched' %}
|
||||
{% if reply_context.content_snippet %}
|
||||
<p class="p-content">{{ reply_context.content_snippet }}</p>
|
||||
{% endif %}
|
||||
<cite>
|
||||
<a class="u-in-reply-to" href="{{ note.in_reply_to }}">
|
||||
<a class="u-url" href="{{ note.in_reply_to }}">
|
||||
{% if reply_context.author_name %}{{ reply_context.author_name }}{% endif %}
|
||||
{% if reply_context.site_name %} on {{ reply_context.site_name }}{% endif %}
|
||||
{% if not reply_context.author_name and not reply_context.site_name %}{{ note.in_reply_to }}{% endif %}
|
||||
@@ -31,16 +33,18 @@
|
||||
</cite>
|
||||
{% else %}
|
||||
<cite>
|
||||
<a class="u-in-reply-to" href="{{ note.in_reply_to }}">{{ note.in_reply_to }}</a>
|
||||
<a class="u-url" href="{{ note.in_reply_to }}">{{ note.in_reply_to }}</a>
|
||||
</cite>
|
||||
{% endif %}
|
||||
</blockquote>
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{{ reaction(note) }}
|
||||
|
||||
{# u-photo placement: Per draft spec, u-photo must be direct child of h-entry, #}
|
||||
{# NOT inside e-content. Media is rendered ABOVE e-content to meet this requirement. #}
|
||||
{{ display_media(note.media) }}
|
||||
{{ display_media(note.media, note.excerpt|truncate(150)) }}
|
||||
|
||||
{# e-content: note content BELOW media (per ADR-057) #}
|
||||
<div class="e-content">
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
{# Reusable media display macro #}
|
||||
{% macro display_media(media_items) %}
|
||||
{# fallback_alt: for photos without a caption, usually the post's text (v2.6.5) #}
|
||||
{% macro display_media(media_items, fallback_alt='') %}
|
||||
{% if media_items %}
|
||||
<div class="note-media">
|
||||
{% for item in media_items %}
|
||||
<figure class="media-item">
|
||||
<img src="{{ url_for('public.media_file', path=item.path) }}"
|
||||
alt="{{ item.caption or 'Image' }}"
|
||||
alt="{{ item.caption or fallback_alt or 'Image' }}"
|
||||
class="u-photo"
|
||||
loading="lazy">
|
||||
{# No figcaption - caption is for alt text only #}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{# One note in a list (home, tag pages). v2.3.0 #}
|
||||
{% from "partials/media.html" import display_media %}
|
||||
{% from "partials/reaction.html" import reaction %}
|
||||
{% macro note_preview(note, author) %}
|
||||
<article class="h-entry note-preview">
|
||||
{# Notes starting with a # heading have a title (Q22) #}
|
||||
@@ -8,7 +9,13 @@
|
||||
<h2 class="p-name"><a href="{{ absolute_url_for('public.note', slug=note.slug) }}">{{ note.title }}</a></h2>
|
||||
{% endif %}
|
||||
|
||||
{{ display_media(note.media) }}
|
||||
{% if note.in_reply_to %}
|
||||
<p class="reaction reply-to"><span class="reaction-verb">↩ In reply to</span> <a class="u-in-reply-to" href="{{ note.in_reply_to }}">{{ note.in_reply_to }}</a></p>
|
||||
{% endif %}
|
||||
|
||||
{{ reaction(note) }}
|
||||
|
||||
{{ display_media(note.media, note.excerpt|truncate(150)) }}
|
||||
|
||||
{% if has_explicit_title %}
|
||||
{# Titled notes are articles: show a summary and link through #}
|
||||
@@ -17,6 +24,7 @@
|
||||
</div>
|
||||
<p class="read-more"><a href="{{ absolute_url_for('public.note', slug=note.slug) }}">Read more →</a></p>
|
||||
{% else %}
|
||||
{# Always present, even empty: without it a text-free photo post gets an implied name (mf2) #}
|
||||
<div class="e-content">
|
||||
{{ note.html|safe }}
|
||||
</div>
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
{# "Liked" / "Reposted" line for likes and reposts (v2.5.0) #}
|
||||
{% macro reaction(note) %}
|
||||
{% if note.reacts_to %}
|
||||
{% set ctx = note.context if note.context is defined else None %}
|
||||
{% if ctx and ctx.title %}{% set label = ctx.title %}{% elif ctx and ctx.content_snippet %}{% set label = ctx.content_snippet|striptags|truncate(90) %}{% elif ctx and (ctx.author_name or ctx.site_name) %}{% set label = 'a post' %}{% else %}{% set label = note.reacts_to %}{% endif %}
|
||||
<p class="reaction reaction-{{ note.kind }}">
|
||||
<span class="reaction-verb">{% if note.kind == 'like' %}★ Liked{% else %}🔁 Reposted{% endif %}</span>
|
||||
<a class="{{ 'u-like-of' if note.kind == 'like' else 'u-repost-of' }}" href="{{ note.reacts_to }}">{{ label }}</a>
|
||||
{% if ctx and (ctx.author_name or ctx.site_name) %}<span class="reaction-by">by {{ ctx.author_name or ctx.site_name }}</span>{% endif %}
|
||||
</p>
|
||||
{% endif %}
|
||||
{% endmacro %}
|
||||
@@ -0,0 +1,223 @@
|
||||
"""
|
||||
Likes and reposts (v2.5.0, ADR-068)
|
||||
|
||||
Fake GoToSocial and Bluesky servers check that likes/reposts of their own
|
||||
posts become native favourites/boosts/likes/reposts, and everything else
|
||||
falls back (reposts) or is skipped (likes).
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from starpunk import create_app, syndication
|
||||
from starpunk.indieauth import create_access_token
|
||||
from starpunk.notes import delete_note, get_note
|
||||
from starpunk.syndication import BlueskyClient, MastodonClient, get_note_syndications, process_due
|
||||
|
||||
SITE = "https://blog.example.com"
|
||||
GTS = "https://social.example.com"
|
||||
FEDI_POST = "https://mastodon.example/@alice/111"
|
||||
BSKY_POST = "https://bsky.app/profile/alice.example/post/3abc"
|
||||
WEB_PAGE = "https://someblog.example/2026/a-post"
|
||||
|
||||
|
||||
class FakeGTS:
|
||||
def __init__(self):
|
||||
self.calls = []
|
||||
self.statuses = {}
|
||||
|
||||
def __call__(self, request):
|
||||
path = request.url.path
|
||||
self.calls.append((request.method, path))
|
||||
if path == "/api/v2/search":
|
||||
q = request.url.params["q"]
|
||||
return httpx.Response(200, json={"statuses": [{"id": "s1"}] if q == FEDI_POST else []})
|
||||
if path in ("/api/v1/statuses/s1/favourite", "/api/v1/statuses/s1/unfavourite",
|
||||
"/api/v1/statuses/s1/reblog", "/api/v1/statuses/s1/unreblog"):
|
||||
return httpx.Response(200, json={"id": "s1"})
|
||||
if path == "/api/v1/statuses" and request.method == "POST":
|
||||
sid = f"p{len(self.statuses) + 1}"
|
||||
self.statuses[sid] = json.loads(request.content)
|
||||
return httpx.Response(200, json={"id": sid, "url": f"{GTS}/@phil/statuses/{sid}"})
|
||||
if path.startswith("/api/v1/statuses/") and request.method == "DELETE":
|
||||
self.statuses.pop(path.rsplit("/", 1)[1], None)
|
||||
return httpx.Response(200, json={})
|
||||
return httpx.Response(500)
|
||||
|
||||
|
||||
class FakePDS:
|
||||
def __init__(self):
|
||||
self.records = {}
|
||||
self.calls = []
|
||||
|
||||
def __call__(self, request):
|
||||
nsid = request.url.path.removeprefix("/xrpc/")
|
||||
self.calls.append(nsid)
|
||||
if nsid == "com.atproto.server.createSession":
|
||||
return httpx.Response(200, json={"did": "did:plc:me", "handle": "me.example",
|
||||
"accessJwt": "a", "refreshJwt": "r"})
|
||||
if nsid == "com.atproto.identity.resolveHandle":
|
||||
return httpx.Response(200, json={"did": "did:plc:alice"})
|
||||
if nsid == "app.bsky.feed.getPosts":
|
||||
return httpx.Response(200, json={"posts": [{"uri": request.url.params["uris"], "cid": "cid1"}]})
|
||||
if nsid == "com.atproto.repo.createRecord":
|
||||
body = json.loads(request.content)
|
||||
uri = f"at://did:plc:me/{body['collection']}/r{len(self.records) + 1}"
|
||||
self.records[uri] = body["record"]
|
||||
return httpx.Response(200, json={"uri": uri, "cid": "c"})
|
||||
if nsid == "com.atproto.repo.deleteRecord":
|
||||
body = json.loads(request.content)
|
||||
self.records.pop(f"at://{body['repo']}/{body['collection']}/{body['rkey']}", None)
|
||||
return httpx.Response(200, json={})
|
||||
return httpx.Response(500, json={"error": nsid})
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def gts():
|
||||
return FakeGTS()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def pds():
|
||||
return FakePDS()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(tmp_path, gts, pds):
|
||||
data = tmp_path / "data"
|
||||
app = create_app({
|
||||
"TESTING": True, "SITE_URL": SITE, "ADMIN_ME": "https://example.com", "SESSION_SECRET": "x",
|
||||
"DATA_PATH": data, "NOTES_PATH": data / "notes", "DATABASE_PATH": data / "starpunk.db",
|
||||
"GOTOSOCIAL_URL": GTS, "GOTOSOCIAL_ACCESS_TOKEN": "t",
|
||||
"BLUESKY_HANDLE": "me.example", "BLUESKY_APP_PASSWORD": "p", "BLUESKY_SERVICE": "https://pds.example",
|
||||
"FEED_CACHE_ENABLED": False,
|
||||
})
|
||||
# The feed's note list is cached at module level across apps
|
||||
from starpunk.routes import public
|
||||
public._feed_cache["notes"] = None
|
||||
public._feed_cache["timestamp"] = None
|
||||
syndication._client_cache[GTS] = MastodonClient(GTS, "t", http=httpx.Client(transport=httpx.MockTransport(gts)))
|
||||
syndication._client_cache["bluesky:https://pds.example:me.example"] = BlueskyClient(
|
||||
"https://pds.example", "me.example", "p", http=httpx.Client(transport=httpx.MockTransport(pds)))
|
||||
yield app
|
||||
syndication._client_cache.clear()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def post(app, monkeypatch):
|
||||
# Don't fetch contexts from the network
|
||||
monkeypatch.setattr("starpunk.reply_context.fetch_reply_context", lambda *a, **k: None)
|
||||
with app.test_request_context():
|
||||
token = create_access_token("https://quill.example/", "create update delete")["access_token"]
|
||||
client = app.test_client()
|
||||
|
||||
def _post(**props):
|
||||
response = client.post("/micropub", json={"type": ["h-entry"], "properties": props},
|
||||
headers={"Authorization": f"Bearer {token}"})
|
||||
assert response.status_code == 201, response.get_json()
|
||||
location = response.headers["Location"]
|
||||
with app.app_context():
|
||||
note = get_note(slug=location.rsplit("/", 1)[1])
|
||||
process_due(limit=10)
|
||||
rows = {r["target"]: r for r in get_note_syndications(note.id)}
|
||||
return note, rows
|
||||
return _post
|
||||
|
||||
|
||||
def page(app, path):
|
||||
return app.test_client().get(path).data.decode()
|
||||
|
||||
|
||||
class TestSite:
|
||||
def test_like_without_comment(self, app, post):
|
||||
note, _ = post(**{"like-of": [WEB_PAGE]})
|
||||
assert note.kind == "like" and note.content == ""
|
||||
assert note.slug not in page(app, "/")
|
||||
likes = page(app, "/likes")
|
||||
assert f'class="u-like-of" href="{WEB_PAGE}"' in likes and "★ Liked" in likes
|
||||
assert note.slug not in page(app, "/feed.json")
|
||||
|
||||
def test_repost_in_stream_and_feed(self, app, post):
|
||||
note, _ = post(**{"repost-of": [WEB_PAGE], "content": ["Worth a read"]})
|
||||
home = page(app, "/")
|
||||
assert f'class="u-repost-of" href="{WEB_PAGE}"' in home and "Worth a read" in home
|
||||
feed = json.loads(page(app, "/feed.json"))
|
||||
item = next(i for i in feed["items"] if note.slug in i["url"])
|
||||
assert "Reposted" in item["content_html"] and WEB_PAGE in item["content_html"]
|
||||
|
||||
def test_webmention_goes_to_target_with_clean_url(self, app, post, monkeypatch):
|
||||
sent = {}
|
||||
monkeypatch.setattr("starpunk.webmention_sender.queue_webmentions_for_note",
|
||||
lambda **kw: sent.update(kw) or 0)
|
||||
app.config["WEBMENTION_ENABLED"] = True
|
||||
note, _ = post(**{"like-of": [WEB_PAGE]})
|
||||
assert sent["in_reply_to"] == WEB_PAGE
|
||||
assert sent["note_url"] == f"{SITE}/note/{note.slug}" # no double slash
|
||||
|
||||
def test_both_kinds_rejected(self, app, post):
|
||||
with pytest.raises(AssertionError):
|
||||
post(**{"like-of": [WEB_PAGE], "repost-of": [WEB_PAGE]})
|
||||
|
||||
|
||||
class TestGoToSocial:
|
||||
def test_like_of_fediverse_post_favourites(self, app, post, gts):
|
||||
note, rows = post(**{"like-of": [FEDI_POST]})
|
||||
assert rows["gotosocial"]["remote_id"] == "favourite:s1"
|
||||
assert rows["gotosocial"]["remote_url"] is None
|
||||
with app.test_request_context():
|
||||
delete_note(id=note.id)
|
||||
process_due(limit=10)
|
||||
assert ("POST", "/api/v1/statuses/s1/unfavourite") in gts.calls
|
||||
|
||||
def test_like_of_web_page_is_not_crossposted(self, app, post, gts):
|
||||
_, rows = post(**{"like-of": [WEB_PAGE]})
|
||||
assert rows["gotosocial"]["state"] == "synced" and rows["gotosocial"]["remote_id"] is None
|
||||
assert gts.statuses == {}
|
||||
|
||||
def test_repost_boosts(self, app, post, gts):
|
||||
_, rows = post(**{"repost-of": [FEDI_POST]})
|
||||
assert rows["gotosocial"]["remote_id"] == "reblog:s1"
|
||||
|
||||
def test_repost_with_comment_posts_status(self, app, post, gts):
|
||||
_, rows = post(**{"repost-of": [FEDI_POST], "content": ["So true"]})
|
||||
(status,) = gts.statuses.values()
|
||||
assert status["status"].startswith("So true") and FEDI_POST in status["status"]
|
||||
assert ("POST", "/api/v1/statuses/s1/reblog") not in gts.calls
|
||||
|
||||
def test_repost_of_web_page_posts_link(self, app, post, gts):
|
||||
post(**{"repost-of": [WEB_PAGE]})
|
||||
(status,) = gts.statuses.values()
|
||||
assert status["status"].startswith("🔁") and WEB_PAGE in status["status"]
|
||||
|
||||
|
||||
class TestBluesky:
|
||||
def test_like_of_bluesky_post(self, app, post, pds):
|
||||
note, rows = post(**{"like-of": [BSKY_POST]})
|
||||
uri = rows["bluesky"]["remote_id"]
|
||||
assert "/app.bsky.feed.like/" in uri
|
||||
assert pds.records[uri]["subject"] == {"uri": "at://did:plc:alice/app.bsky.feed.post/3abc", "cid": "cid1"}
|
||||
with app.test_request_context():
|
||||
delete_note(id=note.id)
|
||||
process_due(limit=10)
|
||||
assert uri not in pds.records
|
||||
|
||||
def test_like_of_web_page_skipped(self, app, post, pds):
|
||||
_, rows = post(**{"like-of": [WEB_PAGE]})
|
||||
assert rows["bluesky"]["remote_id"] is None and pds.records == {}
|
||||
|
||||
def test_repost_and_quote(self, app, post, pds):
|
||||
_, rows = post(**{"repost-of": [BSKY_POST]})
|
||||
assert "/app.bsky.feed.repost/" in rows["bluesky"]["remote_id"]
|
||||
_, rows = post(**{"repost-of": [BSKY_POST], "content": ["Great thread"]})
|
||||
record = pds.records[rows["bluesky"]["remote_id"]]
|
||||
assert record["text"] == "Great thread"
|
||||
assert record["embed"] == {"$type": "app.bsky.embed.record",
|
||||
"record": {"uri": "at://did:plc:alice/app.bsky.feed.post/3abc", "cid": "cid1"}}
|
||||
|
||||
def test_repost_of_web_page_posts_link_card(self, app, post, pds):
|
||||
_, rows = post(**{"repost-of": [WEB_PAGE]})
|
||||
record = pds.records[rows["bluesky"]["remote_id"]]
|
||||
assert record["embed"]["external"]["uri"] == WEB_PAGE
|
||||
assert record["text"].startswith("🔁")
|
||||
@@ -118,7 +118,7 @@ class TestUpdate:
|
||||
def test_categories(self, app, client, h):
|
||||
url = create(client, h, category=["one", "two"])
|
||||
self.update(client, h, url, add={"category": ["three", "One"]})
|
||||
assert source(client, h, url)["properties"]["category"] == ["one", "three", "two"]
|
||||
assert source(client, h, url)["properties"]["category"] == ["one", "two", "three"] # entered order (v2.6.4)
|
||||
self.update(client, h, url, delete={"category": ["two"]})
|
||||
assert source(client, h, url)["properties"]["category"] == ["one", "three"]
|
||||
self.update(client, h, url, delete=["category"])
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
"""
|
||||
Micropub media fixes (v2.6.5): photo posts without text, the same photo
|
||||
twice, and failed creates that left a post (and a database lock) behind.
|
||||
"""
|
||||
|
||||
import io
|
||||
|
||||
from starpunk.database.pool import ConnectionPool
|
||||
from starpunk.notes import list_notes
|
||||
from starpunk.syndication import compose_text
|
||||
from tests.test_micropub_v240 import SITE, app, client, h, jpeg, note_at, source # noqa: F401
|
||||
|
||||
|
||||
def upload(client, h, color="red"):
|
||||
response = client.post("/micropub/media", data={"file": (io.BytesIO(jpeg(color)), "a.jpg")},
|
||||
headers=h, content_type="multipart/form-data")
|
||||
assert response.status_code == 201
|
||||
return response.headers["Location"]
|
||||
|
||||
|
||||
def post(client, h, props):
|
||||
return client.post("/micropub", json={"type": ["h-entry"], "properties": props}, headers=h)
|
||||
|
||||
|
||||
def update(client, h, url, **ops):
|
||||
return client.post("/micropub", json={"action": "update", "url": url, **ops}, headers=h)
|
||||
|
||||
|
||||
class TestPhotoWithoutText:
|
||||
def test_json_photo_only(self, app, client, h):
|
||||
photo = upload(client, h)
|
||||
response = post(client, h, {"photo": [{"value": photo, "alt": "Red"}]})
|
||||
assert response.status_code == 201, response.get_json()
|
||||
url = response.headers["Location"]
|
||||
assert note_at(app, url).content == ""
|
||||
assert source(client, h, url, "photo")["properties"]["photo"] == [{"value": photo, "alt": "Red"}]
|
||||
page = client.get(url.replace(SITE, ""))
|
||||
assert page.status_code == 200 and b"<title>Photo \xc2\xb7 " in page.data
|
||||
|
||||
def test_multipart_photo_only(self, client, h):
|
||||
response = client.post("/micropub", data={"h": "entry", "photo": (io.BytesIO(jpeg()), "p.jpg")},
|
||||
headers=h, content_type="multipart/form-data")
|
||||
assert response.status_code == 201, response.get_json()
|
||||
|
||||
def test_nothing_at_all_is_still_rejected(self, client, h):
|
||||
assert post(client, h, {}).status_code == 400
|
||||
# A photo that can't be found doesn't count
|
||||
assert post(client, h, {"photo": [f"{SITE}/media/nope.jpg"]}).status_code == 400
|
||||
|
||||
def test_text_can_be_cleared_while_photos_remain(self, app, client, h):
|
||||
photo = upload(client, h)
|
||||
url = post(client, h, {"content": ["Hi"], "photo": [photo]}).headers["Location"]
|
||||
assert update(client, h, url, replace={"content": [""]}).status_code == 204
|
||||
assert note_at(app, url).content == ""
|
||||
assert update(client, h, url, replace={"content": ["Back"]}).status_code == 204
|
||||
assert update(client, h, url, delete=["content"]).status_code == 204
|
||||
assert note_at(app, url).content == ""
|
||||
# Without photos the text has to stay
|
||||
response = update(client, h, url, delete=["content", "photo"])
|
||||
assert response.status_code == 400
|
||||
|
||||
def test_text_is_required_without_photos(self, client, h):
|
||||
url = post(client, h, {"content": ["Hi"]}).headers["Location"]
|
||||
assert update(client, h, url, replace={"content": [""]}).status_code == 400
|
||||
assert update(client, h, url, delete=["content"]).status_code == 400
|
||||
|
||||
def test_crosspost_text_starts_with_the_link(self, app, client, h):
|
||||
url = post(client, h, {"photo": [upload(client, h)]}).headers["Location"]
|
||||
with app.app_context():
|
||||
note = note_at(app, url)
|
||||
assert compose_text(note, url, ["cats"], 500) == f"{url}\n\n#cats"
|
||||
|
||||
|
||||
class TestFailures:
|
||||
def test_same_photo_twice_is_shown_once(self, client, h):
|
||||
photo = upload(client, h)
|
||||
response = post(client, h, {"content": ["Twice"], "photo": [photo, photo]})
|
||||
assert response.status_code == 201, response.get_json()
|
||||
assert source(client, h, response.headers["Location"], "photo")["properties"]["photo"] == [{"value": photo, "alt": ""}]
|
||||
|
||||
def test_failed_create_leaves_no_post_and_no_lock(self, app, client, h, monkeypatch):
|
||||
import starpunk.syndication as syndication
|
||||
|
||||
def boom(*args, **kwargs):
|
||||
raise RuntimeError("crosspost store down")
|
||||
|
||||
monkeypatch.setattr(syndication, "select_targets", boom)
|
||||
response = post(client, h, {"content": ["Doomed"], "photo": [upload(client, h)]})
|
||||
assert response.status_code == 500
|
||||
with app.app_context():
|
||||
assert list_notes(published_only=False) == []
|
||||
monkeypatch.undo()
|
||||
# The database is usable straight away
|
||||
assert post(client, h, {"content": ["Next"]}).status_code == 201
|
||||
|
||||
def test_pool_rolls_back_open_transactions(self, tmp_path):
|
||||
pool = ConnectionPool(str(tmp_path / "x.db"), pool_size=1, timeout=1)
|
||||
conn = pool.get_connection()
|
||||
conn.execute("CREATE TABLE t (x)")
|
||||
conn.commit()
|
||||
conn.execute("INSERT INTO t VALUES (1)")
|
||||
assert conn.in_transaction
|
||||
pool.return_connection(conn)
|
||||
assert not conn.in_transaction
|
||||
assert pool.get_connection().execute("SELECT count(*) FROM t").fetchone()[0] == 0
|
||||
@@ -0,0 +1,98 @@
|
||||
"""
|
||||
Rendered posts read as the right post type (v2.6.5).
|
||||
|
||||
Parses note pages and the home page with mf2py, then runs W3C Post Type
|
||||
Discovery (https://www.w3.org/TR/post-type-discovery/) on what a consumer
|
||||
sees. A photo post is any post with a photo, unless it's first a repost,
|
||||
like or reply.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import mf2py
|
||||
|
||||
import starpunk.reply_context as reply_context
|
||||
from tests.test_micropub_v240 import SITE, app, client, h, jpeg # noqa: F401
|
||||
from tests.test_micropub_v265 import upload
|
||||
|
||||
|
||||
def post_type(props):
|
||||
"""W3C Post Type Discovery, section 5 (no events or RSVPs here)."""
|
||||
for prop, kind in [("repost-of", "repost"), ("like-of", "like"), ("in-reply-to", "reply"),
|
||||
("video", "video"), ("photo", "photo")]:
|
||||
if props.get(prop):
|
||||
return kind
|
||||
|
||||
def first(prop):
|
||||
for v in props.get(prop, []):
|
||||
v = v.get("value", "") if isinstance(v, dict) else v
|
||||
if v.strip():
|
||||
return " ".join(v.split())
|
||||
return ""
|
||||
|
||||
content = first("content") or first("summary")
|
||||
name = first("name")
|
||||
if not content or not name or content.startswith(name):
|
||||
return "note"
|
||||
return "article"
|
||||
|
||||
|
||||
def entries(items):
|
||||
for item in items:
|
||||
if "h-entry" in item.get("type", []):
|
||||
yield item
|
||||
yield from entries(item.get("children", []))
|
||||
|
||||
|
||||
def value(v):
|
||||
"""A u-* value, or the url of a nested microformat (h-cite)."""
|
||||
return v.get("value") or v.get("properties", {}).get("url", [None])[0] if isinstance(v, dict) else v
|
||||
|
||||
|
||||
CASES = {
|
||||
"note": ({"content": ["Just a note"]}, "note"),
|
||||
"article": ({"name": ["A Title"], "content": ["Body text here"]}, "article"),
|
||||
"photo without text": ({"photo": ["PHOTO"]}, "photo"),
|
||||
"photo with caption": ({"content": ["Sunset"], "photo": [{"value": "PHOTO", "alt": "Orange sky"}]}, "photo"),
|
||||
"photo with title": ({"name": ["Trip"], "content": ["Day one"], "photo": ["PHOTO"]}, "photo"),
|
||||
"reply": ({"content": ["Agreed"], "in-reply-to": ["https://other.example/post/1"]}, "reply"),
|
||||
"reply with photo": ({"content": ["Mine"], "in-reply-to": ["https://other.example/post/2"],
|
||||
"photo": ["PHOTO"]}, "reply"),
|
||||
"repost": ({"repost-of": ["https://other.example/post/4"]}, "repost"),
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("case", CASES)
|
||||
def test_post_type(app, client, h, monkeypatch, case):
|
||||
monkeypatch.setattr(reply_context, "fetch_reply_context", lambda *a, **k: None, raising=False)
|
||||
props, want = CASES[case]
|
||||
photo = upload(client, h) if "PHOTO" in str(props) else None
|
||||
props = {k: [photo if v == "PHOTO" else ({**v, "value": photo} if isinstance(v, dict) else v)
|
||||
for v in vs] for k, vs in props.items()}
|
||||
response = client.post("/micropub", json={"type": ["h-entry"], "properties": props}, headers=h)
|
||||
assert response.status_code == 201, response.get_json()
|
||||
url = response.headers["Location"]
|
||||
|
||||
(entry,) = entries(mf2py.parse(doc=client.get(url.replace(SITE, "")).data.decode(), url=url)["items"])
|
||||
home = mf2py.parse(doc=client.get("/").data.decode(), url=f"{SITE}/")
|
||||
(listed,) = [e for e in entries(home["items"]) if e["properties"].get("url") == [url]]
|
||||
|
||||
for e in (entry, listed):
|
||||
assert post_type(e["properties"]) == want
|
||||
if want == "reply":
|
||||
assert value(e["properties"]["in-reply-to"][0]) == props["in-reply-to"][0]
|
||||
if want == "photo" and "name" not in props:
|
||||
assert "name" not in e["properties"], "no implied name to show as a caption"
|
||||
|
||||
|
||||
def test_photo_alt_text(app, client, h):
|
||||
"""Alt text is the caption, else the post's text, never made up (v2.6.5)."""
|
||||
def alts(props):
|
||||
url = client.post("/micropub", json={"type": ["h-entry"], "properties": props},
|
||||
headers=h).headers["Location"]
|
||||
(entry,) = entries(mf2py.parse(doc=client.get(url.replace(SITE, "")).data.decode(), url=url)["items"])
|
||||
return [p["alt"] for p in entry["properties"]["photo"]]
|
||||
|
||||
assert alts({"content": ["Beach day"], "photo": [{"value": upload(client, h), "alt": "Waves"}]}) == ["Waves"]
|
||||
assert alts({"content": ["Beach day"], "photo": [upload(client, h, "blue")]}) == ["Beach day"]
|
||||
assert alts({"name": ["Trip"], "content": ["Day one"], "photo": [upload(client, h, "green")]}) == ["Day one"]
|
||||
assert alts({"photo": [upload(client, h, "gray")]}) == ["Image"]
|
||||
@@ -32,6 +32,7 @@ class FakeServer:
|
||||
self.statuses = {}
|
||||
self.calls = []
|
||||
self.media = 0
|
||||
self.uploads = [] # raw multipart bodies
|
||||
self.fail_next = None # status code to return once
|
||||
|
||||
def __call__(self, request: httpx.Request) -> httpx.Response:
|
||||
@@ -43,6 +44,7 @@ class FakeServer:
|
||||
path = request.url.path
|
||||
if path == "/api/v2/media":
|
||||
self.media += 1
|
||||
self.uploads.append(request.read())
|
||||
return httpx.Response(200, json={"id": f"m{self.media}"})
|
||||
if path == "/api/v1/statuses" and request.method == "POST":
|
||||
body = json.loads(request.content)
|
||||
@@ -271,6 +273,51 @@ class TestMedia:
|
||||
assert server.statuses["1"]["media_ids"] == ["m1"]
|
||||
|
||||
|
||||
class TestInlineImages:
|
||||
"""Clients like OutPost upload a photo and embed it in the content (v2.6.6)"""
|
||||
|
||||
def _upload(self, app):
|
||||
from starpunk.media import save_media
|
||||
|
||||
buf = io.BytesIO()
|
||||
Image.new("RGB", (800, 600), "orange").save(buf, format="JPEG")
|
||||
with app.test_request_context():
|
||||
return save_media(buf.getvalue(), "cat.jpg")
|
||||
|
||||
def test_embedded_own_image_is_attached(self, app, server):
|
||||
media = self._upload(app)
|
||||
url = f"https://blog.example.com/media/{media['path']}"
|
||||
publish(app, content=f"")
|
||||
status = server.statuses["1"]
|
||||
assert status["media_ids"] == ["m1"]
|
||||
assert b"Tortie cat on a chair" in server.uploads[0]
|
||||
assert "![" not in status["status"] and url not in status["status"]
|
||||
assert status["status"].startswith("https://blog.example.com/note/")
|
||||
|
||||
def test_text_around_image_is_kept(self, app, server):
|
||||
media = self._upload(app)
|
||||
publish(app, content=f"Look\n\n<img src=\"/media/{media['path']}\" alt=\"cat\">\n\nNice")
|
||||
status = server.statuses["1"]
|
||||
assert status["media_ids"] == ["m1"]
|
||||
assert status["status"].startswith("Look\n\nNice")
|
||||
|
||||
def test_image_both_attached_and_embedded_is_sent_once(self, app, server):
|
||||
from starpunk.media import attach_media_to_note
|
||||
|
||||
media = self._upload(app)
|
||||
with app.test_request_context():
|
||||
note = create_note(content=f"", published=True)
|
||||
attach_media_to_note(note.id, [media["id"]], ["caption"])
|
||||
select_targets(note.id, ["gotosocial"])
|
||||
process_due()
|
||||
assert server.statuses["1"]["media_ids"] == ["m1"]
|
||||
|
||||
def test_other_sites_images_are_ignored(self, app, server):
|
||||
publish(app, content=" hi")
|
||||
assert server.statuses["1"]["media_ids"] == []
|
||||
assert server.uploads == []
|
||||
|
||||
|
||||
class TestEntryPoints:
|
||||
@pytest.fixture
|
||||
def admin(self, app):
|
||||
|
||||
@@ -92,8 +92,31 @@ def pds():
|
||||
return FakePDS()
|
||||
|
||||
|
||||
class FakeWeb:
|
||||
"""Other people's pages, for link cards: url -> (status, headers, body)"""
|
||||
def __init__(self):
|
||||
self.pages = {}
|
||||
self.fetched = []
|
||||
|
||||
def __call__(self, request):
|
||||
url = str(request.url)
|
||||
self.fetched.append(url)
|
||||
status, headers, body = self.pages.get(url, (404, {}, b"nope"))
|
||||
return httpx.Response(status, headers=headers, content=body)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(tmp_path, pds):
|
||||
def web():
|
||||
return FakeWeb()
|
||||
|
||||
|
||||
def routed(pds, web):
|
||||
return httpx.MockTransport(
|
||||
lambda r: pds(r) if r.url.host in ("bsky.example", "pds.example") else web(r))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(tmp_path, pds, web):
|
||||
data = tmp_path / "data"
|
||||
app = create_app({
|
||||
"TESTING": True,
|
||||
@@ -108,7 +131,7 @@ def app(tmp_path, pds):
|
||||
"BLUESKY_SERVICE": SERVICE,
|
||||
})
|
||||
syndication._client_cache[f"bluesky:{SERVICE}:phil.example"] = BlueskyClient(
|
||||
SERVICE, "phil.example", "app-pass", http=httpx.Client(transport=httpx.MockTransport(pds))
|
||||
SERVICE, "phil.example", "app-pass", http=httpx.Client(transport=routed(pds, web))
|
||||
)
|
||||
yield app
|
||||
syndication._client_cache.clear()
|
||||
@@ -258,6 +281,101 @@ class TestMedia:
|
||||
assert image["aspectRatio"]["width"] > 0
|
||||
|
||||
|
||||
ARTICLE = "https://news.example/story"
|
||||
ARTICLE_HTML = b"""<html><head><title>Fallback title</title>
|
||||
<meta property="og:title" content="A Great Story">
|
||||
<meta property="og:description" content="What happened, in short.">
|
||||
<meta property="og:image" content="/img/story.png">
|
||||
</head><body>...</body></html>"""
|
||||
|
||||
|
||||
def png_bytes():
|
||||
buf = io.BytesIO()
|
||||
Image.new("RGB", (60, 40), (10, 120, 200)).save(buf, format="PNG")
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
class TestLinkCards:
|
||||
def test_note_link_gets_the_card(self, app, pds, web):
|
||||
web.pages[ARTICLE] = (200, {"content-type": "text/html"}, ARTICLE_HTML)
|
||||
web.pages["https://news.example/img/story.png"] = (200, {"content-type": "image/png"}, png_bytes())
|
||||
note = publish(app, content=f"Read [this story]({ARTICLE}) now")
|
||||
record = only_record(pds)
|
||||
external = record["embed"]["external"]
|
||||
assert external["uri"] == ARTICLE
|
||||
assert external["title"] == "A Great Story"
|
||||
assert external["description"] == "What happened, in short."
|
||||
assert external["thumb"]["$type"] == "blob"
|
||||
# The link text is clickable and the note's permalink is still in the text
|
||||
permalink = f"https://blog.example.com/note/{note.slug}"
|
||||
assert record["text"].startswith("Read this story now") and permalink in record["text"]
|
||||
raw = record["text"].encode()
|
||||
links = {raw[f["index"]["byteStart"]:f["index"]["byteEnd"]].decode(): f["features"][0]["uri"]
|
||||
for f in record["facets"]}
|
||||
assert links["this story"] == ARTICLE
|
||||
assert links[permalink] == permalink
|
||||
|
||||
def test_bare_url_gets_the_card(self, app, pds, web):
|
||||
web.pages[ARTICLE] = (200, {"content-type": "text/html"}, ARTICLE_HTML)
|
||||
publish(app, content=f"See {ARTICLE}.")
|
||||
assert only_record(pds)["embed"]["external"]["uri"] == ARTICLE
|
||||
|
||||
def test_unreachable_page_still_gets_a_card(self, app, pds, web):
|
||||
note = publish(app, content=f"See {ARTICLE}")
|
||||
external = only_record(pds)["embed"]["external"]
|
||||
assert external == {"uri": ARTICLE, "title": ARTICLE, "description": ""}
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
def test_redirect_to_private_address_is_not_followed(self, app, pds, web, monkeypatch):
|
||||
web.pages[ARTICLE] = (302, {"location": "http://internal.example/admin"}, b"")
|
||||
monkeypatch.setattr("starpunk.micropub._public_address", lambda host: host != "internal.example")
|
||||
app.config["TESTING"] = False
|
||||
try:
|
||||
publish(app, content=f"See {ARTICLE}")
|
||||
finally:
|
||||
app.config["TESTING"] = True
|
||||
assert "http://internal.example/admin" not in web.fetched
|
||||
assert only_record(pds)["embed"]["external"]["title"] == ARTICLE
|
||||
|
||||
def test_images_win_over_link_card(self, app, pds, web):
|
||||
from starpunk.media import attach_media_to_note, save_media
|
||||
with app.test_request_context():
|
||||
note = create_note(content=f"Look {ARTICLE}", published=True)
|
||||
media = save_media(png_bytes(), "a.png")
|
||||
attach_media_to_note(note.id, [media["id"]], ["pic"])
|
||||
select_targets(note.id, ["bluesky"])
|
||||
process_due()
|
||||
assert only_record(pds)["embed"]["$type"] == "app.bsky.embed.images"
|
||||
assert web.fetched == []
|
||||
|
||||
|
||||
class TestLinkText:
|
||||
def test_note_links_in_reading_order(self):
|
||||
from starpunk.syndication import note_links
|
||||
md = "a [x](https://one.example) b https://two.example/p. c <https://three.example> [rel](/local)"
|
||||
assert note_links(md) == ["https://one.example", "https://two.example/p", "https://three.example"]
|
||||
|
||||
def test_link_modes(self):
|
||||
from starpunk.syndication import plain_text_links
|
||||
md = "[a story](https://s.example) and [https://u.example](https://u.example)"
|
||||
assert plain_text_links(md)[0] == "a story and https://u.example"
|
||||
assert plain_text_links(md, "inline")[0] == "a story (https://s.example) and https://u.example"
|
||||
assert plain_text_links(md, "anchor") == ("a story and https://u.example",
|
||||
[("a story", "https://s.example")])
|
||||
|
||||
def test_inline_url_budget(self):
|
||||
from starpunk.syndication import plain_text_links
|
||||
md = "https://bare.example then [one](https://1.example) [two](https://2.example) [one again](https://1.example)"
|
||||
text = plain_text_links(md, "inline", max_urls=2)[0]
|
||||
assert text == "https://bare.example then one (https://1.example) two one again (https://1.example)"
|
||||
|
||||
def test_anchor_facet_byte_offsets(self):
|
||||
text = "Café ☕ this story and more"
|
||||
(facet,) = bluesky_facets(text, [("this story", "https://s.example")])
|
||||
raw = text.encode()
|
||||
assert raw[facet["index"]["byteStart"]:facet["index"]["byteEnd"]] == b"this story"
|
||||
|
||||
|
||||
def test_both_targets_listed_for_micropub(tmp_path):
|
||||
data = tmp_path / "d"
|
||||
app = create_app({
|
||||
|
||||
@@ -0,0 +1,418 @@
|
||||
"""
|
||||
Tests for crossposting to Threads (starpunk/syndication.py, v2.6.0)
|
||||
|
||||
A fake Graph API (httpx.MockTransport) records what StarPunk sends.
|
||||
"""
|
||||
|
||||
import io
|
||||
import json
|
||||
import time
|
||||
from urllib.parse import parse_qsl
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from starpunk import create_app, syndication
|
||||
from starpunk.notes import create_note, delete_note, update_note
|
||||
from starpunk.syndication import (
|
||||
THREADS_REFRESH_EVERY,
|
||||
ThreadsClient,
|
||||
get_note_syndications,
|
||||
process_due,
|
||||
select_targets,
|
||||
threads_length,
|
||||
threads_topic,
|
||||
)
|
||||
|
||||
API = "https://graph.threads.example/v1.0"
|
||||
UID = "1789"
|
||||
|
||||
|
||||
class FakeThreads:
|
||||
def __init__(self):
|
||||
self.containers = {}
|
||||
self.posts = {}
|
||||
self.calls = []
|
||||
self.token = "tok1"
|
||||
self.refreshes = 0
|
||||
self.refresh_fails = False
|
||||
self.pending_polls = 0
|
||||
self.lose_next_publish = False
|
||||
self.fail_next = None
|
||||
self.fail_next_publish = False
|
||||
|
||||
def _json(self, code, body):
|
||||
return httpx.Response(code, json=body)
|
||||
|
||||
def _error(self, code, message, err_code, subcode=None):
|
||||
error = {"message": message, "type": "OAuthException", "code": err_code}
|
||||
if subcode:
|
||||
error["error_subcode"] = subcode
|
||||
return self._json(code, {"error": error})
|
||||
|
||||
def __call__(self, request: httpx.Request) -> httpx.Response:
|
||||
params = dict(request.url.params)
|
||||
if request.content:
|
||||
params.update(parse_qsl(request.content.decode()))
|
||||
path = request.url.path
|
||||
self.calls.append((request.method, path, params))
|
||||
|
||||
if path == "/refresh_access_token":
|
||||
assert params["grant_type"] == "th_refresh_token"
|
||||
if self.refresh_fails or params.get("access_token") != self.token:
|
||||
return self._error(400, "token too new", 190)
|
||||
self.refreshes += 1
|
||||
self.token = f"tok{self.refreshes + 1}"
|
||||
return self._json(200, {"access_token": self.token, "token_type": "bearer",
|
||||
"expires_in": 5184000})
|
||||
|
||||
if params.get("access_token") != self.token:
|
||||
return self._error(400, "Invalid OAuth access token", 190)
|
||||
if self.fail_next:
|
||||
code, err, self.fail_next = *self.fail_next, None
|
||||
return self._error(code, "nope", err)
|
||||
|
||||
path = path.removeprefix("/v1.0/")
|
||||
if path == "me":
|
||||
return self._json(200, {"id": UID, "username": "phil"})
|
||||
if path == f"{UID}/threads" and request.method == "POST":
|
||||
cid = f"c{len(self.containers) + 1}"
|
||||
self.containers[cid] = params
|
||||
return self._json(200, {"id": cid})
|
||||
if path == f"{UID}/threads" and request.method == "GET":
|
||||
return self._json(200, {"data": [
|
||||
{"id": pid, "text": p["text"], "permalink": f"https://www.threads.com/@phil/post/{pid}"}
|
||||
for pid, p in self.posts.items()]})
|
||||
if path == f"{UID}/threads_publish" and self.fail_next_publish:
|
||||
self.fail_next_publish = False
|
||||
return self._error(400, "Fatal", 24, 4279009)
|
||||
if path == f"{UID}/threads_publish":
|
||||
container = self.containers[params["creation_id"]]
|
||||
pid = f"p{len(self.posts) + 1}"
|
||||
self.posts[pid] = container
|
||||
if self.lose_next_publish:
|
||||
self.lose_next_publish = False
|
||||
raise httpx.ReadTimeout("lost reply")
|
||||
return self._json(200, {"id": pid})
|
||||
if path in self.containers and request.method == "GET":
|
||||
if self.pending_polls:
|
||||
self.pending_polls -= 1
|
||||
return self._json(200, {"status": "IN_PROGRESS"})
|
||||
return self._json(200, {"status": "FINISHED"})
|
||||
if path in self.posts and request.method == "GET":
|
||||
return self._json(200, {"id": path, "permalink": f"https://www.threads.com/@phil/post/{path}"})
|
||||
if request.method == "DELETE":
|
||||
if self.posts.pop(path, None) is None:
|
||||
return self._error(400, "Object does not exist", 100, 33)
|
||||
return self._json(200, {"success": True, "deleted_id": path})
|
||||
return self._json(500, {"error": {"message": "unknown", "code": 2}})
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def api():
|
||||
return FakeThreads()
|
||||
|
||||
|
||||
def make_client(app, api, token="tok1"):
|
||||
return ThreadsClient(token, app.config["DATA_PATH"] / "threads_token.json", api=API,
|
||||
http=httpx.Client(transport=httpx.MockTransport(api)), sleep=lambda s: None)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(tmp_path, api):
|
||||
data = tmp_path / "data"
|
||||
app = create_app({
|
||||
"TESTING": True,
|
||||
"SITE_URL": "https://blog.example.com",
|
||||
"ADMIN_ME": "https://example.com",
|
||||
"SESSION_SECRET": "test-secret",
|
||||
"DATA_PATH": data,
|
||||
"NOTES_PATH": data / "notes",
|
||||
"DATABASE_PATH": data / "starpunk.db",
|
||||
"THREADS_ACCESS_TOKEN": "tok1",
|
||||
"THREADS_USERNAME": "phil",
|
||||
})
|
||||
syndication._client_cache["threads"] = make_client(app, api)
|
||||
yield app
|
||||
syndication._client_cache.clear()
|
||||
|
||||
|
||||
def publish(app, content="Hello Threads", tags=None):
|
||||
with app.test_request_context():
|
||||
note = create_note(content=content, published=True, tags=tags)
|
||||
select_targets(note.id, ["threads"])
|
||||
process_due()
|
||||
return note
|
||||
|
||||
|
||||
def only_post(api):
|
||||
(post,) = api.posts.values()
|
||||
return post
|
||||
|
||||
|
||||
def row(app, note):
|
||||
with app.app_context():
|
||||
(r,) = get_note_syndications(note.id)
|
||||
return r
|
||||
|
||||
|
||||
def image_bytes(fmt="JPEG", size=(800, 600)):
|
||||
buf = io.BytesIO()
|
||||
Image.new("RGB", size, (200, 30, 30)).save(buf, format=fmt)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
class TestText:
|
||||
def test_emoji_count_as_bytes(self):
|
||||
assert threads_length("abc") == 3
|
||||
assert threads_length("☕") == 3
|
||||
|
||||
def test_topic_is_first_usable_tag(self):
|
||||
assert threads_topic(["...", "Indie.Web & Co", "x"]) == "IndieWeb Co"
|
||||
assert threads_topic([]) == ""
|
||||
|
||||
|
||||
class TestLifecycle:
|
||||
def test_publish_text_post(self, app, api):
|
||||
note = publish(app, content="Hello **Threads**", tags=["IndieWeb", "Second"])
|
||||
post = only_post(api)
|
||||
permalink = f"https://blog.example.com/note/{note.slug}"
|
||||
assert post["media_type"] == "TEXT"
|
||||
assert post["text"] == f"Hello Threads\n\n{permalink}"
|
||||
assert "link_attachment" not in post # Threads previews the first link itself
|
||||
assert post["topic_tag"] == "IndieWeb"
|
||||
r = row(app, note)
|
||||
assert r["state"] == "synced"
|
||||
assert r["remote_id"] == "p1"
|
||||
assert r["remote_url"] == "https://www.threads.com/@phil/post/p1"
|
||||
|
||||
def test_long_note_fits_500(self, app, api):
|
||||
note = publish(app, content="☕ word " * 300)
|
||||
text = only_post(api)["text"]
|
||||
assert threads_length(text) <= 500
|
||||
assert text.endswith(f"https://blog.example.com/note/{note.slug}")
|
||||
|
||||
def test_waits_for_container(self, app, api):
|
||||
api.pending_polls = 2
|
||||
note = publish(app)
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
def test_edit_leaves_post_alone(self, app, api):
|
||||
note = publish(app, content="first")
|
||||
with app.test_request_context():
|
||||
update_note(id=note.id, content="second")
|
||||
process_due()
|
||||
assert only_post(api)["text"].startswith("first")
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
def test_delete_and_unpublish_remove_post(self, app, api):
|
||||
note = publish(app)
|
||||
with app.test_request_context():
|
||||
update_note(id=note.id, published=False)
|
||||
process_due()
|
||||
assert api.posts == {}
|
||||
update_note(id=note.id, published=True)
|
||||
process_due()
|
||||
assert len(api.posts) == 1
|
||||
delete_note(id=note.id)
|
||||
process_due()
|
||||
assert api.posts == {}
|
||||
|
||||
def test_post_deleted_on_threads_is_fine(self, app, api):
|
||||
note = publish(app)
|
||||
api.posts.clear()
|
||||
with app.test_request_context():
|
||||
delete_note(id=note.id)
|
||||
process_due()
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
|
||||
class TestErrors:
|
||||
def test_lost_reply_does_not_double_post(self, app, api):
|
||||
api.lose_next_publish = True
|
||||
note = publish(app)
|
||||
assert row(app, note)["state"] == "pending"
|
||||
with app.test_request_context():
|
||||
from starpunk.database import get_db
|
||||
get_db(app).execute("UPDATE syndications SET next_attempt_at = datetime('now')")
|
||||
get_db(app).commit()
|
||||
process_due()
|
||||
assert len(api.posts) == 1
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
def test_publish_fatal_is_retried(self, app, api):
|
||||
api.fail_next_publish = True
|
||||
note = publish(app)
|
||||
r = row(app, note)
|
||||
assert r["state"] == "pending" and "Fatal" in r["last_error"] and "subcode 4279009" in r["last_error"]
|
||||
with app.test_request_context():
|
||||
from starpunk.database import get_db
|
||||
get_db(app).execute("UPDATE syndications SET next_attempt_at = datetime('now')")
|
||||
get_db(app).commit()
|
||||
process_due()
|
||||
assert len(api.posts) == 1
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
def test_bad_token_fails_permanently(self, app, api):
|
||||
api.token = "other"
|
||||
note = publish(app)
|
||||
r = row(app, note)
|
||||
assert r["state"] == "failed" and "Invalid OAuth" in r["last_error"]
|
||||
assert "tok1" not in r["last_error"]
|
||||
|
||||
def test_rate_limit_retries(self, app, api):
|
||||
publish(app, content="warm up")
|
||||
api.fail_next = (400, 4) # Graph "application request limit reached"
|
||||
note = publish(app, content="later")
|
||||
assert row(app, note)["state"] == "pending"
|
||||
|
||||
|
||||
class TestLinks:
|
||||
def test_markdown_link_comes_first_for_the_preview(self, app, api):
|
||||
note = publish(app, content="Read [this story](https://news.example/story) now")
|
||||
text = only_post(api)["text"]
|
||||
assert text.startswith("Read this story (https://news.example/story) now")
|
||||
assert text.index("https://news.example/story") < text.index(f"/note/{note.slug}")
|
||||
|
||||
def test_stays_within_five_links(self, app, api):
|
||||
import re
|
||||
links = " ".join(f"[l{i}](https://l{i}.example)" for i in range(7))
|
||||
publish(app, content=f"Lots: {links}")
|
||||
text = only_post(api)["text"]
|
||||
assert len(set(re.findall(r"https?://[^\s)]+", text))) == 5
|
||||
assert "l6" in text # the extra links keep their text
|
||||
|
||||
|
||||
class TestMedia:
|
||||
def _note_with(self, app, images):
|
||||
from starpunk.media import attach_media_to_note, save_media
|
||||
with app.test_request_context():
|
||||
note = create_note(content="pics", published=True)
|
||||
ids = [save_media(data, name)["id"] for data, name in images]
|
||||
attach_media_to_note(note.id, ids, [f"alt {i}" for i in range(len(ids))])
|
||||
select_targets(note.id, ["threads"])
|
||||
process_due()
|
||||
return note
|
||||
|
||||
def test_single_image(self, app, api):
|
||||
self._note_with(app, [(image_bytes(), "a.jpg")])
|
||||
post = only_post(api)
|
||||
assert post["media_type"] == "IMAGE"
|
||||
assert post["image_url"].startswith("https://blog.example.com/media/")
|
||||
assert post["alt_text"] == "alt 0"
|
||||
assert "link_attachment" not in post
|
||||
|
||||
def test_carousel(self, app, api):
|
||||
self._note_with(app, [(image_bytes(), "a.jpg"), (image_bytes("PNG"), "b.png")])
|
||||
post = only_post(api)
|
||||
assert post["media_type"] == "CAROUSEL"
|
||||
children = [api.containers[c] for c in post["children"].split(",")]
|
||||
assert [c["is_carousel_item"] for c in children] == ["true", "true"]
|
||||
assert [c["alt_text"] for c in children] == ["alt 0", "alt 1"]
|
||||
|
||||
def test_unsupported_format_is_left_off(self, app, api):
|
||||
self._note_with(app, [(image_bytes("WEBP"), "a.webp")])
|
||||
assert only_post(api)["media_type"] == "TEXT"
|
||||
|
||||
|
||||
class TestToken:
|
||||
def test_refreshes_daily_and_saves(self, app, api):
|
||||
client = syndication._client_cache["threads"]
|
||||
with app.app_context():
|
||||
client.maintain()
|
||||
assert api.refreshes == 1
|
||||
client.maintain() # same day: nothing
|
||||
assert api.refreshes == 1
|
||||
saved = json.loads((app.config["DATA_PATH"] / "threads_token.json").read_text())
|
||||
assert saved["token"] == "tok2"
|
||||
# A restart picks up the saved token
|
||||
assert make_client(app, api)._token == "tok2"
|
||||
publish(app)
|
||||
assert len(api.posts) == 1
|
||||
|
||||
def test_other_worker_refresh_is_reused(self, app, api):
|
||||
first, second = make_client(app, api), make_client(app, api)
|
||||
with app.app_context():
|
||||
first.maintain()
|
||||
second.maintain()
|
||||
assert api.refreshes == 1
|
||||
assert second._token == "tok2"
|
||||
|
||||
def test_refresh_due_after_a_day(self, app, api, monkeypatch):
|
||||
client = syndication._client_cache["threads"]
|
||||
with app.app_context():
|
||||
client.maintain()
|
||||
later = time.time() + THREADS_REFRESH_EVERY + 1
|
||||
monkeypatch.setattr(syndication.time, "time", lambda: later)
|
||||
client.maintain()
|
||||
assert api.refreshes == 2
|
||||
|
||||
def test_failed_refresh_keeps_token(self, app, api):
|
||||
api.refresh_fails = True
|
||||
client = syndication._client_cache["threads"]
|
||||
with app.app_context():
|
||||
client.maintain()
|
||||
assert client._token == "tok1"
|
||||
publish(app)
|
||||
assert len(api.posts) == 1
|
||||
|
||||
def test_new_config_token_replaces_saved_one(self, app, api):
|
||||
with app.app_context():
|
||||
syndication._client_cache["threads"].maintain()
|
||||
api.token = "fresh"
|
||||
assert make_client(app, api, token="fresh")._token == "fresh"
|
||||
|
||||
def test_worker_runs_upkeep(self, app, api):
|
||||
with app.app_context():
|
||||
process_due()
|
||||
assert api.refreshes == 1
|
||||
|
||||
|
||||
def test_all_targets_listed_for_micropub(tmp_path):
|
||||
data = tmp_path / "d"
|
||||
app = create_app({
|
||||
"TESTING": True, "SITE_URL": "https://b.example", "ADMIN_ME": "https://e.example",
|
||||
"SESSION_SECRET": "x", "DATA_PATH": data, "NOTES_PATH": data / "notes",
|
||||
"DATABASE_PATH": data / "s.db",
|
||||
"GOTOSOCIAL_URL": "https://social.example", "GOTOSOCIAL_ACCESS_TOKEN": "t",
|
||||
"BLUESKY_HANDLE": "phil.example", "BLUESKY_APP_PASSWORD": "p",
|
||||
"THREADS_ACCESS_TOKEN": "x", "THREADS_USERNAME": "@phil",
|
||||
})
|
||||
with app.app_context():
|
||||
from starpunk.syndication import get_targets, resolve_targets
|
||||
assert set(get_targets()) == {"gotosocial", "bluesky", "threads"}
|
||||
assert resolve_targets(["https://www.threads.com/@phil"]) == ["threads"]
|
||||
syndication._client_cache.clear()
|
||||
|
||||
|
||||
class TestLikesReposts:
|
||||
PAGE = "https://other.example/post/1"
|
||||
|
||||
def _post(self, app, monkeypatch, content="", **kind):
|
||||
monkeypatch.setattr("starpunk.reply_context.fetch_reply_context", lambda *a, **k: None)
|
||||
with app.test_request_context():
|
||||
note = create_note(content=content, published=True, **kind)
|
||||
select_targets(note.id, ["threads"])
|
||||
process_due()
|
||||
return note
|
||||
|
||||
def test_like_is_not_crossposted(self, app, api, monkeypatch):
|
||||
note = self._post(app, monkeypatch, like_of=self.PAGE)
|
||||
assert api.posts == {}
|
||||
r = row(app, note)
|
||||
assert r["state"] == "synced" and r["remote_id"] is None
|
||||
|
||||
def test_repost_posts_link(self, app, api, monkeypatch):
|
||||
self._post(app, monkeypatch, repost_of=self.PAGE)
|
||||
post = only_post(api)
|
||||
assert post["text"].startswith("🔁") and self.PAGE in post["text"]
|
||||
assert "link_attachment" not in post
|
||||
|
||||
def test_repost_with_comment_keeps_comment(self, app, api, monkeypatch):
|
||||
note = self._post(app, monkeypatch, content="So **true**", repost_of=self.PAGE)
|
||||
assert only_post(api)["text"].startswith("So true")
|
||||
with app.test_request_context():
|
||||
delete_note(id=note.id)
|
||||
process_due()
|
||||
assert api.posts == {}
|
||||
@@ -0,0 +1,86 @@
|
||||
"""
|
||||
Tags keep the order they were entered in (v2.6.4)
|
||||
|
||||
Public pages list tags alphabetically; crossposts (the Threads topic, which
|
||||
tags fit on Bluesky), the editor and Micropub use the entered order, and
|
||||
saving from the editor or Micropub keeps it.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from starpunk import create_app
|
||||
from starpunk.auth import create_session
|
||||
from starpunk.indieauth import create_access_token
|
||||
from starpunk.notes import create_note, get_note
|
||||
from starpunk.tags import get_note_tags
|
||||
|
||||
ENTERED = ["Zebra", "apple", "Mango"]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(tmp_path):
|
||||
data = tmp_path / "data"
|
||||
data.mkdir()
|
||||
yield create_app(config={
|
||||
"TESTING": True, "DATABASE_PATH": data / "starpunk.db", "DATA_PATH": data,
|
||||
"NOTES_PATH": data / "notes", "SESSION_SECRET": "s", "ADMIN_ME": "https://test.example.com",
|
||||
"SITE_URL": "http://localhost:5000", "DEV_MODE": False,
|
||||
})
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def note(app):
|
||||
with app.test_request_context():
|
||||
return create_note(content="Tagged", published=True, tags=ENTERED)
|
||||
|
||||
|
||||
def names(app, note_id, **kw):
|
||||
with app.app_context():
|
||||
return [t["display_name"] for t in get_note_tags(note_id, **kw)]
|
||||
|
||||
|
||||
def test_alphabetical_by_default_entered_on_request(app, note):
|
||||
assert names(app, note.id) == ["apple", "Mango", "Zebra"]
|
||||
assert names(app, note.id, entered_order=True) == ENTERED
|
||||
|
||||
|
||||
def test_public_page_stays_alphabetical(app, note):
|
||||
html = app.test_client().get(f"/note/{note.slug}").data.decode()
|
||||
assert html.index(">apple<") < html.index(">Mango<") < html.index(">Zebra<")
|
||||
|
||||
|
||||
def test_editor_shows_and_keeps_entered_order(app, note):
|
||||
client = app.test_client()
|
||||
with app.test_request_context():
|
||||
client.set_cookie("starpunk_session", create_session("https://test.example.com"))
|
||||
html = client.get(f"/admin/edit/{note.id}").data.decode()
|
||||
assert 'value="Zebra, apple, Mango"' in html
|
||||
# Saving the form unchanged keeps the order
|
||||
response = client.post(f"/admin/edit/{note.id}", data={
|
||||
"content": "Tagged, edited", "published": "on", "tags": "Zebra, apple, Mango"})
|
||||
assert response.status_code in (200, 302)
|
||||
assert names(app, note.id, entered_order=True) == ENTERED
|
||||
|
||||
|
||||
def test_micropub_source_and_update_keep_entered_order(app, note):
|
||||
with app.test_request_context():
|
||||
token = create_access_token("https://quill.example/", "create update")["access_token"]
|
||||
client = app.test_client()
|
||||
auth = {"Authorization": f"Bearer {token}"}
|
||||
url = f"http://localhost:5000/note/{note.slug}"
|
||||
source = client.get(f"/micropub?q=source&url={url}", headers=auth).get_json()
|
||||
assert source["properties"]["category"] == ENTERED
|
||||
response = client.post("/micropub", headers=auth, json={
|
||||
"action": "update", "url": url, "add": {"category": ["Kiwi"]}})
|
||||
assert response.status_code in (200, 204), response.get_data(as_text=True)
|
||||
assert names(app, note.id, entered_order=True) == ENTERED + ["Kiwi"]
|
||||
|
||||
|
||||
def test_crossposts_use_entered_order(app, note):
|
||||
from starpunk.syndication import compose_text, threads_topic
|
||||
tags = names(app, note.id, entered_order=True)
|
||||
assert threads_topic(tags) == "Zebra"
|
||||
with app.app_context():
|
||||
text = compose_text(get_note(id=note.id, load_content=True),
|
||||
"https://x.example/note/1", tags, 25)
|
||||
assert "#Zebra" in text and "#Mango" not in text # room for the first tags only
|
||||
Reference in new issue
Block a user