Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
565adaeecf | ||
|
|
53d10ed271 |
No files matched your search
@@ -63,6 +63,14 @@ GOTOSOCIAL_NAME=GoToSocial
|
||||
# public, unlisted, private (default: public)
|
||||
GOTOSOCIAL_VISIBILITY=public
|
||||
|
||||
# Threads account (v2.6.0). A long-lived user token with threads_basic,
|
||||
# threads_content_publish and threads_delete; StarPunk refreshes it daily and
|
||||
# keeps the current one in DATA_PATH/threads_token.json.
|
||||
# How to get one: docs/design/v2.6.0/2026-10-31-threads-crossposting.md
|
||||
THREADS_ACCESS_TOKEN=
|
||||
# Your Threads username (for the Micropub syndicate-to uid)
|
||||
THREADS_USERNAME=
|
||||
|
||||
# =============================================================================
|
||||
# DATA STORAGE
|
||||
# =============================================================================
|
||||
|
||||
@@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.6.0] - 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
- **Crossposting to Threads** (ADR-069): set `THREADS_ACCESS_TOKEN` (and optionally `THREADS_USERNAME`) to add a Threads target. Text posts get a link preview; 1 image posts as an image, 2–4 as a carousel, with alt text; the note's first tag becomes the post's topic. Shortened to 500 characters. Edits don't follow (the API can't edit); unpublish and delete remove the post. Likes aren't crossposted to Threads; reposts post "🔁 title url" (with the comment, if any)
|
||||
- The Threads token (60-day lifetime) is refreshed automatically once a day and kept in `data/threads_token.json`, so it survives restarts; setting a new token in the config replaces it. Setup steps: `docs/design/v2.6.0/`
|
||||
|
||||
## [2.5.0] - 2026-11-01
|
||||
|
||||
### Added
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
# ADR-069: Crossposting to Threads
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.6.0, 2026-10-31)
|
||||
|
||||
## Context
|
||||
|
||||
ADR-065 and ADR-066 crosspost notes to GoToSocial and Bluesky. Threads
|
||||
(Meta) has a publishing API, so it can be a third target on the same loop.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Third client, same loop.** `ThreadsClient` is target id `threads`
|
||||
behind the `syndications` table and reconcile loop from ADR-065.
|
||||
2. **Auth.** A long-lived Threads user token (`THREADS_ACCESS_TOKEN`) with
|
||||
`threads_basic`, `threads_content_publish` and `threads_delete`. These
|
||||
tokens expire after 60 days, so StarPunk refreshes the token about once a
|
||||
day (`th_refresh_token`) from the worker loop, even when nothing is being
|
||||
posted. The current token is kept in `DATA_PATH/threads_token.json`
|
||||
(mode 0600) with a hash of the configured token. A different configured
|
||||
token replaces the saved one. A file lock stops gunicorn workers from
|
||||
refreshing at the same time. The file avoids a migration for one value.
|
||||
3. **Posting.** Create a container, poll its status until `FINISHED`, then
|
||||
`threads_publish`. Text-only notes are `TEXT` posts with the permalink as
|
||||
`link_attachment`. One image is an `IMAGE` post and 2–4 images are a
|
||||
`CAROUSEL`. Meta fetches images from their public `/media/` URL. Only
|
||||
JPEG and PNG up to 8 MB are allowed, so GIF and WebP images are left off.
|
||||
Alt text goes in `alt_text`.
|
||||
4. **Content.** Plain text (as for Bluesky), then the permalink, shortened
|
||||
to 500 characters. Non-ASCII characters are counted as UTF-8 bytes,
|
||||
because Threads counts emoji that way. Threads allows one tag per post,
|
||||
so the note's first tag is sent as `topic_tag` and no hashtags are added
|
||||
to the text.
|
||||
5. **Edits don't follow.** The API has no way to edit a post. As with
|
||||
Bluesky, the post stays as first sent, and unpublishing, unticking the
|
||||
target or deleting the note removes it.
|
||||
6. **Likes and reposts (ADR-068).** The API can't like or repost outside
|
||||
content, so likes are not crossposted. Reposts are text posts with
|
||||
"🔁 title" and the URL, plus the comment if there is one, and the
|
||||
reposted page as the link preview.
|
||||
7. **No double posts.** A retried create first looks for a recent post that
|
||||
already contains the permalink.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Meta limits accounts to 250 posts and 100 deletes per day. Rate-limit
|
||||
errors (Graph codes 4, 17, 32, 613) are retried.
|
||||
- If StarPunk is down for more than 60 days, the token expires and a new
|
||||
one is needed.
|
||||
- Getting the first token requires a Meta developer app with the Threads
|
||||
use case. See the setup notes in `docs/design/v2.6.0/`.
|
||||
- Notes with more than five links fail on Threads (a Meta limit). The
|
||||
error is shown in the editor.
|
||||
@@ -0,0 +1,62 @@
|
||||
# v2.6.0: Crossposting to Threads
|
||||
|
||||
Implements ADR-069.
|
||||
|
||||
## What changed
|
||||
|
||||
- `starpunk/syndication.py`: `ThreadsClient` (target `threads`); media
|
||||
entries now carry their public `url`; `process_due()` calls each client's
|
||||
optional `maintain()` hook, which Threads uses to refresh its token.
|
||||
- `starpunk/config.py`: `THREADS_ACCESS_TOKEN`, `THREADS_USERNAME`,
|
||||
`THREADS_NAME`, `THREADS_MAX_CHARS` (500), `THREADS_MAX_MEDIA` (4).
|
||||
- `tests/test_syndication_threads.py`: a fake Graph API covering text,
|
||||
image and carousel posts; container polling; edits not following; deletes
|
||||
(including posts already deleted by hand); a lost publish reply;
|
||||
permanent and rate-limit errors; and token refresh, persistence and
|
||||
replacement.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Variable | Meaning |
|
||||
|---|---|
|
||||
| `THREADS_ACCESS_TOKEN` | Long-lived Threads user token. Enables the target. |
|
||||
| `THREADS_USERNAME` | Optional. Used for the Micropub `syndicate-to` uid (`https://www.threads.com/@name`) and for fallback links. |
|
||||
| `THREADS_NAME` | Label in the editor (default `Threads`) |
|
||||
|
||||
After the first daily refresh, the working token lives in
|
||||
`DATA_PATH/threads_token.json`. The configured value only seeds it.
|
||||
Changing `THREADS_ACCESS_TOKEN` switches to the new token.
|
||||
|
||||
## Getting a token (one time)
|
||||
|
||||
1. At <https://developers.facebook.com/apps>, create an app with the
|
||||
**Access the Threads API** use case.
|
||||
2. Under the use case's permissions, add `threads_basic`,
|
||||
`threads_content_publish` and `threads_delete`.
|
||||
3. Under **App roles → Roles**, add your Threads account as a Threads
|
||||
tester. Accept the invite in Threads (Settings → Account → Website
|
||||
permissions → Invites). The app can stay in development mode.
|
||||
4. In the use case settings, add `https://<site>/` as a redirect URI. Then
|
||||
open:
|
||||
|
||||
https://threads.net/oauth/authorize?client_id=APP_ID&redirect_uri=https://<site>/&scope=threads_basic,threads_content_publish,threads_delete&response_type=code
|
||||
|
||||
After you approve, the browser lands on `https://<site>/?code=...`. Copy
|
||||
the code (drop a trailing `#_`).
|
||||
5. Exchange the code for a short-lived token, then exchange that for a
|
||||
long-lived one:
|
||||
|
||||
curl -s -X POST https://graph.threads.net/oauth/access_token \
|
||||
-F client_id=APP_ID -F client_secret=APP_SECRET \
|
||||
-F grant_type=authorization_code -F redirect_uri=https://<site>/ -F code=CODE
|
||||
curl -s "https://graph.threads.net/access_token?grant_type=th_exchange_token&client_secret=APP_SECRET&access_token=SHORT_TOKEN"
|
||||
|
||||
6. Set `THREADS_ACCESS_TOKEN` to the long-lived `access_token`.
|
||||
|
||||
The app secret is only needed for these steps. StarPunk doesn't use it.
|
||||
|
||||
## Not done
|
||||
|
||||
- Video, GIF and WebP images (Threads takes video by URL. StarPunk only
|
||||
stores still images, and Threads rejects GIF and WebP).
|
||||
- Edits (no API for them).
|
||||
@@ -464,5 +464,5 @@ def create_app(config=None):
|
||||
|
||||
# Package version (Semantic Versioning 2.0.0)
|
||||
# See docs/standards/versioning-strategy.md for details
|
||||
__version__ = "2.5.0"
|
||||
__version_info__ = (2, 1, 0)
|
||||
__version__ = "2.6.0"
|
||||
__version_info__ = (2, 6, 0)
|
||||
@@ -137,6 +137,12 @@ def load_config(app, config_override=None):
|
||||
app.config["BLUESKY_SERVICE"] = os.getenv("BLUESKY_SERVICE", "https://bsky.social")
|
||||
app.config["BLUESKY_NAME"] = os.getenv("BLUESKY_NAME", "Bluesky")
|
||||
app.config["BLUESKY_MAX_CHARS"] = int(os.getenv("BLUESKY_MAX_CHARS", "300"))
|
||||
# Crossposting to Threads (v2.6.0, ADR-069): long-lived token, refreshed automatically
|
||||
app.config["THREADS_ACCESS_TOKEN"] = os.getenv("THREADS_ACCESS_TOKEN", "")
|
||||
app.config["THREADS_USERNAME"] = os.getenv("THREADS_USERNAME", "")
|
||||
app.config["THREADS_NAME"] = os.getenv("THREADS_NAME", "Threads")
|
||||
app.config["THREADS_MAX_CHARS"] = int(os.getenv("THREADS_MAX_CHARS", "500"))
|
||||
app.config["THREADS_MAX_MEDIA"] = int(os.getenv("THREADS_MAX_MEDIA", "4"))
|
||||
app.config["SYNDICATION_POLL_INTERVAL"] = int(os.getenv("SYNDICATION_POLL_INTERVAL", "10"))
|
||||
|
||||
# Apply overrides if provided
|
||||
|
||||
+289
-4
@@ -1,10 +1,10 @@
|
||||
"""
|
||||
Crossposting (POSSE) for StarPunk (v2.1.0 ADR-065, v2.2.0 ADR-066)
|
||||
Crossposting (POSSE) for StarPunk (v2.1.0 ADR-065, v2.2.0 ADR-066, v2.6.0 ADR-069)
|
||||
|
||||
Published notes are copied to syndication targets, and later edits and
|
||||
deletes follow them. Two target kinds exist: a GoToSocial (or any
|
||||
Mastodon-API) account, and a Bluesky account. Bluesky posts can't be edited,
|
||||
so there only deletes follow.
|
||||
deletes follow them. Three target kinds exist: a GoToSocial (or any
|
||||
Mastodon-API) account, a Bluesky account and a Threads account. Bluesky and
|
||||
Threads posts can't be edited, so there only deletes follow.
|
||||
|
||||
Each (note, target) pair has a row in `syndications`. Changing a note marks
|
||||
its rows pending; a background worker then reconciles the remote post with
|
||||
@@ -17,9 +17,15 @@ the note:
|
||||
Rows carry a version so an edit made while a send is in flight is not lost.
|
||||
"""
|
||||
|
||||
import fcntl
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
@@ -28,6 +34,9 @@ from typing import Dict, List, Optional
|
||||
import httpx
|
||||
from flask import current_app
|
||||
|
||||
# httpx logs request URLs at INFO, and Threads takes its token in the URL
|
||||
logging.getLogger("httpx").setLevel(logging.WARNING)
|
||||
|
||||
from starpunk.database import get_db
|
||||
|
||||
MAX_ATTEMPTS = 8
|
||||
@@ -400,6 +409,255 @@ def _fit_image(path: Path, mime_type: str, limit: int) -> tuple:
|
||||
raise SyndicationError(f"{Path(path).name}: image too large for Bluesky", permanent=True)
|
||||
|
||||
|
||||
THREADS_API = "https://graph.threads.net/v1.0"
|
||||
THREADS_IMAGE_TYPES = {"image/jpeg", "image/png"}
|
||||
THREADS_IMAGE_LIMIT = 8_000_000
|
||||
THREADS_REFRESH_EVERY = 24 * 3600 # tokens last 60 days; Meta allows a refresh once one is a day old
|
||||
# Graph API error codes that are temporary (rate limits, service hiccups)
|
||||
_GRAPH_TRANSIENT = {1, 2, 4, 17, 32, 341, 613}
|
||||
|
||||
|
||||
def threads_length(text: str) -> int:
|
||||
"""Length as Threads counts it: emoji (and, to be safe, all non-ASCII) by UTF-8 bytes"""
|
||||
return sum(1 if ord(ch) < 128 else len(ch.encode()) for ch in text)
|
||||
|
||||
|
||||
def threads_topic(tags: List[str]) -> str:
|
||||
"""The first tag usable as a Threads topic (one per post; no '.' or '&', 1-50 chars)"""
|
||||
for tag in tags:
|
||||
topic = re.sub(r"[.&]", "", tag).strip()[:50]
|
||||
if topic:
|
||||
return topic
|
||||
return ""
|
||||
|
||||
|
||||
class ThreadsClient:
|
||||
"""
|
||||
Minimal Threads API client (Meta Graph API)
|
||||
|
||||
Posting is two steps: create a container, wait for it to be ready, then
|
||||
publish it. Meta fetches images from their public URL on this site.
|
||||
Posts can't be edited through the API, so only deletes follow.
|
||||
|
||||
Long-lived tokens expire after 60 days unless refreshed. `maintain()`
|
||||
refreshes the token once a day and keeps it in `token_file`, so it
|
||||
survives restarts. A different token in the config replaces it.
|
||||
"""
|
||||
|
||||
can_edit = False
|
||||
|
||||
def __init__(self, token: str, token_file: Optional[Path] = None, api: str = THREADS_API,
|
||||
http: Optional[httpx.Client] = None, sleep=time.sleep):
|
||||
self.config_token = token
|
||||
self.token_file = Path(token_file) if token_file else None
|
||||
self.api = api.rstrip("/")
|
||||
self.http = http or httpx.Client(timeout=30.0)
|
||||
self.sleep = sleep
|
||||
self.user_id: Optional[str] = None
|
||||
self.username: Optional[str] = None
|
||||
self._lock = threading.Lock()
|
||||
self._token = token
|
||||
self._checked_at = 0.0
|
||||
saved = self._read_file()
|
||||
if saved:
|
||||
self._token = saved["token"]
|
||||
|
||||
# --- Token ---
|
||||
|
||||
def _seed(self) -> str:
|
||||
return hashlib.sha256(self.config_token.encode()).hexdigest()
|
||||
|
||||
def _read_file(self) -> Optional[dict]:
|
||||
"""The saved token, if it descends from the configured one"""
|
||||
if not self.token_file or not self.token_file.exists():
|
||||
return None
|
||||
try:
|
||||
data = json.loads(self.token_file.read_text())
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
return data if data.get("seed") == self._seed() and data.get("token") else None
|
||||
|
||||
def _write_file(self, token: str, refreshed_at: float) -> None:
|
||||
tmp = self.token_file.with_suffix(".tmp")
|
||||
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||
with os.fdopen(fd, "w") as f:
|
||||
json.dump({"seed": self._seed(), "token": token, "refreshed_at": refreshed_at}, f)
|
||||
os.replace(tmp, self.token_file)
|
||||
|
||||
def maintain(self) -> None:
|
||||
"""Refresh the token if a day has passed since the last try (cheap to call often)"""
|
||||
now = time.time()
|
||||
if now - self._checked_at < THREADS_REFRESH_EVERY:
|
||||
return
|
||||
self._checked_at = now
|
||||
if not self.token_file:
|
||||
return
|
||||
# Gunicorn workers share the file: one refreshes, the others pick up its result
|
||||
self.token_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(self.token_file.with_suffix(".lock"), "w") as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX)
|
||||
saved = self._read_file()
|
||||
if saved:
|
||||
self._token = saved["token"]
|
||||
if now - saved.get("refreshed_at", 0) < THREADS_REFRESH_EVERY:
|
||||
self._checked_at = saved["refreshed_at"]
|
||||
return
|
||||
base = self.api.rsplit("/", 1)[0] # refresh lives outside the versioned path
|
||||
try:
|
||||
response = self.http.get(f"{base}/refresh_access_token", params={
|
||||
"grant_type": "th_refresh_token", "access_token": self._token})
|
||||
except httpx.HTTPError as e:
|
||||
current_app.logger.warning(f"Threads token refresh failed: {type(e).__name__}")
|
||||
self._checked_at = now - THREADS_REFRESH_EVERY + 3600 # try again in an hour
|
||||
return
|
||||
if response.status_code < 400:
|
||||
self._token = response.json()["access_token"]
|
||||
current_app.logger.info("Threads access token refreshed")
|
||||
else:
|
||||
# e.g. a token less than a day old: keep it, try again tomorrow
|
||||
current_app.logger.warning(
|
||||
f"Threads token refresh failed: HTTP {response.status_code} {response.text[:200]}")
|
||||
self._write_file(self._token, now)
|
||||
|
||||
# --- Graph plumbing ---
|
||||
|
||||
def _call(self, method: str, path: str, **params) -> dict:
|
||||
params["access_token"] = self._token
|
||||
kwargs = {"params": params} if method in ("GET", "DELETE") else {"data": params}
|
||||
try:
|
||||
response = self.http.request(method, f"{self.api}/{path}", **kwargs)
|
||||
except httpx.HTTPError as e:
|
||||
raise SyndicationError(f"{method} {path}: {type(e).__name__}") # never echo the URL/token
|
||||
try:
|
||||
body = response.json()
|
||||
except ValueError:
|
||||
body = {}
|
||||
error = body.get("error") if isinstance(body, dict) else None
|
||||
if response.status_code >= 400 or error:
|
||||
error = error if isinstance(error, dict) else {}
|
||||
code = error.get("code")
|
||||
message = (f"{method} {path}: HTTP {response.status_code} "
|
||||
f"{error.get('message') or response.text[:200]}")
|
||||
if response.status_code == 404 or (code == 100 and error.get("error_subcode") == 33):
|
||||
raise RemoteGone(message, permanent=True)
|
||||
permanent = (400 <= response.status_code < 500 and response.status_code != 429
|
||||
and code not in _GRAPH_TRANSIENT)
|
||||
raise SyndicationError(message, permanent=permanent)
|
||||
return body
|
||||
|
||||
def _me(self) -> str:
|
||||
with self._lock:
|
||||
if not self.user_id:
|
||||
me = self._call("GET", "me", fields="id,username")
|
||||
self.user_id, self.username = me["id"], me.get("username")
|
||||
return self.user_id
|
||||
|
||||
def _wait(self, container_id: str) -> None:
|
||||
"""Wait until Meta has processed a container (images take a few seconds)"""
|
||||
delay = 1
|
||||
for _ in range(8):
|
||||
status = self._call("GET", container_id, fields="status,error_message")
|
||||
state = status.get("status")
|
||||
if state in ("FINISHED", None):
|
||||
return
|
||||
if state in ("ERROR", "EXPIRED"):
|
||||
raise SyndicationError(
|
||||
f"Threads container {state}: {status.get('error_message') or 'no detail'}",
|
||||
permanent=True)
|
||||
self.sleep(delay)
|
||||
delay = min(delay * 2, 15)
|
||||
raise SyndicationError("Threads container still processing")
|
||||
|
||||
# --- Posts ---
|
||||
|
||||
def _text(self, post: "Post") -> str:
|
||||
body = plain_text(post.note.content)
|
||||
limit = post.max_chars
|
||||
while True:
|
||||
text = compose_text(post.note, post.permalink, [], limit, body=body)
|
||||
over = threads_length(text) - post.max_chars
|
||||
if over <= 0:
|
||||
return text
|
||||
limit -= over
|
||||
|
||||
def _image_container(self, uid: str, media: dict, **extra) -> str:
|
||||
params = {"media_type": "IMAGE", "image_url": media["url"], **extra}
|
||||
if media["description"]:
|
||||
params["alt_text"] = media["description"][:1000]
|
||||
return self._call("POST", f"{uid}/threads", **params)["id"]
|
||||
|
||||
def _repost_text(self, post: "Post") -> str:
|
||||
comment = plain_text(post.note.content)
|
||||
limit = post.max_chars
|
||||
while True:
|
||||
text = compose_repost_text(post, comment, limit)
|
||||
over = threads_length(text) - post.max_chars
|
||||
if over <= 0:
|
||||
return text
|
||||
limit -= over
|
||||
|
||||
def _container(self, uid: str, post: "Post") -> str:
|
||||
if post.kind == "repost":
|
||||
# No native repost of outside content: post "🔁 title url" with a link preview
|
||||
return self._call("POST", f"{uid}/threads", media_type="TEXT",
|
||||
text=self._repost_text(post), link_attachment=post.target_url)["id"]
|
||||
common = {"text": self._text(post)}
|
||||
topic = threads_topic(post.tags)
|
||||
if topic:
|
||||
common["topic_tag"] = topic
|
||||
# Threads takes JPEG/PNG up to 8 MB; others (GIF, WebP) are left off
|
||||
images = [m for m in post.media if m["mime_type"] in THREADS_IMAGE_TYPES
|
||||
and Path(m["path"]).stat().st_size <= THREADS_IMAGE_LIMIT]
|
||||
if not images:
|
||||
return self._call("POST", f"{uid}/threads", media_type="TEXT",
|
||||
link_attachment=post.permalink, **common)["id"]
|
||||
if len(images) == 1:
|
||||
return self._image_container(uid, images[0], **common)
|
||||
children = [self._image_container(uid, m, is_carousel_item="true") for m in images]
|
||||
for child in children:
|
||||
self._wait(child)
|
||||
return self._call("POST", f"{uid}/threads", media_type="CAROUSEL",
|
||||
children=",".join(children), **common)["id"]
|
||||
|
||||
def _profile_url(self) -> str:
|
||||
return f"https://www.threads.com/@{self.username}" if self.username else "https://www.threads.com/"
|
||||
|
||||
def _find_existing(self, uid: str, link: str) -> Optional[dict]:
|
||||
"""A recent post that already has this link (an earlier send whose reply was lost)"""
|
||||
listing = self._call("GET", f"{uid}/threads", fields="id,text,permalink", limit=25)
|
||||
for item in listing.get("data", []):
|
||||
if link in (item.get("text") or ""):
|
||||
return {"id": item["id"], "url": item.get("permalink") or self._profile_url()}
|
||||
return None
|
||||
|
||||
def create(self, post: "Post", key: str, retrying: bool = False) -> dict:
|
||||
if post.kind == "like" and post.target_url:
|
||||
return {"id": None, "url": None} # the API can't like things; likes stay on the site
|
||||
uid = self._me()
|
||||
if retrying:
|
||||
marker = post.target_url if post.kind == "repost" and post.target_url else post.permalink
|
||||
existing = self._find_existing(uid, marker)
|
||||
if existing:
|
||||
return existing
|
||||
container = self._container(uid, post)
|
||||
self._wait(container)
|
||||
media_id = self._call("POST", f"{uid}/threads_publish", creation_id=container)["id"]
|
||||
try:
|
||||
url = self._call("GET", media_id, fields="permalink").get("permalink")
|
||||
except SyndicationError:
|
||||
url = None # it's published; don't fail (and post twice) over the link
|
||||
return {"id": media_id, "url": url or self._profile_url()}
|
||||
|
||||
def update(self, remote_id: str, post: "Post") -> dict:
|
||||
raise NotImplementedError("Threads posts can't be edited through the API")
|
||||
|
||||
def delete(self, remote_id: str) -> None:
|
||||
try:
|
||||
self._call("DELETE", remote_id)
|
||||
except RemoteGone:
|
||||
pass # already gone
|
||||
|
||||
|
||||
@dataclass
|
||||
class Target:
|
||||
id: str # stored in syndications.target
|
||||
@@ -446,6 +704,7 @@ def targets_configured(config) -> bool:
|
||||
return bool(
|
||||
(config.get("GOTOSOCIAL_URL") and config.get("GOTOSOCIAL_ACCESS_TOKEN"))
|
||||
or (config.get("BLUESKY_HANDLE") and config.get("BLUESKY_APP_PASSWORD"))
|
||||
or config.get("THREADS_ACCESS_TOKEN")
|
||||
)
|
||||
|
||||
|
||||
@@ -487,6 +746,23 @@ def get_targets() -> Dict[str, Target]:
|
||||
max_chars=int(config.get("BLUESKY_MAX_CHARS", 300)),
|
||||
max_media=4,
|
||||
)
|
||||
|
||||
token = config.get("THREADS_ACCESS_TOKEN") or ""
|
||||
if token:
|
||||
client = _client_cache.get("threads")
|
||||
if client is None or client.config_token != token:
|
||||
client = ThreadsClient(token, Path(config["DATA_PATH"]) / "threads_token.json",
|
||||
api=config.get("THREADS_API") or THREADS_API)
|
||||
_client_cache["threads"] = client
|
||||
username = (config.get("THREADS_USERNAME") or "").lstrip("@")
|
||||
targets["threads"] = Target(
|
||||
id="threads",
|
||||
uid=f"https://www.threads.com/@{username}" if username else "https://www.threads.com/",
|
||||
name=config.get("THREADS_NAME", "Threads"),
|
||||
client=client,
|
||||
max_chars=int(config.get("THREADS_MAX_CHARS", 500)),
|
||||
max_media=int(config.get("THREADS_MAX_MEDIA", 4)),
|
||||
)
|
||||
return targets
|
||||
|
||||
|
||||
@@ -661,12 +937,14 @@ def _media_files(note_id: int, limit: int) -> List[dict]:
|
||||
from starpunk.media import get_note_media
|
||||
|
||||
media_dir = Path(current_app.config["DATA_PATH"]) / "media"
|
||||
site = current_app.config["SITE_URL"].rstrip("/")
|
||||
files = []
|
||||
for media in get_note_media(note_id)[:limit]:
|
||||
variant = (media.get("variants") or {}).get("large")
|
||||
rel = variant["path"] if variant else media["path"]
|
||||
files.append({
|
||||
"path": media_dir / rel,
|
||||
"url": f"{site}/media/{rel}", # for targets that fetch images themselves
|
||||
"mime_type": media.get("mime_type") or "image/jpeg",
|
||||
"description": media.get("caption") or "",
|
||||
})
|
||||
@@ -785,6 +1063,13 @@ def process_due(limit: int = 5) -> int:
|
||||
targets = get_targets()
|
||||
if not targets:
|
||||
return 0
|
||||
for target in targets.values():
|
||||
maintain = getattr(target.client, "maintain", None)
|
||||
if maintain:
|
||||
try:
|
||||
maintain()
|
||||
except Exception as e:
|
||||
current_app.logger.error(f"Syndication {target.id} upkeep failed: {e}")
|
||||
db = get_db(current_app)
|
||||
rows = db.execute(
|
||||
"""
|
||||
|
||||
@@ -0,0 +1,385 @@
|
||||
"""
|
||||
Tests for crossposting to Threads (starpunk/syndication.py, v2.6.0)
|
||||
|
||||
A fake Graph API (httpx.MockTransport) records what StarPunk sends.
|
||||
"""
|
||||
|
||||
import io
|
||||
import json
|
||||
import time
|
||||
from urllib.parse import parse_qsl
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from starpunk import create_app, syndication
|
||||
from starpunk.notes import create_note, delete_note, update_note
|
||||
from starpunk.syndication import (
|
||||
THREADS_REFRESH_EVERY,
|
||||
ThreadsClient,
|
||||
get_note_syndications,
|
||||
process_due,
|
||||
select_targets,
|
||||
threads_length,
|
||||
threads_topic,
|
||||
)
|
||||
|
||||
API = "https://graph.threads.example/v1.0"
|
||||
UID = "1789"
|
||||
|
||||
|
||||
class FakeThreads:
|
||||
def __init__(self):
|
||||
self.containers = {}
|
||||
self.posts = {}
|
||||
self.calls = []
|
||||
self.token = "tok1"
|
||||
self.refreshes = 0
|
||||
self.refresh_fails = False
|
||||
self.pending_polls = 0
|
||||
self.lose_next_publish = False
|
||||
self.fail_next = None
|
||||
|
||||
def _json(self, code, body):
|
||||
return httpx.Response(code, json=body)
|
||||
|
||||
def _error(self, code, message, err_code, subcode=None):
|
||||
error = {"message": message, "type": "OAuthException", "code": err_code}
|
||||
if subcode:
|
||||
error["error_subcode"] = subcode
|
||||
return self._json(code, {"error": error})
|
||||
|
||||
def __call__(self, request: httpx.Request) -> httpx.Response:
|
||||
params = dict(request.url.params)
|
||||
if request.content:
|
||||
params.update(parse_qsl(request.content.decode()))
|
||||
path = request.url.path
|
||||
self.calls.append((request.method, path, params))
|
||||
|
||||
if path == "/refresh_access_token":
|
||||
assert params["grant_type"] == "th_refresh_token"
|
||||
if self.refresh_fails or params.get("access_token") != self.token:
|
||||
return self._error(400, "token too new", 190)
|
||||
self.refreshes += 1
|
||||
self.token = f"tok{self.refreshes + 1}"
|
||||
return self._json(200, {"access_token": self.token, "token_type": "bearer",
|
||||
"expires_in": 5184000})
|
||||
|
||||
if params.get("access_token") != self.token:
|
||||
return self._error(400, "Invalid OAuth access token", 190)
|
||||
if self.fail_next:
|
||||
code, err, self.fail_next = *self.fail_next, None
|
||||
return self._error(code, "nope", err)
|
||||
|
||||
path = path.removeprefix("/v1.0/")
|
||||
if path == "me":
|
||||
return self._json(200, {"id": UID, "username": "phil"})
|
||||
if path == f"{UID}/threads" and request.method == "POST":
|
||||
cid = f"c{len(self.containers) + 1}"
|
||||
self.containers[cid] = params
|
||||
return self._json(200, {"id": cid})
|
||||
if path == f"{UID}/threads" and request.method == "GET":
|
||||
return self._json(200, {"data": [
|
||||
{"id": pid, "text": p["text"], "permalink": f"https://www.threads.com/@phil/post/{pid}"}
|
||||
for pid, p in self.posts.items()]})
|
||||
if path == f"{UID}/threads_publish":
|
||||
container = self.containers[params["creation_id"]]
|
||||
pid = f"p{len(self.posts) + 1}"
|
||||
self.posts[pid] = container
|
||||
if self.lose_next_publish:
|
||||
self.lose_next_publish = False
|
||||
raise httpx.ReadTimeout("lost reply")
|
||||
return self._json(200, {"id": pid})
|
||||
if path in self.containers and request.method == "GET":
|
||||
if self.pending_polls:
|
||||
self.pending_polls -= 1
|
||||
return self._json(200, {"status": "IN_PROGRESS"})
|
||||
return self._json(200, {"status": "FINISHED"})
|
||||
if path in self.posts and request.method == "GET":
|
||||
return self._json(200, {"id": path, "permalink": f"https://www.threads.com/@phil/post/{path}"})
|
||||
if request.method == "DELETE":
|
||||
if self.posts.pop(path, None) is None:
|
||||
return self._error(400, "Object does not exist", 100, 33)
|
||||
return self._json(200, {"success": True, "deleted_id": path})
|
||||
return self._json(500, {"error": {"message": "unknown", "code": 2}})
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def api():
|
||||
return FakeThreads()
|
||||
|
||||
|
||||
def make_client(app, api, token="tok1"):
|
||||
return ThreadsClient(token, app.config["DATA_PATH"] / "threads_token.json", api=API,
|
||||
http=httpx.Client(transport=httpx.MockTransport(api)), sleep=lambda s: None)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(tmp_path, api):
|
||||
data = tmp_path / "data"
|
||||
app = create_app({
|
||||
"TESTING": True,
|
||||
"SITE_URL": "https://blog.example.com",
|
||||
"ADMIN_ME": "https://example.com",
|
||||
"SESSION_SECRET": "test-secret",
|
||||
"DATA_PATH": data,
|
||||
"NOTES_PATH": data / "notes",
|
||||
"DATABASE_PATH": data / "starpunk.db",
|
||||
"THREADS_ACCESS_TOKEN": "tok1",
|
||||
"THREADS_USERNAME": "phil",
|
||||
})
|
||||
syndication._client_cache["threads"] = make_client(app, api)
|
||||
yield app
|
||||
syndication._client_cache.clear()
|
||||
|
||||
|
||||
def publish(app, content="Hello Threads", tags=None):
|
||||
with app.test_request_context():
|
||||
note = create_note(content=content, published=True, tags=tags)
|
||||
select_targets(note.id, ["threads"])
|
||||
process_due()
|
||||
return note
|
||||
|
||||
|
||||
def only_post(api):
|
||||
(post,) = api.posts.values()
|
||||
return post
|
||||
|
||||
|
||||
def row(app, note):
|
||||
with app.app_context():
|
||||
(r,) = get_note_syndications(note.id)
|
||||
return r
|
||||
|
||||
|
||||
def image_bytes(fmt="JPEG", size=(800, 600)):
|
||||
buf = io.BytesIO()
|
||||
Image.new("RGB", size, (200, 30, 30)).save(buf, format=fmt)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
class TestText:
|
||||
def test_emoji_count_as_bytes(self):
|
||||
assert threads_length("abc") == 3
|
||||
assert threads_length("☕") == 3
|
||||
|
||||
def test_topic_is_first_usable_tag(self):
|
||||
assert threads_topic(["...", "Indie.Web & Co", "x"]) == "IndieWeb Co"
|
||||
assert threads_topic([]) == ""
|
||||
|
||||
|
||||
class TestLifecycle:
|
||||
def test_publish_text_post(self, app, api):
|
||||
note = publish(app, content="Hello **Threads**", tags=["IndieWeb", "Second"])
|
||||
post = only_post(api)
|
||||
permalink = f"https://blog.example.com/note/{note.slug}"
|
||||
assert post["media_type"] == "TEXT"
|
||||
assert post["text"] == f"Hello Threads\n\n{permalink}"
|
||||
assert post["link_attachment"] == permalink
|
||||
assert post["topic_tag"] == "IndieWeb"
|
||||
r = row(app, note)
|
||||
assert r["state"] == "synced"
|
||||
assert r["remote_id"] == "p1"
|
||||
assert r["remote_url"] == "https://www.threads.com/@phil/post/p1"
|
||||
|
||||
def test_long_note_fits_500(self, app, api):
|
||||
note = publish(app, content="☕ word " * 300)
|
||||
text = only_post(api)["text"]
|
||||
assert threads_length(text) <= 500
|
||||
assert text.endswith(f"https://blog.example.com/note/{note.slug}")
|
||||
|
||||
def test_waits_for_container(self, app, api):
|
||||
api.pending_polls = 2
|
||||
note = publish(app)
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
def test_edit_leaves_post_alone(self, app, api):
|
||||
note = publish(app, content="first")
|
||||
with app.test_request_context():
|
||||
update_note(id=note.id, content="second")
|
||||
process_due()
|
||||
assert only_post(api)["text"].startswith("first")
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
def test_delete_and_unpublish_remove_post(self, app, api):
|
||||
note = publish(app)
|
||||
with app.test_request_context():
|
||||
update_note(id=note.id, published=False)
|
||||
process_due()
|
||||
assert api.posts == {}
|
||||
update_note(id=note.id, published=True)
|
||||
process_due()
|
||||
assert len(api.posts) == 1
|
||||
delete_note(id=note.id)
|
||||
process_due()
|
||||
assert api.posts == {}
|
||||
|
||||
def test_post_deleted_on_threads_is_fine(self, app, api):
|
||||
note = publish(app)
|
||||
api.posts.clear()
|
||||
with app.test_request_context():
|
||||
delete_note(id=note.id)
|
||||
process_due()
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
|
||||
class TestErrors:
|
||||
def test_lost_reply_does_not_double_post(self, app, api):
|
||||
api.lose_next_publish = True
|
||||
note = publish(app)
|
||||
assert row(app, note)["state"] == "pending"
|
||||
with app.test_request_context():
|
||||
from starpunk.database import get_db
|
||||
get_db(app).execute("UPDATE syndications SET next_attempt_at = datetime('now')")
|
||||
get_db(app).commit()
|
||||
process_due()
|
||||
assert len(api.posts) == 1
|
||||
assert row(app, note)["state"] == "synced"
|
||||
|
||||
def test_bad_token_fails_permanently(self, app, api):
|
||||
api.token = "other"
|
||||
note = publish(app)
|
||||
r = row(app, note)
|
||||
assert r["state"] == "failed" and "Invalid OAuth" in r["last_error"]
|
||||
assert "tok1" not in r["last_error"]
|
||||
|
||||
def test_rate_limit_retries(self, app, api):
|
||||
publish(app, content="warm up")
|
||||
api.fail_next = (400, 4) # Graph "application request limit reached"
|
||||
note = publish(app, content="later")
|
||||
assert row(app, note)["state"] == "pending"
|
||||
|
||||
|
||||
class TestMedia:
|
||||
def _note_with(self, app, images):
|
||||
from starpunk.media import attach_media_to_note, save_media
|
||||
with app.test_request_context():
|
||||
note = create_note(content="pics", published=True)
|
||||
ids = [save_media(data, name)["id"] for data, name in images]
|
||||
attach_media_to_note(note.id, ids, [f"alt {i}" for i in range(len(ids))])
|
||||
select_targets(note.id, ["threads"])
|
||||
process_due()
|
||||
return note
|
||||
|
||||
def test_single_image(self, app, api):
|
||||
self._note_with(app, [(image_bytes(), "a.jpg")])
|
||||
post = only_post(api)
|
||||
assert post["media_type"] == "IMAGE"
|
||||
assert post["image_url"].startswith("https://blog.example.com/media/")
|
||||
assert post["alt_text"] == "alt 0"
|
||||
assert "link_attachment" not in post
|
||||
|
||||
def test_carousel(self, app, api):
|
||||
self._note_with(app, [(image_bytes(), "a.jpg"), (image_bytes("PNG"), "b.png")])
|
||||
post = only_post(api)
|
||||
assert post["media_type"] == "CAROUSEL"
|
||||
children = [api.containers[c] for c in post["children"].split(",")]
|
||||
assert [c["is_carousel_item"] for c in children] == ["true", "true"]
|
||||
assert [c["alt_text"] for c in children] == ["alt 0", "alt 1"]
|
||||
|
||||
def test_unsupported_format_is_left_off(self, app, api):
|
||||
self._note_with(app, [(image_bytes("WEBP"), "a.webp")])
|
||||
assert only_post(api)["media_type"] == "TEXT"
|
||||
|
||||
|
||||
class TestToken:
|
||||
def test_refreshes_daily_and_saves(self, app, api):
|
||||
client = syndication._client_cache["threads"]
|
||||
with app.app_context():
|
||||
client.maintain()
|
||||
assert api.refreshes == 1
|
||||
client.maintain() # same day: nothing
|
||||
assert api.refreshes == 1
|
||||
saved = json.loads((app.config["DATA_PATH"] / "threads_token.json").read_text())
|
||||
assert saved["token"] == "tok2"
|
||||
# A restart picks up the saved token
|
||||
assert make_client(app, api)._token == "tok2"
|
||||
publish(app)
|
||||
assert len(api.posts) == 1
|
||||
|
||||
def test_other_worker_refresh_is_reused(self, app, api):
|
||||
first, second = make_client(app, api), make_client(app, api)
|
||||
with app.app_context():
|
||||
first.maintain()
|
||||
second.maintain()
|
||||
assert api.refreshes == 1
|
||||
assert second._token == "tok2"
|
||||
|
||||
def test_refresh_due_after_a_day(self, app, api, monkeypatch):
|
||||
client = syndication._client_cache["threads"]
|
||||
with app.app_context():
|
||||
client.maintain()
|
||||
later = time.time() + THREADS_REFRESH_EVERY + 1
|
||||
monkeypatch.setattr(syndication.time, "time", lambda: later)
|
||||
client.maintain()
|
||||
assert api.refreshes == 2
|
||||
|
||||
def test_failed_refresh_keeps_token(self, app, api):
|
||||
api.refresh_fails = True
|
||||
client = syndication._client_cache["threads"]
|
||||
with app.app_context():
|
||||
client.maintain()
|
||||
assert client._token == "tok1"
|
||||
publish(app)
|
||||
assert len(api.posts) == 1
|
||||
|
||||
def test_new_config_token_replaces_saved_one(self, app, api):
|
||||
with app.app_context():
|
||||
syndication._client_cache["threads"].maintain()
|
||||
api.token = "fresh"
|
||||
assert make_client(app, api, token="fresh")._token == "fresh"
|
||||
|
||||
def test_worker_runs_upkeep(self, app, api):
|
||||
with app.app_context():
|
||||
process_due()
|
||||
assert api.refreshes == 1
|
||||
|
||||
|
||||
def test_all_targets_listed_for_micropub(tmp_path):
|
||||
data = tmp_path / "d"
|
||||
app = create_app({
|
||||
"TESTING": True, "SITE_URL": "https://b.example", "ADMIN_ME": "https://e.example",
|
||||
"SESSION_SECRET": "x", "DATA_PATH": data, "NOTES_PATH": data / "notes",
|
||||
"DATABASE_PATH": data / "s.db",
|
||||
"GOTOSOCIAL_URL": "https://social.example", "GOTOSOCIAL_ACCESS_TOKEN": "t",
|
||||
"BLUESKY_HANDLE": "phil.example", "BLUESKY_APP_PASSWORD": "p",
|
||||
"THREADS_ACCESS_TOKEN": "x", "THREADS_USERNAME": "@phil",
|
||||
})
|
||||
with app.app_context():
|
||||
from starpunk.syndication import get_targets, resolve_targets
|
||||
assert set(get_targets()) == {"gotosocial", "bluesky", "threads"}
|
||||
assert resolve_targets(["https://www.threads.com/@phil"]) == ["threads"]
|
||||
syndication._client_cache.clear()
|
||||
|
||||
|
||||
class TestLikesReposts:
|
||||
PAGE = "https://other.example/post/1"
|
||||
|
||||
def _post(self, app, monkeypatch, content="", **kind):
|
||||
monkeypatch.setattr("starpunk.reply_context.fetch_reply_context", lambda *a, **k: None)
|
||||
with app.test_request_context():
|
||||
note = create_note(content=content, published=True, **kind)
|
||||
select_targets(note.id, ["threads"])
|
||||
process_due()
|
||||
return note
|
||||
|
||||
def test_like_is_not_crossposted(self, app, api, monkeypatch):
|
||||
note = self._post(app, monkeypatch, like_of=self.PAGE)
|
||||
assert api.posts == {}
|
||||
r = row(app, note)
|
||||
assert r["state"] == "synced" and r["remote_id"] is None
|
||||
|
||||
def test_repost_posts_link(self, app, api, monkeypatch):
|
||||
self._post(app, monkeypatch, repost_of=self.PAGE)
|
||||
post = only_post(api)
|
||||
assert post["text"].startswith("🔁") and self.PAGE in post["text"]
|
||||
assert post["link_attachment"] == self.PAGE
|
||||
|
||||
def test_repost_with_comment_keeps_comment(self, app, api, monkeypatch):
|
||||
note = self._post(app, monkeypatch, content="So **true**", repost_of=self.PAGE)
|
||||
assert only_post(api)["text"].startswith("So true")
|
||||
with app.test_request_context():
|
||||
delete_note(id=note.id)
|
||||
process_due()
|
||||
assert api.posts == {}
|
||||
Reference in new issue
Block a user