Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ed63e70c46 | ||
|
|
57b02ad6c7 | ||
|
|
94e8c672df | ||
|
|
011f7cc87d |
No files matched your search
@@ -50,6 +50,19 @@ SESSION_LIFETIME=30
|
||||
# Lifetime of Micropub access tokens StarPunk issues, in days (default: 90)
|
||||
INDIEAUTH_TOKEN_LIFETIME_DAYS=90
|
||||
|
||||
# =============================================================================
|
||||
# CROSSPOSTING (optional, v2.1.0)
|
||||
# =============================================================================
|
||||
|
||||
# GoToSocial or Mastodon account to copy new notes to. Create an app token
|
||||
# with scopes: write:statuses write:media
|
||||
GOTOSOCIAL_URL=
|
||||
GOTOSOCIAL_ACCESS_TOKEN=
|
||||
# Name shown in the editor and Micropub apps (default: GoToSocial)
|
||||
GOTOSOCIAL_NAME=GoToSocial
|
||||
# public, unlisted, private (default: public)
|
||||
GOTOSOCIAL_VISIBILITY=public
|
||||
|
||||
# =============================================================================
|
||||
# DATA STORAGE
|
||||
# =============================================================================
|
||||
|
||||
@@ -23,8 +23,11 @@ jobs:
|
||||
run: apk add --no-cache nodejs git docker-cli docker-cli-buildx
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
# The automatic Actions token can't log in to the container registry, so
|
||||
# this uses a personal access token with only the write:package scope,
|
||||
# stored as the repository secret REGISTRY_TOKEN.
|
||||
- name: Log in to the Gitea registry
|
||||
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login "$REGISTRY" -u "${{ github.actor }}" --password-stdin
|
||||
run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login "$REGISTRY" -u "${{ github.repository_owner }}" --password-stdin
|
||||
- name: Use the BuildKit socket
|
||||
run: docker buildx create --name ci --driver remote "$BUILDKIT" --use
|
||||
- name: Build and push
|
||||
|
||||
@@ -7,6 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.1.0] - 2026-10-31
|
||||
|
||||
### Added
|
||||
|
||||
- **Crossposting to GoToSocial** (or any Mastodon-API server), ADR-065
|
||||
- Full note as Markdown with images (alt text from captions), permalink and hashtags
|
||||
- Edits, unpublishing and deletes follow the note
|
||||
- On by default for new notes; untick in the editor, or send `mp-syndicate-to` from Micropub apps
|
||||
- Micropub `q=syndicate-to` and `q=config` list the target
|
||||
- Editor shows crosspost status, errors and a retry button; note pages link the copy with `u-syndication`
|
||||
- Configure with `GOTOSOCIAL_URL` and `GOTOSOCIAL_ACCESS_TOKEN` (scopes `write:statuses write:media`)
|
||||
- Migration 012: `syndications`
|
||||
|
||||
### Fixed
|
||||
|
||||
- Micropub `Location` header pointed at `/notes/<slug>` (404); now the real permalink
|
||||
- "Delete Note" on the edit page submitted the edit form (nested forms)
|
||||
- Search results linked to `/notes/<slug>` (404); now `/note/<slug>`
|
||||
|
||||
## [2.0.2] - 2026-10-31
|
||||
|
||||
### Fixed
|
||||
|
||||
- Container builds in Gitea Actions log in to the registry with the
|
||||
`REGISTRY_TOKEN` secret (a `write:package` token); the automatic Actions
|
||||
token is refused by the registry. No application changes.
|
||||
|
||||
## [2.0.1] - 2026-10-31
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# ADR-065: Direct Syndication (POSSE) to GoToSocial
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.1.0, 2026-10-31)
|
||||
|
||||
## Context
|
||||
|
||||
StarPunk posts were copied to other networks by Echofeed reading the feed.
|
||||
That gives no control per post, no record of where a post went, and edits
|
||||
and deletes never follow. The operator runs GoToSocial locally and wants
|
||||
StarPunk to post there directly, with Bluesky likely next.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Publish here, copy out (POSSE).** Each (note, target) has a row in
|
||||
`syndications` (migration 012). Creating, editing, unpublishing or
|
||||
deleting a note marks its rows pending; a background worker reconciles
|
||||
the remote post: create, update (`PUT /api/v1/statuses/:id`) or delete.
|
||||
2. **Defaults.** New notes go to every configured target unless unticked in
|
||||
the editor or a Micropub client sends `mp-syndicate-to` (an explicit
|
||||
list, possibly empty). Existing notes are never backfilled.
|
||||
3. **Content.** The full note as Markdown (`content_type: text/markdown`),
|
||||
then the permalink and the note's tags as hashtags. Longer than the
|
||||
target's limit: shortened at a word boundary with the link kept.
|
||||
Images are uploaded first (the `large` variant) with their captions as
|
||||
alt text.
|
||||
4. **Delivery.** Rows are claimed atomically so the four gunicorn workers'
|
||||
threads never send twice; an `Idempotency-Key` guards retries; a row
|
||||
`version` keeps edits made during a send. Network errors and 5xx/429
|
||||
retry with exponential backoff (8 attempts); other 4xx fail at once.
|
||||
A remote post deleted by hand is posted again on the next edit.
|
||||
5. **Backlinks.** Synced URLs are shown on the note page as `u-syndication`
|
||||
links, and in the editor with status, last error and a retry button.
|
||||
6. **Targets** are config-driven. One kind exists, a Mastodon-API client
|
||||
(`GOTOSOCIAL_URL`, `GOTOSOCIAL_ACCESS_TOKEN`); it also works for
|
||||
Mastodon. A Bluesky target would add a second client behind the same
|
||||
reconcile loop.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Echofeed must stop posting to GoToSocial or posts appear twice.
|
||||
- The token needs only `write:statuses write:media`.
|
||||
- `syndications.note_id` has no foreign key so a hard-deleted note's
|
||||
remote post can still be deleted.
|
||||
@@ -0,0 +1,29 @@
|
||||
-- Migration 012: Crossposting (POSSE) to syndication targets
|
||||
-- Version: 2.1.0
|
||||
-- Per ADR-065: Direct syndication to GoToSocial / Mastodon-API servers
|
||||
--
|
||||
-- One row per (note, target). The worker reconciles the remote post with the
|
||||
-- note: create, update, or delete as needed. note_id has no foreign key on
|
||||
-- purpose: a row must outlive a hard-deleted note so the remote post can be
|
||||
-- deleted too.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS syndications (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
note_id INTEGER NOT NULL,
|
||||
target TEXT NOT NULL,
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
remote_id TEXT,
|
||||
remote_url TEXT,
|
||||
state TEXT NOT NULL DEFAULT 'pending'
|
||||
CHECK (state IN ('pending', 'synced', 'failed')),
|
||||
version INTEGER NOT NULL DEFAULT 1,
|
||||
attempts INTEGER NOT NULL DEFAULT 0,
|
||||
last_error TEXT,
|
||||
next_attempt_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
claimed_until TIMESTAMP,
|
||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE (note_id, target)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_syndications_due ON syndications(state, next_attempt_at);
|
||||
+12
-2
@@ -336,6 +336,16 @@ def create_app(config=None):
|
||||
if hasattr(app, 'webmention_worker') and app.webmention_worker.is_alive():
|
||||
app.webmention_worker.stop()
|
||||
|
||||
# Start syndication worker when a target is configured (v2.1.0)
|
||||
if (app.config.get("GOTOSOCIAL_URL") and app.config.get("GOTOSOCIAL_ACCESS_TOKEN")
|
||||
and not app.config.get("TESTING", False)):
|
||||
from starpunk.syndication import SyndicationWorker
|
||||
app.syndication_worker = SyndicationWorker(
|
||||
app, poll_interval=app.config["SYNDICATION_POLL_INTERVAL"]
|
||||
)
|
||||
app.syndication_worker.start()
|
||||
app.logger.info("Syndication worker started")
|
||||
|
||||
# Health check endpoint for containers and monitoring
|
||||
@app.route("/health")
|
||||
def health_check():
|
||||
@@ -440,5 +450,5 @@ def create_app(config=None):
|
||||
|
||||
# Package version (Semantic Versioning 2.0.0)
|
||||
# See docs/standards/versioning-strategy.md for details
|
||||
__version__ = "2.0.1"
|
||||
__version_info__ = (2, 0, 1)
|
||||
__version__ = "2.1.0"
|
||||
__version_info__ = (2, 1, 0)
|
||||
@@ -124,6 +124,16 @@ def load_config(app, config_override=None):
|
||||
app.config["WEBMENTION_RATE_LIMIT_WINDOW"] = int(os.getenv("WEBMENTION_RATE_LIMIT_WINDOW", "60"))
|
||||
app.config["TRUSTED_PROXY"] = os.getenv("TRUSTED_PROXY", "false").lower() == "true"
|
||||
|
||||
# Crossposting to GoToSocial / Mastodon-API (v2.1.0, ADR-065)
|
||||
# Enabled when both URL and token are set
|
||||
app.config["GOTOSOCIAL_URL"] = os.getenv("GOTOSOCIAL_URL", "")
|
||||
app.config["GOTOSOCIAL_ACCESS_TOKEN"] = os.getenv("GOTOSOCIAL_ACCESS_TOKEN", "")
|
||||
app.config["GOTOSOCIAL_NAME"] = os.getenv("GOTOSOCIAL_NAME", "GoToSocial")
|
||||
app.config["GOTOSOCIAL_VISIBILITY"] = os.getenv("GOTOSOCIAL_VISIBILITY", "public")
|
||||
app.config["GOTOSOCIAL_MAX_CHARS"] = int(os.getenv("GOTOSOCIAL_MAX_CHARS", "5000"))
|
||||
app.config["GOTOSOCIAL_MAX_MEDIA"] = int(os.getenv("GOTOSOCIAL_MAX_MEDIA", "4"))
|
||||
app.config["SYNDICATION_POLL_INTERVAL"] = int(os.getenv("SYNDICATION_POLL_INTERVAL", "10"))
|
||||
|
||||
# Apply overrides if provided
|
||||
if config_override:
|
||||
app.config.update(config_override)
|
||||
|
||||
+31
-9
@@ -30,6 +30,7 @@ from flask import Request, current_app, jsonify
|
||||
|
||||
from starpunk.notes import create_note, get_note, InvalidNoteDataError, NoteNotFoundError
|
||||
from starpunk.indieauth import check_scope
|
||||
from starpunk.utils import absolute_url_for
|
||||
|
||||
|
||||
# Custom Exceptions
|
||||
@@ -398,6 +399,17 @@ def handle_create(data: dict, token_info: dict):
|
||||
elif isinstance(slug_value, str):
|
||||
custom_slug = slug_value
|
||||
|
||||
# mp-syndicate-to is also a server command, not a property (v2.1.0).
|
||||
# Absent means "all targets"; present means exactly those.
|
||||
requested_targets = None
|
||||
if isinstance(data, dict):
|
||||
for key in ("mp-syndicate-to", "mp-syndicate-to[]"):
|
||||
if key in data:
|
||||
value = data[key]
|
||||
requested_targets = (requested_targets or []) + (
|
||||
value if isinstance(value, list) else [value]
|
||||
)
|
||||
|
||||
# Normalize and extract properties
|
||||
try:
|
||||
properties = normalize_properties(data)
|
||||
@@ -427,10 +439,14 @@ def handle_create(data: dict, token_info: dict):
|
||||
if photos:
|
||||
_attach_photos_to_note(note.id, photos)
|
||||
|
||||
# Build permalink URL
|
||||
# Note: SITE_URL is normalized to include trailing slash (for IndieAuth spec compliance)
|
||||
site_url = current_app.config.get("SITE_URL", "http://localhost:5000")
|
||||
permalink = f"{site_url}notes/{note.slug}"
|
||||
# Crosspost after photos are attached (v2.1.0)
|
||||
from starpunk.syndication import get_targets, resolve_targets, select_targets
|
||||
if requested_targets is None:
|
||||
select_targets(note.id, list(get_targets()))
|
||||
else:
|
||||
select_targets(note.id, resolve_targets(requested_targets))
|
||||
|
||||
permalink = absolute_url_for("public.note", slug=note.slug)
|
||||
|
||||
# Return 201 Created with Location header
|
||||
return "", 201, {"Location": permalink}
|
||||
@@ -444,6 +460,13 @@ def handle_create(data: dict, token_info: dict):
|
||||
)
|
||||
|
||||
|
||||
def _syndicate_to() -> list:
|
||||
"""Configured crosspost targets in Micropub syndicate-to form"""
|
||||
from starpunk.syndication import get_targets
|
||||
|
||||
return [{"uid": t.uid, "name": t.name} for t in get_targets().values()]
|
||||
|
||||
|
||||
def handle_query(args: dict, token_info: dict):
|
||||
"""
|
||||
Handle Micropub query endpoints
|
||||
@@ -451,7 +474,7 @@ def handle_query(args: dict, token_info: dict):
|
||||
Supports:
|
||||
- q=config: Return server configuration
|
||||
- q=source: Return post source in Microformats2 JSON
|
||||
- q=syndicate-to: Return syndication targets (empty for V1)
|
||||
- q=syndicate-to: Return syndication targets
|
||||
|
||||
Args:
|
||||
args: Query string arguments
|
||||
@@ -467,7 +490,7 @@ def handle_query(args: dict, token_info: dict):
|
||||
site_url = current_app.config.get("SITE_URL", "http://localhost:5000").rstrip('/')
|
||||
config = {
|
||||
"media-endpoint": f"{site_url}/micropub/media",
|
||||
"syndicate-to": [], # No syndication targets in V1
|
||||
"syndicate-to": _syndicate_to(),
|
||||
"post-types": [
|
||||
{"type": "note", "name": "Note", "properties": ["content"]},
|
||||
{"type": "photo", "name": "Photo", "properties": ["photo"]}
|
||||
@@ -483,7 +506,7 @@ def handle_query(args: dict, token_info: dict):
|
||||
|
||||
# Extract slug from URL
|
||||
try:
|
||||
# URL format: https://example.com/notes/{slug}
|
||||
# URL format: https://example.com/note/{slug}
|
||||
slug = url.rstrip("/").split("/")[-1]
|
||||
note = get_note(slug)
|
||||
|
||||
@@ -520,8 +543,7 @@ def handle_query(args: dict, token_info: dict):
|
||||
return jsonify(mf2), 200
|
||||
|
||||
elif q == "syndicate-to":
|
||||
# Return syndication targets (none for V1)
|
||||
return jsonify({"syndicate-to": []}), 200
|
||||
return jsonify({"syndicate-to": _syndicate_to()}), 200
|
||||
|
||||
else:
|
||||
return error_response("invalid_request", f"Unknown query: {q}")
|
||||
@@ -851,6 +851,13 @@ def update_note(
|
||||
# Reply context fetch failure should not prevent note update
|
||||
current_app.logger.warning(f"Failed to fetch reply context for {in_reply_to}: {e}")
|
||||
|
||||
# Crossposts follow the note (v2.1.0, best-effort)
|
||||
try:
|
||||
from starpunk.syndication import note_changed
|
||||
note_changed(updated_note.id)
|
||||
except Exception as e:
|
||||
current_app.logger.warning(f"Failed to queue syndication update: {e}")
|
||||
|
||||
# 10. RETURN UPDATED NOTE
|
||||
return updated_note
|
||||
|
||||
@@ -1018,5 +1025,12 @@ def delete_note(
|
||||
)
|
||||
# Don't fail - database record already deleted
|
||||
|
||||
# Delete crossposts too (v2.1.0, best-effort)
|
||||
try:
|
||||
from starpunk.syndication import note_changed
|
||||
note_changed(existing_note.id)
|
||||
except Exception as e:
|
||||
current_app.logger.warning(f"Failed to queue syndication delete: {e}")
|
||||
|
||||
# 6. RETURN (no value)
|
||||
return None
|
||||
@@ -97,7 +97,9 @@ def new_note_form():
|
||||
Decorator: @require_auth
|
||||
Template: templates/admin/new.html
|
||||
"""
|
||||
return render_template("admin/new.html")
|
||||
from starpunk.syndication import get_targets
|
||||
|
||||
return render_template("admin/new.html", syndication_targets=get_targets().values())
|
||||
|
||||
|
||||
@bp.route("/new", methods=["POST"])
|
||||
@@ -182,6 +184,11 @@ def create_note_submit():
|
||||
if errors:
|
||||
flash(f"Note created, but some images failed: {'; '.join(errors)}", "warning")
|
||||
|
||||
# Crosspost after media is attached so images go out too (v2.1.0)
|
||||
if "syndication_form" in request.form:
|
||||
from starpunk.syndication import select_targets
|
||||
select_targets(note.id, request.form.getlist("syndicate_to"))
|
||||
|
||||
flash(f"Note created: {note.slug}", "success")
|
||||
return redirect(url_for("admin.dashboard"))
|
||||
|
||||
@@ -223,7 +230,17 @@ def edit_note_form(note_id: int):
|
||||
tags = get_note_tags(note.id)
|
||||
object.__setattr__(note, '_cached_tags', tags)
|
||||
|
||||
return render_template("admin/edit.html", note=note)
|
||||
from starpunk.syndication import get_note_syndications, get_targets
|
||||
|
||||
syndications = get_note_syndications(note.id)
|
||||
enabled = {s["target"] for s in syndications if s["enabled"]}
|
||||
return render_template(
|
||||
"admin/edit.html",
|
||||
note=note,
|
||||
syndication_targets=get_targets().values(),
|
||||
syndications=syndications,
|
||||
syndication_enabled=enabled,
|
||||
)
|
||||
|
||||
|
||||
@bp.route("/edit/<int:note_id>", methods=["POST"])
|
||||
@@ -274,6 +291,9 @@ def update_note_submit(note_id: int):
|
||||
published=published,
|
||||
tags=tags if tags else None
|
||||
)
|
||||
if "syndication_form" in request.form:
|
||||
from starpunk.syndication import select_targets
|
||||
select_targets(note.id, request.form.getlist("syndicate_to"))
|
||||
flash(f"Note updated: {note.slug}", "success")
|
||||
return redirect(url_for("admin.dashboard"))
|
||||
except ValueError as e:
|
||||
@@ -838,3 +858,14 @@ def revoke_token(token_id):
|
||||
revoke_token_by_id(token_id)
|
||||
flash("App access revoked", "success")
|
||||
return redirect(url_for("admin.tokens"))
|
||||
|
||||
|
||||
@bp.route("/syndication/<int:syndication_id>/retry", methods=["POST"])
|
||||
@require_auth
|
||||
def retry_syndication(syndication_id):
|
||||
"""Retry a crosspost now"""
|
||||
from starpunk.syndication import retry
|
||||
|
||||
retry(syndication_id)
|
||||
flash("Crosspost queued", "success")
|
||||
return redirect(request.referrer or url_for("admin.dashboard"))
|
||||
@@ -331,9 +331,14 @@ def note(slug: str):
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.error(f"Error loading webmentions for note {slug}: {e}")
|
||||
|
||||
from starpunk.syndication import get_note_syndications
|
||||
|
||||
syndications = [s for s in get_note_syndications(note_obj.id) if s["state"] == "synced"]
|
||||
|
||||
return render_template(
|
||||
"note.html",
|
||||
note=note_obj,
|
||||
syndications=syndications,
|
||||
webmentions=webmentions,
|
||||
reply_context=reply_context,
|
||||
**grouped if webmentions else {}
|
||||
|
||||
@@ -108,7 +108,7 @@ def api_search():
|
||||
"title": r["title"] or f"Note from {r['created_at'][:10]}",
|
||||
"excerpt": r["snippet"], # Already has <mark> tags
|
||||
"published_at": r["created_at"],
|
||||
"url": f"/notes/{r['slug']}",
|
||||
"url": f"/note/{r['slug']}",
|
||||
}
|
||||
for r in results
|
||||
],
|
||||
@@ -176,7 +176,7 @@ def search_page():
|
||||
"title": r["title"] or f"Note from {r['created_at'][:10]}",
|
||||
"excerpt": Markup(safe_snippet), # Mark as safe since we've escaped it ourselves
|
||||
"published_at": r["created_at"],
|
||||
"url": f"/notes/{r['slug']}",
|
||||
"url": f"/note/{r['slug']}",
|
||||
})
|
||||
results = formatted_results
|
||||
except Exception as e:
|
||||
|
||||
@@ -0,0 +1,418 @@
|
||||
"""
|
||||
Crossposting (POSSE) for StarPunk (v2.1.0, ADR-065)
|
||||
|
||||
Published notes are copied to syndication targets, and later edits and
|
||||
deletes follow them. Currently one target kind exists: a GoToSocial (or any
|
||||
Mastodon-API) account.
|
||||
|
||||
Each (note, target) pair has a row in `syndications`. Changing a note marks
|
||||
its rows pending; a background worker then reconciles the remote post with
|
||||
the note:
|
||||
|
||||
note published + target enabled, no remote post -> create
|
||||
note published + target enabled, remote post -> update
|
||||
otherwise, remote post exists -> delete
|
||||
|
||||
Rows carry a version so an edit made while a send is in flight is not lost.
|
||||
"""
|
||||
|
||||
import re
|
||||
import threading
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
import httpx
|
||||
from flask import current_app
|
||||
|
||||
from starpunk.database import get_db
|
||||
|
||||
MAX_ATTEMPTS = 8
|
||||
CLAIM_MINUTES = 5
|
||||
|
||||
|
||||
class SyndicationError(Exception):
|
||||
"""A send failed. permanent=True means retrying won't help."""
|
||||
|
||||
def __init__(self, message: str, permanent: bool = False):
|
||||
super().__init__(message)
|
||||
self.permanent = permanent
|
||||
|
||||
|
||||
class RemoteGone(SyndicationError):
|
||||
"""The remote post no longer exists"""
|
||||
|
||||
|
||||
class MastodonClient:
|
||||
"""Minimal Mastodon-API client (works with GoToSocial)"""
|
||||
|
||||
def __init__(self, base_url: str, token: str, visibility: str = "public",
|
||||
http: Optional[httpx.Client] = None):
|
||||
self.base_url = base_url.rstrip("/")
|
||||
self.visibility = visibility
|
||||
self.http = http or httpx.Client(timeout=30.0)
|
||||
self.headers = {"Authorization": f"Bearer {token}"}
|
||||
|
||||
def _call(self, method: str, path: str, **kwargs) -> httpx.Response:
|
||||
headers = {**self.headers, **kwargs.pop("headers", {})}
|
||||
try:
|
||||
response = self.http.request(method, self.base_url + path, headers=headers, **kwargs)
|
||||
except httpx.HTTPError as e:
|
||||
raise SyndicationError(f"{method} {path}: {e}")
|
||||
if response.status_code == 404:
|
||||
raise RemoteGone(f"{method} {path}: not found", permanent=True)
|
||||
if response.status_code >= 400:
|
||||
# 4xx other than rate limiting won't fix itself
|
||||
permanent = 400 <= response.status_code < 500 and response.status_code != 429
|
||||
raise SyndicationError(
|
||||
f"{method} {path}: HTTP {response.status_code} {response.text[:200]}",
|
||||
permanent=permanent,
|
||||
)
|
||||
return response
|
||||
|
||||
def upload_media(self, path: Path, mime_type: str, description: str) -> str:
|
||||
with open(path, "rb") as f:
|
||||
response = self._call(
|
||||
"POST", "/api/v2/media",
|
||||
files={"file": (path.name, f, mime_type)},
|
||||
data={"description": description or ""},
|
||||
)
|
||||
return response.json()["id"]
|
||||
|
||||
def _status_body(self, text: str, media_ids: List[str]) -> dict:
|
||||
return {
|
||||
"status": text,
|
||||
"media_ids": media_ids,
|
||||
"visibility": self.visibility,
|
||||
"content_type": "text/markdown",
|
||||
}
|
||||
|
||||
def create_status(self, text: str, media_ids: List[str], idempotency_key: str) -> dict:
|
||||
response = self._call(
|
||||
"POST", "/api/v1/statuses",
|
||||
json=self._status_body(text, media_ids),
|
||||
headers={"Idempotency-Key": idempotency_key},
|
||||
)
|
||||
data = response.json()
|
||||
return {"id": data["id"], "url": data.get("url") or data.get("uri")}
|
||||
|
||||
def update_status(self, remote_id: str, text: str, media_ids: List[str]) -> dict:
|
||||
response = self._call("PUT", f"/api/v1/statuses/{remote_id}",
|
||||
json=self._status_body(text, media_ids))
|
||||
data = response.json()
|
||||
return {"id": data["id"], "url": data.get("url") or data.get("uri")}
|
||||
|
||||
def delete_status(self, remote_id: str) -> None:
|
||||
try:
|
||||
self._call("DELETE", f"/api/v1/statuses/{remote_id}")
|
||||
except RemoteGone:
|
||||
pass # already gone
|
||||
|
||||
|
||||
@dataclass
|
||||
class Target:
|
||||
id: str # stored in syndications.target
|
||||
uid: str # Micropub syndicate-to uid
|
||||
name: str # shown in the editor and Micropub clients
|
||||
client: MastodonClient
|
||||
max_chars: int = 5000
|
||||
max_media: int = 4
|
||||
|
||||
|
||||
_client_cache: Dict[str, MastodonClient] = {}
|
||||
|
||||
|
||||
def get_targets() -> Dict[str, Target]:
|
||||
"""Configured syndication targets, by id"""
|
||||
config = current_app.config
|
||||
targets: Dict[str, Target] = {}
|
||||
|
||||
url = (config.get("GOTOSOCIAL_URL") or "").rstrip("/")
|
||||
token = config.get("GOTOSOCIAL_ACCESS_TOKEN") or ""
|
||||
if url and token:
|
||||
client = _client_cache.get(url)
|
||||
if client is None:
|
||||
client = MastodonClient(url, token, config.get("GOTOSOCIAL_VISIBILITY", "public"))
|
||||
_client_cache[url] = client
|
||||
targets["gotosocial"] = Target(
|
||||
id="gotosocial",
|
||||
uid=url + "/",
|
||||
name=config.get("GOTOSOCIAL_NAME", "GoToSocial"),
|
||||
client=client,
|
||||
max_chars=int(config.get("GOTOSOCIAL_MAX_CHARS", 5000)),
|
||||
max_media=int(config.get("GOTOSOCIAL_MAX_MEDIA", 4)),
|
||||
)
|
||||
return targets
|
||||
|
||||
|
||||
def resolve_targets(requested: List[str]) -> List[str]:
|
||||
"""Map Micropub uids or target ids to configured target ids"""
|
||||
targets = get_targets()
|
||||
by_uid = {t.uid.rstrip("/"): t.id for t in targets.values()}
|
||||
chosen = []
|
||||
for value in requested:
|
||||
value = (value or "").strip()
|
||||
target_id = value if value in targets else by_uid.get(value.rstrip("/"))
|
||||
if target_id and target_id not in chosen:
|
||||
chosen.append(target_id)
|
||||
return chosen
|
||||
|
||||
|
||||
# --- Queue -----------------------------------------------------------------
|
||||
|
||||
def select_targets(note_id: int, target_ids: List[str]) -> None:
|
||||
"""
|
||||
Set which targets a note goes to, and queue the reconcile
|
||||
|
||||
Targets not listed are disabled (their remote post, if any, is deleted).
|
||||
Nothing is recorded for unlisted targets the note never went to.
|
||||
"""
|
||||
db = get_db(current_app)
|
||||
for target_id in get_targets():
|
||||
enabled = 1 if target_id in target_ids else 0
|
||||
existing = db.execute(
|
||||
"SELECT id FROM syndications WHERE note_id = ? AND target = ?",
|
||||
(note_id, target_id),
|
||||
).fetchone()
|
||||
if existing:
|
||||
db.execute(
|
||||
"""
|
||||
UPDATE syndications
|
||||
SET enabled = ?, state = 'pending', version = version + 1, attempts = 0,
|
||||
last_error = NULL, next_attempt_at = datetime('now'), updated_at = datetime('now')
|
||||
WHERE id = ?
|
||||
""",
|
||||
(enabled, existing["id"]),
|
||||
)
|
||||
elif enabled:
|
||||
db.execute(
|
||||
"INSERT INTO syndications (note_id, target, enabled) VALUES (?, ?, 1)",
|
||||
(note_id, target_id),
|
||||
)
|
||||
db.commit()
|
||||
|
||||
|
||||
def note_changed(note_id: int) -> None:
|
||||
"""Queue a reconcile for every target the note was sent to"""
|
||||
db = get_db(current_app)
|
||||
db.execute(
|
||||
"""
|
||||
UPDATE syndications
|
||||
SET state = 'pending', version = version + 1, attempts = 0, last_error = NULL,
|
||||
next_attempt_at = datetime('now'), updated_at = datetime('now')
|
||||
WHERE note_id = ?
|
||||
""",
|
||||
(note_id,),
|
||||
)
|
||||
db.commit()
|
||||
|
||||
|
||||
def retry(syndication_id: int) -> None:
|
||||
"""Retry a failed or pending row now"""
|
||||
db = get_db(current_app)
|
||||
db.execute(
|
||||
"""
|
||||
UPDATE syndications
|
||||
SET state = 'pending', attempts = 0, last_error = NULL, next_attempt_at = datetime('now')
|
||||
WHERE id = ?
|
||||
""",
|
||||
(syndication_id,),
|
||||
)
|
||||
db.commit()
|
||||
|
||||
|
||||
def get_note_syndications(note_id: int) -> List[dict]:
|
||||
"""Rows for a note, with the target's display name"""
|
||||
targets = get_targets()
|
||||
rows = get_db(current_app).execute(
|
||||
"SELECT * FROM syndications WHERE note_id = ? ORDER BY target", (note_id,)
|
||||
).fetchall()
|
||||
result = []
|
||||
for row in rows:
|
||||
item = dict(row)
|
||||
item["name"] = targets[row["target"]].name if row["target"] in targets else row["target"]
|
||||
result.append(item)
|
||||
return result
|
||||
|
||||
|
||||
# --- Composing --------------------------------------------------------------
|
||||
|
||||
def _hashtag(display_name: str) -> str:
|
||||
word = re.sub(r"[^\w]", "", display_name)
|
||||
return f"#{word}" if word else ""
|
||||
|
||||
|
||||
def compose_text(note, permalink: str, tags: List[str], max_chars: int) -> str:
|
||||
"""Full note as Markdown, then the permalink and hashtags; shortened if too long"""
|
||||
hashtags = " ".join(h for h in (_hashtag(t) for t in tags) if h)
|
||||
tail = f"\n\n{permalink}" + (f"\n\n{hashtags}" if hashtags else "")
|
||||
body = note.content.strip()
|
||||
if len(body) + len(tail) > max_chars:
|
||||
room = max_chars - len(tail) - 1
|
||||
body = body[:room].rsplit(" ", 1)[0].rstrip() + "…"
|
||||
return body + tail
|
||||
|
||||
|
||||
def _media_files(note_id: int, limit: int) -> List[dict]:
|
||||
"""Image files for a note: the 'large' variant when there is one"""
|
||||
from starpunk.media import get_note_media
|
||||
|
||||
media_dir = Path(current_app.config["DATA_PATH"]) / "media"
|
||||
files = []
|
||||
for media in get_note_media(note_id)[:limit]:
|
||||
variant = (media.get("variants") or {}).get("large")
|
||||
rel = variant["path"] if variant else media["path"]
|
||||
files.append({
|
||||
"path": media_dir / rel,
|
||||
"mime_type": media.get("mime_type") or "image/jpeg",
|
||||
"description": media.get("caption") or "",
|
||||
})
|
||||
return files
|
||||
|
||||
|
||||
# --- Worker -----------------------------------------------------------------
|
||||
|
||||
def _claim(db, row_id: int) -> bool:
|
||||
cursor = db.execute(
|
||||
f"""
|
||||
UPDATE syndications SET claimed_until = datetime('now', '+{CLAIM_MINUTES} minutes')
|
||||
WHERE id = ? AND state = 'pending'
|
||||
AND (claimed_until IS NULL OR claimed_until < datetime('now'))
|
||||
""",
|
||||
(row_id,),
|
||||
)
|
||||
db.commit()
|
||||
return cursor.rowcount == 1
|
||||
|
||||
|
||||
def _finish(db, row, remote_id: Optional[str], remote_url: Optional[str]) -> None:
|
||||
# Only mark synced if nothing changed while we were sending
|
||||
db.execute(
|
||||
"""
|
||||
UPDATE syndications
|
||||
SET remote_id = ?, remote_url = ?, claimed_until = NULL, attempts = 0, last_error = NULL,
|
||||
state = CASE WHEN version = ? THEN 'synced' ELSE 'pending' END,
|
||||
updated_at = datetime('now')
|
||||
WHERE id = ?
|
||||
""",
|
||||
(remote_id, remote_url, row["version"], row["id"]),
|
||||
)
|
||||
db.commit()
|
||||
|
||||
|
||||
def _fail(db, row, error: SyndicationError) -> None:
|
||||
attempts = row["attempts"] + 1
|
||||
failed = error.permanent or attempts >= MAX_ATTEMPTS
|
||||
delay = min(30 * (2 ** attempts), 6 * 3600)
|
||||
db.execute(
|
||||
f"""
|
||||
UPDATE syndications
|
||||
SET attempts = ?, last_error = ?, claimed_until = NULL,
|
||||
state = CASE WHEN version = ? THEN ? ELSE 'pending' END,
|
||||
next_attempt_at = datetime('now', '+{delay} seconds'), updated_at = datetime('now')
|
||||
WHERE id = ?
|
||||
""",
|
||||
(attempts, str(error)[:500], row["version"], "failed" if failed else "pending", row["id"]),
|
||||
)
|
||||
db.commit()
|
||||
current_app.logger.warning(
|
||||
f"Syndication {row['target']} for note {row['note_id']} failed "
|
||||
f"(attempt {attempts}{', giving up' if failed else ''}): {error}"
|
||||
)
|
||||
|
||||
|
||||
def _reconcile(row, target: Target) -> tuple:
|
||||
"""Bring the remote post in line with the note. Returns (remote_id, remote_url)."""
|
||||
from starpunk.notes import get_note
|
||||
from starpunk.tags import get_note_tags
|
||||
|
||||
note = get_note(id=row["note_id"], load_content=True)
|
||||
wanted = bool(row["enabled"]) and note is not None and note.published
|
||||
|
||||
if not wanted:
|
||||
if row["remote_id"]:
|
||||
target.client.delete_status(row["remote_id"])
|
||||
return None, None
|
||||
|
||||
site = current_app.config["SITE_URL"].rstrip("/")
|
||||
tags = [t["display_name"] for t in get_note_tags(note.id)]
|
||||
text = compose_text(note, f"{site}/note/{note.slug}", tags, target.max_chars)
|
||||
media_ids = [
|
||||
target.client.upload_media(f["path"], f["mime_type"], f["description"])
|
||||
for f in _media_files(note.id, target.max_media)
|
||||
]
|
||||
|
||||
if row["remote_id"]:
|
||||
try:
|
||||
result = target.client.update_status(row["remote_id"], text, media_ids)
|
||||
return result["id"], result["url"]
|
||||
except RemoteGone:
|
||||
pass # deleted on the remote side; post it again
|
||||
|
||||
result = target.client.create_status(
|
||||
text, media_ids, idempotency_key=f"starpunk-{row['id']}-{row['version']}"
|
||||
)
|
||||
return result["id"], result["url"]
|
||||
|
||||
|
||||
def process_due(limit: int = 5) -> int:
|
||||
"""Reconcile up to `limit` pending rows. Returns how many were handled."""
|
||||
targets = get_targets()
|
||||
if not targets:
|
||||
return 0
|
||||
db = get_db(current_app)
|
||||
rows = db.execute(
|
||||
"""
|
||||
SELECT * FROM syndications
|
||||
WHERE state = 'pending' AND next_attempt_at <= datetime('now')
|
||||
AND (claimed_until IS NULL OR claimed_until < datetime('now'))
|
||||
ORDER BY next_attempt_at LIMIT ?
|
||||
""",
|
||||
(limit,),
|
||||
).fetchall()
|
||||
|
||||
handled = 0
|
||||
for row in rows:
|
||||
if not _claim(db, row["id"]):
|
||||
continue # another worker took it
|
||||
handled += 1
|
||||
target = targets.get(row["target"])
|
||||
if target is None:
|
||||
_fail(db, row, SyndicationError(f"target {row['target']} is not configured", permanent=True))
|
||||
continue
|
||||
try:
|
||||
remote_id, remote_url = _reconcile(row, target)
|
||||
except SyndicationError as e:
|
||||
_fail(db, row, e)
|
||||
continue
|
||||
except Exception as e: # never let one bad note stop the worker
|
||||
_fail(db, row, SyndicationError(f"unexpected error: {e}"))
|
||||
continue
|
||||
_finish(db, row, remote_id, remote_url)
|
||||
if remote_id:
|
||||
current_app.logger.info(f"Syndicated note {row['note_id']} to {row['target']}: {remote_url}")
|
||||
return handled
|
||||
|
||||
|
||||
class SyndicationWorker(threading.Thread):
|
||||
"""Daemon thread that drains the syndication queue"""
|
||||
|
||||
def __init__(self, app, poll_interval: int = 10):
|
||||
super().__init__(daemon=True, name="syndication-worker")
|
||||
self.app = app
|
||||
self.poll_interval = poll_interval
|
||||
self._stop_event = threading.Event()
|
||||
|
||||
def run(self):
|
||||
while not self._stop_event.is_set():
|
||||
try:
|
||||
with self.app.app_context():
|
||||
handled = process_due()
|
||||
except Exception as e:
|
||||
self.app.logger.error(f"Syndication worker error: {e}", exc_info=True)
|
||||
handled = 0
|
||||
if handled == 0:
|
||||
self._stop_event.wait(self.poll_interval)
|
||||
|
||||
def stop(self):
|
||||
self._stop_event.set()
|
||||
@@ -52,15 +52,58 @@
|
||||
<label for="published">Published</label>
|
||||
</div>
|
||||
|
||||
{% if syndication_targets %}
|
||||
<input type="hidden" name="syndication_form" value="1">
|
||||
<fieldset class="form-group">
|
||||
<legend>Crosspost</legend>
|
||||
{% for target in syndication_targets %}
|
||||
<div class="form-checkbox">
|
||||
<input type="checkbox" id="syndicate-{{ target.id }}" name="syndicate_to" value="{{ target.id }}"{% if target.id in syndication_enabled %} checked{% endif %}>
|
||||
<label for="syndicate-{{ target.id }}">{{ target.name }}</label>
|
||||
</div>
|
||||
{% endfor %}
|
||||
</fieldset>
|
||||
{% endif %}
|
||||
|
||||
<div class="form-actions">
|
||||
<button type="submit" class="button button-primary">Update Note</button>
|
||||
<a href="{{ url_for('admin.dashboard') }}" class="button button-secondary">Cancel</a>
|
||||
<form action="{{ url_for('admin.delete_note_submit', note_id=note.id) }}" method="POST" class="delete-form" style="display: inline;" onsubmit="return confirm('Are you sure you want to delete this note? This cannot be undone.');">
|
||||
<input type="hidden" name="confirm" value="yes">
|
||||
<button type="submit" class="button button-danger">Delete Note</button>
|
||||
</form>
|
||||
<button type="submit" form="delete-note-form" class="button button-danger">Delete Note</button>
|
||||
</div>
|
||||
</form>
|
||||
|
||||
{# Outside the edit form: nested forms are not allowed in HTML #}
|
||||
<form id="delete-note-form" action="{{ url_for('admin.delete_note_submit', note_id=note.id) }}" method="POST" class="delete-form" onsubmit="return confirm('Are you sure you want to delete this note? This cannot be undone.');">
|
||||
<input type="hidden" name="confirm" value="yes">
|
||||
</form>
|
||||
|
||||
{% if syndications %}
|
||||
<section class="syndication-status">
|
||||
<h3>Crossposts</h3>
|
||||
<ul>
|
||||
{% for s in syndications %}
|
||||
<li>
|
||||
<strong>{{ s.name }}</strong>:
|
||||
{% if s.state == 'synced' and s.remote_url %}
|
||||
<a href="{{ s.remote_url }}" rel="noopener">posted</a>
|
||||
{% elif s.state == 'synced' %}
|
||||
removed
|
||||
{% elif s.state == 'pending' %}
|
||||
pending{% if s.attempts %} (retry {{ s.attempts }}){% endif %}
|
||||
{% else %}
|
||||
failed
|
||||
{% endif %}
|
||||
{% if s.last_error %}<br><small>{{ s.last_error }}</small>{% endif %}
|
||||
{% if s.state != 'synced' %}
|
||||
<form action="{{ url_for('admin.retry_syndication', syndication_id=s.id) }}" method="POST" style="display: inline;">
|
||||
<button type="submit" class="button button-secondary">Retry now</button>
|
||||
</form>
|
||||
{% endif %}
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
</section>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
|
||||
@@ -65,6 +65,19 @@
|
||||
<label for="published">Publish immediately</label>
|
||||
</div>
|
||||
|
||||
{% if syndication_targets %}
|
||||
<input type="hidden" name="syndication_form" value="1">
|
||||
<fieldset class="form-group">
|
||||
<legend>Crosspost</legend>
|
||||
{% for target in syndication_targets %}
|
||||
<div class="form-checkbox">
|
||||
<input type="checkbox" id="syndicate-{{ target.id }}" name="syndicate_to" value="{{ target.id }}" checked>
|
||||
<label for="syndicate-{{ target.id }}">{{ target.name }}</label>
|
||||
</div>
|
||||
{% endfor %}
|
||||
</fieldset>
|
||||
{% endif %}
|
||||
|
||||
<div class="form-actions">
|
||||
<button type="submit" class="button button-primary">Create Note</button>
|
||||
<a href="{{ url_for('admin.dashboard') }}" class="button button-secondary">Cancel</a>
|
||||
|
||||
@@ -49,6 +49,9 @@
|
||||
|
||||
<footer class="note-meta">
|
||||
{# u-url and u-uid same for notes (per Q23) #}
|
||||
{% for s in syndications if s.remote_url %}
|
||||
<a class="u-syndication" href="{{ s.remote_url }}" rel="syndication">Also on {{ s.name }}</a>
|
||||
{% endfor %}
|
||||
<a class="u-url u-uid" href="{{ absolute_url_for('public.note', slug=note.slug) }}">
|
||||
<time class="dt-published" datetime="{{ note.created_at.isoformat() }}">
|
||||
{{ note.created_at.strftime('%B %d, %Y at %I:%M %p') }}
|
||||
|
||||
@@ -206,7 +206,7 @@ def test_micropub_create_with_custom_slug_form(client, app, mock_valid_token):
|
||||
|
||||
# Verify the custom slug was used
|
||||
location = response.headers['Location']
|
||||
assert location.endswith('/notes/my-custom-slug')
|
||||
assert location.endswith('/note/my-custom-slug')
|
||||
|
||||
# Verify note exists with the custom slug
|
||||
with app.app_context():
|
||||
@@ -236,7 +236,7 @@ def test_micropub_create_with_custom_slug_json(client, app, mock_valid_token):
|
||||
|
||||
# Verify the custom slug was used
|
||||
location = response.headers['Location']
|
||||
assert location.endswith('/notes/json-custom-slug')
|
||||
assert location.endswith('/note/json-custom-slug')
|
||||
|
||||
# Verify note exists with the custom slug
|
||||
with app.app_context():
|
||||
|
||||
@@ -220,8 +220,8 @@ def test_search_api_generates_correct_urls(client, test_notes):
|
||||
|
||||
if data["count"] > 0:
|
||||
result = data["results"][0]
|
||||
assert result["url"].startswith("/notes/")
|
||||
assert result["url"] == f"/notes/{result['slug']}"
|
||||
assert result["url"].startswith("/note/")
|
||||
assert result["url"] == f"/note/{result['slug']}"
|
||||
|
||||
|
||||
def test_search_api_provides_fallback_title(client, app):
|
||||
|
||||
@@ -185,8 +185,8 @@ def test_search_page_links_to_notes(client, test_notes):
|
||||
response = client.get("/search?q=test")
|
||||
assert response.status_code == 200
|
||||
|
||||
# Should contain links to /notes/
|
||||
assert b'href="/notes/' in response.data
|
||||
# Should contain links to /note/
|
||||
assert b'href="/note/' in response.data
|
||||
|
||||
|
||||
def test_search_form_validation(client):
|
||||
|
||||
@@ -0,0 +1,349 @@
|
||||
"""
|
||||
Tests for crossposting to GoToSocial (starpunk/syndication.py, v2.1.0)
|
||||
|
||||
A fake Mastodon-API server (httpx.MockTransport) records what StarPunk sends.
|
||||
"""
|
||||
|
||||
import io
|
||||
import json
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from starpunk import create_app, syndication
|
||||
from starpunk.auth import create_session
|
||||
from starpunk.indieauth import create_access_token
|
||||
from starpunk.notes import create_note, delete_note, update_note
|
||||
from starpunk.syndication import (
|
||||
MastodonClient,
|
||||
compose_text,
|
||||
get_note_syndications,
|
||||
note_changed,
|
||||
process_due,
|
||||
select_targets,
|
||||
)
|
||||
|
||||
GTS = "https://social.example.com"
|
||||
|
||||
|
||||
class FakeServer:
|
||||
def __init__(self):
|
||||
self.statuses = {}
|
||||
self.calls = []
|
||||
self.media = 0
|
||||
self.fail_next = None # status code to return once
|
||||
|
||||
def __call__(self, request: httpx.Request) -> httpx.Response:
|
||||
self.calls.append((request.method, request.url.path))
|
||||
assert request.headers["Authorization"] == "Bearer secret-token"
|
||||
if self.fail_next:
|
||||
code, self.fail_next = self.fail_next, None
|
||||
return httpx.Response(code, json={"error": "nope"})
|
||||
path = request.url.path
|
||||
if path == "/api/v2/media":
|
||||
self.media += 1
|
||||
return httpx.Response(200, json={"id": f"m{self.media}"})
|
||||
if path == "/api/v1/statuses" and request.method == "POST":
|
||||
body = json.loads(request.content)
|
||||
sid = str(len(self.statuses) + 1)
|
||||
self.statuses[sid] = body
|
||||
return httpx.Response(200, json={"id": sid, "url": f"{GTS}/@phil/statuses/{sid}"})
|
||||
if path.startswith("/api/v1/statuses/"):
|
||||
sid = path.rsplit("/", 1)[1]
|
||||
if sid not in self.statuses:
|
||||
return httpx.Response(404, json={"error": "not found"})
|
||||
if request.method == "PUT":
|
||||
self.statuses[sid] = json.loads(request.content)
|
||||
return httpx.Response(200, json={"id": sid, "url": f"{GTS}/@phil/statuses/{sid}"})
|
||||
if request.method == "DELETE":
|
||||
del self.statuses[sid]
|
||||
return httpx.Response(200, json={"id": sid})
|
||||
return httpx.Response(500)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def server():
|
||||
return FakeServer()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(tmp_path, server):
|
||||
data = tmp_path / "data"
|
||||
app = create_app(
|
||||
{
|
||||
"TESTING": True,
|
||||
"SITE_URL": "https://blog.example.com",
|
||||
"ADMIN_ME": "https://example.com",
|
||||
"SESSION_SECRET": "test-secret",
|
||||
"DATA_PATH": data,
|
||||
"NOTES_PATH": data / "notes",
|
||||
"DATABASE_PATH": data / "starpunk.db",
|
||||
"GOTOSOCIAL_URL": GTS,
|
||||
"GOTOSOCIAL_ACCESS_TOKEN": "secret-token",
|
||||
}
|
||||
)
|
||||
syndication._client_cache[GTS] = MastodonClient(
|
||||
GTS, "secret-token", http=httpx.Client(transport=httpx.MockTransport(server))
|
||||
)
|
||||
yield app
|
||||
syndication._client_cache.clear()
|
||||
|
||||
|
||||
def rows(app, note_id):
|
||||
with app.app_context():
|
||||
return get_note_syndications(note_id)
|
||||
|
||||
|
||||
def publish(app, content="Hello fediverse", tags=None, targets=("gotosocial",)):
|
||||
with app.test_request_context():
|
||||
note = create_note(content=content, published=True, tags=tags)
|
||||
select_targets(note.id, list(targets))
|
||||
process_due()
|
||||
return note
|
||||
|
||||
|
||||
class TestCompose:
|
||||
def test_full_note_with_link_and_hashtags(self, app):
|
||||
with app.app_context():
|
||||
note = create_note(content="# Title\n\nSome *markdown*", published=True)
|
||||
text = compose_text(note, "https://blog.example.com/note/x", ["IndieWeb", "Self Hosting"], 5000)
|
||||
assert text == "# Title\n\nSome *markdown*\n\nhttps://blog.example.com/note/x\n\n#IndieWeb #SelfHosting"
|
||||
|
||||
def test_long_note_is_shortened_but_keeps_link(self, app):
|
||||
with app.app_context():
|
||||
note = create_note(content="word " * 2000, published=True)
|
||||
text = compose_text(note, "https://blog.example.com/note/x", [], 500)
|
||||
assert len(text) <= 500
|
||||
assert text.endswith("https://blog.example.com/note/x")
|
||||
assert "…" in text
|
||||
|
||||
|
||||
class TestLifecycle:
|
||||
def test_publish_creates_status(self, app, server):
|
||||
note = publish(app, tags=["Test"])
|
||||
(row,) = rows(app, note.id)
|
||||
assert row["state"] == "synced"
|
||||
assert row["remote_url"] == f"{GTS}/@phil/statuses/1"
|
||||
body = server.statuses["1"]
|
||||
assert body["content_type"] == "text/markdown"
|
||||
assert body["visibility"] == "public"
|
||||
assert f"https://blog.example.com/note/{note.slug}" in body["status"]
|
||||
assert "#Test" in body["status"]
|
||||
|
||||
def test_edit_updates_status(self, app, server):
|
||||
note = publish(app)
|
||||
with app.test_request_context():
|
||||
update_note(id=note.id, content="Edited text")
|
||||
process_due()
|
||||
assert server.statuses["1"]["status"].startswith("Edited text")
|
||||
assert ("PUT", "/api/v1/statuses/1") in server.calls
|
||||
|
||||
def test_delete_deletes_status(self, app, server):
|
||||
note = publish(app)
|
||||
with app.test_request_context():
|
||||
delete_note(id=note.id)
|
||||
process_due()
|
||||
assert server.statuses == {}
|
||||
(row,) = rows(app, note.id)
|
||||
assert row["remote_id"] is None and row["state"] == "synced"
|
||||
|
||||
def test_hard_delete_deletes_status(self, app, server):
|
||||
note = publish(app)
|
||||
with app.test_request_context():
|
||||
delete_note(id=note.id, soft=False)
|
||||
process_due()
|
||||
assert server.statuses == {}
|
||||
|
||||
def test_unpublish_then_republish(self, app, server):
|
||||
note = publish(app)
|
||||
with app.test_request_context():
|
||||
update_note(id=note.id, published=False)
|
||||
process_due()
|
||||
assert server.statuses == {}
|
||||
update_note(id=note.id, published=True)
|
||||
process_due()
|
||||
assert len(server.statuses) == 1
|
||||
|
||||
def test_opt_out_sends_nothing(self, app, server):
|
||||
note = publish(app, targets=())
|
||||
assert server.calls == []
|
||||
assert rows(app, note.id) == []
|
||||
|
||||
def test_unticking_later_deletes_status(self, app, server):
|
||||
note = publish(app)
|
||||
with app.test_request_context():
|
||||
select_targets(note.id, [])
|
||||
process_due()
|
||||
assert server.statuses == {}
|
||||
|
||||
def test_drafts_are_not_sent(self, app, server):
|
||||
with app.test_request_context():
|
||||
note = create_note(content="draft", published=False)
|
||||
select_targets(note.id, ["gotosocial"])
|
||||
process_due()
|
||||
assert server.statuses == {}
|
||||
|
||||
def test_remote_deleted_by_hand_is_reposted_on_edit(self, app, server):
|
||||
note = publish(app)
|
||||
server.statuses.clear()
|
||||
with app.test_request_context():
|
||||
note_changed(note.id)
|
||||
process_due()
|
||||
assert len(server.statuses) == 1
|
||||
|
||||
def test_edit_during_send_is_not_lost(self, app, server):
|
||||
with app.test_request_context():
|
||||
note = create_note(content="v1", published=True)
|
||||
select_targets(note.id, ["gotosocial"])
|
||||
# Simulate an edit arriving while the first send is in flight
|
||||
original = syndication._reconcile
|
||||
|
||||
def edit_mid_send(row, target):
|
||||
result = original(row, target)
|
||||
update_note(id=note.id, content="v2")
|
||||
return result
|
||||
|
||||
syndication._reconcile = edit_mid_send
|
||||
try:
|
||||
process_due()
|
||||
finally:
|
||||
syndication._reconcile = original
|
||||
assert rows(app, note.id)[0]["state"] == "pending"
|
||||
process_due()
|
||||
assert server.statuses["1"]["status"].startswith("v2")
|
||||
|
||||
|
||||
class TestFailures:
|
||||
def test_server_error_retries_later(self, app, server):
|
||||
server.fail_next = 503
|
||||
note = publish(app)
|
||||
(row,) = rows(app, note.id)
|
||||
assert row["state"] == "pending"
|
||||
assert row["attempts"] == 1
|
||||
assert "503" in row["last_error"]
|
||||
|
||||
def test_client_error_fails_permanently(self, app, server):
|
||||
server.fail_next = 422
|
||||
note = publish(app)
|
||||
(row,) = rows(app, note.id)
|
||||
assert row["state"] == "failed"
|
||||
|
||||
def test_retry_button(self, app, server):
|
||||
server.fail_next = 422
|
||||
note = publish(app)
|
||||
(row,) = rows(app, note.id)
|
||||
client = app.test_client()
|
||||
with app.test_request_context():
|
||||
client.set_cookie("starpunk_session", create_session("https://example.com"))
|
||||
client.post(f"/admin/syndication/{row['id']}/retry")
|
||||
with app.test_request_context():
|
||||
process_due()
|
||||
assert rows(app, note.id)[0]["state"] == "synced"
|
||||
|
||||
def test_no_targets_configured_does_nothing(self, tmp_path):
|
||||
data = tmp_path / "d"
|
||||
app = create_app({
|
||||
"TESTING": True, "SITE_URL": "https://b.example", "ADMIN_ME": "https://e.example",
|
||||
"SESSION_SECRET": "x", "DATA_PATH": data, "NOTES_PATH": data / "notes",
|
||||
"DATABASE_PATH": data / "s.db",
|
||||
})
|
||||
with app.test_request_context():
|
||||
note = create_note(content="x", published=True)
|
||||
select_targets(note.id, ["gotosocial"])
|
||||
assert process_due() == 0
|
||||
assert get_note_syndications(note.id) == []
|
||||
|
||||
|
||||
class TestMedia:
|
||||
def test_images_are_uploaded_with_alt_text(self, app, server):
|
||||
from starpunk.media import attach_media_to_note, save_media
|
||||
|
||||
buf = io.BytesIO()
|
||||
Image.new("RGB", (800, 600), "red").save(buf, format="JPEG")
|
||||
with app.test_request_context():
|
||||
note = create_note(content="With a picture", published=True)
|
||||
media = save_media(buf.getvalue(), "red.jpg")
|
||||
attach_media_to_note(note.id, [media["id"]], ["A red square"])
|
||||
select_targets(note.id, ["gotosocial"])
|
||||
process_due()
|
||||
assert ("POST", "/api/v2/media") in server.calls
|
||||
assert server.statuses["1"]["media_ids"] == ["m1"]
|
||||
|
||||
|
||||
class TestEntryPoints:
|
||||
@pytest.fixture
|
||||
def admin(self, app):
|
||||
client = app.test_client()
|
||||
with app.test_request_context():
|
||||
client.set_cookie("starpunk_session", create_session("https://example.com"))
|
||||
return client
|
||||
|
||||
def test_new_note_form_ticks_target_by_default(self, admin):
|
||||
html = admin.get("/admin/new").data.decode()
|
||||
assert 'name="syndicate_to" value="gotosocial" checked' in html
|
||||
|
||||
def test_admin_create_queues(self, app, admin):
|
||||
admin.post("/admin/new", data={
|
||||
"content": "From the editor", "published": "on",
|
||||
"syndication_form": "1", "syndicate_to": "gotosocial",
|
||||
})
|
||||
with app.app_context():
|
||||
from starpunk.database import get_db
|
||||
assert get_db(app).execute("SELECT COUNT(*) FROM syndications").fetchone()[0] == 1
|
||||
|
||||
def test_admin_create_opt_out(self, app, admin):
|
||||
admin.post("/admin/new", data={"content": "Not this one", "published": "on", "syndication_form": "1"})
|
||||
with app.app_context():
|
||||
from starpunk.database import get_db
|
||||
assert get_db(app).execute("SELECT COUNT(*) FROM syndications").fetchone()[0] == 0
|
||||
|
||||
def _token(self, app):
|
||||
with app.test_request_context():
|
||||
return create_access_token("https://quill.example/", "create")["access_token"]
|
||||
|
||||
def test_micropub_lists_target(self, app):
|
||||
token = self._token(app)
|
||||
data = app.test_client().get(
|
||||
"/micropub?q=syndicate-to", headers={"Authorization": f"Bearer {token}"}
|
||||
).get_json()
|
||||
assert data["syndicate-to"] == [{"uid": f"{GTS}/", "name": "GoToSocial"}]
|
||||
|
||||
def test_micropub_default_is_all_targets(self, app):
|
||||
token = self._token(app)
|
||||
response = app.test_client().post(
|
||||
"/micropub", data={"h": "entry", "content": "via quill"},
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
)
|
||||
assert response.status_code == 201
|
||||
assert "/note/" in response.headers["Location"]
|
||||
assert response.headers["Location"].startswith("https://blog.example.com/note/")
|
||||
with app.app_context():
|
||||
from starpunk.database import get_db
|
||||
assert get_db(app).execute("SELECT COUNT(*) FROM syndications").fetchone()[0] == 1
|
||||
|
||||
def test_micropub_explicit_empty_choice(self, app):
|
||||
token = self._token(app)
|
||||
app.test_client().post(
|
||||
"/micropub", json={"type": ["h-entry"], "properties": {"content": ["no crosspost"]},
|
||||
"mp-syndicate-to": []},
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
)
|
||||
with app.app_context():
|
||||
from starpunk.database import get_db
|
||||
assert get_db(app).execute("SELECT COUNT(*) FROM syndications").fetchone()[0] == 0
|
||||
|
||||
def test_note_page_links_to_crosspost(self, app, server):
|
||||
note = publish(app)
|
||||
html = app.test_client().get(f"/note/{note.slug}").data.decode()
|
||||
assert f'class="u-syndication" href="{GTS}/@phil/statuses/1"' in html
|
||||
|
||||
def test_edit_page_shows_status_and_separate_delete_form(self, app, admin, server):
|
||||
note = publish(app)
|
||||
html = admin.get(f"/admin/edit/{note.id}").data.decode()
|
||||
assert 'value="gotosocial" checked' in html
|
||||
assert f"{GTS}/@phil/statuses/1" in html
|
||||
# Delete button belongs to its own form, not the edit form
|
||||
assert 'form="delete-note-form"' in html
|
||||
edit_form = html.split('class="note-form"', 1)[1].split("</form>", 1)[0]
|
||||
assert "<form" not in edit_form
|
||||
Reference in new issue
Block a user