Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
03a1c8fcea | ||
|
|
67c0411fd8 | ||
|
|
f7a936562c | ||
|
|
62e257e0e0 | ||
|
|
22e588377f | ||
|
|
b9e5321df2 | ||
|
|
52ecf0f713 | ||
|
|
5fb521e89e | ||
|
|
bba2178496 | ||
|
|
41c373f46c | ||
|
|
7158713a2a | ||
|
|
73a68157cc | ||
|
|
bda7adc449 | ||
|
|
aa88db50b2 | ||
|
|
c9b882aa46 | ||
|
|
6368365bd6 | ||
|
|
191c99b59b | ||
|
|
b7f8e1612c | ||
|
|
66aecc2bcb | ||
|
|
ccb9f6dce2 | ||
|
|
978c13358b | ||
|
|
9a0d82bab2 | ||
|
|
84411a16d9 | ||
|
|
4ed2033cf9 | ||
|
|
d354dc9733 |
No files matched your search
@@ -130,6 +130,9 @@ CSRF_ENABLED=true
|
||||
# Background worker that verifies received webmentions, sends outgoing ones,
|
||||
# and sends response notifications (v2.14.0) (default: true)
|
||||
# WEBMENTION_WORKER_ENABLED=true
|
||||
# Hosts never sent webmentions, comma-separated, e.g. self-hosted apps that
|
||||
# don't accept them. Links to this site itself are always skipped (v2.18.3)
|
||||
# WEBMENTION_SKIP_HOSTS=git.example.com,photos.example.com
|
||||
# Seconds between queue checks (default: 10) and after an error (default: 60)
|
||||
# WEBMENTION_POLL_INTERVAL=10
|
||||
# WEBMENTION_ERROR_BACKOFF=60
|
||||
|
||||
+152
@@ -7,6 +7,158 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.20.0] - 2026-11-13
|
||||
|
||||
### Added
|
||||
|
||||
- Crossposted replies look like replies. A reply to a post on the same network is posted as a native reply: on GoToSocial it threads under the fediverse post and starts with the author's @mention (so they're notified and their server gets it); on Bluesky it threads under the Bluesky post
|
||||
- A reply to one of your own posts (its page here, or any of its crossposts) threads under that post's crosspost on every network, Threads included
|
||||
- A reply that can't thread on a network (a fediverse post crossposted to Bluesky or Threads, a blog post anywhere) starts with "↩ Replying to <author>" and the post's link. On Bluesky the author's name is the link, to save characters
|
||||
|
||||
Crossposts made before this release are unchanged: edits can't move a post into a thread.
|
||||
|
||||
## [2.19.0] - 2026-11-13
|
||||
|
||||
### Added
|
||||
|
||||
- Replies show what they reply to. A note's page shows a card with the post's author (avatar, name, handle), its text, its first image and when it was posted, linking to the original. The home, tag and likes pages show "↩ In reply to <author>" with a short quote of the post
|
||||
- Context for fediverse and Bluesky posts. These pages have no microformats, so StarPunk asks the post's Mastodon server's API, then this site's GoToSocial instance (which also reaches GoToSocial and locked-down Mastodon servers), or the public Bluesky API. Other pages without microformats use their OpenGraph tags
|
||||
- The avatars and images in reply contexts are saved on this site, like response avatars (the CSP doesn't allow images from other sites). Images are kept at most 640 px wide in `DATA_PATH/previews/`
|
||||
|
||||
### Fixed
|
||||
|
||||
- A reply to a Mastodon post showed only the server's name ("defcon.social"): the page had no h-entry, so the fetch found nothing and the domain stood in for the author. Contexts fetched before this release are fetched once more by the webmention worker
|
||||
- The quoted text of a replied-to post was shown with its HTML tags escaped
|
||||
|
||||
### Changed
|
||||
|
||||
- Migration 024 adds `author_handle`, `photo`, `photo_alt` and `source` to `reply_contexts`
|
||||
|
||||
## [2.18.3] - 2026-11-12
|
||||
|
||||
### Added
|
||||
|
||||
- `WEBMENTION_SKIP_HOSTS`: comma-separated hosts that are never sent webmentions, for self-hosted apps that don't accept them (Gitea, say). The Settings page lists them
|
||||
|
||||
### Fixed
|
||||
|
||||
- StarPunk sent webmentions to links pointing at itself. It only takes webmentions for its posts, and its own public address can time out from inside the container, so these failed and sat in the Log. Links to this site are now skipped
|
||||
- Sends to this site or a skipped host that were already queued or failed are dropped when the worker starts, so they leave the Log
|
||||
|
||||
## [2.18.2] - 2026-11-12
|
||||
|
||||
### Fixed
|
||||
- Tables in notes and articles: words in cells were split mid-word (a narrow
|
||||
first column read "Colo/r", "Blac/k") and cells had no padding or left
|
||||
alignment. Tables now keep words whole and render as a card that fills the
|
||||
column: raised background, rounded border, shaded header row, roomy cells,
|
||||
sans-serif text at reading size, bold first column. Rendered tables are
|
||||
wrapped in `<div class="table-wrap">`, which scrolls sideways on narrow
|
||||
screens.
|
||||
|
||||
### Changed
|
||||
- Public pages are wider: 48rem instead of 40rem (about 85 characters per
|
||||
line at the reading font). Admin pages stay at 60rem.
|
||||
- The site header (title, menu, search) and footer are 60rem wide on every
|
||||
page, so they sit in the same place on public and admin pages; only the
|
||||
content column differs.
|
||||
|
||||
## [2.18.1] - 2026-11-12
|
||||
|
||||
### Changed
|
||||
|
||||
- The MCP `create_note` tool says that articles live at `/article/<slug>` with a slug made from the title, so AI clients leave `slug` out unless they want a specific one
|
||||
|
||||
## [2.18.0] - 2026-11-12
|
||||
|
||||
A tidier admin area.
|
||||
|
||||
### Added
|
||||
- **Log page** (`/admin/log`): everything that is failing right now, newest
|
||||
first. Each entry shows when it happened, what failed (crosspost, response
|
||||
check, outgoing webmention, reply-context fetch, WebSub delivery,
|
||||
notifications, refused silo credentials), a link to the affected post and
|
||||
the error message, with Retry / Check now where that applies. It reads the
|
||||
error columns the workers already keep, so an entry goes away once the
|
||||
next attempt succeeds. The admin nav shows the error count.
|
||||
- **Settings page** (`/admin/settings`) for site-wide settings. Notification
|
||||
status and the test button live here now.
|
||||
- Webmentions, Log and Settings links in the admin nav.
|
||||
|
||||
### Changed
|
||||
- The Webmentions page no longer has the Back to Dashboard button or the
|
||||
Notifications section, and links to the Log instead of counting response
|
||||
check errors.
|
||||
|
||||
## [2.17.1] - 2026-11-12
|
||||
|
||||
### Fixed
|
||||
|
||||
- Crossposting failed in 2.17.0 (never deployed): post URLs were built with Flask's `url_for`, which needs a web request, and crossposts run in a background worker. URLs are now built from `SITE_URL`
|
||||
|
||||
## [2.17.0] - 2026-11-12
|
||||
|
||||
Articles get URLs of their own (ADR-077).
|
||||
|
||||
### Changed
|
||||
|
||||
- Articles live at `/article/<slug>` instead of `/note/<slug>`. The other address redirects (301), so links keep working when a note gains or loses a title. Feeds, pages, Micropub, crossposts, webmentions, notifications, search and the MCP tools all give the new URLs
|
||||
- A new article's slug comes from its title (`Why I Left Twitter` → `why-i-left-twitter`; `-2` if taken). An `mp-slug` still wins, notes keep timestamp slugs, and slugs don't change when a title is edited later
|
||||
- A Micropub update that adds or removes a title answers 201 with the post's new URL in `Location`, as the spec requires
|
||||
|
||||
## [2.16.0] - 2026-11-11
|
||||
|
||||
Articles as the IndieWeb defines them: titles and summaries stored apart from the text (ADR-076).
|
||||
|
||||
**Before upgrading, back up the database and the `data` folder (notes and `.trash`).** Migration 023 moves each note's `# Heading` title out of its Markdown file into the database.
|
||||
|
||||
### Added
|
||||
|
||||
- Notes have a title and a summary of their own. A note with a title is an article. The editor has Title and Summary fields, and an article may have no body text
|
||||
- Micropub `summary` is stored when sent with `content`, can be replaced and deleted, and comes back from `q=source`. A summary on its own is still the content, as Post Type Discovery reads it. The MCP `create_note` and `update_note` tools take a `summary`
|
||||
- Article pages mark up the summary as `p-summary`; Atom entries have `<summary>` and JSON Feed items `summary`
|
||||
|
||||
### Changed
|
||||
|
||||
- A `# Heading` at the top of a note is now just a heading, not a title. Micropub `name` is stored as the title instead of being added to the text; a heading that repeats it is dropped
|
||||
- Feed items have a title only for articles. RSS items and JSON Feed items for notes have none, and Atom entries an empty one, so feed readers no longer show notes as headlines
|
||||
- On the homepage and tag pages an article shows its title, its summary (or the start of its text) and "Read more", without its images
|
||||
- Micropub content sent as both `html` and `text` is stored as the HTML. The text is usually a plain fallback, so articles lost their images, links and headings
|
||||
- Images embedded in Micropub content from other sites are copied into the media library and the text points at the copies, so an article doesn't break when the client's or a CDN's host goes away. Best effort: an image that can't be fetched keeps its URL (up to 20 per post)
|
||||
- Articles are crossposted to GoToSocial, Bluesky and Threads as title, summary (or the start of the text), link and hashtags, without images. On Bluesky the link card is the article. Notes are crossposted in full as before
|
||||
|
||||
### Fixed
|
||||
|
||||
- Feeds showed an article's title twice: as the item title and again at the top of the content
|
||||
- A line starting with a hashtag (`#indieweb is fun`) was shown as a heading on pages and in feeds, and crossposted without its `#`. Only `#` followed by a space starts a heading now, as in CommonMark
|
||||
- HTML content from Micropub is sanitized as is instead of going through Markdown, which added stray line breaks and empty paragraphs
|
||||
- Previews, image alt text and crosspost teasers showed raw Markdown (`**bold**`, `[link](url)`); the excerpt is now plain text
|
||||
## [2.15.0] - 2026-11-11
|
||||
|
||||
MCP server for AI clients (ADR-075).
|
||||
|
||||
### Added
|
||||
|
||||
- An [MCP](https://modelcontextprotocol.io/) server at `SITE_URL/mcp`. AI assistants such as claude.ai custom connectors and Claude Code can search, list and read notes and drafts, list tags, create notes (drafts by default), update, delete and undelete them, and upload photos. Writes go through the Micropub handlers, so scopes, crossposting, webmentions and WebSub behave the same as with any Micropub app
|
||||
- The MCP server runs as a sidecar from the same image (`uvicorn starpunk.mcp_server.app:app --port 8001`) with `BACKGROUND_WORKERS_ENABLED=false`; the proxy sends `/mcp*` and `/.well-known/oauth-protected-resource*` to it. New dependencies: `mcp` 2.3 and `uvicorn` 0.54
|
||||
- The IndieAuth server accepts RFC 8707 `resource` for the MCP endpoint and binds codes and tokens to it (migration 022). Its metadata advertises client ID metadata documents and public clients (`token_endpoint_auth_methods_supported: ["none"]`), so Claude connects without dynamic client registration. The consent screen says when an AI assistant is asking, and Connected Apps marks MCP tokens
|
||||
- `BACKGROUND_WORKERS_ENABLED` (default `true`) turns off the webmention, WebSub and syndication workers
|
||||
|
||||
### Fixed
|
||||
|
||||
- `scopes_supported` in the IndieAuth metadata lists `draft`, which Micropub already honoured
|
||||
|
||||
## [2.14.2] - 2026-11-10
|
||||
|
||||
### Added
|
||||
|
||||
- Quotes have a count in the line under each note (❝) and in the counts on the home, tag and Likes pages. It includes quotes a silo reports without names (Threads, and people who opted out of backfeed), which were stored but shown nowhere
|
||||
|
||||
### Removed
|
||||
|
||||
- The editor pages no longer load `marked` from a CDN and an empty `static/js/preview.js`; nothing used them
|
||||
- The empty `templates/feed.xml`, the v1 `static/indielogin-test.html`, and the `THREADS_API` setting that crossposting read but nothing could set
|
||||
|
||||
## [2.14.1] - 2026-11-10
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -78,6 +78,8 @@ USER starpunk
|
||||
|
||||
# Expose application port
|
||||
EXPOSE 8000
|
||||
# MCP sidecar (same image, run with: uvicorn starpunk.mcp_server.app:app --host 0.0.0.0 --port 8001)
|
||||
EXPOSE 8001
|
||||
|
||||
# Health check
|
||||
# Uses httpx (already in requirements) to verify app is responding
|
||||
|
||||
@@ -23,6 +23,9 @@ what changed in each release is in [CHANGELOG.md](CHANGELOG.md).
|
||||
- Built-in [IndieAuth](https://indieauth.spec.indieweb.org/) server for
|
||||
Micropub apps (consent screen, tokens, revocation, Connected Apps page)
|
||||
- Admin login through any OpenID Connect provider (e.g. Pocket ID)
|
||||
- [MCP](https://modelcontextprotocol.io/) server at `/mcp` so AI assistants
|
||||
(claude.ai connectors, Claude Code…) can search, read and write notes
|
||||
after you approve them, run as an optional sidecar (ADR-075)
|
||||
- [Webmention](https://www.w3.org/TR/webmention/): receiving (verified,
|
||||
moderated, rate-limited) and sending; replies, quotes and mentions shown
|
||||
on each note, with likes, reposts and replies as a line of counts with
|
||||
|
||||
@@ -13,10 +13,11 @@ original v0.x technology choices are kept in
|
||||
|----------|-------------|
|
||||
| [overview.md](overview.md) | System overview: purpose, request flow, modules and routes, background workers, storage, auth model, URL surface, caching, security posture |
|
||||
| [technology-stack.md](technology-stack.md) | Current runtime, Python dependencies, standards implemented, testing and CI |
|
||||
| [database.md](database.md) | SQLite schema after migrations 001–021, migration runner and its lock, connection pool |
|
||||
| [database.md](database.md) | SQLite schema after migrations 001–022, migration runner and its lock, connection pool |
|
||||
| [deployment.md](deployment.md) | Container image stages, CI workflow, release process, `/data` layout, reverse proxy, all environment variables |
|
||||
| [simplified-auth-architecture.md](simplified-auth-architecture.md) | Admin login via OpenID Connect and the built-in IndieAuth server that issues Micropub tokens (ADR-064) |
|
||||
| [indieweb-protocols.md](indieweb-protocols.md) | How Micropub, IndieAuth, Webmention, microformats2, WebSub and the feeds are implemented (including backfeed responses, web actions and notifications), with compliance status and known gaps |
|
||||
| [mcp.md](mcp.md) | The MCP server for AI clients: sidecar, authorization flow, tools (ADR-075) |
|
||||
| [syndication-architecture.md](syndication-architecture.md) | Crossposting (POSSE) to GoToSocial, Bluesky and Threads: syndications table, worker, reconcile rules, backfeed of responses (ADR-073) |
|
||||
|
||||
## Where to start
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
**Version**: 2.14.0 (2026-11-10)
|
||||
**Status**: Current. Verified against a database created by `create_app()`
|
||||
with migrations 001–021.
|
||||
with migrations 001–022.
|
||||
|
||||
StarPunk uses one SQLite file, `starpunk.db` under `DATA_PATH` (`/data` in
|
||||
the container), in WAL mode. Note *content* is not in the database: it lives
|
||||
@@ -75,6 +75,7 @@ Background: [ADR-031](../decisions/ADR-031-database-migration-system-redesign.md
|
||||
| `019_shared_rate_limits.sql` | 2.11.0 | `rate_limits` (counters shared by all processes) |
|
||||
| `020_backfeed.sql` | 2.13.0 | Rebuilds `webmentions` adding `origin`, `remote_id`, the `quote` type and an index on (`target_note_id`, `origin`); `syndications.backfeed_next_at`, `backfeed_checked_at`, `backfeed_claimed_until`, `backfeed_attempts`, `backfeed_error`, `backfeed_backfill`, `likes_count`, `reposts_count`, `quotes_count` |
|
||||
| `021_notifications.sql` | 2.14.0 | `webmentions.notified_at`; existing rows are marked as notified |
|
||||
| `022_indieauth_resource.sql` | 2.15.0 | `indieauth_codes.resource` and `indieauth_tokens.resource` (RFC 8707, MCP tokens; ADR-075) |
|
||||
|
||||
New migrations are numbered `NNN_description.sql`, must be safe to run inside
|
||||
one transaction, and should use `IF NOT EXISTS` where SQLite allows it.
|
||||
@@ -128,12 +129,12 @@ value), `me`, `created_at`, `expires_at`, `last_used_at`, `user_agent`,
|
||||
(primary key), `nonce`, `code_verifier` (PKCE), `next_url`, `expires_at`.
|
||||
Consumed once on callback.
|
||||
|
||||
**`indieauth_codes`**: authorization codes issued to Micropub clients:
|
||||
**`indieauth_codes`**: authorization codes issued to Micropub and MCP clients:
|
||||
`code_hash`, `client_id`, `redirect_uri`, `scope`, `code_challenge`,
|
||||
`expires_at`, `used_at` (single use).
|
||||
`resource` (`SITE_URL/mcp` or NULL), `expires_at`, `used_at` (single use).
|
||||
|
||||
**`indieauth_tokens`**: access tokens: `token_hash`, `client_id`, `scope`,
|
||||
`created_at`, `expires_at`, `last_used_at`, `revoked_at`. Listed and revoked
|
||||
`resource` (`SITE_URL/mcp` for MCP tokens, NULL for Micropub), `created_at`, `expires_at`, `last_used_at`, `revoked_at`. Listed and revoked
|
||||
in `/admin/tokens`.
|
||||
|
||||
**Legacy tables, unused**: `auth_state` (v1 IndieLogin login state), `tokens`
|
||||
|
||||
@@ -125,6 +125,14 @@ WebSub rate limits need; without it, every client appears as the proxy's
|
||||
address and shares one limit. Do not set it if the container port can be
|
||||
reached without the proxy, because the header could then be forged.
|
||||
|
||||
The MCP sidecar (v2.15.0, ADR-075) is a second container from the same
|
||||
image, with the same environment and `/data` volume plus
|
||||
`BACKGROUND_WORKERS_ENABLED=false`, running
|
||||
`uvicorn starpunk.mcp_server.app:app --host 0.0.0.0 --port 8001`. The proxy
|
||||
sends `/mcp*` and `/.well-known/oauth-protected-resource*` to it, keeping the
|
||||
`Host` header; everything else goes to port 8000. Without the sidecar the
|
||||
site simply has no MCP endpoint. See [mcp.md](mcp.md).
|
||||
|
||||
`SITE_URL` must be the public `https://` URL: it is used for absolute links,
|
||||
feed and WebSub topic URLs, the IndieAuth issuer and redirect URIs, and to
|
||||
decide whether the session cookie is `Secure`.
|
||||
@@ -175,6 +183,7 @@ ignored with a warning.
|
||||
| `FLASK_DEBUG` | `0` | Debug off unless `1` (`flask run --debug` sets it locally); the image sets `0` |
|
||||
| `LOG_LEVEL` | `INFO` | `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL` |
|
||||
| `VERSION` | package `__version__` | Overrides the reported version; normally unset |
|
||||
| `BACKGROUND_WORKERS_ENABLED` | `true` | `false` stops the webmention, WebSub and syndication workers; set in the MCP sidecar (v2.15.0) |
|
||||
|
||||
### Feeds
|
||||
|
||||
|
||||
@@ -266,7 +266,7 @@ tombstone, so a re-sent webmention or backfeed doesn't bring it back.
|
||||
Changes that alter what is shown also re-send the note's webmentions.
|
||||
|
||||
Note pages show approved, valid responses. Under the note, a line of counts
|
||||
(♥ likes, 🔁 reposts, 💬 replies, bookmarks) opens a card on hover or focus
|
||||
(♥ likes, 🔁 reposts, 💬 replies, ❝ quotes (v2.14.2), bookmarks) opens a card on hover or focus
|
||||
listing who responded and where from, including anonymous silo counts. Each
|
||||
like, repost and reply count holds an `<indie-action do with>` web action
|
||||
whose fallback links to the note's crossposts (v2.13.2). The Responses
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
# MCP server (v2.15.0)
|
||||
|
||||
How AI clients use StarPunk through the
|
||||
[Model Context Protocol](https://modelcontextprotocol.io/). The decision and
|
||||
alternatives are in [ADR-075](../decisions/ADR-075-mcp-server.md).
|
||||
|
||||
## Shape
|
||||
|
||||
```
|
||||
client ──HTTPS──> reverse proxy
|
||||
/mcp*, /.well-known/oauth-protected-resource* ──> starpunk-mcp:8001 (uvicorn, MCP SDK)
|
||||
everything else (incl. /auth/*, /.well-known/oauth-authorization-server)
|
||||
──> starpunk:8000 (gunicorn, Flask)
|
||||
starpunk and starpunk-mcp: same image, same environment, same /data volume
|
||||
```
|
||||
|
||||
| Module | Role |
|
||||
|---|---|
|
||||
| `starpunk/mcp_server/app.py` | ASGI entry: `uvicorn starpunk.mcp_server.app:app --host 0.0.0.0 --port 8001` |
|
||||
| `starpunk/mcp_server/server.py` | `create_mcp_server(app)` and `create_asgi_app(app=None)`: stateless Streamable HTTP, JSON responses, 16 MB request limit, DNS rebinding protection |
|
||||
| `starpunk/mcp_server/auth.py` | `StarPunkTokenVerifier`: StarPunk tokens to SDK `AccessToken`s |
|
||||
| `starpunk/mcp_server/tools.py` | The tools; each runs in a thread inside `app.test_request_context(base_url=SITE_URL)` |
|
||||
|
||||
`create_asgi_app()` with no argument builds the Flask app from the
|
||||
environment with `BACKGROUND_WORKERS_ENABLED=false`. The sidecar container
|
||||
should also set that variable, for clarity.
|
||||
|
||||
## Authorization flow
|
||||
|
||||
1. The client POSTs to `/mcp` without a token and gets `401` with
|
||||
`WWW-Authenticate: Bearer ... resource_metadata="SITE_URL/.well-known/oauth-protected-resource/mcp"`.
|
||||
2. That document names `SITE_URL/mcp` as the resource and `SITE_URL/` as
|
||||
the authorization server. The client reads
|
||||
`/.well-known/oauth-authorization-server`, which says it accepts client ID
|
||||
metadata documents and public clients.
|
||||
3. The client sends the owner to `/auth/authorize` with its `client_id` URL,
|
||||
PKCE and `resource=SITE_URL/mcp`. StarPunk fetches the client's metadata
|
||||
document to check `redirect_uri` (unless it has the same origin as the
|
||||
`client_id`), and the owner logs in with OIDC and approves. With no
|
||||
`scope`, the default MCP scopes are offered.
|
||||
4. The token endpoint exchanges the code. The token is bound to the
|
||||
resource.
|
||||
5. Each MCP request carries the token. The verifier looks it up and the SDK
|
||||
refuses it unless its resource is `SITE_URL/mcp`.
|
||||
|
||||
Details of codes, tokens and scopes are in
|
||||
[simplified-auth-architecture.md](simplified-auth-architecture.md).
|
||||
|
||||
## Tools
|
||||
|
||||
| Tool | Does | Scope |
|
||||
|---|---|---|
|
||||
| `search_notes(query, include_drafts, limit)` | FTS5 search, with snippets | any; drafts need a posting scope |
|
||||
| `list_notes(include_drafts, limit, offset)` | Newest first, summaries | any; drafts need a posting scope |
|
||||
| `get_note(url_or_slug)` | Microformats2 properties (as Micropub `q=source`) | any; drafts need a posting scope |
|
||||
| `list_tags(filter)` | Tags in use, most used first | posting scope |
|
||||
| `get_site_config()` | Micropub `q=config`, including `syndicate-to` targets | any |
|
||||
| `create_note(content, title, tags, draft=True, photos, in_reply_to, like_of, repost_of, bookmark_of, syndicate_to, slug, published_at)` | Micropub create | `create` or `draft` |
|
||||
| `update_note(url, content, title, tags, add_tags, remove_tags, draft, photos, in_reply_to)` | Micropub update | `update` |
|
||||
| `delete_note(url)`, `undelete_note(url)` | Soft delete and restore | `delete` |
|
||||
| `upload_media(filename, data_base64)` | Save an image, return its URL | `media` or `create` |
|
||||
|
||||
"Posting scope" means `create`, `update`, `delete` or `draft`. Limits are
|
||||
capped at 100. Micropub errors come back as tool errors (`isError`) with the
|
||||
Micropub error description.
|
||||
|
||||
## Operating it
|
||||
|
||||
- Environment: the same as the main container, plus
|
||||
`BACKGROUND_WORKERS_ENABLED=false`.
|
||||
- Proxy: route `/mcp*` and `/.well-known/oauth-protected-resource*` to port
|
||||
8001, keeping the `Host` header (DNS rebinding protection checks it
|
||||
against `SITE_URL`).
|
||||
- Connecting: claude.ai Settings → Connectors → Add custom connector with
|
||||
`SITE_URL/mcp`; Claude Code: `claude mcp add --transport http starpunk
|
||||
SITE_URL/mcp`, then `/mcp` to authenticate.
|
||||
- Revoking: Admin → Connected Apps (MCP tokens are marked "(MCP)").
|
||||
@@ -223,6 +223,9 @@ coordinated through the data volume:
|
||||
| WebSub worker (`websub_worker.py`) | `WEBSUB_ENABLED` and not testing | Same pattern with `websub-worker.lock` | Verifies subscription intents, delivers topic content (signed with `X-Hub-Signature` when the subscriber gave a secret), retries, expires leases; pings an external hub instead if `WEBSUB_HUB_URL` is set |
|
||||
| Syndication worker (`syndication.py`) | At least one crosspost target configured, and not testing | No file lock. Rows in `syndications` are claimed with a 5-minute `claimed_until` lease by an atomic `UPDATE`, so any process may work a row but only one at a time; a `version` column keeps edits made during a send. Threads token refreshes are serialised with a lock file next to `threads_token.json`. | Creates, updates or deletes the remote post for each (note, target) pair; up to 8 attempts. Then, only when `BACKFEED_ENABLED`, backfeed: checks copies for responses, each claimed with a `backfeed_claimed_until` lease (v2.13.0, `backfeed.py`, ADR-073) |
|
||||
|
||||
`BACKGROUND_WORKERS_ENABLED=false` starts none of them; the MCP sidecar
|
||||
(below) sets it so the workers only run in the main container.
|
||||
|
||||
All workers run inside an app context, are daemon threads (they end with the
|
||||
process) and sleep for their poll interval when there is nothing to do. Under
|
||||
the test suite (`TESTING=True`) none of them start; tests call the processing
|
||||
@@ -273,7 +276,14 @@ are in [simplified-auth-architecture.md](simplified-auth-architecture.md).
|
||||
OIDC) approves; the client exchanges the code at `/auth/token` (PKCE
|
||||
required). Codes and tokens are stored hashed. Tokens last
|
||||
`INDIEAUTH_TOKEN_LIFETIME_DAYS` and can be revoked at `/auth/revoke` or in
|
||||
`/admin/tokens`. Scopes: `create`, `update`, `delete`, `media`, `profile`.
|
||||
`/admin/tokens`. Scopes: `create`, `update`, `delete`, `media`, `draft`,
|
||||
`profile`.
|
||||
- **MCP (v2.15.0, ADR-075)**: AI clients use the same flow, identified by a
|
||||
client ID metadata document and asking for `resource=SITE_URL/mcp`. Their
|
||||
tokens are bound to that resource and checked by the MCP sidecar, a second
|
||||
container (uvicorn, official MCP SDK) that the proxy sends `/mcp` to. Its
|
||||
tools read, search and write notes through the Micropub handlers. See
|
||||
[mcp.md](mcp.md).
|
||||
- **Development**: `DEV_MODE=true` adds `/dev/login`, which logs in as
|
||||
`DEV_ADMIN_ME` without any checks. Never enabled in production; start-up
|
||||
logs a warning when it is on.
|
||||
|
||||
@@ -137,8 +137,10 @@ form `{"error", "error_description"}`.
|
||||
"token_endpoint": "<SITE_URL>/auth/token",
|
||||
"revocation_endpoint": "<SITE_URL>/auth/revoke",
|
||||
"revocation_endpoint_auth_methods_supported": ["none"],
|
||||
"scopes_supported": ["create", "update", "delete", "media", "profile"],
|
||||
"scopes_supported": ["create", "update", "delete", "media", "draft", "profile"],
|
||||
"response_types_supported": ["code"],
|
||||
"token_endpoint_auth_methods_supported": ["none"],
|
||||
"client_id_metadata_document_supported": true,
|
||||
"grant_types_supported": ["authorization_code"],
|
||||
"code_challenge_methods_supported": ["S256"],
|
||||
"authorization_response_iss_parameter_supported": true
|
||||
@@ -192,7 +194,7 @@ requested and left checked, issues a code, and redirects with `code`,
|
||||
| Value | 32 random bytes, URL-safe | 32 random bytes, URL-safe |
|
||||
| Stored as | SHA-256 hash | SHA-256 hash |
|
||||
| Lifetime | 10 minutes | `INDIEAUTH_TOKEN_LIFETIME_DAYS` (default 90) |
|
||||
| Bound to | `client_id`, `redirect_uri`, `scope`, `code_challenge` | `client_id`, `scope` |
|
||||
| Bound to | `client_id`, `redirect_uri`, `scope`, `code_challenge`, `resource` | `client_id`, `scope`, `resource` |
|
||||
|
||||
`redeem_authorization_code` handles redemption for both `POST /auth/token`
|
||||
and `POST /auth/authorize`:
|
||||
@@ -213,11 +215,22 @@ refresh tokens: a client re-authorizes when its token expires.
|
||||
|
||||
Expired codes are deleted whenever a new code is issued.
|
||||
|
||||
**Resource indicators (v2.15.0, ADR-075).** An authorization request may
|
||||
include `resource` (RFC 8707). The only accepted value is `SITE_URL/mcp`
|
||||
(a trailing slash is ignored); anything else is `invalid_target`. With it,
|
||||
a request without `scope` is offered `create update delete media draft`, and
|
||||
the consent screen says the client is an AI assistant using MCP. The resource
|
||||
is stored on the code and then on the token. A token request that names a
|
||||
different resource gets `invalid_target`. Micropub clients send no resource
|
||||
and their tokens have none. The MCP endpoint only accepts tokens for its
|
||||
resource; see [mcp.md](mcp.md).
|
||||
|
||||
### Verification and revocation
|
||||
|
||||
- `verify_access_token(token)` hashes the token and accepts it if it is
|
||||
neither revoked nor expired, then updates `last_used_at`. It returns
|
||||
`{me: ADMIN_ME, client_id, scope}`.
|
||||
`{me: ADMIN_ME, client_id, scope, resource, expires_at}`. The legacy
|
||||
`GET /auth/token` still answers with only `me`, `client_id` and `scope`.
|
||||
- `check_scope(required, scope)` matches whole words in the space-separated
|
||||
scope string.
|
||||
- Revocation sets `revoked_at`. It can happen by token value, through
|
||||
@@ -308,8 +321,8 @@ enough. If it is another page, that page must advertise StarPunk's
|
||||
|---|---|---|
|
||||
| `sessions` | Admin browser sessions | `session_token_hash`, `me`, `expires_at`, `last_used_at`, `user_agent`, `ip_address` |
|
||||
| `oidc_login_state` | One pending OIDC login | `state` (PK), `nonce`, `code_verifier`, `next_url`, `expires_at` |
|
||||
| `indieauth_codes` | Authorization codes | `code_hash`, `client_id`, `redirect_uri`, `scope`, `code_challenge`, `expires_at`, `used_at` |
|
||||
| `indieauth_tokens` | Access tokens | `token_hash`, `client_id`, `scope`, `expires_at`, `last_used_at`, `revoked_at` |
|
||||
| `indieauth_codes` | Authorization codes | `code_hash`, `client_id`, `redirect_uri`, `scope`, `code_challenge`, `resource` (migration 022), `expires_at`, `used_at` |
|
||||
| `indieauth_tokens` | Access tokens | `token_hash`, `client_id`, `scope`, `resource` (migration 022), `expires_at`, `last_used_at`, `revoked_at` |
|
||||
|
||||
The legacy v1 tables `auth_state`, `tokens` and `authorization_codes` are
|
||||
not used (see the migration's header comment).
|
||||
@@ -338,7 +351,6 @@ connection pinned, a 5-second limit).
|
||||
|
||||
Known gaps:
|
||||
|
||||
- `scopes_supported` leaves out `draft`, although Micropub honours it and
|
||||
the consent screen grants it when requested. Unknown scopes are also shown
|
||||
and granted as requested; they just have no effect.
|
||||
- Unknown scopes are shown and granted as requested; they just have no
|
||||
effect. (`draft` has been listed in `scopes_supported` since v2.15.0.)
|
||||
- No token introspection endpoint and no refresh tokens.
|
||||
@@ -139,8 +139,8 @@ again with the new content. `_fail()` does the same.
|
||||
a Threads container in `ERROR`/`EXPIRED`, and Graph API auth/permission
|
||||
errors (codes 10, 190, 200).
|
||||
- Other errors retry after `min(30 s × 2^attempts, 6 h)`. After 8 attempts
|
||||
the row becomes `failed`. The error is shown on the edit page with a
|
||||
Retry button. When a row finally fails, the owner also gets a `problems`
|
||||
the row becomes `failed`. The error is shown on the edit page and on the
|
||||
admin Log page (v2.18.0), both with a Retry button. When a row finally fails, the owner also gets a `problems`
|
||||
notification (ADR-074).
|
||||
- `RemoteGone` (404, or Graph error 100/33) during an update means the copy
|
||||
was deleted remotely, so a new one is created. During a delete it counts as
|
||||
@@ -273,7 +273,7 @@ reposts) that are due, each under a lease (`backfeed_claimed_until`):
|
||||
(`site_state.backfeed_backfill_last`). Start and finish times are in
|
||||
`site_state`.
|
||||
- **Refused credentials.** A 401/403 (or Graph auth error) puts the target
|
||||
in `backfeed.denied` until restart; the admin Webmentions page says which
|
||||
in `backfeed.denied` until restart; the admin Log page says which
|
||||
scopes are needed, and a `problems` notification is sent. Crossposting
|
||||
is unaffected.
|
||||
|
||||
|
||||
@@ -48,6 +48,8 @@ vulnerabilities (`pip-audit`); check again before each release.
|
||||
| python-dotenv | `1.2.*` | Loading `.env` |
|
||||
| Pillow | `13.0.*` | Image validation, resizing and variants; converting response avatars to 96 px WebP (`avatars.py`) |
|
||||
| pillow-heif | `1.8.*` | HEIC/HEIF uploads (iPhone photos) |
|
||||
| mcp | `2.3.*` | MCP server for AI clients (`starpunk/mcp_server/`, ADR-075); brings Starlette, pydantic and anyio. Only the sidecar imports it |
|
||||
| uvicorn | `0.54.*` | ASGI server for the MCP sidecar |
|
||||
|
||||
`markupsafe` comes with Flask/Jinja2.
|
||||
|
||||
@@ -72,6 +74,7 @@ front-end framework or bundler, no separate search server.
|
||||
| IndieWeb | Micropub (create, update, delete, undelete, queries, media endpoint), IndieAuth (authorization server with PKCE, metadata endpoint, token revocation), Webmention (send and receive), WebSub (publisher and built-in hub), microformats2 (h-entry, h-feed, h-card markup and parsing), rel=me, web actions (`<indie-action>`), backfeed of responses from the silos |
|
||||
| Login | OpenID Connect authorization code flow with PKCE (provider: Pocket ID) |
|
||||
| Feeds | RSS 2.0, Atom 1.0, JSON Feed 1.1, OPML 2.0, HTTP content negotiation |
|
||||
| AI clients | Model Context Protocol (Streamable HTTP, stateless), OAuth 2.0 Protected Resource Metadata (RFC 9728), Resource Indicators (RFC 8707), Client ID Metadata Documents |
|
||||
| Crossposting | Mastodon client API (GoToSocial), AT Protocol (Bluesky XRPC), Threads Graph API; the same APIs are read for backfeed |
|
||||
|
||||
Protocol details: [indieweb-protocols.md](indieweb-protocols.md),
|
||||
|
||||
@@ -6,6 +6,8 @@ Accepted
|
||||
|
||||
> Its deferral of WebSub is superseded by ADR-071 (v2.10.0), and "no feed caching" by the feed cache added in v1.1.2. RSS is now one of three feed formats (ADR-038) and is served at `/feed.rss` (`/feed.xml` still works). (status updated 2026-11-03)
|
||||
|
||||
> Item titles: since v2.16.0 only articles (notes with a title) have one; notes have none, and no first-line or date title is made up (ADR-076).
|
||||
|
||||
## Context
|
||||
|
||||
Phase 5 requires implementing RSS feed generation for syndicating published notes. We need to decide on the implementation approach, feed format, caching strategy, and technical details for generating a standards-compliant RSS feed.
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
## Status
|
||||
Accepted (implemented in v1.3.0, reviewed in v2.8.0; was: Proposed)
|
||||
|
||||
> Since v2.16.0 `p-name` is the stored title and `p-summary` the author's summary or, in previews, the start of the text (ADR-076).
|
||||
|
||||
## Context
|
||||
StarPunk currently implements basic microformats2 markup:
|
||||
- h-entry on note articles
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-062: Timestamp-Based Slug Format
|
||||
|
||||
## Status
|
||||
Accepted (Supersedes ADR-007)
|
||||
Accepted (Supersedes ADR-007). Articles take their slug from the title since v2.17.0 ([ADR-077](ADR-077-article-urls-and-title-slugs.md)).
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
# ADR-075: MCP server for AI clients
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.15.0, 2026-11-11)
|
||||
|
||||
## Context
|
||||
|
||||
The owner wants AI assistants (claude.ai custom connectors, Claude Code and
|
||||
other [Model Context Protocol](https://modelcontextprotocol.io/) clients) to
|
||||
read, search and write notes. Micropub already covers writing, but MCP
|
||||
clients don't speak Micropub: they discover tools over MCP and get tokens
|
||||
through OAuth 2.1 with protected resource metadata (RFC 9728), resource
|
||||
indicators (RFC 8707), and either dynamic client registration or a Client ID
|
||||
Metadata Document (CIMD).
|
||||
|
||||
StarPunk already runs an OAuth/IndieAuth authorization server (ADR-064) with
|
||||
PKCE, public clients only, and clients identified by a `client_id` URL whose
|
||||
JSON document lists `redirect_uris`. That is what MCP calls CIMD. Claude uses
|
||||
CIMD when the server's metadata says `client_id_metadata_document_supported:
|
||||
true` and lists `none` in `token_endpoint_auth_methods_supported`.
|
||||
|
||||
StarPunk is Flask on gunicorn sync workers (WSGI). The official MCP Python
|
||||
SDK is ASGI.
|
||||
|
||||
Options considered:
|
||||
|
||||
1. **Local stdio bridge** that calls `/micropub` with a stored token, like
|
||||
existing Micropub MCP servers. No server changes, but every client needs
|
||||
the bridge installed, claude.ai and mobile can't use it, and reading is
|
||||
limited to what Micropub queries expose.
|
||||
2. **Hand-written MCP route in Flask**, stateless JSON only. No new
|
||||
dependencies or processes, but StarPunk would own the protocol details
|
||||
(version negotiation, JSON-RPC, Origin checks, auth metadata) and track
|
||||
the spec itself.
|
||||
3. **Official SDK in a sidecar process** from the same image, behind the
|
||||
same reverse proxy, sharing the data volume.
|
||||
4. **Dynamic client registration** (RFC 7591) in the authorization server.
|
||||
Every connection would register a new client, and nothing needs it once
|
||||
CIMD works.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. A sidecar running the official SDK (option 3)
|
||||
|
||||
`starpunk/mcp_server/` builds an `mcp.server.MCPServer` (SDK 2.3) and serves
|
||||
it with uvicorn (`uvicorn starpunk.mcp_server.app:app --port 8001`), as a
|
||||
second container from the same image. The reverse proxy sends `/mcp*` and
|
||||
`/.well-known/oauth-protected-resource*` to it and everything else to
|
||||
gunicorn. The endpoint is `SITE_URL/mcp`.
|
||||
|
||||
The transport is stateless Streamable HTTP with plain JSON responses. No
|
||||
session lives in the process, so restarts and several workers are harmless.
|
||||
|
||||
The sidecar builds the Flask app with `BACKGROUND_WORKERS_ENABLED=false`, so
|
||||
only the main container runs the webmention, WebSub and syndication
|
||||
workers. Each tool runs in a worker thread inside a Flask request context
|
||||
for `SITE_URL`, and calls the same service functions as the web app.
|
||||
SQLite (WAL) handles the second process like another gunicorn worker.
|
||||
|
||||
### 2. StarPunk is the authorization server; the sidecar only checks tokens
|
||||
|
||||
- The authorization server metadata adds
|
||||
`client_id_metadata_document_supported: true` and
|
||||
`token_endpoint_auth_methods_supported: ["none"]`, and `scopes_supported`
|
||||
now includes `draft`.
|
||||
- An authorization request may carry `resource` (RFC 8707). The only resource
|
||||
StarPunk names is `SITE_URL/mcp`; anything else is `invalid_target`. The
|
||||
resource is stored on the code and the token (migration 022), shown on the
|
||||
consent screen, and must match if the token request repeats it.
|
||||
- An MCP request with no `scope` is offered `create update delete media
|
||||
draft`. The owner unticks what they don't want on the consent screen.
|
||||
(The SDK publishes the resource's `scopes_supported` from its
|
||||
required scopes, which StarPunk leaves empty so any token can connect.)
|
||||
- The sidecar's `TokenVerifier` looks tokens up with `verify_access_token`
|
||||
and reports their resource. With `validate_token_resource`, the SDK
|
||||
refuses tokens not issued for `/mcp`, so a Micropub app's token can't be
|
||||
replayed against MCP. MCP tokens still work at `/micropub`; both stay the
|
||||
owner's own clients, and the Connected Apps page marks MCP tokens.
|
||||
- DNS rebinding protection allows only the `SITE_URL` host and origin
|
||||
(plus localhost in `DEV_MODE`).
|
||||
|
||||
### 3. Tools: reading and writing notes, nothing else
|
||||
|
||||
| Tool | Scope |
|
||||
|---|---|
|
||||
| `search_notes`, `list_notes`, `get_note`, `get_site_config` | any token; drafts need `create`, `update`, `delete` or `draft` |
|
||||
| `list_tags` | `create`, `update`, `delete` or `draft` (as Micropub `q=category`) |
|
||||
| `create_note` | `create` or `draft` (draft-only tokens always make drafts) |
|
||||
| `update_note` | `update` |
|
||||
| `delete_note`, `undelete_note` | `delete` |
|
||||
| `upload_media` | `media` or `create` |
|
||||
|
||||
Writing tools build a Micropub JSON request and call `handle_create`,
|
||||
`handle_update` and `handle_delete`. Scope rules, validation, crossposting,
|
||||
webmentions and WebSub notifications are therefore identical to Micropub.
|
||||
`create_note` makes a draft unless told otherwise. Tool annotations mark
|
||||
reading tools read-only and update and delete as destructive.
|
||||
|
||||
Syndication management, webmention moderation and other admin functions are
|
||||
not exposed.
|
||||
|
||||
## Consequences
|
||||
|
||||
- AI clients connect with just the URL: claude.ai via Settings → Connectors,
|
||||
Claude Code via `claude mcp add --transport http starpunk SITE_URL/mcp`.
|
||||
The owner approves them on the usual consent screen, and they appear (and
|
||||
can be revoked) under Connected Apps.
|
||||
- New runtime dependencies: `mcp` (with Starlette, pydantic and anyio) and
|
||||
`uvicorn`. They are only imported by the sidecar.
|
||||
- Deployments run a second container and need two proxy routes. A site that
|
||||
doesn't run the sidecar just has no MCP; nothing else changes.
|
||||
- There are still no refresh tokens. MCP clients re-authorize after
|
||||
`INDIEAUTH_TOKEN_LIFETIME_DAYS`.
|
||||
- SDK upgrades need care: 3.0 is expected to change defaults (for example
|
||||
`validate_token_resource`, which StarPunk already sets).
|
||||
|
||||
## Alternatives
|
||||
|
||||
Options 1, 2 and 4 above. A stdio bridge could still be added later for
|
||||
clients that can't do OAuth; it would use a token from this flow.
|
||||
@@ -0,0 +1,74 @@
|
||||
# ADR-076: Article titles and summaries stored apart from the text
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.16.0, 2026-11-11)
|
||||
|
||||
Article URLs (`/article/<slug>`) and title slugs: [ADR-077](ADR-077-article-urls-and-title-slugs.md) (v2.17.0).
|
||||
|
||||
Supersedes the v2.3.3 design ([micropub-title-and-source-url](../design/v2.3.3/2026-10-31-micropub-title-and-source-url.md))
|
||||
and Q22's "a note has a title when its content starts with `#`". Amends
|
||||
ADR-014 (feed item titles) and ADR-040 (`p-name` and `p-summary`).
|
||||
|
||||
## Context
|
||||
|
||||
An IndieWeb article is an h-entry with a `name` that isn't just the start of
|
||||
its content (W3C Post Type Discovery). Since v1.2.0, StarPunk marked an article
|
||||
by a `# Heading` first line in the Markdown, and v2.3.3 stored a Micropub `name`
|
||||
as that heading. A review against IndieWeb practice found four gaps:
|
||||
|
||||
1. Feeds repeated an article's title: the item title, then the same heading
|
||||
rendered at the top of the content.
|
||||
2. Every feed item had a title, made from a note's first line (or the date),
|
||||
so feed readers showed notes as headlines and couldn't tell them from
|
||||
articles.
|
||||
3. Micropub `summary` was used only when `content` was missing; sent
|
||||
together with content, it was dropped.
|
||||
4. Any note that happened to start with a heading became an article, and its
|
||||
`q=source` changed from what the client sent. Micropub and mf2 treat `name`
|
||||
and `content` as separate properties.
|
||||
|
||||
## Decision
|
||||
|
||||
- **Columns.** `notes.title` and `notes.summary` (migration 023). A note with
|
||||
a title is an article; its Markdown file holds only the body. A heading in
|
||||
the body is just a heading.
|
||||
- **Moving existing titles.** Migration 023 has a Python data step
|
||||
(`migrations.DATA_MIGRATIONS`) that runs once, in the migration's
|
||||
transaction: for each note whose file starts with a heading (`split_title`,
|
||||
trashed files included), it moves the heading into `title`, rewrites the file
|
||||
and updates `content_hash`, then empties `notes_fts` so startup rebuilds it.
|
||||
If anything fails, the changed files are written back and the transaction is
|
||||
rolled back, so 023 is retried on the next start. Tying it to the migration
|
||||
matters: run on every start it would turn new notes' headings into titles.
|
||||
- **Micropub.** `name` → `title`; `summary` → `summary` when `content` is also
|
||||
sent. A summary on its own is still stored as the content, which is how Post
|
||||
Type Discovery reads it. `summary` can be replaced and deleted; `q=source`
|
||||
returns `name` and `summary` when set. If the content starts with a heading
|
||||
equal to `name`, that heading is dropped so it isn't shown twice.
|
||||
- **Feeds.** Only articles have a title: RSS leaves `<title>` out (an item
|
||||
needs a title or a description, and every item has a description), JSON Feed
|
||||
leaves `title` out, and Atom, which requires `<title>`, has an empty one.
|
||||
Atom `<summary>` and JSON Feed `summary` carry the author's summary. Content
|
||||
no longer contains the title, so it isn't repeated.
|
||||
- **Pages.** The article page has `p-name`, `p-summary` (when set) and the body
|
||||
as `e-content`. On the homepage and tag pages an article shows its title,
|
||||
summary (or the start of its text) and "Read more", without images. The
|
||||
photos stay in the markup as hidden `u-photo` values so parsers read the same
|
||||
post type there as on the article page.
|
||||
- **Crossposting.** Articles go to GoToSocial, Bluesky and Threads as title,
|
||||
summary (or the start of the text), link and hashtags, without images, the
|
||||
usual POSSE form for articles. On Bluesky the link card is the article
|
||||
itself. Notes are posted in full as before.
|
||||
- **Editor.** Title and Summary fields; a titled note may have no body.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The migration rewrites note files. Back up the database *and* `data/notes`
|
||||
(and `.trash`) before deploying.
|
||||
- `Note.title` is `None` for notes. Places that need a name for any post use
|
||||
`Note.label` (title, else excerpt, else slug): notifications and the
|
||||
Bluesky card. `Note.has_title` and `Note.body_html` are gone.
|
||||
- Search indexes the title column; untitled notes are still listed by their
|
||||
first line.
|
||||
- A crossposted article that is edited is re-posted as a teaser.
|
||||
@@ -0,0 +1,46 @@
|
||||
# ADR-077: Article URLs and slugs from titles
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (v2.17.0, 2026-11-12)
|
||||
|
||||
Amends ADR-062 (default slugs) for articles and ADR-076 (articles).
|
||||
|
||||
## Context
|
||||
|
||||
ADR-076 (v2.16.0) made a note with a title an article, but articles still got
|
||||
a `/note/<timestamp>` URL like any other note. An article is not a note, and
|
||||
its URL should say so. ADR-062 chose timestamp slugs because short notes have
|
||||
no good words for a URL and their text may be private; an article has a title
|
||||
written to be read, and that is the usual source of its slug on the IndieWeb.
|
||||
|
||||
## Decision
|
||||
|
||||
- **URLs.** Articles live at `/article/<slug>`, other notes at `/note/<slug>`.
|
||||
`Note.permalink`, `Note.url`, `Note.endpoint` and `utils.post_path` /
|
||||
`post_url` build them; nothing else writes `/note/` by hand. Slugs stay
|
||||
unique across both, so a slug still names one post.
|
||||
- **One URL each, the other redirects.** `/note/<slug>` for an article and
|
||||
`/article/<slug>` for a note answer 301 to the right one. Adding or removing
|
||||
a title changes the URL, and links made before (webmentions sent, crossposts,
|
||||
bookmarks) keep working. A deleted post is 410 at either.
|
||||
- **Slug from the title.** When a post is created with a title and no
|
||||
`mp-slug`, the slug is the title made URL-safe (`sanitize_slug`: accents
|
||||
folded, lowercase, hyphens), cut at a word boundary to 80 characters, `-2`,
|
||||
`-3`… if taken, `-article` if reserved. A title with nothing usable (only
|
||||
emoji, say) falls back to the timestamp slug. Notes keep timestamp slugs.
|
||||
- **Slugs don't follow later title changes.** Editing an article's title, or
|
||||
making a note an article, keeps its slug: a slug is the stable part of the
|
||||
URL.
|
||||
- **Micropub.** `Location` and `q=source` give the new URLs. Updates accept
|
||||
either URL. An update that adds or removes a title answers 201 with the new
|
||||
URL in `Location`, as the Micropub spec requires when a post's URL changes;
|
||||
other updates are still 204.
|
||||
- **Incoming webmentions** to either URL find the post.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Production had no articles when this shipped, so no published URL changed.
|
||||
- A note given a title later keeps its timestamp slug under `/article/`.
|
||||
- Stored URLs (webmention sources, crosspost records) are not rewritten; the
|
||||
redirects cover them.
|
||||
+11
-8
@@ -5,8 +5,8 @@ records one decision: its context, what was decided, and the consequences.
|
||||
Superseded ADRs are kept as history; their Status line names the ADR that
|
||||
replaced them.
|
||||
|
||||
**Total ADR files**: 70 (69 numbers: ADR-001 to ADR-044 and ADR-050 to
|
||||
ADR-074, with two files numbered ADR-012). See [Numbering](#numbering).
|
||||
**Total ADR files**: 73 (72 numbers: ADR-001 to ADR-044 and ADR-050 to
|
||||
ADR-077, with two files numbered ADR-012). See [Numbering](#numbering).
|
||||
|
||||
Status values: **Accepted** (in force), **Superseded** (replaced by a later
|
||||
ADR), **Deprecated** (withdrawn, nothing replaces it; none at present),
|
||||
@@ -48,7 +48,7 @@ See also [database architecture](../architecture/database.md).
|
||||
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [062](ADR-062-timestamp-based-slug-format.md) | Timestamp-Based Slug Format | Accepted | Default slugs are `YYYYMMDDHHMMSS` timestamps (v1.5.0); replaces ADR-007. |
|
||||
| [062](ADR-062-timestamp-based-slug-format.md) | Timestamp-Based Slug Format | Accepted* | Default slugs are `YYYYMMDDHHMMSS` timestamps (v1.5.0); replaces ADR-007. Articles take theirs from the title (ADR-077). |
|
||||
| [035](ADR-035-custom-slugs.md) | Custom Slugs in Micropub | Accepted* | Honour `mp-slug`, sanitized, with reserved names protected (v1.1.0). |
|
||||
| [034](ADR-034-full-text-search.md) | Full-Text Search with SQLite FTS5 | Accepted | FTS5 search over note content (v1.1.0). |
|
||||
| [057](ADR-057-media-attachment-model.md) | Media Attachment Model | Accepted | Social-media-style image attachments: up to 4 per note, ordered, with captions. |
|
||||
@@ -58,7 +58,7 @@ See also [database architecture](../architecture/database.md).
|
||||
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [014](ADR-014-rss-feed-implementation.md) | RSS Feed Implementation Strategy | Accepted* | RSS 2.0 feed; its WebSub deferral is superseded by ADR-071. |
|
||||
| [014](ADR-014-rss-feed-implementation.md) | RSS Feed Implementation Strategy | Accepted* | RSS 2.0 feed; its WebSub deferral is superseded by ADR-071, and item titles by ADR-076. |
|
||||
| [038](ADR-038-syndication-formats.md) | Multiple Syndication Format Support | Accepted | RSS, Atom and JSON Feed side by side (v1.1.2). |
|
||||
| [071](ADR-071-websub.md) | WebSub publishing with a built-in hub | Accepted | Built-in WebSub hub at `/websub` for StarPunk's own feeds (v2.10.0, amended v2.10.1). |
|
||||
| [059](ADR-059-full-feed-media-standardization.md) | Full Feed Media Standardization (Option 3) | Proposed | Full Media RSS, podcasts, video and thumbnails in feeds; not scheduled. |
|
||||
@@ -67,7 +67,9 @@ See also [database architecture](../architecture/database.md).
|
||||
|
||||
| ADR | Title | Status | Summary |
|
||||
|---|---|---|---|
|
||||
| [040](ADR-040-microformats2-compliance.md) | Strict Microformats2 Compliance | Accepted | Full h-entry, h-card and h-feed markup (v1.3.0, reviewed v2.8.0). |
|
||||
| [040](ADR-040-microformats2-compliance.md) | Strict Microformats2 Compliance | Accepted* | Full h-entry, h-card and h-feed markup (v1.3.0, reviewed v2.8.0); `p-name`/`p-summary` per ADR-076. |
|
||||
| [076](ADR-076-note-title-and-summary-columns.md) | Article titles and summaries stored apart from the text | Accepted | `notes.title`/`summary` columns; feeds title only articles; article previews and crossposts are title + summary (v2.16.0). |
|
||||
| [077](ADR-077-article-urls-and-title-slugs.md) | Article URLs and slugs from titles | Accepted | Articles at `/article/<slug>` with slugs from their titles; the other prefix redirects (v2.17.0). |
|
||||
| [061](ADR-061-author-discovery.md) | Author Profile Discovery from IndieAuth | Accepted | The author h-card is read from `ADMIN_ME` and cached. |
|
||||
| [063](ADR-063-webmention-implementation.md) | Webmention Implementation | Accepted | Async receiving (202) and sending via a background worker, SSRF guard, rate limits, moderation (v1.6.0, revised up to v2.10.1). *Reconstructed.* |
|
||||
| [028](ADR-028-micropub-implementation.md) | Micropub Implementation Strategy | Accepted* | `/micropub` endpoint for creating posts (v1.0.0-rc.1); tokens now per ADR-064. |
|
||||
@@ -95,6 +97,7 @@ See also [IndieWeb protocols](../architecture/indieweb-protocols.md).
|
||||
| [010](ADR-010-authentication-module-design.md) | Authentication Module Design | Accepted* | Single `auth.py` module and server-side sessions; the login flow is now ADR-064. |
|
||||
| [011](ADR-011-development-authentication-mechanism.md) | Development Authentication Mechanism | Accepted | `DEV_MODE` login bypass for local development only. |
|
||||
| [022](ADR-022-auth-route-prefix-fix.md) | Fix IndieAuth Callback Route Mismatch | Accepted | Auth routes live under `/auth` (v0.9.2). |
|
||||
| [075](ADR-075-mcp-server.md) | MCP server for AI clients | Accepted | `/mcp` served by the official MCP SDK in a sidecar; StarPunk's IndieAuth server issues resource-bound tokens via client ID metadata documents (v2.15.0). |
|
||||
|
||||
### Operations, errors and monitoring
|
||||
|
||||
@@ -169,7 +172,7 @@ was carried out and is not itself superseded.
|
||||
|
||||
## How to create a new ADR
|
||||
|
||||
1. Use the next number after the highest existing one (currently ADR-075).
|
||||
1. Use the next number after the highest existing one (currently ADR-077).
|
||||
2. Name it `ADR-NNN-brief-descriptive-title.md`.
|
||||
3. Follow the format of recent ADRs (e.g. ADR-070, ADR-071):
|
||||
|
||||
@@ -202,5 +205,5 @@ was carried out and is not itself superseded.
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2026-11-10
|
||||
**Total ADR files**: 70
|
||||
**Last Updated**: 2026-11-11
|
||||
**Total ADR files**: 72
|
||||
@@ -21,7 +21,7 @@ is in `../architecture/`; decisions are in `../decisions/` (ADRs).
|
||||
|
||||
## Contents
|
||||
|
||||
[v0.x](#v0x) · [v1.0.0](#v100) · [v1.1.1](#v111) · [v1.1.2](#v112) · [v1.2.0](#v120) · [v1.3.0](#v130) · [v1.3.1](#v131) · [v1.4.0](#v140) · [v1.4.1](#v141) · [v1.4.2](#v142) · [v1.5.0](#v150) · [v1.6.0](#v160) · [v2.0.0](#v200) · [v2.0.1](#v201) · [v2.0.2](#v202) · [v2.1.0](#v210) · [v2.2.0](#v220) · [v2.3.0](#v230) · [v2.3.1](#v231) · [v2.3.2](#v232) · [v2.3.3](#v233) · [v2.4.0](#v240) · [v2.5.0](#v250) · [v2.6.0](#v260) · [v2.6.1](#v261) · [v2.6.2](#v262) · [v2.6.3](#v263) · [v2.6.4](#v264) · [v2.6.5](#v265) · [v2.6.6](#v266) · [v2.6.7](#v267) · [v2.6.8](#v268) · [v2.6.9](#v269) · [v2.7.0](#v270) · [v2.7.1](#v271) · [v2.7.2](#v272) · [v2.8.0](#v280) · [v2.8.1](#v281) · [v2.9.0](#v290) · [v2.10.0](#v2100) · [v2.10.1](#v2101) · [v2.10.2](#v2102) · [v2.11.0](#v2110) · [v2.12.0](#v2120) · [v2.12.1](#v2121) · [v2.13.0](#v2130) · [v2.13.1](#v2131) · [v2.13.2](#v2132) · [v2.13.3](#v2133) · [v2.13.4](#v2134) · [v2.14.0](#v2140) · [v2.14.1](#v2141) · [hotfix](#hotfix)
|
||||
[v0.x](#v0x) · [v1.0.0](#v100) · [v1.1.1](#v111) · [v1.1.2](#v112) · [v1.2.0](#v120) · [v1.3.0](#v130) · [v1.3.1](#v131) · [v1.4.0](#v140) · [v1.4.1](#v141) · [v1.4.2](#v142) · [v1.5.0](#v150) · [v1.6.0](#v160) · [v2.0.0](#v200) · [v2.0.1](#v201) · [v2.0.2](#v202) · [v2.1.0](#v210) · [v2.2.0](#v220) · [v2.3.0](#v230) · [v2.3.1](#v231) · [v2.3.2](#v232) · [v2.3.3](#v233) · [v2.4.0](#v240) · [v2.5.0](#v250) · [v2.6.0](#v260) · [v2.6.1](#v261) · [v2.6.2](#v262) · [v2.6.3](#v263) · [v2.6.4](#v264) · [v2.6.5](#v265) · [v2.6.6](#v266) · [v2.6.7](#v267) · [v2.6.8](#v268) · [v2.6.9](#v269) · [v2.7.0](#v270) · [v2.7.1](#v271) · [v2.7.2](#v272) · [v2.8.0](#v280) · [v2.8.1](#v281) · [v2.9.0](#v290) · [v2.10.0](#v2100) · [v2.10.1](#v2101) · [v2.10.2](#v2102) · [v2.11.0](#v2110) · [v2.12.0](#v2120) · [v2.12.1](#v2121) · [v2.13.0](#v2130) · [v2.13.1](#v2131) · [v2.13.2](#v2132) · [v2.13.3](#v2133) · [v2.13.4](#v2134) · [v2.14.0](#v2140) · [v2.14.1](#v2141) · [v2.14.2](#v2142) · [hotfix](#hotfix)
|
||||
|
||||
## v0.x
|
||||
|
||||
@@ -456,6 +456,10 @@ OIDC admin login and built-in IndieAuth server.
|
||||
|
||||
- [2026-11-10-tags-and-captions.md](v2.14.1/2026-11-10-tags-and-captions.md) — Clearing tags in the editor; captions escaped in feeds
|
||||
|
||||
## v2.14.2
|
||||
|
||||
- [2026-11-10-quotes-cleanup.md](v2.14.2/2026-11-10-quotes-cleanup.md) — Quote counts; dead files removed
|
||||
|
||||
## hotfix
|
||||
|
||||
Not a release folder.
|
||||
|
||||
@@ -343,6 +343,8 @@ Priority order:
|
||||
3. First h-card found (fallback)
|
||||
|
||||
### p-name Only With Explicit Title
|
||||
> Superseded in v2.16.0 (ADR-076): the title is `notes.title`, not a `#` first line. p-name still appears only for titled notes.
|
||||
|
||||
**Per Q22**:
|
||||
- Detected by checking if content starts with `#`
|
||||
- Matches note model's title extraction logic
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
# v2.14.2: Quote counts; dead files removed
|
||||
|
||||
The two remaining items from the v2.14.0 documentation review.
|
||||
|
||||
## Quote counts
|
||||
|
||||
Backfeed stored anonymous quote counts (`syndications.quotes_count`: all
|
||||
Threads quotes, and quotes by people who opted out elsewhere) that nothing
|
||||
displayed. The counts line (`partials/responses.html` `reactions`) now has a
|
||||
quote count after replies: named quotes plus those counts. Its card shows
|
||||
only the anonymous part ("2 on Threads"); named quotes stay listed under
|
||||
Responses. Quotes have no web action (there is no standard verb).
|
||||
`webmention_queries.response_counts` adds `quotes`, so lists of notes show
|
||||
them too.
|
||||
|
||||
## Removed
|
||||
|
||||
- `static/js/preview.js` (empty since the first commit) and the jsDelivr
|
||||
`marked` script the editor pages loaded with it.
|
||||
- `templates/feed.xml` (empty, never rendered) and `static/indielogin-test.html`
|
||||
(a v1 IndieLogin test page).
|
||||
- `syndication.get_targets()` read `THREADS_API` from the config, which
|
||||
`config.py` never loads; the client now always uses the `THREADS_API`
|
||||
constant.
|
||||
|
||||
## Verification
|
||||
|
||||
`tests/test_backfeed.py::test_quote_counts_include_anonymous_ones`; backfeed,
|
||||
syndication, admin, template, public-route, microformats, notification,
|
||||
webmention and feed tests pass. Rendered the counts line with the quote card
|
||||
open in headless Chromium.
|
||||
@@ -0,0 +1,42 @@
|
||||
# v2.16.0: Article titles and summaries
|
||||
|
||||
Implements ADR-076. Prompted by a review of how StarPunk handles IndieWeb
|
||||
articles: titles were repeated in feeds, every feed item had a made-up title,
|
||||
Micropub `summary` was dropped, and any note starting with a heading became an
|
||||
article.
|
||||
|
||||
## What changed
|
||||
|
||||
| Area | Change |
|
||||
|---|---|
|
||||
| Storage | Migration 023 adds `notes.title` and `notes.summary`; `create_note`/`update_note` take `title` and `summary` (`UNCHANGED` sentinel on update; `None`/"" clears) |
|
||||
| Migration data step | `migrations.DATA_MIGRATIONS["023_note_titles.sql"] = _move_note_titles`; `run_migrations(..., data_dir=)` gets `DATA_PATH` from `init_db`. Moves `# Heading` first lines (via `split_title`) into `title`, finds trashed files with `utils.find_trashed_file` (shared with `undelete_note`), updates `content_hash`, empties `notes_fts`. Files are restored if the migration fails |
|
||||
| Model | `Note.title`/`summary` fields; `Note.label` for places that need some name; `has_title`, `body_html` and the first-line `title` property removed |
|
||||
| Micropub | `name` → title, `summary` → summary (only with content), `summary` updatable, `q=source` returns both; a heading repeating `name` is dropped (`drop_title_heading`) |
|
||||
| Feeds | Titles only for articles (RSS none, JSON none, Atom empty); Atom/JSON summaries |
|
||||
| Templates | Article page: `p-name`, `p-summary`, `e-content` without the title. Previews: title, summary or excerpt, "Read more", no images (hidden `u-photo` data kept for Post Type Discovery) |
|
||||
| Crossposting | `compose_text` posts articles as title + `article_teaser` (summary, or 200 characters of plain text) + link + hashtags; no images for articles; Bluesky card is the article |
|
||||
| Editor | Title and Summary fields on new/edit; content not required when titled |
|
||||
| Search | FTS indexes `title`; untitled notes still use their first line |
|
||||
| MCP | `create_note`/`update_note` take `summary`; note summaries include it |
|
||||
|
||||
## Fixes found along the way
|
||||
|
||||
- **Hashtag lines.** Python-Markdown made `#indieweb is fun` an `<h1>`; CommonMark needs a space after `#`. `markdown_ext.HashtagLineExtension` escapes the first `#` of a line where `#`s run straight into text, after fenced code and raw HTML are set aside. It is used for pages and feeds (`MARKDOWN_EXTENSIONS`) and crosspost plain text (`syndication._note_soup`), so the hashtag also reaches Bluesky as a tag. Search's first-line label keeps hashtags too.
|
||||
- **Plain-text excerpt.** `Note.excerpt` only stripped `#` marks, so previews showed `**bold**` and `[link](url)`. It is now `syndication.plain_text` of the content, cut to 200 characters with `…`; `article_teaser` is the summary or this excerpt.
|
||||
|
||||
- **Inline images in Micropub articles.** (1) `extract_content` prefers `html` over `text` and marks it `content_format = 'html'`: clients send `text` as a plain fallback, so images, links and headings were lost. (2) `localize_inline_images` copies images from other sites (Markdown `![]()` or `<img>`) into the media library through the SSRF-checked fetcher used for `photo`, on create and on content replace, and rewrites their URLs; failures keep the original URL, at most 20 per post. Images stay inline (not `u-photo`), so the post remains an article. (3) `Note.html` sanitizes HTML content with nh3 directly instead of passing it through Markdown.
|
||||
|
||||
## Choices made with the owner
|
||||
|
||||
- Title column rather than an editor-only field, as the most standards-compliant option.
|
||||
- Atom entries for notes keep an empty `<title>` (required by RFC 4287).
|
||||
- Homepage and tag pages: articles show title and summary without images.
|
||||
- Crossposts: articles as title + summary + link, no images.
|
||||
|
||||
## Tests
|
||||
|
||||
- `tests/test_articles.py` (new): feeds per format, article page mf2, homepage and tag-page previews, admin create/edit/clear, title-only article.
|
||||
- `tests/test_migration_023.py` (new): titles moved (including trashed notes), hashtag first line untouched, hashes and FTS, runs once, files restored on failure, schema-only without `data_dir`.
|
||||
- Updated for the new behaviour: `test_feed.py`, `test_feeds_{atom,json,rss}.py`, `test_routes_feed.py`, `test_models.py`, `test_microformats.py`, `test_microformats_v280.py`, `test_micropub.py`, `test_micropub_v240.py` (plus new summary and empty-body tests), `test_syndication.py` and `test_syndication_bluesky.py` (article crossposts).
|
||||
- Locally: the 35 affected test files, 872+ tests, pass. CI runs the full suite on the tag.
|
||||
@@ -0,0 +1,29 @@
|
||||
# v2.17.0: Article URLs and title slugs
|
||||
|
||||
Implements ADR-077. Follow-up to v2.16.0 (ADR-076): articles had `/note/`
|
||||
URLs and timestamp slugs.
|
||||
|
||||
## What changed
|
||||
|
||||
| Area | Change |
|
||||
|---|---|
|
||||
| URL helpers | `utils.post_path(slug, title)` and `post_url(slug, title)`; `Note.permalink` uses them, `Note.endpoint` (`public.article`/`public.note`) and `Note.url` (absolute) are new |
|
||||
| Routes | `public.article` at `/article/<slug>`; `note` and `article` share `_permalink(slug, endpoint)`, which 301s to the other route when the post's type doesn't match. `_note_was_deleted` checks both URLs for hard-deleted posts |
|
||||
| Slugs | `slug_utils.generate_title_slug(title, existing)`; `create_note` uses it for titled posts without `custom_slug`, else the timestamp slug |
|
||||
| Micropub | `Location`, `q=source` url via `note.url`; `_note_from_url` accepts `/article/`; update answers 201 + `Location` when a title is added or removed |
|
||||
| Other URL builders | Templates (`note.url`), webmention sending (`post_url`/`note.url`), crosspost permalink, backfeed target, notification links, search results (`article_title` from `notes.title`), MCP tools |
|
||||
| Webmention receiving | `resolve_target_note` accepts `/note/` and `/article/` |
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/test_articles.py::TestArticleUrls`: Micropub URL and slug, timestamp
|
||||
slugs for notes, redirects both ways, updates via the old URL, slug rules
|
||||
(accents, duplicates, reserved, `mp-slug`, emoji, long titles), 201 on
|
||||
adding/removing a title, 410 for a deleted article, webmention targets and
|
||||
homepage links, search result URLs.
|
||||
|
||||
## v2.17.1
|
||||
|
||||
`post_url` first used `url_for`, which fails outside a request (the
|
||||
crosspost and backfeed workers); CI caught it in `test_likes_reposts.py`
|
||||
before 2.17.0 was built. It now joins `SITE_URL` and `post_path`.
|
||||
@@ -0,0 +1,9 @@
|
||||
# v2.18.1: MCP create_note describes article URLs and slugs
|
||||
|
||||
Documentation-only follow-up to ADR-077 (v2.17.x). The MCP `create_note`
|
||||
tool description now says that a title makes the post an article at
|
||||
`/article/<slug>`, that articles get their slug from the title and other
|
||||
notes from the time, and that `slug` is only needed for a specific slug.
|
||||
AI clients read tool descriptions to decide which arguments to send, so
|
||||
this keeps them from making up a slug the title would already give.
|
||||
No behaviour change.
|
||||
@@ -0,0 +1,34 @@
|
||||
# v2.18.3: Don't send webmentions to this site or skipped hosts
|
||||
|
||||
## Why
|
||||
|
||||
The admin Log showed two sends that could never succeed, both "timed out":
|
||||
|
||||
- a note linking to `https://starpunk.thesatelliteoflove.com/` (this site),
|
||||
- a note linking to a Gitea repository on `git.thesatelliteoflove.com`.
|
||||
|
||||
From inside the container both names resolve to the server's public address,
|
||||
which isn't reachable from there (no NAT hairpinning). Routing them to the
|
||||
internal proxy with `extra_hosts` wouldn't help: the SSRF guard blocks
|
||||
private addresses. And neither target wants the webmention: StarPunk only
|
||||
takes webmentions for its posts, and Gitea advertises no endpoint.
|
||||
|
||||
## What changed
|
||||
|
||||
| Area | Change |
|
||||
|---|---|
|
||||
| Config | `WEBMENTION_SKIP_HOSTS` (comma-separated, lowercased) |
|
||||
| Sender | `webmention_sender.skipped_hosts(config)` = `SITE_URL`'s host + the setting; `is_skipped(url, hosts)` matches the exact host, any port. `queue_webmentions_for_note` drops those links before the 10-link cap, and from re-sends on update |
|
||||
| Worker | `_send_webmention` marks a skipped target `no_endpoint` without fetching it (rows queued before the setting changed). At startup `_drop_skipped_targets` does the same for `pending` and `failed` rows, so they leave the Log |
|
||||
| Settings | A Webmentions section lists the skipped hosts |
|
||||
|
||||
`no_endpoint` with `last_error` = "Not sent: …" is used rather than a new
|
||||
status because `webmention_queue.status` has a CHECK constraint; changing it
|
||||
would mean rebuilding the table. The Log only lists `pending` and `failed`.
|
||||
|
||||
Exact host matching is deliberate: skipping `thesatelliteoflove.com` should
|
||||
not skip its subdomains, or the reverse.
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/test_webmention_skip_hosts.py`.
|
||||
@@ -0,0 +1,108 @@
|
||||
# v2.19.0: Reply contexts that show the post
|
||||
|
||||
## Why
|
||||
|
||||
A reply to `https://defcon.social/@deviantollam/117611186265622833` showed
|
||||
"In reply to: defcon.social" and nothing else. Reply contexts were read only
|
||||
from microformats. Mastodon's status page is a JavaScript app shell with no
|
||||
h-entry, so `fetch_reply_context` stored `fetch_status='fetched'` with no
|
||||
content, and `extract_author` fell back to the URL's host for the author name.
|
||||
GoToSocial pages have an h-card but no h-entry; Bluesky pages have neither.
|
||||
The home page didn't use the context at all: it printed the raw URL.
|
||||
|
||||
The data was available elsewhere:
|
||||
|
||||
- Mastodon's public API (`/api/v1/statuses/<id>`) returned the author, text,
|
||||
attachments (with alt text) and date without authentication.
|
||||
- GoToSocial's API needs a token, but this site's own GoToSocial instance can
|
||||
resolve any fediverse URL with signed requests
|
||||
(`/api/v2/search?resolve=true`). The crossposter already does this for
|
||||
native likes and boosts (`MastodonClient.find_status`).
|
||||
- `public.api.bsky.app` answers `app.bsky.feed.getPostThread` for
|
||||
`at://<handle>/app.bsky.feed.post/<rkey>` without logging in. (`getPosts`
|
||||
rejects handles in the URI; `getPostThread` resolves them.)
|
||||
- Most pages carry OpenGraph tags: `og:title`, `og:description`, `og:image`
|
||||
(with `og:image:alt`), `og:site_name`, a published time. Fediverse servers
|
||||
put "Name (@user@host)" in `og:title` and the handle in
|
||||
`profile:username`.
|
||||
|
||||
## Sources, in order
|
||||
|
||||
| # | When | Source | `source` |
|
||||
|---|---|---|---|
|
||||
| 1 | `bsky.app/profile/<actor>/post/<rkey>` | Bluesky AppView API | `bluesky` |
|
||||
| 2 | The page has an h-entry | Microformats (as before), image from `u-photo` or `og:image` | `mf2` |
|
||||
| 3 | Fediverse path (`/@user/<id>`, `/@user/statuses/<id>`, `/users/…/statuses/…`, `/notes/…`, `/notice/…`, `/objects/…`) or a `profile:username` tag | The origin's Mastodon API (numeric Mastodon ids only), else this site's GoToSocial instance | `fediverse` |
|
||||
| 4 | Any other HTML page | OpenGraph tags; author from the page's h-card or `<meta name="author">` | `opengraph` |
|
||||
|
||||
All requests other than the instance lookup go through `fetch_source`, so
|
||||
they get the SSRF checks, size limit and timeouts. The instance lookup uses
|
||||
the crossposting client (`MastodonClient.resolve_status`, which
|
||||
`find_status` now calls too) and is only tried when GoToSocial crossposting
|
||||
is configured.
|
||||
|
||||
Mastodon statuses keep their content warnings: when `spoiler_text` is set,
|
||||
the snippet is "CW: …" and a `sensitive` post's image is left out. Content
|
||||
HTML is sanitized with the webmention sanitizer (nh3; `p br strong em b i a
|
||||
code pre blockquote`, `a[href]`) and cut to about 500 characters. Plain text
|
||||
(Bluesky, OpenGraph) is escaped and split into paragraphs.
|
||||
|
||||
An `mf2` or `opengraph` title that is only the start of the content
|
||||
(an mf2 implied name, a description that repeats the title) is dropped.
|
||||
The URL's host is no longer used as the author's name.
|
||||
|
||||
## Storage (migration 024)
|
||||
|
||||
`reply_contexts` gains `author_handle`, `photo`, `photo_alt` and `source`.
|
||||
Error rows get `source='error'`.
|
||||
|
||||
A row with no `source` was stored before v2.19.0. `fetch_reply_context`
|
||||
fetches such rows again instead of returning them, and the webmention
|
||||
worker calls `refresh_stale()` on every pass, which fetches up to five rows
|
||||
that a note still uses. Each row is fetched once: success stores a source,
|
||||
and a failure keeps the old data and marks it `legacy`.
|
||||
|
||||
## Images
|
||||
|
||||
The CSP only allows images from this site (`img-src 'self' data:`), so
|
||||
remote avatars and images can't be shown directly. As with response avatars
|
||||
(ADR-073 Decision 6), they are saved locally:
|
||||
|
||||
- Author avatars go into the existing `DATA_PATH/avatars/` (96 px squares).
|
||||
- Post images go into `DATA_PATH/previews/`, at most 640 px wide, as WebP,
|
||||
served from `/previews/<sha256>.webp`. Downloads are capped at 8 MB.
|
||||
|
||||
The images are saved right after the fetch. `avatars.fill_missing` also
|
||||
covers reply contexts that notes use, so the worker retries what failed, and
|
||||
`remove_unused` keeps those files (before this release it would have deleted
|
||||
any avatar that wasn't a webmention's). Until there is a copy, the card shows
|
||||
the coloured initial and leaves the image out.
|
||||
|
||||
## Display
|
||||
|
||||
`templates/partials/reply_context.html`:
|
||||
|
||||
- `reply_card(url, ctx)` on the note page: "↩ In reply to", then a card with
|
||||
the avatar, name (linked to the author) and handle, the title if there is
|
||||
one, the content (sanitized HTML), the image with its alt text, and a footer
|
||||
linking to the original with its date and site. Without a fetched context
|
||||
it is the bare link, as before.
|
||||
- `reply_line(url, ctx)` on the home, tag and likes pages: "↩ In reply to
|
||||
<author>" (or the title, "by <author>"), then up to 140 characters of the
|
||||
post as a quote. Without a context it is the raw URL, as before.
|
||||
|
||||
`attach_contexts` now gives replies their context as well as likes and
|
||||
reposts, and the note page uses that when webmentions are disabled.
|
||||
|
||||
Microformats are unchanged for consumers: the card is still the note's
|
||||
`u-in-reply-to h-cite`. Its author h-card and `e-content` are nested inside
|
||||
the h-cite, so they don't become the note's properties. Post Type Discovery
|
||||
still reads the note as a reply (`tests/test_reply_context_sources.py`).
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/test_reply_context_sources.py`, through the mocked network: the
|
||||
Mastodon API, content warnings, resolving through the instance, the
|
||||
OpenGraph fallback for fediverse and other pages, Bluesky, the once-only
|
||||
refresh of old rows (including keeping data when the refresh fails), and the
|
||||
note and home pages (content, local image and avatar, mf2 reply).
|
||||
@@ -0,0 +1,78 @@
|
||||
# v2.20.0: Crossposted replies look like replies
|
||||
|
||||
## Why
|
||||
|
||||
Crossposting ignored `in_reply_to`. A reply to
|
||||
`https://defcon.social/@deviantollam/117611186265622833` went to GoToSocial
|
||||
and Bluesky as an ordinary post (text, permalink, hashtags), with nothing to
|
||||
say it answered anyone.
|
||||
|
||||
## Behaviour
|
||||
|
||||
For each target, in order:
|
||||
|
||||
| Replying to | GoToSocial | Bluesky | Threads |
|
||||
|---|---|---|---|
|
||||
| One of our notes (its page, or a crosspost of it on any network) | Native reply to that note's GoToSocial status | Native reply to its Bluesky post | Native reply (`reply_to_id`) to its Threads post |
|
||||
| A fediverse post | Native reply (`in_reply_to_id`, resolved by our instance) starting with `@acct` | Reply line | Reply line |
|
||||
| A Bluesky post | Reply line | Native reply (`reply.root`/`reply.parent`) | Reply line |
|
||||
| Anything else | Reply line | Reply line | Reply line |
|
||||
|
||||
The reply line is `↩ Replying to <who>`, where *who* is the reply
|
||||
context's author name, else its title, else the URL's host:
|
||||
|
||||
- GoToSocial and Threads: the line, then the URL, then a blank line and the note.
|
||||
On Threads the URL counts towards Meta's limit of five links, so the
|
||||
body gets one link fewer.
|
||||
- Bluesky: the line without the URL. *Who* is a link facet to the post,
|
||||
which saves up to ~100 of the 300 characters.
|
||||
|
||||
`compose_text(..., lead=...)` keeps the line whole and shortens the body
|
||||
instead when the post is too long.
|
||||
|
||||
### The @mention
|
||||
|
||||
The user chose to start a native fediverse reply with the author's
|
||||
mention, as Mastodon and GoToSocial clients do. Without it the author isn't
|
||||
notified, and their server may never receive the reply, so it wouldn't show
|
||||
in their thread. There's no mention when replying to our own account (the
|
||||
search result's account id equals `verify_credentials`), or when the note
|
||||
already contains the mention.
|
||||
|
||||
### Our own posts
|
||||
|
||||
`_own_note_id(url)` maps a URL to a note: a synced `syndications.remote_url`
|
||||
on any target, or `/note/<slug>` and `/article/<slug>` on this site.
|
||||
`_reply_fields` then takes that note's `remote_id` on the target being
|
||||
posted to, as `Post.reply_remote_id`. GoToSocial skips the ids of
|
||||
favourites and boosts (`favourite:…`, `reblog:…`). Bluesky only takes
|
||||
`app.bsky.feed.post` URIs. Threads only takes numeric media ids. The
|
||||
Threads API can't look up other people's posts by URL, so only our own
|
||||
posts thread there.
|
||||
|
||||
### Bluesky threads
|
||||
|
||||
A reply's `root` must be the top of the thread. The parent is fetched with
|
||||
`app.bsky.feed.getPosts`. If the parent is itself a reply, its
|
||||
`record.reply.root` is used; otherwise the parent is the root.
|
||||
|
||||
## Not changed
|
||||
|
||||
- Crossposts made before this release. Edits can't move a status into a
|
||||
thread (GoToSocial, Mastodon), and Bluesky and Threads posts can't be edited at
|
||||
all. Re-threading would mean deleting and reposting, which loses likes and
|
||||
boosts. The user chose to leave them.
|
||||
- An edit of a reply on GoToSocial keeps the status where it is, and its
|
||||
text keeps the mention (`_prepare` resolves the parent again).
|
||||
- Fediverse authors bridged into Bluesky by Bridgy Fed have a copy of
|
||||
their post there. There is no reliable way to find its URI from the
|
||||
original URL, so these get the reply line.
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/test_reply_crossposts.py`: native fediverse reply with mention, no
|
||||
mention for our own account, reply line with name and host fallback,
|
||||
Bluesky root/parent, Bluesky reply line with its link facet, Threads native
|
||||
reply only to our own posts, the line surviving shortening, and a reply to
|
||||
our own note threading on GoToSocial and Bluesky. The reply is addressed both
|
||||
by the note's permalink and by its GoToSocial crosspost URL.
|
||||
@@ -1,5 +1,7 @@
|
||||
# v2.3.3: Micropub titles, q=source URL, single title display
|
||||
|
||||
> Superseded in v2.16.0 by ADR-076: titles are stored in `notes.title`, not as a `# Heading` in the content, and `split_title` is used only to move old headings and drop a heading that repeats `name`.
|
||||
|
||||
Found while building tuipub, a terminal Micropub client, against StarPunk.
|
||||
|
||||
## Problems
|
||||
|
||||
@@ -73,16 +73,6 @@ None open.
|
||||
|
||||
## Low
|
||||
|
||||
### Quote counts without names are not shown
|
||||
- **Description**: Backfeed stores anonymous quote counts (`syndications.quotes_count`: Threads quotes, and quotes by people who opted out elsewhere), but the counts line only adds likes and reposts, so these quotes are counted nowhere on the page.
|
||||
- **Where**: `templates/partials/responses.html` (`reactions`), `starpunk/webmention_queries.py` (`response_counts`)
|
||||
- **Source**: Documentation review 2026-11-10
|
||||
|
||||
### Dead files and settings
|
||||
- **Description**: `static/js/preview.js` is empty, yet the editor pages load it and `marked` from jsDelivr for nothing. `templates/feed.xml` is empty and `static/indielogin-test.html` is a v1 leftover. `syndication.get_targets()` reads a `THREADS_API` setting that `config.py` never loads, so it can't be set.
|
||||
- **Where**: `templates/admin/new.html`, `templates/admin/edit.html`, `static/`, `templates/feed.xml`, `starpunk/syndication.py`
|
||||
- **Source**: Documentation review 2026-11-10
|
||||
|
||||
### Micropub: `location` / `checkin` ignored
|
||||
- **Description**: These properties are accepted but dropped, which spec 3.3.5 allows.
|
||||
- **Where**: `starpunk/micropub.py`
|
||||
@@ -175,6 +165,7 @@ None open.
|
||||
|
||||
These items were removed from the backlog because they have shipped:
|
||||
|
||||
- v2.14.2: quote counts (with anonymous ones) shown; dead editor scripts, files and the `THREADS_API` read removed
|
||||
- v2.14.1: the editor can clear all tags; image captions escaped in RSS, Atom and JSON Feed HTML
|
||||
|
||||
- v2.14.0: notifications of new responses and crosspost/backfeed problems through Apprise (ADR-074)
|
||||
|
||||
@@ -57,3 +57,4 @@ Future work and per-release plans. See `../DOCUMENTATION.md` (PROJECTPLAN).
|
||||
- **[v2.13.4](v2.13.4/RELEASE.md)** - Counts on lists of notes, reply cards link out (Released 2026-11-10)
|
||||
- **[v2.14.0](v2.14.0/RELEASE.md)** - Notifications through Apprise (Released 2026-11-10)
|
||||
- **[v2.14.1](v2.14.1/RELEASE.md)** - Clearing tags in the editor; captions escaped in feeds (Released 2026-11-10)
|
||||
- **[v2.14.2](v2.14.2/RELEASE.md)** - Quote counts; dead files removed (Released 2026-11-10)
|
||||
@@ -0,0 +1,18 @@
|
||||
# v2.14.2
|
||||
|
||||
- **Date**: 2026-11-10
|
||||
- **Status**: Released
|
||||
- **Focus**: Quote counts; dead files removed
|
||||
|
||||
## Added
|
||||
|
||||
- Quote counts (with anonymous ones) in the counts line and on lists of notes
|
||||
|
||||
## Removed
|
||||
|
||||
- Unused editor scripts (`marked` from a CDN, empty `preview.js`), `templates/feed.xml`, `static/indielogin-test.html`, the `THREADS_API` config read
|
||||
|
||||
## Related
|
||||
|
||||
- [CHANGELOG](../../../CHANGELOG.md)
|
||||
- [Design note](../../design/v2.14.2/2026-11-10-quotes-cleanup.md)
|
||||
@@ -0,0 +1,6 @@
|
||||
-- Migration 022: RFC 8707 resource indicators for IndieAuth codes and tokens
|
||||
-- Version: 2.15.0 (ADR-075)
|
||||
-- Tokens for the MCP endpoint are bound to it; Micropub tokens leave it NULL.
|
||||
|
||||
ALTER TABLE indieauth_codes ADD COLUMN resource TEXT;
|
||||
ALTER TABLE indieauth_tokens ADD COLUMN resource TEXT;
|
||||
@@ -0,0 +1,6 @@
|
||||
-- Migration 023: Note titles and summaries (v2.16.0, ADR-076)
|
||||
-- An article's title moves out of its Markdown ('# Heading' first line) into
|
||||
-- notes.title; migrations.py moves existing headings right after this runs.
|
||||
-- notes.summary holds an author-written summary (Micropub 'summary').
|
||||
ALTER TABLE notes ADD COLUMN title TEXT;
|
||||
ALTER TABLE notes ADD COLUMN summary TEXT;
|
||||
@@ -0,0 +1,9 @@
|
||||
-- Migration 024: Richer reply contexts (v2.19.0)
|
||||
-- Fediverse and Bluesky posts have no microformats, so their context now
|
||||
-- comes from the Mastodon or Bluesky API, or the page's OpenGraph tags.
|
||||
-- source records which one was used; rows from before have none, and the
|
||||
-- webmention worker fetches those again once.
|
||||
ALTER TABLE reply_contexts ADD COLUMN author_handle TEXT;
|
||||
ALTER TABLE reply_contexts ADD COLUMN photo TEXT;
|
||||
ALTER TABLE reply_contexts ADD COLUMN photo_alt TEXT;
|
||||
ALTER TABLE reply_contexts ADD COLUMN source TEXT;
|
||||
+5
-1
@@ -1,4 +1,4 @@
|
||||
# StarPunk runtime dependencies (v2.11.0)
|
||||
# StarPunk runtime dependencies (v2.15.0)
|
||||
# Pinned to a minor series; see docs/architecture/technology-stack.md.
|
||||
|
||||
# Web framework and server
|
||||
@@ -25,3 +25,7 @@ apprise==2.2.*
|
||||
# Images
|
||||
Pillow==13.0.*
|
||||
pillow-heif==1.8.*
|
||||
|
||||
# MCP server sidecar (v2.15.0, ADR-075): official SDK, served by uvicorn
|
||||
mcp==2.3.*
|
||||
uvicorn==0.54.*
|
||||
+16
-5
@@ -225,6 +225,14 @@ def create_app(config=None):
|
||||
name = cached_name(app.config.get('DATA_PATH', 'data'), url)
|
||||
return url_for('public.avatar', name=name) if name else None
|
||||
|
||||
# Template filter - local copy of a replied-to post's image, or None (v2.19.0)
|
||||
@app.template_filter('local_preview')
|
||||
def local_preview_filter(url):
|
||||
from flask import url_for
|
||||
from starpunk.avatars import cached_preview
|
||||
name = cached_preview(app.config.get('DATA_PATH', 'data'), url)
|
||||
return url_for('public.preview', name=name) if name else None
|
||||
|
||||
# Template filter - URL to color for facepile (v1.6.0)
|
||||
@app.template_filter('url_to_color')
|
||||
def url_to_color_filter(url):
|
||||
@@ -325,9 +333,12 @@ def create_app(config=None):
|
||||
|
||||
register_error_handlers(app)
|
||||
|
||||
# Background workers, unless this process is the MCP sidecar (v2.15.0, ADR-075)
|
||||
workers = app.config.get("BACKGROUND_WORKERS_ENABLED", True)
|
||||
|
||||
# Start webmention worker thread (v1.6.0 Phase 2)
|
||||
# Per background-worker.md: Skip in test mode
|
||||
if app.config.get('WEBMENTION_WORKER_ENABLED', True) and not app.config.get('TESTING', False):
|
||||
if workers and app.config.get('WEBMENTION_WORKER_ENABLED', True) and not app.config.get('TESTING', False):
|
||||
from starpunk.webmention_worker import WebmentionWorker
|
||||
webmention_worker = WebmentionWorker(
|
||||
app,
|
||||
@@ -348,13 +359,13 @@ def create_app(config=None):
|
||||
|
||||
# Start WebSub worker (v2.10.0)
|
||||
from starpunk.websub_worker import start_worker
|
||||
app.websub_worker = start_worker(app)
|
||||
app.websub_worker = start_worker(app) if workers else None
|
||||
if app.websub_worker:
|
||||
app.logger.info("WebSub worker started")
|
||||
|
||||
# Start syndication worker when a target is configured (v2.1.0)
|
||||
from starpunk.syndication import SyndicationWorker, targets_configured
|
||||
if targets_configured(app.config) and not app.config.get("TESTING", False):
|
||||
if workers and targets_configured(app.config) and not app.config.get("TESTING", False):
|
||||
app.syndication_worker = SyndicationWorker(
|
||||
app, poll_interval=app.config["SYNDICATION_POLL_INTERVAL"]
|
||||
)
|
||||
@@ -465,5 +476,5 @@ def create_app(config=None):
|
||||
|
||||
# Package version (Semantic Versioning 2.0.0)
|
||||
# See docs/standards/versioning-strategy.md for details
|
||||
__version__ = "2.14.1"
|
||||
__version_info__ = (2, 14, 1)
|
||||
__version__ = "2.20.0"
|
||||
__version_info__ = (2, 20, 0)
|
||||
+108
-41
@@ -7,6 +7,9 @@ Each avatar is fetched once (SSRF-safe), shrunk to a 96 px WebP and saved
|
||||
as DATA_PATH/avatars/<sha256 of its URL>.webp. Pages show the saved copy,
|
||||
or the coloured initial until there is one.
|
||||
|
||||
The image of a post being replied to is kept the same way (v2.19.0): at most
|
||||
PREVIEW_WIDTH wide, in DATA_PATH/previews/.
|
||||
|
||||
Paths and URLs are passed in, so the webmention worker can use it. Fetching
|
||||
needs an app context (the SSRF-safe fetcher reads DEV_MODE).
|
||||
"""
|
||||
@@ -24,6 +27,8 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
SIZE = 96
|
||||
MAX_BYTES = 1024 * 1024
|
||||
PREVIEW_WIDTH = 640
|
||||
PREVIEW_MAX_BYTES = 8 * 1024 * 1024
|
||||
# Bluesky avatar on its CDN: .../img/<kind>/plain/<did>/<cid>[@jpeg]
|
||||
BSKY_CDN_RE = re.compile(
|
||||
r"^https://cdn\.bsky\.app/img/[a-z_]+/plain/(did:(?:plc|web):[A-Za-z0-9._:%-]+)/([a-z2-7]+)(?:@\w+)?$")
|
||||
@@ -39,12 +44,64 @@ def folder(data_path) -> Path:
|
||||
return Path(data_path) / "avatars"
|
||||
|
||||
|
||||
def cached_name(data_path, url: Optional[str]) -> Optional[str]:
|
||||
"""File name of the saved copy of this avatar, if there is one"""
|
||||
def preview_folder(data_path) -> Path:
|
||||
return Path(data_path) / "previews"
|
||||
|
||||
|
||||
def _cached(directory: Path, url: Optional[str]) -> Optional[str]:
|
||||
if not url or not url.startswith(("http://", "https://")):
|
||||
return None
|
||||
name = name_for(url)
|
||||
return name if (folder(data_path) / name).is_file() else None
|
||||
return name if (directory / name).is_file() else None
|
||||
|
||||
|
||||
def cached_name(data_path, url: Optional[str]) -> Optional[str]:
|
||||
"""File name of the saved copy of this avatar, if there is one"""
|
||||
return _cached(folder(data_path), url)
|
||||
|
||||
|
||||
def cached_preview(data_path, url: Optional[str]) -> Optional[str]:
|
||||
"""File name of the saved copy of a replied-to post's image, if there is one"""
|
||||
return _cached(preview_folder(data_path), url)
|
||||
|
||||
|
||||
def _square(image):
|
||||
from PIL import Image
|
||||
side = min(image.size)
|
||||
left, top = (image.width - side) // 2, (image.height - side) // 2
|
||||
image = image.crop((left, top, left + side, top + side))
|
||||
return image.resize((SIZE, SIZE), Image.LANCZOS)
|
||||
|
||||
|
||||
def _narrow(image):
|
||||
from PIL import Image
|
||||
if image.width <= PREVIEW_WIDTH:
|
||||
return image
|
||||
height = max(1, round(image.height * PREVIEW_WIDTH / image.width))
|
||||
return image.resize((PREVIEW_WIDTH, height), Image.LANCZOS)
|
||||
|
||||
|
||||
def _save(path: Path, url: Optional[str], site_url: str, shape, max_bytes: int,
|
||||
what: str) -> bool:
|
||||
if not url or not url.startswith(("http://", "https://")):
|
||||
return False
|
||||
if path.is_file():
|
||||
return True
|
||||
from PIL import Image
|
||||
|
||||
try:
|
||||
with Image.open(io.BytesIO(_download(url, site_url, max_bytes))) as image:
|
||||
image.load()
|
||||
image = image.convert("RGBA" if "A" in image.getbands() else "RGB")
|
||||
image = shape(image)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_suffix(".tmp")
|
||||
image.save(tmp, "WEBP", quality=80)
|
||||
tmp.replace(path)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.info(f"Could not save {what} {url}: {type(e).__name__}: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def fetch(data_path, url: Optional[str], site_url: str) -> bool:
|
||||
@@ -52,29 +109,18 @@ def fetch(data_path, url: Optional[str], site_url: str) -> bool:
|
||||
Save a copy of an avatar unless there is one. Returns True if one exists
|
||||
afterwards. Failures are logged and the page keeps showing the initial.
|
||||
"""
|
||||
if not url or not url.startswith(("http://", "https://")):
|
||||
if not url:
|
||||
return False
|
||||
path = folder(data_path) / name_for(url)
|
||||
if path.is_file():
|
||||
return True
|
||||
from PIL import Image
|
||||
return _save(folder(data_path) / name_for(url), url, site_url, _square,
|
||||
MAX_BYTES, "avatar")
|
||||
|
||||
try:
|
||||
with Image.open(io.BytesIO(_download(url, site_url))) as image:
|
||||
image.load()
|
||||
image = image.convert("RGBA" if "A" in image.getbands() else "RGB")
|
||||
side = min(image.size)
|
||||
left, top = (image.width - side) // 2, (image.height - side) // 2
|
||||
image = image.crop((left, top, left + side, top + side))
|
||||
image = image.resize((SIZE, SIZE), Image.LANCZOS)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_suffix(".tmp")
|
||||
image.save(tmp, "WEBP", quality=80)
|
||||
tmp.replace(path)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.info(f"Could not save avatar {url}: {type(e).__name__}: {e}")
|
||||
|
||||
def fetch_preview(data_path, url: Optional[str], site_url: str) -> bool:
|
||||
"""Save a copy of a replied-to post's image unless there is one (v2.19.0)"""
|
||||
if not url:
|
||||
return False
|
||||
return _save(preview_folder(data_path) / name_for(url), url, site_url, _narrow,
|
||||
PREVIEW_MAX_BYTES, "preview")
|
||||
|
||||
|
||||
def _own_instance_host() -> Optional[str]:
|
||||
@@ -85,11 +131,11 @@ def _own_instance_host() -> Optional[str]:
|
||||
return urlparse(current_app.config.get("GOTOSOCIAL_URL") or "").hostname
|
||||
|
||||
|
||||
def _get(url: str, headers: dict) -> bytes:
|
||||
def _get(url: str, headers: dict, max_bytes: int = MAX_BYTES) -> bytes:
|
||||
"""GET a public URL through the SSRF-safe fetcher"""
|
||||
from starpunk.webmention_ssrf import safe_get
|
||||
|
||||
result = safe_get(url, headers=headers, max_bytes=MAX_BYTES, max_redirects=5,
|
||||
result = safe_get(url, headers=headers, max_bytes=max_bytes, max_redirects=5,
|
||||
timeout=10.0, truncate=False)
|
||||
if result.status_code >= 400:
|
||||
raise ValueError(f"HTTP {result.status_code}")
|
||||
@@ -110,7 +156,7 @@ def _pds(did: str, headers: dict) -> str:
|
||||
raise ValueError(f"no PDS for {did}")
|
||||
|
||||
|
||||
def _download(url: str, site_url: str) -> bytes:
|
||||
def _download(url: str, site_url: str, max_bytes: int = MAX_BYTES) -> bytes:
|
||||
"""
|
||||
The avatar's bytes (v2.13.1):
|
||||
- from StarPunk's own GoToSocial directly, as the crossposting client
|
||||
@@ -136,13 +182,13 @@ def _download(url: str, site_url: str) -> bytes:
|
||||
# GoToSocial sends media it hasn't cached to the original
|
||||
# server: that one is fetched like any outside URL (v2.13.2)
|
||||
from urllib.parse import urljoin
|
||||
return _get(urljoin(url, response.headers["location"]), headers)
|
||||
return _get(urljoin(url, response.headers["location"]), headers, max_bytes)
|
||||
if response.status_code >= 400:
|
||||
raise ValueError(f"HTTP {response.status_code}")
|
||||
data = b""
|
||||
for chunk in response.iter_bytes():
|
||||
data += chunk
|
||||
if len(data) > MAX_BYTES:
|
||||
if len(data) > max_bytes:
|
||||
raise ValueError("too large")
|
||||
return data
|
||||
match = BSKY_CDN_RE.match(url)
|
||||
@@ -150,11 +196,26 @@ def _download(url: str, site_url: str) -> bytes:
|
||||
did, cid = match.groups()
|
||||
url = (f"{_pds(did, headers)}/xrpc/com.atproto.sync.getBlob"
|
||||
f"?did={quote(did, safe=':')}&cid={cid}")
|
||||
return _get(url, headers)
|
||||
return _get(url, headers, max_bytes)
|
||||
|
||||
|
||||
def _context_images(conn) -> tuple[list, list]:
|
||||
"""Avatars and post images of reply contexts that notes use (v2.19.0)"""
|
||||
rows = conn.execute(
|
||||
"""
|
||||
SELECT rc.author_photo, rc.photo FROM reply_contexts rc
|
||||
WHERE rc.fetch_status = 'fetched' AND EXISTS (
|
||||
SELECT 1 FROM notes n WHERE n.deleted_at IS NULL AND rc.url IN
|
||||
(n.in_reply_to, n.like_of, n.repost_of, n.bookmark_of))
|
||||
ORDER BY rc.id DESC
|
||||
"""
|
||||
).fetchall()
|
||||
return [r[0] for r in rows if r[0]], [r[1] for r in rows if r[1]]
|
||||
|
||||
|
||||
def fill_missing(db_path, data_path, site_url: str, limit: int = 20) -> int:
|
||||
"""Save avatars of shown responses that have no copy yet. Returns how many were tried."""
|
||||
"""Save avatars of shown responses and reply contexts, and the images of
|
||||
replied-to posts, that have no copy yet. Returns how many were tried."""
|
||||
conn = sqlite3.connect(db_path)
|
||||
try:
|
||||
urls = [row[0] for row in conn.execute(
|
||||
@@ -165,35 +226,41 @@ def fill_missing(db_path, data_path, site_url: str, limit: int = 20) -> int:
|
||||
ORDER BY id DESC
|
||||
"""
|
||||
)]
|
||||
context_avatars, context_photos = _context_images(conn)
|
||||
finally:
|
||||
conn.close()
|
||||
wanted = [(url, cached_name, fetch) for url in dict.fromkeys(urls + context_avatars)]
|
||||
wanted += [(url, cached_preview, fetch_preview) for url in dict.fromkeys(context_photos)]
|
||||
tried = 0
|
||||
for url in urls:
|
||||
for url, cached, save in wanted:
|
||||
if tried >= limit:
|
||||
break
|
||||
if cached_name(data_path, url) or not url.startswith(("http://", "https://")):
|
||||
if cached(data_path, url) or not url.startswith(("http://", "https://")):
|
||||
continue
|
||||
fetch(data_path, url, site_url)
|
||||
save(data_path, url, site_url)
|
||||
tried += 1
|
||||
return tried
|
||||
|
||||
|
||||
def remove_unused(db_path, data_path) -> int:
|
||||
"""Delete saved avatars no stored response uses. Returns how many were removed."""
|
||||
directory = folder(data_path)
|
||||
if not directory.is_dir():
|
||||
return 0
|
||||
"""Delete saved avatars and previews nothing uses. Returns how many were removed."""
|
||||
conn = sqlite3.connect(db_path)
|
||||
try:
|
||||
used = {name_for(row[0]) for row in conn.execute(
|
||||
"SELECT DISTINCT author_photo FROM webmentions WHERE author_photo IS NOT NULL")}
|
||||
context_avatars, context_photos = _context_images(conn)
|
||||
finally:
|
||||
conn.close()
|
||||
used |= {name_for(url) for url in context_avatars}
|
||||
used_previews = {name_for(url) for url in context_photos}
|
||||
removed = 0
|
||||
for path in directory.iterdir():
|
||||
if NAME_RE.match(path.name) and path.name not in used:
|
||||
path.unlink(missing_ok=True)
|
||||
removed += 1
|
||||
for directory, keep in ((folder(data_path), used), (preview_folder(data_path), used_previews)):
|
||||
if not directory.is_dir():
|
||||
continue
|
||||
for path in directory.iterdir():
|
||||
if NAME_RE.match(path.name) and path.name not in keep:
|
||||
path.unlink(missing_ok=True)
|
||||
removed += 1
|
||||
return removed
|
||||
|
||||
|
||||
|
||||
@@ -25,6 +25,7 @@ from typing import Dict, List, Optional
|
||||
from flask import current_app
|
||||
|
||||
from starpunk.database import get_db
|
||||
from starpunk.utils import post_path
|
||||
|
||||
DAY = 24 * 3600
|
||||
SCHEDULE = [ # (note younger than, check every), in seconds
|
||||
@@ -164,7 +165,7 @@ def _due(db, limit: int) -> list:
|
||||
lease = "(s.backfeed_claimed_until IS NULL OR s.backfeed_claimed_until < datetime('now'))"
|
||||
rows = db.execute(
|
||||
f"""
|
||||
SELECT s.*, n.created_at AS note_created, n.slug AS note_slug {_ELIGIBLE}
|
||||
SELECT s.*, n.created_at AS note_created, n.slug AS note_slug, n.title AS note_title {_ELIGIBLE}
|
||||
AND {lease}
|
||||
AND n.created_at >= datetime('now', ?)
|
||||
AND (s.backfeed_next_at IS NULL OR s.backfeed_next_at <= datetime('now'))
|
||||
@@ -176,7 +177,7 @@ def _due(db, limit: int) -> list:
|
||||
return rows
|
||||
backfill = db.execute(
|
||||
f"""
|
||||
SELECT s.*, n.created_at AS note_created, n.slug AS note_slug {_ELIGIBLE}
|
||||
SELECT s.*, n.created_at AS note_created, n.slug AS note_slug, n.title AS note_title {_ELIGIBLE}
|
||||
AND {lease} AND s.backfeed_backfill = 1
|
||||
AND (s.backfeed_next_at IS NULL OR s.backfeed_next_at <= datetime('now'))
|
||||
ORDER BY s.id LIMIT 1
|
||||
@@ -381,7 +382,7 @@ def check(row, target) -> None:
|
||||
# older note (e.g. one linked after the fact). ADR-074 Decision 5.
|
||||
quiet = bool(row["backfeed_backfill"]) or (
|
||||
row["backfeed_checked_at"] is None and _age_seconds(row["note_created"]) > DAY)
|
||||
store(db, row["note_id"], f"{site}/note/{row['note_slug']}", row["target"], harvest, quiet=quiet)
|
||||
store(db, row["note_id"], site + post_path(row["note_slug"], row["note_title"]), row["target"], harvest, quiet=quiet)
|
||||
_store_counts(db, row["id"], hidden_counts(harvest))
|
||||
|
||||
|
||||
|
||||
@@ -106,6 +106,10 @@ def load_config(app, config_override=None):
|
||||
|
||||
# Webmention configuration (v1.6.0)
|
||||
app.config["WEBMENTION_ENABLED"] = os.getenv("WEBMENTION_ENABLED", "true").lower() == "true"
|
||||
# Hosts never sent webmentions, besides this site itself (v2.18.3)
|
||||
app.config["WEBMENTION_SKIP_HOSTS"] = [
|
||||
h.strip().lower() for h in os.getenv("WEBMENTION_SKIP_HOSTS", "").split(",") if h.strip()
|
||||
]
|
||||
|
||||
# WebSub (v2.10.0, ADR-071): built-in hub unless WEBSUB_HUB_URL names another
|
||||
app.config["WEBSUB_ENABLED"] = os.getenv("WEBSUB_ENABLED", "true").lower() == "true"
|
||||
@@ -113,6 +117,8 @@ def load_config(app, config_override=None):
|
||||
app.config["WEBSUB_MAX_SUBSCRIPTIONS"] = int(os.getenv("WEBSUB_MAX_SUBSCRIPTIONS", "1000"))
|
||||
app.config["WEBSUB_MAX_PER_HOST"] = int(os.getenv("WEBSUB_MAX_PER_HOST", "25"))
|
||||
app.config["WEBSUB_POLL_INTERVAL"] = int(os.getenv("WEBSUB_POLL_INTERVAL", "5"))
|
||||
# Off in the MCP sidecar so it never runs a second set of workers (v2.15.0, ADR-075)
|
||||
app.config["BACKGROUND_WORKERS_ENABLED"] = os.getenv("BACKGROUND_WORKERS_ENABLED", "true").lower() == "true"
|
||||
app.config["WEBMENTION_WORKER_ENABLED"] = os.getenv("WEBMENTION_WORKER_ENABLED", "true").lower() == "true"
|
||||
app.config["WEBMENTION_POLL_INTERVAL"] = int(os.getenv("WEBMENTION_POLL_INTERVAL", "10"))
|
||||
app.config["WEBMENTION_ERROR_BACKOFF"] = int(os.getenv("WEBMENTION_ERROR_BACKOFF", "60"))
|
||||
|
||||
@@ -17,10 +17,12 @@ def init_db(app=None):
|
||||
if app:
|
||||
db_path = app.config["DATABASE_PATH"]
|
||||
logger = app.logger
|
||||
data_dir = app.config.get("DATA_PATH")
|
||||
else:
|
||||
# Fallback to default path
|
||||
db_path = Path("./data/starpunk.db")
|
||||
logger = None
|
||||
data_dir = None
|
||||
|
||||
# Ensure parent directory exists
|
||||
db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
@@ -41,4 +43,4 @@ def init_db(app=None):
|
||||
|
||||
# Run migrations
|
||||
from starpunk.migrations import run_migrations
|
||||
run_migrations(db_path, logger=logger)
|
||||
run_migrations(db_path, logger=logger, data_dir=data_dir)
|
||||
@@ -0,0 +1,127 @@
|
||||
"""
|
||||
Error log for the admin Log page (v2.18.0)
|
||||
|
||||
Collects what is failing right now from the tables that already record it:
|
||||
crossposts, response checks (backfeed), outgoing webmentions, reply-context
|
||||
fetches, WebSub deliveries and notifications. Nothing new is stored. An entry
|
||||
disappears once the next attempt succeeds, so the page lists only problems
|
||||
that still need attention.
|
||||
|
||||
Each entry is a dict:
|
||||
when -- timestamp text (UTC) or "" when unknown
|
||||
area -- short label, e.g. "Crosspost to Bluesky"
|
||||
note -- the affected Note, or None for site-wide problems
|
||||
detail -- extra context (a URL or "retrying"), may be ""
|
||||
error -- the recorded error message
|
||||
action -- optional (endpoint, id, label) for a retry button
|
||||
"""
|
||||
|
||||
from flask import current_app
|
||||
|
||||
from starpunk.database import get_db
|
||||
|
||||
SILOS = {"gotosocial": "GoToSocial", "bluesky": "Bluesky", "threads": "Threads"}
|
||||
|
||||
|
||||
def _silo(target: str) -> str:
|
||||
return SILOS.get(target, target)
|
||||
|
||||
|
||||
def _notes(ids) -> dict:
|
||||
"""Load the affected notes once each (deleted notes come back as None)"""
|
||||
from starpunk.notes import get_note
|
||||
|
||||
return {i: get_note(id=i) for i in set(ids) if i is not None}
|
||||
|
||||
|
||||
def entries() -> list:
|
||||
"""Every current error, newest first; site-wide problems come first"""
|
||||
from starpunk import backfeed, notifications
|
||||
|
||||
db = get_db(current_app)
|
||||
rows = []
|
||||
|
||||
for r in db.execute(
|
||||
"SELECT id, note_id, target, state, last_error, updated_at FROM syndications "
|
||||
"WHERE last_error IS NOT NULL AND state != 'synced'"
|
||||
):
|
||||
rows.append({
|
||||
"when": r["updated_at"], "area": f"Crosspost to {_silo(r['target'])}",
|
||||
"note_id": r["note_id"], "detail": "retrying" if r["state"] == "pending" else "",
|
||||
"error": r["last_error"],
|
||||
"action": ("admin.retry_syndication", r["id"], "Retry"),
|
||||
})
|
||||
|
||||
for r in db.execute(
|
||||
"SELECT id, note_id, target, backfeed_error, backfeed_checked_at, remote_url FROM syndications "
|
||||
"WHERE backfeed_error IS NOT NULL"
|
||||
):
|
||||
rows.append({
|
||||
"when": r["backfeed_checked_at"], "area": f"Response check on {_silo(r['target'])}",
|
||||
"note_id": r["note_id"], "detail": r["remote_url"] or "", "error": r["backfeed_error"],
|
||||
"action": ("admin.check_syndication_responses", r["id"], "Check now"),
|
||||
})
|
||||
|
||||
for r in db.execute(
|
||||
"SELECT source_note_id, target_url, status, last_error, COALESCE(started_at, created_at) AS at "
|
||||
"FROM webmention_queue WHERE last_error IS NOT NULL AND status IN ('pending', 'failed')"
|
||||
):
|
||||
rows.append({
|
||||
"when": r["at"], "area": "Webmention sent", "note_id": r["source_note_id"],
|
||||
"detail": r["target_url"] + (" (retrying)" if r["status"] == "pending" else ""),
|
||||
"error": r["last_error"], "action": None,
|
||||
})
|
||||
|
||||
for r in db.execute(
|
||||
"SELECT n.id AS note_id, rc.url, rc.fetch_error, rc.fetched_at, rc.created_at "
|
||||
"FROM reply_contexts rc JOIN notes n ON n.in_reply_to = rc.url "
|
||||
"WHERE rc.fetch_status = 'error' AND n.deleted_at IS NULL"
|
||||
):
|
||||
rows.append({
|
||||
"when": r["fetched_at"] or r["created_at"], "area": "Reply context fetch",
|
||||
"note_id": r["note_id"], "detail": r["url"], "error": r["fetch_error"] or "fetch failed",
|
||||
"action": None,
|
||||
})
|
||||
|
||||
for r in db.execute(
|
||||
"SELECT topic, callback, last_error, COALESCE(last_success_at, verified_at, created_at) AS at "
|
||||
"FROM websub_subscriptions WHERE last_error IS NOT NULL"
|
||||
):
|
||||
rows.append({
|
||||
"when": r["at"], "area": "WebSub delivery", "note_id": None,
|
||||
"detail": f"{r['topic']} → {r['callback']}", "error": r["last_error"], "action": None,
|
||||
})
|
||||
|
||||
rows.sort(key=lambda e: str(e["when"] or ""), reverse=True)
|
||||
|
||||
site = []
|
||||
for silo, message in backfeed.denied.items():
|
||||
site.append({"when": "", "area": f"Response checks on {_silo(silo)}", "note_id": None,
|
||||
"detail": "paused", "error": message, "action": None})
|
||||
notify_error = notifications.last_error(db)
|
||||
if notify_error:
|
||||
when, _, message = notify_error.partition(" UTC ")
|
||||
site.append({"when": when if message else "", "area": "Notifications", "note_id": None,
|
||||
"detail": "", "error": message or notify_error, "action": None})
|
||||
|
||||
entries = site + rows
|
||||
notes = _notes(e["note_id"] for e in entries)
|
||||
for e in entries:
|
||||
e["note"] = notes.get(e.pop("note_id"))
|
||||
return entries
|
||||
|
||||
|
||||
def count() -> int:
|
||||
"""Cheap total for the admin nav badge"""
|
||||
from starpunk import backfeed, notifications
|
||||
|
||||
db = get_db(current_app)
|
||||
total = db.execute(
|
||||
"SELECT (SELECT COUNT(*) FROM syndications WHERE last_error IS NOT NULL AND state != 'synced')"
|
||||
" + (SELECT COUNT(*) FROM syndications WHERE backfeed_error IS NOT NULL)"
|
||||
" + (SELECT COUNT(*) FROM webmention_queue WHERE last_error IS NOT NULL AND status IN ('pending', 'failed'))"
|
||||
" + (SELECT COUNT(*) FROM reply_contexts rc JOIN notes n ON n.in_reply_to = rc.url"
|
||||
" WHERE rc.fetch_status = 'error' AND n.deleted_at IS NULL)"
|
||||
" + (SELECT COUNT(*) FROM websub_subscriptions WHERE last_error IS NOT NULL)"
|
||||
).fetchone()[0]
|
||||
return total + len(backfeed.denied) + (1 if notifications.last_error(db) else 0)
|
||||
@@ -161,7 +161,10 @@ def generate_atom_streaming(
|
||||
|
||||
# Required elements
|
||||
yield f' <id>{_escape_xml(permalink)}</id>\n'
|
||||
yield f' <title>{_escape_xml(note.title)}</title>\n'
|
||||
# atom:title is required but may be empty: untitled notes (ADR-076)
|
||||
yield f' <title>{_escape_xml(note.title or "")}</title>\n'
|
||||
if note.summary:
|
||||
yield f' <summary>{_escape_xml(note.summary)}</summary>\n'
|
||||
|
||||
# Use created_at for both published and updated
|
||||
# (Note model doesn't have updated_at tracking yet)
|
||||
|
||||
@@ -229,8 +229,12 @@ def _build_item_object(site_url: str, note: Note) -> Dict[str, Any]:
|
||||
"url": permalink,
|
||||
}
|
||||
|
||||
# Add title
|
||||
item["title"] = note.title
|
||||
# Title and summary only when the note has them: untitled notes have
|
||||
# no title (JSON Feed 1.1; v2.16.0, ADR-076)
|
||||
if note.title:
|
||||
item["title"] = note.title
|
||||
if note.summary:
|
||||
item["summary"] = note.summary
|
||||
|
||||
# Add image field (URL of first/main image) - per JSON Feed 1.1 spec
|
||||
# Per Q7: Field should be absent (not null) when no media
|
||||
|
||||
+14
-37
@@ -108,7 +108,9 @@ def generate_rss(
|
||||
|
||||
# Set required item elements
|
||||
fe.id(permalink)
|
||||
fe.title(get_note_title(note))
|
||||
title = get_note_title(note)
|
||||
if title: # untitled notes have only a description (RSS 2.0, ADR-076)
|
||||
fe.title(title)
|
||||
fe.link(href=permalink)
|
||||
fe.guid(permalink, permalink=True)
|
||||
|
||||
@@ -263,8 +265,8 @@ def generate_rss_streaming(
|
||||
html_content += clean_html_for_rss(note.feed_html)
|
||||
|
||||
# Build item XML
|
||||
item_xml = f""" <item>
|
||||
<title>{_escape_xml(title)}</title>
|
||||
title_xml = f"\n <title>{_escape_xml(title)}</title>" if title else ""
|
||||
item_xml = f""" <item>{title_xml}
|
||||
<link>{_escape_xml(permalink)}</link>
|
||||
<guid isPermaLink="true">{_escape_xml(permalink)}</guid>
|
||||
<pubDate>{pub_date_str}</pubDate>"""
|
||||
@@ -494,48 +496,23 @@ def format_rfc822_date(dt: datetime) -> str:
|
||||
return dt.strftime("%a, %d %b %Y %H:%M:%S %z")
|
||||
|
||||
|
||||
def get_note_title(note: Note) -> str:
|
||||
def get_note_title(note: Note) -> Optional[str]:
|
||||
"""
|
||||
Extract title from note content
|
||||
An item's title: the note's title (articles), cut to 100 characters
|
||||
|
||||
Attempts to extract a meaningful title from the note. Uses the first
|
||||
line of content (stripped of markdown heading syntax) or falls back
|
||||
to a formatted timestamp if content is unavailable.
|
||||
|
||||
Algorithm:
|
||||
1. Try note.title property (first line, stripped of # syntax)
|
||||
2. Fall back to timestamp if title is unavailable
|
||||
Untitled notes get none (v2.16.0, ADR-076), so readers can tell notes
|
||||
from articles; RSS 2.0 lets an item have just a description.
|
||||
|
||||
Args:
|
||||
note: Note object
|
||||
|
||||
Returns:
|
||||
Title string (max 100 chars, truncated if needed)
|
||||
|
||||
Examples:
|
||||
>>> # Note with heading
|
||||
>>> note = Note(...) # content: "# My First Note\\n\\n..."
|
||||
>>> get_note_title(note)
|
||||
'My First Note'
|
||||
|
||||
>>> # Note without heading (timestamp fallback)
|
||||
>>> note = Note(...) # content: "Just some text"
|
||||
>>> get_note_title(note)
|
||||
'November 18, 2024 at 12:00 PM'
|
||||
Title string, or None for an untitled note
|
||||
"""
|
||||
try:
|
||||
# Use Note's title property (handles extraction logic)
|
||||
title = note.title
|
||||
|
||||
# Truncate to 100 characters for RSS compatibility
|
||||
if len(title) > 100:
|
||||
title = title[:100].strip() + "..."
|
||||
|
||||
return title
|
||||
|
||||
except (FileNotFoundError, OSError, AttributeError):
|
||||
# If title extraction fails, use timestamp
|
||||
return note.created_at.strftime("%B %d, %Y at %I:%M %p")
|
||||
title = note.title
|
||||
if title and len(title) > 100:
|
||||
title = title[:100].strip() + "..."
|
||||
return title or None
|
||||
|
||||
|
||||
def clean_html_for_rss(html: str) -> str:
|
||||
|
||||
+57
-17
@@ -25,7 +25,11 @@ from starpunk.database import get_db
|
||||
from starpunk.utils import absolute_url_for
|
||||
|
||||
CODE_LIFETIME_MINUTES = 10
|
||||
SUPPORTED_SCOPES = ["create", "update", "delete", "media", "profile"]
|
||||
SUPPORTED_SCOPES = ["create", "update", "delete", "media", "draft", "profile"]
|
||||
|
||||
# Scopes offered to an MCP client that asks for none (v2.15.0, ADR-075):
|
||||
# the owner unticks what they don't want on the consent screen
|
||||
MCP_DEFAULT_SCOPES = ["create", "update", "delete", "media", "draft"]
|
||||
|
||||
|
||||
class AuthorizationRequestError(Exception):
|
||||
@@ -56,6 +60,11 @@ def issuer() -> str:
|
||||
return current_app.config["SITE_URL"].rstrip("/") + "/"
|
||||
|
||||
|
||||
def mcp_resource() -> str:
|
||||
"""The MCP endpoint's resource identifier (RFC 8707, v2.15.0)"""
|
||||
return current_app.config["SITE_URL"].rstrip("/") + "/mcp"
|
||||
|
||||
|
||||
def admin_me() -> str:
|
||||
"""The single identity StarPunk issues tokens for"""
|
||||
return current_app.config["ADMIN_ME"]
|
||||
@@ -134,7 +143,7 @@ def validate_authorization_request(params: Dict[str, str]) -> Dict[str, Any]:
|
||||
|
||||
Returns:
|
||||
Normalized request: client_id, redirect_uri, state, scopes,
|
||||
code_challenge
|
||||
code_challenge, resource
|
||||
|
||||
Raises:
|
||||
AuthorizationRequestError
|
||||
@@ -176,17 +185,31 @@ def validate_authorization_request(params: Dict[str, str]) -> Dict[str, Any]:
|
||||
|
||||
scopes = [s for s in (params.get("scope") or "").split() if s]
|
||||
|
||||
# RFC 8707 resource indicator: only the MCP endpoint is a named resource
|
||||
# here (v2.15.0, ADR-075). Micropub clients don't send one.
|
||||
resource = (params.get("resource") or "").strip() or None
|
||||
if resource is not None:
|
||||
if resource.rstrip("/") != mcp_resource():
|
||||
raise AuthorizationRequestError(
|
||||
"invalid_target", "Unknown resource; this server only issues tokens for its MCP endpoint"
|
||||
)
|
||||
resource = mcp_resource()
|
||||
if not scopes:
|
||||
scopes = list(MCP_DEFAULT_SCOPES)
|
||||
|
||||
return {
|
||||
"client_id": client_id,
|
||||
"redirect_uri": redirect_uri,
|
||||
"state": params["state"],
|
||||
"scopes": scopes,
|
||||
"code_challenge": code_challenge,
|
||||
"resource": resource,
|
||||
}
|
||||
|
||||
|
||||
def create_authorization_code(
|
||||
client_id: str, redirect_uri: str, scope: str, code_challenge: str
|
||||
client_id: str, redirect_uri: str, scope: str, code_challenge: str,
|
||||
resource: Optional[str] = None,
|
||||
) -> str:
|
||||
"""Issue a single-use authorization code (stored hashed)"""
|
||||
code = secrets.token_urlsafe(32)
|
||||
@@ -195,8 +218,8 @@ def create_authorization_code(
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO indieauth_codes
|
||||
(code_hash, client_id, redirect_uri, scope, code_challenge, expires_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
(code_hash, client_id, redirect_uri, scope, code_challenge, expires_at, resource)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
_hash(code),
|
||||
@@ -205,6 +228,7 @@ def create_authorization_code(
|
||||
scope,
|
||||
code_challenge,
|
||||
datetime.now(timezone.utc).replace(tzinfo=None) + timedelta(minutes=CODE_LIFETIME_MINUTES),
|
||||
resource,
|
||||
),
|
||||
)
|
||||
db.commit()
|
||||
@@ -218,7 +242,7 @@ def redeem_authorization_code(
|
||||
Redeem an authorization code exactly once
|
||||
|
||||
Returns:
|
||||
{"me": ..., "scope": ..., "client_id": ...}
|
||||
{"me": ..., "scope": ..., "client_id": ..., "resource": ...}
|
||||
|
||||
Raises:
|
||||
InvalidGrantError
|
||||
@@ -240,7 +264,7 @@ def redeem_authorization_code(
|
||||
raise InvalidGrantError("Authorization code is invalid, expired or already used")
|
||||
|
||||
row = db.execute(
|
||||
"SELECT client_id, redirect_uri, scope, code_challenge FROM indieauth_codes WHERE code_hash = ?",
|
||||
"SELECT client_id, redirect_uri, scope, code_challenge, resource FROM indieauth_codes WHERE code_hash = ?",
|
||||
(_hash(code),),
|
||||
).fetchone()
|
||||
|
||||
@@ -252,11 +276,20 @@ def redeem_authorization_code(
|
||||
if not secrets.compare_digest(challenge, row["code_challenge"]):
|
||||
raise InvalidGrantError("PKCE verification failed")
|
||||
|
||||
return {"me": admin_me(), "scope": row["scope"], "client_id": row["client_id"]}
|
||||
return {
|
||||
"me": admin_me(),
|
||||
"scope": row["scope"],
|
||||
"client_id": row["client_id"],
|
||||
"resource": row["resource"],
|
||||
}
|
||||
|
||||
|
||||
def create_access_token(client_id: str, scope: str) -> Dict[str, Any]:
|
||||
"""Issue an access token (stored hashed); returns the token response"""
|
||||
def create_access_token(client_id: str, scope: str, resource: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""Issue an access token (stored hashed); returns the token response
|
||||
|
||||
resource is the RFC 8707 resource the token is for (the MCP endpoint),
|
||||
or None for Micropub tokens.
|
||||
"""
|
||||
token = secrets.token_urlsafe(32)
|
||||
lifetime_days = current_app.config.get("INDIEAUTH_TOKEN_LIFETIME_DAYS", 90)
|
||||
expires_at = datetime.now(timezone.utc).replace(tzinfo=None) + timedelta(days=lifetime_days)
|
||||
@@ -264,10 +297,10 @@ def create_access_token(client_id: str, scope: str) -> Dict[str, Any]:
|
||||
db = get_db(current_app)
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO indieauth_tokens (token_hash, client_id, scope, expires_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
INSERT INTO indieauth_tokens (token_hash, client_id, scope, expires_at, resource)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
""",
|
||||
(_hash(token), client_id, scope, expires_at),
|
||||
(_hash(token), client_id, scope, expires_at, resource),
|
||||
)
|
||||
db.commit()
|
||||
|
||||
@@ -286,14 +319,15 @@ def verify_access_token(token: str) -> Optional[Dict[str, Any]]:
|
||||
Check a bearer token issued by StarPunk
|
||||
|
||||
Returns:
|
||||
{"me", "client_id", "scope"} if active, otherwise None
|
||||
{"me", "client_id", "scope", "resource", "expires_at"} if active,
|
||||
otherwise None
|
||||
"""
|
||||
if not token:
|
||||
return None
|
||||
db = get_db(current_app)
|
||||
row = db.execute(
|
||||
"""
|
||||
SELECT id, client_id, scope FROM indieauth_tokens
|
||||
SELECT id, client_id, scope, resource, expires_at FROM indieauth_tokens
|
||||
WHERE token_hash = ? AND revoked_at IS NULL AND expires_at > datetime('now')
|
||||
""",
|
||||
(_hash(token),),
|
||||
@@ -306,7 +340,13 @@ def verify_access_token(token: str) -> Optional[Dict[str, Any]]:
|
||||
(row["id"],),
|
||||
)
|
||||
db.commit()
|
||||
return {"me": admin_me(), "client_id": row["client_id"], "scope": row["scope"]}
|
||||
return {
|
||||
"me": admin_me(),
|
||||
"client_id": row["client_id"],
|
||||
"scope": row["scope"],
|
||||
"resource": row["resource"],
|
||||
"expires_at": row["expires_at"],
|
||||
}
|
||||
|
||||
|
||||
def revoke_access_token(token: str) -> None:
|
||||
@@ -334,7 +374,7 @@ def list_active_tokens() -> List[Dict[str, Any]]:
|
||||
db = get_db(current_app)
|
||||
rows = db.execute(
|
||||
"""
|
||||
SELECT id, client_id, scope, created_at, expires_at, last_used_at
|
||||
SELECT id, client_id, scope, resource, created_at, expires_at, last_used_at
|
||||
FROM indieauth_tokens
|
||||
WHERE revoked_at IS NULL AND expires_at > datetime('now')
|
||||
ORDER BY created_at DESC
|
||||
|
||||
@@ -1,7 +1,12 @@
|
||||
"""
|
||||
Markdown extension: turn bare URLs into links (v2.3.0)
|
||||
Markdown extensions for notes
|
||||
|
||||
Python-Markdown only links <https://...>; notes often contain pasted URLs.
|
||||
BareUrlExtension: turn bare URLs into links (v2.3.0). Python-Markdown only
|
||||
links <https://...>; notes often contain pasted URLs.
|
||||
|
||||
HashtagLineExtension: a line starting with a hashtag stays text (v2.16.0).
|
||||
Python-Markdown makes '#indieweb is fun' a heading; CommonMark (and every
|
||||
other Markdown a client may use) needs a space after the '#'.
|
||||
"""
|
||||
|
||||
import re
|
||||
@@ -9,6 +14,7 @@ import xml.etree.ElementTree as etree
|
||||
|
||||
from markdown.extensions import Extension
|
||||
from markdown.inlinepatterns import InlineProcessor
|
||||
from markdown.preprocessors import Preprocessor
|
||||
from markdown.treeprocessors import Treeprocessor
|
||||
from markdown.util import AtomicString
|
||||
|
||||
@@ -40,3 +46,20 @@ class BareUrlExtension(Extension):
|
||||
# Below links (160) and <autolinks> (120), so those win
|
||||
md.inlinePatterns.register(_BareUrl(BARE_URL_RE, md), "bare_url", 105)
|
||||
md.treeprocessors.register(_UnnestLinks(md), "unnest_links", 0)
|
||||
|
||||
|
||||
# Up to 3 spaces, then '#'s straight followed by text: a hashtag, not a heading
|
||||
_HASHTAG_LINE_RE = re.compile(r"^( {0,3})(?=#+[^\s#])")
|
||||
|
||||
|
||||
class _HashtagLines(Preprocessor):
|
||||
"""Escape the first '#' of such a line, so it renders as text"""
|
||||
|
||||
def run(self, lines):
|
||||
return [_HASHTAG_LINE_RE.sub(r"\1\\", line) for line in lines]
|
||||
|
||||
|
||||
class HashtagLineExtension(Extension):
|
||||
def extendMarkdown(self, md):
|
||||
# After fenced code (25) and raw HTML (20) are set aside, so code is untouched
|
||||
md.preprocessors.register(_HashtagLines(md), "hashtag_lines", 15)
|
||||
@@ -0,0 +1,13 @@
|
||||
"""
|
||||
MCP server for StarPunk (v2.15.0, ADR-075)
|
||||
|
||||
AI clients (claude.ai connectors, Claude Code, ...) read and write notes
|
||||
through the Model Context Protocol at SITE_URL/mcp. It runs as a sidecar
|
||||
process (uvicorn, the official MCP Python SDK) next to the Flask app and
|
||||
calls the same service functions. StarPunk's own IndieAuth server issues the
|
||||
tokens; this server only checks them (an OAuth resource server).
|
||||
"""
|
||||
|
||||
from starpunk.mcp_server.server import create_asgi_app, create_mcp_server
|
||||
|
||||
__all__ = ["create_asgi_app", "create_mcp_server"]
|
||||
@@ -0,0 +1,13 @@
|
||||
"""
|
||||
ASGI entry point for the MCP sidecar (v2.15.0, ADR-075)
|
||||
|
||||
uvicorn starpunk.mcp_server.app:app --host 0.0.0.0 --port 8001
|
||||
|
||||
Run it with BACKGROUND_WORKERS_ENABLED=false and the same environment and
|
||||
data volume as the main app. Route /mcp and
|
||||
/.well-known/oauth-protected-resource/mcp to it.
|
||||
"""
|
||||
|
||||
from starpunk.mcp_server.server import create_asgi_app
|
||||
|
||||
app = create_asgi_app()
|
||||
@@ -0,0 +1,56 @@
|
||||
"""
|
||||
Bearer token checks for the MCP endpoint (v2.15.0, ADR-075)
|
||||
|
||||
Tokens are StarPunk IndieAuth tokens bound to the MCP resource (RFC 8707).
|
||||
The SDK's bearer middleware turns a missing or rejected token into a 401
|
||||
that points the client at the protected resource metadata, and refuses
|
||||
tokens whose resource isn't this endpoint (validate_token_resource).
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
import anyio.to_thread
|
||||
from flask import Flask
|
||||
from mcp.server.auth.provider import AccessToken
|
||||
|
||||
from starpunk.indieauth import verify_access_token
|
||||
|
||||
|
||||
def _epoch(value) -> Optional[int]:
|
||||
"""SQLite timestamp (datetime or 'YYYY-MM-DD HH:MM:SS' UTC) -> epoch seconds"""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, str):
|
||||
try:
|
||||
value = datetime.fromisoformat(value)
|
||||
except ValueError:
|
||||
return None
|
||||
if value.tzinfo is None:
|
||||
value = value.replace(tzinfo=timezone.utc)
|
||||
return int(value.timestamp())
|
||||
|
||||
|
||||
class StarPunkTokenVerifier:
|
||||
"""TokenVerifier backed by the indieauth_tokens table"""
|
||||
|
||||
def __init__(self, app: Flask):
|
||||
self.app = app
|
||||
|
||||
def _verify(self, token: str) -> Optional[AccessToken]:
|
||||
with self.app.app_context():
|
||||
info = verify_access_token(token)
|
||||
if not info:
|
||||
return None
|
||||
return AccessToken(
|
||||
token=token,
|
||||
client_id=info["client_id"],
|
||||
scopes=(info["scope"] or "").split(),
|
||||
expires_at=_epoch(info.get("expires_at")),
|
||||
resource=info.get("resource"),
|
||||
subject=info["me"],
|
||||
)
|
||||
|
||||
async def verify_token(self, token: str) -> Optional[AccessToken]:
|
||||
# Database work happens off the event loop
|
||||
return await anyio.to_thread.run_sync(self._verify, token)
|
||||
@@ -0,0 +1,95 @@
|
||||
"""
|
||||
The MCP server and its ASGI app (v2.15.0, ADR-075)
|
||||
|
||||
Stateless Streamable HTTP with plain JSON responses: every request stands
|
||||
alone, so any number of uvicorn workers can serve it and nothing is lost on
|
||||
restart.
|
||||
"""
|
||||
|
||||
from typing import Optional
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from flask import Flask
|
||||
from mcp.server import MCPServer
|
||||
from mcp.server.auth.settings import AuthSettings
|
||||
from mcp.server.transport_security import TransportSecuritySettings
|
||||
from pydantic import AnyHttpUrl
|
||||
from starlette.applications import Starlette
|
||||
|
||||
from starpunk.mcp_server.auth import StarPunkTokenVerifier
|
||||
from starpunk.mcp_server.tools import register_tools
|
||||
|
||||
INSTRUCTIONS = """\
|
||||
StarPunk is an IndieWeb site: a personal blog of short notes and longer
|
||||
articles, owned by one person. Notes are Markdown. A note whose first line
|
||||
is '# Heading' is an article with that title. Notes can be drafts
|
||||
(unpublished) or published; publishing may crosspost them to social
|
||||
networks chosen with syndicate_to (see get_site_config). Prefer drafts unless
|
||||
the owner asked to publish. Deleting is a soft delete that undelete_note
|
||||
reverses.
|
||||
"""
|
||||
|
||||
|
||||
def _site(app: Flask) -> str:
|
||||
return app.config["SITE_URL"].rstrip("/")
|
||||
|
||||
|
||||
def create_mcp_server(app: Flask) -> MCPServer:
|
||||
"""The MCP server, with StarPunk's tools and token checks"""
|
||||
site = _site(app)
|
||||
server = MCPServer(
|
||||
name="StarPunk",
|
||||
instructions=INSTRUCTIONS,
|
||||
website_url=site,
|
||||
version=app.config.get("VERSION", ""),
|
||||
token_verifier=StarPunkTokenVerifier(app),
|
||||
auth=AuthSettings(
|
||||
# Same string as the issuer in /.well-known/oauth-authorization-server
|
||||
issuer_url=AnyHttpUrl(site + "/"),
|
||||
resource_server_url=AnyHttpUrl(site + "/mcp"),
|
||||
# Tools check their own scopes; any StarPunk token may connect
|
||||
required_scopes=[],
|
||||
validate_token_resource=True,
|
||||
),
|
||||
)
|
||||
register_tools(server, app)
|
||||
return server
|
||||
|
||||
|
||||
def _transport_security(app: Flask) -> TransportSecuritySettings:
|
||||
"""Only requests for this site's host and from its origin (DNS rebinding)"""
|
||||
site = urlparse(_site(app))
|
||||
hosts = [site.netloc]
|
||||
origins = [f"{site.scheme}://{site.netloc}"]
|
||||
if app.config.get("DEV_MODE"):
|
||||
hosts += ["localhost:*", "127.0.0.1:*"]
|
||||
origins += ["http://localhost:*", "http://127.0.0.1:*"]
|
||||
return TransportSecuritySettings(
|
||||
enable_dns_rebinding_protection=True,
|
||||
allowed_hosts=hosts,
|
||||
allowed_origins=origins,
|
||||
)
|
||||
|
||||
|
||||
def create_asgi_app(app: Optional[Flask] = None) -> Starlette:
|
||||
"""
|
||||
ASGI app serving /mcp and its protected resource metadata
|
||||
|
||||
Args:
|
||||
app: The Flask app to call into. Built from the environment if not
|
||||
given, with background workers off (the main app runs those).
|
||||
"""
|
||||
if app is None:
|
||||
from starpunk import create_app
|
||||
|
||||
app = create_app({"BACKGROUND_WORKERS_ENABLED": False})
|
||||
|
||||
server = create_mcp_server(app)
|
||||
return server.streamable_http_app(
|
||||
streamable_http_path="/mcp",
|
||||
stateless_http=True,
|
||||
json_response=True,
|
||||
# Base64 photos for upload_media: 10 MB of image is ~14 MB encoded
|
||||
max_request_body_size=16 * 1024 * 1024,
|
||||
transport_security=_transport_security(app),
|
||||
)
|
||||
@@ -0,0 +1,372 @@
|
||||
"""
|
||||
MCP tools for StarPunk (v2.15.0, ADR-075)
|
||||
|
||||
Reading tools call the notes, tags and search services. Writing tools build
|
||||
a Micropub request and hand it to the Micropub handlers, so an MCP post goes
|
||||
through exactly the same checks and side effects (scopes, drafts-only
|
||||
tokens, crossposting, webmentions, WebSub) as one from any Micropub client.
|
||||
|
||||
Scopes are the Micropub ones: reading needs any StarPunk token; drafts and
|
||||
tags need create, update, delete or draft; writing needs create (or draft),
|
||||
update, delete or media.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, Optional
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import anyio.to_thread
|
||||
from flask import Flask
|
||||
from mcp.server import MCPServer
|
||||
from mcp.server.auth.middleware.auth_context import get_access_token
|
||||
from mcp.server.mcpserver.exceptions import ToolError
|
||||
from mcp.types import ToolAnnotations
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from starpunk.indieauth import check_scope
|
||||
from starpunk.micropub import MicropubError
|
||||
|
||||
DRAFT_SCOPES = ("create", "update", "delete", "draft")
|
||||
MAX_LIMIT = 100
|
||||
|
||||
|
||||
class Photo(BaseModel):
|
||||
"""A photo for a note"""
|
||||
|
||||
url: str = Field(description="URL from upload_media, or any public image URL")
|
||||
alt: str = Field(default="", description="Alt text describing the photo")
|
||||
|
||||
|
||||
def _token_info() -> dict:
|
||||
"""The caller's token as the Micropub handlers expect it"""
|
||||
token = get_access_token()
|
||||
if token is None:
|
||||
raise ToolError("Not authenticated")
|
||||
return {"me": token.subject, "client_id": token.client_id, "scope": " ".join(token.scopes)}
|
||||
|
||||
|
||||
def _require(info: dict, *scopes: str) -> None:
|
||||
if not any(check_scope(s, info["scope"]) for s in scopes):
|
||||
raise ToolError(f"This needs the {' or '.join(scopes)} scope; reconnect and grant it")
|
||||
|
||||
|
||||
def _sees_drafts(info: dict) -> bool:
|
||||
return any(check_scope(s, info["scope"]) for s in DRAFT_SCOPES)
|
||||
|
||||
|
||||
def _clamp(limit: int) -> int:
|
||||
return max(1, min(int(limit), MAX_LIMIT))
|
||||
|
||||
|
||||
def _note_url(slug: str, title=None) -> str:
|
||||
from starpunk.utils import post_url
|
||||
|
||||
return post_url(slug, title)
|
||||
|
||||
|
||||
def _summary(note) -> dict:
|
||||
from starpunk.utils import iso_utc
|
||||
|
||||
return {
|
||||
"url": note.url,
|
||||
"slug": note.slug,
|
||||
"title": note.title,
|
||||
"summary": note.summary,
|
||||
"kind": note.kind,
|
||||
"status": "published" if note.published else "draft",
|
||||
"published": iso_utc(note.created_at),
|
||||
"excerpt": note.excerpt,
|
||||
}
|
||||
|
||||
|
||||
def _find_note(url_or_slug: str):
|
||||
"""A note by its URL on this site or its slug"""
|
||||
from starpunk.micropub import _note_from_url
|
||||
from starpunk.notes import get_note
|
||||
|
||||
value = (url_or_slug or "").strip()
|
||||
if urlparse(value).scheme:
|
||||
return _note_from_url(value)
|
||||
note = get_note(slug=value.strip("/").split("/")[-1]) if value else None
|
||||
if note is None:
|
||||
raise ToolError("Note not found")
|
||||
return note
|
||||
|
||||
|
||||
def _status_after(url: str) -> dict:
|
||||
"""Summary of the note at url, as it is now"""
|
||||
return _summary(_find_note(url))
|
||||
|
||||
|
||||
def register_tools(server: MCPServer, app: Flask) -> None:
|
||||
"""Add StarPunk's tools to the MCP server"""
|
||||
site = app.config["SITE_URL"].rstrip("/")
|
||||
|
||||
async def run(fn: Callable[..., Any], *args, **kwargs) -> Any:
|
||||
"""Run fn in a worker thread inside a Flask request context for SITE_URL"""
|
||||
|
||||
def call():
|
||||
with app.test_request_context(base_url=site):
|
||||
try:
|
||||
return fn(*args, **kwargs)
|
||||
except MicropubError as e:
|
||||
raise ToolError(e.error_description or e.error) from e
|
||||
except ValueError as e:
|
||||
raise ToolError(str(e)) from e
|
||||
|
||||
return await anyio.to_thread.run_sync(call)
|
||||
|
||||
read_only = ToolAnnotations(read_only_hint=True, open_world_hint=False)
|
||||
|
||||
# Reading
|
||||
|
||||
@server.tool(annotations=read_only)
|
||||
async def search_notes(query: str, include_drafts: bool = False, limit: int = 20) -> dict:
|
||||
"""Full-text search of the site's notes. Returns matches with a snippet;
|
||||
use get_note for the whole note."""
|
||||
info = _token_info()
|
||||
if include_drafts:
|
||||
_require(info, *DRAFT_SCOPES)
|
||||
|
||||
def search():
|
||||
from starpunk.search import has_fts_table, search_notes as fts
|
||||
|
||||
db_path = Path(app.config["DATABASE_PATH"])
|
||||
if not has_fts_table(db_path):
|
||||
raise ToolError("Search is not available on this server")
|
||||
results = fts(query=query, db_path=db_path, published_only=not include_drafts, limit=_clamp(limit))
|
||||
return {
|
||||
"results": [
|
||||
{"url": _note_url(r["slug"], r.get("article_title")), "slug": r["slug"], "title": r.get("title"),
|
||||
"snippet": r.get("snippet")}
|
||||
for r in results
|
||||
]
|
||||
}
|
||||
|
||||
return await run(search)
|
||||
|
||||
@server.tool(annotations=read_only)
|
||||
async def list_notes(include_drafts: bool = False, limit: int = 20, offset: int = 0) -> dict:
|
||||
"""The site's notes, newest first. Drafts are left out unless
|
||||
include_drafts is true."""
|
||||
info = _token_info()
|
||||
if include_drafts:
|
||||
_require(info, *DRAFT_SCOPES)
|
||||
|
||||
def notes():
|
||||
from starpunk.notes import list_notes as list_all
|
||||
|
||||
found = list_all(published_only=not include_drafts, limit=_clamp(limit), offset=max(0, offset))
|
||||
return {"notes": [_summary(n) for n in found], "offset": max(0, offset)}
|
||||
|
||||
return await run(notes)
|
||||
|
||||
@server.tool(annotations=read_only)
|
||||
async def get_note(url_or_slug: str) -> dict:
|
||||
"""One note in full: Markdown content, title (name), tags (category),
|
||||
photos, status, reply/like/repost/bookmark target and crosspost links,
|
||||
as Microformats2 properties."""
|
||||
info = _token_info()
|
||||
|
||||
def one():
|
||||
from starpunk.micropub import note_to_mf2
|
||||
|
||||
note = _find_note(url_or_slug)
|
||||
if not note.published and not _sees_drafts(info):
|
||||
raise ToolError("Note not found")
|
||||
return {"url": note.url, "slug": note.slug,
|
||||
"properties": note_to_mf2(note)["properties"]}
|
||||
|
||||
return await run(one)
|
||||
|
||||
@server.tool(annotations=read_only)
|
||||
async def list_tags(filter: str = "") -> dict:
|
||||
"""Tags in use on the site, most used first, optionally only those
|
||||
containing filter."""
|
||||
info = _token_info()
|
||||
_require(info, *DRAFT_SCOPES)
|
||||
|
||||
def tags():
|
||||
from starpunk.micropub import _all_categories
|
||||
|
||||
return {"tags": _all_categories(filter)}
|
||||
|
||||
return await run(tags)
|
||||
|
||||
@server.tool(annotations=read_only)
|
||||
async def get_site_config() -> dict:
|
||||
"""What the site supports: post types and the crosspost targets
|
||||
(syndicate-to uids) that create_note's syndicate_to accepts."""
|
||||
info = _token_info()
|
||||
|
||||
def config():
|
||||
from starpunk.micropub import handle_query
|
||||
|
||||
response = handle_query({"q": "config"}, info)
|
||||
# Flask views may return (response, status[, headers])
|
||||
if isinstance(response, tuple):
|
||||
response = response[0]
|
||||
return response.get_json()
|
||||
|
||||
return await run(config)
|
||||
|
||||
# Writing
|
||||
|
||||
@server.tool(annotations=ToolAnnotations(destructive_hint=False, open_world_hint=True))
|
||||
async def create_note(
|
||||
content: str = "",
|
||||
title: Optional[str] = None,
|
||||
summary: Optional[str] = None,
|
||||
tags: Optional[list[str]] = None,
|
||||
draft: bool = True,
|
||||
photos: Optional[list[Photo]] = None,
|
||||
in_reply_to: Optional[str] = None,
|
||||
like_of: Optional[str] = None,
|
||||
repost_of: Optional[str] = None,
|
||||
bookmark_of: Optional[str] = None,
|
||||
syndicate_to: Optional[list[str]] = None,
|
||||
slug: Optional[str] = None,
|
||||
published_at: Optional[str] = None,
|
||||
) -> dict:
|
||||
"""Create a note (Markdown). It is a draft unless draft is false;
|
||||
publishing makes it public and crossposts it to the syndicate_to
|
||||
targets. A title makes it an article, at /article/<slug>; summary is its
|
||||
short summary, shown on the homepage and in feeds and crossposts. The
|
||||
slug is made from the title for articles and from the time for other
|
||||
notes, so leave slug out unless a specific one is wanted. At most one of
|
||||
like_of, repost_of and bookmark_of. published_at is an ISO 8601 date."""
|
||||
info = _token_info()
|
||||
_require(info, "create", "draft")
|
||||
|
||||
props: dict[str, list] = {"post-status": ["draft" if draft else "published"]}
|
||||
if content:
|
||||
props["content"] = [content]
|
||||
if title:
|
||||
props["name"] = [title]
|
||||
if summary:
|
||||
props["summary"] = [summary]
|
||||
if tags:
|
||||
props["category"] = list(tags)
|
||||
if photos:
|
||||
props["photo"] = [{"value": p.url, "alt": p.alt} for p in photos]
|
||||
for name, value in (("in-reply-to", in_reply_to), ("like-of", like_of),
|
||||
("repost-of", repost_of), ("bookmark-of", bookmark_of),
|
||||
("published", published_at)):
|
||||
if value:
|
||||
props[name] = [value]
|
||||
data: dict[str, Any] = {"type": ["h-entry"], "properties": props}
|
||||
if syndicate_to:
|
||||
data["mp-syndicate-to"] = list(syndicate_to)
|
||||
if slug:
|
||||
data["mp-slug"] = slug
|
||||
|
||||
def create():
|
||||
from starpunk.micropub import handle_create
|
||||
|
||||
_, _, headers = handle_create(data, info)
|
||||
return _status_after(headers["Location"])
|
||||
|
||||
return await run(create)
|
||||
|
||||
@server.tool(annotations=ToolAnnotations(destructive_hint=True, idempotent_hint=True, open_world_hint=True))
|
||||
async def update_note(
|
||||
url: str,
|
||||
content: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
summary: Optional[str] = None,
|
||||
tags: Optional[list[str]] = None,
|
||||
add_tags: Optional[list[str]] = None,
|
||||
remove_tags: Optional[list[str]] = None,
|
||||
draft: Optional[bool] = None,
|
||||
photos: Optional[list[Photo]] = None,
|
||||
in_reply_to: Optional[str] = None,
|
||||
) -> dict:
|
||||
"""Change a note. Only the fields given change: content replaces the
|
||||
text, title replaces the title ("" removes it), summary likewise, tags replaces all
|
||||
tags, add_tags/remove_tags adjust them, photos replaces all photos,
|
||||
draft false publishes a draft (and crossposts it) and true unpublishes.
|
||||
Edits to published notes follow to their crossposts."""
|
||||
info = _token_info()
|
||||
_require(info, "update")
|
||||
|
||||
replace: dict[str, list] = {}
|
||||
if content is not None:
|
||||
replace["content"] = [content]
|
||||
if title is not None:
|
||||
replace["name"] = [title]
|
||||
if summary is not None:
|
||||
replace["summary"] = [summary]
|
||||
if tags is not None:
|
||||
replace["category"] = list(tags)
|
||||
if draft is not None:
|
||||
replace["post-status"] = ["draft" if draft else "published"]
|
||||
if photos is not None:
|
||||
replace["photo"] = [{"value": p.url, "alt": p.alt} for p in photos]
|
||||
if in_reply_to is not None:
|
||||
replace["in-reply-to"] = [in_reply_to]
|
||||
data: dict[str, Any] = {"action": "update", "url": url, "replace": replace}
|
||||
if add_tags:
|
||||
data["add"] = {"category": list(add_tags)}
|
||||
if remove_tags:
|
||||
data["delete"] = {"category": list(remove_tags)}
|
||||
|
||||
def update():
|
||||
from starpunk.micropub import handle_update
|
||||
|
||||
note = _find_note(url)
|
||||
data["url"] = note.url
|
||||
handle_update(data, info)
|
||||
return _status_after(data["url"])
|
||||
|
||||
return await run(update)
|
||||
|
||||
@server.tool(annotations=ToolAnnotations(destructive_hint=True, idempotent_hint=True, open_world_hint=False))
|
||||
async def delete_note(url: str) -> dict:
|
||||
"""Delete a note. It can be restored with undelete_note."""
|
||||
info = _token_info()
|
||||
_require(info, "delete")
|
||||
|
||||
def delete():
|
||||
from starpunk.micropub import handle_delete
|
||||
|
||||
note_url = _find_note(url).url
|
||||
handle_delete({"url": note_url}, info)
|
||||
return {"url": note_url, "status": "deleted"}
|
||||
|
||||
return await run(delete)
|
||||
|
||||
@server.tool(annotations=ToolAnnotations(destructive_hint=False, idempotent_hint=True, open_world_hint=False))
|
||||
async def undelete_note(url: str) -> dict:
|
||||
"""Restore a deleted note."""
|
||||
info = _token_info()
|
||||
_require(info, "delete")
|
||||
note_url = url if urlparse(url).scheme else f"{site}/note/{url.strip('/')}"
|
||||
|
||||
def undelete():
|
||||
from starpunk.micropub import handle_delete
|
||||
|
||||
handle_delete({"url": note_url}, info, undelete=True)
|
||||
return _status_after(note_url)
|
||||
|
||||
return await run(undelete)
|
||||
|
||||
@server.tool(annotations=ToolAnnotations(destructive_hint=False, open_world_hint=False))
|
||||
async def upload_media(filename: str, data_base64: str) -> dict:
|
||||
"""Upload an image (JPEG, PNG, GIF, WebP or HEIC, base64-encoded) and
|
||||
get its URL, to use as a photo in create_note or update_note."""
|
||||
info = _token_info()
|
||||
_require(info, "media", "create")
|
||||
try:
|
||||
data = base64.b64decode(data_base64, validate=True)
|
||||
except (binascii.Error, ValueError):
|
||||
raise ToolError("data_base64 is not valid base64")
|
||||
|
||||
def upload():
|
||||
from starpunk.media import save_media
|
||||
|
||||
media = save_media(data, filename)
|
||||
return {"url": f"{site}/media/{media['path']}"}
|
||||
|
||||
return await run(upload)
|
||||
+98
-40
@@ -7,7 +7,8 @@ interface for creating posts via external clients.
|
||||
Functions:
|
||||
normalize_properties: Convert form/JSON data to Micropub properties format
|
||||
extract_content: Get content from Micropub properties
|
||||
extract_title: Get the explicit title (name) from Micropub properties
|
||||
extract_title: Get the title (name) from Micropub properties
|
||||
extract_summary: Get the author's summary from Micropub properties
|
||||
extract_tags: Get category tags from Micropub properties
|
||||
handle_create: Process Micropub create action
|
||||
handle_update: Process Micropub update action (v2.4.0)
|
||||
@@ -204,10 +205,11 @@ def extract_content(properties: dict) -> str:
|
||||
|
||||
content = content_list[0]
|
||||
|
||||
# Handle structured content ({"html": "...", "text": "..."})
|
||||
# Handle structured content ({"html": "...", "text": "..."}). The HTML is
|
||||
# what was written; clients send text as a plain fallback, which loses
|
||||
# images, links and headings (v2.16.0; before, text was preferred)
|
||||
if isinstance(content, dict):
|
||||
# Prefer text over html for markdown storage
|
||||
content = content.get("text") or content.get("html", "")
|
||||
content = content.get("html") or content.get("text", "")
|
||||
|
||||
if not isinstance(content, str) or not content.strip():
|
||||
raise MicropubValidationError("Content cannot be empty")
|
||||
@@ -216,9 +218,9 @@ def extract_content(properties: dict) -> str:
|
||||
|
||||
|
||||
def content_format(properties: dict) -> Optional[str]:
|
||||
"""'html' if the content was sent only as HTML (returned as such by q=source)"""
|
||||
"""'html' if the content was sent as HTML (returned as such by q=source)"""
|
||||
content = _first(properties.get("content"))
|
||||
if isinstance(content, dict) and content.get("html") and not content.get("text"):
|
||||
if isinstance(content, dict) and content.get("html"):
|
||||
return "html"
|
||||
return None
|
||||
|
||||
@@ -235,10 +237,7 @@ def _has_text(content) -> bool:
|
||||
|
||||
def extract_title(properties: dict) -> Optional[str]:
|
||||
"""
|
||||
Extract the explicit title (Micropub 'name' property)
|
||||
|
||||
StarPunk has no title field: a note's title is a '# Heading' first line
|
||||
(see Note.title), so the caller stores the name as that heading.
|
||||
Extract the title (Micropub 'name' property), stored in notes.title (ADR-076)
|
||||
|
||||
Args:
|
||||
properties: Normalized Micropub properties dict
|
||||
@@ -252,14 +251,22 @@ def extract_title(properties: dict) -> Optional[str]:
|
||||
return " ".join(name.split()) or None
|
||||
|
||||
|
||||
def add_title_heading(content: str, title: Optional[str]) -> str:
|
||||
"""Prefix content with '# title' unless it already starts with that heading"""
|
||||
if not title:
|
||||
return content
|
||||
first_line = content.lstrip().split("\n", 1)[0]
|
||||
if first_line.startswith("#") and first_line.lstrip("#").strip() == title:
|
||||
return content
|
||||
return f"# {title}\n\n{content}"
|
||||
def extract_summary(properties: dict) -> Optional[str]:
|
||||
"""The author's summary (Micropub 'summary' property, v2.16.0), as plain text"""
|
||||
summary = _first(properties.get("summary"))
|
||||
if isinstance(summary, dict):
|
||||
summary = summary.get("text") or summary.get("value") or ""
|
||||
if not isinstance(summary, str):
|
||||
return None
|
||||
return summary.strip() or None
|
||||
|
||||
|
||||
def drop_title_heading(content: str, title: Optional[str]) -> str:
|
||||
"""Content without a '# heading' first line that only repeats the title"""
|
||||
heading, rest = split_title(content)
|
||||
if title and heading == title:
|
||||
return rest
|
||||
return content
|
||||
|
||||
|
||||
def extract_tags(properties: dict) -> list[str]:
|
||||
@@ -308,7 +315,7 @@ MAX_PHOTOS = 4 # ADR-057
|
||||
MAX_REMOTE_PHOTO_BYTES = 20 * 1024 * 1024
|
||||
|
||||
# Properties we can store, and so can be updated (v2.4.0)
|
||||
UPDATABLE = {"content", "name", "category", "in-reply-to", "post-status", "photo"}
|
||||
UPDATABLE = {"content", "name", "summary", "category", "in-reply-to", "post-status", "photo"}
|
||||
|
||||
|
||||
def _first(value):
|
||||
@@ -407,9 +414,10 @@ def _public_address(host: str) -> bool:
|
||||
return bool(infos)
|
||||
|
||||
|
||||
def _fetch_remote_photo(url: str) -> Optional[int]:
|
||||
def _fetch_remote_photo(url: str) -> Optional[dict]:
|
||||
"""
|
||||
Download a photo hosted elsewhere into the media library. Returns media id.
|
||||
Download a photo hosted elsewhere into the media library. Returns the
|
||||
saved media ({'id', 'path', ...}), or None if it couldn't be fetched.
|
||||
|
||||
Uses the webmention fetcher (v2.9.0): every redirect is checked against
|
||||
private addresses and the connection is pinned to the checked address.
|
||||
@@ -426,7 +434,7 @@ def _fetch_remote_photo(url: str) -> Optional[int]:
|
||||
if result.status_code != 200:
|
||||
raise ValueError(f"HTTP {result.status_code}")
|
||||
name = urlparse(result.url).path.rsplit("/", 1)[-1] or "photo.jpg"
|
||||
return save_media(result.content, name)["id"]
|
||||
return save_media(result.content, name)
|
||||
except Exception as e:
|
||||
current_app.logger.warning(f"Could not fetch photo {url}: {e}")
|
||||
return None
|
||||
@@ -446,7 +454,35 @@ def _media_id_for_url(url: str) -> Optional[int]:
|
||||
return row[0]
|
||||
current_app.logger.warning(f"Photo URL not found in media: {url}")
|
||||
return None
|
||||
return _fetch_remote_photo(url)
|
||||
media = _fetch_remote_photo(url)
|
||||
return media["id"] if media else None
|
||||
|
||||
|
||||
MAX_INLINE_IMAGES = 20
|
||||
|
||||
|
||||
def localize_inline_images(content: str) -> str:
|
||||
"""
|
||||
Copy images embedded in the text from other sites into the media library
|
||||
and point the text at the copies (v2.16.0), so an article doesn't depend
|
||||
on the client's or a CDN's host staying up. Best effort: an image that
|
||||
can't be fetched keeps its URL.
|
||||
"""
|
||||
from html import escape
|
||||
from starpunk.syndication import inline_images
|
||||
|
||||
if not content or "<img" not in content and "![" not in content:
|
||||
return content
|
||||
site_url = current_app.config.get("SITE_URL", "").rstrip("/")
|
||||
remote = [url for url, _ in inline_images(content)
|
||||
if url.startswith(("http://", "https://")) and not url.startswith(site_url + "/")]
|
||||
for url in list(dict.fromkeys(remote))[:MAX_INLINE_IMAGES]:
|
||||
media = _fetch_remote_photo(url)
|
||||
if media:
|
||||
local = f"{site_url}/media/{media['path']}"
|
||||
# As written in Markdown, and as it may be escaped in HTML attributes
|
||||
content = content.replace(url, local).replace(escape(url), local)
|
||||
return content
|
||||
|
||||
|
||||
def _save_uploads(files) -> list[dict]:
|
||||
@@ -513,7 +549,7 @@ def _check_type(data: dict) -> None:
|
||||
|
||||
|
||||
def _note_from_url(url: Optional[str], deleted: bool = False):
|
||||
"""Find the note a URL points at (/note/<slug> on this site)"""
|
||||
"""Find the note a URL points at (/note/<slug> or /article/<slug> on this site)"""
|
||||
from urllib.parse import urlparse
|
||||
from starpunk.database import get_db
|
||||
|
||||
@@ -524,7 +560,7 @@ def _note_from_url(url: Optional[str], deleted: bool = False):
|
||||
if parsed.netloc and site.netloc and parsed.netloc != site.netloc:
|
||||
raise MicropubValidationError("URL is not on this site")
|
||||
parts = [p for p in parsed.path.split("/") if p]
|
||||
if len(parts) != 2 or parts[0] not in ("note", "notes"):
|
||||
if len(parts) != 2 or parts[0] not in ("note", "notes", "article", "articles"):
|
||||
raise MicropubValidationError("URL is not a post on this site")
|
||||
slug = parts[1]
|
||||
if deleted:
|
||||
@@ -596,6 +632,10 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
if has_content or not (like_of or repost_of or bookmark_of or photos or title)
|
||||
else ""
|
||||
)
|
||||
content = localize_inline_images(drop_title_heading(content, title))
|
||||
# A summary with content is the author's summary; alone it is the
|
||||
# content (Post Type Discovery), as extract_content does (v2.16.0)
|
||||
summary = extract_summary(properties) if _has_text(properties.get("content")) else None
|
||||
tags = extract_tags(properties)
|
||||
published_date = extract_published_date(properties)
|
||||
published = extract_post_status(properties) and not extract_visibility_draft(properties)
|
||||
@@ -615,7 +655,9 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
note = None
|
||||
try:
|
||||
note = create_note(
|
||||
content=add_title_heading(content, title),
|
||||
content=content,
|
||||
title=title,
|
||||
summary=summary,
|
||||
published=published,
|
||||
created_at=published_date,
|
||||
custom_slug=custom_slug,
|
||||
@@ -636,7 +678,7 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
from starpunk.syndication import resolve_targets, select_targets
|
||||
select_targets(note.id, resolve_targets(requested_targets))
|
||||
|
||||
return "", 201, {"Location": absolute_url_for("public.note", slug=note.slug)}
|
||||
return "", 201, {"Location": note.url}
|
||||
|
||||
except InvalidNoteDataError as e:
|
||||
raise MicropubValidationError(str(e))
|
||||
@@ -686,7 +728,7 @@ def handle_update(data: dict, token_info: dict):
|
||||
|
||||
from starpunk.tags import get_note_tags
|
||||
|
||||
title, body = split_title(note.content)
|
||||
title, summary, body = note.title, note.summary, note.content
|
||||
tags = [t["display_name"] for t in get_note_tags(note.id, entered_order=True)]
|
||||
photos = _note_photos(note)
|
||||
changes = {}
|
||||
@@ -696,6 +738,8 @@ def handle_update(data: dict, token_info: dict):
|
||||
for prop in delete:
|
||||
if prop == "name":
|
||||
title = None
|
||||
elif prop == "summary":
|
||||
summary = None
|
||||
elif prop == "content":
|
||||
body = ""
|
||||
elif prop == "category":
|
||||
@@ -719,10 +763,12 @@ def handle_update(data: dict, token_info: dict):
|
||||
|
||||
if "content" in replace:
|
||||
# Empty text is fine if the note keeps its photos; update_note checks (v2.6.5)
|
||||
body = extract_content(replace) if _has_text(replace["content"]) else ""
|
||||
body = localize_inline_images(extract_content(replace)) if _has_text(replace["content"]) else ""
|
||||
changes["content_format"] = content_format(replace) if body else None
|
||||
if "name" in replace:
|
||||
title = extract_title(replace)
|
||||
if "summary" in replace:
|
||||
summary = extract_summary(replace)
|
||||
if "category" in replace:
|
||||
tags = extract_tags(replace)
|
||||
if "photo" in replace:
|
||||
@@ -738,17 +784,22 @@ def handle_update(data: dict, token_info: dict):
|
||||
if "photo" in add:
|
||||
photos += extract_photos(add)
|
||||
|
||||
# Text may go only if photos stay (or it's a like, repost or bookmark) (v2.6.5)
|
||||
if not (body or "").strip() and not (photos or note.reacts_to):
|
||||
raise MicropubValidationError("A post needs text or a photo")
|
||||
# Text may go only if photos stay, it has a title, or it's a like, repost
|
||||
# or bookmark (v2.6.5)
|
||||
if not (body or "").strip() and not (photos or title or note.reacts_to):
|
||||
raise MicropubValidationError("A post needs text, a title or a photo")
|
||||
|
||||
# Photos must all resolve before anything changes (v2.9.0)
|
||||
if "photo" in touched:
|
||||
media_ids, captions = _resolve_photos(photos)
|
||||
|
||||
new_content = add_title_heading(body, title)
|
||||
if new_content != note.content:
|
||||
changes["content"] = new_content
|
||||
if title != note.title:
|
||||
changes["title"] = title
|
||||
if summary != note.summary:
|
||||
changes["summary"] = summary
|
||||
body = drop_title_heading(body, title)
|
||||
if body != note.content:
|
||||
changes["content"] = body
|
||||
elif "content_format" in changes and changes["content_format"] == note.content_format:
|
||||
del changes["content_format"]
|
||||
if {"category"} & touched:
|
||||
@@ -769,6 +820,11 @@ def handle_update(data: dict, token_info: dict):
|
||||
current_app.logger.error(f"Failed to update note via Micropub: {e}")
|
||||
raise MicropubError("server_error", "Failed to update post", status_code=500)
|
||||
|
||||
# Adding or removing a title moves the post between /note/ and /article/:
|
||||
# the spec says to answer 201 with the new URL (v2.17.0)
|
||||
if bool(title) != bool(note.title):
|
||||
from starpunk.utils import post_url
|
||||
return "", 201, {"Location": post_url(note.slug, title)}
|
||||
return "", 204
|
||||
|
||||
|
||||
@@ -800,20 +856,22 @@ def note_to_mf2(note, wanted: Optional[list] = None) -> dict:
|
||||
"""
|
||||
A note as Microformats2 JSON, the inverse of create
|
||||
|
||||
A '# Heading' first line is the title, returned as name.
|
||||
The title is returned as name and the author's summary as summary.
|
||||
"""
|
||||
from starpunk.syndication import get_note_syndications
|
||||
|
||||
title, content = split_title(note.content)
|
||||
content = note.content
|
||||
props = {
|
||||
# HTML-authored content comes back as HTML (3.7.2, v2.9.0)
|
||||
"content": [{"html": content} if note.content_format == "html" else content],
|
||||
"published": [iso_utc(note.created_at)],
|
||||
"url": [absolute_url_for("public.note", slug=note.slug)],
|
||||
"url": [note.url],
|
||||
"post-status": ["published" if note.published else "draft"],
|
||||
}
|
||||
if title:
|
||||
props["name"] = [title]
|
||||
if note.title:
|
||||
props["name"] = [note.title]
|
||||
if note.summary:
|
||||
props["summary"] = [note.summary]
|
||||
from starpunk.tags import get_note_tags
|
||||
|
||||
tags = get_note_tags(note.id, entered_order=True)
|
||||
|
||||
+67
-1
@@ -344,7 +344,65 @@ def apply_migration(conn, migration_name, migration_path, logger=None):
|
||||
raise MigrationError(error_msg)
|
||||
|
||||
|
||||
def run_migrations(db_path, logger=None):
|
||||
def _move_note_titles(conn, data_dir, logger):
|
||||
"""
|
||||
Migration 023 data step: move each note's '# Heading' title into notes.title
|
||||
|
||||
Runs once, inside the migration transaction. Rewrites the note files without
|
||||
the heading and returns [(path, original text)] so a failed migration can
|
||||
put them back (see _restore_files).
|
||||
"""
|
||||
from starpunk.models import split_title
|
||||
from starpunk.utils import calculate_content_hash, find_trashed_file, write_note_file
|
||||
|
||||
data_dir = Path(data_dir)
|
||||
written = []
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"SELECT id, slug, file_path FROM notes WHERE title IS NULL"
|
||||
).fetchall()
|
||||
for note_id, slug, file_path in rows:
|
||||
path = data_dir / file_path
|
||||
if not path.exists():
|
||||
path = find_trashed_file(data_dir, file_path)
|
||||
if path is None:
|
||||
logger.warning(f"Migration 023: no file for note {slug}, title not moved")
|
||||
continue
|
||||
text = path.read_text(encoding="utf-8")
|
||||
title, body = split_title(text)
|
||||
if title is None:
|
||||
continue
|
||||
conn.execute(
|
||||
"UPDATE notes SET title = ?, content_hash = ? WHERE id = ?",
|
||||
(title, calculate_content_hash(body), note_id),
|
||||
)
|
||||
write_note_file(path, body)
|
||||
written.append((path, text))
|
||||
if written and table_exists(conn, "notes_fts"):
|
||||
# Emptied so startup rebuilds it with the titles (starpunk/__init__.py)
|
||||
conn.execute("DELETE FROM notes_fts")
|
||||
logger.info(f"Migration 023: moved {len(written)} titles out of note files")
|
||||
return written
|
||||
except Exception:
|
||||
_restore_files(written, logger)
|
||||
raise
|
||||
|
||||
|
||||
def _restore_files(written, logger):
|
||||
"""Put back note files changed by a data step whose migration failed"""
|
||||
for path, text in written:
|
||||
try:
|
||||
path.write_text(text, encoding="utf-8")
|
||||
except OSError as e:
|
||||
logger.critical(f"Could not restore {path} after failed migration: {e}")
|
||||
|
||||
|
||||
# Python steps run right after a migration's SQL, in the same transaction.
|
||||
# Each returns [(path, original text)] for files it changed.
|
||||
DATA_MIGRATIONS = {"023_note_titles.sql": _move_note_titles}
|
||||
|
||||
|
||||
def run_migrations(db_path, logger=None, data_dir=None):
|
||||
"""
|
||||
Run all pending database migrations with concurrency protection
|
||||
|
||||
@@ -374,6 +432,8 @@ def run_migrations(db_path, logger=None):
|
||||
Args:
|
||||
db_path: Path to SQLite database file
|
||||
logger: Optional logger for output
|
||||
data_dir: Data directory, for migrations that also change note files
|
||||
(DATA_MIGRATIONS); skipped when None
|
||||
|
||||
Raises:
|
||||
MigrationError: If any migration fails to apply or lock cannot be acquired
|
||||
@@ -407,6 +467,7 @@ def run_migrations(db_path, logger=None):
|
||||
# BEGIN IMMEDIATE acquires RESERVED lock, preventing other writes
|
||||
# but allowing reads. Escalates to EXCLUSIVE during actual writes.
|
||||
conn.execute("BEGIN IMMEDIATE")
|
||||
written = [] # note files changed by DATA_MIGRATIONS, restored on failure
|
||||
|
||||
try:
|
||||
# Ensure migrations tracking table exists
|
||||
@@ -510,6 +571,9 @@ def run_migrations(db_path, logger=None):
|
||||
|
||||
# Execute migration inside the locked transaction
|
||||
execute_in_transaction(conn, migration_sql)
|
||||
data_step = DATA_MIGRATIONS.get(migration_name)
|
||||
if data_step and data_dir is not None:
|
||||
written += data_step(conn, data_dir, logger)
|
||||
|
||||
# Record migration as applied
|
||||
conn.execute(
|
||||
@@ -547,6 +611,7 @@ def run_migrations(db_path, logger=None):
|
||||
|
||||
except MigrationError:
|
||||
# Migration error - rollback and re-raise
|
||||
_restore_files(written, logger)
|
||||
try:
|
||||
conn.rollback()
|
||||
except Exception as rollback_error:
|
||||
@@ -556,6 +621,7 @@ def run_migrations(db_path, logger=None):
|
||||
|
||||
except Exception as e:
|
||||
# Unexpected error during migration - rollback and wrap
|
||||
_restore_files(written, logger)
|
||||
try:
|
||||
conn.rollback()
|
||||
except Exception as rollback_error:
|
||||
|
||||
+57
-113
@@ -36,9 +36,9 @@ DEFAULT_AUTH_STATE_EXPIRY_MINUTES = 5
|
||||
DEFAULT_TOKEN_EXPIRY_DAYS = 90
|
||||
|
||||
# Constants - Markdown rendering
|
||||
from starpunk.markdown_ext import BareUrlExtension
|
||||
from starpunk.markdown_ext import BareUrlExtension, HashtagLineExtension
|
||||
|
||||
MARKDOWN_EXTENSIONS = ["extra", "codehilite", "nl2br", BareUrlExtension()]
|
||||
MARKDOWN_EXTENSIONS = ["extra", "codehilite", "nl2br", BareUrlExtension(), HashtagLineExtension()]
|
||||
|
||||
# Constants - Content limits
|
||||
MAX_TITLE_LENGTH = 200
|
||||
@@ -62,7 +62,9 @@ ALLOWED_ATTRIBUTES = {
|
||||
def render_markdown(text: str) -> str:
|
||||
"""Note Markdown (or HTML) to sanitized HTML"""
|
||||
html = markdown.markdown(text, extensions=MARKDOWN_EXTENSIONS)
|
||||
return nh3.clean(html, tags=ALLOWED_TAGS, attributes=ALLOWED_ATTRIBUTES)
|
||||
html = nh3.clean(html, tags=ALLOWED_TAGS, attributes=ALLOWED_ATTRIBUTES)
|
||||
# A wrapper lets a table fill the column yet scroll sideways when too wide (v2.18.2)
|
||||
return re.sub(r"<table\b", '<div class="table-wrap"><table', html).replace("</table>", "</table></div>")
|
||||
|
||||
|
||||
# Likes, reposts and bookmarks (v2.5.0, v2.9.0)
|
||||
@@ -76,11 +78,14 @@ EXCERPT_LENGTH = 200
|
||||
|
||||
def split_title(content: str) -> tuple[Optional[str], str]:
|
||||
"""
|
||||
Split a note's '# Heading' first line (its title, per Q22) off its content
|
||||
Split a '# Heading' first line off Markdown content
|
||||
|
||||
Before v2.16.0 an article's title was its '# Heading' first line (Q22).
|
||||
Titles now live in notes.title (ADR-076); migration 023 uses this to move
|
||||
the old headings there, and Micropub to drop a heading that repeats 'name'.
|
||||
|
||||
The first line must be a Markdown heading: 1-6 '#' then a space and text.
|
||||
'#indieweb ...' (a hashtag) or a bare '#' is not a title. This is the one
|
||||
place that decides whether a note has a title (templates use has_title).
|
||||
'#indieweb ...' (a hashtag) or a bare '#' is not a title.
|
||||
|
||||
Returns:
|
||||
(title, remaining content); title is None if the note has no heading
|
||||
@@ -171,6 +176,8 @@ class Note:
|
||||
repost_of: Optional[str] = None # v2.5.0: URL reposted (this note is a repost)
|
||||
bookmark_of: Optional[str] = None # v2.9.0: URL bookmarked (this note is a bookmark)
|
||||
content_format: Optional[str] = None # v2.9.0: 'html' if sent as HTML via Micropub
|
||||
title: Optional[str] = None # v2.16.0: an article's title (p-name), ADR-076
|
||||
summary: Optional[str] = None # v2.16.0: author-written summary (p-summary)
|
||||
_cached_content: Optional[str] = field(
|
||||
default=None, repr=False, compare=False, init=False
|
||||
)
|
||||
@@ -233,6 +240,8 @@ class Note:
|
||||
repost_of=data.get("repost_of"),
|
||||
bookmark_of=data.get("bookmark_of"),
|
||||
content_format=data.get("content_format"),
|
||||
title=data.get("title"),
|
||||
summary=data.get("summary"),
|
||||
)
|
||||
|
||||
@property
|
||||
@@ -291,8 +300,13 @@ class Note:
|
||||
<p>This is my note content...</p>
|
||||
"""
|
||||
if self._cached_html is None:
|
||||
# Render markdown to sanitized HTML (v2.9.0)
|
||||
html = render_markdown(self.content) # triggers lazy load of content
|
||||
if self.content_format == "html":
|
||||
# Sent as HTML via Micropub: sanitize it as is. Through Markdown
|
||||
# it picked up stray <br>s and empty paragraphs (v2.16.0)
|
||||
html = nh3.clean(self.content, tags=ALLOWED_TAGS, attributes=ALLOWED_ATTRIBUTES)
|
||||
else:
|
||||
# Render markdown to sanitized HTML (v2.9.0)
|
||||
html = render_markdown(self.content) # triggers lazy load of content
|
||||
|
||||
# Cache it
|
||||
object.__setattr__(self, "_cached_html", html)
|
||||
@@ -300,124 +314,40 @@ class Note:
|
||||
return self._cached_html
|
||||
|
||||
@property
|
||||
def body_html(self) -> str:
|
||||
"""
|
||||
Rendered HTML without the title heading
|
||||
|
||||
Pages show a titled note's title as their own heading, so the body
|
||||
leaves it out to avoid showing it twice. Untitled notes: same as html.
|
||||
"""
|
||||
title, body = split_title(self.content)
|
||||
if title is None:
|
||||
return self.html
|
||||
return render_markdown(body)
|
||||
|
||||
@property
|
||||
def has_title(self) -> bool:
|
||||
"""True if the note starts with a '# Heading' title (an article)"""
|
||||
try:
|
||||
return split_title(self.content)[0] is not None
|
||||
except (FileNotFoundError, OSError):
|
||||
return False
|
||||
|
||||
@property
|
||||
def title(self) -> str:
|
||||
"""
|
||||
Extract title from content
|
||||
|
||||
Returns first line of content, or uses slug as fallback.
|
||||
Strips markdown heading syntax (# ) if present.
|
||||
|
||||
Returns:
|
||||
Title string
|
||||
|
||||
Examples:
|
||||
>>> # Content: "# My First Note\\n\\nContent here..."
|
||||
>>> note.title
|
||||
'My First Note'
|
||||
|
||||
>>> # Content: "Just a note without heading"
|
||||
>>> note.title
|
||||
'Just a note without heading'
|
||||
"""
|
||||
try:
|
||||
# Get content (may trigger lazy load)
|
||||
content = self.content
|
||||
|
||||
# Split into lines and get first non-empty line
|
||||
lines = content.strip().split("\n")
|
||||
first_line = ""
|
||||
for line in lines:
|
||||
if line.strip():
|
||||
first_line = line.strip()
|
||||
break
|
||||
|
||||
if not first_line:
|
||||
# No content, use slug
|
||||
return self.slug
|
||||
|
||||
# Strip markdown heading syntax
|
||||
if first_line.startswith("#"):
|
||||
# Remove leading # characters and whitespace
|
||||
first_line = first_line.lstrip("#").strip()
|
||||
|
||||
# Truncate to max length
|
||||
if len(first_line) > MAX_TITLE_LENGTH:
|
||||
first_line = first_line[:MAX_TITLE_LENGTH]
|
||||
|
||||
return first_line if first_line else self.slug
|
||||
|
||||
except (FileNotFoundError, OSError):
|
||||
# If file doesn't exist, use slug
|
||||
return self.slug
|
||||
def label(self) -> str:
|
||||
"""A name for places that always need one: the title, else the excerpt, else the slug"""
|
||||
return self.title or " ".join(self.excerpt.split())[:MAX_TITLE_LENGTH] or self.slug
|
||||
|
||||
@property
|
||||
def excerpt(self) -> str:
|
||||
"""
|
||||
Generate short excerpt for previews
|
||||
|
||||
Returns first 200 characters of content (plain text, no markdown).
|
||||
Strips markdown formatting and adds ellipsis if truncated.
|
||||
The first 200 characters of the note as plain text: Markdown (or
|
||||
HTML) rendered and stripped, images left out, links as their text
|
||||
(v2.16.0). Cut at a word boundary with an ellipsis if longer.
|
||||
|
||||
Returns:
|
||||
Excerpt string
|
||||
|
||||
Examples:
|
||||
>>> note.excerpt
|
||||
'This is my note content. It has some interesting points...'
|
||||
'This is my note content. It has some interesting points…'
|
||||
"""
|
||||
from starpunk.syndication import plain_text
|
||||
|
||||
try:
|
||||
# Leave out the title; previews show it separately
|
||||
_, content = split_title(self.content)
|
||||
|
||||
# Remove markdown heading syntax
|
||||
lines = content.split("\n")
|
||||
cleaned_lines = []
|
||||
for line in lines:
|
||||
if line.strip().startswith("#"):
|
||||
# Remove heading syntax
|
||||
cleaned_lines.append(line.lstrip("#").strip())
|
||||
else:
|
||||
cleaned_lines.append(line)
|
||||
|
||||
cleaned_content = "\n".join(cleaned_lines)
|
||||
|
||||
# Take first EXCERPT_LENGTH characters
|
||||
excerpt = cleaned_content.strip()[:EXCERPT_LENGTH]
|
||||
|
||||
# Truncate to word boundary
|
||||
if len(cleaned_content.strip()) > EXCERPT_LENGTH:
|
||||
# Find last space in excerpt
|
||||
text = " ".join(plain_text(self.content).split())
|
||||
if len(text) > EXCERPT_LENGTH:
|
||||
excerpt = text[:EXCERPT_LENGTH]
|
||||
last_space = excerpt.rfind(" ")
|
||||
if last_space > 0:
|
||||
excerpt = excerpt[:last_space]
|
||||
excerpt += "..."
|
||||
text = (excerpt[:last_space] if last_space > 0 else excerpt).rstrip() + "…"
|
||||
|
||||
if not excerpt and self.reacts_to:
|
||||
if not text and self.reacts_to:
|
||||
# A like, repost or bookmark with no comment
|
||||
return f"{REACTION_VERBS[self.kind]} {self.reacts_to}"
|
||||
|
||||
return excerpt
|
||||
return text
|
||||
|
||||
except (FileNotFoundError, OSError):
|
||||
# If file doesn't exist, return slug
|
||||
@@ -455,14 +385,27 @@ class Note:
|
||||
"""
|
||||
Generate permalink (public URL path)
|
||||
|
||||
Returns:
|
||||
URL path string (e.g., '/note/my-first-note')
|
||||
Articles (notes with a title) live under /article/, other notes
|
||||
under /note/ (v2.17.0).
|
||||
|
||||
Examples:
|
||||
>>> note.permalink
|
||||
'/note/my-first-note'
|
||||
Returns:
|
||||
URL path string (e.g., '/note/my-first-note', '/article/my-essay')
|
||||
"""
|
||||
return f"/note/{self.slug}"
|
||||
from starpunk.utils import post_path
|
||||
|
||||
return post_path(self.slug, self.title)
|
||||
|
||||
@property
|
||||
def endpoint(self) -> str:
|
||||
"""The route that shows this note: public.article or public.note (v2.17.0)"""
|
||||
return "public.article" if self.title else "public.note"
|
||||
|
||||
@property
|
||||
def url(self) -> str:
|
||||
"""Absolute permalink, built from SITE_URL (needs an app context)"""
|
||||
from starpunk.utils import post_url
|
||||
|
||||
return post_url(self.slug, self.title)
|
||||
|
||||
@property
|
||||
def is_published(self) -> bool:
|
||||
@@ -533,6 +476,7 @@ class Note:
|
||||
"id": self.id,
|
||||
"slug": self.slug,
|
||||
"title": self.title,
|
||||
"summary": self.summary,
|
||||
"published": self.published,
|
||||
"created_at": self.created_at.strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"updated_at": self.updated_at.strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
|
||||
+49
-26
@@ -40,6 +40,7 @@ from starpunk.utils import (
|
||||
calculate_content_hash,
|
||||
validate_note_path,
|
||||
validate_slug,
|
||||
post_url,
|
||||
)
|
||||
|
||||
|
||||
@@ -149,6 +150,8 @@ def create_note(
|
||||
has_media: bool = False,
|
||||
bookmark_of: Optional[str] = None,
|
||||
content_format: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
summary: Optional[str] = None,
|
||||
) -> Note:
|
||||
"""
|
||||
Create a new note
|
||||
@@ -164,6 +167,8 @@ def create_note(
|
||||
custom_slug: Optional custom slug (from Micropub mp-slug property)
|
||||
tags: Optional list of tag display names (v1.3.0)
|
||||
in_reply_to: Optional URL being replied to (v1.6.0)
|
||||
title: Optional title; a titled note is an article (v2.16.0, ADR-076)
|
||||
summary: Optional author-written summary (v2.16.0)
|
||||
|
||||
Returns:
|
||||
Note object with all metadata and content loaded
|
||||
@@ -176,7 +181,7 @@ def create_note(
|
||||
|
||||
Examples:
|
||||
>>> # Create unpublished draft
|
||||
>>> note = create_note("# My First Note\\n\\nContent here.", published=False)
|
||||
>>> note = create_note("Content here.", title="My First Note", published=False)
|
||||
>>> print(note.slug)
|
||||
'my-first-note'
|
||||
|
||||
@@ -219,7 +224,9 @@ def create_note(
|
||||
"like_of", like_of, "A note can only be one of a like, repost or bookmark"
|
||||
)
|
||||
content = content or ""
|
||||
if not content.strip() and not (like_of or repost_of or bookmark_of or has_media):
|
||||
title = (title or "").strip() or None
|
||||
summary = (summary or "").strip() or None
|
||||
if not content.strip() and not (like_of or repost_of or bookmark_of or has_media or title):
|
||||
raise InvalidNoteDataError(
|
||||
"content", content, "Content cannot be empty or whitespace-only"
|
||||
)
|
||||
@@ -244,9 +251,11 @@ def create_note(
|
||||
if not success:
|
||||
raise InvalidNoteDataError("slug", custom_slug, error)
|
||||
else:
|
||||
# Generate timestamp-based slug (ADR-062)
|
||||
from starpunk.slug_utils import generate_timestamp_slug
|
||||
slug = generate_timestamp_slug(created_at, existing_slugs)
|
||||
# Articles take their slug from the title (v2.17.0); other notes, or a
|
||||
# title with nothing usable in it, get a timestamp slug (ADR-062)
|
||||
from starpunk.slug_utils import generate_timestamp_slug, generate_title_slug
|
||||
slug = (generate_title_slug(title, existing_slugs) if title else None) \
|
||||
or generate_timestamp_slug(created_at, existing_slugs)
|
||||
|
||||
# 4. GENERATE FILE PATH
|
||||
note_path = generate_note_path(slug, created_at, data_dir)
|
||||
@@ -281,11 +290,12 @@ def create_note(
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO notes (slug, file_path, published, created_at, updated_at, content_hash,
|
||||
in_reply_to, like_of, repost_of, bookmark_of, content_format)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
in_reply_to, like_of, repost_of, bookmark_of, content_format,
|
||||
title, summary)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(slug, file_path_rel, published, created_at, updated_at, content_hash,
|
||||
in_reply_to, like_of, repost_of, bookmark_of, content_format),
|
||||
in_reply_to, like_of, repost_of, bookmark_of, content_format, title, summary),
|
||||
)
|
||||
db.commit()
|
||||
except Exception as e:
|
||||
@@ -327,7 +337,7 @@ def create_note(
|
||||
from starpunk.search import update_fts_index, has_fts_table
|
||||
db_path = Path(current_app.config["DATABASE_PATH"])
|
||||
if has_fts_table(db_path):
|
||||
update_fts_index(db, note_id, slug, content)
|
||||
update_fts_index(db, note_id, slug, content, title)
|
||||
db.commit()
|
||||
except Exception as e:
|
||||
# FTS update failure should not prevent note creation
|
||||
@@ -341,7 +351,7 @@ def create_note(
|
||||
db_path = Path(current_app.config["DATABASE_PATH"])
|
||||
site_url = current_app.config["SITE_URL"]
|
||||
from starpunk.utils import absolute_url_for
|
||||
note_url = absolute_url_for("public.note", slug=slug)
|
||||
note_url = post_url(slug, title)
|
||||
|
||||
# Get rendered HTML from note
|
||||
html_content = note.html
|
||||
@@ -641,11 +651,14 @@ def update_note(
|
||||
in_reply_to: Optional[str] = None,
|
||||
has_media: bool = False,
|
||||
content_format=UNCHANGED,
|
||||
title=UNCHANGED,
|
||||
summary=UNCHANGED,
|
||||
) -> Note:
|
||||
"""
|
||||
Update a note's content and/or published status
|
||||
|
||||
content_format (v2.9.0): 'html' or None (Markdown); left alone unless given
|
||||
title, summary (v2.16.0): new value, None or "" to remove; left alone unless given
|
||||
|
||||
Updates note content and/or metadata, maintaining atomic synchronization
|
||||
between file and database. At least one of content or published must
|
||||
@@ -713,8 +726,15 @@ def update_note(
|
||||
if slug is not None and id is not None:
|
||||
raise ValueError("Cannot provide both slug and id")
|
||||
|
||||
if content is None and published is None and tags is None and in_reply_to is None:
|
||||
raise ValueError("Must provide at least one of content, published, tags, or in_reply_to to update")
|
||||
if (content is None and published is None and tags is None and in_reply_to is None
|
||||
and title is UNCHANGED and summary is UNCHANGED):
|
||||
raise ValueError(
|
||||
"Must provide at least one of content, published, tags, in_reply_to, title or summary to update"
|
||||
)
|
||||
if title is not UNCHANGED:
|
||||
title = (title or "").strip() or None
|
||||
if summary is not UNCHANGED:
|
||||
summary = (summary or "").strip() or None
|
||||
|
||||
# Validate content if provided (checked against the note below for
|
||||
# likes/reposts, whose comment may be empty)
|
||||
@@ -729,7 +749,8 @@ def update_note(
|
||||
raise NoteNotFoundError(identifier)
|
||||
old_tag_names = _tag_names(existing_note.id) # for WebSub tag topics (v2.10.0)
|
||||
|
||||
if empty_content and not (existing_note.reacts_to or has_media):
|
||||
new_title = existing_note.title if title is UNCHANGED else title
|
||||
if empty_content and not (existing_note.reacts_to or has_media or new_title):
|
||||
raise InvalidNoteDataError(
|
||||
"content", content, "Content cannot be empty or whitespace-only"
|
||||
)
|
||||
@@ -797,6 +818,14 @@ def update_note(
|
||||
update_fields.append("content_format = ?")
|
||||
params.append(content_format)
|
||||
|
||||
if title is not UNCHANGED:
|
||||
update_fields.append("title = ?")
|
||||
params.append(title)
|
||||
|
||||
if summary is not UNCHANGED:
|
||||
update_fields.append("summary = ?")
|
||||
params.append(summary)
|
||||
|
||||
# Add WHERE clause parameter
|
||||
if slug is not None:
|
||||
where_clause = "slug = ?"
|
||||
@@ -832,13 +861,14 @@ def update_note(
|
||||
# Tag update failure should not prevent note update
|
||||
current_app.logger.warning(f"Failed to update tags for note {existing_note.slug}: {e}")
|
||||
|
||||
# 7. UPDATE FTS INDEX (if available and content changed)
|
||||
if content is not None:
|
||||
# 7. UPDATE FTS INDEX (if available and content or title changed)
|
||||
if content is not None or title is not UNCHANGED:
|
||||
try:
|
||||
from starpunk.search import update_fts_index, has_fts_table
|
||||
db_path = Path(current_app.config["DATABASE_PATH"])
|
||||
if has_fts_table(db_path):
|
||||
update_fts_index(db, existing_note.id, existing_note.slug, content)
|
||||
update_fts_index(db, existing_note.id, existing_note.slug,
|
||||
existing_note.content if content is None else content, new_title)
|
||||
db.commit()
|
||||
except Exception as e:
|
||||
# FTS update failure should not prevent note update
|
||||
@@ -876,7 +906,7 @@ def update_note(
|
||||
db_path = Path(current_app.config["DATABASE_PATH"])
|
||||
site_url = current_app.config["SITE_URL"]
|
||||
from starpunk.utils import absolute_url_for
|
||||
note_url = absolute_url_for("public.note", slug=updated_note.slug)
|
||||
note_url = updated_note.url
|
||||
|
||||
# Get rendered HTML from updated note (if published)
|
||||
html_content = updated_note.html if updated_note.published else None
|
||||
@@ -1129,7 +1159,7 @@ def undelete_note(slug: str) -> Note:
|
||||
NoteNotFoundError: If there is no soft-deleted note with this slug
|
||||
"""
|
||||
import shutil
|
||||
from starpunk.utils import TRASH_DIR_NAME
|
||||
from starpunk.utils import find_trashed_file
|
||||
|
||||
db = get_db(current_app)
|
||||
row = db.execute(
|
||||
@@ -1142,14 +1172,7 @@ def undelete_note(slug: str) -> Note:
|
||||
data_dir = Path(current_app.config["DATA_PATH"])
|
||||
note_path = data_dir / row["file_path"]
|
||||
if not note_path.exists():
|
||||
# delete_note_file put it in .trash/<year>/<month>/<name>
|
||||
parts = Path(row["file_path"]).parts
|
||||
candidates = []
|
||||
if "notes" in parts and len(parts) > parts.index("notes") + 2:
|
||||
i = parts.index("notes")
|
||||
candidates.append(data_dir / TRASH_DIR_NAME / parts[i + 1] / parts[i + 2] / note_path.name)
|
||||
candidates += sorted((data_dir / TRASH_DIR_NAME).glob(f"*/*/{note_path.name}"))
|
||||
source = next((c for c in candidates if c.exists()), None)
|
||||
source = find_trashed_file(data_dir, row["file_path"])
|
||||
if source is None:
|
||||
raise NoteSyncError("undelete", f"Trashed file not found for {slug}", "Cannot restore note file")
|
||||
note_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -24,6 +24,8 @@ import time
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
from urllib.parse import parse_qsl, urlsplit
|
||||
|
||||
from starpunk.utils import post_path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
RESPONSE_EVENTS = ("reply", "mention", "quote", "like", "repost", "bookmark")
|
||||
@@ -142,7 +144,7 @@ def _note_label(conn, note_id: int) -> str:
|
||||
|
||||
try:
|
||||
note = get_note(id=note_id, load_content=True)
|
||||
label = " ".join((note.title or "").split()) if note else ""
|
||||
label = note.label if note else ""
|
||||
except Exception:
|
||||
label = ""
|
||||
return (label[:57] + "…") if len(label) > 60 else (label or "a note")
|
||||
@@ -158,7 +160,7 @@ def _messages(rows, site_url: str, conn) -> List[Tuple[str, str, List[int]]]:
|
||||
who = (row["author_name"] or "someone") + (f" via {via}" if via else "")
|
||||
pending = row["moderation_status"] == "pending"
|
||||
link = (f"{site}/admin/webmentions/{row['id']}" if pending
|
||||
else f"{site}/note/{row['slug']}#responses")
|
||||
else f"{site}{post_path(row['slug'], row['note_title'])}#responses")
|
||||
if row["mention_type"] in GROUPED:
|
||||
groups.setdefault((row["target_note_id"], row["mention_type"], row["slug"], pending), []).append(
|
||||
(row["id"], row["author_name"] or "someone", link))
|
||||
@@ -206,7 +208,7 @@ class Sweeper:
|
||||
logger.warning(f"Dropped {dropped} notification(s) older than {DROP_AFTER_HOURS} hours")
|
||||
rows = conn.execute(
|
||||
"""
|
||||
SELECT w.*, n.slug FROM webmentions w JOIN notes n ON n.id = w.target_note_id
|
||||
SELECT w.*, n.slug, n.title AS note_title FROM webmentions w JOIN notes n ON n.id = w.target_note_id
|
||||
WHERE w.notified_at IS NULL AND w.validation_status = 'valid'
|
||||
AND w.moderation_status IN ('approved', 'pending')
|
||||
ORDER BY w.id LIMIT 100
|
||||
|
||||
+452
-212
@@ -7,41 +7,79 @@ Per database-schema.md Story 2.4:
|
||||
- Store in reply_contexts table
|
||||
- Graceful degradation on fetch failure
|
||||
|
||||
Fediverse and Bluesky pages carry no h-entry, so from v2.19.0 the context
|
||||
comes from the first source that knows the post:
|
||||
|
||||
1. Bluesky post URLs: the public Bluesky AppView API
|
||||
2. Pages with an h-entry: microformats (IndieWeb sites)
|
||||
3. Fediverse posts: the origin's public Mastodon API, else this site's
|
||||
GoToSocial instance resolving the URL (as likes and boosts already do;
|
||||
it fetches with signed requests, so it works for GoToSocial and for
|
||||
Mastodon servers that refuse anonymous API calls)
|
||||
4. Anything else: the page's OpenGraph tags, with the page's h-card as author
|
||||
|
||||
The source used is stored in reply_contexts.source. Rows from before v2.19.0
|
||||
have none; refresh_stale() fetches those again once.
|
||||
|
||||
References:
|
||||
- Database Schema: /docs/design/v1.6.0/database-schema.md
|
||||
- Background Worker: /docs/design/v1.6.0/background-worker.md
|
||||
- Design: /docs/design/v2.19.0/2026-11-13-reply-context-sources.md
|
||||
"""
|
||||
|
||||
import html
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import sqlite3
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional, Dict, Any
|
||||
from urllib.parse import quote, urljoin, urlparse
|
||||
|
||||
import mf2py
|
||||
|
||||
from starpunk import mf2_utils
|
||||
from starpunk.webmention_processor import (
|
||||
fetch_source,
|
||||
extract_author,
|
||||
extract_content,
|
||||
ValidationError
|
||||
sanitize_content,
|
||||
ValidationError,
|
||||
_is_http_url,
|
||||
)
|
||||
import mf2py
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
SNIPPET_LIMIT = 500
|
||||
|
||||
BLUESKY_POST_RE = re.compile(r"^https://bsky\.app/profile/([^/]+)/post/([^/?#]+)")
|
||||
BLUESKY_API = "https://public.api.bsky.app/xrpc/app.bsky.feed.getPostThread"
|
||||
# Mastodon: https://host/@user/123456 (numeric ids; the public API may answer)
|
||||
MASTODON_STATUS_RE = re.compile(r"^/@[^/]+/(\d+)/?$")
|
||||
# Fediverse post paths: Mastodon, GoToSocial, ActivityPub ids, Misskey,
|
||||
# Pleroma/Akkoma
|
||||
FEDIVERSE_PATH_RE = re.compile(
|
||||
r"^/(?:@[^/]+/(?:statuses/)?[A-Za-z0-9]+|users/[^/]+/statuses/[A-Za-z0-9]+|"
|
||||
r"notes/[A-Za-z0-9]+|notice/[A-Za-z0-9]+|objects/[A-Za-z0-9-]+)/?$")
|
||||
# og:title of fediverse posts: "Display Name (@user@host)"
|
||||
FEDI_TITLE_RE = re.compile(r"^(.*?)\s*\((@[^)\s]+)\)\s*$")
|
||||
|
||||
FIELDS = ('title', 'content_snippet', 'published_at', 'author_name', 'author_handle',
|
||||
'author_url', 'author_photo', 'site_name', 'photo', 'photo_alt')
|
||||
|
||||
|
||||
def fetch_reply_context(
|
||||
db_path: Path,
|
||||
reply_url: str,
|
||||
site_url: str
|
||||
site_url: str,
|
||||
refresh: bool = False
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Fetch and parse reply context from a URL.
|
||||
|
||||
Per database-schema.md:
|
||||
- Fetch URL content with SSRF protection
|
||||
- Parse h-entry for title, content, published date
|
||||
- Parse h-card for author information
|
||||
- Parse h-entry, a Mastodon status, a Bluesky post or OpenGraph tags
|
||||
- Store in reply_contexts table
|
||||
- Graceful degradation: errors don't prevent reply creation
|
||||
|
||||
@@ -49,208 +87,434 @@ def fetch_reply_context(
|
||||
db_path: Path to SQLite database
|
||||
reply_url: URL being replied to
|
||||
site_url: Site URL for User-Agent header
|
||||
refresh: Fetch again even if a context is stored
|
||||
|
||||
Returns:
|
||||
Dict with fetch status and context data
|
||||
"""
|
||||
# Check if context already exists
|
||||
existing = get_reply_context(db_path, reply_url)
|
||||
# Contexts stored before v2.19.0 (no source) are fetched again
|
||||
if existing and existing.get('source') and not refresh:
|
||||
logger.debug(f"Reply context already cached for {reply_url}")
|
||||
return existing
|
||||
|
||||
try:
|
||||
conn = sqlite3.connect(db_path)
|
||||
conn.row_factory = sqlite3.Row
|
||||
cursor = conn.cursor()
|
||||
found = _find_context(reply_url, site_url)
|
||||
except Exception as e:
|
||||
if isinstance(e, ValidationError):
|
||||
logger.warning(f"Failed to fetch reply context for {reply_url}: {e}")
|
||||
error = str(e)
|
||||
else:
|
||||
logger.error(f"Error fetching reply context for {reply_url}: {e}", exc_info=True)
|
||||
error = f"Unexpected error: {e}"
|
||||
if existing and existing.get('fetch_status') == 'fetched':
|
||||
# Keep what an earlier fetch found; don't try again
|
||||
_mark_source(db_path, reply_url, existing.get('source') or 'legacy')
|
||||
return existing
|
||||
return _store_minimal_context(db_path, reply_url, error)
|
||||
|
||||
cursor.execute("""
|
||||
SELECT * FROM reply_contexts WHERE url = ?
|
||||
""", (reply_url,))
|
||||
context = _store_context(db_path, reply_url, found)
|
||||
_save_images(context, site_url)
|
||||
logger.info(f"Fetched reply context for {reply_url} ({context['source']})")
|
||||
return context
|
||||
|
||||
existing = cursor.fetchone()
|
||||
|
||||
def refresh_stale(db_path: Path, site_url: str, limit: int = 5) -> int:
|
||||
"""
|
||||
Fetch again the contexts stored before v2.19.0 that notes still use.
|
||||
The webmention worker calls this; each row is fetched once, since the
|
||||
new fetch records its source. Returns how many were fetched.
|
||||
"""
|
||||
conn = sqlite3.connect(db_path)
|
||||
try:
|
||||
urls = [row[0] for row in conn.execute(
|
||||
"""
|
||||
SELECT rc.url FROM reply_contexts rc
|
||||
WHERE rc.source IS NULL AND EXISTS (
|
||||
SELECT 1 FROM notes n WHERE n.deleted_at IS NULL AND rc.url IN
|
||||
(n.in_reply_to, n.like_of, n.repost_of, n.bookmark_of))
|
||||
ORDER BY rc.id DESC LIMIT ?
|
||||
""", (limit,))]
|
||||
finally:
|
||||
conn.close()
|
||||
for url in urls:
|
||||
fetch_reply_context(db_path, url, site_url, refresh=True)
|
||||
return len(urls)
|
||||
|
||||
if existing:
|
||||
logger.debug(f"Reply context already cached for {reply_url}")
|
||||
return dict(existing)
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to check existing context: {e}")
|
||||
# --- Sources ----------------------------------------------------------------
|
||||
|
||||
# Fetch and parse context
|
||||
def _find_context(url: str, site_url: str) -> Dict[str, Any]:
|
||||
"""Context fields and 'source' for a URL; raises ValidationError if nothing could be fetched"""
|
||||
if BLUESKY_POST_RE.match(url):
|
||||
try:
|
||||
found = _from_bluesky(url, site_url)
|
||||
except ValidationError as e:
|
||||
logger.info(f"Bluesky API had no post for {url}: {e}")
|
||||
found = None
|
||||
if found:
|
||||
return found
|
||||
|
||||
page_error = None
|
||||
content = content_type = final_url = None
|
||||
try:
|
||||
# Fetch URL content
|
||||
# fetch_source also returns the URL after redirects (v2.7.0); relative
|
||||
# links in the page resolve against it (v2.10.2)
|
||||
content, content_type, final_url = fetch_source(reply_url, site_url)
|
||||
|
||||
# Parse microformats (only for HTML)
|
||||
if 'html' not in content_type.lower():
|
||||
# Not HTML - store minimal context
|
||||
return _store_minimal_context(db_path, reply_url, "Not HTML content")
|
||||
|
||||
parsed_mf2 = mf2py.parse(doc=content, url=final_url)
|
||||
|
||||
# Extract h-entry data
|
||||
entry_data = _extract_entry_data(parsed_mf2)
|
||||
|
||||
# Extract author data
|
||||
author_data = extract_author(parsed_mf2, reply_url)
|
||||
|
||||
# Store in database
|
||||
context = _store_context(
|
||||
db_path,
|
||||
reply_url,
|
||||
entry_data.get('title'),
|
||||
entry_data.get('content_snippet'),
|
||||
entry_data.get('published_at'),
|
||||
author_data.get('author_name'),
|
||||
author_data.get('author_url'),
|
||||
author_data.get('author_photo'),
|
||||
entry_data.get('site_name')
|
||||
)
|
||||
|
||||
logger.info(f"Fetched reply context for {reply_url}")
|
||||
return context
|
||||
|
||||
content, content_type, final_url = fetch_source(url, site_url)
|
||||
except ValidationError as e:
|
||||
# Fetch/validation error - store error status
|
||||
logger.warning(f"Failed to fetch reply context for {reply_url}: {e}")
|
||||
return _store_minimal_context(db_path, reply_url, str(e))
|
||||
page_error = e
|
||||
|
||||
except Exception as e:
|
||||
# Unexpected error - store error status
|
||||
logger.error(f"Error fetching reply context for {reply_url}: {e}", exc_info=True)
|
||||
return _store_minimal_context(db_path, reply_url, f"Unexpected error: {e}")
|
||||
og: Dict[str, Optional[str]] = {}
|
||||
card: Dict[str, Optional[str]] = {}
|
||||
is_html = content is not None and 'html' in (content_type or '').lower()
|
||||
if is_html:
|
||||
parsed = mf2py.parse(doc=content, url=final_url)
|
||||
entry = mf2_utils.responding_entry(parsed, '')
|
||||
og = _opengraph(content, final_url)
|
||||
if entry:
|
||||
return _from_mf2(parsed, entry, final_url, og)
|
||||
card = _page_card(parsed, final_url)
|
||||
|
||||
if _looks_fediverse(url, og):
|
||||
status = _mastodon_api(url, site_url) or _resolve_with_instance(url)
|
||||
if status:
|
||||
return _from_status(status, url)
|
||||
|
||||
if page_error:
|
||||
raise page_error
|
||||
if not is_html:
|
||||
raise ValidationError("Not HTML content")
|
||||
return _from_opengraph(og, card)
|
||||
|
||||
|
||||
def _extract_entry_data(parsed_mf2: Dict) -> Dict[str, Any]:
|
||||
"""
|
||||
Extract entry data from parsed microformats.
|
||||
|
||||
Args:
|
||||
parsed_mf2: Parsed microformats data from mf2py
|
||||
|
||||
Returns:
|
||||
Dict with title, content_snippet, published_at, site_name
|
||||
"""
|
||||
title = None
|
||||
content_snippet = None
|
||||
published_at = None
|
||||
site_name = None
|
||||
|
||||
# Find h-entry
|
||||
for item in parsed_mf2.get('items', []):
|
||||
if 'h-entry' not in item.get('type', []):
|
||||
continue
|
||||
|
||||
props = item.get('properties', {})
|
||||
|
||||
# Extract title (p-name)
|
||||
name = props.get('name', [])
|
||||
if name:
|
||||
title = name[0]
|
||||
if isinstance(title, dict):
|
||||
title = title.get('value', '')
|
||||
# Truncate if too long
|
||||
if title and len(title) > 200:
|
||||
title = title[:197] + '...'
|
||||
|
||||
# Extract content (reuse processor function)
|
||||
content_snippet = extract_content(parsed_mf2)
|
||||
|
||||
# Extract published date
|
||||
published = props.get('published', [])
|
||||
if published:
|
||||
pub_str = published[0]
|
||||
if isinstance(pub_str, dict):
|
||||
pub_str = pub_str.get('value', '')
|
||||
# Try to parse ISO 8601 datetime
|
||||
try:
|
||||
from dateutil import parser as dateutil_parser
|
||||
published_at = dateutil_parser.parse(pub_str)
|
||||
except Exception:
|
||||
# Parsing failed, leave as None
|
||||
pass
|
||||
|
||||
break # Use first h-entry
|
||||
|
||||
# Try to extract site name from h-card
|
||||
for item in parsed_mf2.get('items', []):
|
||||
if 'h-card' in item.get('type', []):
|
||||
props = item.get('properties', {})
|
||||
name = props.get('name', [])
|
||||
if name:
|
||||
site_name = name[0]
|
||||
if isinstance(site_name, dict):
|
||||
site_name = site_name.get('value', '')
|
||||
break
|
||||
|
||||
def _from_mf2(parsed: dict, entry: dict, page_url: str, og: dict) -> Dict[str, Any]:
|
||||
"""An IndieWeb post's h-entry"""
|
||||
author = mf2_utils.entry_author(parsed, entry, page_url)
|
||||
snippet = extract_content(parsed, entry)
|
||||
title = mf2_utils.first(entry, 'name')
|
||||
if title and _repeats(title, snippet):
|
||||
title = None # implied name: the content again
|
||||
photo, photo_alt = mf2_utils.first(entry, 'photo'), None
|
||||
for value in entry.get('properties', {}).get('photo', [])[:1]:
|
||||
if isinstance(value, dict):
|
||||
photo_alt = value.get('alt') or None
|
||||
if not _is_http_url(photo):
|
||||
photo, photo_alt = og.get('image'), og.get('image_alt')
|
||||
return {
|
||||
'title': title,
|
||||
'content_snippet': content_snippet,
|
||||
'published_at': published_at,
|
||||
'site_name': site_name
|
||||
'source': 'mf2',
|
||||
'title': _clip(title, 200),
|
||||
'content_snippet': snippet,
|
||||
'published_at': _iso(mf2_utils.first(entry, 'published')),
|
||||
'author_name': author['name'],
|
||||
'author_url': author['url'] if _is_http_url(author['url']) else None,
|
||||
'author_photo': author['photo'] if _is_http_url(author['photo']) else None,
|
||||
'site_name': og.get('site_name'),
|
||||
'photo': photo if _is_http_url(photo) else None,
|
||||
'photo_alt': photo_alt,
|
||||
}
|
||||
|
||||
|
||||
def _store_context(
|
||||
db_path: Path,
|
||||
url: str,
|
||||
title: Optional[str],
|
||||
content_snippet: Optional[str],
|
||||
published_at: Optional[datetime],
|
||||
author_name: Optional[str],
|
||||
author_url: Optional[str],
|
||||
author_photo: Optional[str],
|
||||
site_name: Optional[str]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Store reply context in database.
|
||||
def _from_bluesky(url: str, site_url: str) -> Optional[Dict[str, Any]]:
|
||||
"""A Bluesky post from the public AppView API (no login needed)"""
|
||||
actor, rkey = BLUESKY_POST_RE.match(url).groups()
|
||||
uri = quote(f"at://{actor}/app.bsky.feed.post/{rkey}", safe='')
|
||||
data = _get_json(f"{BLUESKY_API}?uri={uri}&depth=0&parentHeight=0", site_url)
|
||||
post = ((data or {}).get('thread') or {}).get('post') if isinstance(data, dict) else None
|
||||
if not post:
|
||||
return None
|
||||
author = post.get('author') or {}
|
||||
record = post.get('record') or {}
|
||||
handle = author.get('handle')
|
||||
photo, photo_alt = _bluesky_image(post.get('embed') or {})
|
||||
return {
|
||||
'source': 'bluesky',
|
||||
'title': None,
|
||||
'content_snippet': _text_snippet(record.get('text')),
|
||||
'published_at': _iso(record.get('createdAt')),
|
||||
'author_name': author.get('displayName') or handle,
|
||||
'author_handle': f"@{handle}" if handle else None,
|
||||
'author_url': f"https://bsky.app/profile/{handle}" if handle else None,
|
||||
'author_photo': author.get('avatar') if _is_http_url(author.get('avatar')) else None,
|
||||
'site_name': 'Bluesky',
|
||||
'photo': photo,
|
||||
'photo_alt': photo_alt,
|
||||
}
|
||||
|
||||
Args:
|
||||
db_path: Database path
|
||||
url: Reply-to URL
|
||||
title: Entry title
|
||||
content_snippet: Content snippet
|
||||
published_at: Published datetime
|
||||
author_name: Author name
|
||||
author_url: Author URL
|
||||
author_photo: Author photo URL
|
||||
site_name: Site name
|
||||
|
||||
Returns:
|
||||
Stored context dict
|
||||
"""
|
||||
def _bluesky_image(embed: dict) -> tuple:
|
||||
"""First image of a post's embed view: (url, alt)"""
|
||||
kind = embed.get('$type', '')
|
||||
if kind.startswith('app.bsky.embed.recordWithMedia'):
|
||||
return _bluesky_image(embed.get('media') or {})
|
||||
if kind.startswith('app.bsky.embed.images'):
|
||||
for image in embed.get('images') or []:
|
||||
if _is_http_url(image.get('thumb')):
|
||||
return image['thumb'], image.get('alt') or None
|
||||
if kind.startswith('app.bsky.embed.video') and _is_http_url(embed.get('thumbnail')):
|
||||
return embed['thumbnail'], embed.get('alt') or None
|
||||
if kind.startswith('app.bsky.embed.external'):
|
||||
external = embed.get('external') or {}
|
||||
if _is_http_url(external.get('thumb')):
|
||||
return external['thumb'], None
|
||||
return None, None
|
||||
|
||||
|
||||
def _looks_fediverse(url: str, og: dict) -> bool:
|
||||
return bool(FEDIVERSE_PATH_RE.match(urlparse(url).path) or og.get('username'))
|
||||
|
||||
|
||||
def _mastodon_api(url: str, site_url: str) -> Optional[dict]:
|
||||
"""The status from its own server's public API (Mastodon post URLs only)"""
|
||||
parts = urlparse(url)
|
||||
m = MASTODON_STATUS_RE.match(parts.path)
|
||||
if not m:
|
||||
return None
|
||||
try:
|
||||
conn = sqlite3.connect(db_path)
|
||||
cursor = conn.cursor()
|
||||
status = _get_json(f"{parts.scheme}://{parts.netloc}/api/v1/statuses/{m.group(1)}", site_url)
|
||||
except ValidationError:
|
||||
return None
|
||||
return status if isinstance(status, dict) and status.get('account') else None
|
||||
|
||||
cursor.execute("""
|
||||
|
||||
def _resolve_with_instance(url: str) -> Optional[dict]:
|
||||
"""The status as this site's GoToSocial instance sees it, if one is configured"""
|
||||
try:
|
||||
from starpunk.syndication import get_targets
|
||||
target = get_targets().get('gotosocial')
|
||||
if target is None:
|
||||
return None
|
||||
return target.client.resolve_status(url)
|
||||
except Exception as e:
|
||||
logger.info(f"Instance could not resolve {url}: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _from_status(status: dict, url: str) -> Dict[str, Any]:
|
||||
"""A Mastodon API status entity"""
|
||||
account = status.get('account') or {}
|
||||
acct = account.get('acct') or account.get('username')
|
||||
host = urlparse(account.get('url') or url).hostname
|
||||
if acct and '@' not in acct and host:
|
||||
acct = f"{acct}@{host}"
|
||||
spoiler = (status.get('spoiler_text') or '').strip()
|
||||
if spoiler:
|
||||
# Keep the content warning: show it, not what it covers
|
||||
snippet = _text_snippet(f"CW: {spoiler}")
|
||||
else:
|
||||
snippet = _snippet(status.get('content') or '') or None
|
||||
photo = photo_alt = None
|
||||
if not status.get('sensitive'):
|
||||
for media in status.get('media_attachments') or []:
|
||||
if media.get('type') in ('image', 'gifv', 'video') and _is_http_url(media.get('preview_url')):
|
||||
photo, photo_alt = media['preview_url'], media.get('description') or None
|
||||
break
|
||||
avatar = account.get('avatar_static') or account.get('avatar')
|
||||
return {
|
||||
'source': 'fediverse',
|
||||
'title': None,
|
||||
'content_snippet': snippet,
|
||||
'published_at': _iso(status.get('created_at')),
|
||||
'author_name': account.get('display_name') or account.get('username'),
|
||||
'author_handle': f"@{acct}" if acct else None,
|
||||
'author_url': account.get('url') if _is_http_url(account.get('url')) else None,
|
||||
'author_photo': avatar if _is_http_url(avatar) else None,
|
||||
'site_name': urlparse(url).hostname,
|
||||
'photo': photo,
|
||||
'photo_alt': photo_alt,
|
||||
}
|
||||
|
||||
|
||||
def _from_opengraph(og: dict, card: dict) -> Dict[str, Any]:
|
||||
"""Any other page: OpenGraph tags, the page's h-card as author"""
|
||||
title = og.get('title')
|
||||
author_name, handle = card.get('name') or og.get('author'), None
|
||||
m = FEDI_TITLE_RE.match(title or '')
|
||||
if m and (og.get('username') or not author_name):
|
||||
# A fediverse post's og:title is its author, not a title
|
||||
author_name, handle, title = m.group(1) or author_name, m.group(2), None
|
||||
elif og.get('username') and title:
|
||||
author_name, handle, title = title, f"@{og['username'].lstrip('@')}", None
|
||||
snippet = _text_snippet(og.get('description'))
|
||||
if title and snippet and _repeats(title, snippet):
|
||||
snippet = None
|
||||
return {
|
||||
'source': 'opengraph',
|
||||
'title': _clip(title, 200),
|
||||
'content_snippet': snippet,
|
||||
'published_at': _iso(og.get('published')),
|
||||
'author_name': author_name,
|
||||
'author_handle': handle,
|
||||
'author_url': card.get('url') if _is_http_url(card.get('url')) else None,
|
||||
'author_photo': card.get('photo') if _is_http_url(card.get('photo')) else None,
|
||||
'site_name': og.get('site_name'),
|
||||
'photo': og.get('image') if _is_http_url(og.get('image')) else None,
|
||||
'photo_alt': og.get('image_alt'),
|
||||
}
|
||||
|
||||
|
||||
OG_FIELDS = {
|
||||
'og:title': 'title', 'twitter:title': 'title',
|
||||
'og:description': 'description', 'twitter:description': 'description',
|
||||
'description': 'description',
|
||||
'og:image': 'image', 'og:image:url': 'image', 'twitter:image': 'image',
|
||||
'og:image:alt': 'image_alt', 'twitter:image:alt': 'image_alt',
|
||||
'og:site_name': 'site_name',
|
||||
'article:published_time': 'published', 'og:article:published_time': 'published',
|
||||
'og:published_time': 'published',
|
||||
'profile:username': 'username', 'og:profile:username': 'username',
|
||||
'author': 'author', 'article:author': 'author',
|
||||
}
|
||||
|
||||
|
||||
def _opengraph(content: str, page_url: str) -> Dict[str, Optional[str]]:
|
||||
"""OpenGraph (and plain) meta tags of a page; the first of each wins"""
|
||||
from lxml import html as lxml_html
|
||||
try:
|
||||
doc = lxml_html.document_fromstring(content)
|
||||
except Exception:
|
||||
return {}
|
||||
found: Dict[str, Optional[str]] = {}
|
||||
for meta in doc.iter('meta'):
|
||||
key = (meta.get('property') or meta.get('name') or '').strip().lower()
|
||||
value = (meta.get('content') or '').strip()
|
||||
field = OG_FIELDS.get(key)
|
||||
if not field or not value:
|
||||
continue
|
||||
if field == 'image_alt':
|
||||
# Only the alt text of the image we keep: the one right after it
|
||||
if 'image' in found and 'image_alt' not in found and not found.get('_alt_done'):
|
||||
found['image_alt'] = value
|
||||
continue
|
||||
if field == 'image':
|
||||
if 'image' in found:
|
||||
found['_alt_done'] = '1'
|
||||
continue
|
||||
value = urljoin(page_url, value)
|
||||
if field == 'author' and _is_http_url(value):
|
||||
continue # article:author is often a profile URL
|
||||
found.setdefault(field, value)
|
||||
found.pop('_alt_done', None)
|
||||
if 'title' not in found:
|
||||
titles = doc.findall('.//title')
|
||||
if titles and (titles[0].text_content() or '').strip():
|
||||
found['title'] = titles[0].text_content().strip()
|
||||
return found
|
||||
|
||||
|
||||
def _page_card(parsed: dict, page_url: str) -> Dict[str, Optional[str]]:
|
||||
"""The page's representative h-card, else its first h-card (GoToSocial pages)"""
|
||||
card = mf2_utils.representative_hcard(parsed, page_url)
|
||||
if card is None:
|
||||
card = next((i for i in mf2_utils.walk(parsed.get('items', []))
|
||||
if mf2_utils.is_type(i, 'h-card')), None)
|
||||
if not card:
|
||||
return {}
|
||||
return {'name': mf2_utils.first(card, 'name'), 'url': mf2_utils.first(card, 'url'),
|
||||
'photo': mf2_utils.first(card, 'photo')}
|
||||
|
||||
|
||||
# --- Helpers ----------------------------------------------------------------
|
||||
|
||||
def _get_json(url: str, site_url: str):
|
||||
"""GET JSON through the SSRF-safe fetcher; None if it isn't JSON"""
|
||||
content, _, _ = fetch_source(url, site_url)
|
||||
try:
|
||||
return json.loads(content)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _snippet(markup: str) -> str:
|
||||
"""Sanitized HTML of at most ~SNIPPET_LIMIT characters, with tags closed"""
|
||||
sanitized = sanitize_content(markup).strip()
|
||||
if len(sanitized) > SNIPPET_LIMIT:
|
||||
# Cutting can leave a tag or attribute open; sanitizing again closes it
|
||||
sanitized = sanitize_content(sanitized[:SNIPPET_LIMIT - 3] + '...')
|
||||
return sanitized
|
||||
|
||||
|
||||
def _text_snippet(text: Optional[str]) -> Optional[str]:
|
||||
"""Plain text as a sanitized HTML snippet"""
|
||||
text = (text or '').strip()
|
||||
if not text:
|
||||
return None
|
||||
if len(text) > SNIPPET_LIMIT:
|
||||
text = text[:SNIPPET_LIMIT - 1].rstrip() + '…'
|
||||
paragraphs = [p.strip() for p in re.split(r'\n\s*\n', text) if p.strip()]
|
||||
return ''.join(f"<p>{html.escape(p).replace(chr(10), '<br>')}</p>" for p in paragraphs)
|
||||
|
||||
|
||||
def _plain(markup: Optional[str]) -> str:
|
||||
return ' '.join(html.unescape(re.sub(r'<[^>]+>', ' ', markup or '')).split())
|
||||
|
||||
|
||||
def _repeats(title: str, snippet: Optional[str]) -> bool:
|
||||
"""True if a title is just the start of the content (mf2 implied names, og:description)"""
|
||||
a, b = ' '.join(title.split()).rstrip('.…'), _plain(snippet).rstrip('.…')
|
||||
return bool(a and b) and (b.startswith(a) or a.startswith(b))
|
||||
|
||||
|
||||
def _clip(text: Optional[str], limit: int) -> Optional[str]:
|
||||
if not text:
|
||||
return None
|
||||
text = ' '.join(text.split())
|
||||
return text if len(text) <= limit else text[:limit - 3] + '...'
|
||||
|
||||
|
||||
def _iso(value: Optional[str]) -> Optional[str]:
|
||||
"""An ISO 8601 date string, normalized; None if it doesn't parse"""
|
||||
if not value:
|
||||
return None
|
||||
from dateutil import parser as dateutil_parser
|
||||
try:
|
||||
return dateutil_parser.isoparse(value).isoformat()
|
||||
except (ValueError, OverflowError):
|
||||
try:
|
||||
return dateutil_parser.parse(value).isoformat()
|
||||
except (ValueError, OverflowError):
|
||||
return None
|
||||
|
||||
|
||||
def _save_images(context: Dict[str, Any], site_url: str) -> None:
|
||||
"""Local copies of the author's avatar and the post's image (best effort;
|
||||
the webmention worker retries what's missing)"""
|
||||
try:
|
||||
from flask import current_app
|
||||
from starpunk import avatars
|
||||
data_path = current_app.config['DATA_PATH']
|
||||
avatars.fetch(data_path, context.get('author_photo'), site_url)
|
||||
avatars.fetch_preview(data_path, context.get('photo'), site_url)
|
||||
except Exception as e:
|
||||
logger.info(f"Could not save reply context images: {e}")
|
||||
|
||||
|
||||
# --- Storage ----------------------------------------------------------------
|
||||
|
||||
def _store_context(db_path: Path, url: str, found: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Store a fetched reply context; returns it as stored"""
|
||||
values = [found.get(field) for field in FIELDS]
|
||||
conn = sqlite3.connect(db_path)
|
||||
try:
|
||||
conn.execute(f"""
|
||||
INSERT OR REPLACE INTO reply_contexts
|
||||
(url, title, content_snippet, published_at,
|
||||
author_name, author_url, author_photo, site_name,
|
||||
fetch_status, fetched_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'fetched', CURRENT_TIMESTAMP)
|
||||
""", (
|
||||
url, title, content_snippet, published_at,
|
||||
author_name, author_url, author_photo, site_name
|
||||
))
|
||||
|
||||
(url, {', '.join(FIELDS)}, source, fetch_status, fetch_error, fetched_at)
|
||||
VALUES (?, {', '.join('?' for _ in FIELDS)}, ?, 'fetched', NULL, CURRENT_TIMESTAMP)
|
||||
""", (url, *values, found['source']))
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
return {
|
||||
'url': url,
|
||||
'title': title,
|
||||
'content_snippet': content_snippet,
|
||||
'published_at': published_at,
|
||||
'author_name': author_name,
|
||||
'author_url': author_url,
|
||||
'author_photo': author_photo,
|
||||
'site_name': site_name,
|
||||
'fetch_status': 'fetched'
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to store reply context: {e}")
|
||||
raise
|
||||
finally:
|
||||
conn.close()
|
||||
return {'url': url, **dict(zip(FIELDS, values)), 'source': found['source'],
|
||||
'fetch_status': 'fetched'}
|
||||
|
||||
|
||||
def _mark_source(db_path: Path, url: str, source: str) -> None:
|
||||
conn = sqlite3.connect(db_path)
|
||||
try:
|
||||
conn.execute("UPDATE reply_contexts SET source = ? WHERE url = ?", (source, url))
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def _store_minimal_context(
|
||||
@@ -269,33 +533,20 @@ def _store_minimal_context(
|
||||
Returns:
|
||||
Minimal context dict with error
|
||||
"""
|
||||
context = {'url': url, 'fetch_status': 'error', 'fetch_error': error, 'source': 'error'}
|
||||
try:
|
||||
conn = sqlite3.connect(db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
conn.execute("""
|
||||
INSERT OR REPLACE INTO reply_contexts
|
||||
(url, fetch_status, fetch_error, created_at)
|
||||
VALUES (?, 'error', ?, CURRENT_TIMESTAMP)
|
||||
(url, fetch_status, fetch_error, source, created_at)
|
||||
VALUES (?, 'error', ?, 'error', CURRENT_TIMESTAMP)
|
||||
""", (url, error))
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
return {
|
||||
'url': url,
|
||||
'fetch_status': 'error',
|
||||
'fetch_error': error
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
# Return the in-memory dict even if storage fails
|
||||
logger.error(f"Failed to store minimal context: {e}")
|
||||
# Return in-memory dict even if storage fails
|
||||
return {
|
||||
'url': url,
|
||||
'fetch_status': 'error',
|
||||
'fetch_error': error
|
||||
}
|
||||
return context
|
||||
|
||||
|
||||
def get_reply_context(db_path: Path, reply_url: str) -> Optional[Dict[str, Any]]:
|
||||
@@ -312,20 +563,9 @@ def get_reply_context(db_path: Path, reply_url: str) -> Optional[Dict[str, Any]]
|
||||
try:
|
||||
conn = sqlite3.connect(db_path)
|
||||
conn.row_factory = sqlite3.Row
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
SELECT * FROM reply_contexts WHERE url = ?
|
||||
""", (reply_url,))
|
||||
|
||||
row = cursor.fetchone()
|
||||
row = conn.execute("SELECT * FROM reply_contexts WHERE url = ?", (reply_url,)).fetchone()
|
||||
conn.close()
|
||||
|
||||
if row:
|
||||
return dict(row)
|
||||
|
||||
return None
|
||||
|
||||
return dict(row) if row else None
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get reply context: {e}")
|
||||
return None
|
||||
+55
-10
@@ -25,6 +25,20 @@ from starpunk.notes import (
|
||||
bp = Blueprint("admin", __name__, url_prefix="/admin")
|
||||
|
||||
|
||||
@bp.context_processor
|
||||
def _admin_nav():
|
||||
"""Error count for the Log link in the admin nav (v2.18.0)"""
|
||||
if not getattr(g, "me", None):
|
||||
return {}
|
||||
from starpunk import error_log
|
||||
|
||||
try:
|
||||
return {"error_count": error_log.count()}
|
||||
except Exception as e: # never break an admin page over the badge
|
||||
logger.warning(f"Could not count errors for the admin nav: {e}")
|
||||
return {"error_count": 0}
|
||||
|
||||
|
||||
@bp.route("/")
|
||||
@require_auth
|
||||
def dashboard():
|
||||
@@ -128,6 +142,7 @@ def create_note_submit():
|
||||
media_files: Multiple file upload (v1.2.0 Phase 3)
|
||||
captions[]: Captions for each media file (v1.2.0 Phase 3)
|
||||
tags: Comma-separated tag list (v1.3.0 Phase 3)
|
||||
title, summary: Optional; a title makes it an article (v2.16.0)
|
||||
|
||||
Returns:
|
||||
Redirect to dashboard on success, back to form on error
|
||||
@@ -141,17 +156,19 @@ def create_note_submit():
|
||||
published = "published" in request.form
|
||||
custom_slug = request.form.get("custom_slug", "").strip()
|
||||
tags_input = request.form.get("tags", "")
|
||||
title = request.form.get("title", "").strip() or None
|
||||
summary = request.form.get("summary", "").strip() or None
|
||||
|
||||
if not content:
|
||||
flash("Content cannot be empty", "error")
|
||||
# With photos, WebSub hears about the note once they're attached (v2.10.1)
|
||||
has_uploads = any(f.filename for f in request.files.getlist('media_files'))
|
||||
|
||||
if not (content or title or has_uploads):
|
||||
flash("A note needs text, a title or an image", "error")
|
||||
return redirect(url_for("admin.new_note_form"))
|
||||
|
||||
# Parse tags (v1.3.0 Phase 3)
|
||||
tags = parse_tag_input(tags_input)
|
||||
|
||||
# With photos, WebSub hears about the note once they're attached (v2.10.1)
|
||||
has_uploads = any(f.filename for f in request.files.getlist('media_files'))
|
||||
|
||||
try:
|
||||
# Create note first (per Q4)
|
||||
note = create_note(
|
||||
@@ -160,6 +177,8 @@ def create_note_submit():
|
||||
custom_slug=custom_slug if custom_slug else None,
|
||||
tags=tags if tags else None,
|
||||
has_media=has_uploads,
|
||||
title=title,
|
||||
summary=summary,
|
||||
)
|
||||
|
||||
# Handle media uploads (v1.2.0 Phase 3)
|
||||
@@ -279,12 +298,14 @@ def update_note_submit(note_id: int):
|
||||
content: Updated markdown content (required)
|
||||
published: Checkbox for published status (optional)
|
||||
tags: Comma-separated tag list (v1.3.0 Phase 3)
|
||||
title, summary: Empty removes them; a form without the field leaves them (v2.16.0)
|
||||
|
||||
Returns:
|
||||
Redirect to dashboard on success, back to form on error
|
||||
|
||||
Decorator: @require_auth
|
||||
"""
|
||||
from starpunk.notes import UNCHANGED
|
||||
from starpunk.tags import parse_tag_input
|
||||
|
||||
# Check if note exists first
|
||||
@@ -296,8 +317,12 @@ def update_note_submit(note_id: int):
|
||||
content = request.form.get("content", "").strip()
|
||||
published = "published" in request.form
|
||||
tags_input = request.form.get("tags", "")
|
||||
title = request.form["title"].strip() if "title" in request.form else UNCHANGED
|
||||
summary = request.form["summary"].strip() if "summary" in request.form else UNCHANGED
|
||||
has_title = existing_note.title if title is UNCHANGED else title
|
||||
|
||||
if not content and not existing_note.reacts_to: # likes/reposts may have no comment
|
||||
# Likes/reposts may have no comment, articles no body
|
||||
if not content and not (existing_note.reacts_to or has_title):
|
||||
flash("Content cannot be empty", "error")
|
||||
return redirect(url_for("admin.edit_note_form", note_id=note_id))
|
||||
|
||||
@@ -311,7 +336,9 @@ def update_note_submit(note_id: int):
|
||||
published=published,
|
||||
# An empty Tags field removes all tags (v2.14.1); a form without
|
||||
# the field leaves them alone
|
||||
tags=tags if "tags" in request.form else None
|
||||
tags=tags if "tags" in request.form else None,
|
||||
title=title,
|
||||
summary=summary,
|
||||
)
|
||||
if "syndication_form" in request.form:
|
||||
from starpunk.syndication import select_targets
|
||||
@@ -517,7 +544,6 @@ def webmentions():
|
||||
status_filter=status_filter,
|
||||
origin_filter=origin_filter,
|
||||
backfeed=backfeed.status(),
|
||||
notify=_notify_status(),
|
||||
page=page,
|
||||
total_pages=total_pages,
|
||||
user_me=g.me
|
||||
@@ -685,7 +711,7 @@ def revoke_token(token_id):
|
||||
|
||||
|
||||
def _notify_status() -> dict:
|
||||
"""Notification settings and last error for the Webmentions page (v2.14.0)"""
|
||||
"""Notification settings and last error for the Settings page (v2.14.0)"""
|
||||
from starpunk import notifications
|
||||
from starpunk.database import get_db
|
||||
|
||||
@@ -698,6 +724,25 @@ def _notify_status() -> dict:
|
||||
}
|
||||
|
||||
|
||||
@bp.route("/settings")
|
||||
@require_auth
|
||||
def settings():
|
||||
"""Site settings: notifications (v2.18.0). Template: templates/admin/settings.html"""
|
||||
from starpunk.webmention_sender import skipped_hosts
|
||||
|
||||
return render_template("admin/settings.html", notify=_notify_status(), user_me=g.me,
|
||||
skip_hosts=sorted(skipped_hosts(current_app.config)))
|
||||
|
||||
|
||||
@bp.route("/log")
|
||||
@require_auth
|
||||
def error_log():
|
||||
"""Current errors with the affected note (v2.18.0). Template: templates/admin/log.html"""
|
||||
from starpunk import error_log as log
|
||||
|
||||
return render_template("admin/log.html", entries=log.entries(), user_me=g.me)
|
||||
|
||||
|
||||
@bp.route("/notifications/test", methods=["POST"])
|
||||
@require_auth
|
||||
def notifications_test():
|
||||
@@ -709,7 +754,7 @@ def notifications_test():
|
||||
notifications._record(current_app.config["DATABASE_PATH"], ok, error)
|
||||
flash("Test notification sent" if ok else f"Test notification failed: {error or 'unknown error'}",
|
||||
"success" if ok else "error")
|
||||
return redirect(url_for("admin.webmentions"))
|
||||
return redirect(url_for("admin.settings"))
|
||||
|
||||
|
||||
@bp.route("/backfeed/backfill", methods=["POST"])
|
||||
|
||||
@@ -51,6 +51,7 @@ _REQUEST_FIELDS = (
|
||||
"code_challenge",
|
||||
"code_challenge_method",
|
||||
"scope",
|
||||
"resource",
|
||||
)
|
||||
|
||||
|
||||
@@ -95,6 +96,11 @@ def metadata():
|
||||
"revocation_endpoint_auth_methods_supported": ["none"],
|
||||
"scopes_supported": SUPPORTED_SCOPES,
|
||||
"response_types_supported": ["code"],
|
||||
# Public clients only (PKCE); clients identify with a client_id
|
||||
# URL whose JSON metadata lists redirect_uris, which is what MCP
|
||||
# calls a Client ID Metadata Document (v2.15.0, ADR-075)
|
||||
"token_endpoint_auth_methods_supported": ["none"],
|
||||
"client_id_metadata_document_supported": True,
|
||||
"grant_types_supported": ["authorization_code"],
|
||||
"code_challenge_methods_supported": ["S256"],
|
||||
"authorization_response_iss_parameter_supported": True,
|
||||
@@ -158,6 +164,7 @@ def approve():
|
||||
auth_request["redirect_uri"],
|
||||
" ".join(granted),
|
||||
auth_request["code_challenge"],
|
||||
resource=auth_request["resource"],
|
||||
)
|
||||
current_app.logger.info(
|
||||
f"IndieAuth: approved {auth_request['client_id']} (scope: {' '.join(granted) or 'none'})"
|
||||
@@ -209,7 +216,12 @@ def token():
|
||||
# Spec: a code issued without scope cannot be used for an access token
|
||||
return _oauth_error("invalid_grant", "No scope was granted; use the authorization endpoint")
|
||||
|
||||
return _json(create_access_token(result["client_id"], result["scope"]))
|
||||
# RFC 8707: a resource named here must be the one the code was issued for
|
||||
requested = (request.form.get("resource") or "").strip().rstrip("/")
|
||||
if requested and requested != (result["resource"] or ""):
|
||||
return _oauth_error("invalid_target", "resource does not match the authorization request")
|
||||
|
||||
return _json(create_access_token(result["client_id"], result["scope"], resource=result["resource"]))
|
||||
|
||||
|
||||
@bp.route("/auth/token", methods=["GET"])
|
||||
@@ -222,7 +234,7 @@ def token_verify():
|
||||
info = verify_access_token(token)
|
||||
if not info:
|
||||
return _oauth_error("unauthorized", "Invalid or expired access token", 401)
|
||||
return _json(info)
|
||||
return _json({k: info[k] for k in ("me", "client_id", "scope")})
|
||||
|
||||
|
||||
@bp.route("/auth/revoke", methods=["POST"])
|
||||
|
||||
@@ -8,7 +8,7 @@ No authentication required for these routes.
|
||||
import hashlib
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from flask import Blueprint, abort, render_template, Response, current_app, request, send_from_directory
|
||||
from flask import Blueprint, abort, redirect, render_template, Response, current_app, request, send_from_directory, url_for
|
||||
|
||||
from starpunk.notes import list_notes, get_note
|
||||
from starpunk.websub import embed_hub
|
||||
@@ -264,6 +264,20 @@ def avatar(name: str):
|
||||
return response
|
||||
|
||||
|
||||
@bp.route("/previews/<name>")
|
||||
def preview(name: str):
|
||||
"""A saved copy of a replied-to post's image (v2.19.0)"""
|
||||
from pathlib import Path
|
||||
from starpunk.avatars import NAME_RE, preview_folder
|
||||
|
||||
if not NAME_RE.match(name):
|
||||
abort(404)
|
||||
response = send_from_directory(
|
||||
preview_folder(Path(current_app.config.get('DATA_PATH', 'data'))), name)
|
||||
response.headers['Cache-Control'] = 'public, max-age=604800'
|
||||
return response
|
||||
|
||||
|
||||
@bp.route("/")
|
||||
def index():
|
||||
"""
|
||||
@@ -310,14 +324,15 @@ def attach_response_counts(notes) -> None:
|
||||
|
||||
|
||||
def attach_contexts(notes) -> None:
|
||||
"""Give likes and reposts the fetched context of the post they point at"""
|
||||
"""Give replies, likes and reposts the fetched context of the post they point at"""
|
||||
from starpunk.webmention_queries import get_reply_context
|
||||
|
||||
for note in notes:
|
||||
context = None
|
||||
if note.reacts_to:
|
||||
url = note.in_reply_to or note.reacts_to
|
||||
if url:
|
||||
try:
|
||||
context = get_reply_context(note.reacts_to)
|
||||
context = get_reply_context(url)
|
||||
except Exception:
|
||||
context = None
|
||||
object.__setattr__(note, 'context', context)
|
||||
@@ -354,18 +369,30 @@ def _note_was_deleted(slug: str) -> bool:
|
||||
).fetchone():
|
||||
return True
|
||||
return db.execute(
|
||||
"SELECT 1 FROM webmention_queue WHERE source_url = ? AND source_note_id IS NULL",
|
||||
(absolute_url_for("public.note", slug=slug),),
|
||||
"SELECT 1 FROM webmention_queue WHERE source_url IN (?, ?) AND source_note_id IS NULL",
|
||||
(absolute_url_for("public.note", slug=slug), absolute_url_for("public.article", slug=slug)),
|
||||
).fetchone() is not None
|
||||
|
||||
|
||||
@bp.route("/note/<slug>")
|
||||
def note(slug: str):
|
||||
"""A note's permalink page; an article's /note/ URL redirects to /article/"""
|
||||
return _permalink(slug, "public.note")
|
||||
|
||||
|
||||
@bp.route("/article/<slug>")
|
||||
def article(slug: str):
|
||||
"""An article's permalink page (v2.17.0); a note's /article/ URL redirects to /note/"""
|
||||
return _permalink(slug, "public.article")
|
||||
|
||||
|
||||
def _permalink(slug: str, endpoint: str):
|
||||
"""
|
||||
Individual note permalink page
|
||||
Individual note or article permalink page
|
||||
|
||||
Args:
|
||||
slug: URL-safe note identifier
|
||||
endpoint: The route the request came in on
|
||||
|
||||
Returns:
|
||||
Rendered note template with full content
|
||||
@@ -390,6 +417,12 @@ def note(slug: str):
|
||||
abort(410)
|
||||
abort(404)
|
||||
|
||||
# Each post has one URL: /article/ for articles, /note/ for notes (v2.17.0).
|
||||
# The other one redirects, so links made before a title was added or
|
||||
# removed keep working.
|
||||
if note_obj.endpoint != endpoint:
|
||||
return redirect(url_for(note_obj.endpoint, slug=slug), code=301)
|
||||
|
||||
# Get media for note (v1.2.0 Phase 3)
|
||||
media = get_note_media(note_obj.id)
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ from pathlib import Path
|
||||
from flask import Blueprint, current_app, g, jsonify, render_template, request
|
||||
|
||||
from starpunk.search import has_fts_table, search_notes
|
||||
from starpunk.utils import post_path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -108,7 +109,7 @@ def api_search():
|
||||
"title": r["title"] or f"Note from {r['created_at'][:10]}",
|
||||
"excerpt": r["snippet"], # Already has <mark> tags
|
||||
"published_at": r["created_at"],
|
||||
"url": f"/note/{r['slug']}",
|
||||
"url": post_path(r["slug"], r.get("article_title")),
|
||||
}
|
||||
for r in results
|
||||
],
|
||||
@@ -176,7 +177,7 @@ def search_page():
|
||||
"title": r["title"] or f"Note from {r['created_at'][:10]}",
|
||||
"excerpt": Markup(safe_snippet), # Mark as safe since we've escaped it ourselves
|
||||
"published_at": r["created_at"],
|
||||
"url": f"/note/{r['slug']}",
|
||||
"url": post_path(r["slug"], r.get("article_title")),
|
||||
})
|
||||
results = formatted_results
|
||||
except Exception as e:
|
||||
|
||||
@@ -139,9 +139,10 @@ def resolve_target_note(target_url: str) -> int | None:
|
||||
# Strip query params and fragments for matching
|
||||
target_path = target_parsed.path.rstrip('/')
|
||||
|
||||
# Check if it's a note URL pattern: /note/{slug}
|
||||
if target_path.startswith('/note/'):
|
||||
slug = target_path[6:] # Remove '/note/' prefix
|
||||
# A note or article URL: /note/{slug} or /article/{slug} (v2.17.0).
|
||||
# Either prefix finds the post, as the page redirects to the right one.
|
||||
prefix, _, slug = target_path.lstrip('/').partition('/')
|
||||
if prefix in ('note', 'article') and '/' not in slug:
|
||||
if slug:
|
||||
try:
|
||||
note = get_note(slug=slug)
|
||||
|
||||
+23
-20
@@ -98,29 +98,32 @@ def has_fts_table(db_path: Path) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def update_fts_index(conn: sqlite3.Connection, note_id: int, slug: str, content: str):
|
||||
def _first_line(content: str) -> str:
|
||||
"""A note's first line without heading marks (result label for untitled notes).
|
||||
A hashtag ('#indieweb ...') is kept."""
|
||||
line = content.split('\n', 1)[0].strip()
|
||||
return re.sub(r'^#{1,6}\s+', '', line)
|
||||
|
||||
|
||||
def update_fts_index(conn: sqlite3.Connection, note_id: int, slug: str, content: str,
|
||||
title: Optional[str] = None):
|
||||
"""
|
||||
Update FTS index for a note (insert or replace)
|
||||
|
||||
Extracts title from first line of content and updates the FTS index.
|
||||
Uses REPLACE to handle both new notes and updates.
|
||||
Indexes the note's title (articles, v2.16.0) or, for untitled notes, its
|
||||
first line. Uses REPLACE to handle both new notes and updates.
|
||||
|
||||
Args:
|
||||
conn: SQLite database connection
|
||||
note_id: Note ID (used as FTS rowid)
|
||||
slug: Note slug
|
||||
content: Full markdown content
|
||||
content: Markdown content
|
||||
title: The note's title, if it has one
|
||||
|
||||
Raises:
|
||||
sqlite3.Error: If FTS update fails
|
||||
"""
|
||||
# Extract title from first line
|
||||
lines = content.split('\n', 1)
|
||||
title = lines[0].strip() if lines else ''
|
||||
|
||||
# Remove markdown heading syntax (# ## ###)
|
||||
if title.startswith('#'):
|
||||
title = title.lstrip('#').strip()
|
||||
title = title or _first_line(content)
|
||||
|
||||
# Limit title length
|
||||
if len(title) > 100:
|
||||
@@ -170,7 +173,7 @@ def rebuild_fts_index(db_path: Path, data_dir: Path):
|
||||
|
||||
# Get all non-deleted notes
|
||||
cursor = conn.execute(
|
||||
"SELECT id, slug, file_path FROM notes WHERE deleted_at IS NULL"
|
||||
"SELECT id, slug, file_path, title FROM notes WHERE deleted_at IS NULL"
|
||||
)
|
||||
|
||||
indexed_count = 0
|
||||
@@ -188,7 +191,7 @@ def rebuild_fts_index(db_path: Path, data_dir: Path):
|
||||
content = note_path.read_text(encoding='utf-8')
|
||||
|
||||
# Update FTS index
|
||||
update_fts_index(conn, row['id'], row['slug'], content)
|
||||
update_fts_index(conn, row['id'], row['slug'], content, row['title'])
|
||||
indexed_count += 1
|
||||
|
||||
except Exception as e:
|
||||
@@ -327,6 +330,7 @@ def search_notes_fts5(
|
||||
notes.id,
|
||||
notes.slug,
|
||||
notes_fts.title,
|
||||
notes.title AS article_title,
|
||||
notes.published,
|
||||
notes.created_at,
|
||||
rank AS relevance,
|
||||
@@ -353,6 +357,7 @@ def search_notes_fts5(
|
||||
'id': row['id'],
|
||||
'slug': row['slug'],
|
||||
'title': row['title'],
|
||||
'article_title': row['article_title'],
|
||||
'snippet': row['snippet'], # Plain string - route must escape HTML while preserving <mark> tags
|
||||
'relevance': row['relevance'],
|
||||
'published': bool(row['published']),
|
||||
@@ -408,6 +413,7 @@ def search_notes_fallback(
|
||||
id,
|
||||
slug,
|
||||
file_path,
|
||||
title,
|
||||
published,
|
||||
created_at
|
||||
FROM notes
|
||||
@@ -446,8 +452,8 @@ def search_notes_fallback(
|
||||
file_path = data_dir / row['file_path']
|
||||
content = read_note_file(file_path)
|
||||
|
||||
# Check if query matches content (case-insensitive)
|
||||
content_lower = content.lower()
|
||||
# Check if query matches title or content (case-insensitive)
|
||||
content_lower = f"{row['title'] or ''}\n{content}".lower()
|
||||
query_lower = query.lower()
|
||||
matches = query_lower in content_lower
|
||||
|
||||
@@ -456,16 +462,13 @@ def search_notes_fallback(
|
||||
matches = any(term.lower() in content_lower for term in terms)
|
||||
|
||||
if matches:
|
||||
# Extract title from first line
|
||||
lines = content.split('\n', 1)
|
||||
title = lines[0].strip() if lines else row['slug']
|
||||
if title.startswith('#'):
|
||||
title = title.lstrip('#').strip()
|
||||
title = row['title'] or _first_line(content) or row['slug']
|
||||
|
||||
results.append({
|
||||
'id': row['id'],
|
||||
'slug': row['slug'],
|
||||
'title': title,
|
||||
'article_title': row['title'],
|
||||
'snippet': generate_snippet(content, query),
|
||||
'relevance': 0.0, # No ranking in fallback mode
|
||||
'published': bool(row['published']),
|
||||
|
||||
@@ -288,6 +288,35 @@ def generate_timestamp_slug(
|
||||
return f"{base_slug}-{suffix}"
|
||||
|
||||
|
||||
TITLE_SLUG_LENGTH = 80
|
||||
|
||||
|
||||
def generate_title_slug(title: str, existing_slugs: Set[str]) -> Optional[str]:
|
||||
"""
|
||||
A slug made from an article's title (v2.17.0)
|
||||
|
||||
"Why I Left Twitter" becomes 'why-i-left-twitter'; long titles are cut at
|
||||
a word boundary, and a taken slug gets -2, -3, ...
|
||||
|
||||
Returns:
|
||||
The slug, or None when the title has nothing usable (e.g. only emoji),
|
||||
so the caller falls back to a timestamp slug.
|
||||
"""
|
||||
slug = sanitize_slug(title or "")
|
||||
if len(slug) > TITLE_SLUG_LENGTH:
|
||||
cut = slug[:TITLE_SLUG_LENGTH + 1]
|
||||
slug = cut.rsplit("-", 1)[0] if "-" in cut else slug[:TITLE_SLUG_LENGTH]
|
||||
slug = slug.strip("-")
|
||||
if not slug:
|
||||
return None
|
||||
if is_reserved_slug(slug):
|
||||
slug = f"{slug}-article"
|
||||
try:
|
||||
return make_slug_unique_with_suffix(slug, existing_slugs)
|
||||
except ValueError: # 99 articles with this title
|
||||
return None
|
||||
|
||||
|
||||
def validate_and_sanitize_custom_slug(custom_slug: str, existing_slugs: Set[str]) -> tuple[bool, Optional[str], Optional[str]]:
|
||||
"""
|
||||
Validate and sanitize a custom slug from Micropub
|
||||
|
||||
+197
-31
@@ -25,6 +25,7 @@ import logging
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
from urllib.parse import urlparse
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
@@ -102,10 +103,14 @@ class MastodonClient:
|
||||
"content_type": "text/markdown",
|
||||
}
|
||||
|
||||
def create_status(self, text: str, media_ids: List[str], idempotency_key: str) -> dict:
|
||||
def create_status(self, text: str, media_ids: List[str], idempotency_key: str,
|
||||
in_reply_to_id: Optional[str] = None) -> dict:
|
||||
body = self._status_body(text, media_ids)
|
||||
if in_reply_to_id:
|
||||
body["in_reply_to_id"] = in_reply_to_id
|
||||
response = self._call(
|
||||
"POST", "/api/v1/statuses",
|
||||
json=self._status_body(text, media_ids),
|
||||
json=body,
|
||||
headers={"Idempotency-Key": idempotency_key},
|
||||
)
|
||||
data = response.json()
|
||||
@@ -127,8 +132,8 @@ class MastodonClient:
|
||||
can_edit = True
|
||||
NATIVE = ("favourite", "reblog") # remote_id prefixes for likes/boosts (v2.5.0)
|
||||
|
||||
def find_status(self, url: str) -> Optional[str]:
|
||||
"""Local id of a fediverse post, fetching it if needed; None if the URL isn't one"""
|
||||
def resolve_status(self, url: str) -> Optional[dict]:
|
||||
"""A fediverse post as a Mastodon status, fetching it if needed; None if the URL isn't one"""
|
||||
try:
|
||||
response = self._call("GET", "/api/v2/search", params={
|
||||
"q": url, "resolve": "true", "type": "statuses", "limit": 1,
|
||||
@@ -136,16 +141,52 @@ class MastodonClient:
|
||||
except RemoteGone:
|
||||
return None
|
||||
statuses = response.json().get("statuses") or []
|
||||
return statuses[0]["id"] if statuses else None
|
||||
return statuses[0] if statuses else None
|
||||
|
||||
def find_status(self, url: str) -> Optional[str]:
|
||||
"""Local id of a fediverse post, fetching it if needed; None if the URL isn't one"""
|
||||
status = self.resolve_status(url)
|
||||
return status["id"] if status else None
|
||||
|
||||
def _reply_parent(self, post: "Post") -> tuple:
|
||||
"""(status id, '@acct ' mention) a reply threads under; (None, '') if it can't (v2.20.0)
|
||||
|
||||
Our own crosspost of the replied-to note, else the replied-to post as
|
||||
this instance resolves it (any fediverse post). Replies to others
|
||||
start with their @mention, as Mastodon and GoToSocial clients do, so
|
||||
they are notified and their server gets the reply."""
|
||||
if not post.reply_to:
|
||||
return None, ""
|
||||
if post.reply_remote_id and post.reply_remote_id.split(":", 1)[0] not in self.NATIVE:
|
||||
return post.reply_remote_id, ""
|
||||
try:
|
||||
status = self.resolve_status(post.reply_to)
|
||||
except SyndicationError as e:
|
||||
current_app.logger.info(f"Could not resolve {post.reply_to}: {e}")
|
||||
return None, ""
|
||||
if not status:
|
||||
return None, ""
|
||||
account = status.get("account") or {}
|
||||
try:
|
||||
mine = account.get("id") == self._account_id()
|
||||
except SyndicationError:
|
||||
mine = False
|
||||
mention = "" if mine or not account.get("acct") else f"@{account['acct']} "
|
||||
return status["id"], mention
|
||||
|
||||
def _prepare(self, post: "Post") -> tuple:
|
||||
"""(text, media ids, id of the status it replies to)"""
|
||||
if post.kind == "repost":
|
||||
return compose_repost_text(post, post.note.content, post.max_chars), []
|
||||
return compose_repost_text(post, post.note.content, post.max_chars), [], None
|
||||
parent, mention = self._reply_parent(post)
|
||||
# Images in the text are sent as attachments instead (v2.6.6)
|
||||
text = compose_text(post.note, post.permalink, post.tags, post.max_chars,
|
||||
body=strip_images(post.note.content))
|
||||
body = strip_images(post.note.content)
|
||||
if mention.strip() and mention.strip() not in body:
|
||||
body = mention + body.lstrip()
|
||||
text = compose_text(post.note, post.permalink, post.tags, post.max_chars, body=body,
|
||||
lead="" if parent else reply_line(post))
|
||||
media_ids = [self.upload_media(f["path"], f["mime_type"], f["description"]) for f in post.media]
|
||||
return text, media_ids
|
||||
return text, media_ids, parent
|
||||
|
||||
def create(self, post: "Post", key: str, retrying: bool = False) -> dict:
|
||||
if post.kind in ("like", "repost") and post.target_url:
|
||||
@@ -160,13 +201,13 @@ class MastodonClient:
|
||||
return {"id": f"{action}:{status_id}", "url": None}
|
||||
if post.kind == "like":
|
||||
return {"id": None, "url": None} # nothing to like here
|
||||
text, media_ids = self._prepare(post)
|
||||
return self.create_status(text, media_ids, idempotency_key=key)
|
||||
text, media_ids, parent = self._prepare(post)
|
||||
return self.create_status(text, media_ids, idempotency_key=key, in_reply_to_id=parent)
|
||||
|
||||
def update(self, remote_id: str, post: "Post") -> dict:
|
||||
if remote_id.split(":", 1)[0] in self.NATIVE:
|
||||
return {"id": remote_id, "url": None} # a favourite/boost has nothing to edit
|
||||
text, media_ids = self._prepare(post)
|
||||
text, media_ids, _ = self._prepare(post) # edits can't move a status in a thread
|
||||
return self.update_status(remote_id, text, media_ids)
|
||||
|
||||
# --- Backfeed (v2.13.0, ADR-073) ---
|
||||
@@ -354,12 +395,44 @@ class BlueskyClient:
|
||||
current_app.logger.info(f"No link card image for {url}: {e}")
|
||||
return {"$type": "app.bsky.embed.external", "external": external}
|
||||
|
||||
def _reply_ref(self, post: "Post") -> Optional[dict]:
|
||||
"""{'root', 'parent'} for a reply to a Bluesky post (ours or anyone's); None otherwise (v2.20.0)"""
|
||||
if not post.reply_to:
|
||||
return None
|
||||
uri = post.reply_remote_id if (post.reply_remote_id or "").startswith("at://") and \
|
||||
f"/{self.POST}/" in post.reply_remote_id else None
|
||||
try:
|
||||
if uri is None:
|
||||
found = self.resolve_post(post.reply_to)
|
||||
if not found:
|
||||
return None
|
||||
uri = found["uri"]
|
||||
views = self._call("GET", "app.bsky.feed.getPosts", params={"uris": uri}).json().get("posts") or []
|
||||
except SyndicationError as e:
|
||||
current_app.logger.info(f"Could not resolve {post.reply_to} on Bluesky: {e}")
|
||||
return None
|
||||
if not views:
|
||||
return None
|
||||
parent = {"uri": views[0]["uri"], "cid": views[0]["cid"]}
|
||||
root = ((views[0].get("record") or {}).get("reply") or {}).get("root") or parent
|
||||
return {"root": {"uri": root["uri"], "cid": root["cid"]}, "parent": parent}
|
||||
|
||||
def _record(self, post: "Post", text: Optional[str] = None, embed: Optional[dict] = None,
|
||||
anchors: List[tuple] = ()) -> dict:
|
||||
anchors: List[tuple] = (), reply: Optional[dict] = None) -> dict:
|
||||
if text is None:
|
||||
body, anchors = plain_text_links(post.note.content, links="anchor")
|
||||
text = compose_text(post.note, post.permalink, post.tags, post.max_chars, body=body)
|
||||
# An article is posted as a teaser (compose_text), so its links aren't in the text
|
||||
body, anchors = (None, []) if post.note.title else \
|
||||
plain_text_links(post.note.content, links="anchor")
|
||||
lead = ""
|
||||
if post.reply_to and not reply:
|
||||
# "↩ Replying to <name>", the name linking to the post (v2.20.0)
|
||||
lead = reply_line(post, with_url=False)
|
||||
anchors = [(reply_label(post), post.reply_to), *anchors]
|
||||
text = compose_text(post.note, post.permalink, post.tags, post.max_chars, body=body,
|
||||
lead=lead)
|
||||
record = {"$type": self.POST, "text": text, "createdAt": self._now()}
|
||||
if reply:
|
||||
record["reply"] = reply
|
||||
facets = bluesky_facets(text, anchors)
|
||||
if facets:
|
||||
record["facets"] = facets
|
||||
@@ -368,14 +441,16 @@ class BlueskyClient:
|
||||
elif post.media:
|
||||
record["embed"] = {"$type": "app.bsky.embed.images",
|
||||
"images": [self._image(m) for m in post.media]}
|
||||
elif note_links(post.note.content):
|
||||
elif not post.note.title and note_links(post.note.content):
|
||||
# The note links somewhere: the card shows that page, not the note
|
||||
record["embed"] = self._link_card(note_links(post.note.content)[0])
|
||||
else:
|
||||
# Articles always get a card for the article itself (v2.16.0)
|
||||
description = article_teaser(post.note) if post.note.title else plain_text(post.note.content)
|
||||
record["embed"] = {"$type": "app.bsky.embed.external", "external": {
|
||||
"uri": post.permalink,
|
||||
"title": post.note.title[:300],
|
||||
"description": plain_text(post.note.content)[:300],
|
||||
"title": post.note.label[:300],
|
||||
"description": description[:300],
|
||||
}}
|
||||
return record
|
||||
|
||||
@@ -455,7 +530,7 @@ class BlueskyClient:
|
||||
existing = self._find_existing(post.permalink)
|
||||
if existing:
|
||||
return existing
|
||||
record = self._record(post)
|
||||
record = self._record(post, reply=self._reply_ref(post))
|
||||
response = self._call("POST", "com.atproto.repo.createRecord",
|
||||
json={"repo": self.did or self.identifier, "collection": self.POST,
|
||||
"record": record})
|
||||
@@ -834,15 +909,26 @@ class ThreadsClient:
|
||||
def _text(self, post: "Post") -> str:
|
||||
# Links written "text (url)"; the first one gets Threads' preview card.
|
||||
# Meta allows 5 different links per post and the permalink is one.
|
||||
body = plain_text(post.note.content, links="inline", max_urls=THREADS_MAX_LINKS - 1)
|
||||
# A reply to someone else's post says so, with its link (v2.20.0);
|
||||
# one to our own Threads post is a native reply instead
|
||||
lead = "" if self._reply_to_id(post) else reply_line(post)
|
||||
links = THREADS_MAX_LINKS - 1 - (1 if lead else 0)
|
||||
body = plain_text(post.note.content, links="inline", max_urls=links)
|
||||
limit = post.max_chars
|
||||
while True:
|
||||
text = compose_text(post.note, post.permalink, [], limit, body=body)
|
||||
text = compose_text(post.note, post.permalink, [], limit, body=body, lead=lead)
|
||||
over = threads_length(text) - post.max_chars
|
||||
if over <= 0:
|
||||
return text
|
||||
limit -= over
|
||||
|
||||
@staticmethod
|
||||
def _reply_to_id(post: "Post") -> Optional[str]:
|
||||
"""Media id of our own Threads post this replies to. The API can't look
|
||||
up other people's posts by URL, so only those thread natively."""
|
||||
remote = post.reply_remote_id or ""
|
||||
return remote if post.reply_to and remote.isdigit() else None
|
||||
|
||||
def _image_container(self, uid: str, media: dict, **extra) -> str:
|
||||
params = {"media_type": "IMAGE", "image_url": media["url"], **extra}
|
||||
if media["description"]:
|
||||
@@ -865,6 +951,8 @@ class ThreadsClient:
|
||||
return self._call("POST", f"{uid}/threads", media_type="TEXT",
|
||||
text=self._repost_text(post))["id"]
|
||||
common = {"text": self._text(post)}
|
||||
if self._reply_to_id(post):
|
||||
common["reply_to_id"] = self._reply_to_id(post)
|
||||
topic = threads_topic(post.tags)
|
||||
if topic:
|
||||
common["topic_tag"] = topic
|
||||
@@ -1002,6 +1090,22 @@ class Post:
|
||||
kind: str = "note" # note, like or repost (v2.5.0)
|
||||
target_url: Optional[str] = None # what a like/repost points at
|
||||
target_title: Optional[str] = None # its fetched title, if any
|
||||
reply_to: Optional[str] = None # what a reply answers (v2.20.0)
|
||||
reply_name: Optional[str] = None # its author (or title), from the reply context
|
||||
reply_remote_id: Optional[str] = None # its crosspost on this target, if it's one of ours
|
||||
|
||||
|
||||
def reply_line(post: "Post", with_url: bool = True) -> str:
|
||||
"""'↩ Replying to <who>' (and the URL) for replies that can't thread natively (v2.20.0)"""
|
||||
if not post.reply_to:
|
||||
return ""
|
||||
who = post.reply_name or urlparse(post.reply_to).hostname or post.reply_to
|
||||
return f"↩ Replying to {who}" + (f"\n{post.reply_to}" if with_url else "")
|
||||
|
||||
|
||||
def reply_label(post: "Post") -> str:
|
||||
"""The linked part of a Bluesky reply line"""
|
||||
return post.reply_name or urlparse(post.reply_to).hostname or post.reply_to
|
||||
|
||||
|
||||
def compose_repost_text(post: "Post", comment: str, max_chars: int) -> str:
|
||||
@@ -1074,8 +1178,7 @@ def get_targets() -> Dict[str, Target]:
|
||||
if token:
|
||||
client = _client_cache.get("threads")
|
||||
if client is None or client.config_token != token:
|
||||
client = ThreadsClient(token, Path(config["DATA_PATH"]) / "threads_token.json",
|
||||
api=config.get("THREADS_API") or THREADS_API)
|
||||
client = ThreadsClient(token, Path(config["DATA_PATH"]) / "threads_token.json")
|
||||
_client_cache["threads"] = client
|
||||
username = (config.get("THREADS_USERNAME") or "").lstrip("@")
|
||||
targets["threads"] = Target(
|
||||
@@ -1186,25 +1289,41 @@ def _hashtag(display_name: str) -> str:
|
||||
return f"#{word}" if word else ""
|
||||
|
||||
|
||||
def article_teaser(note) -> str:
|
||||
"""An article's summary, or the start of its text (v2.16.0)"""
|
||||
return note.summary or note.excerpt
|
||||
|
||||
|
||||
def compose_text(note, permalink: str, tags: List[str], max_chars: int,
|
||||
body: Optional[str] = None) -> str:
|
||||
body: Optional[str] = None, lead: str = "") -> str:
|
||||
"""
|
||||
Full note (Markdown unless `body` is given), then the permalink and
|
||||
hashtags; shortened at a word boundary if too long. On short targets the
|
||||
hashtags may use at most a third of the space.
|
||||
|
||||
An article (a note with a title) is posted as its title and teaser
|
||||
instead of its full text, the usual POSSE form for articles (v2.16.0).
|
||||
"""
|
||||
hashtags = ""
|
||||
for h in (_hashtag(t) for t in tags):
|
||||
if h and len(hashtags) + len(h) + 1 <= max(max_chars // 3, 1):
|
||||
hashtags = f"{hashtags} {h}".strip()
|
||||
tail = f"\n\n{permalink}" + (f"\n\n{hashtags}" if hashtags else "")
|
||||
body = (note.content if body is None else body).strip()
|
||||
if getattr(note, "title", None):
|
||||
body = f"{note.title}\n\n{article_teaser(note)}".strip()
|
||||
else:
|
||||
body = (note.content if body is None else body).strip()
|
||||
if getattr(note, "bookmark_of", None):
|
||||
# A bookmark leads with what it bookmarks (v2.9.0)
|
||||
body = f"🔖 {note.bookmark_of}\n\n{body}".strip()
|
||||
if len(body) + len(tail) > max_chars:
|
||||
room = max_chars - len(tail) - 1
|
||||
body = body[:room].rsplit(" ", 1)[0].rstrip() + "…"
|
||||
# A reply that can't thread says what it replies to (v2.20.0); the lead
|
||||
# is kept whole and the body shortened instead
|
||||
lead = lead.strip()
|
||||
head = f"{lead}\n\n" if lead and body else lead
|
||||
if len(head) + len(body) + len(tail) > max_chars:
|
||||
room = max_chars - len(tail) - len(head) - 1
|
||||
body = body[:max(room, 0)].rsplit(" ", 1)[0].rstrip() + "…"
|
||||
body = head + body
|
||||
# A photo post may have no text: start with the link (v2.6.5)
|
||||
return (body + tail).lstrip()
|
||||
|
||||
@@ -1216,7 +1335,10 @@ def _note_soup(markdown_text: str):
|
||||
import markdown
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
return BeautifulSoup(markdown.markdown(markdown_text, extensions=["fenced_code"]), "html.parser")
|
||||
from starpunk.markdown_ext import HashtagLineExtension
|
||||
|
||||
html = markdown.markdown(markdown_text, extensions=["fenced_code", HashtagLineExtension()])
|
||||
return BeautifulSoup(html, "html.parser")
|
||||
|
||||
|
||||
def _clean_url(url: str) -> str:
|
||||
@@ -1488,6 +1610,48 @@ def _target_title(url: Optional[str]) -> Optional[str]:
|
||||
return " ".join(title.split())[:200] or None
|
||||
|
||||
|
||||
def _own_note_id(url: str) -> Optional[int]:
|
||||
"""The note a URL is, if it's one of ours: its page on this site, or one
|
||||
of its crossposts on any network (v2.20.0)"""
|
||||
db = get_db(current_app)
|
||||
row = db.execute("SELECT note_id FROM syndications WHERE remote_url = ? AND state = 'synced'",
|
||||
(url,)).fetchone()
|
||||
if row:
|
||||
return row["note_id"]
|
||||
site = urlparse(current_app.config.get("SITE_URL", ""))
|
||||
parsed = urlparse(url)
|
||||
parts = [p for p in parsed.path.split("/") if p]
|
||||
if parsed.netloc != site.netloc or len(parts) != 2 or \
|
||||
parts[0] not in ("note", "notes", "article", "articles"):
|
||||
return None
|
||||
row = db.execute("SELECT id FROM notes WHERE slug = ? AND deleted_at IS NULL",
|
||||
(parts[1],)).fetchone()
|
||||
return row["id"] if row else None
|
||||
|
||||
|
||||
def _reply_fields(url: Optional[str], target_id: str) -> dict:
|
||||
"""Post fields for a reply: who it answers, and its crosspost on this target if it's ours"""
|
||||
if not url:
|
||||
return {}
|
||||
fields = {"reply_to": url}
|
||||
try:
|
||||
from starpunk.webmention_queries import get_reply_context
|
||||
context = get_reply_context(url) or {}
|
||||
if context.get("fetch_status") == "fetched":
|
||||
name = context.get("author_name") or context.get("title")
|
||||
fields["reply_name"] = " ".join(name.split())[:100] if name else None
|
||||
note_id = _own_note_id(url)
|
||||
if note_id is not None:
|
||||
row = get_db(current_app).execute(
|
||||
"SELECT remote_id FROM syndications WHERE note_id = ? AND target = ? "
|
||||
"AND state = 'synced' AND remote_id IS NOT NULL", (note_id, target_id)).fetchone()
|
||||
if row:
|
||||
fields["reply_remote_id"] = row["remote_id"]
|
||||
except Exception as e:
|
||||
current_app.logger.info(f"Reply details for {url}: {e}")
|
||||
return fields
|
||||
|
||||
|
||||
def _reconcile(row, target: Target) -> tuple:
|
||||
"""Bring the remote post in line with the note. Returns (remote_id, remote_url)."""
|
||||
from starpunk.notes import get_note
|
||||
@@ -1507,14 +1671,16 @@ def _reconcile(row, target: Target) -> tuple:
|
||||
site = current_app.config["SITE_URL"].rstrip("/")
|
||||
post = Post(
|
||||
note=note,
|
||||
permalink=f"{site}/note/{note.slug}",
|
||||
permalink=note.url,
|
||||
tags=[t["display_name"] for t in get_note_tags(note.id, entered_order=True)],
|
||||
max_chars=target.max_chars,
|
||||
# Articles are posted as a teaser without images (v2.16.0)
|
||||
media=_media_files(note.id, target.max_media, note.content)
|
||||
if note.kind == "note" else [],
|
||||
if note.kind == "note" and not note.title else [],
|
||||
kind=note.kind,
|
||||
target_url=note.reacts_to,
|
||||
target_title=_target_title(note.reacts_to),
|
||||
**(_reply_fields(note.in_reply_to, target.id) if note.kind == "note" else {}),
|
||||
)
|
||||
|
||||
if row["remote_id"]:
|
||||
|
||||
@@ -480,6 +480,27 @@ def validate_note_path(file_path: Path, data_dir: Path) -> bool:
|
||||
# Atomic File Operations
|
||||
|
||||
|
||||
def find_trashed_file(data_dir: Path, file_path: str) -> Optional[Path]:
|
||||
"""
|
||||
Where delete_note_file put a note's file: .trash/<year>/<month>/<name>
|
||||
|
||||
Args:
|
||||
data_dir: Data directory
|
||||
file_path: The note's file_path as stored in the database
|
||||
|
||||
Returns:
|
||||
Path of the trashed file, or None if it can't be found
|
||||
"""
|
||||
parts = Path(file_path).parts
|
||||
name = Path(file_path).name
|
||||
candidates = []
|
||||
if "notes" in parts and len(parts) > parts.index("notes") + 2:
|
||||
i = parts.index("notes")
|
||||
candidates.append(data_dir / TRASH_DIR_NAME / parts[i + 1] / parts[i + 2] / name)
|
||||
candidates += sorted((data_dir / TRASH_DIR_NAME).glob(f"*/*/{name}"))
|
||||
return next((c for c in candidates if c.exists()), None)
|
||||
|
||||
|
||||
def write_note_file(file_path: Path, content: str) -> None:
|
||||
"""
|
||||
Write note content to file atomically
|
||||
@@ -695,6 +716,23 @@ def absolute_url_for(endpoint: str, **values) -> str:
|
||||
return current_app.config["SITE_URL"].rstrip("/") + url_for(endpoint, **values)
|
||||
|
||||
|
||||
def post_path(slug: str, title: Optional[str] = None) -> str:
|
||||
"""A post's URL path: /article/<slug> when it has a title, else /note/<slug> (v2.17.0)"""
|
||||
return f"/{'article' if title else 'note'}/{slug}"
|
||||
|
||||
|
||||
def post_url(slug: str, title: Optional[str] = None) -> str:
|
||||
"""
|
||||
A post's absolute URL (see post_path)
|
||||
|
||||
Built from SITE_URL without url_for, so it works outside a request:
|
||||
crossposting and backfeed run in background workers (v2.17.0).
|
||||
"""
|
||||
from flask import current_app
|
||||
|
||||
return current_app.config["SITE_URL"].rstrip("/") + post_path(slug, title)
|
||||
|
||||
|
||||
def iso_utc(value) -> str:
|
||||
"""
|
||||
Format a timestamp as RFC 3339 UTC for dt-* and Micropub (v2.8.0)
|
||||
|
||||
@@ -112,7 +112,8 @@ def get_webmentions_for_note(note_id: int) -> Dict[str, List[Dict[str, Any]]]:
|
||||
|
||||
def response_counts(note_ids: List[int]) -> Dict[int, Dict[str, int]]:
|
||||
"""
|
||||
Likes, reposts and replies per note, for lists of notes (v2.13.4).
|
||||
Likes, reposts, replies and quotes per note, for lists of notes
|
||||
(v2.13.4; quotes v2.14.2).
|
||||
|
||||
Shown responses (valid and approved) plus the counts silos give without
|
||||
names (Threads; people who opted out of backfeed) on synced crossposts.
|
||||
@@ -126,21 +127,22 @@ def response_counts(note_ids: List[int]) -> Dict[int, Dict[str, int]]:
|
||||
|
||||
def add(note_id, key, n):
|
||||
if n:
|
||||
counts.setdefault(note_id, {'likes': 0, 'reposts': 0, 'replies': 0})[key] += n
|
||||
counts.setdefault(note_id, {'likes': 0, 'reposts': 0, 'replies': 0, 'quotes': 0})[key] += n
|
||||
|
||||
for row in db.execute(
|
||||
f"""
|
||||
SELECT target_note_id, mention_type, COUNT(*) FROM webmentions
|
||||
WHERE target_note_id IN ({marks}) AND validation_status = 'valid'
|
||||
AND moderation_status = 'approved' AND mention_type IN ('like', 'repost', 'reply')
|
||||
AND moderation_status = 'approved' AND mention_type IN ('like', 'repost', 'reply', 'quote')
|
||||
GROUP BY target_note_id, mention_type
|
||||
""",
|
||||
note_ids,
|
||||
):
|
||||
add(row[0], {'like': 'likes', 'repost': 'reposts', 'reply': 'replies'}[row[1]], row[2])
|
||||
add(row[0], {'like': 'likes', 'repost': 'reposts', 'reply': 'replies', 'quote': 'quotes'}[row[1]], row[2])
|
||||
for row in db.execute(
|
||||
f"""
|
||||
SELECT note_id, SUM(COALESCE(likes_count, 0)), SUM(COALESCE(reposts_count, 0))
|
||||
SELECT note_id, SUM(COALESCE(likes_count, 0)), SUM(COALESCE(reposts_count, 0)),
|
||||
SUM(COALESCE(quotes_count, 0))
|
||||
FROM syndications WHERE note_id IN ({marks}) AND state = 'synced'
|
||||
GROUP BY note_id
|
||||
""",
|
||||
@@ -148,6 +150,7 @@ def response_counts(note_ids: List[int]) -> Dict[int, Dict[str, int]]:
|
||||
):
|
||||
add(row[0], 'likes', row[1])
|
||||
add(row[0], 'reposts', row[2])
|
||||
add(row[0], 'quotes', row[3])
|
||||
return counts
|
||||
|
||||
|
||||
|
||||
@@ -22,6 +22,22 @@ from urllib.parse import urlparse
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
SKIPPED_NOTE = "Not sent: links to this site and WEBMENTION_SKIP_HOSTS are skipped"
|
||||
|
||||
|
||||
def skipped_hosts(config) -> set[str]:
|
||||
"""Hosts that never get webmentions: this site and WEBMENTION_SKIP_HOSTS (v2.18.3)"""
|
||||
hosts = set(config.get("WEBMENTION_SKIP_HOSTS") or [])
|
||||
site = urlparse(config.get("SITE_URL") or "").hostname
|
||||
if site:
|
||||
hosts.add(site.lower())
|
||||
return hosts
|
||||
|
||||
|
||||
def is_skipped(url: str, hosts: set[str]) -> bool:
|
||||
"""True if url is on one of the skipped hosts (exact host, any port)"""
|
||||
return (urlparse(url).hostname or "").lower() in hosts
|
||||
|
||||
|
||||
def extract_links_from_html(html_content: str, exclude_code_blocks: bool = True) -> list[str]:
|
||||
"""
|
||||
@@ -189,17 +205,26 @@ def queue_webmentions_for_note(
|
||||
Returns:
|
||||
Number of webmentions queued
|
||||
"""
|
||||
# Links to this site or a skipped host are dropped before the cap (v2.18.3):
|
||||
# this site only takes webmentions for its posts, and its own public
|
||||
# address may not be reachable from inside the container
|
||||
from flask import current_app, has_app_context
|
||||
hosts = skipped_hosts(current_app.config) if has_app_context() else set()
|
||||
|
||||
def wanted(urls):
|
||||
return [u for u in urls if not is_skipped(u, hosts)]
|
||||
|
||||
# Targets for the current content: reply target first, then links in
|
||||
# document order, at most 10
|
||||
links = extract_links_from_html(content_html) if content_html else []
|
||||
current = _dedupe(([in_reply_to] if in_reply_to else []) + links)[:10]
|
||||
current = wanted(_dedupe(([in_reply_to] if in_reply_to else []) + links))[:10]
|
||||
|
||||
if old_content_html is not None:
|
||||
# Update (spec 3.1.4): also re-send to every target notified before,
|
||||
# and to links the edit removed, so they can update or remove the
|
||||
# mention. These are not capped.
|
||||
removed = [u for u in extract_links_from_html(old_content_html) if u not in links]
|
||||
target_urls = _dedupe(current + previous_targets(db_path, note_id) + removed)
|
||||
target_urls = _dedupe(current + wanted(previous_targets(db_path, note_id) + removed))
|
||||
else:
|
||||
target_urls = current
|
||||
|
||||
|
||||
@@ -117,6 +117,12 @@ class WebmentionWorker(threading.Thread):
|
||||
|
||||
# Phase 3: Process outgoing webmentions
|
||||
processed += self._process_pending_sends()
|
||||
# Reply contexts stored before v2.19.0 are fetched once more
|
||||
try:
|
||||
from starpunk.reply_context import refresh_stale
|
||||
refresh_stale(self.db_path, self.site_url)
|
||||
except Exception as e:
|
||||
logger.error(f"Reply context refresh failed: {e}", exc_info=True)
|
||||
# Local copies of response avatars (v2.13.0)
|
||||
self._avatar_upkeep.run(self.db_path, self.app.config['DATA_PATH'],
|
||||
self.site_url)
|
||||
@@ -205,6 +211,40 @@ class WebmentionWorker(threading.Thread):
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to cleanup orphaned items: {e}")
|
||||
|
||||
self._drop_skipped_targets()
|
||||
|
||||
def _skipped_hosts(self) -> set:
|
||||
from starpunk.webmention_sender import skipped_hosts
|
||||
return skipped_hosts(self.app.config) if self.app is not None else set()
|
||||
|
||||
def _drop_skipped_targets(self):
|
||||
"""
|
||||
Stop sending to this site and WEBMENTION_SKIP_HOSTS (v2.18.3)
|
||||
|
||||
Queued or failed sends to those hosts are set to 'no_endpoint' (nothing
|
||||
to send to), so they leave the retry queue and the admin Log.
|
||||
"""
|
||||
from starpunk.webmention_sender import SKIPPED_NOTE, is_skipped
|
||||
|
||||
hosts = self._skipped_hosts()
|
||||
if not hosts:
|
||||
return
|
||||
try:
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
rows = conn.execute(
|
||||
"SELECT id, target_url FROM webmention_queue WHERE status IN ('pending', 'failed')"
|
||||
).fetchall()
|
||||
ids = [(SKIPPED_NOTE, i) for i, url in rows if is_skipped(url, hosts)]
|
||||
if ids:
|
||||
conn.executemany(
|
||||
"UPDATE webmention_queue SET status = 'no_endpoint', last_error = ? WHERE id = ?",
|
||||
ids)
|
||||
conn.commit()
|
||||
logger.info(f"Dropped {len(ids)} queued webmention(s) to skipped hosts")
|
||||
conn.close()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to drop skipped webmention targets: {e}")
|
||||
|
||||
def _process_pending_validations(self) -> int:
|
||||
"""
|
||||
Process pending incoming webmentions
|
||||
@@ -583,6 +623,15 @@ class WebmentionWorker(threading.Thread):
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
from starpunk.webmention_sender import SKIPPED_NOTE, is_skipped
|
||||
if is_skipped(target_url, self._skipped_hosts()):
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
conn.execute("UPDATE webmention_queue SET status = 'no_endpoint', last_error = ? WHERE id = ?",
|
||||
(SKIPPED_NOTE, queue_id))
|
||||
conn.commit()
|
||||
conn.close()
|
||||
return
|
||||
|
||||
try:
|
||||
# Discover endpoint if not cached
|
||||
endpoint = cached_endpoint
|
||||
|
||||
+41
-25
@@ -8,7 +8,7 @@
|
||||
--font-reading: Charter, 'Bitstream Charter', 'Sitka Text', Cambria, 'Noto Serif', Georgia, serif;
|
||||
--font-mono: ui-monospace, 'SF Mono', Monaco, Consolas, 'Liberation Mono', monospace;
|
||||
--spacing-xs: 0.25rem; --spacing-sm: 0.5rem; --spacing-md: 1rem; --spacing-lg: 2rem; --spacing-xl: 4rem;
|
||||
--max-width: 40rem; --border-radius: 6px;
|
||||
--max-width: 48rem; --frame-width: 60rem; --border-radius: 6px;
|
||||
}
|
||||
|
||||
@media (prefers-color-scheme: dark) {
|
||||
@@ -40,6 +40,9 @@ pre code { padding: 0; background: none; }
|
||||
|
||||
/* Page frame */
|
||||
.site-header, main, .site-footer { width: 100%; max-width: var(--max-width); margin-left: auto; margin-right: auto; }
|
||||
/* Header and footer share one width on every page, public and admin (v2.18.2);
|
||||
only the content column (main) differs: 48rem to read, 60rem for admin tables. */
|
||||
.site-header, .site-footer { max-width: var(--frame-width); }
|
||||
main { flex: 1; margin-bottom: var(--spacing-xl); }
|
||||
|
||||
.site-header {
|
||||
@@ -91,6 +94,15 @@ main { flex: 1; margin-bottom: var(--spacing-xl); }
|
||||
.e-content blockquote { margin: 0 0 var(--spacing-md); padding-left: var(--spacing-md); border-left: 3px solid var(--color-border); color: var(--color-text-light); }
|
||||
.e-content ul, .e-content ol { margin: 0 0 var(--spacing-md) 1.25rem; }
|
||||
.e-content img { max-width: 100%; height: auto; }
|
||||
/* Tables (v2.18.2): a card that fills the column, in the page's sans font at
|
||||
the reading size; scrolls sideways when wider than the column. Words in
|
||||
cells are never split (overflow-wrap: anywhere above is for long URLs in
|
||||
prose; in a cell it squeezed columns to a few letters). */
|
||||
.e-content .table-wrap { overflow-x: auto; margin: 0 0 var(--spacing-md); background: var(--color-bg-alt); border: 1px solid var(--color-border); border-radius: var(--border-radius); }
|
||||
.e-content table { width: 100%; border-collapse: collapse; font-family: var(--font-body); font-size: 1.0625rem; line-height: 1.6; }
|
||||
.e-content th, .e-content td { padding: 0.9rem 1.25rem; text-align: left; vertical-align: top; overflow-wrap: normal; }
|
||||
.e-content th { font-weight: 600; white-space: nowrap; background: color-mix(in srgb, var(--color-text) 6%, var(--color-bg-alt)); }
|
||||
.e-content td:first-child { font-weight: 600; }
|
||||
.e-content h1, .e-content h2, .e-content h3 { font-family: var(--font-body); }
|
||||
.reaction { margin-bottom: var(--spacing-sm); font-size: 0.9375rem; color: var(--color-text-light); overflow-wrap: anywhere; }
|
||||
.reaction-verb { font-weight: 600; margin-right: 0.25rem; }
|
||||
@@ -266,30 +278,27 @@ small { display: block; margin-top: var(--spacing-xs); color: var(--color-text-l
|
||||
|
||||
/* Webmention Styles (v1.6.0 Phase 5) */
|
||||
|
||||
/* Reply Context */
|
||||
.reply-context {
|
||||
margin-bottom: var(--spacing-md);
|
||||
padding: var(--spacing-sm);
|
||||
border-left: 3px solid var(--color-link);
|
||||
background: var(--color-bg-alt);
|
||||
}
|
||||
|
||||
.reply-context .reply-label {
|
||||
font-size: 0.875rem;
|
||||
color: var(--color-text-light);
|
||||
display: block;
|
||||
margin-bottom: var(--spacing-xs);
|
||||
}
|
||||
|
||||
.reply-context blockquote {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.reply-context .p-content {
|
||||
font-style: italic;
|
||||
margin-bottom: var(--spacing-xs);
|
||||
}
|
||||
/* Reply Context (card v2.19.0) */
|
||||
.reply-context { margin-bottom: var(--spacing-md); }
|
||||
.reply-context .reply-label { display: block; margin-bottom: var(--spacing-xs); font-size: 0.875rem; font-weight: 600; color: var(--color-text-light); }
|
||||
.reply-card { margin: 0; padding: var(--spacing-sm) var(--spacing-md); border-left: 3px solid var(--color-link); background: var(--color-bg-alt); border-radius: 0 var(--border-radius) var(--border-radius) 0; overflow-wrap: anywhere; }
|
||||
.reply-author { margin-bottom: var(--spacing-xs); }
|
||||
.reply-author .h-card { display: flex; align-items: center; gap: var(--spacing-sm); }
|
||||
.reply-author img, .reply-author .face-initial { width: 40px; height: 40px; border-radius: 50%; flex: none; }
|
||||
.reply-author .face-initial { display: inline-flex; align-items: center; justify-content: center; color: #fff; font-weight: 700; }
|
||||
.reply-who { display: flex; flex-wrap: wrap; column-gap: 0.4rem; line-height: 1.3; min-width: 0; }
|
||||
.reply-who .p-name { font-weight: 600; color: var(--color-text); }
|
||||
.reply-who a.p-name:hover { color: var(--color-link); }
|
||||
.reply-handle { font-size: 0.8125rem; color: var(--color-text-light); }
|
||||
.reply-title { margin: 0 0 var(--spacing-xs); font-weight: 600; }
|
||||
.reply-content { font-size: 0.9375rem; }
|
||||
.reply-content p { margin-bottom: var(--spacing-xs); }
|
||||
.reply-content > :last-child { margin-bottom: 0; }
|
||||
.reply-photo { display: block; max-width: 100%; max-height: 20rem; width: auto; height: auto; margin-top: var(--spacing-sm); border-radius: var(--border-radius); }
|
||||
.reply-source { margin-top: var(--spacing-xs); font-size: 0.8125rem; }
|
||||
.reply-source a { color: var(--color-text-light); }
|
||||
.reply-source a:hover { color: var(--color-link); }
|
||||
.reply-quote { display: block; margin-top: 0.15rem; padding-left: var(--spacing-sm); border-left: 2px solid var(--color-border); color: var(--color-text-light); font-style: italic; quotes: none; }
|
||||
|
||||
/* Webmentions Section */
|
||||
.webmentions {
|
||||
@@ -465,6 +474,13 @@ small { display: block; margin-top: var(--spacing-xs); color: var(--color-text-l
|
||||
.badge-like { background: #FECACA; color: #991B1B; }
|
||||
.badge-repost { background: #D1FAE5; color: #065F46; }
|
||||
.badge-bookmark { background: #FEF3C7; color: #92400E; }
|
||||
.badge-error { background: var(--color-error); color: #fff; }
|
||||
|
||||
/* Admin Log (v2.18.0) */
|
||||
.log-table td { vertical-align: top; }
|
||||
.log-table .button { white-space: nowrap; }
|
||||
.log-error code { white-space: pre-wrap; overflow-wrap: anywhere; font-size: 0.85rem; }
|
||||
.backfeed-status { margin-bottom: var(--spacing-lg); }
|
||||
.badge-mention { background: #E5E7EB; color: #374151; }
|
||||
|
||||
.status-badge {
|
||||
|
||||
@@ -1,27 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>IndieLogin Test Form</title>
|
||||
</head>
|
||||
<body>
|
||||
<h1>IndieLogin Test Form</h1>
|
||||
<p>This is the exact form from IndieLogin.com API docs</p>
|
||||
|
||||
<form action="https://indielogin.com/authorize" method="get">
|
||||
<label for="url">Web Address:</label>
|
||||
<input id="url" type="text" name="me" placeholder="yourdomain.com" value="https://thesatelliteoflove.com" />
|
||||
<p><button type="submit">Sign In</button></p>
|
||||
<input type="hidden" name="client_id" value="https://starpunk.thesatelliteoflove.com/" />
|
||||
<input type="hidden" name="redirect_uri" value="https://starpunk.thesatelliteoflove.com/auth/callback" />
|
||||
<input type="hidden" name="state" value="TESTSTATE123456789" />
|
||||
<input type="hidden" name="code_challenge" value="E9Melhoa2OwvFrEMTJguCHaoeK1t8URWbuGJSstw-cM" />
|
||||
<input type="hidden" name="code_challenge_method" value="S256" />
|
||||
</form>
|
||||
|
||||
<hr>
|
||||
<p><strong>Note:</strong> This uses a fixed code_challenge for testing. In production, this should be generated fresh each time.</p>
|
||||
<p><strong>Form will submit to:</strong> https://indielogin.com/authorize</p>
|
||||
</body>
|
||||
</html>
|
||||
Whitespace-only changes.
@@ -14,6 +14,11 @@
|
||||
<strong>{{ auth_request.client_id }}</strong> wants to sign in as
|
||||
<strong>{{ me }}</strong>.
|
||||
</p>
|
||||
{% if auth_request.resource %}
|
||||
<p>This is an AI assistant asking to use your site through MCP
|
||||
(<code>{{ auth_request.resource }}</code>). It can read your notes and drafts,
|
||||
and do whatever you allow below.</p>
|
||||
{% endif %}
|
||||
<p><small>After approval you will be sent to {{ auth_request.redirect_uri }}</small></p>
|
||||
|
||||
<form action="{{ url_for('indieauth.approve') }}" method="POST" class="login-form"><input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
|
||||
|
||||
@@ -8,7 +8,10 @@
|
||||
<nav class="admin-nav">
|
||||
<a href="{{ url_for('admin.dashboard') }}">Dashboard</a>
|
||||
<a href="{{ url_for('admin.new_note_form') }}">New Note</a>
|
||||
{% if config.WEBMENTION_ENABLED %}<a href="{{ url_for('admin.webmentions') }}">Webmentions</a>{% endif %}
|
||||
<a href="{{ url_for('admin.error_log') }}">Log{% if error_count %} <span class="badge badge-error">{{ error_count }}</span>{% endif %}</a>
|
||||
<a href="{{ url_for('admin.tokens') }}">Connected Apps</a>
|
||||
<a href="{{ url_for('admin.settings') }}">Settings</a>
|
||||
<form action="{{ url_for('auth.logout') }}" method="POST" class="logout-form"><input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
|
||||
<button type="submit" class="button button-secondary">Logout</button>
|
||||
</form>
|
||||
|
||||
@@ -51,7 +51,7 @@
|
||||
<tr>
|
||||
<td class="note-content-preview">
|
||||
<div class="note-excerpt">
|
||||
{{ note.content[:100] }}{% if note.content|length > 100 %}...{% endif %}
|
||||
{% if note.title %}<strong>{{ note.title }}</strong>{% else %}{{ note.content[:100] }}{% if note.content|length > 100 %}...{% endif %}{% endif %}
|
||||
</div>
|
||||
<small class="note-slug">{{ note.slug }}</small>
|
||||
</td>
|
||||
@@ -67,7 +67,7 @@
|
||||
</td>
|
||||
<td class="note-actions">
|
||||
{% if note.published %}
|
||||
<a href="{{ url_for('public.note', slug=note.slug) }}" class="button button-small" target="_blank">
|
||||
<a href="{{ note.url }}" class="button button-small" target="_blank">
|
||||
View
|
||||
</a>
|
||||
{% endif %}
|
||||
|
||||
@@ -11,6 +11,12 @@
|
||||
</p>
|
||||
|
||||
<form action="{{ url_for('admin.update_note_submit', note_id=note.id) }}" method="POST" class="note-form"><input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
|
||||
<div class="form-group">
|
||||
<label for="title">Title (optional)</label>
|
||||
<input type="text" id="title" name="title" value="{{ note.title or '' }}" maxlength="200"
|
||||
placeholder="Leave blank for a note; a title makes it an article">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
{% if note.reacts_to %}
|
||||
<p class="note-meta">{{ {'like': '★ Like of', 'repost': '🔁 Repost of', 'bookmark': '🔖 Bookmark of'}[note.kind] }} <a href="{{ note.reacts_to }}">{{ note.reacts_to }}</a></p>
|
||||
@@ -20,12 +26,18 @@
|
||||
id="content"
|
||||
name="content"
|
||||
rows="{{ 6 if note.reacts_to else 20 }}"
|
||||
{% if not note.reacts_to %}required{% endif %}
|
||||
autofocus
|
||||
>{{ note.content }}</textarea>
|
||||
<small>Use Markdown syntax for formatting</small>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="summary">Summary (optional)</label>
|
||||
<textarea id="summary" name="summary" rows="2"
|
||||
placeholder="A sentence or two for the homepage, feeds and crossposts">{{ note.summary or '' }}</textarea>
|
||||
<small>Articles only. Leave blank to use the start of the text.</small>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="slug">Slug (permanent)</label>
|
||||
<input type="text"
|
||||
@@ -116,8 +128,3 @@
|
||||
</div>
|
||||
{% endblock %}
|
||||
|
||||
{% block head %}
|
||||
{{ super() }}
|
||||
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
|
||||
<script src="{{ url_for('static', filename='js/preview.js') }}"></script>
|
||||
{% endblock %}
|
||||
@@ -0,0 +1,57 @@
|
||||
{% extends "admin/base.html" %}
|
||||
|
||||
{% block title %}Log - StarPunk Admin{% endblock %}
|
||||
|
||||
{% block admin_content %}
|
||||
<div class="log-admin">
|
||||
<h2>Log</h2>
|
||||
<p class="text-muted">What is failing right now. An entry goes away once the next attempt succeeds.</p>
|
||||
|
||||
{% if entries %}
|
||||
<div class="table-scroll">
|
||||
<table class="note-table log-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>When (UTC)</th>
|
||||
<th>What</th>
|
||||
<th>Post</th>
|
||||
<th>Error</th>
|
||||
<th></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for e in entries %}
|
||||
<tr>
|
||||
<td class="note-date">{{ e.when or '' }}</td>
|
||||
<td>
|
||||
{{ e.area }}
|
||||
{% if e.detail %}<br><small class="note-slug">{% if e.detail.startswith('http') %}<a href="{{ e.detail.split(' ')[0] }}" target="_blank" rel="noopener">{{ e.detail|truncate(60) }}</a>{% else %}{{ e.detail|truncate(80) }}{% endif %}</small>{% endif %}
|
||||
</td>
|
||||
<td>
|
||||
{% if e.note %}
|
||||
<a href="{{ url_for('admin.edit_note_form', note_id=e.note.id) }}">{{ (e.note.title or e.note.content or e.note.slug)|truncate(60) }}</a>
|
||||
{% if e.note.published %}<br><small><a href="{{ e.note.url }}" target="_blank">View</a></small>{% endif %}
|
||||
{% else %}
|
||||
<span class="text-muted">Site-wide</span>
|
||||
{% endif %}
|
||||
</td>
|
||||
<td class="log-error"><code>{{ e.error }}</code></td>
|
||||
<td>
|
||||
{% if e.action %}
|
||||
<form method="POST" action="{{ url_for(e.action[0], syndication_id=e.action[1]) }}"><input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
|
||||
<button type="submit" class="button button-small">{{ e.action[2] }}</button>
|
||||
</form>
|
||||
{% endif %}
|
||||
</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{% else %}
|
||||
<div class="empty-state">
|
||||
<p>No errors. Crossposts, response checks, webmentions, WebSub and notifications are all working.</p>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endblock %}
|
||||
@@ -7,6 +7,12 @@
|
||||
<h2>Create New Note</h2>
|
||||
|
||||
<form action="{{ url_for('admin.create_note_submit') }}" method="POST" enctype="multipart/form-data" class="note-form"><input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
|
||||
<div class="form-group">
|
||||
<label for="title">Title (optional)</label>
|
||||
<input type="text" id="title" name="title" value="" maxlength="200"
|
||||
placeholder="Leave blank for a note; a title makes it an article">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="content">Content (Markdown)</label>
|
||||
<textarea
|
||||
@@ -14,12 +20,18 @@
|
||||
name="content"
|
||||
rows="20"
|
||||
placeholder="Write your note in markdown..."
|
||||
required
|
||||
autofocus
|
||||
></textarea>
|
||||
<small>Use Markdown syntax for formatting</small>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="summary">Summary (optional)</label>
|
||||
<textarea id="summary" name="summary" rows="2"
|
||||
placeholder="A sentence or two for the homepage, feeds and crossposts"></textarea>
|
||||
<small>Articles only. Leave blank to use the start of the text.</small>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="custom_slug">Custom Slug (optional)</label>
|
||||
<input type="text"
|
||||
@@ -88,8 +100,6 @@
|
||||
|
||||
{% block head %}
|
||||
{{ super() }}
|
||||
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
|
||||
<script src="{{ url_for('static', filename='js/preview.js') }}"></script>
|
||||
|
||||
<style>
|
||||
.media-preview {
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
{% extends "admin/base.html" %}
|
||||
|
||||
{% block title %}Settings - StarPunk Admin{% endblock %}
|
||||
|
||||
{% block admin_content %}
|
||||
<div class="settings-admin">
|
||||
<h2>Settings</h2>
|
||||
|
||||
{# Notifications (v2.14.0, ADR-074; moved here in v2.18.0) #}
|
||||
<section class="notify-status">
|
||||
<h3>Notifications</h3>
|
||||
{% if notify.enabled %}
|
||||
<p>Sending {{ notify.events|join(', ') }} to {{ notify.services|join(', ') }} at {{ notify.priority }} priority.</p>
|
||||
{% if notify.last_error %}<p class="flash flash-error">Last send failed: {{ notify.last_error }}</p>{% endif %}
|
||||
<form method="POST" action="{{ url_for('admin.notifications_test') }}"><input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
|
||||
<button type="submit" class="button">Send test notification</button>
|
||||
</form>
|
||||
{% else %}
|
||||
<p>Off. Set <code>NOTIFY_URLS</code> to one or more Apprise URLs to be notified of new responses.</p>
|
||||
{% endif %}
|
||||
</section>
|
||||
|
||||
{# Outgoing webmentions (v2.18.3) #}
|
||||
<section>
|
||||
<h3>Webmentions</h3>
|
||||
<p>Never sent to: {% for h in skip_hosts %}<code>{{ h }}</code>{{ ", " if not loop.last }}{% endfor %}.
|
||||
This site is always skipped; add others with <code>WEBMENTION_SKIP_HOSTS</code>.</p>
|
||||
</section>
|
||||
</div>
|
||||
{% endblock %}
|
||||
@@ -5,7 +5,7 @@
|
||||
{% block admin_content %}
|
||||
<div class="tokens-admin">
|
||||
<h2>Connected Apps</h2>
|
||||
<p>Micropub apps you have approved. Revoke any you no longer use.</p>
|
||||
<p>Micropub apps and AI assistants (MCP) you have approved. Revoke any you no longer use.</p>
|
||||
|
||||
{% if tokens %}
|
||||
<div class="table-scroll">
|
||||
@@ -23,7 +23,7 @@
|
||||
<tbody>
|
||||
{% for t in tokens %}
|
||||
<tr>
|
||||
<td>{{ t.client_id }}</td>
|
||||
<td>{{ t.client_id }}{% if t.resource %} <small>(MCP)</small>{% endif %}</td>
|
||||
<td>{{ t.scope }}</td>
|
||||
<td>{{ t.created_at }}</td>
|
||||
<td>{{ t.last_used_at or "never" }}</td>
|
||||
|
||||
@@ -6,35 +6,10 @@
|
||||
<div class="webmentions-admin">
|
||||
<h2>Webmentions</h2>
|
||||
|
||||
<div class="dashboard-actions">
|
||||
<a href="{{ url_for('admin.dashboard') }}" class="button">
|
||||
Back to Dashboard
|
||||
</a>
|
||||
</div>
|
||||
|
||||
{# Notifications (v2.14.0, ADR-074) #}
|
||||
{% if notify %}
|
||||
<section class="notify-status">
|
||||
<h3>Notifications</h3>
|
||||
{% if notify.enabled %}
|
||||
<p>Sending {{ notify.events|join(', ') }} to {{ notify.services|join(', ') }} at {{ notify.priority }} priority.</p>
|
||||
{% if notify.last_error %}<p class="flash flash-error">Last send failed: {{ notify.last_error }}</p>{% endif %}
|
||||
<form method="POST" action="{{ url_for('admin.notifications_test') }}"><input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
|
||||
<button type="submit" class="button">Send test notification</button>
|
||||
</form>
|
||||
{% else %}
|
||||
<p>Off. Set <code>NOTIFY_URLS</code> to one or more Apprise URLs to be notified of new responses.</p>
|
||||
{% endif %}
|
||||
</section>
|
||||
{% endif %}
|
||||
|
||||
{# Backfeed (v2.13.0, ADR-073) #}
|
||||
{% if backfeed and backfeed.enabled %}
|
||||
<section class="backfeed-status">
|
||||
<h3>Backfeed</h3>
|
||||
{% for silo, message in backfeed.denied.items() %}
|
||||
<p class="flash flash-error">{{ silo }}: {{ message }}</p>
|
||||
{% endfor %}
|
||||
{% if backfeed.remaining %}
|
||||
<p>Backfill started {{ backfeed.started }} UTC: {{ backfeed.remaining }} crosspost(s) left to check.</p>
|
||||
{% elif backfeed.finished %}
|
||||
@@ -42,7 +17,7 @@
|
||||
{% else %}
|
||||
<p>No backfill has been run. It checks every crosspost once for responses, however old the note.</p>
|
||||
{% endif %}
|
||||
{% if backfeed.errors %}<p>{{ backfeed.errors }} crosspost(s) had an error at their last check (see each note's edit page).</p>{% endif %}
|
||||
{% if backfeed.errors or backfeed.denied %}<p><a href="{{ url_for('admin.error_log') }}">Response checks have errors: see the Log.</a></p>{% endif %}
|
||||
<form method="POST" action="{{ url_for('admin.backfeed_backfill') }}"><input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
|
||||
<button type="submit" class="button"{% if backfeed.remaining %} disabled{% endif %}>Backfill responses</button>
|
||||
</form>
|
||||
|
||||
Whitespace-only changes.
+12
-32
@@ -1,44 +1,24 @@
|
||||
{% extends "base.html" %}
|
||||
{% from "partials/media.html" import display_media %}
|
||||
{% from "partials/reaction.html" import reaction %}
|
||||
{% from "partials/reply_context.html" import reply_card %}
|
||||
{% import "partials/responses.html" as responses with context %}
|
||||
|
||||
{% block title %}{% if note.has_title %}{{ note.title }}{% elif note.excerpt.strip() %}{{ note.excerpt|truncate(60) }}{% elif note.repost_of %}Repost{% elif note.like_of %}Like{% elif note.bookmark_of %}Bookmark{% elif note.in_reply_to %}Reply{% else %}Photo{% endif %} · {{ config.SITE_NAME }}{% endblock %}
|
||||
{% block title %}{% if note.title %}{{ note.title }}{% elif note.excerpt.strip() %}{{ note.excerpt|truncate(60) }}{% elif note.repost_of %}Repost{% elif note.like_of %}Like{% elif note.bookmark_of %}Bookmark{% elif note.in_reply_to %}Reply{% else %}Photo{% endif %} · {{ config.SITE_NAME }}{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<article class="h-entry">
|
||||
{# Notes starting with a '# Heading' have a title (Q22; Note.has_title) #}
|
||||
{% set has_explicit_title = note.has_title %}
|
||||
|
||||
{# p-name only if note has explicit title (per Q22) #}
|
||||
{% if has_explicit_title %}
|
||||
{# Titled notes are articles: p-name only then, so notes get no implied name (ADR-076) #}
|
||||
{% if note.title %}
|
||||
<h1 class="p-name">{{ note.title }}</h1>
|
||||
{% endif %}
|
||||
{% if note.summary %}
|
||||
<p class="p-summary">{{ note.summary }}</p>
|
||||
{% endif %}
|
||||
|
||||
{# Reply context (v1.6.0 Phase 4) - displayed if this is a reply post #}
|
||||
{# Reply context (v1.6.0 Phase 4; card v2.19.0) - displayed if this is a reply post #}
|
||||
{% if note.in_reply_to %}
|
||||
<div class="reply-context">
|
||||
<span class="reply-label">In reply to:</span>
|
||||
{# u-in-reply-to on the h-cite itself makes it the entry's property (Post Type Discovery) #}
|
||||
<blockquote class="u-in-reply-to h-cite">
|
||||
{% if reply_context and reply_context.fetch_status == 'fetched' %}
|
||||
{% if reply_context.content_snippet %}
|
||||
<p class="p-content">{{ reply_context.content_snippet }}</p>
|
||||
{% endif %}
|
||||
<cite>
|
||||
<a class="u-url" href="{{ note.in_reply_to }}">
|
||||
{% if reply_context.author_name %}{{ reply_context.author_name }}{% endif %}
|
||||
{% if reply_context.site_name %} on {{ reply_context.site_name }}{% endif %}
|
||||
{% if not reply_context.author_name and not reply_context.site_name %}{{ note.in_reply_to }}{% endif %}
|
||||
</a>
|
||||
</cite>
|
||||
{% else %}
|
||||
<cite>
|
||||
<a class="u-url" href="{{ note.in_reply_to }}">{{ note.in_reply_to }}</a>
|
||||
</cite>
|
||||
{% endif %}
|
||||
</blockquote>
|
||||
</div>
|
||||
{{ reply_card(note.in_reply_to, reply_context or (note.context if note.context is defined else None)) }}
|
||||
{% endif %}
|
||||
|
||||
{{ reaction(note) }}
|
||||
@@ -49,12 +29,12 @@
|
||||
|
||||
{# e-content: note content BELOW media (per ADR-057) #}
|
||||
<div class="e-content">
|
||||
{{ note.body_html|safe }}
|
||||
{{ note.html|safe }}
|
||||
</div>
|
||||
|
||||
<footer class="note-meta">
|
||||
{# u-url and u-uid same for notes (per Q23) #}
|
||||
<a class="u-url u-uid" href="{{ absolute_url_for('public.note', slug=note.slug) }}"><time class="dt-published" datetime="{{ note.created_at|iso_utc }}">{{ note.created_at.strftime('%B %-d, %Y · %-I:%M %p') }}</time></a>
|
||||
<a class="u-url u-uid" href="{{ note.url }}"><time class="dt-published" datetime="{{ note.created_at|iso_utc }}">{{ note.created_at.strftime('%B %-d, %Y · %-I:%M %p') }}</time></a>
|
||||
{% if note.updated_at and note.updated_at != note.created_at %}
|
||||
<span class="updated">Updated <time class="dt-updated" datetime="{{ note.updated_at|iso_utc }}">{{ note.updated_at.strftime('%b %-d, %Y') }}</time></span>
|
||||
{% endif %}
|
||||
@@ -79,7 +59,7 @@
|
||||
</footer>
|
||||
|
||||
<div id="responses"></div>
|
||||
{{ responses.reactions(likes, reposts, replies, bookmarks, syndications, absolute_url_for('public.note', slug=note.slug)) }}
|
||||
{{ responses.reactions(likes, reposts, replies, bookmarks, syndications, note.url, quotes) }}
|
||||
|
||||
<nav class="note-nav">
|
||||
<a href="/">← All notes</a>
|
||||
|
||||
@@ -1,30 +1,34 @@
|
||||
{# One note in a list (home, tag pages). v2.3.0 #}
|
||||
{% from "partials/media.html" import display_media %}
|
||||
{% from "partials/reaction.html" import reaction %}
|
||||
{% from "partials/reply_context.html" import reply_line %}
|
||||
{% from "partials/responses.html" import list_counts %}
|
||||
{% macro note_preview(note, author) %}
|
||||
<article class="h-entry note-preview">
|
||||
{# Notes starting with a '# Heading' have a title (Q22; Note.has_title) #}
|
||||
{% set has_explicit_title = note.has_title %}
|
||||
{% if has_explicit_title %}
|
||||
<h2 class="p-name"><a href="{{ absolute_url_for('public.note', slug=note.slug) }}">{{ note.title }}</a></h2>
|
||||
{# Titled notes are articles (ADR-076) #}
|
||||
{% if note.title %}
|
||||
<h2 class="p-name"><a href="{{ note.url }}">{{ note.title }}</a></h2>
|
||||
{% endif %}
|
||||
|
||||
{% if note.in_reply_to %}
|
||||
<p class="reaction reply-to"><span class="reaction-verb">↩ In reply to</span> <a class="u-in-reply-to" href="{{ note.in_reply_to }}">{{ note.in_reply_to }}</a></p>
|
||||
{{ reply_line(note.in_reply_to, note.context if note.context is defined else None) }}
|
||||
{% endif %}
|
||||
|
||||
{{ reaction(note) }}
|
||||
|
||||
{{ display_media(note.media, note.excerpt|truncate(150)) }}
|
||||
|
||||
{% if has_explicit_title %}
|
||||
{# Titled notes are articles: show a summary and link through #}
|
||||
{% if note.title %}
|
||||
{# Articles: title, summary and a link through; images stay on the article page (v2.16.0).
|
||||
The photos stay in the markup, unseen, so parsers read the same post type here as on the page. #}
|
||||
{% for item in note.media or [] %}<data class="u-photo" value="{{ url_for('public.media_file', path=item.path) }}"></data>{% endfor %}
|
||||
{% set summary = note.summary or note.excerpt %}
|
||||
{% if summary.strip() %}
|
||||
<div class="p-summary">
|
||||
<p>{{ note.excerpt }}</p>
|
||||
<p>{{ summary }}</p>
|
||||
</div>
|
||||
<p class="read-more"><a href="{{ absolute_url_for('public.note', slug=note.slug) }}">Read more →</a></p>
|
||||
{% endif %}
|
||||
<p class="read-more"><a href="{{ note.url }}">Read more →</a></p>
|
||||
{% else %}
|
||||
{{ display_media(note.media, note.excerpt|truncate(150)) }}
|
||||
{# Always present, even empty: without it a text-free photo post gets an implied name (mf2) #}
|
||||
<div class="e-content">
|
||||
{{ note.html|safe }}
|
||||
@@ -32,13 +36,13 @@
|
||||
{% endif %}
|
||||
|
||||
<footer class="note-meta">
|
||||
<a class="u-url u-uid" href="{{ absolute_url_for('public.note', slug=note.slug) }}"><time class="dt-published" datetime="{{ note.created_at|iso_utc }}">{{ note.created_at.strftime('%b %-d, %Y') }}</time></a>
|
||||
<a class="u-url u-uid" href="{{ note.url }}"><time class="dt-published" datetime="{{ note.created_at|iso_utc }}">{{ note.created_at.strftime('%b %-d, %Y') }}</time></a>
|
||||
{% if note.tags %}
|
||||
<span class="note-tags">
|
||||
{% for tag in note.tags %}<a class="p-category" rel="tag" href="{{ url_for('public.tag', tag=tag.name) }}">{{ tag.display_name }}</a>{% endfor %}
|
||||
</span>
|
||||
{% endif %}
|
||||
{{ list_counts(note.response_counts if note.response_counts is defined else None, absolute_url_for('public.note', slug=note.slug)) }}
|
||||
{{ list_counts(note.response_counts if note.response_counts is defined else None, note.url) }}
|
||||
{% if author %}
|
||||
{# Single-author site: keep the h-card for parsers, hide it from readers #}
|
||||
<span class="p-author h-card visually-hidden"><a class="p-name u-url" href="{{ author.url or author.me }}">{{ author.name or author.url or author.me }}</a></span>
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
{# The post a reply answers (v2.19.0).
|
||||
The card is the note's u-in-reply-to h-cite (Post Type Discovery reads it
|
||||
as a reply). Avatars and images are this site's saved copies; until there
|
||||
is one, the initial is shown and the image left out. #}
|
||||
|
||||
{% macro _who(ctx, url) %}{{ ctx.author_name or ctx.site_name or url.split('/')[2] }}{% endmacro %}
|
||||
|
||||
{% macro _when(ctx, fmt) %}
|
||||
{%- set when = ctx.published_at -%}
|
||||
{%- if when and when.strftime is defined -%}
|
||||
<time class="dt-published" datetime="{{ when|iso_utc }}">{{ when.strftime(fmt) }}</time>
|
||||
{%- endif -%}
|
||||
{% endmacro %}
|
||||
|
||||
{# Full card, on the note's page #}
|
||||
{% macro reply_card(url, ctx) %}
|
||||
{% set fetched = ctx and ctx.fetch_status == 'fetched' %}
|
||||
<div class="reply-context">
|
||||
<span class="reply-label">↩ In reply to</span>
|
||||
<blockquote class="u-in-reply-to h-cite reply-card">
|
||||
{% if fetched and (ctx.author_name or ctx.content_snippet or ctx.title) %}
|
||||
<header class="reply-author">
|
||||
{% set avatar = ctx.author_photo|local_avatar %}
|
||||
<span class="p-author h-card">
|
||||
{%- if avatar %}<img class="u-photo" src="{{ avatar }}" alt="" width="40" height="40" loading="lazy">
|
||||
{%- else %}<span class="face-initial" style="background-color: {{ (ctx.author_url or url)|url_to_color }}" aria-hidden="true">{{ _who(ctx, url)[0]|upper }}</span>{% endif %}
|
||||
<span class="reply-who">
|
||||
{%- if ctx.author_url %}<a class="p-name u-url" href="{{ ctx.author_url }}">{{ _who(ctx, url) }}</a>
|
||||
{%- else %}<span class="p-name">{{ _who(ctx, url) }}</span>{% endif %}
|
||||
{%- if ctx.author_handle %} <span class="reply-handle">{{ ctx.author_handle }}</span>{% endif -%}
|
||||
</span>
|
||||
</span>
|
||||
</header>
|
||||
{% if ctx.title %}<p class="p-name reply-title">{{ ctx.title }}</p>{% endif %}
|
||||
{% if ctx.content_snippet %}<div class="e-content reply-content">{{ ctx.content_snippet|safe }}</div>{% endif %}
|
||||
{% set image = ctx.photo|local_preview %}
|
||||
{% if image %}<img class="reply-photo" src="{{ image }}" alt="{{ ctx.photo_alt or '' }}" loading="lazy">{% endif %}
|
||||
<footer class="reply-source">
|
||||
<a class="u-url" href="{{ url }}">{{ _when(ctx, '%B %-d, %Y · %-I:%M %p') }}{% if ctx.published_at and ctx.published_at.strftime is defined %} · {% endif %}{{ ctx.site_name or url.split('/')[2] }} ↗</a>
|
||||
</footer>
|
||||
{% else %}
|
||||
<cite><a class="u-url" href="{{ url }}">{{ url }}</a></cite>
|
||||
{% endif %}
|
||||
</blockquote>
|
||||
</div>
|
||||
{% endmacro %}
|
||||
|
||||
{# One line and a short quote, in lists of notes #}
|
||||
{% macro reply_line(url, ctx) %}
|
||||
{% set fetched = ctx and ctx.fetch_status == 'fetched' %}
|
||||
<div class="reaction reply-to">
|
||||
<span class="reaction-verb">↩ In reply to</span>
|
||||
{% if fetched and (ctx.author_name or ctx.title) %}
|
||||
<a class="u-in-reply-to" href="{{ url }}">{{ ctx.title or ctx.author_name }}</a>
|
||||
{%- if ctx.title and ctx.author_name %} <span class="reaction-by">by {{ ctx.author_name }}</span>{% endif %}
|
||||
{% set quote = ctx.content_snippet|striptags|replace('\n', ' ')|trim if ctx.content_snippet else '' %}
|
||||
{% if quote and not ctx.title %}<q class="reply-quote">{{ quote|truncate(140) }}</q>{% endif %}
|
||||
{% else %}
|
||||
<a class="u-in-reply-to" href="{{ url }}">{{ url }}</a>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endmacro %}
|
||||
@@ -64,6 +64,7 @@
|
||||
'like': '<svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><path d="M12 21.35l-1.45-1.32C5.4 15.36 2 12.28 2 8.5 2 5.42 4.42 3 7.5 3c1.74 0 3.41.81 4.5 2.09C13.09 3.81 14.76 3 16.5 3 19.58 3 22 5.42 22 8.5c0 3.78-3.4 6.86-8.55 11.54L12 21.35z"/></svg>',
|
||||
'repost': '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M17 1l4 4-4 4"/><path d="M3 11V9a4 4 0 0 1 4-4h14"/><path d="M7 23l-4-4 4-4"/><path d="M21 13v2a4 4 0 0 1-4 4H3"/></svg>',
|
||||
'reply': '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M21 11.5a8.4 8.4 0 0 1-9 8.4 8.6 8.6 0 0 1-3.9-.9L3 21l1.9-5.1A8.4 8.4 0 0 1 12 3.1a8.4 8.4 0 0 1 9 8.4z"/></svg>',
|
||||
'quote': '<svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><path d="M7.2 5C4.3 6.4 3 8.9 3 12.3V19h7v-7H6.3c.1-2.2 1-3.7 2.9-4.6L7.2 5zm10 0c-2.9 1.4-4.2 3.9-4.2 7.3V19h7v-7h-3.7c.1-2.2 1-3.7 2.9-4.6L17.2 5z"/></svg>',
|
||||
'bookmark': '<svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><path d="M6 2h12a1 1 0 0 1 1 1v19l-7-4-7 4V3a1 1 0 0 1 1-1z"/></svg>',
|
||||
} %}
|
||||
{% set VERBS = {'like': 'Like', 'repost': 'Repost', 'reply': 'Reply'} %}
|
||||
@@ -95,19 +96,22 @@
|
||||
{% endif %}
|
||||
{%- endmacro %}
|
||||
|
||||
{% macro reactions(likes, reposts, replies, bookmarks, syndications, permalink) %}
|
||||
{% set anon_likes, anon_reposts = [], [] %}
|
||||
{% macro reactions(likes, reposts, replies, bookmarks, syndications, permalink, quotes=[]) %}
|
||||
{% set anon_likes, anon_reposts, anon_quotes = [], [], [] %}
|
||||
{% for s in syndications or [] %}
|
||||
{% if s.likes_count %}{% set _ = anon_likes.append((s.name, s.likes_count)) %}{% endif %}
|
||||
{% if s.reposts_count %}{% set _ = anon_reposts.append((s.name, s.reposts_count)) %}{% endif %}
|
||||
{% if s.quotes_count %}{% set _ = anon_quotes.append((s.name, s.quotes_count)) %}{% endif %}
|
||||
{% endfor %}
|
||||
{% set copies = (syndications or [])|selectattr('remote_url')|list %}
|
||||
{% if copies or likes or reposts or replies or bookmarks or anon_likes or anon_reposts %}
|
||||
{% if copies or likes or reposts or replies or quotes or bookmarks or anon_likes or anon_reposts or anon_quotes %}
|
||||
<div class="response-counts">
|
||||
{{ reaction('like', likes or [], 'u-like', anon_likes, 'like', 'likes', copies, permalink) }}
|
||||
{{ reaction('repost', reposts or [], 'u-repost', anon_reposts, 'repost', 'reposts', copies, permalink) }}
|
||||
{# Replies are listed in full below; here just the count and where to reply #}
|
||||
{{ reaction('reply', replies or [], '', [], 'reply', 'replies', copies, permalink, people=False) }}
|
||||
{# Quotes: named ones are listed below; the card shows those without names (v2.14.2) #}
|
||||
{{ reaction('quote', quotes or [], '', anon_quotes, 'quote', 'quotes', people=False) }}
|
||||
{{ reaction('bookmark', bookmarks or [], 'u-bookmark', [], 'bookmark', 'bookmarks') }}
|
||||
</div>
|
||||
{% endif %}
|
||||
@@ -118,7 +122,7 @@
|
||||
{% macro list_counts(counts, permalink) %}
|
||||
{% if counts %}
|
||||
<a class="list-counts" href="{{ permalink }}#responses">
|
||||
{%- for key, kind, one, many in [('likes', 'like', 'like', 'likes'), ('reposts', 'repost', 'repost', 'reposts'), ('replies', 'reply', 'reply', 'replies')] if counts[key] -%}
|
||||
{%- for key, kind, one, many in [('likes', 'like', 'like', 'likes'), ('reposts', 'repost', 'repost', 'reposts'), ('replies', 'reply', 'reply', 'replies'), ('quotes', 'quote', 'quote', 'quotes')] if counts.get(key) -%}
|
||||
<span class="list-count list-count-{{ kind }}" title="{{ counts[key] }} {{ one if counts[key] == 1 else many }}">{{ ICONS[kind]|safe }}{{ counts[key] }}<span class="visually-hidden"> {{ one if counts[key] == 1 else many }}</span></span>
|
||||
{%- endfor -%}
|
||||
</a>
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
"""Admin Log and Settings pages (v2.18.0)"""
|
||||
|
||||
import pytest
|
||||
|
||||
from starpunk import backfeed, create_app, error_log
|
||||
from starpunk.auth import create_session
|
||||
from starpunk.database import get_db
|
||||
from starpunk.notes import create_note
|
||||
|
||||
SITE = "https://blog.example.com"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(tmp_path):
|
||||
app = create_app({
|
||||
"TESTING": True, "SITE_URL": SITE, "ADMIN_ME": "https://example.com",
|
||||
"DATABASE_PATH": tmp_path / "starpunk.db", "DATA_PATH": tmp_path,
|
||||
"NOTES_PATH": tmp_path / "notes", "SESSION_SECRET": "test", "CSRF_ENABLED": False,
|
||||
"BACKGROUND_WORKERS_ENABLED": False, "WEBMENTION_ENABLED": True,
|
||||
})
|
||||
backfeed.denied.clear()
|
||||
yield app
|
||||
backfeed.denied.clear()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def browser(app):
|
||||
with app.test_request_context():
|
||||
token = create_session("https://example.com")
|
||||
client = app.test_client()
|
||||
client.set_cookie("starpunk_session", token)
|
||||
return client
|
||||
|
||||
|
||||
def _failures(app):
|
||||
"""One of each kind of error, on two notes"""
|
||||
with app.app_context():
|
||||
broken = create_note(content="A crosspost that broke", published=True)
|
||||
reply = create_note(content="Replying", published=True)
|
||||
db = get_db(app)
|
||||
db.execute("UPDATE notes SET in_reply_to = 'https://gone.example/post' WHERE id = ?", (reply.id,))
|
||||
db.execute("INSERT INTO syndications (note_id, target, state, last_error, updated_at) "
|
||||
"VALUES (?, 'bluesky', 'failed', 'HTTP 400: post too long', '2026-11-12 10:00:00')",
|
||||
(broken.id,))
|
||||
db.execute("INSERT INTO syndications (note_id, target, state, remote_url, backfeed_error, "
|
||||
"backfeed_checked_at) VALUES (?, 'gotosocial', 'synced', 'https://social.example/1', "
|
||||
"'HTTP 404 from GoToSocial', '2026-11-12 11:00:00')", (broken.id,))
|
||||
db.execute("INSERT INTO webmention_queue (source_url, target_url, source_note_id, status, last_error) "
|
||||
"VALUES (?, 'https://other.example/a', ?, 'failed', 'Endpoint returned 500')",
|
||||
(f"{SITE}/note/{reply.slug}", reply.id))
|
||||
db.execute("INSERT INTO reply_contexts (url, fetch_status, fetch_error) "
|
||||
"VALUES ('https://gone.example/post', 'error', 'HTTP 410 Gone')")
|
||||
db.execute("INSERT INTO syndications (note_id, target, state, last_error) "
|
||||
"VALUES (?, 'threads', 'synced', 'old error, since fixed')", (reply.id,))
|
||||
db.commit()
|
||||
return broken, reply
|
||||
|
||||
|
||||
def test_log_lists_each_error_with_its_post(app, browser):
|
||||
broken, reply = _failures(app)
|
||||
backfeed.denied["threads"] = "Needs a Threads token with threads_manage_replies"
|
||||
|
||||
page = browser.get("/admin/log").get_data(as_text=True)
|
||||
for text in ("Crosspost to Bluesky", "HTTP 400: post too long", "Retry",
|
||||
"Response check on GoToSocial", "HTTP 404 from GoToSocial", "Check now",
|
||||
"Webmention sent", "https://other.example/a", "Endpoint returned 500",
|
||||
"Reply context fetch", "HTTP 410 Gone",
|
||||
"Response checks on Threads", "Needs a Threads token", "Site-wide"):
|
||||
assert text in page, text
|
||||
assert f"/admin/edit/{broken.id}" in page and f"/admin/edit/{reply.id}" in page
|
||||
assert "old error, since fixed" not in page, "synced crossposts aren't errors"
|
||||
assert 'class="badge badge-error">5<' in page
|
||||
|
||||
with app.app_context():
|
||||
assert error_log.count() == 5
|
||||
areas = [e["area"] for e in error_log.entries()]
|
||||
assert areas[0] == "Response checks on Threads", "site-wide problems first"
|
||||
assert areas.index("Response check on GoToSocial") < areas.index("Crosspost to Bluesky"), "newest first"
|
||||
|
||||
|
||||
def test_log_retry_clears_the_crosspost_error(app, browser):
|
||||
broken, _ = _failures(app)
|
||||
with app.app_context():
|
||||
sid = get_db(app).execute("SELECT id FROM syndications WHERE target = 'bluesky'").fetchone()[0]
|
||||
browser.post(f"/admin/syndication/{sid}/retry")
|
||||
assert "HTTP 400: post too long" not in browser.get("/admin/log").get_data(as_text=True)
|
||||
|
||||
|
||||
def test_empty_log_and_no_badge(browser):
|
||||
page = browser.get("/admin/log").get_data(as_text=True)
|
||||
assert "No errors." in page and "badge-error" not in page
|
||||
|
||||
|
||||
def test_notification_error_is_logged(app, browser):
|
||||
with app.app_context():
|
||||
get_db(app).execute("INSERT INTO site_state (key, value) VALUES "
|
||||
"('notify_last_error', '2026-11-12 09:00:00 UTC gotify: 401 Unauthorized')")
|
||||
get_db(app).commit()
|
||||
page = browser.get("/admin/log").get_data(as_text=True)
|
||||
assert "Notifications" in page and "gotify: 401 Unauthorized" in page and "2026-11-12 09:00:00" in page
|
||||
|
||||
|
||||
def test_webmentions_page_is_decluttered(browser):
|
||||
page = browser.get("/admin/webmentions").get_data(as_text=True)
|
||||
assert "Back to Dashboard" not in page
|
||||
assert "Send test notification" not in page and "<h3>Notifications</h3>" not in page
|
||||
|
||||
|
||||
def test_nav_and_settings_page(browser):
|
||||
page = browser.get("/admin/settings").get_data(as_text=True)
|
||||
assert "<h3>Notifications</h3>" in page and "NOTIFY_URLS" in page
|
||||
for link in ("/admin/webmentions", "/admin/log", "/admin/settings", "/admin/tokens"):
|
||||
assert f'href="{link}"' in page, link
|
||||
|
||||
|
||||
def test_pages_require_login(app):
|
||||
client = app.test_client()
|
||||
for path in ("/admin/log", "/admin/settings"):
|
||||
assert client.get(path).status_code == 302
|
||||
@@ -0,0 +1,275 @@
|
||||
"""
|
||||
Articles: titles and summaries stored apart from the text (v2.16.0, ADR-076).
|
||||
|
||||
Feeds give only articles a title and never repeat it in the content; the
|
||||
homepage and tag pages show an article as title, summary and "Read more"
|
||||
without its images; the admin editor has Title and Summary fields.
|
||||
"""
|
||||
|
||||
import json
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
import httpx
|
||||
import mf2py
|
||||
|
||||
from starpunk.auth import create_session
|
||||
from starpunk.notes import create_note, get_note
|
||||
from tests.test_micropub_v240 import SITE, app, client, h, jpeg, source # noqa: F401
|
||||
from tests.test_micropub_v265 import upload
|
||||
|
||||
ATOM = {"a": "http://www.w3.org/2005/Atom"}
|
||||
|
||||
|
||||
def micropub(client, h, **props):
|
||||
response = client.post("/micropub", json={"type": ["h-entry"], "properties": props}, headers=h)
|
||||
assert response.status_code == 201, response.get_json()
|
||||
return response.headers["Location"]
|
||||
|
||||
|
||||
def make_both(app):
|
||||
with app.app_context():
|
||||
article = create_note(content="The **body**.", title="An Article", summary="The gist.",
|
||||
published=True, tags=["essay"])
|
||||
note = create_note(content="# Just a heading\n\nin a note", published=True, tags=["essay"])
|
||||
return article, note
|
||||
|
||||
|
||||
class TestFeeds:
|
||||
def test_rss_title_only_for_articles(self, app, client):
|
||||
article, note = make_both(app)
|
||||
items = {i.find("link").text: i for i in ET.fromstring(client.get("/feed.rss").data).iter("item")}
|
||||
a, n = items[f"{SITE}/article/{article.slug}"], items[f"{SITE}/note/{note.slug}"]
|
||||
assert a.find("title").text == "An Article"
|
||||
assert "An Article" not in a.find("description").text # not repeated in the body
|
||||
assert n.find("title") is None
|
||||
assert n.find("description") is not None
|
||||
|
||||
def test_atom_empty_title_and_summary(self, app, client):
|
||||
article, note = make_both(app)
|
||||
entries = {e.find("a:id", ATOM).text: e
|
||||
for e in ET.fromstring(client.get("/feed.atom").data).findall("a:entry", ATOM)}
|
||||
a, n = entries[f"{SITE}/article/{article.slug}"], entries[f"{SITE}/note/{note.slug}"]
|
||||
assert a.find("a:title", ATOM).text == "An Article"
|
||||
assert a.find("a:summary", ATOM).text == "The gist."
|
||||
assert "An Article" not in a.find("a:content", ATOM).text
|
||||
assert n.find("a:title", ATOM) is not None and not n.find("a:title", ATOM).text
|
||||
assert n.find("a:summary", ATOM) is None
|
||||
|
||||
def test_json_feed(self, app, client):
|
||||
article, note = make_both(app)
|
||||
items = {i["url"]: i for i in json.loads(client.get("/feed.json").data)["items"]}
|
||||
a, n = items[f"{SITE}/article/{article.slug}"], items[f"{SITE}/note/{note.slug}"]
|
||||
assert (a["title"], a["summary"]) == ("An Article", "The gist.")
|
||||
assert "An Article" not in a["content_html"]
|
||||
assert "title" not in n and "summary" not in n
|
||||
assert "<h1>Just a heading</h1>" in n["content_html"]
|
||||
|
||||
|
||||
class TestPages:
|
||||
def test_article_page(self, app, client):
|
||||
article, _ = make_both(app)
|
||||
(entry,) = [i for i in mf2py.parse(doc=client.get(f"/article/{article.slug}").data.decode(),
|
||||
url=f"{SITE}/article/{article.slug}")["items"]
|
||||
if "h-entry" in i["type"]]
|
||||
props = entry["properties"]
|
||||
assert props["name"] == ["An Article"]
|
||||
assert props["summary"] == ["The gist."]
|
||||
assert not props["content"][0]["value"].startswith("An Article")
|
||||
|
||||
def test_homepage_and_tag_page(self, app, client, h):
|
||||
photo = upload(client, h)
|
||||
url = micropub(client, h, name=["Trip"], content=["Day one of the trip."], photo=[photo],
|
||||
category=["essay"])
|
||||
micropub(client, h, content=["A photo note"], photo=[upload(client, h, "blue")], category=["essay"])
|
||||
with app.app_context():
|
||||
slug = url.rsplit("/", 1)[1]
|
||||
article = get_note(slug=slug)
|
||||
for path in ("/", "/tag/essay"):
|
||||
html = client.get(path).data.decode()
|
||||
start = html.index(f'href="{url}"')
|
||||
section = html[html.rindex("<article", 0, start):html.index("</article>", start)]
|
||||
assert "Trip" in section and "Day one of the trip." in section and "Read more" in section
|
||||
assert "<img" not in section and "e-content" not in section
|
||||
assert 'class="u-photo"' in section # parsers still see a photo post
|
||||
assert html.count("<img") >= 1 # the photo note still shows its image
|
||||
assert article.title == "Trip"
|
||||
|
||||
def test_preview_uses_summary_over_excerpt(self, app, client):
|
||||
make_both(app)
|
||||
html = client.get("/").data.decode()
|
||||
assert "The gist." in html and "The <strong>body</strong>." not in html
|
||||
|
||||
|
||||
class TestAdmin:
|
||||
def login(self, app, client):
|
||||
with app.test_request_context():
|
||||
client.set_cookie("starpunk_session", create_session(app.config["ADMIN_ME"]))
|
||||
|
||||
def test_create_and_edit_article(self, app, client):
|
||||
self.login(app, client)
|
||||
response = client.post("/admin/new", data={
|
||||
"title": "From the editor", "summary": "Short.", "content": "Body", "published": "on"})
|
||||
assert response.status_code == 302
|
||||
with app.app_context():
|
||||
from starpunk.notes import list_notes
|
||||
(note,) = list_notes()
|
||||
assert (note.title, note.summary, note.content) == ("From the editor", "Short.", "Body")
|
||||
|
||||
form = client.get(f"/admin/edit/{note.id}").data.decode()
|
||||
assert 'value="From the editor"' in form and ">Short.</textarea>" in form
|
||||
|
||||
client.post(f"/admin/edit/{note.id}", data={"title": "", "summary": "", "content": "Body",
|
||||
"published": "on"})
|
||||
with app.app_context():
|
||||
note = get_note(id=note.id)
|
||||
assert (note.title, note.summary) == (None, None)
|
||||
|
||||
def test_title_only_article(self, app, client):
|
||||
self.login(app, client)
|
||||
client.post("/admin/new", data={"title": "Only a title", "content": "", "published": "on"})
|
||||
with app.app_context():
|
||||
from starpunk.notes import list_notes
|
||||
(note,) = list_notes()
|
||||
assert note.title == "Only a title" and note.content == ""
|
||||
|
||||
|
||||
class TestInlineImages:
|
||||
"""Articles posted with images in their text (v2.16.0)"""
|
||||
|
||||
def remote_host(self, monkeypatch, ok=("/a.jpg",)):
|
||||
fetched = []
|
||||
|
||||
def serve(request): # the fetcher connects to the pinned address, so match on path
|
||||
fetched.append(request.url.path)
|
||||
if request.url.path in ok:
|
||||
return httpx.Response(200, content=jpeg("green"))
|
||||
return httpx.Response(404)
|
||||
|
||||
monkeypatch.setattr("starpunk.webmention_ssrf.TRANSPORT", httpx.MockTransport(serve))
|
||||
monkeypatch.setattr("starpunk.webmention_ssrf.socket.getaddrinfo",
|
||||
lambda *a, **k: [(2, 1, 6, "", ("93.184.216.34", 0))])
|
||||
return fetched
|
||||
|
||||
def test_html_is_kept_over_text_fallback(self, app, client, h, monkeypatch):
|
||||
self.remote_host(monkeypatch)
|
||||
html = '<p>Intro <a href="https://x.example">link</a></p><img src="https://cdn.example/a.jpg" alt="A">'
|
||||
url = micropub(client, h, name=["T"], content=[{"html": html, "text": "Intro link"}])
|
||||
(content,) = source(client, h, url)["properties"]["content"]
|
||||
assert "html" in content and 'href="https://x.example"' in content["html"]
|
||||
assert "<img" in content["html"]
|
||||
|
||||
def test_remote_images_are_copied(self, app, client, h, monkeypatch):
|
||||
fetched = self.remote_host(monkeypatch)
|
||||
url = micropub(client, h, name=["T"], content=[
|
||||
"Intro\n\n\n\n\n\n"
|
||||
"Again "])
|
||||
with app.app_context():
|
||||
note = get_note(slug=url.rsplit("/", 1)[1])
|
||||
assert fetched.count("/a.jpg") == 1 # each image once
|
||||
assert "https://cdn.example/a.jpg" not in note.content
|
||||
assert note.content.count(f"]({SITE}/media/") == 2
|
||||
assert "https://cdn.example/gone.jpg" in note.content # couldn't fetch: left as is
|
||||
assert client.get(note.content.split(f"]({SITE}")[1].split(")")[0]).status_code == 200
|
||||
|
||||
def test_html_images_copied_and_rendered_cleanly(self, app, client, h, monkeypatch):
|
||||
self.remote_host(monkeypatch)
|
||||
url = micropub(client, h, name=["T"], content=[{"html":
|
||||
'<p>Intro</p><img src="https://cdn.example/a.jpg" alt="A"><p>More</p><script>x()</script>'}])
|
||||
with app.app_context():
|
||||
note = get_note(slug=url.rsplit("/", 1)[1])
|
||||
assert note.content_format == "html"
|
||||
assert note.html.startswith("<p>Intro</p><img") and f'src="{SITE}/media/' in note.html
|
||||
assert "<br" not in note.html and "<p></p>" not in note.html and "script" not in note.html
|
||||
|
||||
def test_own_images_not_fetched(self, app, client, h, monkeypatch):
|
||||
fetched = self.remote_host(monkeypatch)
|
||||
micropub(client, h, content=[f""])
|
||||
assert fetched == []
|
||||
|
||||
def test_update_copies_new_images(self, app, client, h, monkeypatch):
|
||||
self.remote_host(monkeypatch)
|
||||
url = micropub(client, h, name=["T"], content=["Before"])
|
||||
client.post("/micropub", json={"action": "update", "url": url,
|
||||
"replace": {"content": [""]}}, headers=h)
|
||||
with app.app_context():
|
||||
assert f"]({SITE}/media/" in get_note(slug=url.rsplit("/", 1)[1]).content
|
||||
|
||||
|
||||
class TestArticleUrls:
|
||||
"""Articles live at /article/<title-slug>, notes at /note/<slug> (v2.17.0)"""
|
||||
|
||||
def test_micropub_article_gets_article_url_from_title(self, app, client, h):
|
||||
url = micropub(client, h, name=["Why I Left Twitter!"], content=["Long story."])
|
||||
assert url == f"{SITE}/article/why-i-left-twitter"
|
||||
assert source(client, h, url)["properties"]["url"] == [url]
|
||||
assert client.get("/article/why-i-left-twitter").status_code == 200
|
||||
|
||||
def test_note_keeps_note_url_and_timestamp_slug(self, app, client, h):
|
||||
url = micropub(client, h, content=["Just a note"])
|
||||
assert url.startswith(f"{SITE}/note/") and url.rsplit("/", 1)[1][:8].isdigit()
|
||||
|
||||
def test_wrong_prefix_redirects(self, app, client, h):
|
||||
article = micropub(client, h, name=["Moved"], content=["Body"])
|
||||
note = micropub(client, h, content=["Plain"])
|
||||
slug, note_slug = article.rsplit("/", 1)[1], note.rsplit("/", 1)[1]
|
||||
response = client.get(f"/note/{slug}")
|
||||
assert (response.status_code, response.location) == (301, f"/article/{slug}")
|
||||
response = client.get(f"/article/{note_slug}")
|
||||
assert (response.status_code, response.location) == (301, f"/note/{note_slug}")
|
||||
assert client.get("/article/no-such-post").status_code == 404
|
||||
|
||||
def test_micropub_accepts_either_url(self, app, client, h):
|
||||
url = micropub(client, h, name=["Either"], content=["Body"])
|
||||
old_style = url.replace("/article/", "/note/")
|
||||
response = client.post("/micropub", json={"action": "update", "url": old_style,
|
||||
"replace": {"summary": ["S"]}}, headers=h)
|
||||
assert response.status_code == 204
|
||||
assert source(client, h, url)["properties"]["summary"] == ["S"]
|
||||
|
||||
def test_title_slugs(self, app):
|
||||
with app.test_request_context():
|
||||
def slug(title, **kw):
|
||||
return create_note(content="Body", title=title, published=True, **kw).slug
|
||||
|
||||
assert slug("Café Culture") == "cafe-culture"
|
||||
assert slug("Café Culture") == "cafe-culture-2"
|
||||
assert slug("Search") == "search-article" # reserved
|
||||
assert slug("Mine", custom_slug="chosen") == "chosen" # mp-slug wins
|
||||
assert slug("🎉🎉").isdigit() # nothing usable: timestamp
|
||||
long = slug("word " * 40)
|
||||
assert len(long) <= 80 and long.endswith("word") and "--" not in long
|
||||
|
||||
def test_update_that_adds_or_removes_title_returns_new_url(self, app, client, h):
|
||||
note = micropub(client, h, content=["Starts as a note"])
|
||||
slug = note.rsplit("/", 1)[1]
|
||||
response = client.post("/micropub", json={"action": "update", "url": note,
|
||||
"replace": {"name": ["Now an Article"]}}, headers=h)
|
||||
assert (response.status_code, response.headers["Location"]) == (201, f"{SITE}/article/{slug}")
|
||||
response = client.post("/micropub", json={"action": "update", "url": f"{SITE}/article/{slug}",
|
||||
"replace": {"content": ["Edited"]}}, headers=h)
|
||||
assert response.status_code == 204 # URL unchanged
|
||||
response = client.post("/micropub", json={"action": "update", "url": f"{SITE}/article/{slug}",
|
||||
"delete": ["name"]}, headers=h)
|
||||
assert (response.status_code, response.headers["Location"]) == (201, f"{SITE}/note/{slug}")
|
||||
|
||||
def test_deleted_article_is_gone(self, app, client, h):
|
||||
url = micropub(client, h, name=["Short-lived"], content=["Body"])
|
||||
assert client.post("/micropub", data={"action": "delete", "url": url}, headers=h).status_code == 204
|
||||
assert client.get("/article/short-lived").status_code == 410
|
||||
|
||||
def test_webmention_target_and_links(self, app, client):
|
||||
article, note = make_both(app)
|
||||
from starpunk.routes.webmention import resolve_target_note
|
||||
with app.test_request_context():
|
||||
assert resolve_target_note(f"{SITE}/article/{article.slug}") == article.id
|
||||
assert resolve_target_note(f"{SITE}/note/{article.slug}") == article.id
|
||||
assert resolve_target_note(f"{SITE}/article/{article.slug}/x") is None
|
||||
html = client.get("/").data.decode()
|
||||
assert f'href="{SITE}/article/{article.slug}"' in html
|
||||
assert f'href="{SITE}/note/{note.slug}"' in html
|
||||
assert f"/note/{article.slug}" not in html
|
||||
|
||||
def test_search_links_to_article(self, app, client):
|
||||
article, _ = make_both(app)
|
||||
results = client.get("/api/search?q=body").get_json()["results"]
|
||||
assert [r["url"] for r in results if r["slug"] == article.slug] == [f"/article/{article.slug}"]
|
||||
+20
-1
@@ -543,7 +543,8 @@ def test_admin_backfill_and_check_now(app, client):
|
||||
assert s["backfeed_next_at"]
|
||||
|
||||
backfeed.denied["gotosocial"] = "Needs a GoToSocial token with read:statuses"
|
||||
assert "Needs a GoToSocial token" in browser.get("/admin/webmentions?origin=gotosocial").get_data(as_text=True)
|
||||
assert "Needs a GoToSocial token" in browser.get("/admin/log").get_data(as_text=True)
|
||||
assert "Response checks have errors" in browser.get("/admin/webmentions").get_data(as_text=True)
|
||||
|
||||
|
||||
def test_threads_avatar_kept_while_its_copy_exists(app, client, web):
|
||||
@@ -639,3 +640,21 @@ def test_home_page_shows_counts(app, client):
|
||||
assert 'title="3 likes"' in page and 'title="2 reposts"' in page and 'title="1 reply"' in page
|
||||
assert f"/note/{quiet.slug}#responses" not in page
|
||||
assert 'id="responses"' in app.test_client().get(f"/note/{note.slug}").get_data(as_text=True)
|
||||
|
||||
|
||||
def test_quote_counts_include_anonymous_ones(app, client):
|
||||
"""v2.14.2: quotes have a count, including those without names (Threads, opted out)"""
|
||||
note, sid = crossposted(app)
|
||||
client.harvest = Harvest([Response(kind="quote", source_url="https://bsky.app/profile/q/post/1",
|
||||
remote_id="at://q/1", author_name="Quoter",
|
||||
content_html="<p>Look</p>")])
|
||||
run(app)
|
||||
with app.app_context():
|
||||
get_db(app).execute("UPDATE syndications SET quotes_count = 2 WHERE id = ?", (sid,))
|
||||
get_db(app).commit()
|
||||
page = app.test_client().get(f"/note/{note.slug}").get_data(as_text=True)
|
||||
quote = page.split('response-count-quote')[1].split('</span>\n</span>')[0]
|
||||
assert '<span class="response-count-n">3</span>' in quote and "2 on GoToSocial" in quote
|
||||
assert "Quoter" in page, "the named quote is listed under Responses"
|
||||
home = app.test_client().get("/").get_data(as_text=True)
|
||||
assert 'title="3 quotes"' in home
|
||||
+15
-29
@@ -55,7 +55,8 @@ def sample_notes(app):
|
||||
notes = []
|
||||
for i in range(5):
|
||||
note = create_note(
|
||||
content=f"# Test Note {i}\n\nThis is test content for note {i}.",
|
||||
content=f"This is test content for note {i}.",
|
||||
title=f"Test Note {i}",
|
||||
published=True,
|
||||
)
|
||||
notes.append(note)
|
||||
@@ -141,7 +142,8 @@ class TestGenerateFeed:
|
||||
import time
|
||||
for i in range(3):
|
||||
create_note(
|
||||
content=f"# Note {i}\n\nContent {i}.",
|
||||
content=f"Content {i}.",
|
||||
title=f"Note {i}",
|
||||
published=True,
|
||||
)
|
||||
time.sleep(0.01) # Ensure distinct timestamps
|
||||
@@ -342,41 +344,25 @@ class TestFormatRFC822Date:
|
||||
class TestGetNoteTitle:
|
||||
"""Test get_note_title() function"""
|
||||
|
||||
def test_get_note_title_with_heading(self, app):
|
||||
"""Test title extraction from note with heading"""
|
||||
def test_get_note_title_article(self, app):
|
||||
"""An article's item title is its title"""
|
||||
with app.app_context():
|
||||
note = create_note(content="# My First Note\n\nContent here.", published=True)
|
||||
title = get_note_title(note)
|
||||
assert title == "My First Note"
|
||||
note = create_note(content="Content here.", title="My First Note", published=True)
|
||||
assert get_note_title(note) == "My First Note"
|
||||
|
||||
def test_get_note_title_without_heading(self, app):
|
||||
"""Test title extraction from note without heading"""
|
||||
def test_get_note_title_untitled_note(self, app):
|
||||
"""Untitled notes have no item title (v2.16.0, ADR-076)"""
|
||||
with app.app_context():
|
||||
note = create_note(content="Just some content without heading.", published=True)
|
||||
title = get_note_title(note)
|
||||
|
||||
# Should use first line (Note.title handles this)
|
||||
assert len(title) > 0
|
||||
assert "Just some content" in title
|
||||
assert get_note_title(note) is None
|
||||
note = create_note(content="# A heading in the body\n\nis not a title", published=True)
|
||||
assert get_note_title(note) is None
|
||||
|
||||
def test_get_note_title_truncates_long_titles(self, app):
|
||||
"""Test title truncation for long titles"""
|
||||
with app.app_context():
|
||||
long_title = "A" * 150
|
||||
note = create_note(content=f"# {long_title}\n\nContent.", published=True)
|
||||
title = get_note_title(note)
|
||||
|
||||
# Should truncate to reasonable length
|
||||
assert len(title) <= 103 # 100 chars + "..."
|
||||
|
||||
def test_get_note_title_minimal_content(self, app):
|
||||
"""Test title extraction with minimal content"""
|
||||
with app.app_context():
|
||||
note = create_note(content="x", published=True)
|
||||
title = get_note_title(note)
|
||||
|
||||
# Should extract something (single character or slug)
|
||||
assert len(title) > 0
|
||||
note = create_note(content="Content.", title="A" * 150, published=True)
|
||||
assert len(get_note_title(note)) <= 103 # 100 chars + "..."
|
||||
|
||||
|
||||
class TestCleanHTMLForRSS:
|
||||
|
||||
@@ -49,7 +49,7 @@ def sample_notes(app):
|
||||
notes = []
|
||||
for i in range(5):
|
||||
note = create_note(
|
||||
content=f"# Test Note {i}\n\nThis is test content for note {i}.",
|
||||
title=f"Test Note {i}", content=f"This is test content for note {i}.",
|
||||
published=True,
|
||||
)
|
||||
notes.append(note)
|
||||
@@ -138,7 +138,7 @@ class TestGenerateAtom:
|
||||
# Create notes with distinct timestamps
|
||||
for i in range(3):
|
||||
create_note(
|
||||
content=f"# Note {i}\n\nContent {i}.",
|
||||
title=f"Note {i}", content=f"Content {i}.",
|
||||
published=True,
|
||||
)
|
||||
time.sleep(0.01)
|
||||
@@ -216,7 +216,7 @@ class TestGenerateAtom:
|
||||
"""Test ATOM feed includes HTML content properly escaped"""
|
||||
with app.app_context():
|
||||
note = create_note(
|
||||
content="# Test\n\nThis is **bold** and *italic*.",
|
||||
title="Test", content="This is **bold** and *italic*.",
|
||||
published=True,
|
||||
)
|
||||
|
||||
@@ -243,7 +243,7 @@ class TestGenerateAtom:
|
||||
"""Test ATOM feed escapes special XML characters"""
|
||||
with app.app_context():
|
||||
note = create_note(
|
||||
content="# Test & Special <Characters>\n\nContent with 'quotes' and \"doubles\".",
|
||||
title="Test & Special <Characters>", content="Content with 'quotes' and \"doubles\".",
|
||||
published=True,
|
||||
)
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ def sample_notes(app):
|
||||
notes = []
|
||||
for i in range(5):
|
||||
note = create_note(
|
||||
content=f"# Test Note {i}\n\nThis is test content for note {i}.",
|
||||
title=f"Test Note {i}", content=f"This is test content for note {i}.",
|
||||
published=True,
|
||||
)
|
||||
notes.append(note)
|
||||
@@ -121,7 +121,7 @@ class TestGenerateJsonFeed:
|
||||
# Create notes with distinct timestamps
|
||||
for i in range(3):
|
||||
create_note(
|
||||
content=f"# Note {i}\n\nContent {i}.",
|
||||
title=f"Note {i}", content=f"Content {i}.",
|
||||
published=True,
|
||||
)
|
||||
time.sleep(0.01)
|
||||
@@ -195,7 +195,7 @@ class TestGenerateJsonFeed:
|
||||
"""Test JSON Feed includes HTML content"""
|
||||
with app.app_context():
|
||||
note = create_note(
|
||||
content="# Test\n\nThis is **bold** and *italic*.",
|
||||
title="Test", content="This is **bold** and *italic*.",
|
||||
published=True,
|
||||
)
|
||||
|
||||
|
||||
@@ -81,7 +81,7 @@ def note_with_single_media(app):
|
||||
with app.app_context():
|
||||
# Create note
|
||||
note = create_note(
|
||||
content="# Test Note\n\nNote with one image.",
|
||||
title="Test Note", content="Note with one image.",
|
||||
published=True
|
||||
)
|
||||
|
||||
@@ -106,7 +106,7 @@ def note_with_multiple_media(app):
|
||||
with app.app_context():
|
||||
# Create note
|
||||
note = create_note(
|
||||
content="# Gallery Note\n\nNote with three images.",
|
||||
title="Gallery Note", content="Note with three images.",
|
||||
published=True
|
||||
)
|
||||
|
||||
@@ -135,7 +135,7 @@ def note_without_media(app):
|
||||
"""Create note without media"""
|
||||
with app.app_context():
|
||||
note = create_note(
|
||||
content="# Plain Note\n\nNote without images.",
|
||||
title="Plain Note", content="Note without images.",
|
||||
published=True
|
||||
)
|
||||
notes = list_notes(published_only=True, limit=1)
|
||||
|
||||
+79
-1
@@ -184,7 +184,9 @@ class TestCodesAndTokens:
|
||||
with app.test_request_context():
|
||||
code = create_authorization_code(CLIENT_ID, REDIRECT_URI, "create", challenge_for(verifier))
|
||||
result = redeem_authorization_code(code, CLIENT_ID, REDIRECT_URI, verifier)
|
||||
assert result == {"me": "https://example.com", "scope": "create", "client_id": CLIENT_ID}
|
||||
assert result == {
|
||||
"me": "https://example.com", "scope": "create", "client_id": CLIENT_ID, "resource": None,
|
||||
}
|
||||
|
||||
token = create_access_token(CLIENT_ID, "create")
|
||||
info = verify_access_token(token["access_token"])
|
||||
@@ -354,3 +356,79 @@ class TestEndpoints:
|
||||
admin_client.post(f"/admin/tokens/{token_id}/revoke")
|
||||
with app.test_request_context():
|
||||
assert verify_access_token(token) is None
|
||||
|
||||
|
||||
MCP_RESOURCE = "https://blog.example.com/mcp"
|
||||
|
||||
|
||||
class TestResourceIndicators:
|
||||
"""RFC 8707 resource binding for MCP clients (v2.15.0, ADR-075)"""
|
||||
|
||||
def test_metadata_advertises_cimd_and_draft(self, client):
|
||||
data = client.get("/.well-known/oauth-authorization-server").get_json()
|
||||
assert data["client_id_metadata_document_supported"] is True
|
||||
assert data["token_endpoint_auth_methods_supported"] == ["none"]
|
||||
assert "draft" in data["scopes_supported"]
|
||||
|
||||
def test_mcp_resource_accepted(self, app):
|
||||
with app.test_request_context():
|
||||
result = validate_authorization_request(auth_params(resource=MCP_RESOURCE + "/"))
|
||||
assert result["resource"] == MCP_RESOURCE
|
||||
|
||||
def test_no_resource_is_micropub(self, app):
|
||||
with app.test_request_context():
|
||||
assert validate_authorization_request(auth_params())["resource"] is None
|
||||
|
||||
def test_unknown_resource_rejected(self, app):
|
||||
with app.test_request_context():
|
||||
with pytest.raises(AuthorizationRequestError) as e:
|
||||
validate_authorization_request(auth_params(resource="https://other.example/mcp"))
|
||||
assert e.value.error == "invalid_target"
|
||||
assert e.value.redirectable
|
||||
|
||||
def test_mcp_client_without_scope_is_offered_defaults(self, app):
|
||||
with app.test_request_context():
|
||||
result = validate_authorization_request(auth_params(scope="", resource=MCP_RESOURCE))
|
||||
assert result["scopes"] == ["create", "update", "delete", "media", "draft"]
|
||||
|
||||
def test_consent_page_names_mcp(self, admin_client):
|
||||
response = admin_client.get("/auth/authorize?" + urlencode(auth_params(resource=MCP_RESOURCE)))
|
||||
assert response.status_code == 200
|
||||
assert b"MCP" in response.data
|
||||
assert b'name="resource" value="https://blog.example.com/mcp"' in response.data
|
||||
|
||||
def _token_form(self, query, **extra):
|
||||
return {
|
||||
"grant_type": "authorization_code",
|
||||
"code": query["code"][0],
|
||||
"client_id": CLIENT_ID,
|
||||
"redirect_uri": REDIRECT_URI,
|
||||
"code_verifier": "v" * 50,
|
||||
**extra,
|
||||
}
|
||||
|
||||
def test_resource_carried_to_token(self, app, admin_client, client):
|
||||
query = approve(admin_client, granted=["create"], resource=MCP_RESOURCE)
|
||||
body = client.post("/auth/token", data=self._token_form(query, resource=MCP_RESOURCE)).get_json()
|
||||
with app.test_request_context():
|
||||
info = verify_access_token(body["access_token"])
|
||||
assert info["resource"] == MCP_RESOURCE
|
||||
assert info["scope"] == "create"
|
||||
|
||||
def test_token_request_resource_must_match(self, admin_client, client):
|
||||
query = approve(admin_client, granted=["create"], resource=MCP_RESOURCE)
|
||||
response = client.post("/auth/token", data=self._token_form(query, resource="https://other.example/"))
|
||||
assert response.status_code == 400
|
||||
assert response.get_json()["error"] == "invalid_target"
|
||||
|
||||
def test_micropub_token_has_no_resource(self, app, admin_client, client):
|
||||
query = approve(admin_client, granted=["create"])
|
||||
body = client.post("/auth/token", data=self._token_form(query)).get_json()
|
||||
with app.test_request_context():
|
||||
assert verify_access_token(body["access_token"])["resource"] is None
|
||||
|
||||
def test_legacy_verification_shape_unchanged(self, app, client):
|
||||
with app.test_request_context():
|
||||
token = create_access_token(CLIENT_ID, "create", resource=MCP_RESOURCE)["access_token"]
|
||||
data = client.get("/auth/token", headers={"Authorization": f"Bearer {token}"}).get_json()
|
||||
assert data == {"me": "https://example.com", "client_id": CLIENT_ID, "scope": "create"}
|
||||
@@ -0,0 +1,9 @@
|
||||
|
||||
|
||||
def test_tables_get_a_scroll_wrapper():
|
||||
"""v2.18.2: tables are wrapped so they can fill the column and scroll sideways"""
|
||||
from starpunk.models import render_markdown
|
||||
|
||||
html = render_markdown("| Color | Why |\n|---|---|\n| Black | Removal |\n\nAfter")
|
||||
assert html.startswith('<div class="table-wrap"><table>') and "</table></div>" in html
|
||||
assert "<p>After</p>" in html
|
||||
@@ -0,0 +1,259 @@
|
||||
"""
|
||||
Tests for the MCP server (starpunk/mcp_server, v2.15.0, ADR-075)
|
||||
|
||||
Requests go through the real ASGI app (Streamable HTTP, stateless JSON) with
|
||||
real StarPunk tokens, so the SDK's bearer checks, resource binding and DNS
|
||||
rebinding protection are exercised along with the tools.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import io
|
||||
import json
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
from starpunk.indieauth import create_access_token, revoke_access_token
|
||||
from starpunk.mcp_server import create_asgi_app
|
||||
|
||||
SITE = "https://blog.example.com"
|
||||
RESOURCE = SITE + "/mcp"
|
||||
ALL_SCOPES = "create update delete media draft"
|
||||
HEADERS = {
|
||||
"Accept": "application/json, text/event-stream",
|
||||
"Content-Type": "application/json",
|
||||
"MCP-Protocol-Version": "2025-11-25",
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app(tmp_path):
|
||||
from starpunk import create_app
|
||||
|
||||
data = tmp_path / "data"
|
||||
return create_app(
|
||||
{
|
||||
"TESTING": True,
|
||||
"SITE_URL": SITE,
|
||||
"ADMIN_ME": "https://example.com",
|
||||
"SESSION_SECRET": "test-secret",
|
||||
"DATA_PATH": data,
|
||||
"NOTES_PATH": data / "notes",
|
||||
"DATABASE_PATH": data / "starpunk.db",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def http(app):
|
||||
with TestClient(create_asgi_app(app), base_url=SITE) as client:
|
||||
yield client
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def token(app):
|
||||
"""Make a token bound to the MCP resource (or another) with given scopes"""
|
||||
|
||||
def make(scope=ALL_SCOPES, resource=RESOURCE):
|
||||
with app.app_context():
|
||||
return create_access_token("https://claude.ai/client.json", scope, resource=resource)["access_token"]
|
||||
|
||||
return make
|
||||
|
||||
|
||||
def rpc(http, token, method, params=None, headers=None):
|
||||
body = {"jsonrpc": "2.0", "id": 1, "method": method}
|
||||
if params is not None:
|
||||
body["params"] = params
|
||||
auth = {"Authorization": f"Bearer {token}"} if token else {}
|
||||
return http.post("/mcp", json=body, headers={**HEADERS, **auth, **(headers or {})})
|
||||
|
||||
|
||||
def call(http, token, name, **arguments):
|
||||
"""Call a tool; returns (is_error, parsed JSON result or error text)"""
|
||||
response = rpc(http, token, "tools/call", {"name": name, "arguments": arguments})
|
||||
assert response.status_code == 200, response.text
|
||||
result = response.json()["result"]
|
||||
text = result["content"][0]["text"]
|
||||
if result.get("isError"):
|
||||
return True, text
|
||||
return False, json.loads(text)
|
||||
|
||||
|
||||
def ok(http, token, name, **arguments):
|
||||
is_error, result = call(http, token, name, **arguments)
|
||||
assert not is_error, result
|
||||
return result
|
||||
|
||||
|
||||
class TestAuth:
|
||||
def test_no_token_points_to_resource_metadata(self, http):
|
||||
response = rpc(http, None, "tools/list")
|
||||
assert response.status_code == 401
|
||||
assert f'resource_metadata="{SITE}/.well-known/oauth-protected-resource/mcp"' in (
|
||||
response.headers["www-authenticate"]
|
||||
)
|
||||
|
||||
def test_protected_resource_metadata(self, http):
|
||||
data = http.get("/.well-known/oauth-protected-resource/mcp").json()
|
||||
assert data["resource"] == RESOURCE
|
||||
# Must equal the issuer in /.well-known/oauth-authorization-server
|
||||
assert data["authorization_servers"] == [SITE + "/"]
|
||||
|
||||
def test_micropub_token_refused(self, http, token):
|
||||
assert rpc(http, token(resource=None), "tools/list").status_code == 401
|
||||
|
||||
def test_other_resource_refused(self, http, token):
|
||||
assert rpc(http, token(resource="https://other.example/mcp"), "tools/list").status_code == 401
|
||||
|
||||
def test_revoked_token_refused(self, app, http, token):
|
||||
value = token()
|
||||
with app.app_context():
|
||||
revoke_access_token(value)
|
||||
assert rpc(http, value, "tools/list").status_code == 401
|
||||
|
||||
def test_unknown_host_refused(self, http, token):
|
||||
response = rpc(http, token(), "tools/list", headers={"Host": "evil.example"})
|
||||
assert response.status_code in (400, 403, 421)
|
||||
|
||||
def test_lists_tools(self, http, token):
|
||||
response = rpc(http, token(), "tools/list")
|
||||
names = {t["name"] for t in response.json()["result"]["tools"]}
|
||||
assert names == {
|
||||
"search_notes", "list_notes", "get_note", "list_tags", "get_site_config",
|
||||
"create_note", "update_note", "delete_note", "undelete_note", "upload_media",
|
||||
}
|
||||
|
||||
def test_read_tools_are_marked_read_only(self, http, token):
|
||||
tools = {t["name"]: t for t in rpc(http, token(), "tools/list").json()["result"]["tools"]}
|
||||
assert tools["list_notes"]["annotations"]["readOnlyHint"] is True
|
||||
assert tools["delete_note"]["annotations"]["destructiveHint"] is True
|
||||
|
||||
|
||||
class TestWriting:
|
||||
def test_create_defaults_to_draft(self, http, token):
|
||||
note = ok(http, token(), "create_note", content="Hello from MCP", tags=["ai"])
|
||||
assert note["status"] == "draft"
|
||||
assert note["url"].startswith(SITE + "/note/")
|
||||
|
||||
def test_create_published_article(self, http, token):
|
||||
tok = token()
|
||||
note = ok(http, tok, "create_note", content="Body", title="An article", draft=False)
|
||||
assert note["status"] == "published"
|
||||
props = ok(http, tok, "get_note", url_or_slug=note["url"])["properties"]
|
||||
assert props["name"] == ["An article"]
|
||||
assert props["post-status"] == ["published"]
|
||||
|
||||
def test_draft_only_token_cannot_publish(self, http, token):
|
||||
note = ok(http, token("draft"), "create_note", content="Sneaky", draft=False)
|
||||
assert note["status"] == "draft"
|
||||
|
||||
def test_create_needs_scope(self, http, token):
|
||||
is_error, message = call(http, token("update"), "create_note", content="x")
|
||||
assert is_error and "create or draft" in message
|
||||
|
||||
def test_update_tags_and_publish(self, http, token):
|
||||
tok = token()
|
||||
note = ok(http, tok, "create_note", content="Tagged", tags=["one", "two"])
|
||||
updated = ok(http, tok, "update_note", url=note["slug"], add_tags=["three"],
|
||||
remove_tags=["one"], draft=False)
|
||||
assert updated["status"] == "published"
|
||||
props = ok(http, tok, "get_note", url_or_slug=note["slug"])["properties"]
|
||||
assert props["category"] == ["two", "three"]
|
||||
|
||||
def test_update_content_and_remove_title(self, http, token):
|
||||
tok = token()
|
||||
note = ok(http, tok, "create_note", content="Old", title="Title")
|
||||
ok(http, tok, "update_note", url=note["url"], content="New", title="")
|
||||
props = ok(http, tok, "get_note", url_or_slug=note["url"])["properties"]
|
||||
assert props["content"] == ["New"]
|
||||
assert "name" not in props
|
||||
|
||||
def test_update_needs_scope(self, http, token):
|
||||
note = ok(http, token(), "create_note", content="x")
|
||||
is_error, _ = call(http, token("create"), "update_note", url=note["url"], content="y")
|
||||
assert is_error
|
||||
|
||||
def test_delete_and_undelete(self, http, token):
|
||||
tok = token()
|
||||
note = ok(http, tok, "create_note", content="Gone soon", draft=False)
|
||||
assert ok(http, tok, "delete_note", url=note["url"])["status"] == "deleted"
|
||||
assert call(http, tok, "get_note", url_or_slug=note["slug"])[0]
|
||||
assert ok(http, tok, "undelete_note", url=note["slug"])["status"] == "published"
|
||||
|
||||
def test_note_on_other_site_refused(self, http, token):
|
||||
is_error, message = call(http, token(), "delete_note", url="https://other.example/note/x")
|
||||
assert is_error and "not on this site" in message
|
||||
|
||||
def test_upload_media_then_attach(self, http, token):
|
||||
tok = token()
|
||||
buffer = io.BytesIO()
|
||||
Image.new("RGB", (40, 30), "red").save(buffer, format="PNG")
|
||||
media = ok(http, tok, "upload_media", filename="red.png",
|
||||
data_base64=base64.b64encode(buffer.getvalue()).decode())
|
||||
assert media["url"].startswith(SITE + "/media/")
|
||||
note = ok(http, tok, "create_note", photos=[{"url": media["url"], "alt": "A red square"}])
|
||||
props = ok(http, tok, "get_note", url_or_slug=note["slug"])["properties"]
|
||||
assert props["photo"][0]["alt"] == "A red square"
|
||||
|
||||
def test_upload_rejects_bad_base64(self, http, token):
|
||||
is_error, message = call(http, token(), "upload_media", filename="x.png", data_base64="!!!")
|
||||
assert is_error and "base64" in message
|
||||
|
||||
|
||||
class TestReading:
|
||||
def test_list_hides_drafts_by_default(self, http, token):
|
||||
tok = token()
|
||||
ok(http, tok, "create_note", content="Draft")
|
||||
ok(http, tok, "create_note", content="Public", draft=False)
|
||||
assert [n["excerpt"] for n in ok(http, tok, "list_notes")["notes"]] == ["Public"]
|
||||
assert len(ok(http, tok, "list_notes", include_drafts=True)["notes"]) == 2
|
||||
|
||||
def test_drafts_need_a_posting_scope(self, http, token):
|
||||
ok(http, token(), "create_note", content="Secret draft")
|
||||
reader = token("media")
|
||||
assert call(http, reader, "list_notes", include_drafts=True)[0]
|
||||
assert ok(http, reader, "list_notes")["notes"] == []
|
||||
|
||||
def test_draft_hidden_from_get_without_scope(self, http, token):
|
||||
note = ok(http, token(), "create_note", content="Secret draft")
|
||||
is_error, _ = call(http, token("media"), "get_note", url_or_slug=note["slug"])
|
||||
assert is_error
|
||||
|
||||
def test_search(self, http, token):
|
||||
tok = token()
|
||||
ok(http, tok, "create_note", content="Pancakes on Sunday", draft=False)
|
||||
ok(http, tok, "create_note", content="Waffles", draft=False)
|
||||
results = ok(http, tok, "search_notes", query="pancakes")["results"]
|
||||
assert len(results) == 1 and results[0]["url"].startswith(SITE + "/note/")
|
||||
|
||||
def test_tags(self, http, token):
|
||||
tok = token()
|
||||
ok(http, tok, "create_note", content="x", tags=["Cats", "dogs"])
|
||||
assert set(ok(http, tok, "list_tags")["tags"]) == {"Cats", "dogs"}
|
||||
assert ok(http, tok, "list_tags", filter="cat")["tags"] == ["Cats"]
|
||||
|
||||
def test_site_config(self, http, token):
|
||||
config = ok(http, token("media"), "get_site_config")
|
||||
assert config["media-endpoint"] == SITE + "/micropub/media"
|
||||
assert "syndicate-to" in config
|
||||
|
||||
|
||||
class TestSidecarApp:
|
||||
def test_workers_off_when_built_from_environment(self, monkeypatch, tmp_path):
|
||||
"""create_asgi_app() without an app never starts background workers"""
|
||||
import starpunk
|
||||
|
||||
seen = {}
|
||||
real = starpunk.create_app
|
||||
|
||||
def spy(config=None):
|
||||
seen.update(config or {})
|
||||
return real({**(config or {}), "TESTING": True, "SITE_URL": SITE,
|
||||
"SESSION_SECRET": "x", "ADMIN_ME": "https://example.com", "DATA_PATH": tmp_path,
|
||||
"NOTES_PATH": tmp_path / "notes", "DATABASE_PATH": tmp_path / "db.sqlite"})
|
||||
|
||||
monkeypatch.setattr(starpunk, "create_app", spy)
|
||||
create_asgi_app()
|
||||
assert seen["BACKGROUND_WORKERS_ENABLED"] is False
|
||||
@@ -57,20 +57,21 @@ class TestNoteHEntry:
|
||||
assert props['url'][0] == props['uid'][0], "u-url and u-uid should match"
|
||||
|
||||
def test_hentry_pname_only_with_explicit_title(self, client, app, data_dir):
|
||||
"""p-name only present when note has explicit title (per Q22)"""
|
||||
"""p-name only present when note has a title (ADR-076, was Q22)"""
|
||||
from starpunk.notes import create_note
|
||||
|
||||
# Create note WITH heading (explicit title)
|
||||
# Create note WITH a title (an article)
|
||||
with app.app_context():
|
||||
note_with_title = create_note(
|
||||
content="# Explicit Title\n\nThis note has a heading.",
|
||||
content="This note has a title.",
|
||||
title="Explicit Title",
|
||||
custom_slug="note-with-title",
|
||||
published=True
|
||||
)
|
||||
|
||||
response = client.get('/note/note-with-title')
|
||||
response = client.get('/article/note-with-title')
|
||||
assert response.status_code == 200, f"Failed to get note: {response.status_code}"
|
||||
parsed = mf2py.parse(doc=response.data.decode(), url='http://localhost/note/note-with-title')
|
||||
parsed = mf2py.parse(doc=response.data.decode(), url='http://localhost/article/note-with-title')
|
||||
|
||||
entries = [item for item in parsed['items'] if 'h-entry' in item.get('type', [])]
|
||||
assert len(entries) > 0, "No h-entry found in parsed HTML"
|
||||
|
||||
@@ -142,7 +142,8 @@ def test_split_title(content, title):
|
||||
@pytest.mark.parametrize("content,kind", [
|
||||
("#indieweb is great, posting from my phone", "note"),
|
||||
("#\n\nbody text", "note"),
|
||||
("# A Title\n\nThe body", "article"),
|
||||
# A heading in the text is no longer a title (v2.16.0, ADR-076)
|
||||
("# A Title\n\nThe body", "note"),
|
||||
])
|
||||
def test_post_type_of_hash_notes(app, client, content, kind):
|
||||
note = make_note(app, content)
|
||||
|
||||
Loaded 100 of 111 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user