Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c89c2cd5ca | ||
|
|
ab6ff60f5c | ||
|
|
57682165a4 | ||
|
|
8c51b52ed6 | ||
|
|
42c77e34b7 | ||
|
|
438465592c |
No files matched your search
@@ -7,6 +7,65 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.9.0] - 2026-11-03
|
||||
|
||||
Micropub review against the [W3C Recommendation](https://www.w3.org/TR/micropub/).
|
||||
|
||||
### Changed
|
||||
|
||||
- **Micropub posts are only crossposted to the targets the client asks for.** If a client doesn't send `mp-syndicate-to`, nothing is crossposted (before, everything was). Posts made in the admin editor are unchanged
|
||||
- Posts sent with `visibility` private or unlisted are saved as drafts instead of being published (StarPunk has no private posts)
|
||||
- A token with only the `draft` scope can create drafts, and nothing else
|
||||
- `q=source` and `q=category` need a token that can post (`create`, `update`, `delete` or `draft`); before, any token could list every post, drafts included
|
||||
- Sending the access token in both the header and the body is refused (400)
|
||||
|
||||
### Added
|
||||
|
||||
- Bookmarks: Micropub `bookmark-of` is stored and shown ("🔖 Bookmarked …", `u-bookmark-of`), returned by `q=source`, listed in `q=config`, and crossposts lead with the bookmarked link (migration 015)
|
||||
- A post with only a `summary` uses it as the text; a post with only a `name` is a titled post with no body
|
||||
|
||||
### Fixed
|
||||
|
||||
- Quill and other JSON clients put `mp-syndicate-to` and `mp-slug` inside `properties`; they were ignored, so posts crossposted everywhere and slugs were not used
|
||||
- HTML in posts is sanitized: scripts, event handlers, iframes and `javascript:` links are removed when notes are shown. Formatting, images, code highlighting, footnotes and microformats are kept
|
||||
- A photo that can't be downloaded or found made the post without it while reporting success. Create and update now fail with 400 and change nothing
|
||||
- Remote photos are fetched the same way as webmentions: every redirect is checked for private addresses and the connection goes to the checked address
|
||||
- JSON values that weren't lists were read a character at a time (`"category": "solo"` became tags s, o, l). Single values are now treated as a one-item list; `delete` values in updates must be lists
|
||||
- Posts written as HTML are returned by `q=source` as `{"html": …}`, as the spec requires; empty `content` is left out
|
||||
- A client-supplied `published` time is stored as UTC like every other post; a date that can't be read is refused instead of silently replaced with now
|
||||
- Malformed JSON gets 400 instead of 500, and error messages no longer include Python internals
|
||||
- `video` and `audio` (not supported) are refused with 400 instead of being dropped; person tags in `category` are ignored instead of failing the post
|
||||
- An update with no `replace`, `add` or `delete` is refused
|
||||
- `Authorization: bearer …` is accepted (the scheme is case-insensitive), and 401 responses include `WWW-Authenticate`
|
||||
|
||||
## [2.8.1] - 2026-11-03
|
||||
|
||||
### Fixed
|
||||
|
||||
- Starting the server after an upgrade with a database migration could fail: several server processes applied the same migration at once and one stopped with "duplicate column name", taking the server down until it restarted (this happened deploying v2.8.0, for about two seconds). The migration lock was released right after it was taken, and again before each migration was recorded. Migrations now run entirely inside the lock
|
||||
|
||||
## [2.8.0] - 2026-11-03
|
||||
|
||||
Microformats2 review: the markup now matches the [microformats2](https://microformats.org/wiki/microformats2) vocabularies, so IndieWeb readers and tools see what the page shows.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Replies to a note weren't marked up as its comments, so readers and threading tools saw none. They are now `u-comment h-cite`s with the replier's h-card (name, link, photo) and the reply's HTML
|
||||
- Likes, reposts, bookmarks and mentions on a note had no microformats at all. Each is now a `u-like`, `u-repost`, `u-bookmark` or `u-mention` h-cite with its author, including likes past the eight faces shown
|
||||
- Responses link to and are dated by the original post (its own URL and published time) instead of the address that sent the webmention and when it arrived. New database columns hold these (migration 014)
|
||||
- Post times had no time zone (and production showed microseconds), so readers could shift them by hours. Times on pages and in Micropub `q=source` are now UTC with a `Z`, matching the Atom feed
|
||||
- A note starting with a hashtag (`#indieweb is great…`) was treated as a titled article: its text became a heading and the body was empty, and a note starting with a bare `#` line got its slug as a title. Only a Markdown heading (`#` then a space) makes a title now
|
||||
- A webmention from one of a note's own crossposts (for example your GoToSocial copy, sent back through Bridgy) showed up as a response on the note. It is now ignored
|
||||
- Micropub and IndieAuth discovery links disappeared from pages when webmentions were turned off
|
||||
- A new site had no `rel="me"` link to its author, so readers couldn't tell whose site it was; pages now always link `rel="me"` and `rel="author"` to the author's home page
|
||||
- Finding the author's profile could pick someone else's h-card from their home page. It now follows the [representative h-card](https://microformats.org/wiki/representative-h-card-parsing) rules
|
||||
- A webmention's author is found by the [authorship algorithm](https://indieweb.org/authorship-spec), from the entry on the source page that actually responds to the note: an author given as a name or link, and the author of a feed the entry is in, now count
|
||||
|
||||
### Added
|
||||
|
||||
- Tag pages and the likes page have a feed URL and author, like the home page
|
||||
- Note previews have `u-uid`
|
||||
|
||||
## [2.7.2] - 2026-11-03
|
||||
|
||||
### Added
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# v2.8.0: Microformats2 review
|
||||
|
||||
Review of StarPunk's published and consumed microformats against
|
||||
microformats2, h-entry, h-card, h-feed, h-cite, representative h-card
|
||||
parsing, the authorship spec and Post Type Discovery. Every page type was
|
||||
rendered, parsed with mf2py and checked with mf2util.
|
||||
|
||||
Posts (h-entry) were largely correct: no implied names, PTD right for
|
||||
note/article/reply/like/repost/photo, author h-card, u-url/u-uid,
|
||||
p-category, u-syndication.
|
||||
|
||||
## Fixes
|
||||
|
||||
| # | Problem | Fix |
|
||||
|---|---|---|
|
||||
| 1 | Replies were unattached `h-cite` children with a plain-text author | `u-comment h-cite`, `p-author h-card` (u-url on an inner link, u-photo, p-name), `e-content` (`templates/partials/responses.html`) |
|
||||
| 2 | Likes/reposts/bookmarks/mentions had no mf2 | `u-like` / `u-repost` / `u-bookmark` / `u-mention` h-cites; faces past 8 are `hidden`, not omitted |
|
||||
| 3 | `dt-*` naive, with microseconds | `iso_utc` filter/helper: RFC 3339 UTC `Z`, no fraction; Micropub `q=source` too |
|
||||
| 4 | `#hashtag` first line = title; bare `#` = slug title | `split_title` requires an ATX heading (`#{1,6}` + space); templates use `Note.has_title` |
|
||||
| 5 | Note's own syndicated copy shown as a mention | Worker rejects a source whose url/uid (or URL) is one of the note's `syndications.remote_url` as invalid |
|
||||
| 6 | Micropub/IndieAuth links gated on WEBMENTION_ENABLED | Only `rel=webmention` is gated |
|
||||
| 7 | No rel=me on a fresh install | `rel=me` and `rel=author` to `author.url or author.me or ADMIN_ME`, always |
|
||||
| 8 | Author discovery guessed h-cards | `mf2_utils.representative_hcard` (uid+url, rel=me, single h-card); parsed with the URL after redirects |
|
||||
| 9 | Webmention entry/author/time | `mf2_utils.responding_entry` (entry whose in-reply-to/like-of/... or content refers to the target), `entry_author` (author property as h-card/URL/name, parent h-feed author, rel=author, representative h-card), `source_post_url` and `source_published` stored (migration 014) |
|
||||
|
||||
Also: tag and likes h-feeds have `u-url` and `p-author`; previews have `u-uid`.
|
||||
|
||||
## Not done
|
||||
|
||||
- Reply-context h-cite does not carry the cached author h-card, name or published time.
|
||||
- Photo alt falls back to "Image" when a photo-only post has no caption.
|
||||
- Person tags (`p-category h-card`) are not supported.
|
||||
- Existing webmentions keep NULL source details until they are re-validated.
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/test_microformats_v280.py`. `tests/test_microformats.py`
|
||||
`test_no_relme_without_author` became `test_relme_to_admin_me_without_author`.
|
||||
@@ -0,0 +1,33 @@
|
||||
# v2.8.1: Migration lock held for the whole run
|
||||
|
||||
## Incident
|
||||
|
||||
Deploying v2.8.0 (migration 014 adds two columns): one gunicorn worker
|
||||
applied 014, another failed with `duplicate column name: source_post_url`,
|
||||
gunicorn shut down ("Worker failed to boot") and Docker restarted the
|
||||
container. About two seconds of downtime; schema correct afterwards.
|
||||
|
||||
## Causes
|
||||
|
||||
`run_migrations` takes `BEGIN IMMEDIATE` so one worker migrates at a time,
|
||||
but the lock was released early twice:
|
||||
|
||||
1. `create_migrations_table()`, called right after `BEGIN IMMEDIATE`,
|
||||
ended with `conn.commit()`. Everything after it ran without the lock.
|
||||
2. `conn.executescript()` COMMITs any open transaction before running the
|
||||
script, so the lock was dropped before the migration was recorded in
|
||||
`schema_migrations`.
|
||||
|
||||
## Fix
|
||||
|
||||
- `create_migrations_table(conn, commit=True)`; `run_migrations` passes
|
||||
`commit=False`.
|
||||
- `execute_in_transaction()` runs each statement from
|
||||
`split_sql_statements()` (built on `sqlite3.complete_statement`, so
|
||||
trigger bodies stay whole) with `conn.execute`, inside the transaction.
|
||||
|
||||
## Verification
|
||||
|
||||
Four processes migrating a database at 013 simultaneously: 6/30 trials
|
||||
failed before, 0/30 after. `tests/test_migration_concurrency.py` runs this
|
||||
five times, plus a rollback test showing a migration commits nothing early.
|
||||
@@ -0,0 +1,51 @@
|
||||
# v2.9.0: Micropub review
|
||||
|
||||
Review against the W3C Micropub Recommendation (https://www.w3.org/TR/micropub/),
|
||||
run through the Flask test client and mapped to the micropub.rocks server
|
||||
tests. Everyday create/update/delete/query/media/auth already passed; the
|
||||
gaps were privacy, security and silent data loss.
|
||||
|
||||
## Decisions (user)
|
||||
|
||||
- No `mp-syndicate-to` from a Micropub client means no crossposting
|
||||
(was: all targets, v2.1.0). The admin editor's default is unchanged.
|
||||
- `visibility` private/unlisted are saved as drafts.
|
||||
- Rendered HTML is sanitized with an allowlist (nh3 defaults plus class,
|
||||
title, id, lang, dir on all tags; video/audio/source/track/picture).
|
||||
|
||||
## Changes
|
||||
|
||||
| Area | Before | After |
|
||||
|---|---|---|
|
||||
| `mp-*` in JSON `properties` | ignored (all targets, timestamp slug) | read as commands, never stored |
|
||||
| absent `mp-syndicate-to` | all targets | none |
|
||||
| `q=source` / `q=category` | any token | create, update, delete or draft scope |
|
||||
| HTML content | rendered raw (stored XSS) | `render_markdown()` = Markdown + nh3 |
|
||||
| `visibility` | ignored (published) | private/unlisted -> draft; unknown -> 400 |
|
||||
| `draft` scope | 403 | creates drafts only |
|
||||
| photo fetch/lookup failure | dropped, 201 | 400, nothing created or changed |
|
||||
| remote photo fetch | first host checked, redirects followed | `webmention_ssrf.safe_get` (per hop, pinned) |
|
||||
| JSON scalars | indexed per character | wrapped on create; update `delete` values must be arrays |
|
||||
| `bookmark-of` | dropped | `notes.bookmark_of`, kind `bookmark` (migration 015) |
|
||||
| HTML-authored content in `q=source` | string | `{"html": ...}` via `notes.content_format` |
|
||||
| `summary` / `name` only | 400 | used as text / titled post |
|
||||
| `video` / `audio` | dropped | 400 |
|
||||
| `published` | stored with offset; garbage -> now | naive UTC; garbage -> 400 |
|
||||
| malformed JSON | 500 | 400 |
|
||||
| update without operations | 204 | 400 |
|
||||
| auth | `Bearer` only; header+body accepted; no challenge | case-insensitive; both -> 400; `WWW-Authenticate` |
|
||||
|
||||
## Kept as is
|
||||
|
||||
- 403 (RFC 6750) for insufficient_scope; the Micropub table says 401.
|
||||
- `add` only for category and photo; `published`, `syndication`,
|
||||
`location` not updatable.
|
||||
- `location`/`checkin` ignored (3.3.5 allows).
|
||||
- GET `access_token` query parameter still accepted.
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/test_micropub_v290.py` (31 of 34 fail on v2.8.1). Updated:
|
||||
likes/reposts helper sends `mp-syndicate-to` in properties;
|
||||
`test_micropub_default_is_no_targets`; remote photo test uses the pinned
|
||||
fetcher; corrupted-image message.
|
||||
@@ -0,0 +1,6 @@
|
||||
-- Migration 014: Source details for received webmentions (v2.8.0)
|
||||
-- The source's own u-url and dt-published, so a response links to and dates
|
||||
-- the original post rather than the URL that sent it and when it arrived.
|
||||
|
||||
ALTER TABLE webmentions ADD COLUMN source_post_url TEXT;
|
||||
ALTER TABLE webmentions ADD COLUMN source_published TIMESTAMP;
|
||||
@@ -0,0 +1,7 @@
|
||||
-- Migration 015: Bookmarks and HTML-authored content (v2.9.0)
|
||||
-- bookmark_of: URL bookmarked (Micropub bookmark-of); the note is a bookmark.
|
||||
-- content_format: 'html' when a Micropub client sent the content as HTML,
|
||||
-- so q=source returns it as {"html": ...} (Micropub 3.7.2). NULL = Markdown.
|
||||
|
||||
ALTER TABLE notes ADD COLUMN bookmark_of TEXT;
|
||||
ALTER TABLE notes ADD COLUMN content_format TEXT;
|
||||
@@ -219,6 +219,10 @@ def create_app(config=None):
|
||||
|
||||
return {'author': author}
|
||||
|
||||
# Template filter - RFC 3339 UTC for dt-* properties (v2.8.0)
|
||||
from starpunk.utils import iso_utc
|
||||
app.add_template_filter(iso_utc, 'iso_utc')
|
||||
|
||||
# Template filter - URL to color for facepile (v1.6.0)
|
||||
@app.template_filter('url_to_color')
|
||||
def url_to_color_filter(url):
|
||||
@@ -459,5 +463,5 @@ def create_app(config=None):
|
||||
|
||||
# Package version (Semantic Versioning 2.0.0)
|
||||
# See docs/standards/versioning-strategy.md for details
|
||||
__version__ = "2.7.2"
|
||||
__version_info__ = (2, 7, 2)
|
||||
__version__ = "2.9.0"
|
||||
__version_info__ = (2, 9, 0)
|
||||
@@ -82,10 +82,12 @@ def discover_author_profile(me_url: str) -> Optional[Dict]:
|
||||
response.raise_for_status()
|
||||
|
||||
# Parse microformats from HTML
|
||||
parsed = mf2py.parse(doc=response.text, url=me_url)
|
||||
# Relative URLs and the representative h-card use the URL after redirects
|
||||
page_url = str(response.url) if isinstance(response.url, (str, httpx.URL)) else me_url
|
||||
parsed = mf2py.parse(doc=response.text, url=page_url)
|
||||
|
||||
# Extract h-card (per Q16: first representative h-card)
|
||||
hcard = _find_representative_hcard(parsed, me_url)
|
||||
# Representative h-card (v2.8.0: per the parsing spec)
|
||||
hcard = _find_representative_hcard(parsed, page_url)
|
||||
|
||||
if not hcard:
|
||||
current_app.logger.warning(f"No h-card found at {me_url}")
|
||||
@@ -132,47 +134,21 @@ def discover_author_profile(me_url: str) -> Optional[Dict]:
|
||||
|
||||
def _find_representative_hcard(parsed: dict, me_url: str) -> Optional[dict]:
|
||||
"""
|
||||
Find representative h-card from parsed microformats
|
||||
Find the representative h-card of the author's page (v2.8.0)
|
||||
|
||||
Per Q16: First representative h-card = first h-card with p-name
|
||||
Per Q18: First h-card with url property matching profile URL
|
||||
Follows https://microformats.org/wiki/representative-h-card-parsing
|
||||
(uid+url match, rel=me match, or the only h-card with a matching url).
|
||||
Other h-cards on the page may be other people, so none is guessed.
|
||||
|
||||
Args:
|
||||
parsed: Parsed microformats data from mf2py
|
||||
me_url: Profile URL for matching
|
||||
me_url: Profile URL (the page that was parsed)
|
||||
|
||||
Returns:
|
||||
h-card dict or None if not found
|
||||
h-card dict or None if the page has no representative h-card
|
||||
"""
|
||||
items = parsed.get('items', [])
|
||||
|
||||
# First try: h-card with matching URL (most specific)
|
||||
for item in items:
|
||||
if 'h-card' in item.get('type', []):
|
||||
properties = item.get('properties', {})
|
||||
urls = properties.get('url', [])
|
||||
|
||||
# Check if any URL matches the profile URL
|
||||
for url in urls:
|
||||
if isinstance(url, dict):
|
||||
url = url.get('value', '')
|
||||
if _normalize_url(url) == _normalize_url(me_url):
|
||||
# Found matching h-card
|
||||
return item
|
||||
|
||||
# Second try: First h-card with p-name (representative h-card)
|
||||
for item in items:
|
||||
if 'h-card' in item.get('type', []):
|
||||
properties = item.get('properties', {})
|
||||
if properties.get('name'):
|
||||
return item
|
||||
|
||||
# Third try: Just use first h-card if any
|
||||
for item in items:
|
||||
if 'h-card' in item.get('type', []):
|
||||
return item
|
||||
|
||||
return None
|
||||
from starpunk.mf2_utils import representative_hcard
|
||||
return representative_hcard(parsed, me_url)
|
||||
|
||||
|
||||
def _get_property(hcard: dict, prop_name: str) -> Optional[str]:
|
||||
|
||||
+6
-2
@@ -93,6 +93,10 @@ def get_optimization_params(file_size: int) -> Tuple[int, int]:
|
||||
return (1280, 85)
|
||||
|
||||
|
||||
# Shown to clients; the parser's own message names internal objects (v2.9.0)
|
||||
UNSUPPORTED_IMAGE = "Not a supported image: use JPEG, PNG, GIF, WebP or HEIC"
|
||||
|
||||
|
||||
def validate_image(file_data: bytes, filename: str) -> Tuple[Image.Image, str, int, int]:
|
||||
"""
|
||||
Validate image file and return PIL Image for memory-efficient processing
|
||||
@@ -158,9 +162,9 @@ def validate_image(file_data: bytes, filename: str) -> Tuple[Image.Image, str, i
|
||||
current_app.logger.info(f'Saved failed upload for analysis: {debug_file}')
|
||||
except RuntimeError:
|
||||
pass # Outside app context (e.g., tests)
|
||||
raise ValueError(f"Invalid or corrupted image: {e}")
|
||||
raise ValueError(UNSUPPORTED_IMAGE) from e
|
||||
else:
|
||||
raise ValueError(f"Invalid or corrupted image: {e}")
|
||||
raise ValueError(UNSUPPORTED_IMAGE) from e
|
||||
|
||||
# Store original format before any conversions
|
||||
original_format = img.format
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
"""
|
||||
Microformats2 consuming helpers (v2.8.0)
|
||||
|
||||
Shared by Webmention processing and author discovery:
|
||||
- representative h-card of a page
|
||||
- the h-entry on a page that responds to a given URL
|
||||
- the author of an h-entry (authorship algorithm)
|
||||
|
||||
References:
|
||||
- https://microformats.org/wiki/representative-h-card-parsing
|
||||
- https://indieweb.org/authorship-spec
|
||||
"""
|
||||
|
||||
from typing import Iterator, Optional
|
||||
from urllib.parse import urldefrag, urlparse
|
||||
|
||||
RESPONSE_PROPERTIES = ('in-reply-to', 'like-of', 'repost-of', 'bookmark-of')
|
||||
|
||||
|
||||
def normalize_url(url: str) -> str:
|
||||
"""URL for comparison: no fragment, no trailing slash, lowercase scheme/host"""
|
||||
if not url:
|
||||
return ''
|
||||
url = urldefrag(url.strip())[0]
|
||||
parsed = urlparse(url)
|
||||
url = parsed._replace(scheme=parsed.scheme.lower(), netloc=parsed.netloc.lower()).geturl()
|
||||
return url.rstrip('/')
|
||||
|
||||
|
||||
def value_of(value) -> Optional[str]:
|
||||
"""Plain value of an mf2 property value (strings, {'value':...}, {'html':...})"""
|
||||
if isinstance(value, dict):
|
||||
value = value.get('value')
|
||||
return value if isinstance(value, str) and value else None
|
||||
|
||||
|
||||
def first(item: dict, prop: str) -> Optional[str]:
|
||||
"""First plain value of an item's property"""
|
||||
for value in item.get('properties', {}).get(prop, []):
|
||||
result = value_of(value)
|
||||
if result:
|
||||
return result
|
||||
return None
|
||||
|
||||
|
||||
def values(item: dict, prop: str) -> list[str]:
|
||||
"""All plain values of an item's property"""
|
||||
return [v for v in (value_of(x) for x in item.get('properties', {}).get(prop, [])) if v]
|
||||
|
||||
|
||||
def walk(items: list) -> Iterator[dict]:
|
||||
"""Every microformat in items, depth first: top level, children, nested properties"""
|
||||
for item in items:
|
||||
if not isinstance(item, dict) or 'type' not in item:
|
||||
continue
|
||||
yield item
|
||||
yield from walk(item.get('children', []))
|
||||
for prop_values in item.get('properties', {}).values():
|
||||
yield from walk([v for v in prop_values if isinstance(v, dict)])
|
||||
|
||||
|
||||
def is_type(item: dict, mf_type: str) -> bool:
|
||||
return mf_type in item.get('type', [])
|
||||
|
||||
|
||||
def representative_hcard(parsed: dict, page_url: str) -> Optional[dict]:
|
||||
"""
|
||||
The page's representative h-card, per the parsing spec, in order:
|
||||
|
||||
1. An h-card whose uid and url both match the page URL
|
||||
2. An h-card whose url matches one of the page's rel=me URLs
|
||||
3. The only h-card on the page, if its url matches the page URL
|
||||
|
||||
Otherwise the page has none.
|
||||
"""
|
||||
page = normalize_url(page_url)
|
||||
cards = [item for item in walk(parsed.get('items', [])) if is_type(item, 'h-card')]
|
||||
|
||||
for card in cards:
|
||||
if page in map(normalize_url, values(card, 'uid')) and \
|
||||
page in map(normalize_url, values(card, 'url')):
|
||||
return card
|
||||
|
||||
rel_me = {normalize_url(u) for u in parsed.get('rels', {}).get('me', [])}
|
||||
for card in cards:
|
||||
if rel_me & set(map(normalize_url, values(card, 'url'))):
|
||||
return card
|
||||
|
||||
if len(cards) == 1 and page in map(normalize_url, values(cards[0], 'url')):
|
||||
return cards[0]
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def responding_entry(parsed: dict, target_url: str) -> Optional[dict]:
|
||||
"""
|
||||
The h-entry on a page that refers to target_url
|
||||
|
||||
Prefers an entry whose in-reply-to / like-of / repost-of / bookmark-of
|
||||
is the target, then one whose content links to it, then the first
|
||||
top-level h-entry.
|
||||
"""
|
||||
target = normalize_url(target_url)
|
||||
entries = [item for item in walk(parsed.get('items', [])) if is_type(item, 'h-entry')]
|
||||
top = [item for item in parsed.get('items', []) if is_type(item, 'h-entry')]
|
||||
|
||||
if not target:
|
||||
return top[0] if top else (entries[0] if entries else None)
|
||||
|
||||
for entry in entries:
|
||||
for prop in RESPONSE_PROPERTIES:
|
||||
if target in map(normalize_url, values(entry, prop)):
|
||||
return entry
|
||||
|
||||
for entry in entries:
|
||||
for content in entry.get('properties', {}).get('content', []):
|
||||
html = content.get('html', '') if isinstance(content, dict) else str(content)
|
||||
if target_url in html or target in html:
|
||||
return entry
|
||||
|
||||
return top[0] if top else (entries[0] if entries else None)
|
||||
|
||||
|
||||
def response_type(entry: dict, target_url: str) -> str:
|
||||
"""reply / like / repost / bookmark if the entry is one to target_url, else mention"""
|
||||
target = normalize_url(target_url)
|
||||
for prop, kind in zip(RESPONSE_PROPERTIES, ('reply', 'like', 'repost', 'bookmark')):
|
||||
if target in map(normalize_url, values(entry, prop)):
|
||||
return kind
|
||||
return 'mention'
|
||||
|
||||
|
||||
def entry_author(parsed: dict, entry: Optional[dict], page_url: str) -> dict:
|
||||
"""
|
||||
Author of an h-entry, following the authorship algorithm without fetching
|
||||
|
||||
1. The entry's author property (an h-card, a URL, or a name)
|
||||
2. The author of an h-feed that contains the entry
|
||||
3. The page's rel=author link
|
||||
4. The page's representative h-card
|
||||
|
||||
Returns:
|
||||
{'name', 'url', 'photo'}; values may be None
|
||||
"""
|
||||
author = None
|
||||
if entry:
|
||||
author = (entry.get('properties', {}).get('author') or [None])[0]
|
||||
if author is None:
|
||||
for feed in walk(parsed.get('items', [])):
|
||||
if is_type(feed, 'h-feed') and entry in feed.get('children', []):
|
||||
author = (feed.get('properties', {}).get('author') or [None])[0]
|
||||
break
|
||||
|
||||
if author is None:
|
||||
rel_author = parsed.get('rels', {}).get('author', [])
|
||||
author = rel_author[0] if rel_author else representative_hcard(parsed, page_url)
|
||||
|
||||
if isinstance(author, dict) and 'type' in author:
|
||||
return {
|
||||
'name': first(author, 'name'),
|
||||
'url': first(author, 'url'),
|
||||
'photo': first(author, 'photo'),
|
||||
}
|
||||
|
||||
text = value_of(author)
|
||||
if text and urlparse(text).scheme in ('http', 'https'):
|
||||
return {'name': None, 'url': text, 'photo': None}
|
||||
return {'name': text, 'url': None, 'photo': None}
|
||||
+157
-82
@@ -25,7 +25,7 @@ References:
|
||||
- IndieAuth Specification: https://www.w3.org/TR/indieauth/
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
from flask import Request, current_app, jsonify
|
||||
@@ -36,7 +36,7 @@ from starpunk.notes import (
|
||||
InvalidNoteDataError, NoteNotFoundError,
|
||||
)
|
||||
from starpunk.indieauth import check_scope
|
||||
from starpunk.utils import absolute_url_for
|
||||
from starpunk.utils import absolute_url_for, iso_utc
|
||||
|
||||
|
||||
# Custom Exceptions
|
||||
@@ -81,9 +81,17 @@ def error_response(error: str, error_description: str, status_code: int = 400):
|
||||
Returns:
|
||||
Tuple of (response, status_code)
|
||||
"""
|
||||
headers = {}
|
||||
if status_code == 401:
|
||||
# RFC 6750 3: tell the client how to authenticate (v2.9.0)
|
||||
challenge = 'Bearer realm="micropub"'
|
||||
if error == "unauthorized" and "Invalid" in error_description:
|
||||
challenge += ', error="invalid_token"'
|
||||
headers["WWW-Authenticate"] = challenge
|
||||
return (
|
||||
jsonify({"error": error, "error_description": error_description}),
|
||||
status_code,
|
||||
headers,
|
||||
)
|
||||
|
||||
|
||||
@@ -104,18 +112,21 @@ def extract_bearer_token(request: Request) -> Optional[str]:
|
||||
Returns:
|
||||
Token string if found, None otherwise
|
||||
"""
|
||||
# Try Authorization header first
|
||||
auth_header = request.headers.get("Authorization", "")
|
||||
if auth_header.startswith("Bearer "):
|
||||
return auth_header[7:] # Remove "Bearer " prefix
|
||||
# The scheme is case-insensitive (RFC 7235)
|
||||
scheme, _, credentials = request.headers.get("Authorization", "").partition(" ")
|
||||
header_token = credentials.strip() if scheme.lower() == "bearer" else None
|
||||
|
||||
# Try form parameter
|
||||
body_token = None
|
||||
if request.method == "POST":
|
||||
return request.form.get("access_token")
|
||||
body_token = request.form.get("access_token")
|
||||
elif request.method == "GET":
|
||||
return request.args.get("access_token")
|
||||
body_token = request.args.get("access_token")
|
||||
|
||||
return None
|
||||
# Only one method may be used (RFC 6750 2) (v2.9.0)
|
||||
if header_token and body_token:
|
||||
raise MicropubValidationError("Send the access token in the header or the body, not both")
|
||||
|
||||
return header_token or body_token
|
||||
|
||||
|
||||
# Property Normalization
|
||||
@@ -135,9 +146,18 @@ def normalize_properties(data: dict) -> dict:
|
||||
Returns:
|
||||
Normalized properties dict with all values as lists
|
||||
"""
|
||||
# JSON format has properties nested
|
||||
# JSON format has properties nested. Values MUST be arrays (3.3.2); a
|
||||
# bare value is wrapped rather than read character by character.
|
||||
# mp-* commands and access_token are not properties (3.2).
|
||||
if "properties" in data:
|
||||
return data["properties"]
|
||||
properties = data["properties"]
|
||||
if not isinstance(properties, dict):
|
||||
raise MicropubValidationError("properties must be an object")
|
||||
return {
|
||||
key: value if isinstance(value, list) else [value]
|
||||
for key, value in properties.items()
|
||||
if not key.startswith("mp-") and key != "access_token"
|
||||
}
|
||||
|
||||
# Form format - convert to properties dict
|
||||
properties = {}
|
||||
@@ -176,7 +196,7 @@ def extract_content(properties: dict) -> str:
|
||||
Raises:
|
||||
MicropubValidationError: If content is missing or empty
|
||||
"""
|
||||
content_list = properties.get("content", [])
|
||||
content_list = properties.get("content") or properties.get("summary") or []
|
||||
|
||||
# Handle both plain text and HTML/text objects
|
||||
if not content_list:
|
||||
@@ -189,12 +209,20 @@ def extract_content(properties: dict) -> str:
|
||||
# Prefer text over html for markdown storage
|
||||
content = content.get("text") or content.get("html", "")
|
||||
|
||||
if not content or not content.strip():
|
||||
if not isinstance(content, str) or not content.strip():
|
||||
raise MicropubValidationError("Content cannot be empty")
|
||||
|
||||
return content.strip()
|
||||
|
||||
|
||||
def content_format(properties: dict) -> Optional[str]:
|
||||
"""'html' if the content was sent only as HTML (returned as such by q=source)"""
|
||||
content = _first(properties.get("content"))
|
||||
if isinstance(content, dict) and content.get("html") and not content.get("text"):
|
||||
return "html"
|
||||
return None
|
||||
|
||||
|
||||
def _has_text(content) -> bool:
|
||||
"""Whether a content property has any text. Likes, reposts (v2.5.0) and
|
||||
photo posts (v2.6.5) may come without."""
|
||||
@@ -245,8 +273,9 @@ def extract_tags(properties: dict) -> list[str]:
|
||||
List of tag strings
|
||||
"""
|
||||
categories = properties.get("category", [])
|
||||
# Filter out empty strings and strip whitespace
|
||||
return [tag.strip() for tag in categories if tag and tag.strip()]
|
||||
# Strings only: person tags (h-card objects) aren't supported and are
|
||||
# ignored, as Micropub says to ignore what isn't recognised (3.3.5)
|
||||
return [tag.strip() for tag in categories if isinstance(tag, str) and tag.strip()]
|
||||
|
||||
|
||||
def extract_published_date(properties: dict) -> Optional[datetime]:
|
||||
@@ -259,18 +288,18 @@ def extract_published_date(properties: dict) -> Optional[datetime]:
|
||||
Returns:
|
||||
Datetime object if published date provided, None otherwise
|
||||
"""
|
||||
published = properties.get("published", [""])[0]
|
||||
published = _first(properties.get("published"))
|
||||
if not published:
|
||||
return None
|
||||
|
||||
try:
|
||||
# Parse ISO 8601 datetime
|
||||
# datetime.fromisoformat handles most ISO formats
|
||||
return datetime.fromisoformat(published.replace("Z", "+00:00"))
|
||||
except (ValueError, AttributeError):
|
||||
# If parsing fails, log and return None (will use current time)
|
||||
current_app.logger.warning(f"Failed to parse published date: {published}")
|
||||
return None
|
||||
value = datetime.fromisoformat(str(published).strip().replace("Z", "+00:00"))
|
||||
except ValueError:
|
||||
raise MicropubValidationError(f"published is not an ISO 8601 date: {published}")
|
||||
# Stored times are naive UTC, like every other note (v2.9.0)
|
||||
if value.tzinfo is not None:
|
||||
value = value.astimezone(timezone.utc).replace(tzinfo=None)
|
||||
return value
|
||||
|
||||
|
||||
# Action Handlers
|
||||
@@ -306,6 +335,24 @@ def extract_post_status(properties: dict) -> bool:
|
||||
raise MicropubValidationError(f"Unsupported post-status: {status}")
|
||||
|
||||
|
||||
def extract_visibility_draft(properties: dict) -> bool:
|
||||
"""
|
||||
True if visibility asks for a non-public post (v2.9.0)
|
||||
|
||||
StarPunk only has public posts and drafts, so private and unlisted posts
|
||||
are kept as drafts rather than published.
|
||||
"""
|
||||
visibility = _first(properties.get("visibility"))
|
||||
if visibility in (None, "", "public"):
|
||||
return False
|
||||
if visibility in ("private", "unlisted"):
|
||||
return True
|
||||
raise MicropubValidationError(f"Unsupported visibility: {visibility}")
|
||||
|
||||
|
||||
UNSUPPORTED_MEDIA = ("video", "audio")
|
||||
|
||||
|
||||
def extract_url_property(properties: dict, name: str) -> Optional[str]:
|
||||
"""A URL property (in-reply-to, like-of, repost-of): string or h-cite"""
|
||||
value = _first(properties.get(name))
|
||||
@@ -361,27 +408,25 @@ def _public_address(host: str) -> bool:
|
||||
|
||||
|
||||
def _fetch_remote_photo(url: str) -> Optional[int]:
|
||||
"""Download a photo hosted elsewhere into the media library. Returns media id."""
|
||||
"""
|
||||
Download a photo hosted elsewhere into the media library. Returns media id.
|
||||
|
||||
Uses the webmention fetcher (v2.9.0): every redirect is checked against
|
||||
private addresses and the connection is pinned to the checked address.
|
||||
"""
|
||||
from urllib.parse import urlparse
|
||||
import httpx
|
||||
from starpunk.media import save_media
|
||||
from starpunk.webmention_ssrf import safe_get, user_agent
|
||||
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in ("http", "https") or not parsed.hostname:
|
||||
return None
|
||||
if not current_app.config.get("TESTING") and not _public_address(parsed.hostname):
|
||||
current_app.logger.warning(f"Refusing to fetch photo from non-public address: {url}")
|
||||
return None
|
||||
try:
|
||||
with httpx.stream("GET", url, timeout=20.0, follow_redirects=True) as response:
|
||||
response.raise_for_status()
|
||||
data = bytearray()
|
||||
for chunk in response.iter_bytes():
|
||||
data += chunk
|
||||
if len(data) > MAX_REMOTE_PHOTO_BYTES:
|
||||
raise ValueError("photo too large")
|
||||
name = parsed.path.rsplit("/", 1)[-1] or "photo.jpg"
|
||||
return save_media(bytes(data), name)["id"]
|
||||
result = safe_get(
|
||||
url, headers={"User-Agent": user_agent(current_app.config.get("SITE_URL", ""))},
|
||||
max_bytes=MAX_REMOTE_PHOTO_BYTES, timeout=20,
|
||||
)
|
||||
if result.status_code != 200:
|
||||
raise ValueError(f"HTTP {result.status_code}")
|
||||
name = urlparse(result.url).path.rsplit("/", 1)[-1] or "photo.jpg"
|
||||
return save_media(result.content, name)["id"]
|
||||
except Exception as e:
|
||||
current_app.logger.warning(f"Could not fetch photo {url}: {e}")
|
||||
return None
|
||||
@@ -430,8 +475,11 @@ def _resolve_photos(photos: list[dict]) -> tuple[list[int], list[str]]:
|
||||
if len(media_ids) == MAX_PHOTOS:
|
||||
break
|
||||
media_id = photo.get("media_id") or _media_id_for_url(photo["url"])
|
||||
if not media_id:
|
||||
# Don't create or change the post without it (v2.9.0)
|
||||
raise MicropubValidationError(f"Could not get photo {photo.get('url')}")
|
||||
# The same photo twice is shown once (v2.6.5)
|
||||
if media_id and media_id not in media_ids:
|
||||
if media_id not in media_ids:
|
||||
media_ids.append(media_id)
|
||||
captions.append(photo.get("alt", ""))
|
||||
return media_ids, captions
|
||||
@@ -442,8 +490,7 @@ def _attach_photos_to_note(note_id: int, photos: list[dict]) -> None:
|
||||
from starpunk.media import attach_media_to_note
|
||||
|
||||
media_ids, captions = _resolve_photos(photos)
|
||||
if media_ids or not photos:
|
||||
attach_media_to_note(note_id, media_ids, captions)
|
||||
attach_media_to_note(note_id, media_ids, captions)
|
||||
|
||||
|
||||
def _note_photos(note) -> list[dict]:
|
||||
@@ -508,47 +555,54 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
Returns:
|
||||
Tuple of (response_body, status_code, headers)
|
||||
"""
|
||||
_require_scope(token_info, "create")
|
||||
_require_scope(token_info, "create", "draft")
|
||||
# A token with only the draft scope may create drafts (v2.9.0)
|
||||
drafts_only = not check_scope("create", token_info.get("scope", ""))
|
||||
_check_type(data)
|
||||
|
||||
# mp-slug and mp-syndicate-to are server commands, not properties
|
||||
custom_slug = None
|
||||
if isinstance(data, dict) and 'mp-slug' in data:
|
||||
slug_value = data.get('mp-slug')
|
||||
if isinstance(slug_value, list) and slug_value:
|
||||
custom_slug = slug_value[0]
|
||||
elif isinstance(slug_value, str):
|
||||
custom_slug = slug_value
|
||||
# mp-slug and mp-syndicate-to are server commands, not properties. Form
|
||||
# posts send them at the top level; JSON clients inside properties too.
|
||||
commands = dict(data) if isinstance(data, dict) else {}
|
||||
if isinstance(commands.get("properties"), dict):
|
||||
commands.update({k: v for k, v in commands["properties"].items() if k.startswith("mp-")})
|
||||
|
||||
# Absent means "all targets"; present means exactly those (v2.1.0)
|
||||
requested_targets = None
|
||||
if isinstance(data, dict):
|
||||
for key in ("mp-syndicate-to", "mp-syndicate-to[]"):
|
||||
if key in data:
|
||||
value = data[key]
|
||||
requested_targets = (requested_targets or []) + (
|
||||
value if isinstance(value, list) else [value]
|
||||
)
|
||||
custom_slug = _first(commands.get("mp-slug"))
|
||||
if custom_slug is not None and not isinstance(custom_slug, str):
|
||||
raise MicropubValidationError("mp-slug must be a string")
|
||||
|
||||
# Crossposting only to the targets asked for: absent means none (v2.9.0)
|
||||
requested_targets = []
|
||||
for key in ("mp-syndicate-to", "mp-syndicate-to[]"):
|
||||
if key in commands:
|
||||
value = commands[key]
|
||||
requested_targets += value if isinstance(value, list) else [value]
|
||||
|
||||
try:
|
||||
properties = normalize_properties(data)
|
||||
like_of = extract_url_property(properties, "like-of")
|
||||
repost_of = extract_url_property(properties, "repost-of")
|
||||
if like_of and repost_of:
|
||||
raise MicropubValidationError("A post can't be both a like and a repost")
|
||||
has_content = _has_text(properties.get("content"))
|
||||
bookmark_of = extract_url_property(properties, "bookmark-of")
|
||||
if sum(bool(u) for u in (like_of, repost_of, bookmark_of)) > 1:
|
||||
raise MicropubValidationError("A post can only be one of a like, repost or bookmark")
|
||||
for prop in UNSUPPORTED_MEDIA:
|
||||
if properties.get(prop) or (files and (files.getlist(prop) or files.getlist(f"{prop}[]"))):
|
||||
raise MicropubValidationError(f"{prop} posts are not supported")
|
||||
has_content = _has_text(properties.get("content")) or _has_text(properties.get("summary"))
|
||||
photos = extract_photos(properties) + _save_uploads(files)
|
||||
# Photo posts may have no text (v2.6.5): Micropub doesn't require content
|
||||
title = extract_title(properties)
|
||||
# Photo posts may have no text (v2.6.5), nor titled posts (v2.9.0)
|
||||
content = (
|
||||
extract_content(properties)
|
||||
if has_content or not (like_of or repost_of or photos)
|
||||
if has_content or not (like_of or repost_of or bookmark_of or photos or title)
|
||||
else ""
|
||||
)
|
||||
title = extract_title(properties)
|
||||
tags = extract_tags(properties)
|
||||
published_date = extract_published_date(properties)
|
||||
published = extract_post_status(properties)
|
||||
published = extract_post_status(properties) and not extract_visibility_draft(properties)
|
||||
if drafts_only:
|
||||
published = False
|
||||
in_reply_to = extract_in_reply_to(properties)
|
||||
fmt = content_format(properties) if has_content else None
|
||||
# Look up and download photos before the note exists, so a bad photo
|
||||
# can't leave a half-made post behind (v2.6.5)
|
||||
media_ids, captions = _resolve_photos(photos)
|
||||
@@ -556,7 +610,7 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
raise
|
||||
except Exception as e:
|
||||
current_app.logger.error(f"Property extraction failed: {e}")
|
||||
raise MicropubValidationError(f"Failed to parse request: {str(e)}")
|
||||
raise MicropubValidationError("Could not read the post's properties")
|
||||
|
||||
note = None
|
||||
try:
|
||||
@@ -569,7 +623,9 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
in_reply_to=in_reply_to,
|
||||
like_of=like_of,
|
||||
repost_of=repost_of,
|
||||
bookmark_of=bookmark_of,
|
||||
has_media=bool(media_ids),
|
||||
content_format=fmt,
|
||||
)
|
||||
|
||||
if media_ids:
|
||||
@@ -577,11 +633,8 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
attach_media_to_note(note.id, media_ids, captions)
|
||||
|
||||
# Crosspost after photos are attached (v2.1.0); drafts wait until published
|
||||
from starpunk.syndication import get_targets, resolve_targets, select_targets
|
||||
if requested_targets is None:
|
||||
select_targets(note.id, list(get_targets()))
|
||||
else:
|
||||
select_targets(note.id, resolve_targets(requested_targets))
|
||||
from starpunk.syndication import resolve_targets, select_targets
|
||||
select_targets(note.id, resolve_targets(requested_targets))
|
||||
|
||||
return "", 201, {"Location": absolute_url_for("public.note", slug=note.slug)}
|
||||
|
||||
@@ -618,9 +671,12 @@ def handle_update(data: dict, token_info: dict):
|
||||
delete = data.get("delete") or {}
|
||||
if not isinstance(replace, dict) or not isinstance(add, dict) or not isinstance(delete, (dict, list)):
|
||||
raise MicropubValidationError("replace and add must be objects; delete an object or list")
|
||||
for prop, values in list(replace.items()) + list(add.items()):
|
||||
for prop, values in list(replace.items()) + list(add.items()) + (
|
||||
list(delete.items()) if isinstance(delete, dict) else []):
|
||||
if not isinstance(values, list):
|
||||
raise MicropubValidationError(f"Values for {prop} must be an array")
|
||||
if not (replace or add or delete):
|
||||
raise MicropubValidationError("An update needs replace, add or delete")
|
||||
touched = set(replace) | set(add) | set(delete)
|
||||
unsupported = sorted(touched - UPDATABLE)
|
||||
if unsupported:
|
||||
@@ -664,6 +720,7 @@ def handle_update(data: dict, token_info: dict):
|
||||
if "content" in replace:
|
||||
# Empty text is fine if the note keeps its photos; update_note checks (v2.6.5)
|
||||
body = extract_content(replace) if _has_text(replace["content"]) else ""
|
||||
changes["content_format"] = content_format(replace) if body else None
|
||||
if "name" in replace:
|
||||
title = extract_title(replace)
|
||||
if "category" in replace:
|
||||
@@ -681,13 +738,19 @@ def handle_update(data: dict, token_info: dict):
|
||||
if "photo" in add:
|
||||
photos += extract_photos(add)
|
||||
|
||||
# Text may go only if photos stay (or it's a like or repost) (v2.6.5)
|
||||
if not (body or "").strip() and not (photos or note.like_of or note.repost_of):
|
||||
# Text may go only if photos stay (or it's a like, repost or bookmark) (v2.6.5)
|
||||
if not (body or "").strip() and not (photos or note.reacts_to):
|
||||
raise MicropubValidationError("A post needs text or a photo")
|
||||
|
||||
# Photos must all resolve before anything changes (v2.9.0)
|
||||
if "photo" in touched:
|
||||
media_ids, captions = _resolve_photos(photos)
|
||||
|
||||
new_content = add_title_heading(body, title)
|
||||
if new_content != note.content:
|
||||
changes["content"] = new_content
|
||||
elif "content_format" in changes and changes["content_format"] == note.content_format:
|
||||
del changes["content_format"]
|
||||
if {"category"} & touched:
|
||||
changes["tags"] = tags
|
||||
|
||||
@@ -695,7 +758,8 @@ def handle_update(data: dict, token_info: dict):
|
||||
if changes:
|
||||
update_note(id=note.id, has_media=bool(photos), **changes)
|
||||
if "photo" in touched:
|
||||
_attach_photos_to_note(note.id, photos)
|
||||
from starpunk.media import attach_media_to_note
|
||||
attach_media_to_note(note.id, media_ids, captions)
|
||||
note_changed(note.id)
|
||||
except InvalidNoteDataError as e:
|
||||
raise MicropubValidationError(str(e))
|
||||
@@ -742,8 +806,9 @@ def note_to_mf2(note, wanted: Optional[list] = None) -> dict:
|
||||
|
||||
title, content = split_title(note.content)
|
||||
props = {
|
||||
"content": [content],
|
||||
"published": [note.created_at.isoformat()],
|
||||
# HTML-authored content comes back as HTML (3.7.2, v2.9.0)
|
||||
"content": [{"html": content} if note.content_format == "html" else content],
|
||||
"published": [iso_utc(note.created_at)],
|
||||
"url": [absolute_url_for("public.note", slug=note.slug)],
|
||||
"post-status": ["published" if note.published else "draft"],
|
||||
}
|
||||
@@ -760,6 +825,10 @@ def note_to_mf2(note, wanted: Optional[list] = None) -> dict:
|
||||
props["like-of"] = [note.like_of]
|
||||
if getattr(note, "repost_of", None):
|
||||
props["repost-of"] = [note.repost_of]
|
||||
if getattr(note, "bookmark_of", None):
|
||||
props["bookmark-of"] = [note.bookmark_of]
|
||||
if not content.strip():
|
||||
del props["content"] # photo posts, likes, reposts and bookmarks without text
|
||||
photos = _note_photos(note)
|
||||
if photos:
|
||||
props["photo"] = [{"value": p["url"], "alt": p["alt"]} for p in photos]
|
||||
@@ -812,6 +881,11 @@ def handle_query(args, token_info: dict):
|
||||
|
||||
q = args.get("q")
|
||||
|
||||
# Posts (drafts included) and tags are for apps that may post (v2.9.0);
|
||||
# config and syndicate-to are fine for any token
|
||||
if q in ("source", "category"):
|
||||
_require_scope(token_info, "create", "update", "delete", "draft")
|
||||
|
||||
if q == "config":
|
||||
site_url = current_app.config.get("SITE_URL", "http://localhost:5000").rstrip('/')
|
||||
return jsonify({
|
||||
@@ -825,6 +899,7 @@ def handle_query(args, token_info: dict):
|
||||
{"type": "reply", "name": "Reply", "properties": ["in-reply-to", "content"]},
|
||||
{"type": "like", "name": "Like", "properties": ["like-of"]},
|
||||
{"type": "repost", "name": "Repost", "properties": ["repost-of", "content"]},
|
||||
{"type": "bookmark", "name": "Bookmark", "properties": ["bookmark-of", "content"]},
|
||||
],
|
||||
}), 200
|
||||
|
||||
@@ -853,4 +928,4 @@ def handle_query(args, token_info: dict):
|
||||
if q == "category":
|
||||
return jsonify({"categories": _all_categories(args.get("filter") or "")}), 200
|
||||
|
||||
return error_response("invalid_request", f"Unknown query: {q}")
|
||||
return error_response("invalid_request", f"Unknown query: {q}" if q else "Missing q parameter")
|
||||
+42
-6
@@ -31,12 +31,47 @@ class MigrationError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def create_migrations_table(conn):
|
||||
def split_sql_statements(script: str) -> list[str]:
|
||||
"""
|
||||
Split a SQL script into complete statements (v2.8.1)
|
||||
|
||||
Uses sqlite3.complete_statement, so semicolons inside strings and
|
||||
trigger bodies (BEGIN ... END) don't split a statement.
|
||||
"""
|
||||
statements = []
|
||||
current = ""
|
||||
for line in script.splitlines(keepends=True):
|
||||
current += line
|
||||
if sqlite3.complete_statement(current):
|
||||
if current.strip().strip(";").strip():
|
||||
statements.append(current.strip())
|
||||
current = ""
|
||||
if current.strip():
|
||||
statements.append(current.strip())
|
||||
return statements
|
||||
|
||||
|
||||
def execute_in_transaction(conn, script: str) -> None:
|
||||
"""
|
||||
Run a SQL script inside the connection's open transaction
|
||||
|
||||
Not conn.executescript(): that COMMITs first, which released the
|
||||
migration lock mid-migration and let another gunicorn worker apply the
|
||||
same migration again ("duplicate column name", v2.8.0 deploy).
|
||||
"""
|
||||
for statement in split_sql_statements(script):
|
||||
conn.execute(statement)
|
||||
|
||||
|
||||
def create_migrations_table(conn, commit=True):
|
||||
"""
|
||||
Create schema_migrations tracking table if it doesn't exist
|
||||
|
||||
Args:
|
||||
conn: SQLite connection
|
||||
commit: Commit afterwards. run_migrations passes False: committing
|
||||
there released its BEGIN IMMEDIATE lock straight away, so
|
||||
gunicorn workers applied migrations concurrently (v2.8.1)
|
||||
"""
|
||||
conn.execute("""
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
@@ -51,7 +86,8 @@ def create_migrations_table(conn):
|
||||
ON schema_migrations(migration_name)
|
||||
""")
|
||||
|
||||
conn.commit()
|
||||
if commit:
|
||||
conn.commit()
|
||||
|
||||
|
||||
def is_schema_current(conn):
|
||||
@@ -287,7 +323,7 @@ def apply_migration(conn, migration_name, migration_path, logger=None):
|
||||
|
||||
# Execute migration in transaction
|
||||
conn.execute("BEGIN")
|
||||
conn.executescript(migration_sql)
|
||||
execute_in_transaction(conn, migration_sql)
|
||||
|
||||
# Record migration as applied
|
||||
conn.execute(
|
||||
@@ -374,7 +410,7 @@ def run_migrations(db_path, logger=None):
|
||||
|
||||
try:
|
||||
# Ensure migrations tracking table exists
|
||||
create_migrations_table(conn)
|
||||
create_migrations_table(conn, commit=False)
|
||||
|
||||
# Quick check: have migrations already been applied by another worker?
|
||||
cursor = conn.execute("SELECT COUNT(*) FROM schema_migrations")
|
||||
@@ -472,8 +508,8 @@ def run_migrations(db_path, logger=None):
|
||||
|
||||
logger.debug(f"Applying migration: {migration_name}")
|
||||
|
||||
# Execute migration (already in transaction)
|
||||
conn.executescript(migration_sql)
|
||||
# Execute migration inside the locked transaction
|
||||
execute_in_transaction(conn, migration_sql)
|
||||
|
||||
# Record migration as applied
|
||||
conn.execute(
|
||||
|
||||
+61
-19
@@ -11,10 +11,12 @@ from dataclasses import dataclass, field, replace
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
import re
|
||||
import sqlite3
|
||||
|
||||
# Third-party imports
|
||||
import markdown
|
||||
import nh3
|
||||
|
||||
# Local imports
|
||||
from starpunk.utils import (
|
||||
@@ -40,6 +42,35 @@ MARKDOWN_EXTENSIONS = ["extra", "codehilite", "nl2br", BareUrlExtension()]
|
||||
|
||||
# Constants - Content limits
|
||||
MAX_TITLE_LENGTH = 200
|
||||
|
||||
# HTML allowed in rendered notes (v2.9.0). Markdown passes raw HTML through,
|
||||
# and Micropub clients may send HTML, so scripts, event handlers, iframes and
|
||||
# javascript: links are stripped. class keeps code highlighting and
|
||||
# microformats; id keeps footnote links.
|
||||
ALLOWED_TAGS = set(nh3.ALLOWED_TAGS) | {"video", "audio", "source", "track", "picture"}
|
||||
ALLOWED_ATTRIBUTES = {
|
||||
**{tag: set(attrs) for tag, attrs in nh3.ALLOWED_ATTRIBUTES.items()},
|
||||
"*": {"class", "title", "id", "lang", "dir"},
|
||||
"img": set(nh3.ALLOWED_ATTRIBUTES.get("img", ())) | {"loading", "srcset", "sizes"},
|
||||
"video": {"src", "controls", "poster", "width", "height", "loop", "muted", "playsinline", "preload"},
|
||||
"audio": {"src", "controls", "loop", "muted", "preload"},
|
||||
"source": {"src", "type", "srcset", "media", "sizes"},
|
||||
"track": {"src", "kind", "label", "srclang", "default"},
|
||||
}
|
||||
|
||||
|
||||
def render_markdown(text: str) -> str:
|
||||
"""Note Markdown (or HTML) to sanitized HTML"""
|
||||
html = markdown.markdown(text, extensions=MARKDOWN_EXTENSIONS)
|
||||
return nh3.clean(html, tags=ALLOWED_TAGS, attributes=ALLOWED_ATTRIBUTES)
|
||||
|
||||
|
||||
# Likes, reposts and bookmarks (v2.5.0, v2.9.0)
|
||||
REACTION_VERBS = {"like": "Liked", "repost": "Reposted", "bookmark": "Bookmarked"}
|
||||
REACTION_ICONS = {"like": "★", "repost": "🔁", "bookmark": "🔖"}
|
||||
|
||||
# A Markdown ATX heading: up to 6 '#', whitespace, then text
|
||||
TITLE_RE = re.compile(r"#{1,6}[ \t]+(\S.*)$")
|
||||
EXCERPT_LENGTH = 200
|
||||
|
||||
|
||||
@@ -47,17 +78,19 @@ def split_title(content: str) -> tuple[Optional[str], str]:
|
||||
"""
|
||||
Split a note's '# Heading' first line (its title, per Q22) off its content
|
||||
|
||||
The first line must be a Markdown heading: 1-6 '#' then a space and text.
|
||||
'#indieweb ...' (a hashtag) or a bare '#' is not a title. This is the one
|
||||
place that decides whether a note has a title (templates use has_title).
|
||||
|
||||
Returns:
|
||||
(title, remaining content); title is None if the note has no heading
|
||||
"""
|
||||
stripped = content.lstrip()
|
||||
if not stripped.startswith("#"):
|
||||
return None, content
|
||||
first_line, _, rest = stripped.partition("\n")
|
||||
title = first_line.lstrip("#").strip()
|
||||
if not title:
|
||||
match = TITLE_RE.match(first_line)
|
||||
if not match:
|
||||
return None, content
|
||||
return title, rest.strip()
|
||||
return match.group(1).strip().rstrip("#").strip(), rest.strip()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -136,6 +169,8 @@ class Note:
|
||||
in_reply_to: Optional[str] = None # v1.6.0: URL being replied to
|
||||
like_of: Optional[str] = None # v2.5.0: URL liked (this note is a like)
|
||||
repost_of: Optional[str] = None # v2.5.0: URL reposted (this note is a repost)
|
||||
bookmark_of: Optional[str] = None # v2.9.0: URL bookmarked (this note is a bookmark)
|
||||
content_format: Optional[str] = None # v2.9.0: 'html' if sent as HTML via Micropub
|
||||
_cached_content: Optional[str] = field(
|
||||
default=None, repr=False, compare=False, init=False
|
||||
)
|
||||
@@ -196,6 +231,8 @@ class Note:
|
||||
in_reply_to=data.get("in_reply_to"),
|
||||
like_of=data.get("like_of"),
|
||||
repost_of=data.get("repost_of"),
|
||||
bookmark_of=data.get("bookmark_of"),
|
||||
content_format=data.get("content_format"),
|
||||
)
|
||||
|
||||
@property
|
||||
@@ -254,11 +291,8 @@ class Note:
|
||||
<p>This is my note content...</p>
|
||||
"""
|
||||
if self._cached_html is None:
|
||||
# Render markdown to HTML
|
||||
html = markdown.markdown(
|
||||
self.content, # This triggers lazy load of content
|
||||
extensions=MARKDOWN_EXTENSIONS,
|
||||
)
|
||||
# Render markdown to sanitized HTML (v2.9.0)
|
||||
html = render_markdown(self.content) # triggers lazy load of content
|
||||
|
||||
# Cache it
|
||||
object.__setattr__(self, "_cached_html", html)
|
||||
@@ -276,7 +310,15 @@ class Note:
|
||||
title, body = split_title(self.content)
|
||||
if title is None:
|
||||
return self.html
|
||||
return markdown.markdown(body, extensions=MARKDOWN_EXTENSIONS)
|
||||
return render_markdown(body)
|
||||
|
||||
@property
|
||||
def has_title(self) -> bool:
|
||||
"""True if the note starts with a '# Heading' title (an article)"""
|
||||
try:
|
||||
return split_title(self.content)[0] is not None
|
||||
except (FileNotFoundError, OSError):
|
||||
return False
|
||||
|
||||
@property
|
||||
def title(self) -> str:
|
||||
@@ -372,9 +414,8 @@ class Note:
|
||||
excerpt += "..."
|
||||
|
||||
if not excerpt and self.reacts_to:
|
||||
# A like or repost with no comment
|
||||
verb = "Liked" if self.kind == "like" else "Reposted"
|
||||
return f"{verb} {self.reacts_to}"
|
||||
# A like, repost or bookmark with no comment
|
||||
return f"{REACTION_VERBS[self.kind]} {self.reacts_to}"
|
||||
|
||||
return excerpt
|
||||
|
||||
@@ -384,17 +425,19 @@ class Note:
|
||||
|
||||
@property
|
||||
def kind(self) -> str:
|
||||
"""'like', 'repost' or 'note' (v2.5.0)"""
|
||||
"""'like', 'repost', 'bookmark' (v2.9.0) or 'note' (v2.5.0)"""
|
||||
if self.like_of:
|
||||
return "like"
|
||||
if self.repost_of:
|
||||
return "repost"
|
||||
if self.bookmark_of:
|
||||
return "bookmark"
|
||||
return "note"
|
||||
|
||||
@property
|
||||
def reacts_to(self) -> Optional[str]:
|
||||
"""The URL a like or repost points at"""
|
||||
return self.like_of or self.repost_of
|
||||
"""The URL a like, repost or bookmark points at"""
|
||||
return self.like_of or self.repost_of or self.bookmark_of
|
||||
|
||||
@property
|
||||
def feed_html(self) -> str:
|
||||
@@ -403,9 +446,8 @@ class Note:
|
||||
|
||||
if not self.reacts_to:
|
||||
return self.html
|
||||
verb = "Liked" if self.kind == "like" else "Reposted"
|
||||
url = escape(self.reacts_to, quote=True)
|
||||
line = f'<p>{"★" if self.kind == "like" else "🔁"} {verb} <a href="{url}">{url}</a></p>'
|
||||
line = f'<p>{REACTION_ICONS[self.kind]} {REACTION_VERBS[self.kind]} <a href="{url}">{url}</a></p>'
|
||||
return line + (self.html or "")
|
||||
|
||||
@property
|
||||
|
||||
+28
-10
@@ -133,6 +133,10 @@ def _get_existing_slugs(db) -> set[str]:
|
||||
# Core CRUD Functions
|
||||
|
||||
|
||||
# update_note: leave a field as it is
|
||||
UNCHANGED = object()
|
||||
|
||||
|
||||
def create_note(
|
||||
content: str,
|
||||
published: bool = False,
|
||||
@@ -143,6 +147,8 @@ def create_note(
|
||||
like_of: Optional[str] = None,
|
||||
repost_of: Optional[str] = None,
|
||||
has_media: bool = False,
|
||||
bookmark_of: Optional[str] = None,
|
||||
content_format: Optional[str] = None,
|
||||
) -> Note:
|
||||
"""
|
||||
Create a new note
|
||||
@@ -207,10 +213,13 @@ def create_note(
|
||||
"""
|
||||
# 1. VALIDATION (before any changes)
|
||||
# Likes and reposts may have no comment (v2.5.0), photo posts no text (v2.6.5)
|
||||
if like_of and repost_of:
|
||||
raise InvalidNoteDataError("like_of", like_of, "A note can't be both a like and a repost")
|
||||
# Bookmarks too (v2.9.0)
|
||||
if sum(bool(u) for u in (like_of, repost_of, bookmark_of)) > 1:
|
||||
raise InvalidNoteDataError(
|
||||
"like_of", like_of, "A note can only be one of a like, repost or bookmark"
|
||||
)
|
||||
content = content or ""
|
||||
if not content.strip() and not (like_of or repost_of or has_media):
|
||||
if not content.strip() and not (like_of or repost_of or bookmark_of or has_media):
|
||||
raise InvalidNoteDataError(
|
||||
"content", content, "Content cannot be empty or whitespace-only"
|
||||
)
|
||||
@@ -271,10 +280,12 @@ def create_note(
|
||||
try:
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO notes (slug, file_path, published, created_at, updated_at, content_hash, in_reply_to, like_of, repost_of)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
INSERT INTO notes (slug, file_path, published, created_at, updated_at, content_hash,
|
||||
in_reply_to, like_of, repost_of, bookmark_of, content_format)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(slug, file_path_rel, published, created_at, updated_at, content_hash, in_reply_to, like_of, repost_of),
|
||||
(slug, file_path_rel, published, created_at, updated_at, content_hash,
|
||||
in_reply_to, like_of, repost_of, bookmark_of, content_format),
|
||||
)
|
||||
db.commit()
|
||||
except Exception as e:
|
||||
@@ -342,7 +353,7 @@ def create_note(
|
||||
note_id=note_id,
|
||||
note_url=note_url,
|
||||
content_html=html_content,
|
||||
in_reply_to=in_reply_to or like_of or repost_of
|
||||
in_reply_to=in_reply_to or like_of or repost_of or bookmark_of
|
||||
)
|
||||
except Exception as e:
|
||||
# Webmention queueing failure should not prevent note creation
|
||||
@@ -350,7 +361,7 @@ def create_note(
|
||||
|
||||
# 12. FETCH REPLY CONTEXT (v1.6.0 Phase 3)
|
||||
# Fetch context for the post being replied to, liked or reposted
|
||||
context_url = in_reply_to or like_of or repost_of
|
||||
context_url = in_reply_to or like_of or repost_of or bookmark_of
|
||||
if context_url:
|
||||
try:
|
||||
from starpunk.reply_context import fetch_reply_context
|
||||
@@ -607,10 +618,13 @@ def update_note(
|
||||
tags: Optional[list[str]] = None,
|
||||
in_reply_to: Optional[str] = None,
|
||||
has_media: bool = False,
|
||||
content_format=UNCHANGED,
|
||||
) -> Note:
|
||||
"""
|
||||
Update a note's content and/or published status
|
||||
|
||||
content_format (v2.9.0): 'html' or None (Markdown); left alone unless given
|
||||
|
||||
Updates note content and/or metadata, maintaining atomic synchronization
|
||||
between file and database. At least one of content or published must
|
||||
be provided.
|
||||
@@ -692,7 +706,7 @@ def update_note(
|
||||
identifier = slug if slug is not None else id
|
||||
raise NoteNotFoundError(identifier)
|
||||
|
||||
if empty_content and not (existing_note.like_of or existing_note.repost_of or has_media):
|
||||
if empty_content and not (existing_note.reacts_to or has_media):
|
||||
raise InvalidNoteDataError(
|
||||
"content", content, "Content cannot be empty or whitespace-only"
|
||||
)
|
||||
@@ -756,6 +770,10 @@ def update_note(
|
||||
update_fields.append("in_reply_to = ?")
|
||||
params.append(in_reply_to)
|
||||
|
||||
if content_format is not UNCHANGED:
|
||||
update_fields.append("content_format = ?")
|
||||
params.append(content_format)
|
||||
|
||||
# Add WHERE clause parameter
|
||||
if slug is not None:
|
||||
where_clause = "slug = ?"
|
||||
@@ -841,7 +859,7 @@ def update_note(
|
||||
html_content = updated_note.html if updated_note.published else None
|
||||
|
||||
# Determine in_reply_to (use new value if provided, otherwise existing)
|
||||
reply_to = (in_reply_to if in_reply_to is not None else updated_note.in_reply_to) or updated_note.like_of or updated_note.repost_of
|
||||
reply_to = (in_reply_to if in_reply_to is not None else updated_note.in_reply_to) or updated_note.reacts_to
|
||||
|
||||
# Queue webmentions (best-effort)
|
||||
# Pass old_html for URL diffing (Phase 6)
|
||||
|
||||
@@ -37,7 +37,7 @@ from starpunk.indieauth import (
|
||||
validate_authorization_request,
|
||||
verify_access_token,
|
||||
)
|
||||
from starpunk.micropub import extract_bearer_token
|
||||
from starpunk.micropub import MicropubError, extract_bearer_token
|
||||
from starpunk.utils import absolute_url_for
|
||||
|
||||
bp = Blueprint("indieauth", __name__)
|
||||
@@ -215,7 +215,11 @@ def token():
|
||||
@bp.route("/auth/token", methods=["GET"])
|
||||
def token_verify():
|
||||
"""Legacy token verification for clients that still call it"""
|
||||
info = verify_access_token(extract_bearer_token(request))
|
||||
try:
|
||||
token = extract_bearer_token(request)
|
||||
except MicropubError as e:
|
||||
return _oauth_error("invalid_request", e.error_description)
|
||||
info = verify_access_token(token)
|
||||
if not info:
|
||||
return _oauth_error("unauthorized", "Invalid or expired access token", 401)
|
||||
return _json(info)
|
||||
|
||||
@@ -70,7 +70,10 @@ def micropub_endpoint():
|
||||
}
|
||||
"""
|
||||
# Extract and verify token
|
||||
token = extract_bearer_token(request)
|
||||
try:
|
||||
token = extract_bearer_token(request)
|
||||
except MicropubError as e:
|
||||
return error_response(e.error, e.error_description, e.status_code)
|
||||
if not token:
|
||||
return error_response("unauthorized", "No access token provided", 401)
|
||||
|
||||
@@ -94,7 +97,10 @@ def micropub_endpoint():
|
||||
try:
|
||||
# Parse request based on content type
|
||||
if "application/json" in content_type:
|
||||
data = request.get_json() or {}
|
||||
# Malformed JSON is the client's error, not ours (v2.9.0)
|
||||
data = request.get_json(silent=True)
|
||||
if not isinstance(data, dict):
|
||||
return error_response("invalid_request", "Request body must be a JSON object", 400)
|
||||
action = data.get("action", "create")
|
||||
else:
|
||||
# Form-encoded or multipart (files handled separately)
|
||||
@@ -136,7 +142,10 @@ def media_endpoint():
|
||||
from starpunk.media import save_media
|
||||
|
||||
# Extract and verify token
|
||||
token = extract_bearer_token(request)
|
||||
try:
|
||||
token = extract_bearer_token(request)
|
||||
except MicropubError as e:
|
||||
return error_response(e.error, e.error_description, e.status_code)
|
||||
if not token:
|
||||
return error_response("unauthorized", "No access token provided", 401)
|
||||
|
||||
|
||||
@@ -1002,6 +1002,9 @@ def compose_text(note, permalink: str, tags: List[str], max_chars: int,
|
||||
hashtags = f"{hashtags} {h}".strip()
|
||||
tail = f"\n\n{permalink}" + (f"\n\n{hashtags}" if hashtags else "")
|
||||
body = (note.content if body is None else body).strip()
|
||||
if getattr(note, "bookmark_of", None):
|
||||
# A bookmark leads with what it bookmarks (v2.9.0)
|
||||
body = f"🔖 {note.bookmark_of}\n\n{body}".strip()
|
||||
if len(body) + len(tail) > max_chars:
|
||||
room = max_chars - len(tail) - 1
|
||||
body = body[:room].rsplit(" ", 1)[0].rstrip() + "…"
|
||||
|
||||
+24
-1
@@ -11,7 +11,7 @@ import hashlib
|
||||
import re
|
||||
import secrets
|
||||
import shutil
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
@@ -693,3 +693,26 @@ def absolute_url_for(endpoint: str, **values) -> str:
|
||||
from flask import current_app, url_for
|
||||
|
||||
return current_app.config["SITE_URL"].rstrip("/") + url_for(endpoint, **values)
|
||||
|
||||
|
||||
def iso_utc(value) -> str:
|
||||
"""
|
||||
Format a timestamp as RFC 3339 UTC for dt-* and Micropub (v2.8.0)
|
||||
|
||||
Stored times are naive UTC. Without an offset, microformats parsers read
|
||||
them as local time, so this adds 'Z' and drops microseconds (HTML
|
||||
<time datetime> allows at most 3 fractional digits).
|
||||
|
||||
Args:
|
||||
value: datetime (naive = UTC) or ISO 8601 string
|
||||
|
||||
Returns:
|
||||
e.g. '2026-11-01T16:25:12Z', or '' for empty values
|
||||
"""
|
||||
if not value:
|
||||
return ''
|
||||
if isinstance(value, str):
|
||||
value = datetime.fromisoformat(value.strip().replace('Z', '+00:00'))
|
||||
if value.tzinfo is None:
|
||||
value = value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ')
|
||||
@@ -19,10 +19,11 @@ import logging
|
||||
import mf2py
|
||||
import nh3
|
||||
import httpx
|
||||
from datetime import datetime
|
||||
from typing import Dict, Any, Optional
|
||||
from urllib.parse import urldefrag, urlparse
|
||||
|
||||
from starpunk import mf2_utils
|
||||
from starpunk.utils import iso_utc
|
||||
from starpunk.webmention_ssrf import (
|
||||
FetchTooLargeError, SSRFBlockedError, safe_get, user_agent,
|
||||
)
|
||||
@@ -147,66 +148,13 @@ def verify_link_presence(content: str, target_url: str, content_type: str,
|
||||
|
||||
def detect_mention_type(parsed_mf2: Dict, target_url: str) -> str:
|
||||
"""
|
||||
Detect mention type from microformats.
|
||||
|
||||
Priority: reply > like > repost > bookmark > mention
|
||||
|
||||
Args:
|
||||
parsed_mf2: Parsed microformats data from mf2py
|
||||
target_url: Target URL being mentioned
|
||||
Detect mention type from the h-entry that responds to the target
|
||||
|
||||
Returns:
|
||||
Mention type: 'reply', 'like', 'repost', 'bookmark', or 'mention'
|
||||
'reply', 'like', 'repost', 'bookmark', or 'mention'
|
||||
"""
|
||||
target_normalized = urlparse(target_url)._replace(fragment='').geturl().rstrip('/')
|
||||
|
||||
# Find h-entry that references target
|
||||
for item in parsed_mf2.get('items', []):
|
||||
if 'h-entry' not in item.get('type', []):
|
||||
continue
|
||||
|
||||
props = item.get('properties', {})
|
||||
|
||||
# Check in-reply-to (highest priority)
|
||||
for reply_to in props.get('in-reply-to', []):
|
||||
url = reply_to if isinstance(reply_to, str) else reply_to.get('value', '')
|
||||
if urlparse(url)._replace(fragment='').geturl().rstrip('/') == target_normalized:
|
||||
return 'reply'
|
||||
|
||||
# Check like-of
|
||||
for like in props.get('like-of', []):
|
||||
url = like if isinstance(like, str) else like.get('value', '')
|
||||
if urlparse(url)._replace(fragment='').geturl().rstrip('/') == target_normalized:
|
||||
return 'like'
|
||||
|
||||
# Check repost-of
|
||||
for repost in props.get('repost-of', []):
|
||||
url = repost if isinstance(repost, str) else repost.get('value', '')
|
||||
if urlparse(url)._replace(fragment='').geturl().rstrip('/') == target_normalized:
|
||||
return 'repost'
|
||||
|
||||
# Check bookmark-of
|
||||
for bookmark in props.get('bookmark-of', []):
|
||||
url = bookmark if isinstance(bookmark, str) else bookmark.get('value', '')
|
||||
if urlparse(url)._replace(fragment='').geturl().rstrip('/') == target_normalized:
|
||||
return 'bookmark'
|
||||
|
||||
# Default fallback
|
||||
return 'mention'
|
||||
|
||||
|
||||
def _first_str(props: Dict, name: str) -> Optional[str]:
|
||||
"""
|
||||
First value of an mf2 property as a string
|
||||
|
||||
mf2py gives u-photo with alt text as {'value': url, 'alt': ...} and nested
|
||||
microformats as dicts with a 'value'; reduce those to the plain value.
|
||||
"""
|
||||
values = props.get(name) or [None]
|
||||
value = values[0]
|
||||
if isinstance(value, dict):
|
||||
value = value.get('value')
|
||||
return value if isinstance(value, str) and value else None
|
||||
entry = mf2_utils.responding_entry(parsed_mf2, target_url)
|
||||
return mf2_utils.response_type(entry, target_url) if entry else 'mention'
|
||||
|
||||
|
||||
def _is_http_url(value: Optional[str]) -> bool:
|
||||
@@ -217,59 +165,27 @@ def _is_http_url(value: Optional[str]) -> bool:
|
||||
return parsed.scheme.lower() in ('http', 'https') and bool(parsed.netloc)
|
||||
|
||||
|
||||
def extract_author(parsed_mf2: Dict, source_url: str) -> Dict[str, Optional[str]]:
|
||||
def extract_author(parsed_mf2: Dict, source_url: str,
|
||||
entry: Optional[Dict] = None) -> Dict[str, Optional[str]]:
|
||||
"""
|
||||
Extract author information from h-card.
|
||||
Author of the source's h-entry (authorship algorithm, without fetching)
|
||||
|
||||
Args:
|
||||
parsed_mf2: Parsed microformats data from mf2py
|
||||
source_url: Source URL (fallback for author URL)
|
||||
source_url: Source URL (fallback for author URL and name)
|
||||
entry: The h-entry to attribute (default: first top-level h-entry)
|
||||
|
||||
Returns:
|
||||
Dict with author_name, author_url, author_photo
|
||||
Dict with author_name, author_url, author_photo; only http(s) URLs
|
||||
are kept, so they are safe to render as links (XSS)
|
||||
"""
|
||||
author_name = None
|
||||
author_url = None
|
||||
author_photo = None
|
||||
if entry is None:
|
||||
entry = mf2_utils.responding_entry(parsed_mf2, '')
|
||||
author = mf2_utils.entry_author(parsed_mf2, entry, source_url)
|
||||
|
||||
# Find h-entry first
|
||||
entry = None
|
||||
for item in parsed_mf2.get('items', []):
|
||||
if 'h-entry' in item.get('type', []):
|
||||
entry = item
|
||||
break
|
||||
|
||||
# Check for author property in h-entry
|
||||
if entry:
|
||||
author_prop = entry.get('properties', {}).get('author', [])
|
||||
if author_prop:
|
||||
author_data = author_prop[0]
|
||||
if isinstance(author_data, dict):
|
||||
# h-card object
|
||||
props = author_data.get('properties', {})
|
||||
author_name = _first_str(props, 'name')
|
||||
author_url = _first_str(props, 'url')
|
||||
author_photo = _first_str(props, 'photo')
|
||||
|
||||
# If no author in h-entry, find representative h-card
|
||||
if not author_name:
|
||||
for item in parsed_mf2.get('items', []):
|
||||
if 'h-card' in item.get('type', []):
|
||||
props = item.get('properties', {})
|
||||
author_name = _first_str(props, 'name')
|
||||
author_url = _first_str(props, 'url')
|
||||
author_photo = _first_str(props, 'photo')
|
||||
break
|
||||
|
||||
# Fallback to hostname if no author found
|
||||
if not author_name:
|
||||
author_name = urlparse(source_url).netloc
|
||||
|
||||
# Only http(s) URLs are rendered as links (blocks javascript: and data:)
|
||||
if not _is_http_url(author_url):
|
||||
author_url = source_url
|
||||
if not _is_http_url(author_photo):
|
||||
author_photo = None
|
||||
author_url = author['url'] if _is_http_url(author['url']) else source_url
|
||||
author_photo = author['photo'] if _is_http_url(author['photo']) else None
|
||||
author_name = author['name'] or urlparse(author_url).netloc
|
||||
|
||||
return {
|
||||
'author_name': author_name,
|
||||
@@ -278,45 +194,56 @@ def extract_author(parsed_mf2: Dict, source_url: str) -> Dict[str, Optional[str]
|
||||
}
|
||||
|
||||
|
||||
def extract_content(parsed_mf2: Dict) -> Optional[str]:
|
||||
def extract_content(parsed_mf2: Dict, entry: Optional[Dict] = None) -> Optional[str]:
|
||||
"""
|
||||
Extract content snippet from h-entry.
|
||||
Content snippet of an h-entry (e-content, else p-summary)
|
||||
|
||||
Args:
|
||||
parsed_mf2: Parsed microformats data from mf2py
|
||||
entry: The h-entry (default: first top-level h-entry)
|
||||
|
||||
Returns:
|
||||
Sanitized content snippet (max 500 chars) or None
|
||||
"""
|
||||
# Find h-entry
|
||||
for item in parsed_mf2.get('items', []):
|
||||
if 'h-entry' not in item.get('type', []):
|
||||
continue
|
||||
if entry is None:
|
||||
entry = mf2_utils.responding_entry(parsed_mf2, '')
|
||||
if not entry:
|
||||
return None
|
||||
|
||||
props = item.get('properties', {})
|
||||
props = entry.get('properties', {})
|
||||
for content in props.get('content', []):
|
||||
html = (content.get('html') or content.get('value')) if isinstance(content, dict) else content
|
||||
if html:
|
||||
return _snippet(html)
|
||||
|
||||
# Try e-content first (HTML)
|
||||
content_prop = props.get('content', [])
|
||||
if content_prop:
|
||||
content_data = content_prop[0]
|
||||
if isinstance(content_data, dict):
|
||||
content = content_data.get('html', '') or content_data.get('value', '')
|
||||
else:
|
||||
content = content_data
|
||||
summary = mf2_utils.first(entry, 'summary')
|
||||
return _snippet(summary) if summary else None
|
||||
|
||||
if content:
|
||||
return _snippet(content)
|
||||
|
||||
# Try p-summary as fallback
|
||||
summary = props.get('summary', [])
|
||||
if summary:
|
||||
content = summary[0]
|
||||
if isinstance(content, dict):
|
||||
content = content.get('value', '')
|
||||
if content:
|
||||
return _snippet(content)
|
||||
def extract_source_details(entry: Optional[Dict]) -> Dict[str, Optional[str]]:
|
||||
"""
|
||||
The source post's own URL, uid and published time
|
||||
|
||||
return None
|
||||
Returns:
|
||||
source_post_url, source_uid (http(s) only) and source_published
|
||||
(RFC 3339 UTC, or None if missing or unparseable)
|
||||
"""
|
||||
if not entry:
|
||||
return {'source_post_url': None, 'source_uid': None, 'source_published': None}
|
||||
|
||||
url = mf2_utils.first(entry, 'url')
|
||||
uid = mf2_utils.first(entry, 'uid')
|
||||
published = mf2_utils.first(entry, 'published')
|
||||
try:
|
||||
published = iso_utc(published) or None
|
||||
except (ValueError, TypeError):
|
||||
published = None
|
||||
|
||||
return {
|
||||
'source_post_url': url if _is_http_url(url) else None,
|
||||
'source_uid': uid,
|
||||
'source_published': published,
|
||||
}
|
||||
|
||||
|
||||
def _snippet(html: str, limit: int = 500) -> str:
|
||||
@@ -399,14 +326,19 @@ def process_webmention(source_url: str, target_url: str, site_url: str) -> Dict[
|
||||
}
|
||||
content_snippet = None
|
||||
|
||||
source_details = extract_source_details(None)
|
||||
|
||||
if parsed_mf2:
|
||||
mention_type = detect_mention_type(parsed_mf2, target_url)
|
||||
author_info = extract_author(parsed_mf2, source_url)
|
||||
content_snippet = extract_content(parsed_mf2)
|
||||
entry = mf2_utils.responding_entry(parsed_mf2, target_url)
|
||||
mention_type = mf2_utils.response_type(entry, target_url) if entry else 'mention'
|
||||
author_info = extract_author(parsed_mf2, final_url, entry)
|
||||
content_snippet = extract_content(parsed_mf2, entry)
|
||||
source_details = extract_source_details(entry)
|
||||
|
||||
return {
|
||||
'valid': True,
|
||||
'mention_type': mention_type,
|
||||
**author_info,
|
||||
'content_snippet': content_snippet
|
||||
'content_snippet': content_snippet,
|
||||
**source_details,
|
||||
}
|
||||
@@ -74,9 +74,13 @@ def get_webmentions_for_note(note_id: int) -> Dict[str, List[Dict[str, Any]]]:
|
||||
|
||||
for row in rows:
|
||||
wm = dict(row)
|
||||
# Parse received_at timestamp
|
||||
if isinstance(wm.get('received_at'), str):
|
||||
wm['received_at'] = datetime.fromisoformat(wm['received_at'].replace('Z', '+00:00'))
|
||||
# Parse timestamps (source_published may be missing or malformed)
|
||||
for key in ('received_at', 'source_published'):
|
||||
if isinstance(wm.get(key), str):
|
||||
try:
|
||||
wm[key] = datetime.fromisoformat(wm[key].replace('Z', '+00:00'))
|
||||
except ValueError:
|
||||
wm[key] = None
|
||||
|
||||
mention_type = wm.get('mention_type', 'mention')
|
||||
if mention_type == 'reply':
|
||||
|
||||
@@ -27,6 +27,7 @@ from lxml import html as lxml_html
|
||||
|
||||
from starpunk.webmention_processor import process_webmention, TransientError, ValidationError
|
||||
from starpunk.webmention_sender import note_responses_changed
|
||||
from starpunk.mf2_utils import normalize_url
|
||||
from starpunk.webmention_ssrf import (
|
||||
FetchTooLargeError, SSRFBlockedError, safe_get, safe_post, user_agent,
|
||||
)
|
||||
@@ -263,6 +264,23 @@ class WebmentionWorker(threading.Thread):
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to re-send webmentions for note {target_note_id}: {e}")
|
||||
|
||||
def _is_own_syndicated_copy(self, note_id: Optional[int], source_url: str,
|
||||
result: Dict[str, Any]) -> bool:
|
||||
"""True if the source is one of the note's own u-syndication copies"""
|
||||
if note_id is None:
|
||||
return False
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
try:
|
||||
copies = {normalize_url(row[0]) for row in conn.execute(
|
||||
"SELECT remote_url FROM syndications WHERE note_id = ? AND remote_url IS NOT NULL",
|
||||
(note_id,),
|
||||
)}
|
||||
finally:
|
||||
conn.close()
|
||||
source = {normalize_url(u) for u in (
|
||||
source_url, result.get('source_post_url'), result.get('source_uid')) if u}
|
||||
return bool(copies & source)
|
||||
|
||||
def _display_state(self, webmention_id: int) -> Optional[tuple]:
|
||||
"""
|
||||
What the note page shows for this webmention, or None if hidden
|
||||
@@ -305,6 +323,11 @@ class WebmentionWorker(threading.Thread):
|
||||
# Process webmention (fetch, validate, extract)
|
||||
result = process_webmention(source_url, target_url, self.site_url)
|
||||
|
||||
# A copy of this very note on a silo (e.g. backfed by Bridgy) is
|
||||
# not a response to it
|
||||
if self._is_own_syndicated_copy(target_note_id, source_url, result):
|
||||
raise ValidationError("Source is this note's own syndicated copy")
|
||||
|
||||
# Update database with successful validation
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
cursor = conn.cursor()
|
||||
@@ -330,6 +353,8 @@ class WebmentionWorker(threading.Thread):
|
||||
author_url = ?,
|
||||
author_photo = ?,
|
||||
content_snippet = ?,
|
||||
source_post_url = ?,
|
||||
source_published = ?,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
WHERE id = ?
|
||||
""", (
|
||||
@@ -339,6 +364,8 @@ class WebmentionWorker(threading.Thread):
|
||||
result['author_url'],
|
||||
result['author_photo'],
|
||||
result['content_snippet'],
|
||||
result.get('source_post_url'),
|
||||
result.get('source_published'),
|
||||
webmention_id
|
||||
))
|
||||
|
||||
@@ -395,7 +422,7 @@ class WebmentionWorker(threading.Thread):
|
||||
# All other errors are 'error' status
|
||||
if transient:
|
||||
status = 'error'
|
||||
elif any(x in error.lower() for x in ['does not link', 'returned 4']):
|
||||
elif any(x in error.lower() for x in ['does not link', 'returned 4', 'own syndicated copy']):
|
||||
status = 'invalid'
|
||||
else:
|
||||
status = 'error'
|
||||
|
||||
@@ -332,6 +332,9 @@ small { display: block; margin-top: var(--spacing-xs); color: var(--color-text-l
|
||||
}
|
||||
|
||||
.webmention-author a {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: var(--spacing-sm);
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
@@ -361,6 +364,13 @@ small { display: block; margin-top: var(--spacing-xs); color: var(--color-text-l
|
||||
margin-left: 0;
|
||||
}
|
||||
|
||||
.facepile .face > .h-card,
|
||||
.facepile .face > .h-card > a {
|
||||
display: block;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
}
|
||||
|
||||
.facepile .face img {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
@@ -392,6 +402,12 @@ small { display: block; margin-top: var(--spacing-xs); color: var(--color-text-l
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
.webmention-mention .h-card img {
|
||||
border-radius: 50%;
|
||||
vertical-align: middle;
|
||||
margin-right: var(--spacing-xs);
|
||||
}
|
||||
|
||||
.webmention-mention p {
|
||||
margin: var(--spacing-xs) 0 0 0;
|
||||
font-size: 0.875rem;
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
<form action="{{ url_for('admin.update_note_submit', note_id=note.id) }}" method="POST" class="note-form">
|
||||
<div class="form-group">
|
||||
{% if note.reacts_to %}
|
||||
<p class="note-meta">{{ '★ Like of' if note.kind == 'like' else '🔁 Repost of' }} <a href="{{ note.reacts_to }}">{{ note.reacts_to }}</a></p>
|
||||
<p class="note-meta">{{ {'like': '★ Like of', 'repost': '🔁 Repost of', 'bookmark': '🔖 Bookmark of'}[note.kind] }} <a href="{{ note.reacts_to }}">{{ note.reacts_to }}</a></p>
|
||||
{% endif %}
|
||||
<label for="content">{% if note.reacts_to %}Comment (optional, Markdown){% else %}Content (Markdown){% endif %}</label>
|
||||
<textarea
|
||||
|
||||
+11
-1
@@ -12,15 +12,25 @@
|
||||
{# Webmention endpoint (v1.6.0) #}
|
||||
{% if config.WEBMENTION_ENABLED %}
|
||||
<link rel="webmention" href="/webmention">
|
||||
{% endif %}
|
||||
|
||||
{# Micropub and IndieAuth discovery (independent of Webmention, v2.8.0) #}
|
||||
<link rel="micropub" href="{{ absolute_url_for('micropub.micropub_endpoint') }}">
|
||||
<link rel="indieauth-metadata" href="{{ absolute_url_for('indieauth.metadata') }}">
|
||||
<link rel="authorization_endpoint" href="{{ absolute_url_for('indieauth.authorize') }}">
|
||||
<link rel="token_endpoint" href="{{ absolute_url_for('indieauth.token') }}">
|
||||
|
||||
{# Author's home page: rel=me makes the h-card linking to it this site's
|
||||
representative h-card; rel=author is the authorship fallback (v2.8.0) #}
|
||||
{% set author_home = (author.url or author.me) if author else config.ADMIN_ME %}
|
||||
{% if author_home %}
|
||||
<link rel="me" href="{{ author_home }}">
|
||||
<link rel="author" href="{{ author_home }}">
|
||||
{% endif %}
|
||||
|
||||
{# rel-me links from discovered author profile (v1.2.0 Phase 2) #}
|
||||
{% if author and author.rel_me_links %}
|
||||
{% for profile_url in author.rel_me_links %}
|
||||
{% for profile_url in author.rel_me_links if profile_url != author_home %}
|
||||
<link rel="me" href="{{ profile_url }}">
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
|
||||
@@ -7,6 +7,12 @@
|
||||
<div class="h-feed">
|
||||
<h1 class="p-name page-title">Likes</h1>
|
||||
|
||||
{# Feed URL and author, as on the home page (v2.8.0) #}
|
||||
<a class="u-url" href="{{ absolute_url_for(request.endpoint, **request.view_args) }}" hidden>Feed URL</a>
|
||||
{% if author %}
|
||||
<span class="p-author h-card" hidden><a class="p-name u-url" href="{{ author.url or author.me }}">{{ author.name or author.url or author.me }}</a></span>
|
||||
{% endif %}
|
||||
|
||||
{% if notes %}
|
||||
{% for note in notes %}
|
||||
{{ note_preview(note, author) }}
|
||||
|
||||
+11
-81
@@ -1,13 +1,14 @@
|
||||
{% extends "base.html" %}
|
||||
{% from "partials/media.html" import display_media %}
|
||||
{% from "partials/reaction.html" import reaction %}
|
||||
{% import "partials/responses.html" as responses with context %}
|
||||
|
||||
{% block title %}{% if note.content.strip().startswith('#') %}{{ note.title }}{% elif note.excerpt.strip() %}{{ note.excerpt|truncate(60) }}{% elif note.repost_of %}Repost{% elif note.like_of %}Like{% elif note.in_reply_to %}Reply{% else %}Photo{% endif %} · {{ config.SITE_NAME }}{% endblock %}
|
||||
{% block title %}{% if note.has_title %}{{ note.title }}{% elif note.excerpt.strip() %}{{ note.excerpt|truncate(60) }}{% elif note.repost_of %}Repost{% elif note.like_of %}Like{% elif note.bookmark_of %}Bookmark{% elif note.in_reply_to %}Reply{% else %}Photo{% endif %} · {{ config.SITE_NAME }}{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<article class="h-entry">
|
||||
{# Detect if note has explicit title (starts with # heading) - per Q22 #}
|
||||
{% set has_explicit_title = note.content.strip().startswith('#') %}
|
||||
{# Notes starting with a '# Heading' have a title (Q22; Note.has_title) #}
|
||||
{% set has_explicit_title = note.has_title %}
|
||||
|
||||
{# p-name only if note has explicit title (per Q22) #}
|
||||
{% if has_explicit_title %}
|
||||
@@ -53,9 +54,9 @@
|
||||
|
||||
<footer class="note-meta">
|
||||
{# u-url and u-uid same for notes (per Q23) #}
|
||||
<a class="u-url u-uid" href="{{ absolute_url_for('public.note', slug=note.slug) }}"><time class="dt-published" datetime="{{ note.created_at.isoformat() }}">{{ note.created_at.strftime('%B %-d, %Y · %-I:%M %p') }}</time></a>
|
||||
<a class="u-url u-uid" href="{{ absolute_url_for('public.note', slug=note.slug) }}"><time class="dt-published" datetime="{{ note.created_at|iso_utc }}">{{ note.created_at.strftime('%B %-d, %Y · %-I:%M %p') }}</time></a>
|
||||
{% if note.updated_at and note.updated_at != note.created_at %}
|
||||
<span class="updated">Updated <time class="dt-updated" datetime="{{ note.updated_at.isoformat() }}">{{ note.updated_at.strftime('%b %-d, %Y') }}</time></span>
|
||||
<span class="updated">Updated <time class="dt-updated" datetime="{{ note.updated_at|iso_utc }}">{{ note.updated_at.strftime('%b %-d, %Y') }}</time></span>
|
||||
{% endif %}
|
||||
{# rel="tag" per microformats2 p-category #}
|
||||
{% if note.tags %}
|
||||
@@ -89,106 +90,35 @@
|
||||
{% if replies %}
|
||||
<div class="webmention-group webmention-replies">
|
||||
<h3>{{ replies|length }} {{ 'Reply' if replies|length == 1 else 'Replies' }}</h3>
|
||||
{% for reply in replies %}
|
||||
<article class="webmention webmention-reply h-cite">
|
||||
<div class="webmention-author">
|
||||
{% if reply.author_photo %}
|
||||
<img src="{{ reply.author_photo }}" alt="" width="48" height="48" loading="lazy">
|
||||
{% endif %}
|
||||
<a class="p-author" href="{{ reply.author_url or reply.source_url }}">
|
||||
{{ reply.author_name or 'Anonymous' }}
|
||||
</a>
|
||||
</div>
|
||||
{% if reply.content_snippet %}
|
||||
<div class="p-content">{{ reply.content_snippet|safe }}</div>
|
||||
{% endif %}
|
||||
<a class="u-url webmention-date" href="{{ reply.source_url }}">
|
||||
<time class="dt-published" datetime="{{ reply.received_at.isoformat() }}">
|
||||
{{ reply.received_at.strftime('%B %d, %Y') }}
|
||||
</time>
|
||||
</a>
|
||||
</article>
|
||||
{% endfor %}
|
||||
{% for wm in replies %}{{ responses.reply(wm) }}{% endfor %}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if likes %}
|
||||
<div class="webmention-group webmention-likes">
|
||||
<h3>{{ likes|length }} {{ 'Like' if likes|length == 1 else 'Likes' }}</h3>
|
||||
<div class="facepile">
|
||||
{% for wm in likes[:8] %}
|
||||
<a href="{{ wm.author_url or wm.source_url }}" class="face" title="{{ wm.author_name or 'Anonymous' }}">
|
||||
{% if wm.author_photo %}
|
||||
<img src="{{ wm.author_photo }}" alt="{{ wm.author_name or '' }}" width="32" height="32" loading="lazy">
|
||||
{% else %}
|
||||
<span class="face-initial" style="background-color: {{ wm.author_url|url_to_color }}">
|
||||
{{ (wm.author_name or '?')[0]|upper }}
|
||||
</span>
|
||||
{% endif %}
|
||||
</a>
|
||||
{% endfor %}
|
||||
{% if likes|length > 8 %}
|
||||
<span class="face-overflow">+{{ likes|length - 8 }} others</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{{ responses.facepile(likes, 'u-like') }}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if reposts %}
|
||||
<div class="webmention-group webmention-reposts">
|
||||
<h3>{{ reposts|length }} {{ 'Repost' if reposts|length == 1 else 'Reposts' }}</h3>
|
||||
<div class="facepile">
|
||||
{% for wm in reposts[:8] %}
|
||||
<a href="{{ wm.author_url or wm.source_url }}" class="face" title="{{ wm.author_name or 'Anonymous' }}">
|
||||
{% if wm.author_photo %}
|
||||
<img src="{{ wm.author_photo }}" alt="{{ wm.author_name or '' }}" width="32" height="32" loading="lazy">
|
||||
{% else %}
|
||||
<span class="face-initial" style="background-color: {{ wm.author_url|url_to_color }}">
|
||||
{{ (wm.author_name or '?')[0]|upper }}
|
||||
</span>
|
||||
{% endif %}
|
||||
</a>
|
||||
{% endfor %}
|
||||
{% if reposts|length > 8 %}
|
||||
<span class="face-overflow">+{{ reposts|length - 8 }} others</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{{ responses.facepile(reposts, 'u-repost') }}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if bookmarks %}
|
||||
<div class="webmention-group webmention-bookmarks">
|
||||
<h3>{{ bookmarks|length }} {{ 'Bookmark' if bookmarks|length == 1 else 'Bookmarks' }}</h3>
|
||||
<div class="facepile">
|
||||
{% for wm in bookmarks[:8] %}
|
||||
<a href="{{ wm.author_url or wm.source_url }}" class="face" title="{{ wm.author_name or 'Anonymous' }}">
|
||||
{% if wm.author_photo %}
|
||||
<img src="{{ wm.author_photo }}" alt="{{ wm.author_name or '' }}" width="32" height="32" loading="lazy">
|
||||
{% else %}
|
||||
<span class="face-initial" style="background-color: {{ wm.author_url|url_to_color }}">
|
||||
{{ (wm.author_name or '?')[0]|upper }}
|
||||
</span>
|
||||
{% endif %}
|
||||
</a>
|
||||
{% endfor %}
|
||||
{% if bookmarks|length > 8 %}
|
||||
<span class="face-overflow">+{{ bookmarks|length - 8 }} others</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{{ responses.facepile(bookmarks, 'u-bookmark') }}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
{% if mentions %}
|
||||
<div class="webmention-group webmention-mentions">
|
||||
<h3>{{ mentions|length }} {{ 'Mention' if mentions|length == 1 else 'Mentions' }}</h3>
|
||||
{% for mention in mentions %}
|
||||
<div class="webmention webmention-mention">
|
||||
<a href="{{ mention.source_url }}">{{ mention.author_name or mention.source_url }}</a>
|
||||
{% if mention.content_snippet %}
|
||||
<p>{{ mention.content_snippet|striptags|truncate(200) }}</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endfor %}
|
||||
{% for wm in mentions %}{{ responses.mention(wm) }}{% endfor %}
|
||||
</div>
|
||||
{% endif %}
|
||||
</section>
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
{% from "partials/reaction.html" import reaction %}
|
||||
{% macro note_preview(note, author) %}
|
||||
<article class="h-entry note-preview">
|
||||
{# Notes starting with a # heading have a title (Q22) #}
|
||||
{% set has_explicit_title = note.content.strip().startswith('#') %}
|
||||
{# Notes starting with a '# Heading' have a title (Q22; Note.has_title) #}
|
||||
{% set has_explicit_title = note.has_title %}
|
||||
{% if has_explicit_title %}
|
||||
<h2 class="p-name"><a href="{{ absolute_url_for('public.note', slug=note.slug) }}">{{ note.title }}</a></h2>
|
||||
{% endif %}
|
||||
@@ -31,7 +31,7 @@
|
||||
{% endif %}
|
||||
|
||||
<footer class="note-meta">
|
||||
<a class="u-url" href="{{ absolute_url_for('public.note', slug=note.slug) }}"><time class="dt-published" datetime="{{ note.created_at.isoformat() }}">{{ note.created_at.strftime('%b %-d, %Y') }}</time></a>
|
||||
<a class="u-url u-uid" href="{{ absolute_url_for('public.note', slug=note.slug) }}"><time class="dt-published" datetime="{{ note.created_at|iso_utc }}">{{ note.created_at.strftime('%b %-d, %Y') }}</time></a>
|
||||
{% if note.tags %}
|
||||
<span class="note-tags">
|
||||
{% for tag in note.tags %}<a class="p-category" rel="tag" href="{{ url_for('public.tag', tag=tag.name) }}">{{ tag.display_name }}</a>{% endfor %}
|
||||
|
||||
@@ -4,8 +4,8 @@
|
||||
{% set ctx = note.context if note.context is defined else None %}
|
||||
{% if ctx and ctx.title %}{% set label = ctx.title %}{% elif ctx and ctx.content_snippet %}{% set label = ctx.content_snippet|striptags|truncate(90) %}{% elif ctx and (ctx.author_name or ctx.site_name) %}{% set label = 'a post' %}{% else %}{% set label = note.reacts_to %}{% endif %}
|
||||
<p class="reaction reaction-{{ note.kind }}">
|
||||
<span class="reaction-verb">{% if note.kind == 'like' %}★ Liked{% else %}🔁 Reposted{% endif %}</span>
|
||||
<a class="{{ 'u-like-of' if note.kind == 'like' else 'u-repost-of' }}" href="{{ note.reacts_to }}">{{ label }}</a>
|
||||
<span class="reaction-verb">{{ {'like': '★ Liked', 'repost': '🔁 Reposted', 'bookmark': '🔖 Bookmarked'}[note.kind] }}</span>
|
||||
<a class="u-{{ note.kind }}-of" href="{{ note.reacts_to }}">{{ label }}</a>
|
||||
{% if ctx and (ctx.author_name or ctx.site_name) %}<span class="reaction-by">by {{ ctx.author_name or ctx.site_name }}</span>{% endif %}
|
||||
</p>
|
||||
{% endif %}
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
{# Received webmentions as microformats2 responses (v2.8.0)
|
||||
Each response is an h-cite in a property of the note's h-entry:
|
||||
u-comment (replies), u-like, u-repost, u-bookmark, u-mention.
|
||||
See https://indieweb.org/comments-presentation and microformats.org/wiki/h-entry #}
|
||||
|
||||
{# When the source said when it was posted, use that; else when we received it #}
|
||||
{% macro response_time(wm, fmt='%B %-d, %Y') %}
|
||||
{% set when = wm.source_published or wm.received_at %}
|
||||
<time class="dt-published" datetime="{{ when|iso_utc }}">{{ when.strftime(fmt) if when.strftime is defined else when }}</time>
|
||||
{%- endmacro %}
|
||||
|
||||
{# The response's own URL: the source's u-url if it gave one #}
|
||||
{% macro response_url(wm) %}{{ wm.source_post_url or wm.source_url }}{% endmacro %}
|
||||
|
||||
{# Author h-card: name, URL and photo. u-url is on an inner link: on the
|
||||
h-card element itself it would belong to the enclosing h-cite #}
|
||||
{% macro author_card(wm, class='', photo_size=48, show_name=True) %}
|
||||
<span class="p-author h-card {{ class }}"><a class="u-url" href="{{ wm.author_url or wm.source_url }}" title="{{ wm.author_name or 'Anonymous' }}">
|
||||
{%- if wm.author_photo %}<img class="u-photo" src="{{ wm.author_photo }}" alt="" width="{{ photo_size }}" height="{{ photo_size }}" loading="lazy">
|
||||
{%- elif not show_name %}<span class="face-initial" style="background-color: {{ wm.author_url|url_to_color }}" aria-hidden="true">{{ (wm.author_name or '?')[0]|upper }}</span>
|
||||
{%- endif %}
|
||||
<span class="p-name{{ '' if show_name else ' visually-hidden' }}">{{ wm.author_name or 'Anonymous' }}</span>
|
||||
</a></span>
|
||||
{%- endmacro %}
|
||||
|
||||
{# A reply, shown as a comment #}
|
||||
{% macro reply(wm) %}
|
||||
<article class="webmention webmention-reply u-comment h-cite">
|
||||
<div class="webmention-author">{{ author_card(wm) }}</div>
|
||||
{% if wm.content_snippet %}
|
||||
<div class="e-content">{{ wm.content_snippet|safe }}</div>
|
||||
{% endif %}
|
||||
<a class="u-url webmention-date" href="{{ response_url(wm) }}">{{ response_time(wm) }}</a>
|
||||
</article>
|
||||
{%- endmacro %}
|
||||
|
||||
{# Likes, reposts or bookmarks as a facepile; prop is u-like / u-repost / u-bookmark.
|
||||
Faces past `shown` are hidden on screen but kept for parsers. #}
|
||||
{% macro facepile(items, prop, shown=8) %}
|
||||
<div class="facepile">
|
||||
{% for wm in items %}
|
||||
<span class="face {{ prop }} h-cite"{% if loop.index > shown %} hidden{% endif %}>
|
||||
{{ author_card(wm, photo_size=32, show_name=False) }}
|
||||
<a class="u-url" href="{{ response_url(wm) }}" hidden>{{ response_time(wm) }}</a>
|
||||
</span>
|
||||
{% endfor %}
|
||||
{% if items|length > shown %}
|
||||
<span class="face-overflow">+{{ items|length - shown }} others</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{%- endmacro %}
|
||||
|
||||
{# A post that mentions this note without replying #}
|
||||
{% macro mention(wm) %}
|
||||
<div class="webmention webmention-mention u-mention h-cite">
|
||||
{{ author_card(wm, photo_size=24) }}
|
||||
{% if wm.content_snippet %}
|
||||
<p class="p-content">{{ wm.content_snippet|striptags|truncate(200) }}</p>
|
||||
{% endif %}
|
||||
<a class="u-url webmention-date" href="{{ response_url(wm) }}">{{ response_time(wm) }}</a>
|
||||
</div>
|
||||
{%- endmacro %}
|
||||
@@ -7,6 +7,12 @@
|
||||
<div class="h-feed">
|
||||
<h1 class="p-name page-title">Notes tagged <span class="tag-name">{{ tag.display_name }}</span></h1>
|
||||
|
||||
{# Feed URL and author, as on the home page (v2.8.0) #}
|
||||
<a class="u-url" href="{{ absolute_url_for(request.endpoint, **request.view_args) }}" hidden>Feed URL</a>
|
||||
{% if author %}
|
||||
<span class="p-author h-card" hidden><a class="p-name u-url" href="{{ author.url or author.me }}">{{ author.name or author.url or author.me }}</a></span>
|
||||
{% endif %}
|
||||
|
||||
{% if notes %}
|
||||
{% for note in notes %}
|
||||
{{ note_preview(note, author) }}
|
||||
|
||||
@@ -114,6 +114,11 @@ def post(app, monkeypatch):
|
||||
client = app.test_client()
|
||||
|
||||
def _post(**props):
|
||||
# Crosspost to every configured target; absent means none (v2.9.0).
|
||||
# Sent inside properties, as JSON clients like Quill do.
|
||||
with app.app_context():
|
||||
from starpunk.syndication import get_targets
|
||||
props["mp-syndicate-to"] = [t.uid for t in get_targets().values()]
|
||||
response = client.post("/micropub", json={"type": ["h-entry"], "properties": props},
|
||||
headers={"Authorization": f"Bearer {token}"})
|
||||
assert response.status_code == 201, response.get_json()
|
||||
|
||||
@@ -168,7 +168,7 @@ class TestImageValidation:
|
||||
with pytest.raises(ValueError) as exc_info:
|
||||
validate_image(corrupted_data, 'corrupt.jpg')
|
||||
|
||||
assert "Invalid or corrupted" in str(exc_info.value)
|
||||
assert "Not a supported image" in str(exc_info.value)
|
||||
|
||||
|
||||
class TestHEICSupport:
|
||||
|
||||
@@ -289,16 +289,20 @@ class TestRelMe:
|
||||
assert 'https://github.com/testuser' in rels['me']
|
||||
assert 'https://mastodon.social/@testuser' in rels['me']
|
||||
|
||||
def test_no_relme_without_author(self, client, app):
|
||||
"""No rel=me links if no author profile"""
|
||||
def test_relme_to_admin_me_without_author(self, client, app):
|
||||
"""Without a discovered profile, rel=me and rel=author still point at ADMIN_ME (v2.8.0)
|
||||
|
||||
The home page's p-author h-card links to the author's site; rel=me to
|
||||
the same URL makes it the representative h-card.
|
||||
"""
|
||||
with patch('starpunk.author_discovery.get_author_profile', return_value=None):
|
||||
response = client.get('/')
|
||||
|
||||
parsed = mf2py.parse(doc=response.data.decode(), url='http://localhost/')
|
||||
|
||||
rels = parsed.get('rels', {})
|
||||
# Should not have rel=me, or it should be empty
|
||||
assert len(rels.get('me', [])) == 0, "Should not have rel=me without author"
|
||||
assert rels.get('me') == [app.config['ADMIN_ME']]
|
||||
assert rels.get('author') == [app.config['ADMIN_ME']]
|
||||
|
||||
|
||||
class TestV130Microformats:
|
||||
|
||||
@@ -0,0 +1,268 @@
|
||||
"""
|
||||
Microformats2 fixes (v2.8.0)
|
||||
|
||||
Pages are rendered and parsed with mf2py; assertions are on the parsed data,
|
||||
which is what IndieWeb readers and Webmention receivers see.
|
||||
"""
|
||||
|
||||
import re
|
||||
import sqlite3
|
||||
from unittest.mock import patch
|
||||
|
||||
import mf2py
|
||||
import pytest
|
||||
|
||||
from starpunk import mf2_utils
|
||||
from starpunk.models import split_title
|
||||
from starpunk.webmention_processor import extract_author, extract_source_details
|
||||
from starpunk.webmention_worker import WebmentionWorker
|
||||
|
||||
BASE = 'http://localhost:5000'
|
||||
|
||||
|
||||
def parse(client, path):
|
||||
response = client.get(path)
|
||||
assert response.status_code == 200
|
||||
return mf2py.parse(doc=response.data.decode(), url=BASE + path)
|
||||
|
||||
|
||||
def entry_of(parsed):
|
||||
return next(i for i in parsed['items'] if 'h-entry' in i['type'])
|
||||
|
||||
|
||||
def make_note(app, content, **kw):
|
||||
from starpunk.notes import create_note
|
||||
with app.test_request_context():
|
||||
return create_note(content, published=True, **kw)
|
||||
|
||||
|
||||
def add_mention(app, note, kind, n, photo=True, **cols):
|
||||
db = sqlite3.connect(app.config['DATABASE_PATH'])
|
||||
fields = {
|
||||
'source_url': f'https://src.example/{kind}/{n}',
|
||||
'target_url': f'{BASE}/note/{note.slug}',
|
||||
'target_note_id': note.id,
|
||||
'mention_type': kind,
|
||||
'author_name': f'{kind.title()} Person {n}',
|
||||
'author_url': f'https://{kind}{n}.example/',
|
||||
'author_photo': f'https://{kind}{n}.example/me.jpg' if photo else None,
|
||||
'content_snippet': f'<p>{kind} text <a href="https://x.example/">link</a></p>',
|
||||
'validation_status': 'valid',
|
||||
'moderation_status': 'approved',
|
||||
**cols,
|
||||
}
|
||||
db.execute(f"INSERT INTO webmentions ({', '.join(fields)}) VALUES ({', '.join('?' * len(fields))})",
|
||||
tuple(fields.values()))
|
||||
db.commit()
|
||||
db.close()
|
||||
|
||||
|
||||
# A1/A2: received responses are properties of the entry
|
||||
|
||||
def test_responses_are_entry_properties(app, client):
|
||||
note = make_note(app, "A note")
|
||||
add_mention(app, note, 'reply', 1, source_post_url='https://src.example/reply/1#canonical',
|
||||
source_published='2026-11-01T10:00:00Z')
|
||||
for n in range(10):
|
||||
add_mention(app, note, 'like', n, photo=n % 2 == 0)
|
||||
add_mention(app, note, 'repost', 1)
|
||||
add_mention(app, note, 'bookmark', 1)
|
||||
add_mention(app, note, 'mention', 1)
|
||||
|
||||
props = entry_of(parse(client, f'/note/{note.slug}'))['properties']
|
||||
assert 'children' not in entry_of(parse(client, f'/note/{note.slug}'))
|
||||
|
||||
(comment,) = props['comment']
|
||||
assert comment['type'] == ['h-cite']
|
||||
author = comment['properties']['author'][0]
|
||||
assert author['type'] == ['h-card']
|
||||
assert author['properties']['name'] == ['Reply Person 1']
|
||||
assert author['properties']['url'] == ['https://reply1.example/']
|
||||
assert author['properties']['photo'][0] in ('https://reply1.example/me.jpg',
|
||||
{'value': 'https://reply1.example/me.jpg', 'alt': ''})
|
||||
assert '<a href="https://x.example/"' in comment['properties']['content'][0]['html']
|
||||
# the source's own URL and published time, not ours
|
||||
assert comment['properties']['url'] == ['https://src.example/reply/1#canonical']
|
||||
assert comment['properties']['published'] == ['2026-11-01T10:00:00Z']
|
||||
|
||||
# all ten likes, including the ones past the visible eight
|
||||
assert len(props['like']) == 10
|
||||
names = {like['properties']['author'][0]['properties']['name'][0] for like in props['like']}
|
||||
assert names == {f'Like Person {n}' for n in range(10)}
|
||||
assert props['like'][0]['properties']['url'] == ['https://src.example/like/0']
|
||||
|
||||
assert props['repost'][0]['properties']['author'][0]['properties']['name'] == ['Repost Person 1']
|
||||
assert props['bookmark'][0]['type'] == ['h-cite']
|
||||
assert props['mention'][0]['properties']['author'][0]['properties']['url'] == ['https://mention1.example/']
|
||||
|
||||
|
||||
# A3: dt-* are RFC 3339 UTC
|
||||
|
||||
UTC = re.compile(r'^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ$')
|
||||
|
||||
|
||||
def test_dates_are_utc(app, client):
|
||||
from starpunk.notes import update_note
|
||||
note = make_note(app, "Dated")
|
||||
with app.test_request_context():
|
||||
update_note(id=note.id, content="Dated, edited")
|
||||
add_mention(app, note, 'reply', 1)
|
||||
|
||||
props = entry_of(parse(client, f'/note/{note.slug}'))['properties']
|
||||
assert UTC.match(props['published'][0]), props['published']
|
||||
assert UTC.match(props['updated'][0]), props['updated']
|
||||
assert UTC.match(props['comment'][0]['properties']['published'][0])
|
||||
|
||||
home = parse(client, '/')['items'][0]['children'][0]['properties']
|
||||
assert UTC.match(home['published'][0])
|
||||
|
||||
|
||||
def test_micropub_source_published_is_utc():
|
||||
from datetime import datetime, timezone, timedelta
|
||||
from starpunk.utils import iso_utc
|
||||
assert iso_utc(datetime(2026, 11, 1, 16, 25, 12, 921768)) == '2026-11-01T16:25:12Z'
|
||||
assert iso_utc(datetime(2026, 11, 1, 9, 0, tzinfo=timezone(timedelta(hours=-7)))) == '2026-11-01T16:00:00Z'
|
||||
assert iso_utc('2026-11-01T16:25:15+0000') == '2026-11-01T16:25:15Z'
|
||||
|
||||
|
||||
# A4: only a real Markdown heading is a title
|
||||
|
||||
@pytest.mark.parametrize("content,title", [
|
||||
("# Title\n\nBody", "Title"),
|
||||
("## Sub title ##\n\nBody", "Sub title"),
|
||||
("#indieweb is great", None),
|
||||
("#\n\nbody text", None),
|
||||
("####### seven", None),
|
||||
("Plain note", None),
|
||||
])
|
||||
def test_split_title(content, title):
|
||||
assert split_title(content)[0] == title
|
||||
|
||||
|
||||
@pytest.mark.parametrize("content,kind", [
|
||||
("#indieweb is great, posting from my phone", "note"),
|
||||
("#\n\nbody text", "note"),
|
||||
("# A Title\n\nThe body", "article"),
|
||||
])
|
||||
def test_post_type_of_hash_notes(app, client, content, kind):
|
||||
note = make_note(app, content)
|
||||
for path in (f'/note/{note.slug}', '/'):
|
||||
parsed = parse(client, path)
|
||||
entry = entry_of(parsed) if path != '/' else parsed['items'][0]['children'][0]
|
||||
props = entry['properties']
|
||||
if kind == 'note':
|
||||
assert 'name' not in props
|
||||
assert props['content'][0]['value'].strip()
|
||||
else:
|
||||
assert props['name'] == ['A Title']
|
||||
|
||||
|
||||
# A5: the note's own syndicated copy is not a response
|
||||
|
||||
def test_own_syndicated_copy_rejected(app):
|
||||
note = make_note(app, "Crossposted")
|
||||
db = sqlite3.connect(app.config['DATABASE_PATH'])
|
||||
db.execute("INSERT INTO syndications (note_id, target, remote_url) VALUES (?, 'gotosocial', ?)",
|
||||
(note.id, 'https://social.example/@me/statuses/1'))
|
||||
db.execute("INSERT INTO webmentions (source_url, target_url, target_note_id) VALUES (?, ?, ?)",
|
||||
('https://brid.gy/post/mastodon/@someone/1', f'{BASE}/note/{note.slug}', note.id))
|
||||
db.commit()
|
||||
|
||||
result = {'valid': True, 'mention_type': 'mention', 'author_name': 'Me', 'author_url': 'https://me.example/',
|
||||
'author_photo': None, 'content_snippet': 'x',
|
||||
'source_post_url': 'https://social.example/@me/statuses/1', 'source_uid': None,
|
||||
'source_published': None}
|
||||
worker = WebmentionWorker(app, db_path=app.config['DATABASE_PATH'], site_url=BASE)
|
||||
with patch("starpunk.webmention_worker.process_webmention", return_value=result):
|
||||
worker._validate_webmention(1, 'https://brid.gy/post/mastodon/@someone/1', 'x', note.id)
|
||||
|
||||
status, error = db.execute("SELECT validation_status, validation_error FROM webmentions").fetchone()
|
||||
assert status == 'invalid' and 'own syndicated copy' in error
|
||||
|
||||
|
||||
# B1/B3: discovery links
|
||||
|
||||
def test_head_links_without_webmention(app, client):
|
||||
app.config['WEBMENTION_ENABLED'] = False
|
||||
rels = parse(client, '/')['rels']
|
||||
assert 'webmention' not in rels
|
||||
for rel in ('micropub', 'indieauth-metadata', 'authorization_endpoint', 'token_endpoint'):
|
||||
assert rels.get(rel), rel
|
||||
|
||||
|
||||
def test_home_has_representative_hcard(app, client):
|
||||
# Fresh install: discovery fails, the fallback profile is used
|
||||
parsed = parse(client, '/')
|
||||
card = mf2_utils.representative_hcard(parsed, BASE + '/')
|
||||
assert card and card['properties']['url'] == [app.config['ADMIN_ME']]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("path", ['/likes', '/tag/python'])
|
||||
def test_list_feeds_have_url_and_author(app, client, path):
|
||||
make_note(app, "Tagged", tags=["python"])
|
||||
feed = parse(client, path)['items'][0]
|
||||
assert feed['type'] == ['h-feed']
|
||||
assert feed['properties']['url'] == [BASE + path]
|
||||
assert feed['properties']['author'][0]['type'] == ['h-card']
|
||||
|
||||
|
||||
# B2: representative h-card
|
||||
|
||||
def _cards(html, url='https://me.example/'):
|
||||
return mf2py.parse(doc=html, url=url)
|
||||
|
||||
|
||||
def test_representative_hcard_rules():
|
||||
other = '<div class="h-card"><a class="p-name u-url" href="https://bob.example/">Bob</a></div>'
|
||||
# someone else's h-card only: none
|
||||
assert mf2_utils.representative_hcard(_cards(other), 'https://me.example/') is None
|
||||
# uid + url
|
||||
me = '<div class="h-card"><a class="p-name u-url u-uid" href="/">Me</a></div>'
|
||||
assert mf2_utils.first(mf2_utils.representative_hcard(_cards(other + me), 'https://me.example/'), 'name') == 'Me'
|
||||
# rel=me
|
||||
relme = ('<a rel="me" href="https://social.example/@me">x</a>'
|
||||
'<div class="h-card"><a class="p-name u-url" href="https://social.example/@me">Me</a></div>')
|
||||
assert mf2_utils.first(mf2_utils.representative_hcard(_cards(other + relme), 'https://me.example/'), 'name') == 'Me'
|
||||
# only one h-card, url = page
|
||||
single = '<div class="h-card"><a class="p-name u-url" href="/">Me</a></div>'
|
||||
assert mf2_utils.first(mf2_utils.representative_hcard(_cards(single), 'https://me.example/'), 'name') == 'Me'
|
||||
|
||||
|
||||
# B4: the responding entry and its author
|
||||
|
||||
FEED_PAGE = '''
|
||||
<div class="h-feed">
|
||||
<div class="p-author h-card"><a class="p-name u-url" href="https://feed.example/">Feed Owner</a></div>
|
||||
<div class="h-entry"><a class="u-url" href="https://feed.example/1">one</a>
|
||||
<div class="e-content">unrelated</div></div>
|
||||
<div class="h-entry"><a class="u-url" href="https://feed.example/2">two</a>
|
||||
<time class="dt-published" datetime="2026-11-01T09:00:00-07:00">x</time>
|
||||
<a class="u-in-reply-to" href="https://me.example/note/a">re</a>
|
||||
<div class="e-content">my reply</div></div>
|
||||
</div>'''
|
||||
|
||||
|
||||
def test_responding_entry_and_feed_author():
|
||||
parsed = mf2py.parse(doc=FEED_PAGE, url='https://feed.example/')
|
||||
entry = mf2_utils.responding_entry(parsed, 'https://me.example/note/a#frag')
|
||||
assert mf2_utils.first(entry, 'url') == 'https://feed.example/2'
|
||||
assert mf2_utils.response_type(entry, 'https://me.example/note/a') == 'reply'
|
||||
|
||||
author = extract_author(parsed, 'https://feed.example/', entry)
|
||||
assert author['author_name'] == 'Feed Owner'
|
||||
assert author['author_url'] == 'https://feed.example/'
|
||||
|
||||
details = extract_source_details(entry)
|
||||
assert details['source_post_url'] == 'https://feed.example/2'
|
||||
assert details['source_published'] == '2026-11-01T16:00:00Z'
|
||||
|
||||
|
||||
def test_author_as_url_or_name():
|
||||
url_author = '<div class="h-entry"><a class="u-author" href="https://al.example/">al</a></div>'
|
||||
parsed = mf2py.parse(doc=url_author, url='https://s.example/')
|
||||
assert extract_author(parsed, 'https://s.example/')['author_url'] == 'https://al.example/'
|
||||
|
||||
name_author = '<div class="h-entry"><span class="p-author">Al Smith</span></div>'
|
||||
parsed = mf2py.parse(doc=name_author, url='https://s.example/')
|
||||
author = extract_author(parsed, 'https://s.example/')
|
||||
assert author['author_name'] == 'Al Smith' and author['author_url'] == 'https://s.example/'
|
||||
@@ -87,8 +87,11 @@ class TestCreate:
|
||||
assert "photo" in source(client, h, response.headers["Location"])["properties"]
|
||||
|
||||
def test_remote_photo_is_downloaded(self, app, client, h, monkeypatch):
|
||||
transport = httpx.MockTransport(lambda req: httpx.Response(200, content=jpeg("blue")))
|
||||
monkeypatch.setattr(httpx, "stream", lambda method, url, **kw: httpx.Client(transport=transport).stream(method, url))
|
||||
# Fetched through the SSRF-checked, address-pinned fetcher (v2.9.0)
|
||||
monkeypatch.setattr("starpunk.webmention_ssrf.TRANSPORT",
|
||||
httpx.MockTransport(lambda req: httpx.Response(200, content=jpeg("blue"))))
|
||||
monkeypatch.setattr("starpunk.webmention_ssrf.socket.getaddrinfo",
|
||||
lambda *a, **k: [(2, 1, 6, "", ("93.184.216.34", 0))])
|
||||
url = create(client, h, photo=[{"value": "https://photos.example/cat.jpg", "alt": "A cat"}])
|
||||
(photo,) = source(client, h, url)["properties"]["photo"]
|
||||
assert photo["value"].startswith(f"{SITE}/media/") and photo["alt"] == "A cat"
|
||||
|
||||
@@ -0,0 +1,249 @@
|
||||
"""
|
||||
Micropub compliance fixes (v2.9.0)
|
||||
|
||||
Section numbers refer to the W3C Micropub Recommendation.
|
||||
"""
|
||||
|
||||
import io
|
||||
import sqlite3
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from tests.test_micropub_v240 import SITE, app, auth, client, create, h, jpeg, note_at, source # noqa: F401
|
||||
|
||||
|
||||
def syndications(app, url):
|
||||
note = note_at(app, url)
|
||||
db = sqlite3.connect(app.config["DATABASE_PATH"])
|
||||
return sorted(r[0] for r in db.execute("SELECT target FROM syndications WHERE note_id = ?", (note.id,)))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def targets(app):
|
||||
app.config.update(BLUESKY_HANDLE="me.example", BLUESKY_APP_PASSWORD="p")
|
||||
from starpunk import syndication
|
||||
syndication._client_cache.clear()
|
||||
with app.app_context():
|
||||
return {t.uid: name for name, t in syndication.get_targets().items()}
|
||||
|
||||
|
||||
# Crossposting: mp-* in JSON properties; absent means none
|
||||
|
||||
def test_mp_syndicate_to_inside_json_properties(app, client, h, targets):
|
||||
bsky_uid = next(uid for uid, name in targets.items() if name == "bluesky")
|
||||
url = create(client, h, **{"mp-syndicate-to": [bsky_uid]})
|
||||
assert syndications(app, url) == ["bluesky"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("extra", [{}, {"mp-syndicate-to": []}])
|
||||
def test_no_choice_means_no_crossposting(app, client, h, targets, extra):
|
||||
url = create(client, h, **extra)
|
||||
assert syndications(app, url) == []
|
||||
|
||||
|
||||
def test_form_mp_syndicate_to(app, client, h, targets):
|
||||
bsky_uid = next(uid for uid, name in targets.items() if name == "bluesky")
|
||||
response = client.post("/micropub", data={"content": "x", "mp-syndicate-to[]": bsky_uid}, headers=h)
|
||||
assert syndications(app, response.headers["Location"]) == ["bluesky"]
|
||||
|
||||
|
||||
def test_mp_slug_inside_json_properties(app, client, h):
|
||||
url = create(client, h, **{"mp-slug": ["my-slug"]})
|
||||
assert url.endswith("/note/my-slug")
|
||||
assert "mp-slug" not in source(client, h, url)["properties"]
|
||||
|
||||
|
||||
# Scopes
|
||||
|
||||
def test_source_needs_a_posting_scope(app, client, h):
|
||||
create(client, h, **{"post-status": ["draft"]})
|
||||
profile_only = auth(app, "profile")
|
||||
assert client.get("/micropub?q=source", headers=profile_only).status_code == 403
|
||||
assert client.get("/micropub?q=category", headers=profile_only).status_code == 403
|
||||
assert client.get("/micropub?q=config", headers=profile_only).status_code == 200
|
||||
|
||||
|
||||
def test_draft_scope_only_creates_drafts(app, client):
|
||||
url = create(client, auth(app, "draft"))
|
||||
assert note_at(app, url).published is False
|
||||
|
||||
|
||||
# Visibility
|
||||
|
||||
@pytest.mark.parametrize("visibility,published", [("public", True), ("unlisted", False), ("private", False)])
|
||||
def test_visibility(app, client, h, visibility, published):
|
||||
url = create(client, h, visibility=[visibility])
|
||||
assert note_at(app, url).published is published
|
||||
|
||||
|
||||
def test_unknown_visibility_rejected(client, h):
|
||||
body = {"type": ["h-entry"], "properties": {"content": ["x"], "visibility": ["friends"]}}
|
||||
assert client.post("/micropub", json=body, headers=h).status_code == 400
|
||||
|
||||
|
||||
# Photos that can't be used fail the request
|
||||
|
||||
def test_unreachable_photo_is_an_error(app, client, h, monkeypatch):
|
||||
monkeypatch.setattr("starpunk.webmention_ssrf.TRANSPORT", httpx.MockTransport(lambda r: httpx.Response(404)))
|
||||
monkeypatch.setattr("starpunk.webmention_ssrf.socket.getaddrinfo",
|
||||
lambda *a, **k: [(2, 1, 6, "", ("93.184.216.34", 0))])
|
||||
body = {"type": ["h-entry"], "properties": {"content": ["x"], "photo": ["https://p.example/404.jpg"]}}
|
||||
assert client.post("/micropub", json=body, headers=h).status_code == 400
|
||||
assert client.get("/micropub?q=source", headers=h).get_json()["items"] == []
|
||||
|
||||
|
||||
def test_unknown_local_media_is_an_error_on_update(app, client, h):
|
||||
url = create(client, h)
|
||||
response = client.post("/micropub", json={"action": "update", "url": url,
|
||||
"replace": {"photo": [f"{SITE}/media/2026/11/nope.jpg"]}}, headers=h)
|
||||
assert response.status_code == 400
|
||||
|
||||
|
||||
def test_redirect_to_private_address_is_not_fetched(app, client, h, monkeypatch):
|
||||
def resolve(host, *a, **k):
|
||||
return [(2, 1, 6, "", ("10.0.0.5" if host == "internal.example" else "93.184.216.34", 0))]
|
||||
|
||||
def site(request):
|
||||
return httpx.Response(302, headers={"Location": "http://internal.example/secret.jpg"})
|
||||
|
||||
monkeypatch.setattr("starpunk.webmention_ssrf.socket.getaddrinfo", resolve)
|
||||
monkeypatch.setattr("starpunk.webmention_ssrf.TRANSPORT", httpx.MockTransport(site))
|
||||
body = {"type": ["h-entry"], "properties": {"content": ["x"], "photo": ["https://p.example/a.jpg"]}}
|
||||
assert client.post("/micropub", json=body, headers=h).status_code == 400
|
||||
|
||||
|
||||
# JSON values must be arrays (3.3.2)
|
||||
|
||||
def test_scalar_json_values_are_not_split(app, client, h):
|
||||
body = {"type": ["h-entry"], "properties": {"content": "whole text", "category": "solo", "name": "Title"}}
|
||||
response = client.post("/micropub", json=body, headers=h)
|
||||
props = source(client, h, response.headers["Location"])["properties"]
|
||||
assert props["content"] == ["whole text"]
|
||||
assert props["category"] == ["solo"]
|
||||
assert props["name"] == ["Title"]
|
||||
|
||||
|
||||
def test_update_delete_values_must_be_arrays(app, client, h):
|
||||
url = create(client, h, category=["a", "b"])
|
||||
response = client.post("/micropub", json={"action": "update", "url": url, "delete": {"category": "a"}}, headers=h)
|
||||
assert response.status_code == 400
|
||||
|
||||
|
||||
def test_update_needs_an_operation(app, client, h):
|
||||
url = create(client, h)
|
||||
assert client.post("/micropub", json={"action": "update", "url": url}, headers=h).status_code == 400
|
||||
|
||||
|
||||
def test_person_tag_categories_are_ignored(app, client, h):
|
||||
url = create(client, h, category=["kept", {"type": ["h-card"], "properties": {"url": ["https://a.example/"]}}])
|
||||
assert source(client, h, url)["properties"]["category"] == ["kept"]
|
||||
|
||||
|
||||
# Properties that used to be dropped
|
||||
|
||||
def test_bookmark_of(app, client, h):
|
||||
body = {"type": ["h-entry"], "properties": {"bookmark-of": ["https://read.example/post"]}}
|
||||
url = client.post("/micropub", json=body, headers=h).headers["Location"]
|
||||
note = note_at(app, url)
|
||||
assert note.kind == "bookmark" and note.bookmark_of == "https://read.example/post"
|
||||
props = source(client, h, url)["properties"]
|
||||
assert props["bookmark-of"] == ["https://read.example/post"] and "content" not in props
|
||||
page = client.get(url.replace(SITE, "")).data.decode()
|
||||
assert 'class="u-bookmark-of" href="https://read.example/post"' in page
|
||||
|
||||
|
||||
def test_summary_or_name_without_content(app, client, h):
|
||||
response = client.post("/micropub", json={"type": ["h-entry"], "properties": {"summary": ["Just a summary"]}},
|
||||
headers=h)
|
||||
assert source(client, h, response.headers["Location"])["properties"]["content"] == ["Just a summary"]
|
||||
|
||||
response = client.post("/micropub", json={"type": ["h-entry"], "properties": {"name": ["Only a title"]}},
|
||||
headers=h)
|
||||
assert source(client, h, response.headers["Location"])["properties"]["name"] == ["Only a title"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("prop", ["video", "audio"])
|
||||
def test_unsupported_media_is_refused(client, h, prop):
|
||||
body = {"type": ["h-entry"], "properties": {"content": ["x"], prop: ["https://m.example/clip"]}}
|
||||
assert client.post("/micropub", json=body, headers=h).status_code == 400
|
||||
upload = {"content": "x", prop: (io.BytesIO(b"data"), "clip.mp4")}
|
||||
assert client.post("/micropub", data=upload, headers=h, content_type="multipart/form-data").status_code == 400
|
||||
|
||||
|
||||
# HTML content (3.7.2) and sanitizing
|
||||
|
||||
def test_html_content_comes_back_as_html(app, client, h):
|
||||
url = create(client, h, content=[{"html": "<p>Some <b>bold</b> text</p>"}])
|
||||
assert source(client, h, url)["properties"]["content"] == [{"html": "<p>Some <b>bold</b> text</p>"}]
|
||||
url = create(client, h, content=["plain *markdown*"])
|
||||
assert source(client, h, url)["properties"]["content"] == ["plain *markdown*"]
|
||||
|
||||
|
||||
def test_update_with_html_content(app, client, h):
|
||||
url = create(client, h)
|
||||
client.post("/micropub", json={"action": "update", "url": url,
|
||||
"replace": {"content": [{"html": "<p>new</p>"}]}}, headers=h)
|
||||
assert source(client, h, url)["properties"]["content"] == [{"html": "<p>new</p>"}]
|
||||
|
||||
|
||||
def test_html_is_sanitized_on_the_page(app, client, h):
|
||||
url = create(client, h, content=[{"html": '<p>hi<script>alert(1)</script><img src="/x.jpg" onerror="evil()">'
|
||||
'<a href="javascript:alert(2)">j</a><iframe src="https://e.example"></iframe></p>'}])
|
||||
page = client.get(url.replace(SITE, "")).data.decode()
|
||||
for bad in ("<script>alert(1)", "onerror", "javascript:alert", "<iframe"):
|
||||
assert bad not in page
|
||||
assert '<img src="/x.jpg">' in page
|
||||
|
||||
|
||||
# published (3.3)
|
||||
|
||||
def test_published_is_stored_as_utc(app, client, h):
|
||||
url = create(client, h, published=["2020-02-03T04:05:06-08:00"])
|
||||
assert source(client, h, url)["properties"]["published"] == ["2020-02-03T12:05:06Z"]
|
||||
created = note_at(app, url).created_at
|
||||
assert created.tzinfo is None and (created.hour, created.minute) == (12, 5)
|
||||
|
||||
|
||||
def test_unparseable_published_is_an_error(client, h):
|
||||
body = {"type": ["h-entry"], "properties": {"content": ["x"], "published": ["yesterday"]}}
|
||||
assert client.post("/micropub", json=body, headers=h).status_code == 400
|
||||
|
||||
|
||||
# Errors (3.8) and authentication (5)
|
||||
|
||||
@pytest.mark.parametrize("payload", ["{not json", "[1, 2]"])
|
||||
def test_malformed_json_is_400(client, h, payload):
|
||||
response = client.post("/micropub", data=payload, headers={**h, "Content-Type": "application/json"})
|
||||
assert response.status_code == 400
|
||||
assert response.get_json()["error"] == "invalid_request"
|
||||
|
||||
|
||||
def test_bearer_scheme_is_case_insensitive(client, h):
|
||||
lower = {"Authorization": h["Authorization"].replace("Bearer", "bearer")}
|
||||
assert client.get("/micropub?q=config", headers=lower).status_code == 200
|
||||
|
||||
|
||||
def test_token_in_header_and_body_is_400(client, h):
|
||||
response = client.post("/micropub", data={"content": "x", "access_token": "other"}, headers=h)
|
||||
assert response.status_code == 400
|
||||
|
||||
|
||||
def test_401_has_www_authenticate(client):
|
||||
response = client.get("/micropub?q=config")
|
||||
assert response.status_code == 401
|
||||
assert response.headers["WWW-Authenticate"].startswith("Bearer")
|
||||
response = client.get("/micropub?q=config", headers={"Authorization": "Bearer nope"})
|
||||
assert 'error="invalid_token"' in response.headers["WWW-Authenticate"]
|
||||
|
||||
|
||||
def test_media_error_does_not_leak_internals(client, h):
|
||||
response = client.post("/micropub/media", data={"file": (io.BytesIO(b"not an image"), "a.jpg")},
|
||||
headers=h, content_type="multipart/form-data")
|
||||
assert response.status_code == 400
|
||||
assert "BytesIO" not in response.get_json()["error_description"]
|
||||
|
||||
|
||||
def test_config_lists_bookmarks(client, h):
|
||||
types = {t["type"] for t in client.get("/micropub?q=config", headers=h).get_json()["post-types"]}
|
||||
assert "bookmark" in types
|
||||
@@ -0,0 +1,86 @@
|
||||
"""
|
||||
Migrations under concurrent gunicorn workers (v2.8.1)
|
||||
|
||||
The v2.8.0 deploy failed to boot: one worker applied migration 014 while
|
||||
another applied it again ("duplicate column name"), because
|
||||
conn.executescript() committed and released the migration lock early.
|
||||
"""
|
||||
|
||||
import multiprocessing
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
from starpunk.database.schema import INITIAL_SCHEMA_SQL
|
||||
from starpunk.migrations import execute_in_transaction, run_migrations, split_sql_statements
|
||||
|
||||
MIGRATIONS = Path(__file__).parent.parent / "migrations"
|
||||
|
||||
|
||||
def test_script_runs_inside_the_transaction(tmp_path):
|
||||
conn = sqlite3.connect(tmp_path / "t.db")
|
||||
conn.execute("CREATE TABLE t (a INTEGER)")
|
||||
conn.commit()
|
||||
|
||||
conn.execute("BEGIN IMMEDIATE")
|
||||
execute_in_transaction(conn, "ALTER TABLE t ADD COLUMN b TEXT;\nALTER TABLE t ADD COLUMN c TEXT;")
|
||||
conn.rollback()
|
||||
|
||||
assert [row[1] for row in conn.execute("PRAGMA table_info(t)")] == ["a"]
|
||||
|
||||
|
||||
def test_split_keeps_trigger_bodies_together():
|
||||
script = """
|
||||
-- comment; with a semicolon
|
||||
CREATE TABLE t (a TEXT DEFAULT 'x;y');
|
||||
CREATE TRIGGER tr AFTER DELETE ON t BEGIN
|
||||
DELETE FROM t WHERE a = old.a;
|
||||
SELECT 1;
|
||||
END;
|
||||
"""
|
||||
statements = split_sql_statements(script)
|
||||
assert len(statements) == 2
|
||||
assert statements[1].startswith("CREATE TRIGGER") and statements[1].endswith("END;")
|
||||
|
||||
|
||||
def _migrate(db_path, barrier, errors):
|
||||
try:
|
||||
barrier.wait()
|
||||
run_migrations(db_path)
|
||||
except Exception as e: # report to the parent
|
||||
errors.put(repr(e))
|
||||
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.parametrize("trial", range(5)) # the race hit ~1 in 5 runs
|
||||
def test_concurrent_workers_apply_each_migration_once(tmp_path, trial):
|
||||
# A database at migration 013, like production before v2.8.0
|
||||
db_path = tmp_path / "starpunk.db"
|
||||
conn = sqlite3.connect(db_path)
|
||||
conn.executescript(INITIAL_SCHEMA_SQL) # as init_db does
|
||||
conn.close()
|
||||
run_migrations(db_path)
|
||||
conn = sqlite3.connect(db_path)
|
||||
conn.execute("DELETE FROM schema_migrations WHERE migration_name = '014_webmention_source_details.sql'")
|
||||
conn.commit()
|
||||
columns = [row[1] for row in conn.execute("PRAGMA table_info(webmentions)")]
|
||||
if "source_post_url" in columns:
|
||||
conn.execute("ALTER TABLE webmentions DROP COLUMN source_post_url")
|
||||
conn.execute("ALTER TABLE webmentions DROP COLUMN source_published")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
ctx = multiprocessing.get_context("fork")
|
||||
barrier, errors = ctx.Barrier(4), ctx.Queue()
|
||||
workers = [ctx.Process(target=_migrate, args=(db_path, barrier, errors)) for _ in range(4)]
|
||||
for w in workers:
|
||||
w.start()
|
||||
for w in workers:
|
||||
w.join(timeout=60)
|
||||
|
||||
assert errors.empty(), errors.get()
|
||||
conn = sqlite3.connect(db_path)
|
||||
assert conn.execute(
|
||||
"SELECT COUNT(*) FROM schema_migrations WHERE migration_name = '014_webmention_source_details.sql'"
|
||||
).fetchone()[0] == 1
|
||||
@@ -302,12 +302,13 @@ class TestErrorHandling:
|
||||
|
||||
def test_rollback_failure_causes_system_exit(self, temp_db):
|
||||
"""Test that rollback failure raises SystemExit"""
|
||||
with patch('sqlite3.connect') as mock_connect:
|
||||
with patch('sqlite3.connect') as mock_connect, \
|
||||
patch('starpunk.migrations.execute_in_transaction',
|
||||
side_effect=Exception("Migration failed")):
|
||||
mock_conn = MagicMock()
|
||||
mock_connect.return_value = mock_conn
|
||||
|
||||
# Make both migration and rollback fail
|
||||
mock_conn.executescript.side_effect = Exception("Migration failed")
|
||||
mock_conn.rollback.side_effect = Exception("Rollback failed")
|
||||
mock_conn.execute.return_value.fetchone.return_value = (0,)
|
||||
|
||||
|
||||
@@ -356,7 +356,8 @@ class TestEntryPoints:
|
||||
).get_json()
|
||||
assert data["syndicate-to"] == [{"uid": f"{GTS}/", "name": "GoToSocial"}]
|
||||
|
||||
def test_micropub_default_is_all_targets(self, app):
|
||||
def test_micropub_default_is_no_targets(self, app):
|
||||
"""Absent mp-syndicate-to means no crossposting (v2.9.0)"""
|
||||
token = self._token(app)
|
||||
response = app.test_client().post(
|
||||
"/micropub", data={"h": "entry", "content": "via quill"},
|
||||
@@ -367,7 +368,7 @@ class TestEntryPoints:
|
||||
assert response.headers["Location"].startswith("https://blog.example.com/note/")
|
||||
with app.app_context():
|
||||
from starpunk.database import get_db
|
||||
assert get_db(app).execute("SELECT COUNT(*) FROM syndications").fetchone()[0] == 1
|
||||
assert get_db(app).execute("SELECT COUNT(*) FROM syndications").fetchone()[0] == 0
|
||||
|
||||
def test_micropub_explicit_empty_choice(self, app):
|
||||
token = self._token(app)
|
||||
|
||||
Reference in new issue
Block a user