fix: Micropub photo posts and failed creates (v2.6.5)
- A create that failed after the note was written (e.g. the same photo twice) answered 500 but left the post published without photos. Photos are now resolved first, and a failed note is removed. - Pooled connections roll back unfinished writes, so one failure no longer locks the database for the next ~10 s. - The same photo twice in one post is shown once. - Photo posts no longer need text (Micropub doesn't require content, and q=config already advertised a photo-only post type). Updates may clear the text while photos remain. Crossposts start with the link; the page is titled "Photo". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d16463ef8a
commit
d0bfc80c2e
10 files changed
+247
-31
No files matched your search
@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.6.5] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
- A Micropub post that failed partway (for example, the same photo listed twice) answered "Failed to create post" but stayed published without its photos, so a client retrying would post it twice. Photos are now looked up before the post is made, and a post that fails after that is removed again
|
||||
- After such a failure, every request failed with "database is locked" for about 10 seconds: database connections now roll back unfinished writes when they're returned to the pool
|
||||
- The same photo listed twice in one post is shown once instead of failing the post
|
||||
|
||||
### Changed
|
||||
|
||||
- Photo posts no longer need text. Micropub doesn't require `content`, and `q=config` already offered a Photo post type with only `photo`. A Micropub update may also clear the text of a post that keeps its photos. Crossposts of a photo post without text start with its link, and its page is titled "Photo"
|
||||
|
||||
## [2.6.4] - 2026-11-01
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
# v2.6.5: Micropub media fixes
|
||||
|
||||
Found while adding media uploads to tuipub, a Micropub client written to the
|
||||
spec rather than to StarPunk.
|
||||
|
||||
## Problems
|
||||
|
||||
1. **Failed creates left a post behind.** `handle_create` made the note, then
|
||||
attached photos. Listing one photo twice broke the `note_media` unique key,
|
||||
so the client got a 500 while the note stayed published without photos.
|
||||
A retry would post it twice.
|
||||
2. **Then the database stayed locked.** `attach_media_to_note` didn't roll
|
||||
back, and `ConnectionPool.return_connection` pooled the connection with
|
||||
its write transaction still open. Other requests got "database is locked"
|
||||
until the 10 s timeout.
|
||||
3. **Photo posts needed text.** The Micropub spec doesn't require `content`,
|
||||
and `q=config` advertised a `photo` post type with only `photo`, yet
|
||||
creates without content got 400 "Content is required".
|
||||
|
||||
## Changes
|
||||
|
||||
- `database/pool.py`: `return_connection` rolls back an open transaction
|
||||
(or closes the connection if that fails).
|
||||
- `media.py`: `attach_media_to_note` rolls back on error.
|
||||
- `micropub.py`:
|
||||
- `handle_create` resolves photos (by media URL, downloading remote ones)
|
||||
before `create_note`. If attaching or crosspost selection fails, it
|
||||
hard-deletes the note before answering 500.
|
||||
- `_resolve_photos` skips repeated media.
|
||||
- Content is optional when photos resolve. Content-less posts whose photos
|
||||
all fail still get a 400 from `create_note`.
|
||||
- `handle_update` allows `replace: {content: [""]}` and `delete: ["content"]`
|
||||
while the note keeps photos (or is a like or repost). Otherwise it answers
|
||||
400 "A post needs text or a photo".
|
||||
- `notes.py`: `create_note` and `update_note` take `has_media`, so
|
||||
empty content is allowed for photo posts.
|
||||
- `syndication.py`: `compose_text` drops the leading blank line when there's
|
||||
no body.
|
||||
|
||||
## Not changed (noted for later)
|
||||
|
||||
- More than 4 photos are still cut to 4 without an error (ADR-057).
|
||||
- `video` and `audio` are ignored. This is allowed by the spec, and they
|
||||
aren't in `q=config` post-types.
|
||||
- Media endpoint errors for non-images expose Python text
|
||||
("cannot identify image file <_io.BytesIO …>").
|
||||
|
||||
## Tests
|
||||
|
||||
`tests/test_micropub_v265.py`. The full suite runs with
|
||||
`SESSION_SECRET=test`. 8 tests in `test_media_upload.py` and
|
||||
`test_migration_race_condition.py` already fail on main, and still fail here.
|
||||
@@ -464,5 +464,5 @@ def create_app(config=None):
|
||||
|
||||
# Package version (Semantic Versioning 2.0.0)
|
||||
# See docs/standards/versioning-strategy.md for details
|
||||
__version__ = "2.6.4"
|
||||
__version__ = "2.6.5"
|
||||
__version_info__ = (2, 6, 4)
|
||||
@@ -107,6 +107,16 @@ class ConnectionPool:
|
||||
if not conn:
|
||||
return
|
||||
|
||||
# A request that failed mid-write must not hand an open transaction
|
||||
# (and its write lock) to the next request (v2.6.5)
|
||||
if conn.in_transaction:
|
||||
try:
|
||||
conn.rollback()
|
||||
except Exception:
|
||||
conn.close()
|
||||
self._stats['connections_closed'] += 1
|
||||
return
|
||||
|
||||
with self._lock:
|
||||
if len(self._pool) < self.pool_size:
|
||||
# Return to pool
|
||||
|
||||
+16
-12
@@ -868,20 +868,24 @@ def attach_media_to_note(note_id: int, media_ids: List[int], captions: List[str]
|
||||
|
||||
db = get_db(current_app)
|
||||
|
||||
# Delete existing associations (for edit case)
|
||||
db.execute("DELETE FROM note_media WHERE note_id = ?", (note_id,))
|
||||
try:
|
||||
# Delete existing associations (for edit case)
|
||||
db.execute("DELETE FROM note_media WHERE note_id = ?", (note_id,))
|
||||
|
||||
# Insert new associations
|
||||
for i, (media_id, caption) in enumerate(zip(media_ids, captions)):
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO note_media (note_id, media_id, display_order, caption)
|
||||
VALUES (?, ?, ?, ?)
|
||||
""",
|
||||
(note_id, media_id, i, caption or None)
|
||||
)
|
||||
# Insert new associations
|
||||
for i, (media_id, caption) in enumerate(zip(media_ids, captions)):
|
||||
db.execute(
|
||||
"""
|
||||
INSERT INTO note_media (note_id, media_id, display_order, caption)
|
||||
VALUES (?, ?, ?, ?)
|
||||
""",
|
||||
(note_id, media_id, i, caption or None)
|
||||
)
|
||||
|
||||
db.commit()
|
||||
db.commit()
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
|
||||
|
||||
def get_note_media(note_id: int) -> List[Dict]:
|
||||
|
||||
+42
-12
@@ -195,6 +195,16 @@ def extract_content(properties: dict) -> str:
|
||||
return content.strip()
|
||||
|
||||
|
||||
def _has_text(content) -> bool:
|
||||
"""Whether a content property has any text. Likes, reposts (v2.5.0) and
|
||||
photo posts (v2.6.5) may come without."""
|
||||
values = content if isinstance(content, list) else [content]
|
||||
return any(
|
||||
(c.strip() if isinstance(c, str) else (c.get("text") or c.get("html") or "").strip())
|
||||
for c in values if c
|
||||
)
|
||||
|
||||
|
||||
def extract_title(properties: dict) -> Optional[str]:
|
||||
"""
|
||||
Extract the explicit title (Micropub 'name' property)
|
||||
@@ -420,7 +430,8 @@ def _resolve_photos(photos: list[dict]) -> tuple[list[int], list[str]]:
|
||||
if len(media_ids) == MAX_PHOTOS:
|
||||
break
|
||||
media_id = photo.get("media_id") or _media_id_for_url(photo["url"])
|
||||
if media_id:
|
||||
# The same photo twice is shown once (v2.6.5)
|
||||
if media_id and media_id not in media_ids:
|
||||
media_ids.append(media_id)
|
||||
captions.append(photo.get("alt", ""))
|
||||
return media_ids, captions
|
||||
@@ -525,24 +536,29 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
repost_of = extract_url_property(properties, "repost-of")
|
||||
if like_of and repost_of:
|
||||
raise MicropubValidationError("A post can't be both a like and a repost")
|
||||
# Likes and reposts may come without a comment (v2.5.0)
|
||||
has_content = any(
|
||||
(c.strip() if isinstance(c, str) else (c.get("text") or c.get("html") or "").strip())
|
||||
for c in properties.get("content", []) if c
|
||||
has_content = _has_text(properties.get("content"))
|
||||
photos = extract_photos(properties) + _save_uploads(files)
|
||||
# Photo posts may have no text (v2.6.5): Micropub doesn't require content
|
||||
content = (
|
||||
extract_content(properties)
|
||||
if has_content or not (like_of or repost_of or photos)
|
||||
else ""
|
||||
)
|
||||
content = extract_content(properties) if has_content or not (like_of or repost_of) else ""
|
||||
title = extract_title(properties)
|
||||
tags = extract_tags(properties)
|
||||
published_date = extract_published_date(properties)
|
||||
published = extract_post_status(properties)
|
||||
in_reply_to = extract_in_reply_to(properties)
|
||||
photos = extract_photos(properties) + _save_uploads(files)
|
||||
# Look up and download photos before the note exists, so a bad photo
|
||||
# can't leave a half-made post behind (v2.6.5)
|
||||
media_ids, captions = _resolve_photos(photos)
|
||||
except MicropubError:
|
||||
raise
|
||||
except Exception as e:
|
||||
current_app.logger.error(f"Property extraction failed: {e}")
|
||||
raise MicropubValidationError(f"Failed to parse request: {str(e)}")
|
||||
|
||||
note = None
|
||||
try:
|
||||
note = create_note(
|
||||
content=add_title_heading(content, title),
|
||||
@@ -553,10 +569,12 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
in_reply_to=in_reply_to,
|
||||
like_of=like_of,
|
||||
repost_of=repost_of,
|
||||
has_media=bool(media_ids),
|
||||
)
|
||||
|
||||
if photos:
|
||||
_attach_photos_to_note(note.id, photos)
|
||||
if media_ids:
|
||||
from starpunk.media import attach_media_to_note
|
||||
attach_media_to_note(note.id, media_ids, captions)
|
||||
|
||||
# Crosspost after photos are attached (v2.1.0); drafts wait until published
|
||||
from starpunk.syndication import get_targets, resolve_targets, select_targets
|
||||
@@ -573,6 +591,13 @@ def handle_create(data: dict, token_info: dict, files=None):
|
||||
raise
|
||||
except Exception as e:
|
||||
current_app.logger.error(f"Failed to create note via Micropub: {e}")
|
||||
# The client is told it failed, so don't leave the post up (v2.6.5)
|
||||
if note is not None:
|
||||
try:
|
||||
from starpunk.notes import delete_note
|
||||
delete_note(id=note.id, soft=False)
|
||||
except Exception as cleanup:
|
||||
current_app.logger.error(f"Could not remove failed note {note.slug}: {cleanup}")
|
||||
raise MicropubError("server_error", "Failed to create post", status_code=500)
|
||||
|
||||
|
||||
@@ -616,7 +641,7 @@ def handle_update(data: dict, token_info: dict):
|
||||
if prop == "name":
|
||||
title = None
|
||||
elif prop == "content":
|
||||
raise MicropubValidationError("content can't be deleted")
|
||||
body = ""
|
||||
elif prop == "category":
|
||||
tags = []
|
||||
elif prop == "photo":
|
||||
@@ -637,7 +662,8 @@ def handle_update(data: dict, token_info: dict):
|
||||
raise MicropubValidationError(f"Can't delete individual values of {prop}")
|
||||
|
||||
if "content" in replace:
|
||||
body = extract_content(replace)
|
||||
# Empty text is fine if the note keeps its photos; update_note checks (v2.6.5)
|
||||
body = extract_content(replace) if _has_text(replace["content"]) else ""
|
||||
if "name" in replace:
|
||||
title = extract_title(replace)
|
||||
if "category" in replace:
|
||||
@@ -655,6 +681,10 @@ def handle_update(data: dict, token_info: dict):
|
||||
if "photo" in add:
|
||||
photos += extract_photos(add)
|
||||
|
||||
# Text may go only if photos stay (or it's a like or repost) (v2.6.5)
|
||||
if not (body or "").strip() and not (photos or note.like_of or note.repost_of):
|
||||
raise MicropubValidationError("A post needs text or a photo")
|
||||
|
||||
new_content = add_title_heading(body, title)
|
||||
if new_content != note.content:
|
||||
changes["content"] = new_content
|
||||
@@ -663,7 +693,7 @@ def handle_update(data: dict, token_info: dict):
|
||||
|
||||
try:
|
||||
if changes:
|
||||
update_note(id=note.id, **changes)
|
||||
update_note(id=note.id, has_media=bool(photos), **changes)
|
||||
if "photo" in touched:
|
||||
_attach_photos_to_note(note.id, photos)
|
||||
note_changed(note.id)
|
||||
|
||||
+6
-4
@@ -142,6 +142,7 @@ def create_note(
|
||||
in_reply_to: Optional[str] = None,
|
||||
like_of: Optional[str] = None,
|
||||
repost_of: Optional[str] = None,
|
||||
has_media: bool = False,
|
||||
) -> Note:
|
||||
"""
|
||||
Create a new note
|
||||
@@ -205,11 +206,11 @@ def create_note(
|
||||
- created_at and updated_at set to same value initially
|
||||
"""
|
||||
# 1. VALIDATION (before any changes)
|
||||
# Likes and reposts may have no comment (v2.5.0)
|
||||
# Likes and reposts may have no comment (v2.5.0), photo posts no text (v2.6.5)
|
||||
if like_of and repost_of:
|
||||
raise InvalidNoteDataError("like_of", like_of, "A note can't be both a like and a repost")
|
||||
content = content or ""
|
||||
if not content.strip() and not (like_of or repost_of):
|
||||
if not content.strip() and not (like_of or repost_of or has_media):
|
||||
raise InvalidNoteDataError(
|
||||
"content", content, "Content cannot be empty or whitespace-only"
|
||||
)
|
||||
@@ -604,7 +605,8 @@ def update_note(
|
||||
content: Optional[str] = None,
|
||||
published: Optional[bool] = None,
|
||||
tags: Optional[list[str]] = None,
|
||||
in_reply_to: Optional[str] = None
|
||||
in_reply_to: Optional[str] = None,
|
||||
has_media: bool = False,
|
||||
) -> Note:
|
||||
"""
|
||||
Update a note's content and/or published status
|
||||
@@ -690,7 +692,7 @@ def update_note(
|
||||
identifier = slug if slug is not None else id
|
||||
raise NoteNotFoundError(identifier)
|
||||
|
||||
if empty_content and not (existing_note.like_of or existing_note.repost_of):
|
||||
if empty_content and not (existing_note.like_of or existing_note.repost_of or has_media):
|
||||
raise InvalidNoteDataError(
|
||||
"content", content, "Content cannot be empty or whitespace-only"
|
||||
)
|
||||
|
||||
@@ -1003,7 +1003,8 @@ def compose_text(note, permalink: str, tags: List[str], max_chars: int,
|
||||
if len(body) + len(tail) > max_chars:
|
||||
room = max_chars - len(tail) - 1
|
||||
body = body[:room].rsplit(" ", 1)[0].rstrip() + "…"
|
||||
return body + tail
|
||||
# A photo post may have no text: start with the link (v2.6.5)
|
||||
return (body + tail).lstrip()
|
||||
|
||||
|
||||
_BLOCKS = ["p", "h1", "h2", "h3", "h4", "h5", "h6", "pre", "blockquote", "ul", "ol", "hr", "table"]
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
{% from "partials/media.html" import display_media %}
|
||||
{% from "partials/reaction.html" import reaction %}
|
||||
|
||||
{% block title %}{% if note.content.strip().startswith('#') %}{{ note.title }}{% else %}{{ note.excerpt|truncate(60) }}{% endif %} · {{ config.SITE_NAME }}{% endblock %}
|
||||
{% block title %}{% if note.content.strip().startswith('#') %}{{ note.title }}{% elif note.excerpt.strip() %}{{ note.excerpt|truncate(60) }}{% elif note.like_of %}Like{% elif note.repost_of %}Repost{% else %}Photo{% endif %} · {{ config.SITE_NAME }}{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<article class="h-entry">
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
"""
|
||||
Micropub media fixes (v2.6.5): photo posts without text, the same photo
|
||||
twice, and failed creates that left a post (and a database lock) behind.
|
||||
"""
|
||||
|
||||
import io
|
||||
|
||||
from starpunk.database.pool import ConnectionPool
|
||||
from starpunk.notes import list_notes
|
||||
from starpunk.syndication import compose_text
|
||||
from tests.test_micropub_v240 import SITE, app, client, h, jpeg, note_at, source # noqa: F401
|
||||
|
||||
|
||||
def upload(client, h, color="red"):
|
||||
response = client.post("/micropub/media", data={"file": (io.BytesIO(jpeg(color)), "a.jpg")},
|
||||
headers=h, content_type="multipart/form-data")
|
||||
assert response.status_code == 201
|
||||
return response.headers["Location"]
|
||||
|
||||
|
||||
def post(client, h, props):
|
||||
return client.post("/micropub", json={"type": ["h-entry"], "properties": props}, headers=h)
|
||||
|
||||
|
||||
def update(client, h, url, **ops):
|
||||
return client.post("/micropub", json={"action": "update", "url": url, **ops}, headers=h)
|
||||
|
||||
|
||||
class TestPhotoWithoutText:
|
||||
def test_json_photo_only(self, app, client, h):
|
||||
photo = upload(client, h)
|
||||
response = post(client, h, {"photo": [{"value": photo, "alt": "Red"}]})
|
||||
assert response.status_code == 201, response.get_json()
|
||||
url = response.headers["Location"]
|
||||
assert note_at(app, url).content == ""
|
||||
assert source(client, h, url, "photo")["properties"]["photo"] == [{"value": photo, "alt": "Red"}]
|
||||
page = client.get(url.replace(SITE, ""))
|
||||
assert page.status_code == 200 and b"<title>Photo \xc2\xb7 " in page.data
|
||||
|
||||
def test_multipart_photo_only(self, client, h):
|
||||
response = client.post("/micropub", data={"h": "entry", "photo": (io.BytesIO(jpeg()), "p.jpg")},
|
||||
headers=h, content_type="multipart/form-data")
|
||||
assert response.status_code == 201, response.get_json()
|
||||
|
||||
def test_nothing_at_all_is_still_rejected(self, client, h):
|
||||
assert post(client, h, {}).status_code == 400
|
||||
# A photo that can't be found doesn't count
|
||||
assert post(client, h, {"photo": [f"{SITE}/media/nope.jpg"]}).status_code == 400
|
||||
|
||||
def test_text_can_be_cleared_while_photos_remain(self, app, client, h):
|
||||
photo = upload(client, h)
|
||||
url = post(client, h, {"content": ["Hi"], "photo": [photo]}).headers["Location"]
|
||||
assert update(client, h, url, replace={"content": [""]}).status_code == 204
|
||||
assert note_at(app, url).content == ""
|
||||
assert update(client, h, url, replace={"content": ["Back"]}).status_code == 204
|
||||
assert update(client, h, url, delete=["content"]).status_code == 204
|
||||
assert note_at(app, url).content == ""
|
||||
# Without photos the text has to stay
|
||||
response = update(client, h, url, delete=["content", "photo"])
|
||||
assert response.status_code == 400
|
||||
|
||||
def test_text_is_required_without_photos(self, client, h):
|
||||
url = post(client, h, {"content": ["Hi"]}).headers["Location"]
|
||||
assert update(client, h, url, replace={"content": [""]}).status_code == 400
|
||||
assert update(client, h, url, delete=["content"]).status_code == 400
|
||||
|
||||
def test_crosspost_text_starts_with_the_link(self, app, client, h):
|
||||
url = post(client, h, {"photo": [upload(client, h)]}).headers["Location"]
|
||||
with app.app_context():
|
||||
note = note_at(app, url)
|
||||
assert compose_text(note, url, ["cats"], 500) == f"{url}\n\n#cats"
|
||||
|
||||
|
||||
class TestFailures:
|
||||
def test_same_photo_twice_is_shown_once(self, client, h):
|
||||
photo = upload(client, h)
|
||||
response = post(client, h, {"content": ["Twice"], "photo": [photo, photo]})
|
||||
assert response.status_code == 201, response.get_json()
|
||||
assert source(client, h, response.headers["Location"], "photo")["properties"]["photo"] == [{"value": photo, "alt": ""}]
|
||||
|
||||
def test_failed_create_leaves_no_post_and_no_lock(self, app, client, h, monkeypatch):
|
||||
import starpunk.syndication as syndication
|
||||
|
||||
def boom(*args, **kwargs):
|
||||
raise RuntimeError("crosspost store down")
|
||||
|
||||
monkeypatch.setattr(syndication, "select_targets", boom)
|
||||
response = post(client, h, {"content": ["Doomed"], "photo": [upload(client, h)]})
|
||||
assert response.status_code == 500
|
||||
with app.app_context():
|
||||
assert list_notes(published_only=False) == []
|
||||
monkeypatch.undo()
|
||||
# The database is usable straight away
|
||||
assert post(client, h, {"content": ["Next"]}).status_code == 201
|
||||
|
||||
def test_pool_rolls_back_open_transactions(self, tmp_path):
|
||||
pool = ConnectionPool(str(tmp_path / "x.db"), pool_size=1, timeout=1)
|
||||
conn = pool.get_connection()
|
||||
conn.execute("CREATE TABLE t (x)")
|
||||
conn.commit()
|
||||
conn.execute("INSERT INTO t VALUES (1)")
|
||||
assert conn.in_transaction
|
||||
pool.return_connection(conn)
|
||||
assert not conn.in_transaction
|
||||
assert pool.get_connection().execute("SELECT count(*) FROM t").fetchone()[0] == 0
|
||||
Reference in new issue
Block a user