StarPunk/docs/design/feed-media-option2-design.md

Feed Media Enhancement Design: Option 2 (RSS + Media RSS Extension)

Overview

This design document specifies the implementation of Option 2 for feed media support: adding Media RSS namespace elements to RSS feeds and the image field to JSON Feed items. This provides improved feed reader compatibility for notes with attached images.

Target Version: v1.2.x Estimated Effort: 4-6 hours Prerequisites: Media attachment model implemented (ADR-057)

Current State

RSS Feed (starpunk/feeds/rss.py)

• Embeds media as <img> tags within the <description> CDATA section
• Uses feedgen library for RSS 2.0 generation
• No <enclosure> elements
• No Media RSS namespace

JSON Feed (starpunk/feeds/json_feed.py)

• Includes media in attachments array (per JSON Feed 1.1 spec)
• Includes media as <img> tags in content_html
• No top-level image field for items

Note Model

• Media accessed via note.media property (list of dicts)
• Each media item has: path, mime_type, size, caption (optional)

Design Goals

1. Standards Compliance: Follow Media RSS spec and JSON Feed 1.1 spec
2. Backward Compatibility: Keep existing HTML embedding for universal reader support
3. Feed Reader Optimization: Add structured metadata for enhanced display
4. Minimal Changes: Modify only feed generation, no database changes

Files to Modify

1. starpunk/feeds/rss.py

Changes Required:

A. Add Media RSS Namespace to Feed Generator

Location: generate_rss() function and generate_rss_streaming() function

# Add namespace registration before generating XML
# For feedgen-based generation:
fg.load_extension('media', rss=True)  # feedgen has built-in media extension

# For streaming generation, add to opening RSS tag:
'<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/">\n'

B. Add RSS <enclosure> Element (First Image Only)

Per RSS 2.0 spec, only ONE enclosure per item is allowed. Use the first image.

# In item generation, after setting description:
if hasattr(note, 'media') and note.media:
    first_media = note.media[0]
    media_url = f"{site_url}/media/{first_media['path']}"
    fe.enclosure(
        url=media_url,
        length=str(first_media.get('size', 0)),
        type=first_media.get('mime_type', 'image/jpeg')
    )

C. Add Media RSS Elements (All Images)

For each image, add <media:content> and optional <media:description>:

# Using feedgen's media extension:
for media_item in note.media:
    media_url = f"{site_url}/media/{media_item['path']}"

    # Add media:content
    fe.media.content({
        'url': media_url,
        'type': media_item.get('mime_type', 'image/jpeg'),
        'medium': 'image',
        'fileSize': str(media_item.get('size', 0))
    })

    # Add media:description if caption exists
    if media_item.get('caption'):
        fe.media.description(media_item['caption'], type='plain')

# Add media:thumbnail for first image
if note.media:
    first_media = note.media[0]
    fe.media.thumbnail({
        'url': f"{site_url}/media/{first_media['path']}"
    })

D. Expected XML Output Structure

For an item with 2 images:

<item>
  <title>My Note Title</title>
  <link>https://example.com/note/my-slug</link>
  <guid isPermaLink="true">https://example.com/note/my-slug</guid>
  <pubDate>Mon, 09 Dec 2024 12:00:00 +0000</pubDate>

  <!-- Standard RSS enclosure (first image only) -->
  <enclosure url="https://example.com/media/2024/12/image1.jpg"
             length="245760"
             type="image/jpeg"/>

  <!-- Media RSS elements (all images) -->
  <media:content url="https://example.com/media/2024/12/image1.jpg"
                 type="image/jpeg"
                 medium="image"
                 fileSize="245760"/>
  <media:content url="https://example.com/media/2024/12/image2.jpg"
                 type="image/jpeg"
                 medium="image"
                 fileSize="198432"/>

  <!-- Thumbnail (first image) -->
  <media:thumbnail url="https://example.com/media/2024/12/image1.jpg"/>

  <!-- Caption if present -->
  <media:description type="plain">Photo from today's hike</media:description>

  <!-- Description with embedded HTML (for legacy readers) -->
  <description><![CDATA[
    <div class="media">
      <img src="https://example.com/media/2024/12/image1.jpg" alt="Photo from today's hike" />
      <img src="https://example.com/media/2024/12/image2.jpg" alt="" />
    </div>
    <p>Note content here...</p>
  ]]></description>
</item>

2. starpunk/feeds/json_feed.py

Changes Required:

A. Add image Field to Item Objects

Per JSON Feed 1.1 spec, image is "the URL of the main image for the item."

Location: _build_item_object() function

def _build_item_object(site_url: str, note: Note) -> Dict[str, Any]:
    # ... existing code ...

    # Add image field (URL of first/main image)
    # Per JSON Feed 1.1: "the URL of the main image for the item"
    if hasattr(note, 'media') and note.media:
        first_media = note.media[0]
        item["image"] = f"{site_url}/media/{first_media['path']}"

    # ... rest of existing code (content_html, attachments, etc.) ...

B. Expected JSON Output Structure

{
  "id": "https://example.com/note/my-slug",
  "url": "https://example.com/note/my-slug",
  "title": "My Note Title",
  "date_published": "2024-12-09T12:00:00Z",

  "image": "https://example.com/media/2024/12/image1.jpg",

  "content_html": "<div class=\"media\"><img src=\"https://example.com/media/2024/12/image1.jpg\" alt=\"Photo from today's hike\" /><img src=\"https://example.com/media/2024/12/image2.jpg\" alt=\"\" /></div><p>Note content here...</p>",

  "attachments": [
    {
      "url": "https://example.com/media/2024/12/image1.jpg",
      "mime_type": "image/jpeg",
      "title": "Photo from today's hike",
      "size_in_bytes": 245760
    },
    {
      "url": "https://example.com/media/2024/12/image2.jpg",
      "mime_type": "image/jpeg",
      "size_in_bytes": 198432
    }
  ],

  "_starpunk": {
    "permalink_path": "/note/my-slug",
    "word_count": 42
  }
}

Implementation Details

RSS Implementation: feedgen vs Manual Streaming

For generate_rss() (feedgen-based):

The feedgen library has a media extension. Check if it's available:

# Test if feedgen supports media extension
from feedgen.ext.media import MediaExtension

# If supported, use:
fg.register_extension('media', MediaExtension, rss=True)

If feedgen's media extension is insufficient, consider manual XML injection after feedgen generates the base XML.

For generate_rss_streaming() (manual XML):

Modify the streaming generator to include media elements. This requires:

1. Update the opening RSS tag to include media namespace
2. Add <enclosure> element after <pubDate>
3. Add <media:content> elements for each image
4. Add <media:thumbnail> for first image
5. Add <media:description> if caption exists

JSON Feed Implementation

Straightforward addition in _build_item_object():

# Add image field if media exists
if hasattr(note, 'media') and note.media:
    first_media = note.media[0]
    item["image"] = f"{site_url}/media/{first_media['path']}"

Testing Requirements

Unit Tests to Add/Modify

File: tests/test_feeds_rss.py (create or extend)

def test_rss_enclosure_for_note_with_media():
    """RSS item should include enclosure element for first image."""
    # Create note with media
    # Generate RSS
    # Parse XML, verify <enclosure> present with correct attributes

def test_rss_media_content_for_all_images():
    """RSS item should include media:content for each image."""
    # Create note with 2 images
    # Generate RSS
    # Parse XML, verify 2 <media:content> elements

def test_rss_media_thumbnail_for_first_image():
    """RSS item should include media:thumbnail for first image."""
    # Create note with media
    # Generate RSS
    # Parse XML, verify <media:thumbnail> present

def test_rss_media_description_for_caption():
    """RSS item should include media:description if caption exists."""
    # Create note with captioned image
    # Generate RSS
    # Parse XML, verify <media:description> present

def test_rss_no_media_elements_without_attachments():
    """RSS item without media should have no media elements."""
    # Create note without media
    # Generate RSS
    # Parse XML, verify no enclosure or media:* elements

def test_rss_namespace_declaration():
    """RSS feed should declare media namespace."""
    # Generate any RSS feed
    # Verify xmlns:media attribute in root element

File: tests/test_feeds_json.py (create or extend)

def test_json_feed_image_field_for_note_with_media():
    """JSON Feed item should include image field for first image."""
    # Create note with media
    # Generate JSON feed
    # Parse JSON, verify "image" field present with correct URL

def test_json_feed_no_image_field_without_media():
    """JSON Feed item without media should not have image field."""
    # Create note without media
    # Generate JSON feed
    # Parse JSON, verify "image" field not present

def test_json_feed_image_uses_first_media():
    """JSON Feed image field should use first media item URL."""
    # Create note with 3 images
    # Generate JSON feed
    # Verify "image" URL matches first image path

Feed Validation Tests

Manual Validation (document in test plan):

1. W3C Feed Validator: https://validator.w3.org/feed/

   • Submit generated RSS feed
   • Verify no errors for media:* elements
   • Note: Validator may warn about unknown extensions (acceptable)

2. Feed Reader Testing:

   • Feedly: Verify images display in article preview
   • NetNewsWire: Check media thumbnail in list view
   • Feedbin: Test image extraction
   • RSS.app: Verify enclosure handling

3. JSON Feed Validator: Use online JSON Feed validator

   • Verify image field accepted
   • Verify attachments array valid

Integration Tests

def test_rss_route_with_media_notes(client, app):
    """GET /feed.xml with media notes returns valid RSS with media elements."""
    # Create test notes with media
    # Request /feed.xml
    # Verify response contains media namespace and elements

def test_json_route_with_media_notes(client, app):
    """GET /feed.json with media notes returns JSON with image fields."""
    # Create test notes with media
    # Request /feed.json
    # Verify response contains image fields

Reference Documentation

Media RSS Specification

• URL: https://www.rssboard.org/media-rss
• Key Elements Used:
  • media:content - Primary media reference
  • media:thumbnail - Preview image
  • media:description - Caption text

JSON Feed 1.1 Specification

• URL: https://jsonfeed.org/version/1.1/
• Key Fields Used:
  • image (item level) - "the URL of the main image for the item"
  • attachments - Array of attachment objects (already implemented)

RSS 2.0 Enclosure Specification

• URL: https://www.rssboard.org/rss-specification#ltenclosuregtSubelementOfLtitemgt
• Constraint: Only ONE enclosure per item allowed
• Required Attributes: url, length, type

Feed Reader Compatibility Notes

Media RSS Support

Reader	media:content	media:thumbnail	enclosure
Feedly	Yes	Yes	Yes
Inoreader	Yes	Yes	Yes
NetNewsWire	Partial	Yes	Yes
Feedbin	Yes	Yes	Yes
RSS.app	Yes	Yes	Yes
The Old Reader	Yes	Partial	Yes

JSON Feed Image Support

Reader	image field	attachments
Feedly	Yes	Yes
NetNewsWire	Yes	Yes
Reeder	Yes	Yes
Feedbin	Yes	Yes

Note: The HTML-embedded images in description/content_html serve as fallback for readers that don't support Media RSS or JSON Feed attachments.

Rollout Plan

1. Implement RSS Changes

   • Add namespace declaration
   • Add enclosure element
   • Add media:content elements
   • Add media:thumbnail
   • Add media:description for captions

2. Implement JSON Feed Changes

   • Add image field to item builder

3. Add Tests

   • Unit tests for both feed types
   • Integration tests for routes

4. Manual Validation

   • Test with W3C validator
   • Test in 3+ feed readers

5. Deploy

   • Release as part of v1.2.x

Future Considerations (Option 3)

This design explicitly does NOT include:

• Multiple image sizes/thumbnails (deferred to ADR-059)
• Video support (deferred to v1.4.0)
• Audio/podcast support (deferred to v1.3.0+)
• Full Media RSS attribute set (width, height, duration)

These are documented in ADR-059: Full Feed Media Standardization for future releases.

Summary of Changes

File	Change
starpunk/feeds/rss.py	Add media namespace, enclosure, media:content, media:thumbnail, media:description
starpunk/feeds/json_feed.py	Add image field to items with media
tests/test_feeds_rss.py	Add 6 new test cases for media elements
tests/test_feeds_json.py	Add 3 new test cases for image field

Total Estimated Changes: ~100-150 lines of new code + ~100 lines of tests