Compare commits
48 Commits
dd822a35b5
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 06a8aabc01 | |||
| 4ee2c189ae | |||
| c94cb377d3 | |||
| 2bd971f3d6 | |||
| 84e693fe57 | |||
| 975046abc7 | |||
| 21fa7acfbb | |||
| b689e02e64 | |||
| 1b45a64920 | |||
| 3f1f82a749 | |||
| 92e7bdd342 | |||
| 0acefa4670 | |||
| 9dcc5c5710 | |||
| 7be2fb0f62 | |||
| 730eb8d58c | |||
| 6682339a86 | |||
| d416463242 | |||
| 25b8cbd79d | |||
| 042505d5a6 | |||
| 72f3d4a55e | |||
| e8ff0a0dcb | |||
| 9bc6780a8e | |||
| e4e481d7cf | |||
| 07f351fef7 | |||
| fd92a1d1eb | |||
| 68d1a1d879 | |||
| 00f21d2a51 | |||
| 83dc488457 | |||
| c64feaea23 | |||
| 501a711050 | |||
| 1b51c82656 | |||
| 5ea9c8f330 | |||
| 98692c35db | |||
| 61cba2fa6d | |||
| cba24ab06f | |||
| 9b26de7b05 | |||
| 3222620cee | |||
| 247eb34c36 | |||
| 41b65703f9 | |||
| f901aa2242 | |||
| 5ca8b7e9b4 | |||
| 3d80e1af51 | |||
| 372064b116 | |||
| 377027e79a | |||
| f10d0679da | |||
| 927db4aea0 | |||
| 27501f6381 | |||
| 10d85bb78b |
336
CHANGELOG.md
336
CHANGELOG.md
@@ -7,9 +7,337 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
## [1.2.0-rc.1] - 2025-11-28
|
## [1.5.0] - 2025-12-17
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
||||||
|
- **Timestamp-Based Slugs** - Default slug generation now uses timestamp format (ADR-062)
|
||||||
|
- Format: YYYYMMDDHHMMSS (e.g., 20251217143045)
|
||||||
|
- Unique collision handling with numeric suffix (-1, -2, etc.)
|
||||||
|
- More semantic and sortable than random slugs
|
||||||
|
- Custom slugs via `mp-slug` or web UI still supported
|
||||||
|
- Applies to all new notes created via Micropub or web interface
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **Debug File Management** - Enhanced control and cleanup for failed uploads
|
||||||
|
- Debug file saving now controlled by `DEBUG_SAVE_FAILED_UPLOADS` config (default: false in production)
|
||||||
|
- Automatic cleanup of debug files older than 7 days on app startup
|
||||||
|
- Disk space protection with 100MB limit for debug directory
|
||||||
|
- Filename sanitization in debug paths to prevent path traversal
|
||||||
|
- Debug feature designed for development/troubleshooting only
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **IndieAuth Authentication** - Corrected W3C IndieAuth specification compliance
|
||||||
|
- Authentication now discovers endpoints from user's profile URL per specification
|
||||||
|
- Uses `response_type=id` for authentication-only flow (identity verification)
|
||||||
|
- Removed hardcoded indielogin.com service URL
|
||||||
|
- URL comparison handles trailing slashes and case differences correctly
|
||||||
|
- User-friendly error messages when endpoint discovery fails
|
||||||
|
- DEPRECATED: `INDIELOGIN_URL` config no longer used (warning shown if set)
|
||||||
|
|
||||||
|
- **Feed Generation Performance** - Eliminated N+1 query pattern in feed generation
|
||||||
|
- Implemented batch loading for note media in `_get_cached_notes()`
|
||||||
|
- Single query loads media for all 50 feed notes instead of 50 separate queries
|
||||||
|
- Significant performance improvement for RSS/Atom/JSON Feed generation
|
||||||
|
- Feed cache still prevents repeated queries for same feed requests
|
||||||
|
- Other N+1 patterns in lower-traffic areas deferred (see BACKLOG.md)
|
||||||
|
|
||||||
|
- **Variant Generation Atomicity** - Fixed orphaned files on database failure
|
||||||
|
- Variant files now written to temporary location first
|
||||||
|
- Files moved to final location only after successful database commit
|
||||||
|
- Prevents disk bloat from failed variant generation
|
||||||
|
- Rollback mechanism cleans up temporary files on error
|
||||||
|
- Database and filesystem state now consistent
|
||||||
|
|
||||||
|
### Technical Details
|
||||||
|
|
||||||
|
- **Migration 010**: Add debug cleanup tracking table (optional)
|
||||||
|
- **New Functions**:
|
||||||
|
- `cleanup_old_debug_files()` - Automatic debug file cleanup
|
||||||
|
- `sanitize_filename()` - Safe filename generation for debug paths
|
||||||
|
- `get_media_for_notes()` - Batch media loading to prevent N+1 queries
|
||||||
|
- **Modified Functions**:
|
||||||
|
- `generate_slug()` - Timestamp-based default slug generation
|
||||||
|
- `save_media()` - Debug file handling with sanitization
|
||||||
|
- `generate_all_variants()` - Atomic variant generation with temp files
|
||||||
|
- `_get_cached_notes()` - Batch media loading integration
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
- `DEBUG_SAVE_FAILED_UPLOADS` - Enable debug file saving (default: false in production)
|
||||||
|
- `DEBUG_MAX_AGE_DAYS` - Debug file retention period (default: 7)
|
||||||
|
- `DEBUG_MAX_SIZE_MB` - Debug directory size limit (default: 100)
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
|
||||||
|
- Enhanced MPO format test coverage
|
||||||
|
- All 5 broken tests removed (test suite cleanup)
|
||||||
|
- Brittle test assertions fixed
|
||||||
|
- Complete test coverage for new batch loading functions
|
||||||
|
- Atomic variant generation tests with rollback scenarios
|
||||||
|
|
||||||
|
### Standards Compliance
|
||||||
|
|
||||||
|
- ADR-062: Timestamp-Based Default Slug Generation
|
||||||
|
- Maintains backward compatibility with custom slugs
|
||||||
|
- No breaking changes to Micropub or web interface
|
||||||
|
|
||||||
|
### Related Documentation
|
||||||
|
|
||||||
|
- ADR-062: Timestamp-Based Default Slug Generation
|
||||||
|
- Implementation reports in `docs/design/v1.5.0/`
|
||||||
|
- Architect reviews documenting all design decisions
|
||||||
|
|
||||||
|
## [1.4.2] - 2025-12-16
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- HEIC/HEIF image format support for iPhone photo uploads
|
||||||
|
- MPO (Multi-Picture Object) format support for iPhone depth/portrait photos
|
||||||
|
- Automatic HEIC/MPO to JPEG conversion (browsers cannot display these formats)
|
||||||
|
- Graceful error handling if pillow-heif library not installed
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- Increased maximum input image dimensions from 4096x4096 to 12000x12000 to support modern phone cameras (48MP+); images are still optimized to smaller sizes for web delivery
|
||||||
|
|
||||||
|
### Dependencies
|
||||||
|
|
||||||
|
- Added `pillow-heif` for HEIC image processing
|
||||||
|
- Updated `Pillow` from 10.0.* to 10.1.* (required by pillow-heif)
|
||||||
|
|
||||||
|
## [1.4.1] - 2025-12-16
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Media upload failures are now logged for debugging and observability
|
||||||
|
- Validation errors (invalid format, file too large) logged at WARNING level
|
||||||
|
- Optimization failures logged at WARNING level
|
||||||
|
- Variant generation failures logged at WARNING level (upload continues)
|
||||||
|
- Unexpected errors logged at ERROR level with error type and message
|
||||||
|
- Successful uploads logged at INFO level with file details
|
||||||
|
- Removed duplicate logging in Micropub media endpoint
|
||||||
|
|
||||||
|
## [1.4.0rc1] - 2025-12-16
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Large Image Support** - Accept and optimize images up to 50MB (Phase 1)
|
||||||
|
- Increased file size limit from 10MB to 50MB for image uploads
|
||||||
|
- Tiered resize strategy based on input size:
|
||||||
|
- <=10MB: 2048px max dimension, 95% quality
|
||||||
|
- 10-25MB: 1600px max dimension, 90% quality
|
||||||
|
- 25-50MB: 1280px max dimension, 85% quality
|
||||||
|
- Iterative quality reduction if output still >10MB after optimization
|
||||||
|
- Rejects if optimization cannot achieve <=10MB target (minimum 640px at 70% quality)
|
||||||
|
- Animated GIF detection with 10MB size limit (cannot be resized)
|
||||||
|
- All optimized images kept at or under 10MB for web performance
|
||||||
|
|
||||||
|
- **Image Variants** - Multiple image sizes for responsive delivery (Phase 2)
|
||||||
|
- Four variants generated automatically on upload:
|
||||||
|
- thumb: 150x150 center crop for thumbnails
|
||||||
|
- small: 320px width for mobile/low bandwidth
|
||||||
|
- medium: 640px width for standard display
|
||||||
|
- large: 1280px width for high-res display
|
||||||
|
- original: As uploaded (optimized, <=2048px)
|
||||||
|
- Variants stored in date-organized folders (data/media/YYYY/MM/)
|
||||||
|
- Database tracking in new `media_variants` table
|
||||||
|
- Synchronous/eager generation on upload
|
||||||
|
- Only new uploads get variants (existing media unchanged)
|
||||||
|
- Cascade deletion with parent media
|
||||||
|
- Variants included in `get_note_media()` response (backwards compatible)
|
||||||
|
|
||||||
|
- **Micropub Media Endpoint** - W3C-compliant media upload (Phase 3)
|
||||||
|
- New POST `/micropub/media` endpoint for file uploads
|
||||||
|
- Multipart/form-data with single file part named 'file'
|
||||||
|
- Bearer token authentication with `create` scope (no new scope needed)
|
||||||
|
- Returns 201 Created with Location header on success
|
||||||
|
- Automatic variant generation on upload
|
||||||
|
- OAuth 2.0 error format for all error responses
|
||||||
|
- Media endpoint advertised in `q=config` query response
|
||||||
|
- Photo property support in Micropub create requests:
|
||||||
|
- Simple URL strings: `"photo": ["https://example.com/image.jpg"]`
|
||||||
|
- Structured with alt text: `"photo": [{"value": "url", "alt": "description"}]`
|
||||||
|
- Multiple photos supported (max 4 per ADR-057)
|
||||||
|
- External URLs logged and ignored (no download)
|
||||||
|
- Photo captions preserved from alt text
|
||||||
|
|
||||||
|
- **Enhanced Feed Media** - Full Media RSS and JSON Feed variants (Phase 4)
|
||||||
|
- RSS 2.0: Complete Media RSS namespace support
|
||||||
|
- `media:group` element for multiple sizes of same image
|
||||||
|
- `media:content` for each variant with dimensions and file size
|
||||||
|
- `media:thumbnail` element for preview images
|
||||||
|
- `media:title` for captions (when present)
|
||||||
|
- `isDefault` attribute on largest available variant
|
||||||
|
- JSON Feed 1.1: `_starpunk` extension with variants
|
||||||
|
- All variants with URLs, dimensions, and sizes
|
||||||
|
- Configurable `about` URL for extension documentation
|
||||||
|
- Default: `https://github.com/yourusername/starpunk`
|
||||||
|
- Override via `STARPUNK_ABOUT_URL` config
|
||||||
|
- ATOM 1.0: Enclosures with title attribute for captions
|
||||||
|
- Backwards compatible: Feeds work with and without variants
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **Image Optimization** - Enhanced for large file handling
|
||||||
|
- `optimize_image()` now accepts `original_size` parameter
|
||||||
|
- Returns both optimized image and bytes (avoids re-saving)
|
||||||
|
- Iterative quality reduction loop for difficult-to-compress images
|
||||||
|
- Safety check prevents infinite loops (minimum 640px dimension)
|
||||||
|
|
||||||
|
- **Media Storage** - Extended with variant support
|
||||||
|
- `save_media()` generates variants synchronously after saving original
|
||||||
|
- Variants cleaned up automatically on generation failure
|
||||||
|
- Database records original as 'original' variant type
|
||||||
|
- File size passed efficiently without redundant I/O
|
||||||
|
|
||||||
|
### Technical Details
|
||||||
|
|
||||||
|
- **Migration 009**: Add `media_variants` table
|
||||||
|
- Tracks variant_type, path, dimensions, and file size
|
||||||
|
- Foreign key to media table with cascade delete
|
||||||
|
- Unique constraint on (media_id, variant_type)
|
||||||
|
- Index on media_id for efficient lookups
|
||||||
|
|
||||||
|
- **New Functions**:
|
||||||
|
- `get_optimization_params(file_size)` - Tiered resize strategy
|
||||||
|
- `generate_variant()` - Single variant generation
|
||||||
|
- `generate_all_variants()` - Full variant set with DB storage
|
||||||
|
- `extract_photos()` - Micropub photo property parsing
|
||||||
|
- `_attach_photos_to_note()` - Photo attachment to notes
|
||||||
|
|
||||||
|
- **Modified Functions**:
|
||||||
|
- `validate_image()` - 50MB limit, animated GIF detection
|
||||||
|
- `optimize_image()` - Size-aware tiered optimization
|
||||||
|
- `save_media()` - Variant generation integration
|
||||||
|
- `get_note_media()` - Includes variants (when present)
|
||||||
|
- `handle_query()` - Advertises media-endpoint in config
|
||||||
|
- `handle_create()` - Photo property extraction and attachment
|
||||||
|
- `generate_rss_streaming()` - Media RSS support
|
||||||
|
- `_build_item_object()` - JSON Feed variants
|
||||||
|
- `generate_atom_streaming()` - Enclosure title attributes
|
||||||
|
|
||||||
|
- **Configuration Options**:
|
||||||
|
- `STARPUNK_ABOUT_URL` - JSON Feed extension documentation URL
|
||||||
|
|
||||||
|
### Standards Compliance
|
||||||
|
|
||||||
|
- W3C Micropub Media Endpoint Specification
|
||||||
|
- Media RSS 2.0 Specification (RSS Board)
|
||||||
|
- JSON Feed 1.1 with custom extension
|
||||||
|
- OAuth 2.0 Bearer Token Authentication
|
||||||
|
- RFC 3339 date formats in feeds
|
||||||
|
|
||||||
|
### Storage Impact
|
||||||
|
|
||||||
|
- Variants use approximately 4x storage per image:
|
||||||
|
- Original: 100%
|
||||||
|
- Large (1280px): ~50%
|
||||||
|
- Medium (640px): ~25%
|
||||||
|
- Small (320px): ~12%
|
||||||
|
- Thumb (150x150): ~3%
|
||||||
|
- Typical 500KB optimized image → ~900KB total with variants
|
||||||
|
- Only new uploads generate variants (existing media unchanged)
|
||||||
|
|
||||||
|
### Backwards Compatibility
|
||||||
|
|
||||||
|
- Existing media files work unchanged
|
||||||
|
- No variants generated for pre-v1.4.0 uploads
|
||||||
|
- Feeds handle media with and without variants gracefully
|
||||||
|
- `get_note_media()` only includes 'variants' key when variants exist
|
||||||
|
- All existing Micropub clients continue to work
|
||||||
|
|
||||||
|
### Related Documentation
|
||||||
|
|
||||||
|
- ADR-057: Media Attachment Model
|
||||||
|
- ADR-058: Image Optimization Strategy
|
||||||
|
- ADR-059: Full Feed Media Standardization
|
||||||
|
- Design: `/docs/design/v1.4.0/media-implementation-design.md`
|
||||||
|
|
||||||
|
## [1.3.1] - 2025-12-10
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Feed Tags/Categories** - Tags now appear in all syndication feed formats
|
||||||
|
- RSS 2.0: `<category>` elements for each tag
|
||||||
|
- Atom 1.0: `<category term="slug" label="Display Name"/>` per RFC 4287
|
||||||
|
- JSON Feed 1.1: `tags` array with display names
|
||||||
|
- Tags omitted from feeds when note has no tags
|
||||||
|
|
||||||
|
### Technical Details
|
||||||
|
|
||||||
|
- Enhanced: `starpunk/feeds/rss.py` with category elements
|
||||||
|
- Enhanced: `starpunk/feeds/atom.py` with category elements
|
||||||
|
- Enhanced: `starpunk/feeds/json_feed.py` with tags array
|
||||||
|
- Enhanced: `starpunk/routes/public.py` pre-loads tags for feed generation
|
||||||
|
|
||||||
|
## [1.3.0] - 2025-12-10
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Tag/Category System** - Complete tag support with hierarchical organization
|
||||||
|
- Tag creation and management via web UI and Micropub
|
||||||
|
- Support for Micropub `category` property in JSON and form-encoded requests
|
||||||
|
- Tag archive pages at `/tags/{tag}` with all tagged notes
|
||||||
|
- Tag cloud display on homepage showing all used tags
|
||||||
|
- Tag filtering in database queries (list_notes_by_tag)
|
||||||
|
- Reserved tag validation (prevents tags like 'api', 'admin', etc.)
|
||||||
|
- Comprehensive tag management in admin dashboard
|
||||||
|
- Database schema: tags table with slug and name fields
|
||||||
|
- Many-to-many relationship between notes and tags
|
||||||
|
- Automatic tag cleanup (removes orphaned tags)
|
||||||
|
|
||||||
|
- **Strict Microformats2 Compliance** - Enhanced h-entry markup for parsers
|
||||||
|
- p-category property for each tag in note markup
|
||||||
|
- dt-updated property displays when note is modified
|
||||||
|
- dt-published always shown for temporal context
|
||||||
|
- u-uid property matches u-url for permalink stability
|
||||||
|
- Proper h-feed structure on homepage and tag archives
|
||||||
|
- p-name property only when note has explicit title (# heading)
|
||||||
|
- e-content wraps full note content
|
||||||
|
- Nested h-card for author within each h-entry
|
||||||
|
- Homepage displays as complete h-feed with feed properties
|
||||||
|
|
||||||
|
- **h-feed Properties** - Proper feed markup on collection pages
|
||||||
|
- Homepage marked as h-feed with p-name "Recent Notes"
|
||||||
|
- Tag archive pages marked as h-feed with descriptive p-name
|
||||||
|
- Each feed contains multiple h-entry items
|
||||||
|
- Feed structure validates with Microformats2 parsers
|
||||||
|
- Supports feed readers and IndieWeb aggregators
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **Template Structure** - Reorganized for better Microformats2 compliance
|
||||||
|
- Homepage template now wraps entries in proper h-feed
|
||||||
|
- Note display templates use semantic h-entry markup
|
||||||
|
- Tag display integrated throughout note views
|
||||||
|
- Consistent Microformats2 patterns across all pages
|
||||||
|
|
||||||
|
### Technical Details
|
||||||
|
|
||||||
|
- Migration 006: Add tags table and note_tags junction table
|
||||||
|
- New module: `starpunk/tags.py` with tag CRUD operations
|
||||||
|
- Enhanced: `starpunk/notes.py` with tag relationship handling
|
||||||
|
- Enhanced: `starpunk/micropub.py` with category property support
|
||||||
|
- Enhanced: Templates with p-category and h-feed markup
|
||||||
|
- All tests passing (580+ tests)
|
||||||
|
- 100% backward compatible with existing notes
|
||||||
|
|
||||||
|
## [1.2.0] - 2025-12-09
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **Feed Media Enhancement** - Media RSS and JSON Feed image support for improved feed reader compatibility
|
||||||
|
- RSS feeds now include Media RSS namespace (xmlns:media) for structured media metadata
|
||||||
|
- RSS enclosure element added for first image (per RSS 2.0 spec)
|
||||||
|
- Media RSS media:content elements for all images with type, medium, and fileSize attributes
|
||||||
|
- Media RSS media:thumbnail element for first image preview
|
||||||
|
- JSON Feed items include "image" field with first image URL (per JSON Feed 1.1 spec)
|
||||||
|
- Image field absent (not null) when no media attached
|
||||||
|
- Both feed formats maintain existing HTML embedding for universal reader support
|
||||||
|
- Provides enhanced display in modern feed readers (Feedly, Inoreader, NetNewsWire)
|
||||||
|
|
||||||
- **Custom Slug Input Field** - Web UI now supports custom slugs (v1.2.0 Phase 1)
|
- **Custom Slug Input Field** - Web UI now supports custom slugs (v1.2.0 Phase 1)
|
||||||
- Added optional custom slug field to note creation form
|
- Added optional custom slug field to note creation form
|
||||||
- Slugs are read-only after creation to preserve permalinks
|
- Slugs are read-only after creation to preserve permalinks
|
||||||
@@ -53,6 +381,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
- Multiple u-photo properties in Microformats2 markup
|
- Multiple u-photo properties in Microformats2 markup
|
||||||
- Media files cached immutably (1 year) for performance
|
- Media files cached immutably (1 year) for performance
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **Media Display on Homepage** - Images now display correctly on homepage, not just individual note pages
|
||||||
|
- **Responsive Image Sizing** - Images constrained to container width with proper CSS
|
||||||
|
- **Caption Display** - Captions now used as alt text only, not displayed as visible text
|
||||||
|
- **Logging Correlation ID** - Fixed crash in non-request contexts (app init, memory monitor)
|
||||||
|
|
||||||
## [1.1.2] - 2025-11-28
|
## [1.1.2] - 2025-11-28
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|||||||
85
CLAUDE.md
85
CLAUDE.md
@@ -8,97 +8,50 @@ This file contains operational instructions for Claude agents working on this pr
|
|||||||
- All Python commands must be run with `uv run` prefix
|
- All Python commands must be run with `uv run` prefix
|
||||||
- Example: `uv run pytest`, `uv run flask run`
|
- Example: `uv run pytest`, `uv run flask run`
|
||||||
|
|
||||||
|
## Agent Protocol (All Agents)
|
||||||
|
|
||||||
|
**IMPORTANT**: All agents must review `docs/DOCUMENTATION.md` before starting work. This file is the authoritative source for documentation organization and supersedes any other instructions.
|
||||||
|
|
||||||
## Agent-Architect Protocol
|
## Agent-Architect Protocol
|
||||||
|
|
||||||
When invoking the agent-architect, always remind it to:
|
When invoking the agent-architect, always remind it to:
|
||||||
|
|
||||||
1. Review documentation in docs/ before working on the task it is given
|
1. Review `docs/DOCUMENTATION.md` for documentation organization standards
|
||||||
- docs/architecture, docs/decisions, docs/standards are of particular interest
|
|
||||||
|
|
||||||
2. Give it the map of the documentation folder as described in the "Understanding the docs/ Structure" section below
|
2. Review documentation in docs/ before working on the task it is given
|
||||||
|
- docs/architecture, docs/decisions, docs/standards are of particular interest
|
||||||
|
|
||||||
3. Search for authoritative documentation for any web standard it is implementing on https://www.w3.org/
|
3. Search for authoritative documentation for any web standard it is implementing on https://www.w3.org/
|
||||||
|
|
||||||
4. If it is reviewing a developers implementation report and it is accepts the completed work it should go back and update the project plan to reflect the completed work
|
4. If it is reviewing a developers implementation report and it accepts the completed work it should go back and update the project plan to reflect the completed work
|
||||||
|
|
||||||
## Agent-Developer Protocol
|
## Agent-Developer Protocol
|
||||||
|
|
||||||
When invoking the agent-developer, always remind it to:
|
When invoking the agent-developer, always remind it to:
|
||||||
|
|
||||||
1. **Document work in reports**
|
1. Review `docs/DOCUMENTATION.md` for documentation organization standards
|
||||||
- Create implementation reports in `docs/reports/`
|
|
||||||
- Include date in filename: `YYYY-MM-DD-description.md`
|
|
||||||
|
|
||||||
2. **Update the changelog**
|
2. **Document work in design folder**
|
||||||
|
- Create implementation reports in `docs/design/{version}/`
|
||||||
|
- Include date in filename: `YYYY-MM-DD-description.md`
|
||||||
|
- All developer interaction (questions, responses, reports, reviews) goes in design/{version}/
|
||||||
|
|
||||||
|
3. **Update the changelog**
|
||||||
- Add entries to `CHANGELOG.md` for user-facing changes
|
- Add entries to `CHANGELOG.md` for user-facing changes
|
||||||
- Follow existing format
|
- Follow existing format
|
||||||
|
|
||||||
3. **Version number management**
|
4. **Version number management**
|
||||||
- Increment version numbers according to `docs/standards/versioning-strategy.md`
|
- Increment version numbers according to `docs/standards/versioning-strategy.md`
|
||||||
- Update version in `starpunk/__init__.py`
|
- Update version in `starpunk/__init__.py`
|
||||||
|
|
||||||
4. **Follow git protocol**
|
5. **Follow git protocol**
|
||||||
- Adhere to git branching strategy in `docs/standards/git-branching-strategy.md`
|
- Adhere to git branching strategy in `docs/standards/git-branching-strategy.md`
|
||||||
- Create feature branches for non-trivial changes
|
- Create feature branches for non-trivial changes
|
||||||
- Write clear commit messages
|
- Write clear commit messages
|
||||||
|
|
||||||
## Documentation Navigation
|
## Documentation
|
||||||
|
|
||||||
### Understanding the docs/ Structure
|
See `docs/DOCUMENTATION.md` for the authoritative documentation structure, navigation guidance, and key references.
|
||||||
|
|
||||||
The `docs/` folder is organized by document type and purpose:
|
|
||||||
|
|
||||||
- **`docs/architecture/`** - System design overviews, component diagrams, architectural patterns
|
|
||||||
- **`docs/decisions/`** - Architecture Decision Records (ADRs), numbered sequentially (ADR-001, ADR-002, etc.)
|
|
||||||
- **`docs/deployment/`** - Deployment guides, infrastructure setup, operations documentation
|
|
||||||
- **`docs/design/`** - Detailed design documents, feature specifications, phase plans
|
|
||||||
- **`docs/examples/`** - Example implementations, code samples, usage patterns
|
|
||||||
- **`docs/migration/`** - Migration guides for upgrading between versions and configuration changes
|
|
||||||
- **`docs/projectplan/`** - Project roadmaps, implementation plans, feature scope definitions
|
|
||||||
- **`docs/releases/`** - Release-specific documentation, release notes, version information
|
|
||||||
- **`docs/reports/`** - Implementation reports from developers (dated: YYYY-MM-DD-description.md)
|
|
||||||
- **`docs/reviews/`** - Architectural reviews, design critiques, retrospectives
|
|
||||||
- **`docs/security/`** - Security-related documentation, vulnerability analyses, best practices
|
|
||||||
- **`docs/standards/`** - Coding standards, conventions, processes, workflows
|
|
||||||
|
|
||||||
### Where to Find Documentation
|
|
||||||
|
|
||||||
- **Before implementing a feature**: Check `docs/decisions/` for relevant ADRs and `docs/design/` for specifications
|
|
||||||
- **Understanding system architecture**: Start with `docs/architecture/overview.md`
|
|
||||||
- **Coding guidelines**: See `docs/standards/` for language-specific standards and best practices
|
|
||||||
- **Past implementation context**: Review `docs/reports/` for similar work (sorted by date)
|
|
||||||
- **Project roadmap and scope**: Refer to `docs/projectplan/`
|
|
||||||
|
|
||||||
### Where to Create New Documentation
|
|
||||||
|
|
||||||
**Create an ADR (`docs/decisions/`)** when:
|
|
||||||
- Making architectural decisions that affect system design
|
|
||||||
- Choosing between competing technical approaches
|
|
||||||
- Establishing patterns that others should follow
|
|
||||||
- Format: `ADR-NNN-brief-title.md` (find next number sequentially)
|
|
||||||
|
|
||||||
**Create a design doc (`docs/design/`)** when:
|
|
||||||
- Planning a complex feature implementation
|
|
||||||
- Detailing technical specifications
|
|
||||||
- Documenting multi-phase development plans
|
|
||||||
|
|
||||||
**Create an implementation report (`docs/reports/`)** when:
|
|
||||||
- Completing significant development work
|
|
||||||
- Documenting implementation details for architect review
|
|
||||||
- Format: `YYYY-MM-DD-brief-description.md`
|
|
||||||
|
|
||||||
**Update standards (`docs/standards/`)** when:
|
|
||||||
- Establishing new coding conventions
|
|
||||||
- Documenting processes or workflows
|
|
||||||
- Creating checklists or guidelines
|
|
||||||
|
|
||||||
### Key Documentation References
|
|
||||||
|
|
||||||
- **Architecture**: See `docs/architecture/overview.md`
|
|
||||||
- **Implementation Plan**: See `docs/projectplan/v1/implementation-plan.md`
|
|
||||||
- **Feature Scope**: See `docs/projectplan/v1/feature-scope.md`
|
|
||||||
- **Coding Standards**: See `docs/standards/python-coding-standards.md`
|
|
||||||
- **Testing**: See `docs/standards/testing-checklist.md`
|
|
||||||
|
|
||||||
## Project Philosophy
|
## Project Philosophy
|
||||||
|
|
||||||
|
|||||||
18
README.md
18
README.md
@@ -2,13 +2,13 @@
|
|||||||
|
|
||||||
A minimal, self-hosted IndieWeb CMS for publishing notes with RSS syndication.
|
A minimal, self-hosted IndieWeb CMS for publishing notes with RSS syndication.
|
||||||
|
|
||||||
**Current Version**: 1.1.0
|
**Current Version**: 1.2.0
|
||||||
|
|
||||||
## Versioning
|
## Versioning
|
||||||
|
|
||||||
StarPunk follows [Semantic Versioning 2.0.0](https://semver.org/):
|
StarPunk follows [Semantic Versioning 2.0.0](https://semver.org/):
|
||||||
- Version format: `MAJOR.MINOR.PATCH`
|
- Version format: `MAJOR.MINOR.PATCH`
|
||||||
- Current: `1.1.0` (stable release)
|
- Current: `1.2.0` (stable release)
|
||||||
- Check version: `python -c "from starpunk import __version__; print(__version__)"`
|
- Check version: `python -c "from starpunk import __version__; print(__version__)"`
|
||||||
- See changes: [CHANGELOG.md](CHANGELOG.md)
|
- See changes: [CHANGELOG.md](CHANGELOG.md)
|
||||||
- Versioning strategy: [docs/standards/versioning-strategy.md](docs/standards/versioning-strategy.md)
|
- Versioning strategy: [docs/standards/versioning-strategy.md](docs/standards/versioning-strategy.md)
|
||||||
@@ -29,10 +29,14 @@ StarPunk is designed for a single user who wants to:
|
|||||||
- **File-based storage**: Notes are markdown files, owned by you
|
- **File-based storage**: Notes are markdown files, owned by you
|
||||||
- **IndieAuth authentication**: Use your own website as identity
|
- **IndieAuth authentication**: Use your own website as identity
|
||||||
- **Micropub support**: Full W3C Micropub specification compliance
|
- **Micropub support**: Full W3C Micropub specification compliance
|
||||||
- **RSS feed**: Automatic syndication
|
- **Media attachments**: Upload and display images with your notes
|
||||||
|
- **Microformats2**: Full h-entry, h-card, and h-feed markup for IndieWeb compatibility
|
||||||
|
- **Author discovery**: Automatic profile discovery from your IndieWeb identity
|
||||||
|
- **RSS, ATOM, JSON Feed**: Multiple syndication formats with Media RSS support
|
||||||
|
- **Custom slugs**: Control your note permalinks
|
||||||
- **No database lock-in**: SQLite for metadata, files for content
|
- **No database lock-in**: SQLite for metadata, files for content
|
||||||
- **Self-hostable**: Run on your own server
|
- **Self-hostable**: Run on your own server
|
||||||
- **Minimal dependencies**: 6 core dependencies, no build tools
|
- **Minimal dependencies**: Core dependencies, no build tools
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
@@ -154,8 +158,10 @@ See [docs/architecture/](docs/architecture/) for complete documentation.
|
|||||||
StarPunk implements:
|
StarPunk implements:
|
||||||
- [Micropub](https://micropub.spec.indieweb.org/) - Publishing API
|
- [Micropub](https://micropub.spec.indieweb.org/) - Publishing API
|
||||||
- [IndieAuth](https://www.w3.org/TR/indieauth/) - Authentication
|
- [IndieAuth](https://www.w3.org/TR/indieauth/) - Authentication
|
||||||
- [Microformats2](http://microformats.org/) - Semantic HTML markup
|
- [Microformats2](http://microformats.org/) - h-entry, h-card, h-feed markup
|
||||||
- [RSS 2.0](https://www.rssboard.org/rss-specification) - Feed syndication
|
- [RSS 2.0](https://www.rssboard.org/rss-specification) with Media RSS extensions
|
||||||
|
- [ATOM 1.0](https://validator.w3.org/feed/docs/atom.html) - Syndication format
|
||||||
|
- [JSON Feed 1.1](https://jsonfeed.org/version/1.1) - Modern feed format
|
||||||
|
|
||||||
## Deployment
|
## Deployment
|
||||||
|
|
||||||
|
|||||||
57
docs/DOCUMENTATION.md
Normal file
57
docs/DOCUMENTATION.md
Normal file
@@ -0,0 +1,57 @@
|
|||||||
|
# PURPOSE
|
||||||
|
|
||||||
|
This document describes how documentation in this folder should be organized and supersedes any other instructions.
|
||||||
|
|
||||||
|
# FOLDERS
|
||||||
|
|
||||||
|
## ARCHITECTURE
|
||||||
|
|
||||||
|
The architecture folder should contain documentation reflecting the current design of the system and should be updated at the end of each release to ensure it is current.
|
||||||
|
|
||||||
|
## DECISIONS
|
||||||
|
|
||||||
|
This folder contains any architectural decisions, documented as ADRs.
|
||||||
|
|
||||||
|
- Format: `ADR-NNN-brief-title.md` (numbered sequentially)
|
||||||
|
- Create an ADR when making architectural decisions, choosing between technical approaches, or establishing patterns
|
||||||
|
|
||||||
|
## DESIGN
|
||||||
|
|
||||||
|
This folder is used by the architect to document implementation designs to be handed off to the developer. These designs should be sorted into subfolders reflecting the semantic version number of the release in question (e.g., `v1.0.0/`, `v1.1.1/`).
|
||||||
|
|
||||||
|
All developer interaction belongs in the appropriate version subfolder:
|
||||||
|
- Implementation designs and specifications
|
||||||
|
- Developer questions to the architect
|
||||||
|
- Architect responses
|
||||||
|
- Implementation reports (format: `YYYY-MM-DD-description.md`)
|
||||||
|
- Implementation reviews
|
||||||
|
|
||||||
|
## PROJECTPLAN
|
||||||
|
|
||||||
|
This folder contains documents relating to the future state of the project. There should be a single BACKLOG.md file that lists future features by priority as well as bugs (which are assumed to be high priority). Items in this file can have one of the following priorities:
|
||||||
|
|
||||||
|
- Critical - Items that break existing functionality
|
||||||
|
- High
|
||||||
|
- Medium
|
||||||
|
- Low
|
||||||
|
|
||||||
|
In addition to the backlog file each version should have a folder named for its semantic version with a RELEASE.md file which lists the features and bugs to be addressed in that release.
|
||||||
|
|
||||||
|
## STANDARDS
|
||||||
|
|
||||||
|
Includes any standards written by the architect that the developer needs to reference during development. Any deprecated standards should be moved to the DEPRECATED subfolder when appropriate.
|
||||||
|
|
||||||
|
# WHERE TO FIND DOCUMENTATION
|
||||||
|
|
||||||
|
- **Before implementing a feature**: Check `decisions/` for relevant ADRs and `design/{version}/` for specifications
|
||||||
|
- **Understanding system architecture**: Start with `architecture/`
|
||||||
|
- **Coding guidelines**: See `standards/`
|
||||||
|
- **Past implementation context**: Review `design/{version}/` for similar work
|
||||||
|
- **Project roadmap and scope**: Refer to `projectplan/`
|
||||||
|
|
||||||
|
# KEY REFERENCES
|
||||||
|
|
||||||
|
- **Architecture**: `architecture/`
|
||||||
|
- **Coding Standards**: `standards/python-coding-standards.md`
|
||||||
|
- **Testing**: `standards/testing-checklist.md`
|
||||||
|
- **Project Backlog**: `projectplan/BACKLOG.md`
|
||||||
181
docs/decisions/ADR-012-flaky-test-removal.md
Normal file
181
docs/decisions/ADR-012-flaky-test-removal.md
Normal file
@@ -0,0 +1,181 @@
|
|||||||
|
# ADR-012: Flaky Test Removal and Test Quality Standards
|
||||||
|
|
||||||
|
## Status
|
||||||
|
Proposed
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The test suite contains several categories of flaky tests that pass inconsistently. These tests consume developer time without providing proportional value. Per the project philosophy ("Every line of code must justify its existence"), we must evaluate whether these tests should be kept, fixed, or removed.
|
||||||
|
|
||||||
|
## Analysis by Test Category
|
||||||
|
|
||||||
|
### 1. Migration Race Condition Tests (`test_migration_race_condition.py`)
|
||||||
|
|
||||||
|
**Failing Tests:**
|
||||||
|
- `test_debug_level_for_early_retries` - Log message matching
|
||||||
|
- `test_new_connection_per_retry` - Connection count assertions
|
||||||
|
- `test_concurrent_workers_barrier_sync` - Multiprocessing pickle errors
|
||||||
|
- `test_sequential_worker_startup` - Missing table errors
|
||||||
|
- `test_worker_late_arrival` - Missing table errors
|
||||||
|
- `test_single_worker_performance` - Missing table errors
|
||||||
|
- `test_concurrent_workers_performance` - Pickle errors
|
||||||
|
|
||||||
|
**Value Analysis:**
|
||||||
|
- The migration retry logic with exponential backoff is *critical* for production deployments with multiple Gunicorn workers
|
||||||
|
- However, the flaky tests are testing implementation details (log levels, exact connection counts) rather than behavior
|
||||||
|
- The multiprocessing tests fundamentally cannot work reliably because:
|
||||||
|
1. `multiprocessing.Manager().Barrier()` objects cannot be pickled for `Pool.map()`
|
||||||
|
2. The worker functions require Flask app context that doesn't transfer across processes
|
||||||
|
3. SQLite database files in temp directories may not be accessible across process boundaries
|
||||||
|
|
||||||
|
**Root Cause:** Test design is flawed. These are attempting integration/stress tests using unit test infrastructure.
|
||||||
|
|
||||||
|
**Recommendation: REMOVE the multiprocessing tests entirely. KEEP and FIX the unit tests.**
|
||||||
|
|
||||||
|
Specifically:
|
||||||
|
- **REMOVE:** `TestConcurrentExecution` class (all 3 tests) - fundamentally broken by design
|
||||||
|
- **REMOVE:** `TestPerformance` class (both tests) - same multiprocessing issues
|
||||||
|
- **KEEP:** `TestRetryLogic` - valuable, just needs mock fixes
|
||||||
|
- **KEEP:** `TestGraduatedLogging` - valuable, needs logger configuration fixes
|
||||||
|
- **KEEP:** `TestConnectionManagement` - valuable, needs assertion fixes
|
||||||
|
- **KEEP:** `TestErrorHandling` - valuable, tests critical rollback behavior
|
||||||
|
- **KEEP:** `TestBeginImmediateTransaction` - valuable, tests locking mechanism
|
||||||
|
|
||||||
|
**Rationale for removal:** If we need to test concurrent migration behavior, that requires:
|
||||||
|
1. A proper integration test framework (not pytest unit tests)
|
||||||
|
2. External process spawning (not multiprocessing.Pool)
|
||||||
|
3. Real filesystem isolation
|
||||||
|
4. This is out of scope for V1 - the code works; the tests are the problem
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2. Feed Route Tests (`test_routes_feeds.py`)
|
||||||
|
|
||||||
|
**Failing Assertions:**
|
||||||
|
- Tests checking for exact `<?xml version="1.0"` but code produces `<?xml version='1.0'` (single quotes)
|
||||||
|
- Tests checking for exact Content-Type with charset but response may vary
|
||||||
|
- Tests checking for exact `<rss version="2.0"` string
|
||||||
|
|
||||||
|
**Value Analysis:**
|
||||||
|
- These tests ARE valuable - they verify feed output format
|
||||||
|
- The tests are NOT flaky per se; they are *brittle* due to over-specific assertions
|
||||||
|
|
||||||
|
**Root Cause:** Tests are asserting implementation details (quote style) rather than semantics (valid XML).
|
||||||
|
|
||||||
|
**Recommendation: FIX by loosening assertions**
|
||||||
|
|
||||||
|
Current (brittle):
|
||||||
|
```python
|
||||||
|
assert b'<?xml version="1.0"' in response.data
|
||||||
|
```
|
||||||
|
|
||||||
|
Better (semantic):
|
||||||
|
```python
|
||||||
|
assert b'<?xml version=' in response.data
|
||||||
|
assert b"encoding=" in response.data
|
||||||
|
```
|
||||||
|
|
||||||
|
The test file already has SOME tests using the correct pattern (lines 72, 103, 265). The ATOM test on line 84 is the outlier - it should match the RSS tests.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3. Feed Streaming Test (`test_routes_feed.py`)
|
||||||
|
|
||||||
|
**Failing Test:** `test_feed_route_streaming`
|
||||||
|
|
||||||
|
**Current assertion (line 124):**
|
||||||
|
```python
|
||||||
|
assert "ETag" in response.headers
|
||||||
|
```
|
||||||
|
|
||||||
|
**But the test comment says:**
|
||||||
|
```python
|
||||||
|
# Cached responses include ETags for conditional requests
|
||||||
|
# (Phase 3 caching was added, replacing streaming for better performance)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Value Analysis:**
|
||||||
|
- The test title says "streaming" but the implementation uses caching
|
||||||
|
- The test is actually correct (ETag SHOULD be present)
|
||||||
|
- If ETag is NOT present, that's a bug in the feed caching implementation
|
||||||
|
|
||||||
|
**Root Cause:** This is not a flaky test - if it fails, there's an actual bug. The test name is misleading but the assertion is correct.
|
||||||
|
|
||||||
|
**Recommendation: KEEP and RENAME to `test_feed_route_caching`**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 4. Search Security Tests (`test_search_security.py`)
|
||||||
|
|
||||||
|
**Analysis of the file:** After reviewing, I see no obviously flaky tests. All tests are:
|
||||||
|
- Testing XSS prevention (correct)
|
||||||
|
- Testing SQL injection prevention (correct)
|
||||||
|
- Testing input validation (correct)
|
||||||
|
- Testing pagination limits (correct)
|
||||||
|
|
||||||
|
**Possible flakiness sources:**
|
||||||
|
- FTS5 special character handling varies by SQLite version
|
||||||
|
- Tests that accept multiple status codes (200, 400, 500) are defensive, not flaky
|
||||||
|
|
||||||
|
**Recommendation: KEEP all tests**
|
||||||
|
|
||||||
|
If specific flakiness is identified, it's likely due to SQLite FTS5 version differences, which should be documented rather than the tests removed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
### Remove Entirely
|
||||||
|
1. `TestConcurrentExecution` class from `test_migration_race_condition.py`
|
||||||
|
2. `TestPerformance` class from `test_migration_race_condition.py`
|
||||||
|
|
||||||
|
### Fix Tests (Developer Action Items)
|
||||||
|
1. **`test_routes_feeds.py` line 84:** Change `assert b'<?xml version="1.0"'` to `assert b'<?xml version='`
|
||||||
|
2. **`test_routes_feed.py` line 117:** Rename test from `test_feed_route_streaming` to `test_feed_route_caching`
|
||||||
|
3. **`test_migration_race_condition.py`:** Fix logger configuration in `TestGraduatedLogging` tests to ensure DEBUG level is captured
|
||||||
|
4. **`test_migration_race_condition.py`:** Fix mock setup in `test_new_connection_per_retry` to accurately count connection attempts
|
||||||
|
|
||||||
|
### Keep As-Is
|
||||||
|
1. All tests in `test_search_security.py`
|
||||||
|
2. All non-multiprocessing tests in `test_migration_race_condition.py` (after fixes)
|
||||||
|
3. All other tests in `test_routes_feeds.py` and `test_routes_feed.py`
|
||||||
|
|
||||||
|
## Rationale
|
||||||
|
|
||||||
|
1. **Project philosophy alignment:** Tests that cannot reliably pass do not justify their existence. They waste developer time and erode confidence in the test suite.
|
||||||
|
|
||||||
|
2. **Pragmatic approach:** The migration concurrency code is tested in production by virtue of running with multiple Gunicorn workers. Manual testing during deployment is more reliable than broken multiprocessing tests.
|
||||||
|
|
||||||
|
3. **Test semantics over implementation:** Tests should verify behavior, not implementation details like quote styles in XML.
|
||||||
|
|
||||||
|
4. **Maintainability:** A smaller, reliable test suite is better than a larger, flaky one.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
### Positive
|
||||||
|
- Faster, more reliable CI/CD pipeline
|
||||||
|
- Increased developer confidence in test results
|
||||||
|
- Reduced time spent debugging test infrastructure
|
||||||
|
- Tests that fail actually indicate bugs
|
||||||
|
|
||||||
|
### Negative
|
||||||
|
- Reduced test coverage for concurrent migration scenarios
|
||||||
|
- Manual testing required for multi-worker deployments
|
||||||
|
|
||||||
|
### Mitigations
|
||||||
|
- Document the multi-worker testing procedure in deployment docs
|
||||||
|
- Consider adding integration tests in a separate test category (not run in CI) for concurrent scenarios
|
||||||
|
|
||||||
|
## Alternatives Considered
|
||||||
|
|
||||||
|
### 1. Fix the multiprocessing tests
|
||||||
|
**Rejected:** Would require significant refactoring to use subprocess spawning instead of multiprocessing.Pool. The complexity is not justified for V1 given the code works correctly in production.
|
||||||
|
|
||||||
|
### 2. Mark tests as `@pytest.mark.skip`
|
||||||
|
**Rejected:** Skipped tests are just noise. They either work and should run, or they don't work and should be removed. "Skip" is procrastination.
|
||||||
|
|
||||||
|
### 3. Use pytest-xdist for parallel testing
|
||||||
|
**Rejected:** Does not solve the fundamental issue of needing to spawn external processes with proper app context.
|
||||||
|
|
||||||
|
### 4. Move to integration test framework (e.g., testcontainers)
|
||||||
|
**Considered for future:** This is the correct long-term solution but is out of scope for V1. Should be considered for V2 if concurrent migration testing is deemed critical.
|
||||||
281
docs/decisions/ADR-059-full-feed-media-standardization.md
Normal file
281
docs/decisions/ADR-059-full-feed-media-standardization.md
Normal file
@@ -0,0 +1,281 @@
|
|||||||
|
# ADR-059: Full Feed Media Standardization (Option 3)
|
||||||
|
|
||||||
|
## Status
|
||||||
|
Proposed (For v1.3.0 Backlog)
|
||||||
|
|
||||||
|
## Context
|
||||||
|
StarPunk v1.2.0 introduced media attachments for notes (images). The initial implementation embeds media as HTML in feed description fields. Option 2 (implemented in v1.2.x) adds Media RSS extension elements and JSON Feed image fields for better feed reader compatibility.
|
||||||
|
|
||||||
|
This ADR documents Option 3: Full Standardization, which provides comprehensive media support across all syndication formats, including video, audio, and advanced features. This is planned for v1.3.0 or later.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
Document the scope of "Full Standardization" for feed media support to be implemented in a future release. This option goes beyond Option 2's basic Media RSS support to include:
|
||||||
|
|
||||||
|
1. **Complete Media RSS Specification Support**
|
||||||
|
2. **Podcast RSS Support (RSS 2.0 enclosures for audio)**
|
||||||
|
3. **Video Support**
|
||||||
|
4. **Multiple Image Sizes/Thumbnails**
|
||||||
|
5. **Full JSON Feed 1.1 Media Compliance**
|
||||||
|
|
||||||
|
## Scope of Full Standardization
|
||||||
|
|
||||||
|
### 1. Complete Media RSS Implementation
|
||||||
|
|
||||||
|
**Research Required**: Full Media RSS specification at https://www.rssboard.org/media-rss
|
||||||
|
|
||||||
|
**Elements to Implement**:
|
||||||
|
- `<media:content>` with full attribute support:
|
||||||
|
- `url` (required) - Direct URL to media file
|
||||||
|
- `fileSize` - Size in bytes
|
||||||
|
- `type` - MIME type
|
||||||
|
- `medium` - Type: "image", "audio", "video", "document", "executable"
|
||||||
|
- `isDefault` - Boolean for default rendition
|
||||||
|
- `expression` - "full", "sample", "nonstop"
|
||||||
|
- `bitrate` - Kilobits per second
|
||||||
|
- `framerate` - Frames per second (video)
|
||||||
|
- `samplingrate` - Samples per second (audio)
|
||||||
|
- `channels` - Audio channels
|
||||||
|
- `duration` - Seconds
|
||||||
|
- `height` / `width` - Dimensions in pixels
|
||||||
|
- `lang` - RFC-3066 language code
|
||||||
|
|
||||||
|
- `<media:group>` - Container for multiple renditions of same content
|
||||||
|
- `<media:thumbnail>` - Multiple sizes with url, width, height, time
|
||||||
|
- `<media:title>` - Media title (type="plain" or "html")
|
||||||
|
- `<media:description>` - Media description (type="plain" or "html")
|
||||||
|
- `<media:keywords>` - Comma-separated keywords
|
||||||
|
- `<media:category>` - Categorization with scheme attribute
|
||||||
|
- `<media:credit>` - Credit attribution with role and scheme
|
||||||
|
- `<media:copyright>` - Copyright information
|
||||||
|
- `<media:rating>` - Content rating (scheme-based)
|
||||||
|
- `<media:hash>` - MD5/SHA-1 hash for integrity
|
||||||
|
- `<media:player>` - Embeddable player URL
|
||||||
|
|
||||||
|
**Effort Estimate**: 8-12 hours
|
||||||
|
|
||||||
|
### 2. Podcast RSS Support
|
||||||
|
|
||||||
|
**Research Required**:
|
||||||
|
- Apple Podcast RSS specification
|
||||||
|
- Google Podcast RSS requirements
|
||||||
|
- Podcast Index namespace (podcast:)
|
||||||
|
|
||||||
|
**Elements to Implement**:
|
||||||
|
- iTunes namespace (`xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"`):
|
||||||
|
- `<itunes:summary>` - Episode summary
|
||||||
|
- `<itunes:duration>` - Audio duration (HH:MM:SS)
|
||||||
|
- `<itunes:image>` - Episode artwork
|
||||||
|
- `<itunes:explicit>` - Content rating
|
||||||
|
- `<itunes:episode>` - Episode number
|
||||||
|
- `<itunes:season>` - Season number
|
||||||
|
- `<itunes:episodeType>` - "full", "trailer", "bonus"
|
||||||
|
- `<itunes:author>` - Author name
|
||||||
|
- `<itunes:owner>` - Owner contact
|
||||||
|
|
||||||
|
- Standard RSS `<enclosure>` for audio:
|
||||||
|
- `url` - Direct audio file URL
|
||||||
|
- `length` - File size in bytes
|
||||||
|
- `type` - MIME type (audio/mpeg, audio/mp4, etc.)
|
||||||
|
|
||||||
|
**Database Changes**:
|
||||||
|
- Add `duration` column to `note_media` table
|
||||||
|
- Add `media_type` enum (image, audio, video)
|
||||||
|
- Consider `podcast_metadata` table for series-level data
|
||||||
|
|
||||||
|
**Effort Estimate**: 10-16 hours
|
||||||
|
|
||||||
|
### 3. Video Support
|
||||||
|
|
||||||
|
**Research Required**:
|
||||||
|
- Video hosting considerations (storage, bandwidth)
|
||||||
|
- Supported formats (mp4, webm, ogg)
|
||||||
|
- Transcoding requirements
|
||||||
|
- Poster image generation
|
||||||
|
|
||||||
|
**Implementation Scope**:
|
||||||
|
- Accept video uploads via Micropub media endpoint
|
||||||
|
- Generate poster thumbnails automatically
|
||||||
|
- Include in Media RSS with proper video attributes:
|
||||||
|
- `medium="video"`
|
||||||
|
- `framerate`, `duration`, `bitrate`
|
||||||
|
- Associated `<media:thumbnail>` for poster
|
||||||
|
|
||||||
|
- HTML5 `<video>` element in feed description
|
||||||
|
- Consider video hosting limits (file size, duration)
|
||||||
|
|
||||||
|
**Database Changes**:
|
||||||
|
- Video-specific metadata in `media` table
|
||||||
|
- Poster image path
|
||||||
|
- Transcoding status (if needed)
|
||||||
|
|
||||||
|
**Effort Estimate**: 16-24 hours (significant)
|
||||||
|
|
||||||
|
### 4. Multiple Image Sizes (Thumbnails)
|
||||||
|
|
||||||
|
**Research Required**:
|
||||||
|
- Responsive image best practices
|
||||||
|
- WebP generation
|
||||||
|
- srcset/sizes patterns
|
||||||
|
|
||||||
|
**Implementation Scope**:
|
||||||
|
- Generate multiple sizes on upload:
|
||||||
|
- Thumbnail: 150x150 (square crop)
|
||||||
|
- Small: 320px width
|
||||||
|
- Medium: 640px width
|
||||||
|
- Large: 1280px width
|
||||||
|
- Original: preserved
|
||||||
|
|
||||||
|
- Store all sizes in `media_variants` table
|
||||||
|
- Include in Media RSS:
|
||||||
|
```xml
|
||||||
|
<media:group>
|
||||||
|
<media:content url="large.jpg" isDefault="true" width="1280" />
|
||||||
|
<media:content url="medium.jpg" width="640" />
|
||||||
|
<media:content url="small.jpg" width="320" />
|
||||||
|
</media:group>
|
||||||
|
<media:thumbnail url="thumb.jpg" width="150" height="150" />
|
||||||
|
```
|
||||||
|
|
||||||
|
- JSON Feed: Use `image` for default, include variants in `_starpunk` extension
|
||||||
|
|
||||||
|
**Database Changes**:
|
||||||
|
- `media_variants` table: media_id, variant_type, path, width, height, size_bytes
|
||||||
|
- Add `has_variants` boolean to `media` table
|
||||||
|
|
||||||
|
**Effort Estimate**: 8-12 hours
|
||||||
|
|
||||||
|
### 5. Full JSON Feed 1.1 Media Compliance
|
||||||
|
|
||||||
|
**Research Required**: JSON Feed 1.1 specification for extensions
|
||||||
|
|
||||||
|
**Implementation Scope**:
|
||||||
|
- Top-level `image` field (URL of first image, per spec)
|
||||||
|
- Top-level `banner_image` if applicable
|
||||||
|
- Item-level `image` field (main/featured image)
|
||||||
|
- Item-level `banner_image` for posts with banners
|
||||||
|
- Complete `attachments` array:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"url": "https://example.com/media/image.jpg",
|
||||||
|
"mime_type": "image/jpeg",
|
||||||
|
"title": "Image caption",
|
||||||
|
"size_in_bytes": 245760,
|
||||||
|
"duration_in_seconds": null
|
||||||
|
}
|
||||||
|
```
|
||||||
|
- Audio attachments with `duration_in_seconds`
|
||||||
|
- Video attachments (if supported)
|
||||||
|
|
||||||
|
**Effort Estimate**: 4-6 hours
|
||||||
|
|
||||||
|
### 6. ATOM Feed Media Extensions
|
||||||
|
|
||||||
|
**Research Required**:
|
||||||
|
- ATOM Media extension namespace
|
||||||
|
- `<link rel="enclosure">` best practices
|
||||||
|
|
||||||
|
**Implementation Scope**:
|
||||||
|
- `<link rel="enclosure">` for each media item
|
||||||
|
- `type` attribute with MIME type
|
||||||
|
- `length` attribute with file size
|
||||||
|
- `title` attribute with caption
|
||||||
|
- Consider `<link rel="related">` for thumbnails
|
||||||
|
|
||||||
|
**Effort Estimate**: 3-5 hours
|
||||||
|
|
||||||
|
## Total Effort Estimate
|
||||||
|
|
||||||
|
| Feature | Minimum | Maximum |
|
||||||
|
|---------|---------|---------|
|
||||||
|
| Complete Media RSS | 8 hours | 12 hours |
|
||||||
|
| Podcast RSS Support | 10 hours | 16 hours |
|
||||||
|
| Video Support | 16 hours | 24 hours |
|
||||||
|
| Multiple Image Sizes | 8 hours | 12 hours |
|
||||||
|
| JSON Feed Compliance | 4 hours | 6 hours |
|
||||||
|
| ATOM Extensions | 3 hours | 5 hours |
|
||||||
|
| **Total** | **49 hours** | **75 hours** |
|
||||||
|
|
||||||
|
**Note**: Video support is the most complex feature and could be deferred to v1.4.0 "Media" release.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Before implementing Full Standardization:
|
||||||
|
|
||||||
|
1. **Option 2 Complete**: Basic Media RSS and JSON Feed `image` field
|
||||||
|
2. **Image Optimization**: ADR-058 image optimization strategy implemented
|
||||||
|
3. **Media Storage Architecture**: Clear path for large file storage
|
||||||
|
4. **Test Infrastructure**: Feed validation tests in place
|
||||||
|
|
||||||
|
## Implementation Phases
|
||||||
|
|
||||||
|
### Phase A: Enhanced Image Support (v1.3.0)
|
||||||
|
- Multiple image sizes/thumbnails
|
||||||
|
- Full Media RSS for images
|
||||||
|
- Enhanced JSON Feed attachments
|
||||||
|
- **Effort**: 12-18 hours
|
||||||
|
|
||||||
|
### Phase B: Audio Support (v1.3.x or v1.4.0)
|
||||||
|
- Podcast RSS implementation
|
||||||
|
- Audio duration extraction
|
||||||
|
- iTunes namespace
|
||||||
|
- **Effort**: 10-16 hours
|
||||||
|
|
||||||
|
### Phase C: Video Support (v1.4.0 "Media")
|
||||||
|
- Video upload handling
|
||||||
|
- Poster generation
|
||||||
|
- Video in feeds
|
||||||
|
- **Effort**: 16-24 hours
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
### Positive
|
||||||
|
- Best-in-class feed reader compatibility
|
||||||
|
- Podcast distribution capability
|
||||||
|
- Video content support
|
||||||
|
- Professional media syndication
|
||||||
|
- Future-proof architecture
|
||||||
|
|
||||||
|
### Negative
|
||||||
|
- Significant implementation effort (50-75 hours total)
|
||||||
|
- Increased storage requirements
|
||||||
|
- More complex feed generation
|
||||||
|
- Processing overhead for image variants
|
||||||
|
- Larger codebase to maintain
|
||||||
|
|
||||||
|
### Neutral
|
||||||
|
- Aligns with media-focused v1.4.0 roadmap
|
||||||
|
- Phased implementation possible
|
||||||
|
- Optional features can be configuration-gated
|
||||||
|
|
||||||
|
## Alternatives Considered
|
||||||
|
|
||||||
|
### Alternative 1: Minimal Enhancement (Option 2 Only)
|
||||||
|
Just implement basic Media RSS and JSON Feed image field.
|
||||||
|
- **Pros**: Low effort, immediate benefit
|
||||||
|
- **Cons**: Misses podcast/video opportunity
|
||||||
|
|
||||||
|
### Alternative 2: Third-Party Media Service
|
||||||
|
Use external service (Cloudinary, etc.) for media processing.
|
||||||
|
- **Pros**: Offloads complexity
|
||||||
|
- **Cons**: External dependency, cost, data ownership concerns
|
||||||
|
|
||||||
|
### Alternative 3: Plugin Architecture
|
||||||
|
Make media support pluggable for advanced features.
|
||||||
|
- **Pros**: Keeps core simple
|
||||||
|
- **Cons**: Added architectural complexity
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [Media RSS Specification](https://www.rssboard.org/media-rss)
|
||||||
|
- [JSON Feed 1.1 Specification](https://jsonfeed.org/version/1.1)
|
||||||
|
- [Apple Podcast RSS Requirements](https://podcasters.apple.com/support/823-podcast-requirements)
|
||||||
|
- [Podcast Index Namespace](https://github.com/Podcastindex-org/podcast-namespace)
|
||||||
|
- [RSS 2.0 Enclosure Specification](https://www.rssboard.org/rss-specification#ltenclosuregtSubelementOfLtitemgt)
|
||||||
|
- [ADR-057: Media Attachment Model](/home/phil/Projects/starpunk/docs/decisions/ADR-057-media-attachment-model.md)
|
||||||
|
- [ADR-058: Image Optimization Strategy](/home/phil/Projects/starpunk/docs/decisions/ADR-058-image-optimization-strategy.md)
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
This ADR documents the scope of Full Standardization (Option 3) for the project backlog. Implementation should be scheduled for v1.3.0 and v1.4.0 releases according to the phased approach outlined above.
|
||||||
|
|
||||||
|
**Immediate Action**: Implement Option 2 (ADR-060) for v1.2.x release.
|
||||||
|
**Future Action**: Review and refine this scope when scheduling v1.3.0 work.
|
||||||
197
docs/decisions/ADR-062-timestamp-based-slug-format.md
Normal file
197
docs/decisions/ADR-062-timestamp-based-slug-format.md
Normal file
@@ -0,0 +1,197 @@
|
|||||||
|
# ADR-062: Timestamp-Based Slug Format
|
||||||
|
|
||||||
|
## Status
|
||||||
|
Accepted (Supersedes ADR-007)
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
ADR-007 established a content-based slug generation algorithm that extracts the first 5 words from note content to create URL slugs. While this approach provides readable, SEO-friendly URLs, it has drawbacks for a personal note-taking system:
|
||||||
|
|
||||||
|
1. **Privacy Concerns**: The slug reveals note content in the URL. Private thoughts or draft content become visible in URLs that may be shared or logged.
|
||||||
|
|
||||||
|
2. **SEO Irrelevance**: StarPunk is designed for personal IndieWeb notes, not public blogs. Notes are typically short-form content (similar to tweets or status updates) where SEO optimization provides no meaningful benefit.
|
||||||
|
|
||||||
|
3. **Unpredictable Slugs**: Users cannot predict what slug will be generated without examining their content carefully.
|
||||||
|
|
||||||
|
4. **Edge Case Handling**: Content-based slugs require complex fallback logic for short content, unicode-only content, or special characters.
|
||||||
|
|
||||||
|
5. **Collision Complexity**: Similar notes require random suffix generation (e.g., `hello-world-a7c9`), which undermines the readability goal.
|
||||||
|
|
||||||
|
The user has explicitly stated: "Notes don't need SEO."
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Change the default slug format from content-based to timestamp-based:
|
||||||
|
|
||||||
|
**New Default Format**: `YYYYMMDDHHMMSS`
|
||||||
|
- Example: `20251216143052`
|
||||||
|
- Compact, sortable, predictable
|
||||||
|
- 14 characters total
|
||||||
|
|
||||||
|
**Collision Handling**: Sequential numeric suffix
|
||||||
|
- First collision: `20251216143052-1`
|
||||||
|
- Second collision: `20251216143052-2`
|
||||||
|
- Simple, predictable, no randomness
|
||||||
|
|
||||||
|
**Custom Slugs**: Continue to support user-specified slugs via `mp-slug` Micropub property and the web UI custom slug field. When provided, custom slugs take precedence.
|
||||||
|
|
||||||
|
### Algorithm Specification
|
||||||
|
|
||||||
|
```python
|
||||||
|
def generate_slug(custom_slug: str = None, created_at: datetime = None) -> str:
|
||||||
|
"""Generate a URL-safe slug for a note.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
custom_slug: User-provided custom slug (takes precedence if provided)
|
||||||
|
created_at: Note creation timestamp (defaults to now)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
URL-safe slug string
|
||||||
|
"""
|
||||||
|
if custom_slug:
|
||||||
|
return sanitize_slug(custom_slug)
|
||||||
|
|
||||||
|
# Default: timestamp-based
|
||||||
|
timestamp = (created_at or datetime.now()).strftime("%Y%m%d%H%M%S")
|
||||||
|
return ensure_unique_slug(timestamp)
|
||||||
|
|
||||||
|
def ensure_unique_slug(base_slug: str) -> str:
|
||||||
|
"""Ensure slug is unique, adding numeric suffix if needed."""
|
||||||
|
if not slug_exists(base_slug):
|
||||||
|
return base_slug
|
||||||
|
|
||||||
|
suffix = 1
|
||||||
|
while slug_exists(f"{base_slug}-{suffix}"):
|
||||||
|
suffix += 1
|
||||||
|
return f"{base_slug}-{suffix}"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Examples
|
||||||
|
|
||||||
|
| Scenario | Generated Slug |
|
||||||
|
|----------|----------------|
|
||||||
|
| Normal note at 2:30:52 PM on Dec 16, 2025 | `20251216143052` |
|
||||||
|
| Second note in same second | `20251216143052-1` |
|
||||||
|
| Third note in same second | `20251216143052-2` |
|
||||||
|
| Note with custom slug "my-custom-slug" | `my-custom-slug` |
|
||||||
|
|
||||||
|
## Rationale
|
||||||
|
|
||||||
|
### Timestamp Format (Score: 9/10)
|
||||||
|
|
||||||
|
**Pros**:
|
||||||
|
- **Privacy**: URL reveals nothing about content
|
||||||
|
- **Predictability**: User knows exactly what slug format to expect
|
||||||
|
- **Sortability**: Chronological sorting by URL is possible
|
||||||
|
- **Simplicity**: No complex word extraction or normalization
|
||||||
|
- **Collision Rarity**: Same-second creation is rare; handled cleanly when it occurs
|
||||||
|
- **Compact**: 14 characters vs potentially 50+ for content-based
|
||||||
|
|
||||||
|
**Cons**:
|
||||||
|
- **Not Memorable**: `20251216143052` is harder to remember than `hello-world`
|
||||||
|
- **No SEO Value**: Search engines prefer descriptive URLs
|
||||||
|
|
||||||
|
**Why Cons Don't Matter**:
|
||||||
|
- StarPunk is for personal notes, not public SEO-optimized content
|
||||||
|
- Notes are accessed via UI, feeds, or bookmarks, not memorized URLs
|
||||||
|
- Users wanting memorable URLs can use custom slugs
|
||||||
|
|
||||||
|
### Sequential Suffix (Score: 10/10)
|
||||||
|
|
||||||
|
**Pros**:
|
||||||
|
- **Deterministic**: No randomness; same collision always gets same suffix
|
||||||
|
- **Simple**: No cryptographic random generation needed
|
||||||
|
- **Readable**: `-1`, `-2` are clear and obvious
|
||||||
|
- **Debuggable**: Easy to understand collision resolution
|
||||||
|
|
||||||
|
**Cons**:
|
||||||
|
- **Enumerable**: Sequential numbers could be probed
|
||||||
|
- Not a real security concern for note slugs
|
||||||
|
|
||||||
|
### Comparison with ADR-007 Approach
|
||||||
|
|
||||||
|
| Aspect | ADR-007 (Content-Based) | ADR-062 (Timestamp) |
|
||||||
|
|--------|------------------------|---------------------|
|
||||||
|
| Privacy | Reveals content | Reveals only time |
|
||||||
|
| Complexity | High (word extraction, normalization, unicode handling) | Low (strftime) |
|
||||||
|
| SEO | Good | None |
|
||||||
|
| Predictability | Low | High |
|
||||||
|
| Collision handling | Random suffix | Sequential suffix |
|
||||||
|
| Fallback cases | Many (short content, unicode, etc.) | None |
|
||||||
|
| Code lines | ~50 | ~15 |
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
### Positive
|
||||||
|
|
||||||
|
1. **Simplified Code**: Remove complex word extraction, unicode normalization, and multiple fallback paths
|
||||||
|
2. **Better Privacy**: Note content never appears in URLs
|
||||||
|
3. **Predictable Output**: Users always know what slug format to expect
|
||||||
|
4. **Fewer Edge Cases**: No special handling for short content, unicode, or special characters
|
||||||
|
5. **Cleaner Collisions**: Sequential suffixes are more intuitive than random strings
|
||||||
|
|
||||||
|
### Negative
|
||||||
|
|
||||||
|
1. **Migration**: Existing notes keep their content-based slugs (no migration needed)
|
||||||
|
2. **Not Human-Readable**: URLs don't describe content
|
||||||
|
3. **No SEO**: Search engines won't benefit from descriptive URLs
|
||||||
|
|
||||||
|
### Mitigations
|
||||||
|
|
||||||
|
**Human-Readable URLs**:
|
||||||
|
- Users wanting descriptive URLs can use custom slugs via `mp-slug`
|
||||||
|
- The web UI custom slug field remains available
|
||||||
|
- This is opt-in rather than default
|
||||||
|
|
||||||
|
**SEO**:
|
||||||
|
- IndieWeb notes are typically not SEO targets
|
||||||
|
- Content is in the page body where search engines can index it
|
||||||
|
- Microformats2 markup provides semantic meaning
|
||||||
|
|
||||||
|
## Backward Compatibility
|
||||||
|
|
||||||
|
- Existing notes retain their slugs (no data migration)
|
||||||
|
- New notes use timestamp format by default
|
||||||
|
- Custom slug functionality unchanged
|
||||||
|
- All existing URLs remain valid
|
||||||
|
|
||||||
|
## Testing Requirements
|
||||||
|
|
||||||
|
- Test default slug generation produces timestamp format
|
||||||
|
- Test collision handling with sequential suffixes
|
||||||
|
- Test custom slugs still take precedence
|
||||||
|
- Test edge case: multiple notes in same second
|
||||||
|
- Test reserved slug rejection still works
|
||||||
|
- Verify existing tests for custom slugs pass
|
||||||
|
|
||||||
|
## Implementation Notes
|
||||||
|
|
||||||
|
Changes required in:
|
||||||
|
- `starpunk/slug_utils.py`: Update `generate_slug()` function
|
||||||
|
- `starpunk/notes.py`: Remove content parameter from slug generation call
|
||||||
|
- Tests: Update expected slug formats
|
||||||
|
|
||||||
|
Estimated effort: Small (1-2 hours implementation, 1 hour testing)
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- ADR-007: Slug Generation Algorithm (Superseded by this ADR)
|
||||||
|
- ADR-035: Custom Slugs (Unchanged; complements this decision)
|
||||||
|
- IndieWeb Permalink Best Practices: https://indieweb.org/permalink
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- [ ] Default slugs use `YYYYMMDDHHMMSS` format
|
||||||
|
- [ ] Collision handling uses sequential suffix (`-1`, `-2`, etc.)
|
||||||
|
- [ ] Custom slugs via `mp-slug` continue to work
|
||||||
|
- [ ] Custom slugs via web UI continue to work
|
||||||
|
- [ ] Reserved slug validation unchanged
|
||||||
|
- [ ] Existing notes unaffected
|
||||||
|
- [ ] All tests pass
|
||||||
|
- [ ] Code complexity reduced
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Approved**: 2025-12-16
|
||||||
|
**Architect**: StarPunk Architect Agent
|
||||||
|
**Supersedes**: ADR-007 (Slug Generation Algorithm)
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
# Deployment Documentation Index
|
|
||||||
|
|
||||||
This directory contains deployment guides, infrastructure setup instructions, and operations documentation for StarPunk CMS.
|
|
||||||
|
|
||||||
## Deployment Guides
|
|
||||||
|
|
||||||
- **[container-deployment.md](container-deployment.md)** - Container-based deployment guide (Docker, Podman)
|
|
||||||
|
|
||||||
## Deployment Options
|
|
||||||
|
|
||||||
### Container Deployment (Recommended)
|
|
||||||
Container deployment provides:
|
|
||||||
- Consistent environment across platforms
|
|
||||||
- Easy updates and rollbacks
|
|
||||||
- Resource isolation
|
|
||||||
- Simplified dependency management
|
|
||||||
|
|
||||||
See: [container-deployment.md](container-deployment.md)
|
|
||||||
|
|
||||||
### Manual Deployment
|
|
||||||
For manual deployment without containers:
|
|
||||||
- Follow [../standards/development-setup.md](../standards/development-setup.md)
|
|
||||||
- Configure systemd service
|
|
||||||
- Set up reverse proxy (nginx/Caddy)
|
|
||||||
- Configure SSL/TLS certificates
|
|
||||||
|
|
||||||
### Cloud Deployment
|
|
||||||
StarPunk can be deployed to:
|
|
||||||
- Any container platform (Kubernetes, Docker Swarm)
|
|
||||||
- VPS providers (DigitalOcean, Linode, Vultr)
|
|
||||||
- PaaS platforms supporting containers
|
|
||||||
|
|
||||||
## Related Documentation
|
|
||||||
- **[../standards/development-setup.md](../standards/development-setup.md)** - Development environment setup
|
|
||||||
- **[../architecture/](../architecture/)** - System architecture
|
|
||||||
- **[README.md](../../README.md)** - Quick start guide
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Last Updated**: 2025-11-25
|
|
||||||
**Maintained By**: Documentation Manager Agent
|
|
||||||
@@ -0,0 +1,173 @@
|
|||||||
|
# IndieAuth Endpoint Discovery Hotfix - Implementation Report
|
||||||
|
|
||||||
|
**Date:** 2025-12-17
|
||||||
|
**Type:** Production Hotfix
|
||||||
|
**Priority:** Critical
|
||||||
|
**Status:** Implementation Complete
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Successfully implemented the IndieAuth endpoint discovery hotfix as specified in the design document. The authentication flow now correctly discovers endpoints from the user's profile URL per the W3C IndieAuth specification, instead of hardcoding the indielogin.com service.
|
||||||
|
|
||||||
|
## Implementation Steps
|
||||||
|
|
||||||
|
All implementation steps were completed successfully:
|
||||||
|
|
||||||
|
### Step 1: Update starpunk/config.py - Remove INDIELOGIN_URL
|
||||||
|
- Removed `INDIELOGIN_URL` config line (previously line 37)
|
||||||
|
- Added deprecation warning for users still setting `INDIELOGIN_URL` in environment
|
||||||
|
- Warning directs users to remove the deprecated config
|
||||||
|
|
||||||
|
### Step 2: Update starpunk/auth.py - Add imports and use endpoint discovery
|
||||||
|
- Added imports: `discover_endpoints`, `DiscoveryError`, `normalize_url` from `starpunk.auth_external`
|
||||||
|
- Rewrote `initiate_login()`:
|
||||||
|
- Now discovers authorization_endpoint from the user's profile URL
|
||||||
|
- Uses discovered endpoint instead of hardcoded INDIELOGIN_URL
|
||||||
|
- Raises DiscoveryError if endpoint discovery fails or no authorization_endpoint found
|
||||||
|
- Rewrote `handle_callback()`:
|
||||||
|
- Discovers authorization_endpoint from ADMIN_ME profile
|
||||||
|
- Uses authorization_endpoint for authentication-only flow (per IndieAuth spec)
|
||||||
|
- Does NOT include `grant_type` parameter (not needed for auth-only flows)
|
||||||
|
- Uses `normalize_url()` for URL comparison to handle trailing slashes and case differences
|
||||||
|
|
||||||
|
### Step 3: Update starpunk/auth_external.py - Relax endpoint validation
|
||||||
|
- Changed endpoint validation in `_fetch_and_parse()`:
|
||||||
|
- Now requires at least one endpoint (authorization_endpoint OR token_endpoint)
|
||||||
|
- Previously required token_endpoint to be present
|
||||||
|
- This allows profiles with only authorization_endpoint to work for login
|
||||||
|
- Micropub will still require token_endpoint and fail gracefully with 401
|
||||||
|
|
||||||
|
### Step 4: Update starpunk/routes/auth.py - Import and handle DiscoveryError
|
||||||
|
- Added import for `DiscoveryError` from `starpunk.auth_external`
|
||||||
|
- Added exception handler in `login_initiate()`:
|
||||||
|
- Catches DiscoveryError
|
||||||
|
- Logs technical details at ERROR level
|
||||||
|
- Shows user-friendly message: "Unable to verify your profile URL. Please check that it's correct and try again."
|
||||||
|
- Redirects back to login form
|
||||||
|
|
||||||
|
### Step 5: Update tests/test_auth.py - Mock discover_endpoints()
|
||||||
|
- Removed `INDIELOGIN_URL` from test app fixture
|
||||||
|
- Updated all tests that call `initiate_login()` or `handle_callback()`:
|
||||||
|
- Added `@patch("starpunk.auth.discover_endpoints")` decorator
|
||||||
|
- Mock returns both authorization_endpoint and token_endpoint
|
||||||
|
- Updated assertions to check for discovered endpoint instead of indielogin.com
|
||||||
|
- Tests updated:
|
||||||
|
- `TestInitiateLogin.test_initiate_login_success`
|
||||||
|
- `TestInitiateLogin.test_initiate_login_stores_state`
|
||||||
|
- `TestHandleCallback.test_handle_callback_success`
|
||||||
|
- `TestHandleCallback.test_handle_callback_unauthorized_user`
|
||||||
|
- `TestHandleCallback.test_handle_callback_indielogin_error`
|
||||||
|
- `TestHandleCallback.test_handle_callback_no_identity`
|
||||||
|
- `TestLoggingIntegration.test_initiate_login_logs_at_debug`
|
||||||
|
- `TestLoggingIntegration.test_initiate_login_info_level`
|
||||||
|
- `TestLoggingIntegration.test_handle_callback_logs_http_details`
|
||||||
|
|
||||||
|
### Step 6: Update tests/test_auth_external.py - Fix error message
|
||||||
|
- Updated `test_discover_endpoints_no_token_endpoint`:
|
||||||
|
- Changed assertion from "No token endpoint found" to "No IndieAuth endpoints found"
|
||||||
|
- Matches new relaxed validation error message
|
||||||
|
|
||||||
|
### Step 7: Run tests to verify implementation
|
||||||
|
- All 51 tests in `tests/test_auth.py` pass
|
||||||
|
- All 35 tests in `tests/test_auth_external.py` pass
|
||||||
|
- All 32 tests in `tests/test_routes_admin.py` pass
|
||||||
|
- No regressions detected
|
||||||
|
|
||||||
|
## Files Modified
|
||||||
|
|
||||||
|
| File | Lines Changed | Description |
|
||||||
|
|------|--------------|-------------|
|
||||||
|
| `starpunk/config.py` | 9 added, 1 removed | Removed INDIELOGIN_URL, added deprecation warning |
|
||||||
|
| `starpunk/auth.py` | 1 added, 84 replaced | Added imports, rewrote initiate_login() and handle_callback() |
|
||||||
|
| `starpunk/auth_external.py` | 6 replaced | Relaxed endpoint validation |
|
||||||
|
| `starpunk/routes/auth.py` | 5 added | Added DiscoveryError import and exception handling |
|
||||||
|
| `tests/test_auth.py` | 1 removed, 43 modified | Removed INDIELOGIN_URL from fixture, added mocks |
|
||||||
|
| `tests/test_auth_external.py` | 2 modified | Updated error message assertion |
|
||||||
|
|
||||||
|
## Key Implementation Details
|
||||||
|
|
||||||
|
### Authorization Endpoint Usage
|
||||||
|
Per IndieAuth spec and architect clarifications:
|
||||||
|
- Authentication-only flows POST to the **authorization_endpoint** (not token_endpoint)
|
||||||
|
- The `grant_type` parameter is NOT included (only for access token flows)
|
||||||
|
- This differs from the previous implementation which incorrectly used indielogin.com's endpoints
|
||||||
|
|
||||||
|
### URL Normalization
|
||||||
|
The implementation now uses `normalize_url()` when comparing the returned 'me' URL with ADMIN_ME:
|
||||||
|
- Handles trailing slash differences (https://example.com vs https://example.com/)
|
||||||
|
- Handles case differences (https://Example.com vs https://example.com)
|
||||||
|
- This is spec-compliant behavior that was previously missing
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
- Discovery failures are caught and logged at ERROR level
|
||||||
|
- User-facing error message is simplified and friendly
|
||||||
|
- Technical details remain in logs for debugging
|
||||||
|
|
||||||
|
### Backward Compatibility
|
||||||
|
- Deprecation warning added for INDIELOGIN_URL environment variable
|
||||||
|
- Existing .env files with INDIELOGIN_URL will log a warning but continue to work
|
||||||
|
- Users are instructed to remove the deprecated config
|
||||||
|
|
||||||
|
## Testing Results
|
||||||
|
|
||||||
|
### Unit Tests
|
||||||
|
- `tests/test_auth.py`: 51/51 passed
|
||||||
|
- `tests/test_auth_external.py`: 35/35 passed
|
||||||
|
- `tests/test_routes_admin.py`: 32/32 passed
|
||||||
|
|
||||||
|
### Integration
|
||||||
|
All modified tests now correctly mock endpoint discovery and validate the new behavior.
|
||||||
|
|
||||||
|
## Issues Encountered
|
||||||
|
|
||||||
|
No significant issues encountered during implementation. The design document was thorough and all architect clarifications were addressed:
|
||||||
|
|
||||||
|
1. Import placement - Moved to top-level as specified
|
||||||
|
2. URL normalization - Included as intentional bugfix
|
||||||
|
3. Endpoint selection - Used authorization_endpoint for auth-only flows
|
||||||
|
4. Validation relaxation - Allowed profiles with only authorization_endpoint
|
||||||
|
5. Test strategy - Mocked discover_endpoints() and removed INDIELOGIN_URL
|
||||||
|
6. grant_type parameter - Correctly omitted for auth-only flows
|
||||||
|
7. Error messages - Simplified user-facing messages
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
1. Manual testing recommended:
|
||||||
|
- Test login flow with actual IndieAuth profile
|
||||||
|
- Verify endpoint discovery logs appear
|
||||||
|
- Test with profiles that have custom endpoints
|
||||||
|
- Verify error message appears for profiles without endpoints
|
||||||
|
|
||||||
|
2. Deployment:
|
||||||
|
- Update production .env to remove INDIELOGIN_URL (optional - will show warning)
|
||||||
|
- Deploy changes
|
||||||
|
- Monitor logs for "Discovered authorization_endpoint" messages
|
||||||
|
- Verify login works for admin user
|
||||||
|
|
||||||
|
3. Documentation:
|
||||||
|
- Update CHANGELOG.md with hotfix entry
|
||||||
|
- Consider adding migration guide if needed
|
||||||
|
|
||||||
|
## Verification Checklist
|
||||||
|
|
||||||
|
- [x] All specified files modified
|
||||||
|
- [x] All code changes follow architect's design exactly
|
||||||
|
- [x] Tests updated and passing
|
||||||
|
- [x] Error messages user-friendly
|
||||||
|
- [x] Logging appropriate for debugging
|
||||||
|
- [x] URL normalization implemented
|
||||||
|
- [x] Endpoint validation relaxed correctly
|
||||||
|
- [x] No regressions in existing tests
|
||||||
|
- [x] Implementation report created
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
The hotfix has been successfully implemented according to the architect's design. The authentication flow now correctly implements the W3C IndieAuth specification for endpoint discovery. All tests pass and no regressions were detected.
|
||||||
|
|
||||||
|
The critical production bug preventing user login should be resolved once this code is deployed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Developer:** Claude (Fullstack Developer Subagent)
|
||||||
|
**Date Completed:** 2025-12-17
|
||||||
|
**Ready for Review:** Yes
|
||||||
@@ -0,0 +1,608 @@
|
|||||||
|
# IndieAuth Endpoint Discovery Hotfix
|
||||||
|
|
||||||
|
**Date:** 2025-12-17
|
||||||
|
**Type:** Production Hotfix
|
||||||
|
**Priority:** Critical
|
||||||
|
**Status:** Ready for Implementation
|
||||||
|
|
||||||
|
## Problem Summary
|
||||||
|
|
||||||
|
Users cannot log in to StarPunk. The root cause is that the authentication code ignores endpoint discovery and hardcodes `INDIELOGIN_URL` instead of discovering the authorization and token endpoints from the user's profile URL.
|
||||||
|
|
||||||
|
**Root Cause:** The `starpunk/auth.py` module uses `INDIELOGIN_URL` config instead of discovering endpoints from the user's profile URL as required by the IndieAuth specification. This is a regression - the system used to respect discovered endpoints.
|
||||||
|
|
||||||
|
**Note:** The PKCE error message in the callback is a symptom, not the cause. Once we use the correct discovered endpoints, PKCE will not be required (since the user's actual IndieAuth server doesn't require it).
|
||||||
|
|
||||||
|
## Specification Requirements
|
||||||
|
|
||||||
|
### W3C IndieAuth Spec (https://www.w3.org/TR/indieauth/)
|
||||||
|
|
||||||
|
- Clients MUST discover `authorization_endpoint` from user's profile URL
|
||||||
|
- Clients MUST discover `token_endpoint` from user's profile URL
|
||||||
|
- Discovery via HTTP Link headers (highest priority) or HTML `<link>` elements
|
||||||
|
|
||||||
|
## Implementation Plan
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
|
||||||
|
The fix reuses the existing `discover_endpoints()` function from `auth_external.py` in the login flow. Changes are minimal and focused:
|
||||||
|
|
||||||
|
1. Use `discover_endpoints()` in `initiate_login()` to get the `authorization_endpoint`
|
||||||
|
2. Use `discover_endpoints()` in `handle_callback()` to get the `token_endpoint`
|
||||||
|
3. Remove `INDIELOGIN_URL` config (with deprecation warning)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Step 1: Update config.py - Remove INDIELOGIN_URL
|
||||||
|
|
||||||
|
In `/home/phil/Projects/starpunk/starpunk/config.py`:
|
||||||
|
|
||||||
|
**Change 1:** Remove the INDIELOGIN_URL config line (line 37):
|
||||||
|
|
||||||
|
```python
|
||||||
|
# DELETE THIS LINE:
|
||||||
|
app.config["INDIELOGIN_URL"] = os.getenv("INDIELOGIN_URL", "https://indielogin.com")
|
||||||
|
```
|
||||||
|
|
||||||
|
**Change 2:** Add deprecation warning for INDIELOGIN_URL (add after the TOKEN_ENDPOINT warning, around line 47):
|
||||||
|
|
||||||
|
```python
|
||||||
|
# DEPRECATED: INDIELOGIN_URL no longer used (hotfix 2025-12-17)
|
||||||
|
# Authorization endpoint is now discovered from ADMIN_ME profile per IndieAuth spec
|
||||||
|
if 'INDIELOGIN_URL' in os.environ:
|
||||||
|
app.logger.warning(
|
||||||
|
"INDIELOGIN_URL is deprecated and will be ignored. "
|
||||||
|
"Remove it from your configuration. "
|
||||||
|
"The authorization endpoint is now discovered automatically from your ADMIN_ME profile."
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Step 2: Update auth.py - Use Endpoint Discovery
|
||||||
|
|
||||||
|
In `/home/phil/Projects/starpunk/starpunk/auth.py`:
|
||||||
|
|
||||||
|
**Change 1:** Add import for endpoint discovery (after line 42):
|
||||||
|
|
||||||
|
```python
|
||||||
|
from starpunk.auth_external import discover_endpoints, DiscoveryError, normalize_url
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Note:** The `normalize_url` import is at the top level (not inside `handle_callback()`) for consistency with the existing code style.
|
||||||
|
|
||||||
|
**Change 2:** Update `initiate_login()` to use discovered authorization_endpoint (replace lines 251-318):
|
||||||
|
|
||||||
|
```python
|
||||||
|
def initiate_login(me_url: str) -> str:
|
||||||
|
"""
|
||||||
|
Initiate IndieAuth authentication flow with endpoint discovery.
|
||||||
|
|
||||||
|
Per W3C IndieAuth spec, discovers authorization_endpoint from user's
|
||||||
|
profile URL rather than using a hardcoded service.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
me_url: User's IndieWeb identity URL
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Redirect URL to discovered authorization endpoint
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: Invalid me_url format
|
||||||
|
DiscoveryError: Failed to discover endpoints from profile
|
||||||
|
"""
|
||||||
|
# Validate URL format
|
||||||
|
if not is_valid_url(me_url):
|
||||||
|
raise ValueError(f"Invalid URL format: {me_url}")
|
||||||
|
|
||||||
|
current_app.logger.debug(f"Auth: Validating me URL: {me_url}")
|
||||||
|
|
||||||
|
# Discover authorization endpoint from user's profile URL
|
||||||
|
# Per IndieAuth spec: clients MUST discover endpoints, not hardcode them
|
||||||
|
try:
|
||||||
|
endpoints = discover_endpoints(me_url)
|
||||||
|
except DiscoveryError as e:
|
||||||
|
current_app.logger.error(f"Auth: Endpoint discovery failed for {me_url}: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
auth_endpoint = endpoints.get('authorization_endpoint')
|
||||||
|
if not auth_endpoint:
|
||||||
|
raise DiscoveryError(
|
||||||
|
f"No authorization_endpoint found at {me_url}. "
|
||||||
|
"Ensure your profile has IndieAuth link elements or headers."
|
||||||
|
)
|
||||||
|
|
||||||
|
current_app.logger.info(f"Auth: Discovered authorization_endpoint: {auth_endpoint}")
|
||||||
|
|
||||||
|
# Generate CSRF state token
|
||||||
|
state = _generate_state_token()
|
||||||
|
current_app.logger.debug(f"Auth: Generated state token: {_redact_token(state, 8)}")
|
||||||
|
|
||||||
|
# Store state in database (5-minute expiry)
|
||||||
|
db = get_db(current_app)
|
||||||
|
expires_at = datetime.utcnow() + timedelta(minutes=5)
|
||||||
|
redirect_uri = f"{current_app.config['SITE_URL']}auth/callback"
|
||||||
|
|
||||||
|
db.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO auth_state (state, expires_at, redirect_uri)
|
||||||
|
VALUES (?, ?, ?)
|
||||||
|
""",
|
||||||
|
(state, expires_at, redirect_uri),
|
||||||
|
)
|
||||||
|
db.commit()
|
||||||
|
|
||||||
|
# Build authorization URL
|
||||||
|
params = {
|
||||||
|
"me": me_url,
|
||||||
|
"client_id": current_app.config["SITE_URL"],
|
||||||
|
"redirect_uri": redirect_uri,
|
||||||
|
"state": state,
|
||||||
|
"response_type": "code",
|
||||||
|
}
|
||||||
|
|
||||||
|
current_app.logger.debug(
|
||||||
|
f"Auth: Building authorization URL with params:\n"
|
||||||
|
f" me: {me_url}\n"
|
||||||
|
f" client_id: {current_app.config['SITE_URL']}\n"
|
||||||
|
f" redirect_uri: {redirect_uri}\n"
|
||||||
|
f" state: {_redact_token(state, 8)}\n"
|
||||||
|
f" response_type: code"
|
||||||
|
)
|
||||||
|
|
||||||
|
auth_url = f"{auth_endpoint}?{urlencode(params)}"
|
||||||
|
|
||||||
|
current_app.logger.debug(f"Auth: Complete authorization URL: {auth_url}")
|
||||||
|
current_app.logger.info(f"Auth: Authentication initiated for {me_url}")
|
||||||
|
|
||||||
|
return auth_url
|
||||||
|
```
|
||||||
|
|
||||||
|
**Change 3:** Update `handle_callback()` to use discovered authorization_endpoint (replace lines 321-474):
|
||||||
|
|
||||||
|
> **Important:** Per IndieAuth spec, authentication-only flows (identity verification without access tokens) POST to the **authorization_endpoint**, NOT the token_endpoint. The `grant_type` parameter is NOT included for authentication-only flows.
|
||||||
|
|
||||||
|
```python
|
||||||
|
def handle_callback(code: str, state: str, iss: Optional[str] = None) -> Optional[str]:
|
||||||
|
"""
|
||||||
|
Handle IndieAuth callback with endpoint discovery.
|
||||||
|
|
||||||
|
Discovers authorization_endpoint from ADMIN_ME profile and exchanges
|
||||||
|
authorization code for identity verification.
|
||||||
|
|
||||||
|
Per IndieAuth spec: Authentication-only flows POST to the authorization
|
||||||
|
endpoint (not token endpoint) and do not include grant_type.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
code: Authorization code from authorization server
|
||||||
|
state: CSRF state token
|
||||||
|
iss: Issuer identifier (optional, for security validation)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Session token if successful, None otherwise
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
InvalidStateError: State token validation failed
|
||||||
|
UnauthorizedError: User not authorized as admin
|
||||||
|
IndieLoginError: Code exchange failed
|
||||||
|
"""
|
||||||
|
current_app.logger.debug(f"Auth: Verifying state token: {_redact_token(state, 8)}")
|
||||||
|
|
||||||
|
# Verify state token (CSRF protection)
|
||||||
|
if not _verify_state_token(state):
|
||||||
|
current_app.logger.warning(
|
||||||
|
"Auth: Invalid state token received (possible CSRF or expired token)"
|
||||||
|
)
|
||||||
|
raise InvalidStateError("Invalid or expired state token")
|
||||||
|
|
||||||
|
current_app.logger.debug("Auth: State token valid")
|
||||||
|
|
||||||
|
# Discover authorization endpoint from ADMIN_ME profile
|
||||||
|
admin_me = current_app.config.get("ADMIN_ME")
|
||||||
|
if not admin_me:
|
||||||
|
current_app.logger.error("Auth: ADMIN_ME not configured")
|
||||||
|
raise IndieLoginError("ADMIN_ME not configured")
|
||||||
|
|
||||||
|
try:
|
||||||
|
endpoints = discover_endpoints(admin_me)
|
||||||
|
except DiscoveryError as e:
|
||||||
|
current_app.logger.error(f"Auth: Endpoint discovery failed: {e}")
|
||||||
|
raise IndieLoginError(f"Failed to discover endpoints: {e}")
|
||||||
|
|
||||||
|
# Use authorization_endpoint for authentication-only flow (identity verification)
|
||||||
|
# Per IndieAuth spec: auth-only flows POST to authorization_endpoint, not token_endpoint
|
||||||
|
auth_endpoint = endpoints.get('authorization_endpoint')
|
||||||
|
if not auth_endpoint:
|
||||||
|
raise IndieLoginError(
|
||||||
|
f"No authorization_endpoint found at {admin_me}. "
|
||||||
|
"Ensure your profile has IndieAuth endpoints configured."
|
||||||
|
)
|
||||||
|
|
||||||
|
current_app.logger.debug(f"Auth: Using authorization_endpoint: {auth_endpoint}")
|
||||||
|
|
||||||
|
# Verify issuer if provided (security check - optional)
|
||||||
|
if iss:
|
||||||
|
current_app.logger.debug(f"Auth: Issuer provided: {iss}")
|
||||||
|
|
||||||
|
# Prepare code verification request
|
||||||
|
# Note: grant_type is NOT included for authentication-only flows per IndieAuth spec
|
||||||
|
token_exchange_data = {
|
||||||
|
"code": code,
|
||||||
|
"client_id": current_app.config["SITE_URL"],
|
||||||
|
"redirect_uri": f"{current_app.config['SITE_URL']}auth/callback",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Log the request
|
||||||
|
_log_http_request(
|
||||||
|
method="POST",
|
||||||
|
url=auth_endpoint,
|
||||||
|
data=token_exchange_data,
|
||||||
|
)
|
||||||
|
|
||||||
|
current_app.logger.debug(
|
||||||
|
"Auth: Sending code verification request to authorization endpoint:\n"
|
||||||
|
" Method: POST\n"
|
||||||
|
" URL: %s\n"
|
||||||
|
" Data: code=%s, client_id=%s, redirect_uri=%s",
|
||||||
|
auth_endpoint,
|
||||||
|
_redact_token(code),
|
||||||
|
token_exchange_data["client_id"],
|
||||||
|
token_exchange_data["redirect_uri"],
|
||||||
|
)
|
||||||
|
|
||||||
|
# Exchange code for identity at authorization endpoint (authentication-only flow)
|
||||||
|
try:
|
||||||
|
response = httpx.post(
|
||||||
|
auth_endpoint,
|
||||||
|
data=token_exchange_data,
|
||||||
|
timeout=10.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
current_app.logger.debug(
|
||||||
|
"Auth: Received code verification response:\n"
|
||||||
|
" Status: %d\n"
|
||||||
|
" Headers: %s\n"
|
||||||
|
" Body: %s",
|
||||||
|
response.status_code,
|
||||||
|
{k: v for k, v in dict(response.headers).items()
|
||||||
|
if k.lower() not in ["set-cookie", "authorization"]},
|
||||||
|
_redact_token(response.text) if response.text else "(empty)",
|
||||||
|
)
|
||||||
|
|
||||||
|
_log_http_response(
|
||||||
|
status_code=response.status_code,
|
||||||
|
headers=dict(response.headers),
|
||||||
|
body=response.text,
|
||||||
|
)
|
||||||
|
|
||||||
|
response.raise_for_status()
|
||||||
|
except httpx.RequestError as e:
|
||||||
|
current_app.logger.error(f"Auth: Authorization endpoint request failed: {e}")
|
||||||
|
raise IndieLoginError(f"Failed to verify code: {e}")
|
||||||
|
except httpx.HTTPStatusError as e:
|
||||||
|
current_app.logger.error(
|
||||||
|
f"Auth: Authorization endpoint returned error: {e.response.status_code} - {e.response.text}"
|
||||||
|
)
|
||||||
|
raise IndieLoginError(
|
||||||
|
f"Authorization endpoint returned error: {e.response.status_code}"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Parse response
|
||||||
|
try:
|
||||||
|
data = response.json()
|
||||||
|
except Exception as e:
|
||||||
|
current_app.logger.error(f"Auth: Failed to parse authorization endpoint response: {e}")
|
||||||
|
raise IndieLoginError("Invalid JSON response from authorization endpoint")
|
||||||
|
|
||||||
|
me = data.get("me")
|
||||||
|
|
||||||
|
if not me:
|
||||||
|
current_app.logger.error("Auth: No identity returned from authorization endpoint")
|
||||||
|
raise IndieLoginError("No identity returned from authorization endpoint")
|
||||||
|
|
||||||
|
current_app.logger.debug(f"Auth: Received identity: {me}")
|
||||||
|
|
||||||
|
# Verify this is the admin user
|
||||||
|
current_app.logger.info(f"Auth: Verifying admin authorization for me={me}")
|
||||||
|
|
||||||
|
# Normalize URLs for comparison (handles trailing slashes and case differences)
|
||||||
|
# This is correct per IndieAuth spec - the returned 'me' is the canonical form
|
||||||
|
if normalize_url(me) != normalize_url(admin_me):
|
||||||
|
current_app.logger.warning(
|
||||||
|
f"Auth: Unauthorized login attempt: {me} (expected {admin_me})"
|
||||||
|
)
|
||||||
|
raise UnauthorizedError(f"User {me} is not authorized")
|
||||||
|
|
||||||
|
current_app.logger.debug("Auth: Admin verification passed")
|
||||||
|
|
||||||
|
# Create session
|
||||||
|
session_token = create_session(me)
|
||||||
|
|
||||||
|
# Trigger author profile discovery (v1.2.0 Phase 2)
|
||||||
|
# Per Q14: Never block login, always allow fallback
|
||||||
|
try:
|
||||||
|
from starpunk.author_discovery import get_author_profile
|
||||||
|
author_profile = get_author_profile(me, refresh=True)
|
||||||
|
current_app.logger.info(f"Author profile refreshed for {me}")
|
||||||
|
except Exception as e:
|
||||||
|
current_app.logger.warning(f"Author discovery failed: {e}")
|
||||||
|
# Continue login anyway - never block per Q14
|
||||||
|
|
||||||
|
return session_token
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Step 3: Update auth_external.py - Relax Endpoint Validation
|
||||||
|
|
||||||
|
The existing `_fetch_and_parse()` function requires `token_endpoint` to be present. We need to relax this since some profiles may only have `authorization_endpoint` (for authentication-only flows).
|
||||||
|
|
||||||
|
In `/home/phil/Projects/starpunk/starpunk/auth_external.py`, update the validation in `_fetch_and_parse()` (around lines 302-307):
|
||||||
|
|
||||||
|
**Change:** Make token_endpoint not strictly required (allow authentication-only profiles):
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Validate we found at least one endpoint
|
||||||
|
# - authorization_endpoint: Required for authentication-only flows (admin login)
|
||||||
|
# - token_endpoint: Required for Micropub token verification
|
||||||
|
# Having at least one allows the appropriate flow to work
|
||||||
|
if 'token_endpoint' not in endpoints and 'authorization_endpoint' not in endpoints:
|
||||||
|
raise DiscoveryError(
|
||||||
|
f"No IndieAuth endpoints found at {profile_url}. "
|
||||||
|
"Ensure your profile has authorization_endpoint or token_endpoint configured."
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Step 4: Update routes/auth.py - Handle DiscoveryError
|
||||||
|
|
||||||
|
In `/home/phil/Projects/starpunk/starpunk/routes/auth.py`:
|
||||||
|
|
||||||
|
**Change 1:** Add import for DiscoveryError (update lines 20-29):
|
||||||
|
|
||||||
|
```python
|
||||||
|
from starpunk.auth import (
|
||||||
|
IndieLoginError,
|
||||||
|
InvalidStateError,
|
||||||
|
UnauthorizedError,
|
||||||
|
destroy_session,
|
||||||
|
handle_callback,
|
||||||
|
initiate_login,
|
||||||
|
require_auth,
|
||||||
|
verify_session,
|
||||||
|
)
|
||||||
|
from starpunk.auth_external import DiscoveryError
|
||||||
|
```
|
||||||
|
|
||||||
|
**Change 2:** Handle DiscoveryError in login_initiate() (update lines 79-85):
|
||||||
|
|
||||||
|
> **Note:** The user-facing error message is kept simple. Technical details are logged but not shown to users.
|
||||||
|
|
||||||
|
```python
|
||||||
|
try:
|
||||||
|
# Initiate IndieAuth flow
|
||||||
|
auth_url = initiate_login(me_url)
|
||||||
|
return redirect(auth_url)
|
||||||
|
except ValueError as e:
|
||||||
|
flash(str(e), "error")
|
||||||
|
return redirect(url_for("auth.login_form"))
|
||||||
|
except DiscoveryError as e:
|
||||||
|
current_app.logger.error(f"Endpoint discovery failed for {me_url}: {e}")
|
||||||
|
flash("Unable to verify your profile URL. Please check that it's correct and try again.", "error")
|
||||||
|
return redirect(url_for("auth.login_form"))
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Summary
|
||||||
|
|
||||||
|
| File | Change Type | Description |
|
||||||
|
|------|-------------|-------------|
|
||||||
|
| `starpunk/config.py` | Edit | Remove INDIELOGIN_URL, add deprecation warning |
|
||||||
|
| `starpunk/auth.py` | Edit | Use endpoint discovery instead of hardcoded URL |
|
||||||
|
| `starpunk/auth_external.py` | Edit | Relax endpoint validation (allow auth-only flow) |
|
||||||
|
| `starpunk/routes/auth.py` | Edit | Handle DiscoveryError exception |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing Requirements
|
||||||
|
|
||||||
|
### Manual Testing
|
||||||
|
|
||||||
|
1. **Login Flow Test**
|
||||||
|
- Navigate to `/auth/login`
|
||||||
|
- Enter ADMIN_ME URL
|
||||||
|
- Verify redirect goes to discovered authorization_endpoint (not hardcoded indielogin.com)
|
||||||
|
- Complete login and verify session is created
|
||||||
|
|
||||||
|
2. **Endpoint Discovery Test**
|
||||||
|
- Test with profile that declares custom endpoints
|
||||||
|
- Verify discovered endpoints are used, not defaults
|
||||||
|
|
||||||
|
### Existing Test Updates
|
||||||
|
|
||||||
|
**Update test fixture in `tests/test_auth.py`:**
|
||||||
|
|
||||||
|
Remove `INDIELOGIN_URL` from the app fixture (line 51):
|
||||||
|
|
||||||
|
```python
|
||||||
|
@pytest.fixture
|
||||||
|
def app(tmp_path):
|
||||||
|
"""Create Flask app for testing"""
|
||||||
|
from starpunk import create_app
|
||||||
|
|
||||||
|
test_data_dir = tmp_path / "data"
|
||||||
|
test_data_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
app = create_app(
|
||||||
|
{
|
||||||
|
"TESTING": True,
|
||||||
|
"SITE_URL": "http://localhost:5000/",
|
||||||
|
"ADMIN_ME": "https://example.com",
|
||||||
|
"SESSION_SECRET": secrets.token_hex(32),
|
||||||
|
"SESSION_LIFETIME": 30,
|
||||||
|
# REMOVED: "INDIELOGIN_URL": "https://indielogin.com",
|
||||||
|
"DATA_PATH": test_data_dir,
|
||||||
|
"NOTES_PATH": test_data_dir / "notes",
|
||||||
|
"DATABASE_PATH": test_data_dir / "starpunk.db",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return app
|
||||||
|
```
|
||||||
|
|
||||||
|
**Update existing tests that use `httpx.post` mock:**
|
||||||
|
|
||||||
|
Tests in `TestInitiateLogin` and `TestHandleCallback` need to mock `discover_endpoints()` in addition to `httpx.post`. Example pattern:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@patch("starpunk.auth.discover_endpoints")
|
||||||
|
@patch("starpunk.auth.httpx.post")
|
||||||
|
def test_handle_callback_success(self, mock_post, mock_discover, app, db, client):
|
||||||
|
"""Test successful callback handling"""
|
||||||
|
# Mock endpoint discovery
|
||||||
|
mock_discover.return_value = {
|
||||||
|
'authorization_endpoint': 'https://auth.example.com/authorize',
|
||||||
|
'token_endpoint': 'https://auth.example.com/token'
|
||||||
|
}
|
||||||
|
|
||||||
|
# Rest of test remains the same...
|
||||||
|
```
|
||||||
|
|
||||||
|
**Update `TestInitiateLogin.test_initiate_login_success`:**
|
||||||
|
|
||||||
|
The assertion checking for `indielogin.com` needs to change to check for the mocked endpoint:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@patch("starpunk.auth.discover_endpoints")
|
||||||
|
def test_initiate_login_success(self, mock_discover, app, db):
|
||||||
|
"""Test successful login initiation"""
|
||||||
|
mock_discover.return_value = {
|
||||||
|
'authorization_endpoint': 'https://auth.example.com/authorize',
|
||||||
|
'token_endpoint': 'https://auth.example.com/token'
|
||||||
|
}
|
||||||
|
|
||||||
|
with app.app_context():
|
||||||
|
me_url = "https://example.com"
|
||||||
|
auth_url = initiate_login(me_url)
|
||||||
|
|
||||||
|
# Changed: Check for discovered endpoint instead of indielogin.com
|
||||||
|
assert "auth.example.com/authorize" in auth_url
|
||||||
|
assert "me=https%3A%2F%2Fexample.com" in auth_url
|
||||||
|
# ... rest of assertions
|
||||||
|
```
|
||||||
|
|
||||||
|
### New Automated Tests to Add
|
||||||
|
|
||||||
|
```python
|
||||||
|
# tests/test_auth_endpoint_discovery.py
|
||||||
|
|
||||||
|
def test_initiate_login_uses_endpoint_discovery(client, mocker):
|
||||||
|
"""Verify login uses discovered endpoint, not hardcoded"""
|
||||||
|
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
|
||||||
|
mock_discover.return_value = {
|
||||||
|
'authorization_endpoint': 'https://custom-auth.example.com/authorize',
|
||||||
|
'token_endpoint': 'https://custom-auth.example.com/token'
|
||||||
|
}
|
||||||
|
|
||||||
|
response = client.post('/auth/login', data={'me': 'https://example.com'})
|
||||||
|
|
||||||
|
assert response.status_code == 302
|
||||||
|
assert 'custom-auth.example.com' in response.headers['Location']
|
||||||
|
|
||||||
|
|
||||||
|
def test_callback_uses_discovered_authorization_endpoint(client, mocker):
|
||||||
|
"""Verify callback uses discovered authorization endpoint (not token endpoint)"""
|
||||||
|
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
|
||||||
|
mock_discover.return_value = {
|
||||||
|
'authorization_endpoint': 'https://custom-auth.example.com/authorize',
|
||||||
|
'token_endpoint': 'https://custom-auth.example.com/token'
|
||||||
|
}
|
||||||
|
mock_post = mocker.patch('starpunk.auth.httpx.post')
|
||||||
|
# Setup state token and mock httpx response
|
||||||
|
# Verify code exchange POSTs to authorization_endpoint, not token_endpoint
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def test_discovery_error_shows_user_friendly_message(client, mocker):
|
||||||
|
"""Verify discovery failures show helpful error"""
|
||||||
|
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
|
||||||
|
mock_discover.side_effect = DiscoveryError("No endpoints found")
|
||||||
|
|
||||||
|
response = client.post('/auth/login', data={'me': 'https://example.com'})
|
||||||
|
|
||||||
|
assert response.status_code == 302
|
||||||
|
# Should redirect back to login form with flash message
|
||||||
|
|
||||||
|
|
||||||
|
def test_url_normalization_handles_trailing_slash(app, mocker):
|
||||||
|
"""Verify URL normalization allows trailing slash differences"""
|
||||||
|
# ADMIN_ME without trailing slash, auth server returns with trailing slash
|
||||||
|
# Should still authenticate successfully
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def test_url_normalization_handles_case_differences(app, mocker):
|
||||||
|
"""Verify URL normalization is case-insensitive"""
|
||||||
|
# ADMIN_ME: https://Example.com, auth server returns: https://example.com
|
||||||
|
# Should still authenticate successfully
|
||||||
|
pass
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Rollback Plan
|
||||||
|
|
||||||
|
If issues occur after deployment:
|
||||||
|
|
||||||
|
1. **Code:** Revert to previous commit
|
||||||
|
2. **Config:** Re-add INDIELOGIN_URL to .env if needed
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Post-Deployment Verification
|
||||||
|
|
||||||
|
1. Verify login works with the user's actual profile URL
|
||||||
|
2. Check logs for "Discovered authorization_endpoint" message
|
||||||
|
3. Test logout and re-login cycle
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architect Q&A (2025-12-17)
|
||||||
|
|
||||||
|
Developer questions answered by the architect prior to implementation:
|
||||||
|
|
||||||
|
### Q1: Import Placement
|
||||||
|
**Q:** Should `normalize_url` import be inside the function or at top level?
|
||||||
|
**A:** Move to top level with other imports for consistency. The design has been updated.
|
||||||
|
|
||||||
|
### Q2: URL Normalization Behavior Change
|
||||||
|
**Q:** Is the URL normalization change intentional?
|
||||||
|
**A:** Yes, this is an intentional bugfix. The current exact-match behavior is incorrect per IndieAuth spec. URLs differing only in trailing slashes or case should be considered equivalent for identity purposes. The `normalize_url()` function already exists in `auth_external.py` and is used by `verify_external_token()`.
|
||||||
|
|
||||||
|
### Q3: Which Endpoint for Authentication Flow?
|
||||||
|
**Q:** Should we use token_endpoint or authorization_endpoint?
|
||||||
|
**A:** Use **authorization_endpoint** for authentication-only flows. Per IndieAuth spec: "the client makes a POST request to the authorization endpoint to verify the authorization code and retrieve the final user profile URL." The design has been corrected.
|
||||||
|
|
||||||
|
### Q4: Endpoint Validation Relaxation
|
||||||
|
**Q:** Is relaxed endpoint validation acceptable?
|
||||||
|
**A:** Yes. Login requires `authorization_endpoint`, Micropub requires `token_endpoint`. Requiring at least one is correct. If only auth endpoint exists, login works but Micropub fails gracefully (401).
|
||||||
|
|
||||||
|
### Q5: Test Update Strategy
|
||||||
|
**Q:** Remove INDIELOGIN_URL and/or mock discover_endpoints()?
|
||||||
|
**A:** Both. Remove `INDIELOGIN_URL` from fixtures, add `discover_endpoints()` mocking to existing tests. Detailed guidance added to Testing Requirements section.
|
||||||
|
|
||||||
|
### Q6: grant_type Parameter
|
||||||
|
**Q:** Should we include grant_type in the code exchange?
|
||||||
|
**A:** No. Authentication-only flows do not include `grant_type`. This parameter is only required when POSTing to the token_endpoint for access tokens. The design has been corrected.
|
||||||
|
|
||||||
|
### Q7: Error Message Verbosity
|
||||||
|
**Q:** Should we simplify the user-facing error message?
|
||||||
|
**A:** Yes. User-facing message should be simple: "Unable to verify your profile URL. Please check that it's correct and try again." Technical details are logged at ERROR level. The design has been updated.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- W3C IndieAuth Specification: https://www.w3.org/TR/indieauth/
|
||||||
|
- IndieAuth Endpoint Discovery: https://www.w3.org/TR/indieauth/#discovery-by-clients
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user