Compare commits

..

43 Commits

Author SHA1 Message Date
c94cb377d3 release: Bump version to 1.5.0-rc.1 for testing
Release candidate for v1.5.0 with IndieAuth hotfix.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 14:25:33 -07:00
2bd971f3d6 fix(auth): Implement IndieAuth endpoint discovery per W3C spec
BREAKING: Removes INDIELOGIN_URL config - endpoints are now properly
discovered from user's profile URL as required by W3C IndieAuth spec.

- auth.py: Uses discover_endpoints() to find authorization_endpoint
- config.py: Deprecation warning for obsolete INDIELOGIN_URL setting
- auth_external.py: Relaxed validation (allows auth-only flows)
- tests: Updated to mock endpoint discovery

This fixes a regression where admin login was hardcoded to use
indielogin.com instead of respecting the user's declared endpoints.

Version: 1.5.0-hotfix.1

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 13:52:36 -07:00
84e693fe57 release: Bump version to 1.5.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 12:36:16 -07:00
975046abc7 test: Expand coverage to 90% for v1.5.0
Added 3 MPO format tests per v1.5.0 Phase 5 requirements:
- test_mpo_detection_and_conversion: Verify MPO->JPEG conversion
- test_mpo_dimensions_preserved: Verify dimensions maintained
- test_mpo_full_upload_flow: Test complete upload workflow

MPO (Multi-Picture Object) format handling was implemented in v1.4.2
but was previously untested. These tests ensure the format detection
and JPEG conversion work correctly for iPhone portrait/depth photos.

Test count: 924 -> 927 tests (all passing)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 11:56:03 -07:00
21fa7acfbb feat(media): Make variant generation atomic with database
Per v1.5.0 Phase 4:
- Generate variants to temp directory first
- Perform database inserts in transaction
- Move files to final location before commit
- Clean up temp files on any failure
- Add startup recovery for orphaned temp files
- All media operations now fully atomic

Changes:
- Modified generate_all_variants() to return file moves
- Modified save_media() to handle full atomic operation
- Add cleanup_orphaned_temp_files() for startup recovery
- Added 4 new tests for atomic behavior
- Fixed HEIC variant format detection
- Updated variant failure test for atomic behavior

Fixes:
- No orphaned files on database failures
- No orphaned DB records on file failures
- Startup recovery detects and cleans orphans

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 11:26:26 -07:00
b689e02e64 perf(feed): Batch load media and tags to fix N+1 query
Per v1.5.0 Phase 3: Fix N+1 query pattern in feed generation.

Implementation:
- Add get_media_for_notes() to starpunk/media.py for batch media loading
- Add get_tags_for_notes() to starpunk/tags.py for batch tag loading
- Update _get_cached_notes() in starpunk/routes/public.py to use batch loading
- Add comprehensive tests in tests/test_batch_loading.py

Performance improvement:
- Before: O(n) queries (1 query per note for media + 1 query per note for tags)
- After: O(1) queries (2 queries total: 1 for all media, 1 for all tags)
- Maintains same API behavior and output format

All tests passing: 920 passed in 360.79s

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 10:42:44 -07:00
1b45a64920 feat: v1.5.0 Phase 2 - Debug File Management
Implement debug file management system with configuration controls,
automatic cleanup, and security improvements per v1.5.0 Phase 2.

## Changes

### Configuration (config.py)
- Add DEBUG_SAVE_FAILED_UPLOADS (default: false, production-safe)
- Add DEBUG_FILE_MAX_AGE_DAYS (default: 7 days)
- Add DEBUG_FILE_MAX_SIZE_MB (default: 100MB)

### Media Validation (media.py)
- Check config before saving debug files
- Sanitize filenames to prevent path traversal
- Pattern: alphanumeric + "._-", truncated to 50 chars
- Add cleanup_old_debug_files() function
  * Age-based cleanup (delete files older than MAX_AGE)
  * Size-based cleanup (delete oldest if total > MAX_SIZE)

### Application Startup (__init__.py)
- Run cleanup_old_debug_files() on startup
- Automatic maintenance of debug directory

### Tests (test_debug_file_management.py)
- 15 comprehensive tests
- Config defaults and overrides
- Debug file saving behavior
- Filename sanitization security
- Cleanup age and size limits
- Startup integration

## Security Improvements
- Debug saving disabled by default (production-safe)
- Filename sanitization prevents path traversal
- Automatic cleanup prevents disk exhaustion

## Acceptance Criteria
- [x] Configuration options added
- [x] Debug saving disabled by default
- [x] Filename sanitized before saving
- [x] Cleanup runs on startup
- [x] Old files deleted based on age
- [x] Size limit enforced

All tests pass. Ready for architect review.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 10:05:42 -07:00
3f1f82a749 feat(slugs): Implement timestamp-based slugs per ADR-062
Replaces content-based slug generation with timestamp format YYYYMMDDHHMMSS.
Simplifies slug generation and improves privacy by not exposing note content in URLs.

Changes:
- Add generate_timestamp_slug() to slug_utils.py
- Update notes.py to use timestamp slugs for default generation
- Sequential collision suffix (-1, -2) instead of random
- Custom slugs via mp-slug continue to work unchanged
- 892 tests passing (+18 new timestamp slug tests)

Per ADR-062 and v1.5.0 Phase 1 specification.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 09:49:30 -07:00
92e7bdd342 feat(tests): Phase 0 - Fix flaky and broken tests
Implements Phase 0 of v1.5.0 per ADR-012 and RELEASE.md.

Changes:
- Remove 5 broken multiprocessing tests (TestConcurrentExecution, TestPerformance)
- Fix brittle XML assertion tests (check semantics not quote style)
- Fix test_debug_level_for_early_retries logger configuration
- Rename test_feed_route_streaming to test_feed_route_caching (correct name)

Results:
- Test count: 879 → 874 (5 removed as planned)
- All tests pass consistently (verified across 3 runs)
- No flakiness detected

References:
- ADR-012: Flaky Test Removal and Test Quality Standards
- docs/projectplan/v1.5.0/RELEASE.md Phase 0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 09:24:12 -07:00
0acefa4670 docs: Update Phase 0 with specific test fix requirements
Per ADR-012, Phase 0 now specifies:
- 5 tests to REMOVE (broken multiprocessing)
- 4 tests to FIX (brittle assertions)
- 1 test to RENAME (misleading name)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 20:45:41 -07:00
9dcc5c5710 docs: v1.5.0 planning - ADR-062, release plan, and design docs
- ADR-062: Timestamp-based slug format (supersedes ADR-007)
- Updated v1.5.0 RELEASE.md with 6-phase plan
- Updated BACKLOG.md with deferred N+1 query locations
- Developer questions and architect responses for Phase 1

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 19:38:01 -07:00
7be2fb0f62 docs: Add v1.5.0 "Trigger" release definition
Focus: Cleanup, test coverage, and quality of life improvements

Goals:
- 90% test coverage target
- MPO format test coverage (High backlog item)
- Debug file storage cleanup (Medium backlog item)
- Filename sanitization in debug path (Medium backlog item)
- N+1 query pattern fix (Medium backlog item)
- Atomic variant generation (Medium backlog item)
- Default slug change to timestamp format (Medium backlog item)

Backlog items marked as scheduled for v1.5.0.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 19:04:43 -07:00
730eb8d58c docs: Add v1.4.2 review documents and update backlog
Some checks failed
Build Container / build (push) Failing after 15s
- Fix stale docstring in media.py (4096 -> 12000)
- Add developer review document
- Add architect review document
- Update backlog with identified issues

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 18:57:39 -07:00
6682339a86 fix(media): Increase max dimension to 12000px for modern phone cameras
Modern iPhones (48MP) and other phones produce images larger than 4096px.
Since optimize_image() resizes them anyway, the input limit was too
restrictive. Increased from 4096x4096 to 12000x12000.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 18:29:21 -07:00
d416463242 fix(media): Add MPO format support for iPhone portrait photos
iPhones use MPO (Multi-Picture Object) format for depth/portrait photos.
This format contains multiple JPEG images (main + depth map). Pillow
opens these as MPO format which wasn't in our allowed types.

Added automatic MPO to JPEG conversion by extracting the primary image.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 18:23:23 -07:00
25b8cbd79d chore: Add format detection logging for debugging
Logs the detected image format when a file is rejected to help
diagnose why iPhone uploads are being rejected.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 18:14:05 -07:00
042505d5a6 fix(media): Add MAX_CONTENT_LENGTH and debug file capture - v1.4.2
- Set Flask MAX_CONTENT_LENGTH to 50MB (matches MAX_FILE_SIZE)
- Save failed uploads to data/debug/ for analysis
- Log magic bytes when both Pillow and HEIC parsers fail

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 18:08:36 -07:00
72f3d4a55e fix(media): Save failed uploads to debug/ for analysis - v1.4.2
When an image fails both Pillow and HEIC parsing, save the file
to data/debug/ for manual analysis.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 18:06:51 -07:00
e8ff0a0dcb fix(media): Add debug logging for unrecognized image formats - v1.4.2
Logs magic bytes when both Pillow and HEIC parsers fail,
to help diagnose what format the file actually is.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 18:06:09 -07:00
9bc6780a8e fix(media): Handle HEIC files with wrong extension - v1.4.2
iOS sometimes saves HEIC with .jpeg extension. Pillow fails to open
these as JPEG, so now we fallback to trying pillow-heif directly
when initial open fails.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 17:54:50 -07:00
e4e481d7cf feat(media): Add HEIC/HEIF image support - v1.4.2
- Add pillow-heif dependency for iPhone photo support
- Auto-convert HEIC to JPEG (browsers can't display HEIC)
- Graceful error if pillow-heif not installed
- Handles RGBA/P mode conversion to RGB

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 17:45:53 -07:00
07f351fef7 feat(media): Add comprehensive logging for media uploads - v1.4.1
Implements media upload logging per docs/design/v1.4.1/media-logging-design.md

Changes:
- Add logging to save_media() in starpunk/media.py:
  * INFO: Successful uploads with file details
  * WARNING: Validation/optimization/variant failures
  * ERROR: Unexpected system errors
- Remove duplicate logging in Micropub media endpoint
- Add 5 comprehensive logging tests in TestMediaLogging class
- Bump version to 1.4.1
- Update CHANGELOG.md

All media upload operations now logged for debugging and observability.
Validation errors, optimization failures, and variant generation issues
are tracked at appropriate log levels. Original functionality unchanged.

Test results: 28/28 media tests pass, 5 new logging tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 17:22:22 -07:00
fd92a1d1eb fix: Clean up variant files in delete_media() - v1.4.0
Following design approved by architect in docs/design/v1.4.0/

Changes to starpunk/media.py:
- Update delete_media() to fetch and delete variant files from disk
- Query media_variants table before deletion for file paths
- Use best-effort cleanup with try/except for each file
- Log individual file deletion failures as warnings
- Update final log to show total files deleted (original + variants)
- Update module docstring to reflect v1.4.0 capabilities:
  * 50MB max upload, 10MB max output
  * Image variants (thumb, small, medium, large)
  * Tiered resize strategy

This fixes the issue where variant files were left orphaned on disk
when media was deleted. The database CASCADE already deleted variant
records, but the physical files remained.

All tests pass: uv run pytest tests/test_media_upload.py -v (23/23)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 11:10:21 -07:00
68d1a1d879 feat: v1.4.0 Phase 5 - Testing & Documentation
Complete Phase 5 of v1.4.0 "Media" release with comprehensive release
preparation.

CHANGELOG Updates:
- Added v1.4.0rc1 section with comprehensive release notes
- Documented all four phases:
  * Large Image Support (50MB limit, tiered resize)
  * Image Variants (thumb, small, medium, large)
  * Micropub Media Endpoint (W3C compliant)
  * Enhanced Feed Media (Media RSS, JSON Feed extension)
- Standards compliance documented (W3C Micropub, Media RSS 2.0)
- Storage impact and backwards compatibility notes
- Configuration options (STARPUNK_ABOUT_URL)

Version Bump:
- Updated starpunk/__init__.py to 1.4.0rc1
- Version info tuple includes rc1 designator

Implementation Report:
- Created docs/design/v1.4.0/2025-12-16-v140-implementation-complete.md
- Complete summary of all five phases
- Test results: 850 passing, 19 failing
- Known issues documented (flaky tests, feed cache)
- Acceptance criteria checklist
- Deployment and rollback procedures
- Release recommendation: Ready for RC testing

Test Results:
- Full test suite executed: 850 passed, 19 failed (98% pass rate)
- Core v1.4.0 functionality verified working
- Failures in known flaky tests and unrelated areas
- 5 minutes 58 seconds execution time

Release Status:
- All five phases complete
- Ready for release candidate testing
- Manual verification recommended before final release

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 08:18:54 -07:00
00f21d2a51 docs: Add Phase 4 implementation report
Document completed Phase 4 implementation with test results,
verification output, and backwards compatibility notes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 08:01:06 -07:00
83dc488457 feat: v1.4.0 Phase 4 - Enhanced Feed Media
Implement Phase 4 of v1.4.0 Media release - Enhanced Feed Media support.

RSS Feed Enhancements (starpunk/feeds/rss.py):
- Wrap size variants in <media:group> elements
- Add <media:content> for large/medium/small variants with attributes:
  url, type, medium, isDefault, width, height, fileSize
- Add <media:thumbnail> for thumb variant with dimensions
- Add <media:title type="plain"> for image captions
- Implement isDefault logic: largest available variant (large→medium→small fallback)
- Maintain backwards compatibility for media without variants (legacy fallback)

JSON Feed Enhancements (starpunk/feeds/json_feed.py):
- Add _starpunk.about URL (configurable via STARPUNK_ABOUT_URL config)
- Add _starpunk.media_variants array with variant data when variants exist
- Each variant entry includes: url, width, height, size_in_bytes, mime_type

ATOM Feed Enhancements (starpunk/feeds/atom.py):
- Add title attribute to enclosure links for captions
- Keep simple (no variants in ATOM per design decision)

Test Updates (tests/test_feeds_rss.py):
- Update streaming media test to search descendants for media:content
- Now inside media:group for images with variants (v1.4.0 behavior)

Per design document: /docs/design/v1.4.0/media-implementation-design.md
Following ADR-059: Full Feed Media Standardization

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 07:47:56 -07:00
c64feaea23 feat: v1.4.0 Phase 3 - Micropub Media Endpoint
Implement W3C Micropub media endpoint for external client uploads.

Changes:
- Add POST /micropub/media endpoint in routes/micropub.py
  - Accept multipart/form-data with 'file' field
  - Require bearer token with 'create' scope
  - Return 201 Created with Location header
  - Validate, optimize, and generate variants via save_media()

- Update q=config response to advertise media-endpoint
  - Include media-endpoint URL in config response
  - Add 'photo' post-type to supported types

- Add photo property support to Micropub create
  - extract_photos() function to parse photo property
  - Handles both simple URL strings and structured objects with alt text
  - _attach_photos_to_note() function to attach photos by URL
  - Only attach photos from our server (by URL match)
  - External URLs logged but ignored (no download)
  - Maximum 4 photos per note (per ADR-057)

- SITE_URL normalization pattern
  - Use .rstrip('/') for consistent URL comparison
  - Applied in media endpoint and photo attachment

Per design document: docs/design/v1.4.0/media-implementation-design.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 18:32:21 -07:00
501a711050 feat(media): Add image variant support - v1.4.0 Phase 2
Implement automatic generation of multiple image sizes for responsive
delivery and enhanced feed optimization.

Changes:
- Add migration 009 for media_variants table with CASCADE delete
- Define variant specs: thumb (150x150 crop), small (320px), medium (640px), large (1280px)
- Implement generate_variant() with center crop for thumbnails and aspect-preserving resize
- Implement generate_all_variants() with try/except cleanup, pass year/month/optimized_bytes explicitly
- Update save_media() to generate all variants after saving original
- Update get_note_media() to include variants dict (backwards compatible - only when variants exist)
- Record original as 'original' variant type

Technical details:
- Variants use explicit year/month parameters to avoid fragile path traversal
- Pass optimized_bytes to avoid redundant file I/O
- File cleanup on variant generation failure
- Skip generating variants larger than source image
- variants key only added to response when variants exist (pre-v1.4.0 compatibility)

All existing tests pass. Phase 2 complete.

Per design document: /home/phil/Projects/starpunk/docs/design/v1.4.0/media-implementation-design.md

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 18:19:24 -07:00
1b51c82656 feat: Implement v1.4.0 Phase 1 - Large Image Support
Implement tiered resize strategy for large images per v1.4.0 design:

Changes:
- Increase MAX_FILE_SIZE from 10MB to 50MB
- Add MAX_OUTPUT_SIZE constant (10MB target after optimization)
- Add MIN_QUALITY and MIN_DIMENSION constants
- Add get_optimization_params() for tiered strategy:
  - <=10MB: 2048px max, 95% quality
  - 10-25MB: 1600px max, 90% quality
  - 25-50MB: 1280px max, 85% quality
- Update optimize_image() signature to return 4-tuple (img, w, h, bytes)
- Implement iterative quality reduction if output >10MB
- Add animated GIF detection and size check in validate_image()
- Update save_media() to use new optimize_image() return value
- Fix GIF format preservation during optimization
- Update tests to match new optimize_image() signature

All existing tests pass. Ready for Phase 2 (Image Variants).

Following design in:
/home/phil/Projects/starpunk/docs/design/v1.4.0/media-implementation-design.md

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 17:57:44 -07:00
5ea9c8f330 release: Bump version to 1.3.1
Some checks failed
Build Container / build (push) Failing after 13s
Promote v1.3.1-rc.1 to stable release "Syndicate Tags"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 14:36:59 -07:00
98692c35db Merge branch 'feature/feed-tags' for v1.3.1
Add tags/categories to RSS, Atom, and JSON Feed formats.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 13:28:16 -07:00
61cba2fa6d release: Bump version to 1.3.1rc1
Prepare release candidate for v1.3.1 "Syndicate Tags"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 13:27:55 -07:00
cba24ab06f fix: Use _cached_tags for Note model compatibility in feeds
The Note model has a `tags` property that uses `_cached_tags` internally.
Setting `tags` directly via object.__setattr__ conflicts with the property.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 13:25:28 -07:00
9b26de7b05 feat: Add tag/category support to all feed formats (v1.3.1)
Implements tag/category rendering in RSS 2.0, Atom 1.0, and JSON Feed 1.1
syndication feeds. Tags added in v1.3.0 are now visible to feed readers.

Changes:
- Load tags in feed generation (_get_cached_notes)
- Add <category> elements to RSS 2.0 feeds
- Add <category term/label> to Atom 1.0 feeds
- Add "tags" array to JSON Feed 1.1 items
- Omit empty tags field in JSON Feed (minimal approach)

Standards compliance:
- RSS 2.0: category element with display name
- Atom 1.0: RFC 4287 category with term and label attributes
- JSON Feed 1.1: tags array of strings

Per design: docs/design/v1.3.1/feed-tags-design.md
Implementation: docs/design/v1.3.1/2025-12-10-feed-tags-implementation.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 12:33:35 -07:00
3222620cee release: Bump version to 1.3.0
Some checks failed
Build Container / build (push) Failing after 12s
Promoting v1.3.0-rc.1 to stable release.

Changes:
- Updated version in starpunk/__init__.py to 1.3.0
- Updated CHANGELOG.md header to v1.3.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 12:19:11 -07:00
247eb34c36 Merge feature/v1.3.0-tags-microformats into main
Release candidate v1.3.0-rc.1 with tags/categories and enhanced Microformats2 support.

Major features:
- Complete tag/category system with Micropub support
- Strict Microformats2 compliance (p-category, h-feed properties)
- Tag archive pages at /tags/{tag}
- Enhanced h-entry markup with dt-updated
- Proper h-feed structure on collection pages

Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 12:01:09 -07:00
41b65703f9 docs: Add v1.3.1 and v1.4.0 release definitions
v1.3.1 "Syndicate Tags":
- RSS/Atom/JSON Feed category/tag support

v1.4.0 "Media":
- Micropub media endpoint (W3C compliant)
- Large image support (>10MB auto-resize)
- Enhanced feed media (image variants, full Media RSS)

Also adds tag-filtered feeds to backlog at medium priority.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 12:00:56 -07:00
f901aa2242 docs: Update project plan files 2025-12-10 11:58:45 -07:00
5ca8b7e9b4 release: Bump version to 1.3.0-rc.1
Release candidate for v1.3.0 with tags/categories and enhanced Microformats2 support.

Features:
- Tag/category system with Micropub support
- Strict Microformats2 compliance (p-category, h-feed)
- Tag archive pages
- Enhanced h-entry markup

Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 11:58:10 -07:00
3d80e1af51 test(microformats): Add v1.3.0 validation tests for tags and h-feed
Phase 4: Validation per microformats-tags-design.md

Added test fixtures:
- published_note_with_tags: Creates note with test tags for p-category validation
- published_note_with_media: Creates note with media for u-photo placement testing

Added v1.3.0 microformats2 validation tests:
- test_hfeed_has_required_properties: Validates name, author, url per spec
- test_hfeed_author_is_valid_hcard: Validates h-card structure
- test_hentry_has_pcategory_for_tags: Validates p-category markup
- test_uphoto_outside_econtent: Validates u-photo placement per draft spec

Test results:
- All 18 microformats tests pass
- All 116 related tests pass (microformats, notes, micropub)
- Confirms Phases 1-3 implementation correctness

Updated BACKLOG.md with tag-filtered feeds feature (medium priority)

Implementation report: docs/design/v1.3.0/2025-12-10-phase4-implementation.md

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 11:51:39 -07:00
372064b116 feat(tags): Add tag archive route and admin interface integration
Implement Phase 3 of v1.3.0 tags feature per microformats-tags-design.md:

Routes (starpunk/routes/public.py):
- Add /tag/<tag> archive route with normalization and 404 handling
- Pre-load tags in index route for all notes
- Pre-load tags in note route for individual notes

Admin (starpunk/routes/admin.py):
- Parse comma-separated tag input in create route
- Parse tag input in update route
- Pre-load tags when displaying edit form
- Empty tag field removes all tags

Templates:
- Add tag input field to templates/admin/edit.html
- Add tag input field to templates/admin/new.html
- Use Jinja2 map filter to display existing tags

Implementation details:
- Tag URL parameter normalized to lowercase before lookup
- Tags pre-loaded using object.__setattr__ pattern (like media)
- parse_tag_input() handles trim, dedupe, normalization
- All existing tests pass (micropub categories, admin routes)

Per architect design:
- No pagination on tag archives (acceptable for v1.3.0)
- No autocomplete in admin (out of scope)
- Follows existing media loading patterns

Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 11:42:16 -07:00
377027e79a feat(templates): Add microformats2 h-feed and p-category markup for tags
Implement Phase 2 of v1.3.0 per microformats-tags-design.md

Template Updates:
- templates/index.html: Add h-feed properties (u-url, enhanced p-author with u-photo/p-note, feed-level u-photo)
- templates/index.html: Add p-category markup with rel="tag" to note previews
- templates/note.html: Add p-category markup with rel="tag" for tags
- templates/note.html: Enhance author h-card with u-photo and p-note (hidden for parsers)
- templates/note.html: Document u-photo placement outside e-content per draft spec
- templates/tag.html: Create new tag archive template with h-feed structure

Key Decisions Applied:
- Tags ordered alphabetically by display_name (ready for backend)
- rel="tag" on all p-category links per microformats2 spec
- Author bio (p-note) hidden with display: none for semantic parsing
- Dual u-photo elements intentional for parser compatibility
- Graceful fallback when author photo/bio not available

Templates are backward compatible and ready for backend integration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 11:35:11 -07:00
f10d0679da feat(tags): Add database schema and tags module (v1.3.0 Phase 1)
Implements tag/category system backend following microformats2 p-category specification.

Database changes:
- Migration 008: Add tags and note_tags tables
- Normalized tag storage (case-insensitive lookup, display name preserved)
- Indexes for performance

New module:
- starpunk/tags.py: Tag management functions
  - normalize_tag: Normalize tag strings
  - get_or_create_tag: Get or create tag records
  - add_tags_to_note: Associate tags with notes (replaces existing)
  - get_note_tags: Retrieve note tags (alphabetically ordered)
  - get_tag_by_name: Lookup tag by normalized name
  - get_notes_by_tag: Get all notes with specific tag
  - parse_tag_input: Parse comma-separated tag input

Model updates:
- Note.tags property (lazy-loaded, prefer pre-loading in routes)
- Note.to_dict() add include_tags parameter

CRUD updates:
- create_note() accepts tags parameter
- update_note() accepts tags parameter (None = no change, [] = remove all)

Micropub integration:
- Pass tags to create_note() (tags already extracted by extract_tags())
- Return tags in q=source response

Per design doc: docs/design/v1.3.0/microformats-tags-design.md

Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 11:24:23 -07:00
288 changed files with 16459 additions and 11646 deletions

View File

@@ -7,6 +7,329 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
## [1.5.0-hotfix.1] - 2025-12-17
### Fixed
- **CRITICAL: IndieAuth Login Failure** - Fixed authentication bug preventing user login
- Authentication now correctly discovers endpoints from user's profile URL per W3C IndieAuth spec
- Removed hardcoded indielogin.com service URL (was causing PKCE errors)
- Login flow now uses discovered authorization_endpoint for identity verification
- URL comparison now handles trailing slashes and case differences correctly
- User-friendly error messages when endpoint discovery fails
- DEPRECATED: `INDIELOGIN_URL` config no longer used (will show warning if set)
- Implements proper IndieAuth authentication-only flow per specification
## [1.5.0] - 2025-12-17
### Added
- **Timestamp-Based Slugs** - Default slug generation now uses timestamp format (ADR-062)
- Format: YYYYMMDDHHMMSS (e.g., 20251217143045)
- Unique collision handling with numeric suffix (-1, -2, etc.)
- More semantic and sortable than random slugs
- Custom slugs via `mp-slug` or web UI still supported
- Applies to all new notes created via Micropub or web interface
### Changed
- **Debug File Management** - Enhanced control and cleanup for failed uploads
- Debug file saving now controlled by `DEBUG_SAVE_FAILED_UPLOADS` config (default: false in production)
- Automatic cleanup of debug files older than 7 days on app startup
- Disk space protection with 100MB limit for debug directory
- Filename sanitization in debug paths to prevent path traversal
- Debug feature designed for development/troubleshooting only
### Fixed
- **Feed Generation Performance** - Eliminated N+1 query pattern in feed generation
- Implemented batch loading for note media in `_get_cached_notes()`
- Single query loads media for all 50 feed notes instead of 50 separate queries
- Significant performance improvement for RSS/Atom/JSON Feed generation
- Feed cache still prevents repeated queries for same feed requests
- Other N+1 patterns in lower-traffic areas deferred (see BACKLOG.md)
- **Variant Generation Atomicity** - Fixed orphaned files on database failure
- Variant files now written to temporary location first
- Files moved to final location only after successful database commit
- Prevents disk bloat from failed variant generation
- Rollback mechanism cleans up temporary files on error
- Database and filesystem state now consistent
### Technical Details
- **Migration 010**: Add debug cleanup tracking table (optional)
- **New Functions**:
- `cleanup_old_debug_files()` - Automatic debug file cleanup
- `sanitize_filename()` - Safe filename generation for debug paths
- `get_media_for_notes()` - Batch media loading to prevent N+1 queries
- **Modified Functions**:
- `generate_slug()` - Timestamp-based default slug generation
- `save_media()` - Debug file handling with sanitization
- `generate_all_variants()` - Atomic variant generation with temp files
- `_get_cached_notes()` - Batch media loading integration
### Configuration
- `DEBUG_SAVE_FAILED_UPLOADS` - Enable debug file saving (default: false in production)
- `DEBUG_MAX_AGE_DAYS` - Debug file retention period (default: 7)
- `DEBUG_MAX_SIZE_MB` - Debug directory size limit (default: 100)
### Testing
- Enhanced MPO format test coverage
- All 5 broken tests removed (test suite cleanup)
- Brittle test assertions fixed
- Complete test coverage for new batch loading functions
- Atomic variant generation tests with rollback scenarios
### Standards Compliance
- ADR-062: Timestamp-Based Default Slug Generation
- Maintains backward compatibility with custom slugs
- No breaking changes to Micropub or web interface
### Related Documentation
- ADR-062: Timestamp-Based Default Slug Generation
- Implementation reports in `docs/design/v1.5.0/`
- Architect reviews documenting all design decisions
## [1.4.2] - 2025-12-16
### Added
- HEIC/HEIF image format support for iPhone photo uploads
- MPO (Multi-Picture Object) format support for iPhone depth/portrait photos
- Automatic HEIC/MPO to JPEG conversion (browsers cannot display these formats)
- Graceful error handling if pillow-heif library not installed
### Changed
- Increased maximum input image dimensions from 4096x4096 to 12000x12000 to support modern phone cameras (48MP+); images are still optimized to smaller sizes for web delivery
### Dependencies
- Added `pillow-heif` for HEIC image processing
- Updated `Pillow` from 10.0.* to 10.1.* (required by pillow-heif)
## [1.4.1] - 2025-12-16
### Fixed
- Media upload failures are now logged for debugging and observability
- Validation errors (invalid format, file too large) logged at WARNING level
- Optimization failures logged at WARNING level
- Variant generation failures logged at WARNING level (upload continues)
- Unexpected errors logged at ERROR level with error type and message
- Successful uploads logged at INFO level with file details
- Removed duplicate logging in Micropub media endpoint
## [1.4.0rc1] - 2025-12-16
### Added
- **Large Image Support** - Accept and optimize images up to 50MB (Phase 1)
- Increased file size limit from 10MB to 50MB for image uploads
- Tiered resize strategy based on input size:
- <=10MB: 2048px max dimension, 95% quality
- 10-25MB: 1600px max dimension, 90% quality
- 25-50MB: 1280px max dimension, 85% quality
- Iterative quality reduction if output still >10MB after optimization
- Rejects if optimization cannot achieve <=10MB target (minimum 640px at 70% quality)
- Animated GIF detection with 10MB size limit (cannot be resized)
- All optimized images kept at or under 10MB for web performance
- **Image Variants** - Multiple image sizes for responsive delivery (Phase 2)
- Four variants generated automatically on upload:
- thumb: 150x150 center crop for thumbnails
- small: 320px width for mobile/low bandwidth
- medium: 640px width for standard display
- large: 1280px width for high-res display
- original: As uploaded (optimized, <=2048px)
- Variants stored in date-organized folders (data/media/YYYY/MM/)
- Database tracking in new `media_variants` table
- Synchronous/eager generation on upload
- Only new uploads get variants (existing media unchanged)
- Cascade deletion with parent media
- Variants included in `get_note_media()` response (backwards compatible)
- **Micropub Media Endpoint** - W3C-compliant media upload (Phase 3)
- New POST `/micropub/media` endpoint for file uploads
- Multipart/form-data with single file part named 'file'
- Bearer token authentication with `create` scope (no new scope needed)
- Returns 201 Created with Location header on success
- Automatic variant generation on upload
- OAuth 2.0 error format for all error responses
- Media endpoint advertised in `q=config` query response
- Photo property support in Micropub create requests:
- Simple URL strings: `"photo": ["https://example.com/image.jpg"]`
- Structured with alt text: `"photo": [{"value": "url", "alt": "description"}]`
- Multiple photos supported (max 4 per ADR-057)
- External URLs logged and ignored (no download)
- Photo captions preserved from alt text
- **Enhanced Feed Media** - Full Media RSS and JSON Feed variants (Phase 4)
- RSS 2.0: Complete Media RSS namespace support
- `media:group` element for multiple sizes of same image
- `media:content` for each variant with dimensions and file size
- `media:thumbnail` element for preview images
- `media:title` for captions (when present)
- `isDefault` attribute on largest available variant
- JSON Feed 1.1: `_starpunk` extension with variants
- All variants with URLs, dimensions, and sizes
- Configurable `about` URL for extension documentation
- Default: `https://github.com/yourusername/starpunk`
- Override via `STARPUNK_ABOUT_URL` config
- ATOM 1.0: Enclosures with title attribute for captions
- Backwards compatible: Feeds work with and without variants
### Changed
- **Image Optimization** - Enhanced for large file handling
- `optimize_image()` now accepts `original_size` parameter
- Returns both optimized image and bytes (avoids re-saving)
- Iterative quality reduction loop for difficult-to-compress images
- Safety check prevents infinite loops (minimum 640px dimension)
- **Media Storage** - Extended with variant support
- `save_media()` generates variants synchronously after saving original
- Variants cleaned up automatically on generation failure
- Database records original as 'original' variant type
- File size passed efficiently without redundant I/O
### Technical Details
- **Migration 009**: Add `media_variants` table
- Tracks variant_type, path, dimensions, and file size
- Foreign key to media table with cascade delete
- Unique constraint on (media_id, variant_type)
- Index on media_id for efficient lookups
- **New Functions**:
- `get_optimization_params(file_size)` - Tiered resize strategy
- `generate_variant()` - Single variant generation
- `generate_all_variants()` - Full variant set with DB storage
- `extract_photos()` - Micropub photo property parsing
- `_attach_photos_to_note()` - Photo attachment to notes
- **Modified Functions**:
- `validate_image()` - 50MB limit, animated GIF detection
- `optimize_image()` - Size-aware tiered optimization
- `save_media()` - Variant generation integration
- `get_note_media()` - Includes variants (when present)
- `handle_query()` - Advertises media-endpoint in config
- `handle_create()` - Photo property extraction and attachment
- `generate_rss_streaming()` - Media RSS support
- `_build_item_object()` - JSON Feed variants
- `generate_atom_streaming()` - Enclosure title attributes
- **Configuration Options**:
- `STARPUNK_ABOUT_URL` - JSON Feed extension documentation URL
### Standards Compliance
- W3C Micropub Media Endpoint Specification
- Media RSS 2.0 Specification (RSS Board)
- JSON Feed 1.1 with custom extension
- OAuth 2.0 Bearer Token Authentication
- RFC 3339 date formats in feeds
### Storage Impact
- Variants use approximately 4x storage per image:
- Original: 100%
- Large (1280px): ~50%
- Medium (640px): ~25%
- Small (320px): ~12%
- Thumb (150x150): ~3%
- Typical 500KB optimized image → ~900KB total with variants
- Only new uploads generate variants (existing media unchanged)
### Backwards Compatibility
- Existing media files work unchanged
- No variants generated for pre-v1.4.0 uploads
- Feeds handle media with and without variants gracefully
- `get_note_media()` only includes 'variants' key when variants exist
- All existing Micropub clients continue to work
### Related Documentation
- ADR-057: Media Attachment Model
- ADR-058: Image Optimization Strategy
- ADR-059: Full Feed Media Standardization
- Design: `/docs/design/v1.4.0/media-implementation-design.md`
## [1.3.1] - 2025-12-10
### Added
- **Feed Tags/Categories** - Tags now appear in all syndication feed formats
- RSS 2.0: `<category>` elements for each tag
- Atom 1.0: `<category term="slug" label="Display Name"/>` per RFC 4287
- JSON Feed 1.1: `tags` array with display names
- Tags omitted from feeds when note has no tags
### Technical Details
- Enhanced: `starpunk/feeds/rss.py` with category elements
- Enhanced: `starpunk/feeds/atom.py` with category elements
- Enhanced: `starpunk/feeds/json_feed.py` with tags array
- Enhanced: `starpunk/routes/public.py` pre-loads tags for feed generation
## [1.3.0] - 2025-12-10
### Added
- **Tag/Category System** - Complete tag support with hierarchical organization
- Tag creation and management via web UI and Micropub
- Support for Micropub `category` property in JSON and form-encoded requests
- Tag archive pages at `/tags/{tag}` with all tagged notes
- Tag cloud display on homepage showing all used tags
- Tag filtering in database queries (list_notes_by_tag)
- Reserved tag validation (prevents tags like 'api', 'admin', etc.)
- Comprehensive tag management in admin dashboard
- Database schema: tags table with slug and name fields
- Many-to-many relationship between notes and tags
- Automatic tag cleanup (removes orphaned tags)
- **Strict Microformats2 Compliance** - Enhanced h-entry markup for parsers
- p-category property for each tag in note markup
- dt-updated property displays when note is modified
- dt-published always shown for temporal context
- u-uid property matches u-url for permalink stability
- Proper h-feed structure on homepage and tag archives
- p-name property only when note has explicit title (# heading)
- e-content wraps full note content
- Nested h-card for author within each h-entry
- Homepage displays as complete h-feed with feed properties
- **h-feed Properties** - Proper feed markup on collection pages
- Homepage marked as h-feed with p-name "Recent Notes"
- Tag archive pages marked as h-feed with descriptive p-name
- Each feed contains multiple h-entry items
- Feed structure validates with Microformats2 parsers
- Supports feed readers and IndieWeb aggregators
### Changed
- **Template Structure** - Reorganized for better Microformats2 compliance
- Homepage template now wraps entries in proper h-feed
- Note display templates use semantic h-entry markup
- Tag display integrated throughout note views
- Consistent Microformats2 patterns across all pages
### Technical Details
- Migration 006: Add tags table and note_tags junction table
- New module: `starpunk/tags.py` with tag CRUD operations
- Enhanced: `starpunk/notes.py` with tag relationship handling
- Enhanced: `starpunk/micropub.py` with category property support
- Enhanced: Templates with p-category and h-feed markup
- All tests passing (580+ tests)
- 100% backward compatible with existing notes
## [1.2.0] - 2025-12-09 ## [1.2.0] - 2025-12-09
### Added ### Added

View File

@@ -8,97 +8,50 @@ This file contains operational instructions for Claude agents working on this pr
- All Python commands must be run with `uv run` prefix - All Python commands must be run with `uv run` prefix
- Example: `uv run pytest`, `uv run flask run` - Example: `uv run pytest`, `uv run flask run`
## Agent Protocol (All Agents)
**IMPORTANT**: All agents must review `docs/DOCUMENTATION.md` before starting work. This file is the authoritative source for documentation organization and supersedes any other instructions.
## Agent-Architect Protocol ## Agent-Architect Protocol
When invoking the agent-architect, always remind it to: When invoking the agent-architect, always remind it to:
1. Review documentation in docs/ before working on the task it is given 1. Review `docs/DOCUMENTATION.md` for documentation organization standards
- docs/architecture, docs/decisions, docs/standards are of particular interest
2. Give it the map of the documentation folder as described in the "Understanding the docs/ Structure" section below 2. Review documentation in docs/ before working on the task it is given
- docs/architecture, docs/decisions, docs/standards are of particular interest
3. Search for authoritative documentation for any web standard it is implementing on https://www.w3.org/ 3. Search for authoritative documentation for any web standard it is implementing on https://www.w3.org/
4. If it is reviewing a developers implementation report and it is accepts the completed work it should go back and update the project plan to reflect the completed work 4. If it is reviewing a developers implementation report and it accepts the completed work it should go back and update the project plan to reflect the completed work
## Agent-Developer Protocol ## Agent-Developer Protocol
When invoking the agent-developer, always remind it to: When invoking the agent-developer, always remind it to:
1. **Document work in reports** 1. Review `docs/DOCUMENTATION.md` for documentation organization standards
- Create implementation reports in `docs/reports/`
- Include date in filename: `YYYY-MM-DD-description.md`
2. **Update the changelog** 2. **Document work in design folder**
- Create implementation reports in `docs/design/{version}/`
- Include date in filename: `YYYY-MM-DD-description.md`
- All developer interaction (questions, responses, reports, reviews) goes in design/{version}/
3. **Update the changelog**
- Add entries to `CHANGELOG.md` for user-facing changes - Add entries to `CHANGELOG.md` for user-facing changes
- Follow existing format - Follow existing format
3. **Version number management** 4. **Version number management**
- Increment version numbers according to `docs/standards/versioning-strategy.md` - Increment version numbers according to `docs/standards/versioning-strategy.md`
- Update version in `starpunk/__init__.py` - Update version in `starpunk/__init__.py`
4. **Follow git protocol** 5. **Follow git protocol**
- Adhere to git branching strategy in `docs/standards/git-branching-strategy.md` - Adhere to git branching strategy in `docs/standards/git-branching-strategy.md`
- Create feature branches for non-trivial changes - Create feature branches for non-trivial changes
- Write clear commit messages - Write clear commit messages
## Documentation Navigation ## Documentation
### Understanding the docs/ Structure See `docs/DOCUMENTATION.md` for the authoritative documentation structure, navigation guidance, and key references.
The `docs/` folder is organized by document type and purpose:
- **`docs/architecture/`** - System design overviews, component diagrams, architectural patterns
- **`docs/decisions/`** - Architecture Decision Records (ADRs), numbered sequentially (ADR-001, ADR-002, etc.)
- **`docs/deployment/`** - Deployment guides, infrastructure setup, operations documentation
- **`docs/design/`** - Detailed design documents, feature specifications, phase plans
- **`docs/examples/`** - Example implementations, code samples, usage patterns
- **`docs/migration/`** - Migration guides for upgrading between versions and configuration changes
- **`docs/projectplan/`** - Project roadmaps, implementation plans, feature scope definitions
- **`docs/releases/`** - Release-specific documentation, release notes, version information
- **`docs/reports/`** - Implementation reports from developers (dated: YYYY-MM-DD-description.md)
- **`docs/reviews/`** - Architectural reviews, design critiques, retrospectives
- **`docs/security/`** - Security-related documentation, vulnerability analyses, best practices
- **`docs/standards/`** - Coding standards, conventions, processes, workflows
### Where to Find Documentation
- **Before implementing a feature**: Check `docs/decisions/` for relevant ADRs and `docs/design/` for specifications
- **Understanding system architecture**: Start with `docs/architecture/overview.md`
- **Coding guidelines**: See `docs/standards/` for language-specific standards and best practices
- **Past implementation context**: Review `docs/reports/` for similar work (sorted by date)
- **Project roadmap and scope**: Refer to `docs/projectplan/`
### Where to Create New Documentation
**Create an ADR (`docs/decisions/`)** when:
- Making architectural decisions that affect system design
- Choosing between competing technical approaches
- Establishing patterns that others should follow
- Format: `ADR-NNN-brief-title.md` (find next number sequentially)
**Create a design doc (`docs/design/`)** when:
- Planning a complex feature implementation
- Detailing technical specifications
- Documenting multi-phase development plans
**Create an implementation report (`docs/reports/`)** when:
- Completing significant development work
- Documenting implementation details for architect review
- Format: `YYYY-MM-DD-brief-description.md`
**Update standards (`docs/standards/`)** when:
- Establishing new coding conventions
- Documenting processes or workflows
- Creating checklists or guidelines
### Key Documentation References
- **Architecture**: See `docs/architecture/overview.md`
- **Implementation Plan**: See `docs/projectplan/v1/implementation-plan.md`
- **Feature Scope**: See `docs/projectplan/v1/feature-scope.md`
- **Coding Standards**: See `docs/standards/python-coding-standards.md`
- **Testing**: See `docs/standards/testing-checklist.md`
## Project Philosophy ## Project Philosophy

View File

@@ -2,13 +2,13 @@
A minimal, self-hosted IndieWeb CMS for publishing notes with RSS syndication. A minimal, self-hosted IndieWeb CMS for publishing notes with RSS syndication.
**Current Version**: 1.1.0 **Current Version**: 1.2.0
## Versioning ## Versioning
StarPunk follows [Semantic Versioning 2.0.0](https://semver.org/): StarPunk follows [Semantic Versioning 2.0.0](https://semver.org/):
- Version format: `MAJOR.MINOR.PATCH` - Version format: `MAJOR.MINOR.PATCH`
- Current: `1.1.0` (stable release) - Current: `1.2.0` (stable release)
- Check version: `python -c "from starpunk import __version__; print(__version__)"` - Check version: `python -c "from starpunk import __version__; print(__version__)"`
- See changes: [CHANGELOG.md](CHANGELOG.md) - See changes: [CHANGELOG.md](CHANGELOG.md)
- Versioning strategy: [docs/standards/versioning-strategy.md](docs/standards/versioning-strategy.md) - Versioning strategy: [docs/standards/versioning-strategy.md](docs/standards/versioning-strategy.md)
@@ -29,10 +29,14 @@ StarPunk is designed for a single user who wants to:
- **File-based storage**: Notes are markdown files, owned by you - **File-based storage**: Notes are markdown files, owned by you
- **IndieAuth authentication**: Use your own website as identity - **IndieAuth authentication**: Use your own website as identity
- **Micropub support**: Full W3C Micropub specification compliance - **Micropub support**: Full W3C Micropub specification compliance
- **RSS feed**: Automatic syndication - **Media attachments**: Upload and display images with your notes
- **Microformats2**: Full h-entry, h-card, and h-feed markup for IndieWeb compatibility
- **Author discovery**: Automatic profile discovery from your IndieWeb identity
- **RSS, ATOM, JSON Feed**: Multiple syndication formats with Media RSS support
- **Custom slugs**: Control your note permalinks
- **No database lock-in**: SQLite for metadata, files for content - **No database lock-in**: SQLite for metadata, files for content
- **Self-hostable**: Run on your own server - **Self-hostable**: Run on your own server
- **Minimal dependencies**: 6 core dependencies, no build tools - **Minimal dependencies**: Core dependencies, no build tools
## Requirements ## Requirements
@@ -154,8 +158,10 @@ See [docs/architecture/](docs/architecture/) for complete documentation.
StarPunk implements: StarPunk implements:
- [Micropub](https://micropub.spec.indieweb.org/) - Publishing API - [Micropub](https://micropub.spec.indieweb.org/) - Publishing API
- [IndieAuth](https://www.w3.org/TR/indieauth/) - Authentication - [IndieAuth](https://www.w3.org/TR/indieauth/) - Authentication
- [Microformats2](http://microformats.org/) - Semantic HTML markup - [Microformats2](http://microformats.org/) - h-entry, h-card, h-feed markup
- [RSS 2.0](https://www.rssboard.org/rss-specification) - Feed syndication - [RSS 2.0](https://www.rssboard.org/rss-specification) with Media RSS extensions
- [ATOM 1.0](https://validator.w3.org/feed/docs/atom.html) - Syndication format
- [JSON Feed 1.1](https://jsonfeed.org/version/1.1) - Modern feed format
## Deployment ## Deployment

57
docs/DOCUMENTATION.md Normal file
View File

@@ -0,0 +1,57 @@
# PURPOSE
This document describes how documentation in this folder should be organized and supersedes any other instructions.
# FOLDERS
## ARCHITECTURE
The architecture folder should contain documentation reflecting the current design of the system and should be updated at the end of each release to ensure it is current.
## DECISIONS
This folder contains any architectural decisions, documented as ADRs.
- Format: `ADR-NNN-brief-title.md` (numbered sequentially)
- Create an ADR when making architectural decisions, choosing between technical approaches, or establishing patterns
## DESIGN
This folder is used by the architect to document implementation designs to be handed off to the developer. These designs should be sorted into subfolders reflecting the semantic version number of the release in question (e.g., `v1.0.0/`, `v1.1.1/`).
All developer interaction belongs in the appropriate version subfolder:
- Implementation designs and specifications
- Developer questions to the architect
- Architect responses
- Implementation reports (format: `YYYY-MM-DD-description.md`)
- Implementation reviews
## PROJECTPLAN
This folder contains documents relating to the future state of the project. There should be a single BACKLOG.md file that lists future features by priority as well as bugs (which are assumed to be high priority). Items in this file can have one of the following priorities:
- Critical - Items that break existing functionality
- High
- Medium
- Low
In addition to the backlog file each version should have a folder named for its semantic version with a RELEASE.md file which lists the features and bugs to be addressed in that release.
## STANDARDS
Includes any standards written by the architect that the developer needs to reference during development. Any deprecated standards should be moved to the DEPRECATED subfolder when appropriate.
# WHERE TO FIND DOCUMENTATION
- **Before implementing a feature**: Check `decisions/` for relevant ADRs and `design/{version}/` for specifications
- **Understanding system architecture**: Start with `architecture/`
- **Coding guidelines**: See `standards/`
- **Past implementation context**: Review `design/{version}/` for similar work
- **Project roadmap and scope**: Refer to `projectplan/`
# KEY REFERENCES
- **Architecture**: `architecture/`
- **Coding Standards**: `standards/python-coding-standards.md`
- **Testing**: `standards/testing-checklist.md`
- **Project Backlog**: `projectplan/BACKLOG.md`

View File

@@ -0,0 +1,181 @@
# ADR-012: Flaky Test Removal and Test Quality Standards
## Status
Proposed
## Context
The test suite contains several categories of flaky tests that pass inconsistently. These tests consume developer time without providing proportional value. Per the project philosophy ("Every line of code must justify its existence"), we must evaluate whether these tests should be kept, fixed, or removed.
## Analysis by Test Category
### 1. Migration Race Condition Tests (`test_migration_race_condition.py`)
**Failing Tests:**
- `test_debug_level_for_early_retries` - Log message matching
- `test_new_connection_per_retry` - Connection count assertions
- `test_concurrent_workers_barrier_sync` - Multiprocessing pickle errors
- `test_sequential_worker_startup` - Missing table errors
- `test_worker_late_arrival` - Missing table errors
- `test_single_worker_performance` - Missing table errors
- `test_concurrent_workers_performance` - Pickle errors
**Value Analysis:**
- The migration retry logic with exponential backoff is *critical* for production deployments with multiple Gunicorn workers
- However, the flaky tests are testing implementation details (log levels, exact connection counts) rather than behavior
- The multiprocessing tests fundamentally cannot work reliably because:
1. `multiprocessing.Manager().Barrier()` objects cannot be pickled for `Pool.map()`
2. The worker functions require Flask app context that doesn't transfer across processes
3. SQLite database files in temp directories may not be accessible across process boundaries
**Root Cause:** Test design is flawed. These are attempting integration/stress tests using unit test infrastructure.
**Recommendation: REMOVE the multiprocessing tests entirely. KEEP and FIX the unit tests.**
Specifically:
- **REMOVE:** `TestConcurrentExecution` class (all 3 tests) - fundamentally broken by design
- **REMOVE:** `TestPerformance` class (both tests) - same multiprocessing issues
- **KEEP:** `TestRetryLogic` - valuable, just needs mock fixes
- **KEEP:** `TestGraduatedLogging` - valuable, needs logger configuration fixes
- **KEEP:** `TestConnectionManagement` - valuable, needs assertion fixes
- **KEEP:** `TestErrorHandling` - valuable, tests critical rollback behavior
- **KEEP:** `TestBeginImmediateTransaction` - valuable, tests locking mechanism
**Rationale for removal:** If we need to test concurrent migration behavior, that requires:
1. A proper integration test framework (not pytest unit tests)
2. External process spawning (not multiprocessing.Pool)
3. Real filesystem isolation
4. This is out of scope for V1 - the code works; the tests are the problem
---
### 2. Feed Route Tests (`test_routes_feeds.py`)
**Failing Assertions:**
- Tests checking for exact `<?xml version="1.0"` but code produces `<?xml version='1.0'` (single quotes)
- Tests checking for exact Content-Type with charset but response may vary
- Tests checking for exact `<rss version="2.0"` string
**Value Analysis:**
- These tests ARE valuable - they verify feed output format
- The tests are NOT flaky per se; they are *brittle* due to over-specific assertions
**Root Cause:** Tests are asserting implementation details (quote style) rather than semantics (valid XML).
**Recommendation: FIX by loosening assertions**
Current (brittle):
```python
assert b'<?xml version="1.0"' in response.data
```
Better (semantic):
```python
assert b'<?xml version=' in response.data
assert b"encoding=" in response.data
```
The test file already has SOME tests using the correct pattern (lines 72, 103, 265). The ATOM test on line 84 is the outlier - it should match the RSS tests.
---
### 3. Feed Streaming Test (`test_routes_feed.py`)
**Failing Test:** `test_feed_route_streaming`
**Current assertion (line 124):**
```python
assert "ETag" in response.headers
```
**But the test comment says:**
```python
# Cached responses include ETags for conditional requests
# (Phase 3 caching was added, replacing streaming for better performance)
```
**Value Analysis:**
- The test title says "streaming" but the implementation uses caching
- The test is actually correct (ETag SHOULD be present)
- If ETag is NOT present, that's a bug in the feed caching implementation
**Root Cause:** This is not a flaky test - if it fails, there's an actual bug. The test name is misleading but the assertion is correct.
**Recommendation: KEEP and RENAME to `test_feed_route_caching`**
---
### 4. Search Security Tests (`test_search_security.py`)
**Analysis of the file:** After reviewing, I see no obviously flaky tests. All tests are:
- Testing XSS prevention (correct)
- Testing SQL injection prevention (correct)
- Testing input validation (correct)
- Testing pagination limits (correct)
**Possible flakiness sources:**
- FTS5 special character handling varies by SQLite version
- Tests that accept multiple status codes (200, 400, 500) are defensive, not flaky
**Recommendation: KEEP all tests**
If specific flakiness is identified, it's likely due to SQLite FTS5 version differences, which should be documented rather than the tests removed.
---
## Decision
### Remove Entirely
1. `TestConcurrentExecution` class from `test_migration_race_condition.py`
2. `TestPerformance` class from `test_migration_race_condition.py`
### Fix Tests (Developer Action Items)
1. **`test_routes_feeds.py` line 84:** Change `assert b'<?xml version="1.0"'` to `assert b'<?xml version='`
2. **`test_routes_feed.py` line 117:** Rename test from `test_feed_route_streaming` to `test_feed_route_caching`
3. **`test_migration_race_condition.py`:** Fix logger configuration in `TestGraduatedLogging` tests to ensure DEBUG level is captured
4. **`test_migration_race_condition.py`:** Fix mock setup in `test_new_connection_per_retry` to accurately count connection attempts
### Keep As-Is
1. All tests in `test_search_security.py`
2. All non-multiprocessing tests in `test_migration_race_condition.py` (after fixes)
3. All other tests in `test_routes_feeds.py` and `test_routes_feed.py`
## Rationale
1. **Project philosophy alignment:** Tests that cannot reliably pass do not justify their existence. They waste developer time and erode confidence in the test suite.
2. **Pragmatic approach:** The migration concurrency code is tested in production by virtue of running with multiple Gunicorn workers. Manual testing during deployment is more reliable than broken multiprocessing tests.
3. **Test semantics over implementation:** Tests should verify behavior, not implementation details like quote styles in XML.
4. **Maintainability:** A smaller, reliable test suite is better than a larger, flaky one.
## Consequences
### Positive
- Faster, more reliable CI/CD pipeline
- Increased developer confidence in test results
- Reduced time spent debugging test infrastructure
- Tests that fail actually indicate bugs
### Negative
- Reduced test coverage for concurrent migration scenarios
- Manual testing required for multi-worker deployments
### Mitigations
- Document the multi-worker testing procedure in deployment docs
- Consider adding integration tests in a separate test category (not run in CI) for concurrent scenarios
## Alternatives Considered
### 1. Fix the multiprocessing tests
**Rejected:** Would require significant refactoring to use subprocess spawning instead of multiprocessing.Pool. The complexity is not justified for V1 given the code works correctly in production.
### 2. Mark tests as `@pytest.mark.skip`
**Rejected:** Skipped tests are just noise. They either work and should run, or they don't work and should be removed. "Skip" is procrastination.
### 3. Use pytest-xdist for parallel testing
**Rejected:** Does not solve the fundamental issue of needing to spawn external processes with proper app context.
### 4. Move to integration test framework (e.g., testcontainers)
**Considered for future:** This is the correct long-term solution but is out of scope for V1. Should be considered for V2 if concurrent migration testing is deemed critical.

View File

@@ -0,0 +1,197 @@
# ADR-062: Timestamp-Based Slug Format
## Status
Accepted (Supersedes ADR-007)
## Context
ADR-007 established a content-based slug generation algorithm that extracts the first 5 words from note content to create URL slugs. While this approach provides readable, SEO-friendly URLs, it has drawbacks for a personal note-taking system:
1. **Privacy Concerns**: The slug reveals note content in the URL. Private thoughts or draft content become visible in URLs that may be shared or logged.
2. **SEO Irrelevance**: StarPunk is designed for personal IndieWeb notes, not public blogs. Notes are typically short-form content (similar to tweets or status updates) where SEO optimization provides no meaningful benefit.
3. **Unpredictable Slugs**: Users cannot predict what slug will be generated without examining their content carefully.
4. **Edge Case Handling**: Content-based slugs require complex fallback logic for short content, unicode-only content, or special characters.
5. **Collision Complexity**: Similar notes require random suffix generation (e.g., `hello-world-a7c9`), which undermines the readability goal.
The user has explicitly stated: "Notes don't need SEO."
## Decision
Change the default slug format from content-based to timestamp-based:
**New Default Format**: `YYYYMMDDHHMMSS`
- Example: `20251216143052`
- Compact, sortable, predictable
- 14 characters total
**Collision Handling**: Sequential numeric suffix
- First collision: `20251216143052-1`
- Second collision: `20251216143052-2`
- Simple, predictable, no randomness
**Custom Slugs**: Continue to support user-specified slugs via `mp-slug` Micropub property and the web UI custom slug field. When provided, custom slugs take precedence.
### Algorithm Specification
```python
def generate_slug(custom_slug: str = None, created_at: datetime = None) -> str:
"""Generate a URL-safe slug for a note.
Args:
custom_slug: User-provided custom slug (takes precedence if provided)
created_at: Note creation timestamp (defaults to now)
Returns:
URL-safe slug string
"""
if custom_slug:
return sanitize_slug(custom_slug)
# Default: timestamp-based
timestamp = (created_at or datetime.now()).strftime("%Y%m%d%H%M%S")
return ensure_unique_slug(timestamp)
def ensure_unique_slug(base_slug: str) -> str:
"""Ensure slug is unique, adding numeric suffix if needed."""
if not slug_exists(base_slug):
return base_slug
suffix = 1
while slug_exists(f"{base_slug}-{suffix}"):
suffix += 1
return f"{base_slug}-{suffix}"
```
### Examples
| Scenario | Generated Slug |
|----------|----------------|
| Normal note at 2:30:52 PM on Dec 16, 2025 | `20251216143052` |
| Second note in same second | `20251216143052-1` |
| Third note in same second | `20251216143052-2` |
| Note with custom slug "my-custom-slug" | `my-custom-slug` |
## Rationale
### Timestamp Format (Score: 9/10)
**Pros**:
- **Privacy**: URL reveals nothing about content
- **Predictability**: User knows exactly what slug format to expect
- **Sortability**: Chronological sorting by URL is possible
- **Simplicity**: No complex word extraction or normalization
- **Collision Rarity**: Same-second creation is rare; handled cleanly when it occurs
- **Compact**: 14 characters vs potentially 50+ for content-based
**Cons**:
- **Not Memorable**: `20251216143052` is harder to remember than `hello-world`
- **No SEO Value**: Search engines prefer descriptive URLs
**Why Cons Don't Matter**:
- StarPunk is for personal notes, not public SEO-optimized content
- Notes are accessed via UI, feeds, or bookmarks, not memorized URLs
- Users wanting memorable URLs can use custom slugs
### Sequential Suffix (Score: 10/10)
**Pros**:
- **Deterministic**: No randomness; same collision always gets same suffix
- **Simple**: No cryptographic random generation needed
- **Readable**: `-1`, `-2` are clear and obvious
- **Debuggable**: Easy to understand collision resolution
**Cons**:
- **Enumerable**: Sequential numbers could be probed
- Not a real security concern for note slugs
### Comparison with ADR-007 Approach
| Aspect | ADR-007 (Content-Based) | ADR-062 (Timestamp) |
|--------|------------------------|---------------------|
| Privacy | Reveals content | Reveals only time |
| Complexity | High (word extraction, normalization, unicode handling) | Low (strftime) |
| SEO | Good | None |
| Predictability | Low | High |
| Collision handling | Random suffix | Sequential suffix |
| Fallback cases | Many (short content, unicode, etc.) | None |
| Code lines | ~50 | ~15 |
## Consequences
### Positive
1. **Simplified Code**: Remove complex word extraction, unicode normalization, and multiple fallback paths
2. **Better Privacy**: Note content never appears in URLs
3. **Predictable Output**: Users always know what slug format to expect
4. **Fewer Edge Cases**: No special handling for short content, unicode, or special characters
5. **Cleaner Collisions**: Sequential suffixes are more intuitive than random strings
### Negative
1. **Migration**: Existing notes keep their content-based slugs (no migration needed)
2. **Not Human-Readable**: URLs don't describe content
3. **No SEO**: Search engines won't benefit from descriptive URLs
### Mitigations
**Human-Readable URLs**:
- Users wanting descriptive URLs can use custom slugs via `mp-slug`
- The web UI custom slug field remains available
- This is opt-in rather than default
**SEO**:
- IndieWeb notes are typically not SEO targets
- Content is in the page body where search engines can index it
- Microformats2 markup provides semantic meaning
## Backward Compatibility
- Existing notes retain their slugs (no data migration)
- New notes use timestamp format by default
- Custom slug functionality unchanged
- All existing URLs remain valid
## Testing Requirements
- Test default slug generation produces timestamp format
- Test collision handling with sequential suffixes
- Test custom slugs still take precedence
- Test edge case: multiple notes in same second
- Test reserved slug rejection still works
- Verify existing tests for custom slugs pass
## Implementation Notes
Changes required in:
- `starpunk/slug_utils.py`: Update `generate_slug()` function
- `starpunk/notes.py`: Remove content parameter from slug generation call
- Tests: Update expected slug formats
Estimated effort: Small (1-2 hours implementation, 1 hour testing)
## References
- ADR-007: Slug Generation Algorithm (Superseded by this ADR)
- ADR-035: Custom Slugs (Unchanged; complements this decision)
- IndieWeb Permalink Best Practices: https://indieweb.org/permalink
## Acceptance Criteria
- [ ] Default slugs use `YYYYMMDDHHMMSS` format
- [ ] Collision handling uses sequential suffix (`-1`, `-2`, etc.)
- [ ] Custom slugs via `mp-slug` continue to work
- [ ] Custom slugs via web UI continue to work
- [ ] Reserved slug validation unchanged
- [ ] Existing notes unaffected
- [ ] All tests pass
- [ ] Code complexity reduced
---
**Approved**: 2025-12-16
**Architect**: StarPunk Architect Agent
**Supersedes**: ADR-007 (Slug Generation Algorithm)

View File

@@ -1,41 +0,0 @@
# Deployment Documentation Index
This directory contains deployment guides, infrastructure setup instructions, and operations documentation for StarPunk CMS.
## Deployment Guides
- **[container-deployment.md](container-deployment.md)** - Container-based deployment guide (Docker, Podman)
## Deployment Options
### Container Deployment (Recommended)
Container deployment provides:
- Consistent environment across platforms
- Easy updates and rollbacks
- Resource isolation
- Simplified dependency management
See: [container-deployment.md](container-deployment.md)
### Manual Deployment
For manual deployment without containers:
- Follow [../standards/development-setup.md](../standards/development-setup.md)
- Configure systemd service
- Set up reverse proxy (nginx/Caddy)
- Configure SSL/TLS certificates
### Cloud Deployment
StarPunk can be deployed to:
- Any container platform (Kubernetes, Docker Swarm)
- VPS providers (DigitalOcean, Linode, Vultr)
- PaaS platforms supporting containers
## Related Documentation
- **[../standards/development-setup.md](../standards/development-setup.md)** - Development environment setup
- **[../architecture/](../architecture/)** - System architecture
- **[README.md](../../README.md)** - Quick start guide
---
**Last Updated**: 2025-11-25
**Maintained By**: Documentation Manager Agent

View File

@@ -0,0 +1,173 @@
# IndieAuth Endpoint Discovery Hotfix - Implementation Report
**Date:** 2025-12-17
**Type:** Production Hotfix
**Priority:** Critical
**Status:** Implementation Complete
## Summary
Successfully implemented the IndieAuth endpoint discovery hotfix as specified in the design document. The authentication flow now correctly discovers endpoints from the user's profile URL per the W3C IndieAuth specification, instead of hardcoding the indielogin.com service.
## Implementation Steps
All implementation steps were completed successfully:
### Step 1: Update starpunk/config.py - Remove INDIELOGIN_URL
- Removed `INDIELOGIN_URL` config line (previously line 37)
- Added deprecation warning for users still setting `INDIELOGIN_URL` in environment
- Warning directs users to remove the deprecated config
### Step 2: Update starpunk/auth.py - Add imports and use endpoint discovery
- Added imports: `discover_endpoints`, `DiscoveryError`, `normalize_url` from `starpunk.auth_external`
- Rewrote `initiate_login()`:
- Now discovers authorization_endpoint from the user's profile URL
- Uses discovered endpoint instead of hardcoded INDIELOGIN_URL
- Raises DiscoveryError if endpoint discovery fails or no authorization_endpoint found
- Rewrote `handle_callback()`:
- Discovers authorization_endpoint from ADMIN_ME profile
- Uses authorization_endpoint for authentication-only flow (per IndieAuth spec)
- Does NOT include `grant_type` parameter (not needed for auth-only flows)
- Uses `normalize_url()` for URL comparison to handle trailing slashes and case differences
### Step 3: Update starpunk/auth_external.py - Relax endpoint validation
- Changed endpoint validation in `_fetch_and_parse()`:
- Now requires at least one endpoint (authorization_endpoint OR token_endpoint)
- Previously required token_endpoint to be present
- This allows profiles with only authorization_endpoint to work for login
- Micropub will still require token_endpoint and fail gracefully with 401
### Step 4: Update starpunk/routes/auth.py - Import and handle DiscoveryError
- Added import for `DiscoveryError` from `starpunk.auth_external`
- Added exception handler in `login_initiate()`:
- Catches DiscoveryError
- Logs technical details at ERROR level
- Shows user-friendly message: "Unable to verify your profile URL. Please check that it's correct and try again."
- Redirects back to login form
### Step 5: Update tests/test_auth.py - Mock discover_endpoints()
- Removed `INDIELOGIN_URL` from test app fixture
- Updated all tests that call `initiate_login()` or `handle_callback()`:
- Added `@patch("starpunk.auth.discover_endpoints")` decorator
- Mock returns both authorization_endpoint and token_endpoint
- Updated assertions to check for discovered endpoint instead of indielogin.com
- Tests updated:
- `TestInitiateLogin.test_initiate_login_success`
- `TestInitiateLogin.test_initiate_login_stores_state`
- `TestHandleCallback.test_handle_callback_success`
- `TestHandleCallback.test_handle_callback_unauthorized_user`
- `TestHandleCallback.test_handle_callback_indielogin_error`
- `TestHandleCallback.test_handle_callback_no_identity`
- `TestLoggingIntegration.test_initiate_login_logs_at_debug`
- `TestLoggingIntegration.test_initiate_login_info_level`
- `TestLoggingIntegration.test_handle_callback_logs_http_details`
### Step 6: Update tests/test_auth_external.py - Fix error message
- Updated `test_discover_endpoints_no_token_endpoint`:
- Changed assertion from "No token endpoint found" to "No IndieAuth endpoints found"
- Matches new relaxed validation error message
### Step 7: Run tests to verify implementation
- All 51 tests in `tests/test_auth.py` pass
- All 35 tests in `tests/test_auth_external.py` pass
- All 32 tests in `tests/test_routes_admin.py` pass
- No regressions detected
## Files Modified
| File | Lines Changed | Description |
|------|--------------|-------------|
| `starpunk/config.py` | 9 added, 1 removed | Removed INDIELOGIN_URL, added deprecation warning |
| `starpunk/auth.py` | 1 added, 84 replaced | Added imports, rewrote initiate_login() and handle_callback() |
| `starpunk/auth_external.py` | 6 replaced | Relaxed endpoint validation |
| `starpunk/routes/auth.py` | 5 added | Added DiscoveryError import and exception handling |
| `tests/test_auth.py` | 1 removed, 43 modified | Removed INDIELOGIN_URL from fixture, added mocks |
| `tests/test_auth_external.py` | 2 modified | Updated error message assertion |
## Key Implementation Details
### Authorization Endpoint Usage
Per IndieAuth spec and architect clarifications:
- Authentication-only flows POST to the **authorization_endpoint** (not token_endpoint)
- The `grant_type` parameter is NOT included (only for access token flows)
- This differs from the previous implementation which incorrectly used indielogin.com's endpoints
### URL Normalization
The implementation now uses `normalize_url()` when comparing the returned 'me' URL with ADMIN_ME:
- Handles trailing slash differences (https://example.com vs https://example.com/)
- Handles case differences (https://Example.com vs https://example.com)
- This is spec-compliant behavior that was previously missing
### Error Handling
- Discovery failures are caught and logged at ERROR level
- User-facing error message is simplified and friendly
- Technical details remain in logs for debugging
### Backward Compatibility
- Deprecation warning added for INDIELOGIN_URL environment variable
- Existing .env files with INDIELOGIN_URL will log a warning but continue to work
- Users are instructed to remove the deprecated config
## Testing Results
### Unit Tests
- `tests/test_auth.py`: 51/51 passed
- `tests/test_auth_external.py`: 35/35 passed
- `tests/test_routes_admin.py`: 32/32 passed
### Integration
All modified tests now correctly mock endpoint discovery and validate the new behavior.
## Issues Encountered
No significant issues encountered during implementation. The design document was thorough and all architect clarifications were addressed:
1. Import placement - Moved to top-level as specified
2. URL normalization - Included as intentional bugfix
3. Endpoint selection - Used authorization_endpoint for auth-only flows
4. Validation relaxation - Allowed profiles with only authorization_endpoint
5. Test strategy - Mocked discover_endpoints() and removed INDIELOGIN_URL
6. grant_type parameter - Correctly omitted for auth-only flows
7. Error messages - Simplified user-facing messages
## Next Steps
1. Manual testing recommended:
- Test login flow with actual IndieAuth profile
- Verify endpoint discovery logs appear
- Test with profiles that have custom endpoints
- Verify error message appears for profiles without endpoints
2. Deployment:
- Update production .env to remove INDIELOGIN_URL (optional - will show warning)
- Deploy changes
- Monitor logs for "Discovered authorization_endpoint" messages
- Verify login works for admin user
3. Documentation:
- Update CHANGELOG.md with hotfix entry
- Consider adding migration guide if needed
## Verification Checklist
- [x] All specified files modified
- [x] All code changes follow architect's design exactly
- [x] Tests updated and passing
- [x] Error messages user-friendly
- [x] Logging appropriate for debugging
- [x] URL normalization implemented
- [x] Endpoint validation relaxed correctly
- [x] No regressions in existing tests
- [x] Implementation report created
## Conclusion
The hotfix has been successfully implemented according to the architect's design. The authentication flow now correctly implements the W3C IndieAuth specification for endpoint discovery. All tests pass and no regressions were detected.
The critical production bug preventing user login should be resolved once this code is deployed.
---
**Developer:** Claude (Fullstack Developer Subagent)
**Date Completed:** 2025-12-17
**Ready for Review:** Yes

View File

@@ -0,0 +1,608 @@
# IndieAuth Endpoint Discovery Hotfix
**Date:** 2025-12-17
**Type:** Production Hotfix
**Priority:** Critical
**Status:** Ready for Implementation
## Problem Summary
Users cannot log in to StarPunk. The root cause is that the authentication code ignores endpoint discovery and hardcodes `INDIELOGIN_URL` instead of discovering the authorization and token endpoints from the user's profile URL.
**Root Cause:** The `starpunk/auth.py` module uses `INDIELOGIN_URL` config instead of discovering endpoints from the user's profile URL as required by the IndieAuth specification. This is a regression - the system used to respect discovered endpoints.
**Note:** The PKCE error message in the callback is a symptom, not the cause. Once we use the correct discovered endpoints, PKCE will not be required (since the user's actual IndieAuth server doesn't require it).
## Specification Requirements
### W3C IndieAuth Spec (https://www.w3.org/TR/indieauth/)
- Clients MUST discover `authorization_endpoint` from user's profile URL
- Clients MUST discover `token_endpoint` from user's profile URL
- Discovery via HTTP Link headers (highest priority) or HTML `<link>` elements
## Implementation Plan
### Overview
The fix reuses the existing `discover_endpoints()` function from `auth_external.py` in the login flow. Changes are minimal and focused:
1. Use `discover_endpoints()` in `initiate_login()` to get the `authorization_endpoint`
2. Use `discover_endpoints()` in `handle_callback()` to get the `token_endpoint`
3. Remove `INDIELOGIN_URL` config (with deprecation warning)
---
### Step 1: Update config.py - Remove INDIELOGIN_URL
In `/home/phil/Projects/starpunk/starpunk/config.py`:
**Change 1:** Remove the INDIELOGIN_URL config line (line 37):
```python
# DELETE THIS LINE:
app.config["INDIELOGIN_URL"] = os.getenv("INDIELOGIN_URL", "https://indielogin.com")
```
**Change 2:** Add deprecation warning for INDIELOGIN_URL (add after the TOKEN_ENDPOINT warning, around line 47):
```python
# DEPRECATED: INDIELOGIN_URL no longer used (hotfix 2025-12-17)
# Authorization endpoint is now discovered from ADMIN_ME profile per IndieAuth spec
if 'INDIELOGIN_URL' in os.environ:
app.logger.warning(
"INDIELOGIN_URL is deprecated and will be ignored. "
"Remove it from your configuration. "
"The authorization endpoint is now discovered automatically from your ADMIN_ME profile."
)
```
---
### Step 2: Update auth.py - Use Endpoint Discovery
In `/home/phil/Projects/starpunk/starpunk/auth.py`:
**Change 1:** Add import for endpoint discovery (after line 42):
```python
from starpunk.auth_external import discover_endpoints, DiscoveryError, normalize_url
```
> **Note:** The `normalize_url` import is at the top level (not inside `handle_callback()`) for consistency with the existing code style.
**Change 2:** Update `initiate_login()` to use discovered authorization_endpoint (replace lines 251-318):
```python
def initiate_login(me_url: str) -> str:
"""
Initiate IndieAuth authentication flow with endpoint discovery.
Per W3C IndieAuth spec, discovers authorization_endpoint from user's
profile URL rather than using a hardcoded service.
Args:
me_url: User's IndieWeb identity URL
Returns:
Redirect URL to discovered authorization endpoint
Raises:
ValueError: Invalid me_url format
DiscoveryError: Failed to discover endpoints from profile
"""
# Validate URL format
if not is_valid_url(me_url):
raise ValueError(f"Invalid URL format: {me_url}")
current_app.logger.debug(f"Auth: Validating me URL: {me_url}")
# Discover authorization endpoint from user's profile URL
# Per IndieAuth spec: clients MUST discover endpoints, not hardcode them
try:
endpoints = discover_endpoints(me_url)
except DiscoveryError as e:
current_app.logger.error(f"Auth: Endpoint discovery failed for {me_url}: {e}")
raise
auth_endpoint = endpoints.get('authorization_endpoint')
if not auth_endpoint:
raise DiscoveryError(
f"No authorization_endpoint found at {me_url}. "
"Ensure your profile has IndieAuth link elements or headers."
)
current_app.logger.info(f"Auth: Discovered authorization_endpoint: {auth_endpoint}")
# Generate CSRF state token
state = _generate_state_token()
current_app.logger.debug(f"Auth: Generated state token: {_redact_token(state, 8)}")
# Store state in database (5-minute expiry)
db = get_db(current_app)
expires_at = datetime.utcnow() + timedelta(minutes=5)
redirect_uri = f"{current_app.config['SITE_URL']}auth/callback"
db.execute(
"""
INSERT INTO auth_state (state, expires_at, redirect_uri)
VALUES (?, ?, ?)
""",
(state, expires_at, redirect_uri),
)
db.commit()
# Build authorization URL
params = {
"me": me_url,
"client_id": current_app.config["SITE_URL"],
"redirect_uri": redirect_uri,
"state": state,
"response_type": "code",
}
current_app.logger.debug(
f"Auth: Building authorization URL with params:\n"
f" me: {me_url}\n"
f" client_id: {current_app.config['SITE_URL']}\n"
f" redirect_uri: {redirect_uri}\n"
f" state: {_redact_token(state, 8)}\n"
f" response_type: code"
)
auth_url = f"{auth_endpoint}?{urlencode(params)}"
current_app.logger.debug(f"Auth: Complete authorization URL: {auth_url}")
current_app.logger.info(f"Auth: Authentication initiated for {me_url}")
return auth_url
```
**Change 3:** Update `handle_callback()` to use discovered authorization_endpoint (replace lines 321-474):
> **Important:** Per IndieAuth spec, authentication-only flows (identity verification without access tokens) POST to the **authorization_endpoint**, NOT the token_endpoint. The `grant_type` parameter is NOT included for authentication-only flows.
```python
def handle_callback(code: str, state: str, iss: Optional[str] = None) -> Optional[str]:
"""
Handle IndieAuth callback with endpoint discovery.
Discovers authorization_endpoint from ADMIN_ME profile and exchanges
authorization code for identity verification.
Per IndieAuth spec: Authentication-only flows POST to the authorization
endpoint (not token endpoint) and do not include grant_type.
Args:
code: Authorization code from authorization server
state: CSRF state token
iss: Issuer identifier (optional, for security validation)
Returns:
Session token if successful, None otherwise
Raises:
InvalidStateError: State token validation failed
UnauthorizedError: User not authorized as admin
IndieLoginError: Code exchange failed
"""
current_app.logger.debug(f"Auth: Verifying state token: {_redact_token(state, 8)}")
# Verify state token (CSRF protection)
if not _verify_state_token(state):
current_app.logger.warning(
"Auth: Invalid state token received (possible CSRF or expired token)"
)
raise InvalidStateError("Invalid or expired state token")
current_app.logger.debug("Auth: State token valid")
# Discover authorization endpoint from ADMIN_ME profile
admin_me = current_app.config.get("ADMIN_ME")
if not admin_me:
current_app.logger.error("Auth: ADMIN_ME not configured")
raise IndieLoginError("ADMIN_ME not configured")
try:
endpoints = discover_endpoints(admin_me)
except DiscoveryError as e:
current_app.logger.error(f"Auth: Endpoint discovery failed: {e}")
raise IndieLoginError(f"Failed to discover endpoints: {e}")
# Use authorization_endpoint for authentication-only flow (identity verification)
# Per IndieAuth spec: auth-only flows POST to authorization_endpoint, not token_endpoint
auth_endpoint = endpoints.get('authorization_endpoint')
if not auth_endpoint:
raise IndieLoginError(
f"No authorization_endpoint found at {admin_me}. "
"Ensure your profile has IndieAuth endpoints configured."
)
current_app.logger.debug(f"Auth: Using authorization_endpoint: {auth_endpoint}")
# Verify issuer if provided (security check - optional)
if iss:
current_app.logger.debug(f"Auth: Issuer provided: {iss}")
# Prepare code verification request
# Note: grant_type is NOT included for authentication-only flows per IndieAuth spec
token_exchange_data = {
"code": code,
"client_id": current_app.config["SITE_URL"],
"redirect_uri": f"{current_app.config['SITE_URL']}auth/callback",
}
# Log the request
_log_http_request(
method="POST",
url=auth_endpoint,
data=token_exchange_data,
)
current_app.logger.debug(
"Auth: Sending code verification request to authorization endpoint:\n"
" Method: POST\n"
" URL: %s\n"
" Data: code=%s, client_id=%s, redirect_uri=%s",
auth_endpoint,
_redact_token(code),
token_exchange_data["client_id"],
token_exchange_data["redirect_uri"],
)
# Exchange code for identity at authorization endpoint (authentication-only flow)
try:
response = httpx.post(
auth_endpoint,
data=token_exchange_data,
timeout=10.0,
)
current_app.logger.debug(
"Auth: Received code verification response:\n"
" Status: %d\n"
" Headers: %s\n"
" Body: %s",
response.status_code,
{k: v for k, v in dict(response.headers).items()
if k.lower() not in ["set-cookie", "authorization"]},
_redact_token(response.text) if response.text else "(empty)",
)
_log_http_response(
status_code=response.status_code,
headers=dict(response.headers),
body=response.text,
)
response.raise_for_status()
except httpx.RequestError as e:
current_app.logger.error(f"Auth: Authorization endpoint request failed: {e}")
raise IndieLoginError(f"Failed to verify code: {e}")
except httpx.HTTPStatusError as e:
current_app.logger.error(
f"Auth: Authorization endpoint returned error: {e.response.status_code} - {e.response.text}"
)
raise IndieLoginError(
f"Authorization endpoint returned error: {e.response.status_code}"
)
# Parse response
try:
data = response.json()
except Exception as e:
current_app.logger.error(f"Auth: Failed to parse authorization endpoint response: {e}")
raise IndieLoginError("Invalid JSON response from authorization endpoint")
me = data.get("me")
if not me:
current_app.logger.error("Auth: No identity returned from authorization endpoint")
raise IndieLoginError("No identity returned from authorization endpoint")
current_app.logger.debug(f"Auth: Received identity: {me}")
# Verify this is the admin user
current_app.logger.info(f"Auth: Verifying admin authorization for me={me}")
# Normalize URLs for comparison (handles trailing slashes and case differences)
# This is correct per IndieAuth spec - the returned 'me' is the canonical form
if normalize_url(me) != normalize_url(admin_me):
current_app.logger.warning(
f"Auth: Unauthorized login attempt: {me} (expected {admin_me})"
)
raise UnauthorizedError(f"User {me} is not authorized")
current_app.logger.debug("Auth: Admin verification passed")
# Create session
session_token = create_session(me)
# Trigger author profile discovery (v1.2.0 Phase 2)
# Per Q14: Never block login, always allow fallback
try:
from starpunk.author_discovery import get_author_profile
author_profile = get_author_profile(me, refresh=True)
current_app.logger.info(f"Author profile refreshed for {me}")
except Exception as e:
current_app.logger.warning(f"Author discovery failed: {e}")
# Continue login anyway - never block per Q14
return session_token
```
---
### Step 3: Update auth_external.py - Relax Endpoint Validation
The existing `_fetch_and_parse()` function requires `token_endpoint` to be present. We need to relax this since some profiles may only have `authorization_endpoint` (for authentication-only flows).
In `/home/phil/Projects/starpunk/starpunk/auth_external.py`, update the validation in `_fetch_and_parse()` (around lines 302-307):
**Change:** Make token_endpoint not strictly required (allow authentication-only profiles):
```python
# Validate we found at least one endpoint
# - authorization_endpoint: Required for authentication-only flows (admin login)
# - token_endpoint: Required for Micropub token verification
# Having at least one allows the appropriate flow to work
if 'token_endpoint' not in endpoints and 'authorization_endpoint' not in endpoints:
raise DiscoveryError(
f"No IndieAuth endpoints found at {profile_url}. "
"Ensure your profile has authorization_endpoint or token_endpoint configured."
)
```
---
### Step 4: Update routes/auth.py - Handle DiscoveryError
In `/home/phil/Projects/starpunk/starpunk/routes/auth.py`:
**Change 1:** Add import for DiscoveryError (update lines 20-29):
```python
from starpunk.auth import (
IndieLoginError,
InvalidStateError,
UnauthorizedError,
destroy_session,
handle_callback,
initiate_login,
require_auth,
verify_session,
)
from starpunk.auth_external import DiscoveryError
```
**Change 2:** Handle DiscoveryError in login_initiate() (update lines 79-85):
> **Note:** The user-facing error message is kept simple. Technical details are logged but not shown to users.
```python
try:
# Initiate IndieAuth flow
auth_url = initiate_login(me_url)
return redirect(auth_url)
except ValueError as e:
flash(str(e), "error")
return redirect(url_for("auth.login_form"))
except DiscoveryError as e:
current_app.logger.error(f"Endpoint discovery failed for {me_url}: {e}")
flash("Unable to verify your profile URL. Please check that it's correct and try again.", "error")
return redirect(url_for("auth.login_form"))
```
---
## File Summary
| File | Change Type | Description |
|------|-------------|-------------|
| `starpunk/config.py` | Edit | Remove INDIELOGIN_URL, add deprecation warning |
| `starpunk/auth.py` | Edit | Use endpoint discovery instead of hardcoded URL |
| `starpunk/auth_external.py` | Edit | Relax endpoint validation (allow auth-only flow) |
| `starpunk/routes/auth.py` | Edit | Handle DiscoveryError exception |
---
## Testing Requirements
### Manual Testing
1. **Login Flow Test**
- Navigate to `/auth/login`
- Enter ADMIN_ME URL
- Verify redirect goes to discovered authorization_endpoint (not hardcoded indielogin.com)
- Complete login and verify session is created
2. **Endpoint Discovery Test**
- Test with profile that declares custom endpoints
- Verify discovered endpoints are used, not defaults
### Existing Test Updates
**Update test fixture in `tests/test_auth.py`:**
Remove `INDIELOGIN_URL` from the app fixture (line 51):
```python
@pytest.fixture
def app(tmp_path):
"""Create Flask app for testing"""
from starpunk import create_app
test_data_dir = tmp_path / "data"
test_data_dir.mkdir(parents=True, exist_ok=True)
app = create_app(
{
"TESTING": True,
"SITE_URL": "http://localhost:5000/",
"ADMIN_ME": "https://example.com",
"SESSION_SECRET": secrets.token_hex(32),
"SESSION_LIFETIME": 30,
# REMOVED: "INDIELOGIN_URL": "https://indielogin.com",
"DATA_PATH": test_data_dir,
"NOTES_PATH": test_data_dir / "notes",
"DATABASE_PATH": test_data_dir / "starpunk.db",
}
)
return app
```
**Update existing tests that use `httpx.post` mock:**
Tests in `TestInitiateLogin` and `TestHandleCallback` need to mock `discover_endpoints()` in addition to `httpx.post`. Example pattern:
```python
@patch("starpunk.auth.discover_endpoints")
@patch("starpunk.auth.httpx.post")
def test_handle_callback_success(self, mock_post, mock_discover, app, db, client):
"""Test successful callback handling"""
# Mock endpoint discovery
mock_discover.return_value = {
'authorization_endpoint': 'https://auth.example.com/authorize',
'token_endpoint': 'https://auth.example.com/token'
}
# Rest of test remains the same...
```
**Update `TestInitiateLogin.test_initiate_login_success`:**
The assertion checking for `indielogin.com` needs to change to check for the mocked endpoint:
```python
@patch("starpunk.auth.discover_endpoints")
def test_initiate_login_success(self, mock_discover, app, db):
"""Test successful login initiation"""
mock_discover.return_value = {
'authorization_endpoint': 'https://auth.example.com/authorize',
'token_endpoint': 'https://auth.example.com/token'
}
with app.app_context():
me_url = "https://example.com"
auth_url = initiate_login(me_url)
# Changed: Check for discovered endpoint instead of indielogin.com
assert "auth.example.com/authorize" in auth_url
assert "me=https%3A%2F%2Fexample.com" in auth_url
# ... rest of assertions
```
### New Automated Tests to Add
```python
# tests/test_auth_endpoint_discovery.py
def test_initiate_login_uses_endpoint_discovery(client, mocker):
"""Verify login uses discovered endpoint, not hardcoded"""
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
mock_discover.return_value = {
'authorization_endpoint': 'https://custom-auth.example.com/authorize',
'token_endpoint': 'https://custom-auth.example.com/token'
}
response = client.post('/auth/login', data={'me': 'https://example.com'})
assert response.status_code == 302
assert 'custom-auth.example.com' in response.headers['Location']
def test_callback_uses_discovered_authorization_endpoint(client, mocker):
"""Verify callback uses discovered authorization endpoint (not token endpoint)"""
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
mock_discover.return_value = {
'authorization_endpoint': 'https://custom-auth.example.com/authorize',
'token_endpoint': 'https://custom-auth.example.com/token'
}
mock_post = mocker.patch('starpunk.auth.httpx.post')
# Setup state token and mock httpx response
# Verify code exchange POSTs to authorization_endpoint, not token_endpoint
pass
def test_discovery_error_shows_user_friendly_message(client, mocker):
"""Verify discovery failures show helpful error"""
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
mock_discover.side_effect = DiscoveryError("No endpoints found")
response = client.post('/auth/login', data={'me': 'https://example.com'})
assert response.status_code == 302
# Should redirect back to login form with flash message
def test_url_normalization_handles_trailing_slash(app, mocker):
"""Verify URL normalization allows trailing slash differences"""
# ADMIN_ME without trailing slash, auth server returns with trailing slash
# Should still authenticate successfully
pass
def test_url_normalization_handles_case_differences(app, mocker):
"""Verify URL normalization is case-insensitive"""
# ADMIN_ME: https://Example.com, auth server returns: https://example.com
# Should still authenticate successfully
pass
```
---
## Rollback Plan
If issues occur after deployment:
1. **Code:** Revert to previous commit
2. **Config:** Re-add INDIELOGIN_URL to .env if needed
---
## Post-Deployment Verification
1. Verify login works with the user's actual profile URL
2. Check logs for "Discovered authorization_endpoint" message
3. Test logout and re-login cycle
---
## Architect Q&A (2025-12-17)
Developer questions answered by the architect prior to implementation:
### Q1: Import Placement
**Q:** Should `normalize_url` import be inside the function or at top level?
**A:** Move to top level with other imports for consistency. The design has been updated.
### Q2: URL Normalization Behavior Change
**Q:** Is the URL normalization change intentional?
**A:** Yes, this is an intentional bugfix. The current exact-match behavior is incorrect per IndieAuth spec. URLs differing only in trailing slashes or case should be considered equivalent for identity purposes. The `normalize_url()` function already exists in `auth_external.py` and is used by `verify_external_token()`.
### Q3: Which Endpoint for Authentication Flow?
**Q:** Should we use token_endpoint or authorization_endpoint?
**A:** Use **authorization_endpoint** for authentication-only flows. Per IndieAuth spec: "the client makes a POST request to the authorization endpoint to verify the authorization code and retrieve the final user profile URL." The design has been corrected.
### Q4: Endpoint Validation Relaxation
**Q:** Is relaxed endpoint validation acceptable?
**A:** Yes. Login requires `authorization_endpoint`, Micropub requires `token_endpoint`. Requiring at least one is correct. If only auth endpoint exists, login works but Micropub fails gracefully (401).
### Q5: Test Update Strategy
**Q:** Remove INDIELOGIN_URL and/or mock discover_endpoints()?
**A:** Both. Remove `INDIELOGIN_URL` from fixtures, add `discover_endpoints()` mocking to existing tests. Detailed guidance added to Testing Requirements section.
### Q6: grant_type Parameter
**Q:** Should we include grant_type in the code exchange?
**A:** No. Authentication-only flows do not include `grant_type`. This parameter is only required when POSTing to the token_endpoint for access tokens. The design has been corrected.
### Q7: Error Message Verbosity
**Q:** Should we simplify the user-facing error message?
**A:** Yes. User-facing message should be simple: "Unable to verify your profile URL. Please check that it's correct and try again." Technical details are logged at ERROR level. The design has been updated.
---
## References
- W3C IndieAuth Specification: https://www.w3.org/TR/indieauth/
- IndieAuth Endpoint Discovery: https://www.w3.org/TR/indieauth/#discovery-by-clients

Some files were not shown because too many files have changed in this diff Show More