StarPunk/docs/projectplan/v1.5.0/RELEASE.md

StarPunk v1.5.0 Release Plan

Status: Approved Codename: "Trigger" Focus: Stability, Test Coverage, and Technical Debt Reduction Last Updated: 2025-12-17

Overview

v1.5.0 is a quality-focused release that addresses failing tests, increases test coverage to 90%, implements critical fixes from the v1.4.x review cycle, and resolves technical debt. No new user-facing features are planned.

Goals

1. Fix Failing Tests - Resolve all 19 currently failing tests
2. 90% Test Coverage - Increase overall test coverage to 90% minimum
3. Technical Debt Reduction - Address 6 specific backlog items
4. Code Quality - Better error handling, atomicity, and performance

Phased Implementation Plan

Phase 0: Test Fixes (Critical Path)

Priority: Must complete first - unblocks all other phases

Scope

Address flaky and broken tests per ADR-012 (Flaky Test Removal):

REMOVE (5 tests) - Architecturally broken multiprocessing tests:

Test	Reason
test_concurrent_workers_barrier_sync	Cannot pickle Barrier objects
test_sequential_worker_startup	Missing Flask app context
test_worker_late_arrival	Missing Flask app context
test_single_worker_performance	Cannot pickle local functions
test_concurrent_workers_performance	Cannot pickle local functions

FIX (4 tests) - Valuable tests needing adjustments:

Test	Fix Required
test_debug_level_for_early_retries	Configure logger level in test
test_new_connection_per_retry	Adjust assertion count
Feed XML tests	Change <?xml version="1.0" to <?xml version= (don't assert quote style)
test_feed_json_endpoint	Don't require charset in Content-Type

RENAME (1 test):

Test	New Name
test_feed_route_streaming	test_feed_route_caching (test is correct, name misleading)

Approach

1. Remove the 5 broken multiprocessing tests (they cannot work due to Python limitations)
2. Fix the brittle feed assertion tests (check semantics, not quote style)
3. Fix the 4 migration tests that have value but need mock/assertion adjustments
4. Rename misleading test
5. Document changes in implementation report

Acceptance Criteria

• All remaining tests pass consistently (run 3x to verify no flakiness)
• 5 broken tests removed with justification in ADR-012
• No new test skips added
• Test count reduced from 879 to 874

Dependencies

None - this is the first phase

---

Phase 1: Timestamp-Based Slugs

Priority: High - Addresses user-reported issue

Scope

Implement ADR-062 to change default slug format from content-based to timestamp-based.

Implementation

1. Update starpunk/slug_utils.py:

   • Change generate_slug() to use timestamp format YYYYMMDDHHMMSS
   • Update collision handling to use sequential suffix (-1, -2, etc.)
   • Preserve custom slug functionality

2. Update starpunk/notes.py:

   • Remove content parameter from slug generation calls

3. Update tests:

   • Modify expected slug formats in test assertions
   • Add tests for new timestamp format
   • Add tests for sequential collision handling

Acceptance Criteria

• Default slugs use YYYYMMDDHHMMSS format
• Collision handling uses -1, -2 suffix
• Custom slugs via mp-slug work unchanged
• Custom slugs via web UI work unchanged
• Existing notes unaffected
• ADR-062 referenced in code comments

Dependencies

• Phase 0 complete (all tests passing)

---

Phase 2: Debug File Management

Priority: Medium - Security and operations concern

Scope

Implement cleanup mechanism for failed upload debug files and add configuration controls.

Implementation

1. Add configuration options:

   DEBUG_SAVE_FAILED_UPLOADS = False  # Default: disabled in production
   DEBUG_FILE_MAX_AGE_DAYS = 7        # Auto-delete threshold
   DEBUG_FILE_MAX_SIZE_MB = 100       # Maximum debug folder size
   

2. Implement cleanup logic in starpunk/media.py:

   • Check config before saving debug files
   • Implement cleanup_old_debug_files() function
   • Add size limit check before saving

3. Add startup cleanup:

   • Run cleanup on application startup
   • Log cleanup actions

4. Sanitize filenames:

   • Sanitize filename before debug path construction
   • Pattern: "".join(c for c in filename if c.isalnum() or c in "._-")[:50]

Acceptance Criteria

• Debug files disabled by default
• Files older than 7 days auto-deleted when enabled
• Folder size limited to 100MB
• Filenames sanitized (no path traversal)
• Cleanup runs on startup
• Tests cover all scenarios

Dependencies

• Phase 0 complete

---

Phase 3: N+1 Query Fix (Feed Generation)

Priority: Medium - Performance improvement

Scope

Fix N+1 query pattern in _get_cached_notes() only. Other N+1 patterns are deferred (documented in BACKLOG.md).

Implementation

1. Create batch loading functions in starpunk/media.py:

   def get_media_for_notes(note_ids: List[int]) -> Dict[int, List[dict]]:
       """Batch load media for multiple notes in single query."""
   

2. Create batch loading functions in starpunk/tags.py:

   def get_tags_for_notes(note_ids: List[int]) -> Dict[int, List[dict]]:
       """Batch load tags for multiple notes in single query."""
   

3. Update _get_cached_notes() in starpunk/routes/public.py:

   • Use batch loading instead of per-note queries
   • Maintain same output format

Acceptance Criteria

• Feed generation uses batch queries
• Query count reduced from O(n) to O(1) for media/tags
• No change to API behavior
• Performance improvement verified in tests
• Other N+1 locations documented in BACKLOG.md (not fixed)

Dependencies

• Phase 0 complete

---

Phase 4: Atomic Variant Generation

Priority: Medium - Data integrity

Scope

Make variant file generation atomic with database commits to prevent orphaned files.

Implementation

1. Modify generate_all_variants() in starpunk/media.py:

   • Write variants to temporary directory first
   • Perform database inserts in transaction
   • Move files to final location after successful commit
   • Clean up temp files on any failure

2. Add startup recovery:

   • Detect orphaned variant files on startup
   • Log warnings for orphans found
   • Optionally clean up orphans

Flow

1. Generate variants to temp directory (data/media/.tmp/)
2. BEGIN TRANSACTION
3. INSERT media record
4. INSERT variant records
5. COMMIT
6. Move files from temp to final location
7. On failure: ROLLBACK, delete temp files

Acceptance Criteria

• No orphaned files on database failures
• No orphaned DB records on file failures
• Atomic operation for all media saves
• Startup recovery detects orphans
• Tests simulate failure scenarios

Dependencies

• Phase 0 complete

---

Phase 5: Test Coverage Expansion

Priority: Medium - Quality assurance

Scope

Increase overall test coverage to 90% minimum.

Approach

1. Run coverage report: uv run pytest --cov=starpunk --cov-report=html
2. Identify modules below 90% coverage
3. Prioritize based on risk and complexity
4. Write tests for uncovered paths

Known Coverage Gaps (to verify)

• MPO format handling (untested)
• Edge cases in error paths
• Configuration validation paths
• Startup/shutdown hooks

Specific Test Additions

1. MPO Format Tests (tests/test_media_upload.py):

   • test_mpo_detection_and_conversion()
   • test_mpo_corrupted_file()
   • test_mpo_single_frame()

2. Debug File Tests (new test file or extend test_media_upload.py):

   • test_debug_file_disabled_by_default()
   • test_debug_file_cleanup_old_files()
   • test_debug_file_size_limit()
   • test_debug_filename_sanitization()

3. Batch Loading Tests:

   • test_get_media_for_notes_batch()
   • test_get_tags_for_notes_batch()
   • test_batch_with_empty_list()

Acceptance Criteria

• Overall coverage >= 90%
• No module below 85% coverage
• All new code in v1.5.0 has 100% coverage
• MPO handling fully tested

Dependencies

• Phases 1-4 complete (tests cover new functionality)

---

Out of Scope

Items explicitly excluded from v1.5.0:

Item	Reason
Rate limiting	Handled by reverse proxy (Caddy/Nginx)
Schema changes	Not needed for v1.5.0 fixes
New user features	Quality-focused release
N+1 fixes in admin/search	Low traffic, deferred to BACKLOG
UI changes	No frontend work planned

---

Recommendation: Single Release vs. Multiple Patches

Recommendation: Single v1.5.0 Release

Rationale

1. Phase Dependencies: Most phases depend on Phase 0 (test fixes). Splitting would create artificial release boundaries.

2. Cognitive Overhead: Multiple patch releases (1.5.1, 1.5.2, etc.) require:

   • Multiple changelog entries
   • Multiple version bumps
   • Multiple release notes
   • More git tags/branches

3. Test Coverage Integration: Test coverage work (Phase 5) tests functionality from Phases 1-4. Separating them creates incomplete test coverage.

4. User Experience: Users prefer fewer, more significant updates over many small patches.

5. Scope Alignment: All v1.5.0 work is internally focused (no external API changes). Users see one "quality improvement" release.

Exception

If Phase 0 (test fixes) reveals critical bugs affecting production, those fixes should be:

• Backported to a v1.4.3 patch release on the current branch
• Then merged forward to v1.5.0

Alternative Considered

Splitting into:

• v1.5.0: Phase 0 (test fixes) + Phase 1 (slugs)
• v1.5.1: Phase 2-4 (technical debt)
• v1.5.2: Phase 5 (test coverage)

Rejected because test coverage work must test the new functionality, making separation counterproductive.

---

Success Criteria

#	Criterion	Verification
1	All tests pass	uv run pytest shows 0 failures
2	Coverage >= 90%	uv run pytest --cov=starpunk
3	MPO tested	MPO tests in test suite
4	Debug cleanup works	Manual verification + tests
5	N+1 fixed in feed	Performance tests show improvement
6	Variants atomic	Failure simulation tests pass
7	Slugs timestamp-based	New notes use YYYYMMDDHHMMSS format
8	No regressions	Full test suite passes
9	ADRs documented	ADR-062 in /docs/decisions/

---

Related Documentation

• ADR-062: Timestamp-Based Slug Format (supersedes ADR-007)
• ADR-007: Slug Generation Algorithm (superseded)
• BACKLOG.md: Deferred N+1 query locations documented
• v1.4.2 Architect Review: Source of many v1.5.0 items

---

Implementation Timeline

Phase	Estimated Effort	Dependencies
Phase 0: Test Fixes	2-4 hours	None
Phase 1: Timestamp Slugs	2-3 hours	Phase 0
Phase 2: Debug Files	3-4 hours	Phase 0
Phase 3: N+1 Fix	3-4 hours	Phase 0
Phase 4: Atomic Variants	4-6 hours	Phase 0
Phase 5: Coverage	4-8 hours	Phases 1-4

Total Estimated: 18-29 hours

---

Post-Release

After v1.5.0 ships, update BACKLOG.md to move completed items to "Recently Completed" section:

• MPO Format Test Coverage
• Debug File Storage Cleanup
• Filename Sanitization
• N+1 Query Fix (Feed Generation - partial)
• Atomic Variant Generation
• Default Slug Change