StarPunk/docs/projectplan/v1.5.0/RELEASE.md

# StarPunk v1.5.0 Release Plan

**Status**: Approved
**Codename**: "Trigger"
**Focus**: Stability, Test Coverage, and Technical Debt Reduction
**Last Updated**: 2025-12-17

## Overview

v1.5.0 is a quality-focused release that addresses failing tests, increases test coverage to 90%, implements critical fixes from the v1.4.x review cycle, and resolves technical debt. No new user-facing features are planned.

## Goals

1. **Fix Failing Tests** - Resolve all 19 currently failing tests
2. **90% Test Coverage** - Increase overall test coverage to 90% minimum
3. **Technical Debt Reduction** - Address 6 specific backlog items
4. **Code Quality** - Better error handling, atomicity, and performance

## Phased Implementation Plan

### Phase 0: Test Fixes (Critical Path)
**Priority**: Must complete first - unblocks all other phases

#### Scope
Address flaky and broken tests per ADR-012 (Flaky Test Removal):

**REMOVE (5 tests)** - Architecturally broken multiprocessing tests:
| Test | Reason |
|------|--------|
| `test_concurrent_workers_barrier_sync` | Cannot pickle Barrier objects |
| `test_sequential_worker_startup` | Missing Flask app context |
| `test_worker_late_arrival` | Missing Flask app context |
| `test_single_worker_performance` | Cannot pickle local functions |
| `test_concurrent_workers_performance` | Cannot pickle local functions |

**FIX (4 tests)** - Valuable tests needing adjustments:
| Test | Fix Required |
|------|--------------|
| `test_debug_level_for_early_retries` | Configure logger level in test |
| `test_new_connection_per_retry` | Adjust assertion count |
| Feed XML tests | Change `<?xml version="1.0"` to `<?xml version=` (don't assert quote style) |
| `test_feed_json_endpoint` | Don't require charset in Content-Type |

**RENAME (1 test)**:
| Test | New Name |
|------|----------|
| `test_feed_route_streaming` | `test_feed_route_caching` (test is correct, name misleading) |

#### Approach
1. Remove the 5 broken multiprocessing tests (they cannot work due to Python limitations)
2. Fix the brittle feed assertion tests (check semantics, not quote style)
3. Fix the 4 migration tests that have value but need mock/assertion adjustments
4. Rename misleading test
5. Document changes in implementation report

#### Acceptance Criteria
- [ ] All remaining tests pass consistently (run 3x to verify no flakiness)
- [ ] 5 broken tests removed with justification in ADR-012
- [ ] No new test skips added
- [ ] Test count reduced from 879 to 874

#### Dependencies
None - this is the first phase

---

### Phase 1: Timestamp-Based Slugs
**Priority**: High - Addresses user-reported issue

#### Scope
Implement ADR-062 to change default slug format from content-based to timestamp-based.

#### Implementation
1. Update `starpunk/slug_utils.py`:
   - Change `generate_slug()` to use timestamp format `YYYYMMDDHHMMSS`
   - Update collision handling to use sequential suffix (`-1`, `-2`, etc.)
   - Preserve custom slug functionality

2. Update `starpunk/notes.py`:
   - Remove content parameter from slug generation calls

3. Update tests:
   - Modify expected slug formats in test assertions
   - Add tests for new timestamp format
   - Add tests for sequential collision handling

#### Acceptance Criteria
- [ ] Default slugs use `YYYYMMDDHHMMSS` format
- [ ] Collision handling uses `-1`, `-2` suffix
- [ ] Custom slugs via `mp-slug` work unchanged
- [ ] Custom slugs via web UI work unchanged
- [ ] Existing notes unaffected
- [ ] ADR-062 referenced in code comments

#### Dependencies
- Phase 0 complete (all tests passing)

---

### Phase 2: Debug File Management
**Priority**: Medium - Security and operations concern

#### Scope
Implement cleanup mechanism for failed upload debug files and add configuration controls.

#### Implementation
1. Add configuration options:
   ```python
   DEBUG_SAVE_FAILED_UPLOADS = False  # Default: disabled in production
   DEBUG_FILE_MAX_AGE_DAYS = 7        # Auto-delete threshold
   DEBUG_FILE_MAX_SIZE_MB = 100       # Maximum debug folder size
   ```

2. Implement cleanup logic in `starpunk/media.py`:
   - Check config before saving debug files
   - Implement `cleanup_old_debug_files()` function
   - Add size limit check before saving

3. Add startup cleanup:
   - Run cleanup on application startup
   - Log cleanup actions

4. Sanitize filenames:
   - Sanitize filename before debug path construction
   - Pattern: `"".join(c for c in filename if c.isalnum() or c in "._-")[:50]`

#### Acceptance Criteria
- [ ] Debug files disabled by default
- [ ] Files older than 7 days auto-deleted when enabled
- [ ] Folder size limited to 100MB
- [ ] Filenames sanitized (no path traversal)
- [ ] Cleanup runs on startup
- [ ] Tests cover all scenarios

#### Dependencies
- Phase 0 complete

---

### Phase 3: N+1 Query Fix (Feed Generation)
**Priority**: Medium - Performance improvement

#### Scope
Fix N+1 query pattern in `_get_cached_notes()` only. Other N+1 patterns are deferred (documented in BACKLOG.md).

#### Implementation
1. Create batch loading functions in `starpunk/media.py`:
   ```python
   def get_media_for_notes(note_ids: List[int]) -> Dict[int, List[dict]]:
       """Batch load media for multiple notes in single query."""
   ```

2. Create batch loading functions in `starpunk/tags.py`:
   ```python
   def get_tags_for_notes(note_ids: List[int]) -> Dict[int, List[dict]]:
       """Batch load tags for multiple notes in single query."""
   ```

3. Update `_get_cached_notes()` in `starpunk/routes/public.py`:
   - Use batch loading instead of per-note queries
   - Maintain same output format

#### Acceptance Criteria
- [ ] Feed generation uses batch queries
- [ ] Query count reduced from O(n) to O(1) for media/tags
- [ ] No change to API behavior
- [ ] Performance improvement verified in tests
- [ ] Other N+1 locations documented in BACKLOG.md (not fixed)

#### Dependencies
- Phase 0 complete

---

### Phase 4: Atomic Variant Generation
**Priority**: Medium - Data integrity

#### Scope
Make variant file generation atomic with database commits to prevent orphaned files.

#### Implementation
1. Modify `generate_all_variants()` in `starpunk/media.py`:
   - Write variants to temporary directory first
   - Perform database inserts in transaction
   - Move files to final location after successful commit
   - Clean up temp files on any failure

2. Add startup recovery:
   - Detect orphaned variant files on startup
   - Log warnings for orphans found
   - Optionally clean up orphans

#### Flow
```
1. Generate variants to temp directory (data/media/.tmp/)
2. BEGIN TRANSACTION
3. INSERT media record
4. INSERT variant records
5. COMMIT
6. Move files from temp to final location
7. On failure: ROLLBACK, delete temp files
```

#### Acceptance Criteria
- [ ] No orphaned files on database failures
- [ ] No orphaned DB records on file failures
- [ ] Atomic operation for all media saves
- [ ] Startup recovery detects orphans
- [ ] Tests simulate failure scenarios

#### Dependencies
- Phase 0 complete

---

### Phase 5: Test Coverage Expansion
**Priority**: Medium - Quality assurance

#### Scope
Increase overall test coverage to 90% minimum.

#### Approach
1. Run coverage report: `uv run pytest --cov=starpunk --cov-report=html`
2. Identify modules below 90% coverage
3. Prioritize based on risk and complexity
4. Write tests for uncovered paths

#### Known Coverage Gaps (to verify)
- MPO format handling (untested)
- Edge cases in error paths
- Configuration validation paths
- Startup/shutdown hooks

#### Specific Test Additions
1. **MPO Format Tests** (`tests/test_media_upload.py`):
   - `test_mpo_detection_and_conversion()`
   - `test_mpo_corrupted_file()`
   - `test_mpo_single_frame()`

2. **Debug File Tests** (new test file or extend `test_media_upload.py`):
   - `test_debug_file_disabled_by_default()`
   - `test_debug_file_cleanup_old_files()`
   - `test_debug_file_size_limit()`
   - `test_debug_filename_sanitization()`

3. **Batch Loading Tests**:
   - `test_get_media_for_notes_batch()`
   - `test_get_tags_for_notes_batch()`
   - `test_batch_with_empty_list()`

#### Acceptance Criteria
- [ ] Overall coverage >= 90%
- [ ] No module below 85% coverage
- [ ] All new code in v1.5.0 has 100% coverage
- [ ] MPO handling fully tested

#### Dependencies
- Phases 1-4 complete (tests cover new functionality)

---

## Out of Scope

Items explicitly excluded from v1.5.0:

| Item | Reason |
|------|--------|
| Rate limiting | Handled by reverse proxy (Caddy/Nginx) |
| Schema changes | Not needed for v1.5.0 fixes |
| New user features | Quality-focused release |
| N+1 fixes in admin/search | Low traffic, deferred to BACKLOG |
| UI changes | No frontend work planned |

---

## Recommendation: Single Release vs. Multiple Patches

**Recommendation: Single v1.5.0 Release**

### Rationale

1. **Phase Dependencies**: Most phases depend on Phase 0 (test fixes). Splitting would create artificial release boundaries.

2. **Cognitive Overhead**: Multiple patch releases (1.5.1, 1.5.2, etc.) require:
   - Multiple changelog entries
   - Multiple version bumps
   - Multiple release notes
   - More git tags/branches

3. **Test Coverage Integration**: Test coverage work (Phase 5) tests functionality from Phases 1-4. Separating them creates incomplete test coverage.

4. **User Experience**: Users prefer fewer, more significant updates over many small patches.

5. **Scope Alignment**: All v1.5.0 work is internally focused (no external API changes). Users see one "quality improvement" release.

### Exception

If Phase 0 (test fixes) reveals critical bugs affecting production, those fixes should be:
- Backported to a v1.4.3 patch release on the current branch
- Then merged forward to v1.5.0

### Alternative Considered

Splitting into:
- v1.5.0: Phase 0 (test fixes) + Phase 1 (slugs)
- v1.5.1: Phase 2-4 (technical debt)
- v1.5.2: Phase 5 (test coverage)

**Rejected** because test coverage work must test the new functionality, making separation counterproductive.

---

## Success Criteria

| # | Criterion | Verification |
|---|-----------|--------------|
| 1 | All tests pass | `uv run pytest` shows 0 failures |
| 2 | Coverage >= 90% | `uv run pytest --cov=starpunk` |
| 3 | MPO tested | MPO tests in test suite |
| 4 | Debug cleanup works | Manual verification + tests |
| 5 | N+1 fixed in feed | Performance tests show improvement |
| 6 | Variants atomic | Failure simulation tests pass |
| 7 | Slugs timestamp-based | New notes use `YYYYMMDDHHMMSS` format |
| 8 | No regressions | Full test suite passes |
| 9 | ADRs documented | ADR-062 in `/docs/decisions/` |

---

## Related Documentation

- ADR-062: Timestamp-Based Slug Format (supersedes ADR-007)
- ADR-007: Slug Generation Algorithm (superseded)
- BACKLOG.md: Deferred N+1 query locations documented
- v1.4.2 Architect Review: Source of many v1.5.0 items

---

## Implementation Timeline

| Phase | Estimated Effort | Dependencies |
|-------|------------------|--------------|
| Phase 0: Test Fixes | 2-4 hours | None |
| Phase 1: Timestamp Slugs | 2-3 hours | Phase 0 |
| Phase 2: Debug Files | 3-4 hours | Phase 0 |
| Phase 3: N+1 Fix | 3-4 hours | Phase 0 |
| Phase 4: Atomic Variants | 4-6 hours | Phase 0 |
| Phase 5: Coverage | 4-8 hours | Phases 1-4 |

**Total Estimated**: 18-29 hours

---

## Post-Release

After v1.5.0 ships, update BACKLOG.md to move completed items to "Recently Completed" section:
- MPO Format Test Coverage
- Debug File Storage Cleanup
- Filename Sanitization
- N+1 Query Fix (Feed Generation - partial)
- Atomic Variant Generation
- Default Slug Change