docs: Add v1.1.0 architecture and validation documentation
- ADR-033: Database migration redesign - ADR-034: Full-text search with FTS5 - ADR-035: Custom slugs in Micropub - ADR-036: IndieAuth token verification method - ADR-039: Micropub URL construction fix - Implementation plan and decisions - Architecture specifications - Validation reports for implementation and search UI 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
345
docs/reports/v1.1.0-implementation-plan.md
Normal file
345
docs/reports/v1.1.0-implementation-plan.md
Normal file
@@ -0,0 +1,345 @@
|
||||
# StarPunk v1.1.0 Implementation Plan
|
||||
|
||||
## Executive Summary
|
||||
Version 1.1.0 focuses on three high-value features that enhance usability while maintaining our minimal philosophy. This release addresses critical technical debt (migration system), adds essential functionality (search), and improves user control (custom slugs).
|
||||
|
||||
## Release Overview
|
||||
- **Version**: 1.1.0
|
||||
- **Codename**: "Searchlight"
|
||||
- **Theme**: Enhanced discovery and control
|
||||
- **Estimated Effort**: 16-20 hours
|
||||
- **Priority**: High (addresses user feedback and technical debt)
|
||||
|
||||
## Critical Issue: RSS Feed Ordering
|
||||
|
||||
### Investigation Results
|
||||
**Finding**: The RSS feed is already correctly implemented in reverse chronological order (newest first).
|
||||
|
||||
**Evidence**:
|
||||
- `list_notes()` function defaults to `order_dir="DESC"` (descending = newest first)
|
||||
- SQL query uses `ORDER BY created_at DESC`
|
||||
- Feed generation receives notes in correct order
|
||||
|
||||
**Conclusion**: No bug exists. The user's perception may be incorrect, or they may be seeing cached content.
|
||||
|
||||
**Action Required**: None. Document the correct behavior and suggest cache clearing if users report chronological ordering.
|
||||
|
||||
## Feature Components
|
||||
|
||||
### 1. Database Migration System Redesign (CRITICAL)
|
||||
**Priority**: CRITICAL - Must be done first
|
||||
**ADR**: ADR-033
|
||||
**Effort**: 4-6 hours
|
||||
|
||||
#### Problem
|
||||
- Duplicate schema definitions in SCHEMA_SQL and migration files
|
||||
- Risk of schema drift between fresh installs and upgrades
|
||||
- Violates DRY principle
|
||||
|
||||
#### Solution Architecture
|
||||
```python
|
||||
# New structure
|
||||
INITIAL_SCHEMA_SQL = """-- v1.0.0 schema frozen in time"""
|
||||
migrations = [
|
||||
# Only changes after v1.0.0
|
||||
"001_add_search_index.sql",
|
||||
"002_add_custom_fields.sql"
|
||||
]
|
||||
|
||||
def initialize_database():
|
||||
if fresh_install():
|
||||
execute(INITIAL_SCHEMA_SQL)
|
||||
mark_as_v1_0_0()
|
||||
apply_pending_migrations()
|
||||
```
|
||||
|
||||
#### Implementation Tasks
|
||||
1. **Extract v1.0.0 Schema** (1 hour)
|
||||
- Document current production schema
|
||||
- Create INITIAL_SCHEMA_SQL constant
|
||||
- Verify against existing installations
|
||||
|
||||
2. **Refactor Migration System** (2 hours)
|
||||
- Modify `migrations.py` to use new approach
|
||||
- Separate fresh install from upgrade path
|
||||
- Update version tracking logic
|
||||
|
||||
3. **Migration Files Cleanup** (1 hour)
|
||||
- Remove redundant schema from existing migrations
|
||||
- Keep only incremental changes
|
||||
- Verify migration sequence
|
||||
|
||||
4. **Testing** (2 hours)
|
||||
- Test fresh installation path
|
||||
- Test upgrade from v1.0.0
|
||||
- Test upgrade from v1.0.1
|
||||
- Verify schema consistency
|
||||
|
||||
#### Risks
|
||||
- Breaking existing installations if not careful
|
||||
- Must maintain backward compatibility
|
||||
- Need thorough testing of both paths
|
||||
|
||||
### 2. Full-Text Search with FTS5
|
||||
**Priority**: HIGH - Most requested feature
|
||||
**ADR**: ADR-034
|
||||
**Effort**: 6-8 hours
|
||||
|
||||
#### Architecture Design
|
||||
```sql
|
||||
-- FTS virtual table
|
||||
CREATE VIRTUAL TABLE notes_fts USING fts5(
|
||||
slug UNINDEXED,
|
||||
title,
|
||||
content,
|
||||
tokenize='porter unicode61'
|
||||
);
|
||||
|
||||
-- Sync triggers
|
||||
CREATE TRIGGER notes_fts_sync_insert ...
|
||||
CREATE TRIGGER notes_fts_sync_update ...
|
||||
CREATE TRIGGER notes_fts_sync_delete ...
|
||||
```
|
||||
|
||||
#### API Design
|
||||
```python
|
||||
@app.route('/api/search')
|
||||
def search():
|
||||
query = request.args.get('q')
|
||||
results = db.execute("""
|
||||
SELECT slug, snippet(notes_fts, 2, '<mark>', '</mark>', '...', 30)
|
||||
FROM notes_fts
|
||||
WHERE notes_fts MATCH ?
|
||||
ORDER BY rank
|
||||
LIMIT 20
|
||||
""", [query])
|
||||
return jsonify(results)
|
||||
```
|
||||
|
||||
#### Implementation Tasks
|
||||
1. **Database Schema** (2 hours)
|
||||
- Create FTS5 migration
|
||||
- Implement sync triggers
|
||||
- Build initial index
|
||||
|
||||
2. **Search API** (2 hours)
|
||||
- Create `/api/search` endpoint
|
||||
- Implement query validation
|
||||
- Add result ranking and snippets
|
||||
- Handle pagination
|
||||
|
||||
3. **Search UI** (2 hours)
|
||||
- Add search box to navigation
|
||||
- Create results page template
|
||||
- Implement result highlighting
|
||||
- Add query syntax help
|
||||
|
||||
4. **Testing** (2 hours)
|
||||
- Test various query types
|
||||
- Benchmark performance
|
||||
- Verify trigger synchronization
|
||||
- Test Unicode content
|
||||
|
||||
#### Performance Targets
|
||||
- Index building: <1ms per note
|
||||
- Search latency: <10ms for 10,000 notes
|
||||
- Index size: ~30% of text size
|
||||
|
||||
### 3. Custom Slugs via Micropub
|
||||
**Priority**: MEDIUM - Standards compliance
|
||||
**ADR**: ADR-035
|
||||
**Effort**: 4-5 hours
|
||||
|
||||
#### Design
|
||||
```python
|
||||
def create_note_with_slug(content, custom_slug=None):
|
||||
if custom_slug:
|
||||
slug = sanitize_slug(custom_slug)
|
||||
if not validate_slug(slug):
|
||||
raise InvalidSlugError()
|
||||
if slug_exists(slug):
|
||||
slug = make_unique(slug)
|
||||
else:
|
||||
slug = generate_slug(content)
|
||||
|
||||
return create_note(content, slug=slug)
|
||||
```
|
||||
|
||||
#### Validation Rules
|
||||
- Pattern: `^[a-z0-9]+(?:-[a-z0-9]+)*(?:/[a-z0-9]+(?:-[a-z0-9]+)*)*$`
|
||||
- Max length: 200 characters
|
||||
- Reserved words: `api`, `admin`, `auth`, `feed`
|
||||
- Uniqueness with auto-increment on conflict
|
||||
|
||||
#### Implementation Tasks
|
||||
1. **Core Slug Logic** (2 hours)
|
||||
- Add slug parameter to note creation
|
||||
- Implement validation function
|
||||
- Add uniqueness checking
|
||||
- Handle conflicts
|
||||
|
||||
2. **Micropub Integration** (1 hour)
|
||||
- Extract `mp-slug` property
|
||||
- Pass to note creation
|
||||
- Handle validation errors
|
||||
- Return proper responses
|
||||
|
||||
3. **Testing** (1.5 hours)
|
||||
- Test valid/invalid slugs
|
||||
- Test conflict resolution
|
||||
- Test with real Micropub clients
|
||||
- Test backward compatibility
|
||||
|
||||
#### Security Considerations
|
||||
- Prevent path traversal (`../`)
|
||||
- Block reserved system routes
|
||||
- Enforce character whitelist
|
||||
- Normalize case (lowercase only)
|
||||
|
||||
## Implementation Sequence
|
||||
|
||||
### Phase 1: Critical Foundation (Week 1)
|
||||
1. **Migration System Redesign** (FIRST - blocks everything else)
|
||||
- Must be complete before adding new migrations
|
||||
- Ensures clean path for FTS5 tables
|
||||
- 4-6 hours
|
||||
|
||||
### Phase 2: Core Features (Week 1-2)
|
||||
2. **Full-Text Search**
|
||||
- Can begin after migration system ready
|
||||
- High user value, most requested
|
||||
- 6-8 hours
|
||||
|
||||
3. **Custom Slugs**
|
||||
- Can be done in parallel with search
|
||||
- Lower complexity, good for end of sprint
|
||||
- 4-5 hours
|
||||
|
||||
### Phase 3: Polish & Release (Week 2)
|
||||
4. **Integration Testing** (2 hours)
|
||||
5. **Documentation Updates** (1 hour)
|
||||
6. **Release Process** (1 hour)
|
||||
|
||||
## Risk Analysis
|
||||
|
||||
### High Risks
|
||||
1. **Migration System Breaking Changes**
|
||||
- Mitigation: Extensive testing, backup instructions
|
||||
- Contingency: Rollback procedure documented
|
||||
|
||||
2. **FTS5 Not Available**
|
||||
- Mitigation: Check SQLite version in setup
|
||||
- Contingency: Graceful degradation
|
||||
|
||||
### Medium Risks
|
||||
1. **Search Performance Issues**
|
||||
- Mitigation: Index size monitoring
|
||||
- Contingency: Add caching layer
|
||||
|
||||
2. **Slug Conflicts**
|
||||
- Mitigation: Auto-increment suffix
|
||||
- Contingency: Return clear error messages
|
||||
|
||||
### Low Risks
|
||||
1. **Increased Database Size**
|
||||
- Expected: ~30% increase from FTS
|
||||
- Acceptable for functionality gained
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Functional Requirements
|
||||
- [ ] Migration system uses single schema source
|
||||
- [ ] Search returns relevant results in <10ms
|
||||
- [ ] Custom slugs accepted via Micropub
|
||||
- [ ] All existing tests pass
|
||||
- [ ] No breaking changes to API
|
||||
|
||||
### Performance Requirements
|
||||
- [ ] Search latency <10ms for 1000 notes
|
||||
- [ ] Migration completes in <1 second
|
||||
- [ ] No degradation in note creation time
|
||||
|
||||
### Quality Requirements
|
||||
- [ ] 100% backward compatibility
|
||||
- [ ] No data loss during migration
|
||||
- [ ] Clear error messages for invalid slugs
|
||||
- [ ] Search results properly escaped (XSS prevention)
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
- Migration path logic
|
||||
- Slug validation functions
|
||||
- Search query parsing
|
||||
- FTS trigger behavior
|
||||
|
||||
### Integration Tests
|
||||
- Fresh install flow
|
||||
- Upgrade from v1.0.0/v1.0.1
|
||||
- Micropub with custom slugs
|
||||
- Search API responses
|
||||
|
||||
### Manual Testing
|
||||
- Search UI functionality
|
||||
- Various search queries
|
||||
- Micropub client compatibility
|
||||
- Performance benchmarks
|
||||
|
||||
## Documentation Updates
|
||||
|
||||
### User Documentation
|
||||
- Search syntax guide
|
||||
- Custom slug usage
|
||||
- Migration instructions
|
||||
|
||||
### Developer Documentation
|
||||
- New migration system explanation
|
||||
- FTS5 implementation details
|
||||
- Slug validation rules
|
||||
|
||||
### API Documentation
|
||||
- `/api/search` endpoint
|
||||
- `mp-slug` property handling
|
||||
- Error response formats
|
||||
|
||||
## Release Checklist
|
||||
|
||||
### Pre-Release
|
||||
- [ ] All tests passing
|
||||
- [ ] Documentation updated
|
||||
- [ ] CHANGELOG.md updated
|
||||
- [ ] Version bumped to 1.1.0
|
||||
- [ ] Migration tested on copy of production
|
||||
|
||||
### Release
|
||||
- [ ] Tag v1.1.0
|
||||
- [ ] Build container
|
||||
- [ ] Update release notes
|
||||
- [ ] Announce features
|
||||
|
||||
### Post-Release
|
||||
- [ ] Monitor for issues
|
||||
- [ ] Gather user feedback
|
||||
- [ ] Plan v1.2.0 based on feedback
|
||||
|
||||
## Estimated Timeline
|
||||
|
||||
### Week 1 (20-25 hours)
|
||||
- Day 1-2: Migration system redesign (6h)
|
||||
- Day 3-4: Full-text search implementation (8h)
|
||||
- Day 5: Custom slugs implementation (5h)
|
||||
|
||||
### Week 2 (5-8 hours)
|
||||
- Day 1: Integration testing (2h)
|
||||
- Day 2: Documentation and release prep (3h)
|
||||
- Day 3: Release and monitoring
|
||||
|
||||
**Total Estimated Effort**: 16-20 hours of focused development
|
||||
|
||||
## Conclusion
|
||||
|
||||
Version 1.1.0 represents a significant improvement in usability while maintaining our minimal philosophy. The migration system redesign eliminates technical debt, full-text search adds essential functionality, and custom slugs improve standards compliance.
|
||||
|
||||
The implementation should proceed in the order specified, with the migration system being absolutely critical to complete first. Each feature has been designed to be simple, elegant, and maintainable.
|
||||
|
||||
Remember our core principle: "Every line of code must justify its existence."
|
||||
Reference in New Issue
Block a user