phil/StarPunk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Add v1.1.0 architecture and validation documentation

Browse Source 
- ADR-033: Database migration redesign
- ADR-034: Full-text search with FTS5
- ADR-035: Custom slugs in Micropub
- ADR-036: IndieAuth token verification method
- ADR-039: Micropub URL construction fix
- Implementation plan and decisions
- Architecture specifications
- Validation reports for implementation and search UI

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Phil Skentelbery
2025-11-25 10:39:58 -07:00
parent 8f71ff36ec
commit 82bb1499d5
13 changed files with 3324 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

446
docs/architecture/v1.1.0-implementation-decisions.md Normal file
View File

@@ -0,0 +1,446 @@
# V1.1.0 Implementation Decisions - Architectural Guidance
## Overview
This document provides definitive architectural decisions for all 29 questions raised during v1.1.0 implementation planning. Each decision is final and actionable.
---
## RSS Feed Fix Decisions
### Q1: No Bug Exists - Action Required?
**Decision**: Add a regression test and close as "working as intended"
**Rationale**: Since the RSS feed is already correctly ordered (newest first), we should document this as the intended behavior and prevent future regressions.
**Implementation**:
1. Add test case: `test_feed_order_newest_first()` in `tests/test_feed.py`
2. Add comment above line 96 in `feed.py`: `# Notes are already DESC ordered from database`
3. Close the issue with note: "Verified feed order is correct (newest first)"
### Q2: Line 96 Loop - Keep As-Is?
**Decision**: Keep the current implementation unchanged
**Rationale**: The `for note in notes[:limit]:` loop is correct because notes are already sorted DESC by created_at from the database query.
**Implementation**: No code change needed. Add clarifying comment if not already present.
---
## Migration System Redesign (ADR-033)
### Q3: INITIAL_SCHEMA_SQL Storage Location
**Decision**: Store in `starpunk/database.py` as a module-level constant
**Rationale**: Keeps schema definitions close to database initialization code.
**Implementation**:
```python
# In starpunk/database.py, after imports:
INITIAL_SCHEMA_SQL = """
-- V1.0.0 Schema - DO NOT MODIFY
-- All changes must go in migration files
[... original schema from v1.0.0 ...]
"""
```
### Q4: Existing SCHEMA_SQL Variable
**Decision**: Keep both with clear naming
**Implementation**:
1. Rename current `SCHEMA_SQL` to `INITIAL_SCHEMA_SQL`
2. Add new variable `CURRENT_SCHEMA_SQL` that will be built from initial + migrations
3. Document the purpose of each in comments
### Q5: Modify init_db() Detection
**Decision**: Yes, modify `init_db()` to detect fresh install
**Implementation**:
```python
def init_db(app=None):
"""Initialize database with proper schema"""
conn = get_db_connection()
# Check if this is a fresh install
cursor = conn.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='migrations'")
is_fresh = cursor.fetchone() is None
if is_fresh:
# Fresh install: use initial schema
conn.executescript(INITIAL_SCHEMA_SQL)
conn.execute("INSERT INTO migrations (version, applied_at) VALUES ('initial', CURRENT_TIMESTAMP)")
# Apply any pending migrations
apply_pending_migrations(conn)
```
### Q6: Users Upgrading from v1.0.1
**Decision**: Automatic migration on application start
**Rationale**: Zero-downtime upgrade with automatic schema updates.
**Implementation**:
1. Application detects current version via migrations table
2. Applies only new migrations (005+)
3. No manual intervention required
4. Add startup log: "Database migrated to v1.1.0"
### Q7: Existing Migrations 001-004
**Decision**: Leave existing migrations unchanged
**Rationale**: These are historical records and changing them would break existing deployments.
**Implementation**: Do not modify files. They remain for upgrade path from older versions.
### Q8: Testing Both Paths
**Decision**: Create two separate test scenarios
**Implementation**:
```python
# tests/test_migrations.py
def test_fresh_install():
"""Test database creation from scratch"""
# Start with no database
# Run init_db()
# Verify all tables exist with correct schema
def test_upgrade_from_v1_0_1():
"""Test upgrade path"""
# Create database with v1.0.1 schema
# Add sample data
# Run init_db()
# Verify migrations applied
# Verify data preserved
```
---
## Full-Text Search (ADR-034)
### Q9: Title Source
**Decision**: Extract title from first line of markdown content
**Rationale**: Notes table doesn't have a title column. Follow existing pattern where title is derived from content.
**Implementation**:
```sql
-- Use SQL to extract first line as title
substr(content, 1, instr(content || char(10), char(10)) - 1) as title
```
### Q10: Trigger Implementation
**Decision**: Use SQL expression to extract title, not a custom function
**Rationale**: Simpler, no UDF required, portable across SQLite versions.
**Implementation**:
```sql
CREATE TRIGGER notes_fts_insert AFTER INSERT ON notes
BEGIN
INSERT INTO notes_fts (rowid, slug, title, content)
SELECT
NEW.id,
NEW.slug,
substr(content, 1, min(60, ifnull(nullif(instr(content, char(10)), 0) - 1, length(content)))),
content
FROM note_files WHERE file_path = NEW.file_path;
END;
```
### Q11: Migration 005 Scope
**Decision**: Yes, create everything in one migration
**Rationale**: Atomic operation ensures consistency.
**Implementation in `migrations/005_add_full_text_search.sql`:
1. Create FTS5 virtual table
2. Create all three triggers (INSERT, UPDATE, DELETE)
3. Build initial index from existing notes
4. All in single transaction
### Q12: Search Endpoint URL
**Decision**: `/api/search`
**Rationale**: Consistent with existing API pattern, RESTful design.
**Implementation**: Register route in `app.py` or API blueprint.
### Q13: Template Files Needing Modification
**Decision**: Modify `base.html` for search box, create new `search.html` for results
**Implementation**:
- `templates/base.html`: Add search form in navigation
- `templates/search.html`: New template for search results page
- `templates/partials/search-result.html`: Result item component
### Q14: Search Filtering by Authentication
**Decision**: Yes, filter by published status
**Implementation**:
```python
if not is_authenticated():
query += " AND published = 1"
```
### Q15: FTS5 Unavailable Handling
**Decision**: Disable search gracefully with warning
**Rationale**: Better UX than failing to start.
**Implementation**:
```python
def check_fts5_support():
try:
conn.execute("CREATE VIRTUAL TABLE test_fts USING fts5(content)")
conn.execute("DROP TABLE test_fts")
return True
except sqlite3.OperationalError:
app.logger.warning("FTS5 not available - search disabled")
return False
```
---
## Custom Slugs (ADR-035)
### Q16: mp-slug Extraction Location
**Decision**: In `handle_create()` function after properties normalization
**Implementation**:
```python
def handle_create(request: Request) -> dict:
properties = normalize_properties(request)
# Extract custom slug if provided
custom_slug = properties.get('mp-slug', [None])[0]
# Continue with note creation...
```
### Q17: Slug Validation Functions Location
**Decision**: Create new module `starpunk/slug_utils.py`
**Rationale**: Slug handling is complex enough to warrant its own module.
**Implementation**: New file with functions: `validate_slug()`, `sanitize_slug()`, `ensure_unique_slug()`
### Q18: RESERVED_SLUGS Storage
**Decision**: Module constant in `slug_utils.py`
**Implementation**:
```python
# starpunk/slug_utils.py
RESERVED_SLUGS = frozenset([
'api', 'admin', 'auth', 'feed', 'static',
'login', 'logout', 'settings', 'micropub'
])
```
### Q19: Conflict Resolution Strategy
**Decision**: Use sequential numbers (-2, -3, etc.)
**Rationale**: Predictable, easier to debug, standard practice.
**Implementation**:
```python
def make_unique_slug(base_slug: str, max_attempts: int = 99) -> str:
for i in range(2, max_attempts + 2):
candidate = f"{base_slug}-{i}"
if not slug_exists(candidate):
return candidate
raise ValueError(f"Could not create unique slug after {max_attempts} attempts")
```
### Q20: Hierarchical Slugs Support
**Decision**: No, defer to v1.2.0
**Rationale**: Adds routing complexity, not essential for v1.1.0.
**Implementation**: Validate slugs don't contain `/`. Add to roadmap for v1.2.0.
### Q21: Existing Slug Field Sufficient?
**Decision**: Yes, current schema is sufficient
**Rationale**: `slug TEXT UNIQUE NOT NULL` already enforces uniqueness.
**Implementation**: No migration needed.
### Q22: Micropub Error Format
**Decision**: Follow Micropub spec exactly
**Implementation**:
```python
return jsonify({
"error": "invalid_request",
"error_description": f"Invalid slug format: {reason}"
}), 400
```
---
## General Implementation Decisions
### Q23: Implementation Sequence
**Decision**: Follow sequence but document design for all components first
**Rationale**: Design clarity prevents rework.
**Implementation**:
1. Day 1: Document all component designs
2. Days 2-4: Implement in sequence
3. Day 5: Integration testing
### Q24: Branching Strategy
**Decision**: Single feature branch: `feature/v1.1.0`
**Rationale**: Components are interdependent, easier to test together.
**Implementation**:
```bash
git checkout -b feature/v1.1.0
# All work happens here
# PR to main when complete
```
### Q25: Test Writing Strategy
**Decision**: Write tests immediately after each component
**Rationale**: Ensures each component works before moving on.
**Implementation**:
1. Implement feature
2. Write tests
3. Verify tests pass
4. Move to next component
### Q26: Version Bump Timing
**Decision**: Bump version in final commit before merge
**Rationale**: Version represents released code, not development code.
**Implementation**:
1. Complete all features
2. Update `__version__` to "1.1.0"
3. Update CHANGELOG.md
4. Commit: "chore: bump version to 1.1.0"
### Q27: New Migration Numbering
**Decision**: Continue sequential: 005, 006, etc.
**Implementation**:
- `005_add_full_text_search.sql`
- `006_add_custom_slug_support.sql` (if needed)
### Q28: Progress Documentation
**Decision**: Daily updates in `/docs/reports/v1.1.0-progress.md`
**Implementation**:
```markdown
# V1.1.0 Implementation Progress
## Day 1 - [Date]
### Completed
- [ ] Task 1
- [ ] Task 2
### Blockers
- None
### Notes
- Implementation detail...
```
### Q29: Backwards Compatibility Verification
**Decision**: Test suite with v1.0.1 data
**Implementation**:
1. Create test database with v1.0.1 schema
2. Add sample data
3. Run upgrade
4. Verify all existing features work
5. Verify API compatibility
---
## Developer Observations - Responses
### Migration System Complexity
**Response**: Allocate extra 2 hours. Better to overdeliver than rush.
### FTS5 Title Extraction
**Response**: Correct - index full content only in v1.1.0. Title extraction is display concern.
### Search UI Template Review
**Response**: Keep minimal - search box in nav, simple results page. No JavaScript.
### Testing Time Optimistic
**Response**: Add 2 hours buffer for testing. Quality over speed.
### Slug Validation Security
**Response**: Yes, add fuzzing tests for slug validation. Security is non-negotiable.
### Performance Benchmarking
**Response**: Defer to v1.2.0. Focus on correctness in v1.1.0.
---
## Implementation Checklist Order
1. **Day 1 - Design & Setup**
- [ ] Create feature branch
- [ ] Write component designs
- [ ] Set up test fixtures
2. **Day 2 - Migration System**
- [ ] Implement INITIAL_SCHEMA_SQL
- [ ] Refactor init_db()
- [ ] Write migration tests
- [ ] Test both paths
3. **Day 3 - Full-Text Search**
- [ ] Create migration 005
- [ ] Implement search endpoint
- [ ] Add search UI
- [ ] Write search tests
4. **Day 4 - Custom Slugs**
- [ ] Create slug_utils.py
- [ ] Modify micropub.py
- [ ] Add validation
- [ ] Write slug tests
5. **Day 5 - Integration**
- [ ] Full system testing
- [ ] Update documentation
- [ ] Bump version
- [ ] Create PR
---
## Risk Mitigations
1. **Database Corruption**: Test migrations on copy first
2. **Search Performance**: Limit results to 100 maximum
3. **Slug Conflicts**: Clear error messages for users
4. **Upgrade Failures**: Provide rollback instructions
5. **FTS5 Missing**: Graceful degradation
---
## Success Criteria
- [ ] All existing tests pass
- [ ] New tests for all features
- [ ] No breaking changes to API
- [ ] Documentation updated
- [ ] Performance acceptable (<100ms responses)
- [ ] Security review passed
- [ ] Backwards compatible with v1.0.1 data
---
## Notes
- This document represents final architectural decisions
- Any deviations require ADR and approval
- Focus on simplicity and correctness
- When in doubt, defer complexity to v1.2.0