that initial commit

2025-11-18 19:21:31 -07:00
commit a68fd570c7
69 changed files with 31070 additions and 0 deletions
--- a/docs/design/phase-2.1-quick-reference.md
+++ b/docs/design/phase-2.1-quick-reference.md
@@ -0,0 +1,616 @@
+# Phase 2.1: Notes Management - Quick Reference
+
+## Overview
+
+Quick reference guide for implementing Phase 2.1: Notes Management (CRUD operations) in StarPunk.
+
+**File**: `starpunk/notes.py`
+**Estimated Time**: 6-8 hours
+**Dependencies**: `utils.py`, `models.py`, `database.py`
+
+---
+
+## Function Checklist
+
+### Required Functions
+
+- [ ] **create_note(content, published=False, created_at=None) -> Note**
+  - Generate unique slug
+  - Write file atomically
+  - Insert database record
+  - Return Note object
+
+- [ ] **get_note(slug=None, id=None, load_content=True) -> Optional[Note]**
+  - Query database by slug or id
+  - Load content from file if requested
+  - Return Note or None
+
+- [ ] **list_notes(published_only=False, limit=50, offset=0, order_by='created_at', order_dir='DESC') -> list[Note]**
+  - Query database with filters
+  - Support pagination
+  - No file I/O (metadata only)
+  - Return list of Notes
+
+- [ ] **update_note(slug=None, id=None, content=None, published=None) -> Note**
+  - Update file if content changed
+  - Update database record
+  - Return updated Note
+
+- [ ] **delete_note(slug=None, id=None, soft=True) -> None**
+  - Soft delete: mark deleted_at in database
+  - Hard delete: remove file and database record
+  - Return None
+
+### Custom Exceptions
+
+- [ ] **NoteNotFoundError(Exception)**
+  - Raised when note doesn't exist
+
+- [ ] **InvalidNoteDataError(Exception)**
+  - Raised for invalid content/parameters
+
+- [ ] **NoteSyncError(Exception)**
+  - Raised when file/database sync fails
+
+---
+
+## Implementation Order
+
+### Step 1: Module Setup (15 minutes)
+
+```python
+# starpunk/notes.py
+"""Notes management for StarPunk"""
+
+# Imports
+from datetime import datetime
+from pathlib import Path
+from typing import Optional
+from flask import current_app
+from starpunk.database import get_db
+from starpunk.models import Note
+from starpunk.utils import (
+    generate_slug, make_slug_unique, generate_note_path,
+    ensure_note_directory, write_note_file, read_note_file,
+    delete_note_file, calculate_content_hash,
+    validate_note_path, validate_slug
+)
+
+# Exception classes (define all 3)
+```
+
+**Time**: 15 minutes
+
+### Step 2: create_note() (90 minutes)
+
+**Algorithm**:
+1. Validate content not empty
+2. Set created_at to now if not provided
+3. Query existing slugs from database
+4. Generate unique slug
+5. Generate file path
+6. Validate path (security)
+7. Calculate content hash
+8. Write file (ensure_note_directory + write_note_file)
+9. Insert database record
+10. If DB fails: delete file, raise NoteSyncError
+11. If success: commit, return Note object
+
+**Testing**:
+- Create basic note
+- Create with empty content (should fail)
+- Create with duplicate slug (should add suffix)
+- Create with specific timestamp
+- File write fails (should not create DB record)
+
+**Time**: 90 minutes (45 min implementation + 45 min testing)
+
+### Step 3: get_note() (45 minutes)
+
+**Algorithm**:
+1. Validate parameters (exactly one of slug or id)
+2. Query database
+3. Return None if not found
+4. Create Note.from_row()
+5. Optionally verify integrity (log warning if mismatch)
+6. Return Note
+
+**Testing**:
+- Get by slug
+- Get by id
+- Get nonexistent (returns None)
+- Get with both parameters (should fail)
+- Get without loading content
+
+**Time**: 45 minutes (25 min implementation + 20 min testing)
+
+### Step 4: list_notes() (60 minutes)
+
+**Algorithm**:
+1. Validate order_by (whitelist check)
+2. Validate order_dir (ASC/DESC)
+3. Validate limit (max 1000)
+4. Build SQL query with filters
+5. Add ORDER BY and LIMIT/OFFSET
+6. Execute query
+7. Create Note objects (don't load content)
+8. Return list
+
+**Testing**:
+- List all notes
+- List published only
+- List with pagination
+- List with different ordering
+- Invalid order_by (should fail - SQL injection test)
+
+**Time**: 60 minutes (35 min implementation + 25 min testing)
+
+### Step 5: update_note() (90 minutes)
+
+**Algorithm**:
+1. Validate parameters
+2. Get existing note (raises NoteNotFoundError if missing)
+3. Validate content if provided
+4. Setup paths and timestamps
+5. If content changed: write new file, calculate new hash
+6. Build UPDATE query for changed fields
+7. Execute database update
+8. If DB fails: log error, raise NoteSyncError
+9. If success: commit, return updated Note
+
+**Testing**:
+- Update content only
+- Update published only
+- Update both
+- Update nonexistent (should fail)
+- Update with empty content (should fail)
+- Update with no changes (should fail)
+
+**Time**: 90 minutes (50 min implementation + 40 min testing)
+
+### Step 6: delete_note() (60 minutes)
+
+**Algorithm**:
+1. Validate parameters
+2. Get existing note (return if None - idempotent)
+3. Validate path
+4. If soft delete:
+   - UPDATE notes SET deleted_at = now WHERE id = ?
+   - Optionally move file to trash (best effort)
+5. If hard delete:
+   - DELETE FROM notes WHERE id = ?
+   - Delete file (best effort)
+6. Return None
+
+**Testing**:
+- Soft delete
+- Hard delete
+- Delete nonexistent (should succeed)
+- Delete already deleted (should succeed)
+
+**Time**: 60 minutes (35 min implementation + 25 min testing)
+
+### Step 7: Integration Tests (60 minutes)
+
+**Full CRUD cycle test**:
+1. Create note
+2. Retrieve note
+3. Update content
+4. Update published status
+5. List notes (verify appears)
+6. Delete note
+7. Verify gone
+
+**Sync tests**:
+- Verify file exists after create
+- Verify DB record exists after create
+- Verify file updated after update
+- Verify file deleted after hard delete
+- Verify DB record deleted after hard delete
+
+**Time**: 60 minutes
+
+### Step 8: Documentation and Cleanup (30 minutes)
+
+- Review all docstrings
+- Format with Black
+- Run flake8
+- Check type hints
+- Review error messages
+
+**Time**: 30 minutes
+
+---
+
+## Common Pitfalls
+
+### 1. Forgetting to Commit Transactions
+
+```python
+# BAD - no commit
+db.execute("INSERT INTO notes ...")
+
+# GOOD - explicit commit
+db.execute("INSERT INTO notes ...")
+db.commit()
+```
+
+### 2. Not Cleaning Up on Failure
+
+```python
+# BAD - orphaned file if DB fails
+write_note_file(path, content)
+db.execute("INSERT ...")  # What if this fails?
+
+# GOOD - cleanup on failure
+write_note_file(path, content)
+try:
+    db.execute("INSERT ...")
+    db.commit()
+except Exception as e:
+    path.unlink()  # Delete file we created
+    raise NoteSyncError(...)
+```
+
+### 3. SQL Injection in ORDER BY
+
+```python
+# BAD - SQL injection risk
+order_by = request.args.get('order')
+query = f"SELECT * FROM notes ORDER BY {order_by}"
+
+# GOOD - whitelist validation
+ALLOWED = ['id', 'slug', 'created_at', 'updated_at']
+if order_by not in ALLOWED:
+    raise ValueError(f"Invalid order_by: {order_by}")
+query = f"SELECT * FROM notes ORDER BY {order_by}"
+```
+
+### 4. Not Using Parameterized Queries
+
+```python
+# BAD - SQL injection risk
+slug = request.args.get('slug')
+query = f"SELECT * FROM notes WHERE slug = '{slug}'"
+
+# GOOD - parameterized query
+query = "SELECT * FROM notes WHERE slug = ?"
+db.execute(query, (slug,))
+```
+
+### 5. Forgetting Path Validation
+
+```python
+# BAD - directory traversal risk
+note_path = data_dir / file_path
+write_note_file(note_path, content)
+
+# GOOD - validate path
+note_path = data_dir / file_path
+if not validate_note_path(note_path, data_dir):
+    raise NoteSyncError(...)
+write_note_file(note_path, content)
+```
+
+### 6. Not Handling None in Optional Parameters
+
+```python
+# BAD - will crash on None
+if slug and id:
+    raise ValueError(...)
+
+# GOOD - explicit None checks
+if slug is None and id is None:
+    raise ValueError("Must provide slug or id")
+if slug is not None and id is not None:
+    raise ValueError("Cannot provide both")
+```
+
+---
+
+## Testing Checklist
+
+### Unit Tests
+
+- [ ] create_note with valid content
+- [ ] create_note with empty content (fail)
+- [ ] create_note with duplicate slug (unique suffix)
+- [ ] create_note with specific timestamp
+- [ ] create_note with unicode content
+- [ ] create_note file write failure (no DB record)
+- [ ] get_note by slug
+- [ ] get_note by id
+- [ ] get_note nonexistent (returns None)
+- [ ] get_note with invalid parameters
+- [ ] get_note without loading content
+- [ ] list_notes all
+- [ ] list_notes published only
+- [ ] list_notes with pagination
+- [ ] list_notes with ordering
+- [ ] list_notes with invalid order_by (fail)
+- [ ] update_note content
+- [ ] update_note published
+- [ ] update_note both
+- [ ] update_note nonexistent (fail)
+- [ ] update_note empty content (fail)
+- [ ] delete_note soft
+- [ ] delete_note hard
+- [ ] delete_note nonexistent (succeed)
+
+### Integration Tests
+
+- [ ] Full CRUD cycle (create → read → update → delete)
+- [ ] File-database sync maintained throughout lifecycle
+- [ ] Orphaned files cleaned up on DB failure
+- [ ] Soft-deleted notes excluded from queries
+- [ ] Hard-deleted notes removed from DB and filesystem
+
+### Performance Tests
+
+- [ ] list_notes with 1000 notes (< 10ms)
+- [ ] get_note (< 10ms)
+- [ ] create_note (< 20ms)
+
+---
+
+## Time Estimates
+
+| Task | Time |
+|------|------|
+| Module setup | 15 min |
+| create_note() | 90 min |
+| get_note() | 45 min |
+| list_notes() | 60 min |
+| update_note() | 90 min |
+| delete_note() | 60 min |
+| Integration tests | 60 min |
+| Documentation/cleanup | 30 min |
+| **Total** | **7.5 hours** |
+
+Add 30-60 minutes for unexpected issues and debugging.
+
+---
+
+## Key Design Decisions
+
+### 1. File Operations Before Database
+
+**Rationale**: Fail fast on disk issues before database changes.
+
+**Pattern**:
+```python
+# Write file first
+write_note_file(path, content)
+
+# Then update database
+db.execute("INSERT ...")
+db.commit()
+
+# If DB fails, cleanup file
+```
+
+### 2. Best-Effort File Cleanup
+
+**Rationale**: Database is source of truth. Missing files can be recreated or cleaned up later.
+
+**Pattern**:
+```python
+try:
+    path.unlink()
+except OSError:
+    logger.warning("Cleanup failed")
+    # Don't fail - log and continue
+```
+
+### 3. Idempotent Delete
+
+**Rationale**: DELETE operations should succeed even if already deleted.
+
+**Pattern**:
+```python
+note = get_note(slug=slug)
+if note is None:
+    return  # Already deleted, that's fine
+# ... proceed with delete
+```
+
+### 4. Lazy Content Loading
+
+**Rationale**: list_notes() should not trigger file I/O for every note.
+
+**Pattern**:
+```python
+# list_notes creates Notes without loading content
+notes = [Note.from_row(row, data_dir) for row in rows]
+
+# Content loaded on access
+for note in notes:
+    print(note.slug)      # Fast (metadata)
+    print(note.content)   # Triggers file I/O
+```
+
+### 5. Parameterized Queries Only
+
+**Rationale**: Prevent SQL injection.
+
+**Pattern**:
+```python
+# Always use parameter binding
+db.execute("SELECT * FROM notes WHERE slug = ?", (slug,))
+
+# Never use string interpolation
+db.execute(f"SELECT * FROM notes WHERE slug = '{slug}'")  # NO!
+```
+
+---
+
+## Dependencies Reference
+
+### From utils.py
+
+```python
+generate_slug(content, created_at) -> str
+make_slug_unique(base_slug, existing_slugs) -> str
+validate_slug(slug) -> bool
+generate_note_path(slug, created_at, data_dir) -> Path
+ensure_note_directory(note_path) -> Path
+write_note_file(file_path, content) -> None
+read_note_file(file_path) -> str
+delete_note_file(file_path, soft=False, data_dir=None) -> None
+calculate_content_hash(content) -> str
+validate_note_path(file_path, data_dir) -> bool
+```
+
+### From models.py
+
+```python
+Note.from_row(row, data_dir) -> Note
+Note.content -> str (property, lazy-loaded)
+Note.to_dict(include_content=False) -> dict
+Note.verify_integrity() -> bool
+```
+
+### From database.py
+
+```python
+get_db() -> sqlite3.Connection
+db.execute(query, params) -> Cursor
+db.commit() -> None
+db.rollback() -> None
+```
+
+---
+
+## Error Handling Quick Reference
+
+### When to Raise vs Return None
+
+| Scenario | Action |
+|----------|--------|
+| Note not found in get_note() | Return None |
+| Note not found in update_note() | Raise NoteNotFoundError |
+| Note not found in delete_note() | Return None (idempotent) |
+| Empty content | Raise InvalidNoteDataError |
+| File write fails | Raise NoteSyncError |
+| Database fails | Raise NoteSyncError |
+| Invalid parameters | Raise ValueError |
+
+### Error Message Examples
+
+```python
+# NoteNotFoundError
+raise NoteNotFoundError(
+    slug,
+    f"Note '{slug}' does not exist or has been deleted"
+)
+
+# InvalidNoteDataError
+raise InvalidNoteDataError(
+    'content',
+    content,
+    "Note content cannot be empty. Please provide markdown content."
+)
+
+# NoteSyncError
+raise NoteSyncError(
+    'create',
+    f"Database insert failed: {str(e)}",
+    f"Failed to create note. File written but database update failed."
+)
+```
+
+---
+
+## Security Checklist
+
+- [ ] All SQL queries use parameterized binding (no string interpolation)
+- [ ] order_by field validated against whitelist
+- [ ] All file paths validated with validate_note_path()
+- [ ] No symlinks followed in path operations
+- [ ] Content validated (not empty)
+- [ ] Slug validated before use in file paths
+- [ ] No code execution from user content
+
+---
+
+## Final Checks
+
+Before submitting Phase 2.1 as complete:
+
+- [ ] All 5 functions implemented
+- [ ] All 3 exceptions implemented
+- [ ] Full type hints on all functions
+- [ ] Comprehensive docstrings with examples
+- [ ] Test coverage > 90%
+- [ ] All tests passing
+- [ ] Black formatting applied
+- [ ] flake8 linting passes (no errors)
+- [ ] Integration test passes (full CRUD cycle)
+- [ ] No orphaned files in test runs
+- [ ] No orphaned database records in test runs
+- [ ] Error messages are clear and actionable
+- [ ] Performance targets met:
+  - [ ] create_note < 20ms
+  - [ ] get_note < 10ms
+  - [ ] list_notes < 10ms
+  - [ ] update_note < 20ms
+  - [ ] delete_note < 10ms
+
+---
+
+## Quick Command Reference
+
+```bash
+# Run tests
+pytest tests/test_notes.py -v
+
+# Check coverage
+pytest tests/test_notes.py --cov=starpunk.notes --cov-report=term-missing
+
+# Format code
+black starpunk/notes.py tests/test_notes.py
+
+# Lint code
+flake8 starpunk/notes.py --max-line-length=100
+
+# Type check (optional)
+mypy starpunk/notes.py
+```
+
+---
+
+## What's Next?
+
+After completing Phase 2.1:
+
+**Phase 3: Authentication**
+- IndieLogin OAuth flow
+- Session management
+- Admin access control
+
+**Phase 4: Web Routes**
+- Admin interface (create/edit/delete notes)
+- Public note views
+- Template rendering
+
+**Phase 5: Micropub**
+- Micropub endpoint
+- Token validation
+- IndieWeb API compliance
+
+**Phase 6: RSS Feed**
+- Feed generation
+- RFC-822 date formatting
+- Published notes only
+
+---
+
+## Help & Resources
+
+- **Full Design Doc**: [phase-2.1-notes-management.md](/home/phil/Projects/starpunk/docs/design/phase-2.1-notes-management.md)
+- **Utilities Design**: [phase-1.1-core-utilities.md](/home/phil/Projects/starpunk/docs/design/phase-1.1-core-utilities.md)
+- **Models Design**: [phase-1.2-data-models.md](/home/phil/Projects/starpunk/docs/design/phase-1.2-data-models.md)
+- **Coding Standards**: [python-coding-standards.md](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
+- **Implementation Plan**: [implementation-plan.md](/home/phil/Projects/starpunk/docs/projectplan/v1/implementation-plan.md)
+
+Remember: "Every line of code must justify its existence. When in doubt, leave it out."