StarPunk/docs/design/phase-2.1-quick-reference.md

# Phase 2.1: Notes Management - Quick Reference

## Overview

Quick reference guide for implementing Phase 2.1: Notes Management (CRUD operations) in StarPunk.

**File**: `starpunk/notes.py`
**Estimated Time**: 6-8 hours
**Dependencies**: `utils.py`, `models.py`, `database.py`

---

## Function Checklist

### Required Functions

- [ ] **create_note(content, published=False, created_at=None) -> Note**
  - Generate unique slug
  - Write file atomically
  - Insert database record
  - Return Note object

- [ ] **get_note(slug=None, id=None, load_content=True) -> Optional[Note]**
  - Query database by slug or id
  - Load content from file if requested
  - Return Note or None

- [ ] **list_notes(published_only=False, limit=50, offset=0, order_by='created_at', order_dir='DESC') -> list[Note]**
  - Query database with filters
  - Support pagination
  - No file I/O (metadata only)
  - Return list of Notes

- [ ] **update_note(slug=None, id=None, content=None, published=None) -> Note**
  - Update file if content changed
  - Update database record
  - Return updated Note

- [ ] **delete_note(slug=None, id=None, soft=True) -> None**
  - Soft delete: mark deleted_at in database
  - Hard delete: remove file and database record
  - Return None

### Custom Exceptions

- [ ] **NoteNotFoundError(Exception)**
  - Raised when note doesn't exist

- [ ] **InvalidNoteDataError(Exception)**
  - Raised for invalid content/parameters

- [ ] **NoteSyncError(Exception)**
  - Raised when file/database sync fails

---

## Implementation Order

### Step 1: Module Setup (15 minutes)

```python
# starpunk/notes.py
"""Notes management for StarPunk"""

# Imports
from datetime import datetime
from pathlib import Path
from typing import Optional
from flask import current_app
from starpunk.database import get_db
from starpunk.models import Note
from starpunk.utils import (
    generate_slug, make_slug_unique, generate_note_path,
    ensure_note_directory, write_note_file, read_note_file,
    delete_note_file, calculate_content_hash,
    validate_note_path, validate_slug
)

# Exception classes (define all 3)
```

**Time**: 15 minutes

### Step 2: create_note() (90 minutes)

**Algorithm**:
1. Validate content not empty
2. Set created_at to now if not provided
3. Query existing slugs from database
4. Generate unique slug
5. Generate file path
6. Validate path (security)
7. Calculate content hash
8. Write file (ensure_note_directory + write_note_file)
9. Insert database record
10. If DB fails: delete file, raise NoteSyncError
11. If success: commit, return Note object

**Testing**:
- Create basic note
- Create with empty content (should fail)
- Create with duplicate slug (should add suffix)
- Create with specific timestamp
- File write fails (should not create DB record)

**Time**: 90 minutes (45 min implementation + 45 min testing)

### Step 3: get_note() (45 minutes)

**Algorithm**:
1. Validate parameters (exactly one of slug or id)
2. Query database
3. Return None if not found
4. Create Note.from_row()
5. Optionally verify integrity (log warning if mismatch)
6. Return Note

**Testing**:
- Get by slug
- Get by id
- Get nonexistent (returns None)
- Get with both parameters (should fail)
- Get without loading content

**Time**: 45 minutes (25 min implementation + 20 min testing)

### Step 4: list_notes() (60 minutes)

**Algorithm**:
1. Validate order_by (whitelist check)
2. Validate order_dir (ASC/DESC)
3. Validate limit (max 1000)
4. Build SQL query with filters
5. Add ORDER BY and LIMIT/OFFSET
6. Execute query
7. Create Note objects (don't load content)
8. Return list

**Testing**:
- List all notes
- List published only
- List with pagination
- List with different ordering
- Invalid order_by (should fail - SQL injection test)

**Time**: 60 minutes (35 min implementation + 25 min testing)

### Step 5: update_note() (90 minutes)

**Algorithm**:
1. Validate parameters
2. Get existing note (raises NoteNotFoundError if missing)
3. Validate content if provided
4. Setup paths and timestamps
5. If content changed: write new file, calculate new hash
6. Build UPDATE query for changed fields
7. Execute database update
8. If DB fails: log error, raise NoteSyncError
9. If success: commit, return updated Note

**Testing**:
- Update content only
- Update published only
- Update both
- Update nonexistent (should fail)
- Update with empty content (should fail)
- Update with no changes (should fail)

**Time**: 90 minutes (50 min implementation + 40 min testing)

### Step 6: delete_note() (60 minutes)

**Algorithm**:
1. Validate parameters
2. Get existing note (return if None - idempotent)
3. Validate path
4. If soft delete:
   - UPDATE notes SET deleted_at = now WHERE id = ?
   - Optionally move file to trash (best effort)
5. If hard delete:
   - DELETE FROM notes WHERE id = ?
   - Delete file (best effort)
6. Return None

**Testing**:
- Soft delete
- Hard delete
- Delete nonexistent (should succeed)
- Delete already deleted (should succeed)

**Time**: 60 minutes (35 min implementation + 25 min testing)

### Step 7: Integration Tests (60 minutes)

**Full CRUD cycle test**:
1. Create note
2. Retrieve note
3. Update content
4. Update published status
5. List notes (verify appears)
6. Delete note
7. Verify gone

**Sync tests**:
- Verify file exists after create
- Verify DB record exists after create
- Verify file updated after update
- Verify file deleted after hard delete
- Verify DB record deleted after hard delete

**Time**: 60 minutes

### Step 8: Documentation and Cleanup (30 minutes)

- Review all docstrings
- Format with Black
- Run flake8
- Check type hints
- Review error messages

**Time**: 30 minutes

---

## Common Pitfalls

### 1. Forgetting to Commit Transactions

```python
# BAD - no commit
db.execute("INSERT INTO notes ...")

# GOOD - explicit commit
db.execute("INSERT INTO notes ...")
db.commit()
```

### 2. Not Cleaning Up on Failure

```python
# BAD - orphaned file if DB fails
write_note_file(path, content)
db.execute("INSERT ...")  # What if this fails?

# GOOD - cleanup on failure
write_note_file(path, content)
try:
    db.execute("INSERT ...")
    db.commit()
except Exception as e:
    path.unlink()  # Delete file we created
    raise NoteSyncError(...)
```

### 3. SQL Injection in ORDER BY

```python
# BAD - SQL injection risk
order_by = request.args.get('order')
query = f"SELECT * FROM notes ORDER BY {order_by}"

# GOOD - whitelist validation
ALLOWED = ['id', 'slug', 'created_at', 'updated_at']
if order_by not in ALLOWED:
    raise ValueError(f"Invalid order_by: {order_by}")
query = f"SELECT * FROM notes ORDER BY {order_by}"
```

### 4. Not Using Parameterized Queries

```python
# BAD - SQL injection risk
slug = request.args.get('slug')
query = f"SELECT * FROM notes WHERE slug = '{slug}'"

# GOOD - parameterized query
query = "SELECT * FROM notes WHERE slug = ?"
db.execute(query, (slug,))
```

### 5. Forgetting Path Validation

```python
# BAD - directory traversal risk
note_path = data_dir / file_path
write_note_file(note_path, content)

# GOOD - validate path
note_path = data_dir / file_path
if not validate_note_path(note_path, data_dir):
    raise NoteSyncError(...)
write_note_file(note_path, content)
```

### 6. Not Handling None in Optional Parameters

```python
# BAD - will crash on None
if slug and id:
    raise ValueError(...)

# GOOD - explicit None checks
if slug is None and id is None:
    raise ValueError("Must provide slug or id")
if slug is not None and id is not None:
    raise ValueError("Cannot provide both")
```

---

## Testing Checklist

### Unit Tests

- [ ] create_note with valid content
- [ ] create_note with empty content (fail)
- [ ] create_note with duplicate slug (unique suffix)
- [ ] create_note with specific timestamp
- [ ] create_note with unicode content
- [ ] create_note file write failure (no DB record)
- [ ] get_note by slug
- [ ] get_note by id
- [ ] get_note nonexistent (returns None)
- [ ] get_note with invalid parameters
- [ ] get_note without loading content
- [ ] list_notes all
- [ ] list_notes published only
- [ ] list_notes with pagination
- [ ] list_notes with ordering
- [ ] list_notes with invalid order_by (fail)
- [ ] update_note content
- [ ] update_note published
- [ ] update_note both
- [ ] update_note nonexistent (fail)
- [ ] update_note empty content (fail)
- [ ] delete_note soft
- [ ] delete_note hard
- [ ] delete_note nonexistent (succeed)

### Integration Tests

- [ ] Full CRUD cycle (create → read → update → delete)
- [ ] File-database sync maintained throughout lifecycle
- [ ] Orphaned files cleaned up on DB failure
- [ ] Soft-deleted notes excluded from queries
- [ ] Hard-deleted notes removed from DB and filesystem

### Performance Tests

- [ ] list_notes with 1000 notes (< 10ms)
- [ ] get_note (< 10ms)
- [ ] create_note (< 20ms)

---

## Time Estimates

| Task | Time |
|------|------|
| Module setup | 15 min |
| create_note() | 90 min |
| get_note() | 45 min |
| list_notes() | 60 min |
| update_note() | 90 min |
| delete_note() | 60 min |
| Integration tests | 60 min |
| Documentation/cleanup | 30 min |
| **Total** | **7.5 hours** |

Add 30-60 minutes for unexpected issues and debugging.

---

## Key Design Decisions

### 1. File Operations Before Database

**Rationale**: Fail fast on disk issues before database changes.

**Pattern**:
```python
# Write file first
write_note_file(path, content)

# Then update database
db.execute("INSERT ...")
db.commit()

# If DB fails, cleanup file
```

### 2. Best-Effort File Cleanup

**Rationale**: Database is source of truth. Missing files can be recreated or cleaned up later.

**Pattern**:
```python
try:
    path.unlink()
except OSError:
    logger.warning("Cleanup failed")
    # Don't fail - log and continue
```

### 3. Idempotent Delete

**Rationale**: DELETE operations should succeed even if already deleted.

**Pattern**:
```python
note = get_note(slug=slug)
if note is None:
    return  # Already deleted, that's fine
# ... proceed with delete
```

### 4. Lazy Content Loading

**Rationale**: list_notes() should not trigger file I/O for every note.

**Pattern**:
```python
# list_notes creates Notes without loading content
notes = [Note.from_row(row, data_dir) for row in rows]

# Content loaded on access
for note in notes:
    print(note.slug)      # Fast (metadata)
    print(note.content)   # Triggers file I/O
```

### 5. Parameterized Queries Only

**Rationale**: Prevent SQL injection.

**Pattern**:
```python
# Always use parameter binding
db.execute("SELECT * FROM notes WHERE slug = ?", (slug,))

# Never use string interpolation
db.execute(f"SELECT * FROM notes WHERE slug = '{slug}'")  # NO!
```

---

## Dependencies Reference

### From utils.py

```python
generate_slug(content, created_at) -> str
make_slug_unique(base_slug, existing_slugs) -> str
validate_slug(slug) -> bool
generate_note_path(slug, created_at, data_dir) -> Path
ensure_note_directory(note_path) -> Path
write_note_file(file_path, content) -> None
read_note_file(file_path) -> str
delete_note_file(file_path, soft=False, data_dir=None) -> None
calculate_content_hash(content) -> str
validate_note_path(file_path, data_dir) -> bool
```

### From models.py

```python
Note.from_row(row, data_dir) -> Note
Note.content -> str (property, lazy-loaded)
Note.to_dict(include_content=False) -> dict
Note.verify_integrity() -> bool
```

### From database.py

```python
get_db() -> sqlite3.Connection
db.execute(query, params) -> Cursor
db.commit() -> None
db.rollback() -> None
```

---

## Error Handling Quick Reference

### When to Raise vs Return None

| Scenario | Action |
|----------|--------|
| Note not found in get_note() | Return None |
| Note not found in update_note() | Raise NoteNotFoundError |
| Note not found in delete_note() | Return None (idempotent) |
| Empty content | Raise InvalidNoteDataError |
| File write fails | Raise NoteSyncError |
| Database fails | Raise NoteSyncError |
| Invalid parameters | Raise ValueError |

### Error Message Examples

```python
# NoteNotFoundError
raise NoteNotFoundError(
    slug,
    f"Note '{slug}' does not exist or has been deleted"
)

# InvalidNoteDataError
raise InvalidNoteDataError(
    'content',
    content,
    "Note content cannot be empty. Please provide markdown content."
)

# NoteSyncError
raise NoteSyncError(
    'create',
    f"Database insert failed: {str(e)}",
    f"Failed to create note. File written but database update failed."
)
```

---

## Security Checklist

- [ ] All SQL queries use parameterized binding (no string interpolation)
- [ ] order_by field validated against whitelist
- [ ] All file paths validated with validate_note_path()
- [ ] No symlinks followed in path operations
- [ ] Content validated (not empty)
- [ ] Slug validated before use in file paths
- [ ] No code execution from user content

---

## Final Checks

Before submitting Phase 2.1 as complete:

- [ ] All 5 functions implemented
- [ ] All 3 exceptions implemented
- [ ] Full type hints on all functions
- [ ] Comprehensive docstrings with examples
- [ ] Test coverage > 90%
- [ ] All tests passing
- [ ] Black formatting applied
- [ ] flake8 linting passes (no errors)
- [ ] Integration test passes (full CRUD cycle)
- [ ] No orphaned files in test runs
- [ ] No orphaned database records in test runs
- [ ] Error messages are clear and actionable
- [ ] Performance targets met:
  - [ ] create_note < 20ms
  - [ ] get_note < 10ms
  - [ ] list_notes < 10ms
  - [ ] update_note < 20ms
  - [ ] delete_note < 10ms

---

## Quick Command Reference

```bash
# Run tests
pytest tests/test_notes.py -v

# Check coverage
pytest tests/test_notes.py --cov=starpunk.notes --cov-report=term-missing

# Format code
black starpunk/notes.py tests/test_notes.py

# Lint code
flake8 starpunk/notes.py --max-line-length=100

# Type check (optional)
mypy starpunk/notes.py
```

---

## What's Next?

After completing Phase 2.1:

**Phase 3: Authentication**
- IndieLogin OAuth flow
- Session management
- Admin access control

**Phase 4: Web Routes**
- Admin interface (create/edit/delete notes)
- Public note views
- Template rendering

**Phase 5: Micropub**
- Micropub endpoint
- Token validation
- IndieWeb API compliance

**Phase 6: RSS Feed**
- Feed generation
- RFC-822 date formatting
- Published notes only

---

## Help & Resources

- **Full Design Doc**: [phase-2.1-notes-management.md](/home/phil/Projects/starpunk/docs/design/phase-2.1-notes-management.md)
- **Utilities Design**: [phase-1.1-core-utilities.md](/home/phil/Projects/starpunk/docs/design/phase-1.1-core-utilities.md)
- **Models Design**: [phase-1.2-data-models.md](/home/phil/Projects/starpunk/docs/design/phase-1.2-data-models.md)
- **Coding Standards**: [python-coding-standards.md](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
- **Implementation Plan**: [implementation-plan.md](/home/phil/Projects/starpunk/docs/projectplan/v1/implementation-plan.md)

Remember: "Every line of code must justify its existence. When in doubt, leave it out."