feat(tags): Add database schema and tags module (v1.3.0 Phase 1)

Implements tag/category system backend following microformats2 p-category specification. Database changes: - Migration 008: Add tags and note_tags tables - Normalized tag storage (case-insensitive lookup, display name preserved) - Indexes for performance New module: - starpunk/tags.py: Tag management functions - normalize_tag: Normalize tag strings - get_or_create_tag: Get or create tag records - add_tags_to_note: Associate tags with notes (replaces existing) - get_note_tags: Retrieve note tags (alphabetically ordered) - get_tag_by_name: Lookup tag by normalized name - get_notes_by_tag: Get all notes with specific tag - parse_tag_input: Parse comma-separated tag input Model updates: - Note.tags property (lazy-loaded, prefer pre-loading in routes) - Note.to_dict() add include_tags parameter CRUD updates: - create_note() accepts tags parameter - update_note() accepts tags parameter (None = no change, [] = remove all) Micropub integration: - Pass tags to create_note() (tags already extracted by extract_tags()) - Return tags in q=source response Per design doc: docs/design/v1.3.0/microformats-tags-design.md Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 11:24:23 -07:00
parent 927db4aea0
commit f10d0679da
188 changed files with 601 additions and 945 deletions
--- a/docs/design/v1.1.1/v1.1.1-architecture-overview.md
+++ b/docs/design/v1.1.1/v1.1.1-architecture-overview.md
@@ -0,0 +1,379 @@
+# v1.1.1 "Polish" Architecture Overview
+
+## Executive Summary
+
+StarPunk v1.1.1 introduces production-focused improvements without changing the core architecture. The release adds configurability, observability, and robustness while maintaining full backward compatibility.
+
+## Architectural Principles
+
+### Core Principles (Unchanged)
+1. **Simplicity First**: Every feature must justify its complexity
+2. **Standards Compliance**: Full IndieWeb specification adherence
+3. **No External Dependencies**: Use Python stdlib where possible
+4. **Progressive Enhancement**: Core functionality without JavaScript
+5. **Data Portability**: User data remains exportable
+
+### v1.1.1 Additions
+6. **Observable by Default**: Production visibility built-in
+7. **Graceful Degradation**: Features degrade rather than fail
+8. **Configuration over Code**: Behavior adjustable without changes
+9. **Zero Breaking Changes**: Perfect backward compatibility
+
+## System Architecture
+
+### High-Level Component View
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                     StarPunk v1.1.1                       │
+├─────────────────────────────────────────────────────────┤
+│                    Configuration Layer                     │
+│                  (Environment Variables)                   │
+├─────────────────────────────────────────────────────────┤
+│                    Application Layer                       │
+│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐│
+│  │   Auth   │  │ Micropub │  │  Search  │  │   Web    ││
+│  └──────────┘  └──────────┘  └──────────┘  └──────────┘│
+├─────────────────────────────────────────────────────────┤
+│                 Monitoring & Logging Layer                │
+│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
+│  │ Performance  │  │  Structured  │  │    Error     │  │
+│  │  Monitoring  │  │   Logging    │  │   Handling   │  │
+│  └──────────────┘  └──────────────┘  └──────────────┘  │
+├─────────────────────────────────────────────────────────┤
+│                     Data Access Layer                     │
+│  ┌──────────────────────┐  ┌──────────────────────┐    │
+│  │  Connection Pool     │  │   Search Engine      │    │
+│  │  ┌────┐...┌────┐    │  │  ┌──────┐┌────────┐ │    │
+│  │  │Conn│   │Conn│    │  │  │ FTS5 ││Fallback│ │    │
+│  │  └────┘   └────┘    │  │  └──────┘└────────┘ │    │
+│  └──────────────────────┘  └──────────────────────┘    │
+├─────────────────────────────────────────────────────────┤
+│                    SQLite Database                        │
+│                    (WAL mode, FTS5)                       │
+└─────────────────────────────────────────────────────────┘
+```
+
+### Request Flow
+
+```
+HTTP Request
+     ↓
+[Logging Middleware: Start Request ID]
+     ↓
+[Performance Middleware: Start Timer]
+     ↓
+[Session Middleware: Validate/Extend]
+     ↓
+[Error Handling Wrapper]
+     ↓
+Route Handler
+     ├→ [Database: Connection Pool]
+     ├→ [Search: FTS5 or Fallback]
+     ├→ [Monitoring: Record Metrics]
+     └→ [Logging: Structured Output]
+     ↓
+Response Generation
+     ↓
+[Performance Middleware: Stop Timer, Record]
+     ↓
+[Logging Middleware: Log Request]
+     ↓
+HTTP Response
+```
+
+## New Components
+
+### 1. Configuration System
+
+**Location**: `starpunk/config.py`
+
+**Responsibilities**:
+- Load environment variables
+- Provide type-safe access
+- Define defaults
+- Validate configuration
+
+**Design Pattern**: Singleton with lazy loading
+
+```python
+Configuration
+├── get_bool(key, default)
+├── get_int(key, default)
+├── get_float(key, default)
+└── get_str(key, default)
+```
+
+### 2. Performance Monitoring
+
+**Location**: `starpunk/monitoring/`
+
+**Components**:
+- `collector.py`: Metrics collection and storage
+- `db_monitor.py`: Database performance tracking
+- `memory.py`: Memory usage monitoring
+- `http.py`: HTTP request tracking
+
+**Design Pattern**: Observer with circular buffer
+
+```python
+MetricsCollector
+├── CircularBuffer (1000 metrics)
+├── SlowQueryLog (100 queries)
+├── MemoryTracker (background thread)
+└── Dashboard (read-only view)
+```
+
+### 3. Structured Logging
+
+**Location**: `starpunk/logging.py`
+
+**Features**:
+- JSON formatting in production
+- Human-readable in development
+- Request correlation IDs
+- Log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+
+**Design Pattern**: Decorator with context injection
+
+### 4. Error Handling
+
+**Location**: `starpunk/errors.py`
+
+**Hierarchy**:
+```
+StarPunkError (Base)
+├── ValidationError (400)
+├── AuthenticationError (401)
+├── NotFoundError (404)
+├── DatabaseError (500)
+├── ConfigurationError (500)
+└── TransientError (503)
+```
+
+**Design Pattern**: Exception hierarchy with middleware
+
+### 5. Connection Pool
+
+**Location**: `starpunk/database/pool.py`
+
+**Features**:
+- Thread-safe pool management
+- Configurable pool size
+- Connection health checks
+- Usage statistics
+
+**Design Pattern**: Object pool with semaphore
+
+## Data Flow Improvements
+
+### Search Data Flow
+
+```
+Search Request
+     ↓
+Check Config: SEARCH_ENABLED?
+     ├─No→ Return "Search Disabled"
+     └─Yes↓
+        Check FTS5 Available?
+           ├─Yes→ FTS5 Search Engine
+           │      ├→ Execute FTS5 Query
+           │      ├→ Calculate Relevance
+           │      └→ Highlight Terms
+           └─No→ Fallback Search Engine
+                 ├→ Execute LIKE Query
+                 ├→ No Relevance Score
+                 └→ Basic Highlighting
+```
+
+### Error Flow
+
+```
+Exception Occurs
+     ↓
+Catch in Middleware
+     ↓
+Categorize Error
+     ├→ User Error: Log INFO, Return Helpful Message
+     ├→ System Error: Log ERROR, Return Generic Message
+     ├→ Transient Error: Retry with Backoff
+     └→ Config Error: Fail Fast at Startup
+```
+
+## Database Schema Changes
+
+### Sessions Table Enhancement
+```sql
+CREATE TABLE sessions (
+    id TEXT PRIMARY KEY,
+    user_id TEXT NOT NULL,
+    created_at TIMESTAMP NOT NULL,
+    expires_at TIMESTAMP NOT NULL,
+    last_activity TIMESTAMP,
+    remember BOOLEAN DEFAULT FALSE,
+    INDEX idx_sessions_expires (expires_at),
+    INDEX idx_sessions_user (user_id)
+);
+```
+
+## Performance Characteristics
+
+### Metrics
+| Operation | v1.1.0 | v1.1.1 Target | v1.1.1 Actual |
+|-----------|---------|---------------|---------------|
+| Request Latency | ~50ms | <50ms | TBD |
+| Search Response | ~100ms | <100ms (FTS5) <500ms (fallback) | TBD |
+| RSS Generation | ~200ms | <100ms | TBD |
+| Memory per Request | ~2MB | <1MB | TBD |
+| Monitoring Overhead | N/A | <1% | TBD |
+
+### Scalability
+- Connection pool: Handles 20+ concurrent requests
+- Metrics buffer: Fixed 1MB memory overhead
+- RSS streaming: O(1) memory complexity
+- Session cleanup: Automatic background process
+
+## Security Enhancements
+
+### Input Validation
+- Unicode normalization in slugs
+- XSS prevention in search highlighting
+- SQL injection prevention via parameterization
+
+### Session Security
+- Configurable timeout
+- HTTP-only cookies
+- Secure flag in production
+- CSRF protection maintained
+
+### Error Information
+- Sensitive data never in errors
+- Stack traces only in debug mode
+- Rate limiting on error endpoints
+
+## Deployment Architecture
+
+### Environment Variables
+```
+Production Server
+├── STARPUNK_* Configuration
+├── Process Manager (systemd/supervisor)
+├── Reverse Proxy (nginx/caddy)
+└── SQLite Database File
+```
+
+### Health Monitoring
+```
+Load Balancer
+├→ /health (liveness)
+└→ /health/ready (readiness)
+```
+
+## Testing Architecture
+
+### Test Isolation
+```
+Test Suite
+├── Isolated Database per Test
+├── Mocked Time/Random
+├── Controlled Configuration
+└── Deterministic Execution
+```
+
+### Performance Testing
+```
+Benchmarks
+├── Baseline Measurements
+├── With Monitoring Enabled
+├── Memory Profiling
+└── Load Testing
+```
+
+## Migration Path
+
+### From v1.1.0 to v1.1.1
+1. Install new version
+2. Run migrations (automatic)
+3. Configure as needed (optional)
+4. Restart service
+
+### Rollback Plan
+1. Restore previous version
+2. No database changes to revert
+3. Remove new config vars (optional)
+
+## Observability
+
+### Metrics Available
+- Request count and latency
+- Database query performance
+- Memory usage over time
+- Error rates by type
+- Session statistics
+
+### Logging Output
+```json
+{
+  "timestamp": "2025-11-25T10:00:00Z",
+  "level": "INFO",
+  "logger": "starpunk.micropub",
+  "message": "Note created",
+  "request_id": "abc123",
+  "user": "alice@example.com",
+  "duration_ms": 45
+}
+```
+
+## Future Considerations
+
+### Extensibility Points
+1. **Monitoring Plugins**: Hook for external monitoring
+2. **Search Providers**: Interface for alternative search
+3. **Cache Layer**: Ready for Redis/Memcached
+4. **Queue System**: Prepared for async operations
+
+### Technical Debt Addressed
+1. ✅ Test race conditions fixed
+2. ✅ Unicode handling improved
+3. ✅ Memory usage optimized
+4. ✅ Error handling standardized
+5. ✅ Configuration centralized
+
+## Design Decisions Summary
+
+| Decision | Rationale | Alternative Considered |
+|----------|-----------|----------------------|
+| Environment variables for config | 12-factor app, container-friendly | Config files |
+| Built-in monitoring | Zero dependencies, privacy | External APM |
+| Connection pooling | Reduce latency, handle concurrency | Single connection |
+| Structured logging | Production parsing, debugging | Plain text logs |
+| Graceful degradation | Reliability, user experience | Fail fast |
+
+## Risks and Mitigations
+
+| Risk | Impact | Mitigation |
+|------|--------|------------|
+| FTS5 not available | Slow search | Automatic fallback to LIKE |
+| Memory leak in monitoring | OOM | Circular buffer with fixed size |
+| Configuration complexity | User confusion | Sensible defaults, clear docs |
+| Performance regression | Slow responses | Comprehensive benchmarking |
+
+## Success Metrics
+
+1. **Reliability**: 99.9% uptime capability
+2. **Performance**: <1% overhead from monitoring
+3. **Usability**: Zero configuration required to upgrade
+4. **Observability**: Full visibility into production
+5. **Compatibility**: 100% backward compatible
+
+## Documentation References
+
+- [Configuration System](/home/phil/Projects/starpunk/docs/decisions/ADR-052-configuration-system-architecture.md)
+- [Performance Monitoring](/home/phil/Projects/starpunk/docs/decisions/ADR-053-performance-monitoring-strategy.md)
+- [Structured Logging](/home/phil/Projects/starpunk/docs/decisions/ADR-054-structured-logging-architecture.md)
+- [Error Handling](/home/phil/Projects/starpunk/docs/decisions/ADR-055-error-handling-philosophy.md)
+- [Implementation Guide](/home/phil/Projects/starpunk/docs/design/v1.1.1/implementation-guide.md)
+
+---
+
+This architecture maintains StarPunk's commitment to simplicity while adding production-grade capabilities. Every addition has been carefully considered to ensure it provides value without unnecessary complexity.