StarPunk/docs/design/v1.1.1/v1.1.1-architecture-overview.md

# v1.1.1 "Polish" Architecture Overview

## Executive Summary

StarPunk v1.1.1 introduces production-focused improvements without changing the core architecture. The release adds configurability, observability, and robustness while maintaining full backward compatibility.

## Architectural Principles

### Core Principles (Unchanged)
1. **Simplicity First**: Every feature must justify its complexity
2. **Standards Compliance**: Full IndieWeb specification adherence
3. **No External Dependencies**: Use Python stdlib where possible
4. **Progressive Enhancement**: Core functionality without JavaScript
5. **Data Portability**: User data remains exportable

### v1.1.1 Additions
6. **Observable by Default**: Production visibility built-in
7. **Graceful Degradation**: Features degrade rather than fail
8. **Configuration over Code**: Behavior adjustable without changes
9. **Zero Breaking Changes**: Perfect backward compatibility

## System Architecture

### High-Level Component View

```
┌─────────────────────────────────────────────────────────┐
│                     StarPunk v1.1.1                       │
├─────────────────────────────────────────────────────────┤
│                    Configuration Layer                     │
│                  (Environment Variables)                   │
├─────────────────────────────────────────────────────────┤
│                    Application Layer                       │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐│
│  │   Auth   │  │ Micropub │  │  Search  │  │   Web    ││
│  └──────────┘  └──────────┘  └──────────┘  └──────────┘│
├─────────────────────────────────────────────────────────┤
│                 Monitoring & Logging Layer                │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │ Performance  │  │  Structured  │  │    Error     │  │
│  │  Monitoring  │  │   Logging    │  │   Handling   │  │
│  └──────────────┘  └──────────────┘  └──────────────┘  │
├─────────────────────────────────────────────────────────┤
│                     Data Access Layer                     │
│  ┌──────────────────────┐  ┌──────────────────────┐    │
│  │  Connection Pool     │  │   Search Engine      │    │
│  │  ┌────┐...┌────┐    │  │  ┌──────┐┌────────┐ │    │
│  │  │Conn│   │Conn│    │  │  │ FTS5 ││Fallback│ │    │
│  │  └────┘   └────┘    │  │  └──────┘└────────┘ │    │
│  └──────────────────────┘  └──────────────────────┘    │
├─────────────────────────────────────────────────────────┤
│                    SQLite Database                        │
│                    (WAL mode, FTS5)                       │
└─────────────────────────────────────────────────────────┘
```

### Request Flow

```
HTTP Request
     ↓
[Logging Middleware: Start Request ID]
     ↓
[Performance Middleware: Start Timer]
     ↓
[Session Middleware: Validate/Extend]
     ↓
[Error Handling Wrapper]
     ↓
Route Handler
     ├→ [Database: Connection Pool]
     ├→ [Search: FTS5 or Fallback]
     ├→ [Monitoring: Record Metrics]
     └→ [Logging: Structured Output]
     ↓
Response Generation
     ↓
[Performance Middleware: Stop Timer, Record]
     ↓
[Logging Middleware: Log Request]
     ↓
HTTP Response
```

## New Components

### 1. Configuration System

**Location**: `starpunk/config.py`

**Responsibilities**:
- Load environment variables
- Provide type-safe access
- Define defaults
- Validate configuration

**Design Pattern**: Singleton with lazy loading

```python
Configuration
├── get_bool(key, default)
├── get_int(key, default)
├── get_float(key, default)
└── get_str(key, default)
```

### 2. Performance Monitoring

**Location**: `starpunk/monitoring/`

**Components**:
- `collector.py`: Metrics collection and storage
- `db_monitor.py`: Database performance tracking
- `memory.py`: Memory usage monitoring
- `http.py`: HTTP request tracking

**Design Pattern**: Observer with circular buffer

```python
MetricsCollector
├── CircularBuffer (1000 metrics)
├── SlowQueryLog (100 queries)
├── MemoryTracker (background thread)
└── Dashboard (read-only view)
```

### 3. Structured Logging

**Location**: `starpunk/logging.py`

**Features**:
- JSON formatting in production
- Human-readable in development
- Request correlation IDs
- Log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)

**Design Pattern**: Decorator with context injection

### 4. Error Handling

**Location**: `starpunk/errors.py`

**Hierarchy**:
```
StarPunkError (Base)
├── ValidationError (400)
├── AuthenticationError (401)
├── NotFoundError (404)
├── DatabaseError (500)
├── ConfigurationError (500)
└── TransientError (503)
```

**Design Pattern**: Exception hierarchy with middleware

### 5. Connection Pool

**Location**: `starpunk/database/pool.py`

**Features**:
- Thread-safe pool management
- Configurable pool size
- Connection health checks
- Usage statistics

**Design Pattern**: Object pool with semaphore

## Data Flow Improvements

### Search Data Flow

```
Search Request
     ↓
Check Config: SEARCH_ENABLED?
     ├─No→ Return "Search Disabled"
     └─Yes↓
        Check FTS5 Available?
           ├─Yes→ FTS5 Search Engine
           │      ├→ Execute FTS5 Query
           │      ├→ Calculate Relevance
           │      └→ Highlight Terms
           └─No→ Fallback Search Engine
                 ├→ Execute LIKE Query
                 ├→ No Relevance Score
                 └→ Basic Highlighting
```

### Error Flow

```
Exception Occurs
     ↓
Catch in Middleware
     ↓
Categorize Error
     ├→ User Error: Log INFO, Return Helpful Message
     ├→ System Error: Log ERROR, Return Generic Message
     ├→ Transient Error: Retry with Backoff
     └→ Config Error: Fail Fast at Startup
```

## Database Schema Changes

### Sessions Table Enhancement
```sql
CREATE TABLE sessions (
    id TEXT PRIMARY KEY,
    user_id TEXT NOT NULL,
    created_at TIMESTAMP NOT NULL,
    expires_at TIMESTAMP NOT NULL,
    last_activity TIMESTAMP,
    remember BOOLEAN DEFAULT FALSE,
    INDEX idx_sessions_expires (expires_at),
    INDEX idx_sessions_user (user_id)
);
```

## Performance Characteristics

### Metrics
| Operation | v1.1.0 | v1.1.1 Target | v1.1.1 Actual |
|-----------|---------|---------------|---------------|
| Request Latency | ~50ms | <50ms | TBD |
| Search Response | ~100ms | <100ms (FTS5) <500ms (fallback) | TBD |
| RSS Generation | ~200ms | <100ms | TBD |
| Memory per Request | ~2MB | <1MB | TBD |
| Monitoring Overhead | N/A | <1% | TBD |

### Scalability
- Connection pool: Handles 20+ concurrent requests
- Metrics buffer: Fixed 1MB memory overhead
- RSS streaming: O(1) memory complexity
- Session cleanup: Automatic background process

## Security Enhancements

### Input Validation
- Unicode normalization in slugs
- XSS prevention in search highlighting
- SQL injection prevention via parameterization

### Session Security
- Configurable timeout
- HTTP-only cookies
- Secure flag in production
- CSRF protection maintained

### Error Information
- Sensitive data never in errors
- Stack traces only in debug mode
- Rate limiting on error endpoints

## Deployment Architecture

### Environment Variables
```
Production Server
├── STARPUNK_* Configuration
├── Process Manager (systemd/supervisor)
├── Reverse Proxy (nginx/caddy)
└── SQLite Database File
```

### Health Monitoring
```
Load Balancer
├→ /health (liveness)
└→ /health/ready (readiness)
```

## Testing Architecture

### Test Isolation
```
Test Suite
├── Isolated Database per Test
├── Mocked Time/Random
├── Controlled Configuration
└── Deterministic Execution
```

### Performance Testing
```
Benchmarks
├── Baseline Measurements
├── With Monitoring Enabled
├── Memory Profiling
└── Load Testing
```

## Migration Path

### From v1.1.0 to v1.1.1
1. Install new version
2. Run migrations (automatic)
3. Configure as needed (optional)
4. Restart service

### Rollback Plan
1. Restore previous version
2. No database changes to revert
3. Remove new config vars (optional)

## Observability

### Metrics Available
- Request count and latency
- Database query performance
- Memory usage over time
- Error rates by type
- Session statistics

### Logging Output
```json
{
  "timestamp": "2025-11-25T10:00:00Z",
  "level": "INFO",
  "logger": "starpunk.micropub",
  "message": "Note created",
  "request_id": "abc123",
  "user": "alice@example.com",
  "duration_ms": 45
}
```

## Future Considerations

### Extensibility Points
1. **Monitoring Plugins**: Hook for external monitoring
2. **Search Providers**: Interface for alternative search
3. **Cache Layer**: Ready for Redis/Memcached
4. **Queue System**: Prepared for async operations

### Technical Debt Addressed
1. ✅ Test race conditions fixed
2. ✅ Unicode handling improved
3. ✅ Memory usage optimized
4. ✅ Error handling standardized
5. ✅ Configuration centralized

## Design Decisions Summary

| Decision | Rationale | Alternative Considered |
|----------|-----------|----------------------|
| Environment variables for config | 12-factor app, container-friendly | Config files |
| Built-in monitoring | Zero dependencies, privacy | External APM |
| Connection pooling | Reduce latency, handle concurrency | Single connection |
| Structured logging | Production parsing, debugging | Plain text logs |
| Graceful degradation | Reliability, user experience | Fail fast |

## Risks and Mitigations

| Risk | Impact | Mitigation |
|------|--------|------------|
| FTS5 not available | Slow search | Automatic fallback to LIKE |
| Memory leak in monitoring | OOM | Circular buffer with fixed size |
| Configuration complexity | User confusion | Sensible defaults, clear docs |
| Performance regression | Slow responses | Comprehensive benchmarking |

## Success Metrics

1. **Reliability**: 99.9% uptime capability
2. **Performance**: <1% overhead from monitoring
3. **Usability**: Zero configuration required to upgrade
4. **Observability**: Full visibility into production
5. **Compatibility**: 100% backward compatible

## Documentation References

- [Configuration System](/home/phil/Projects/starpunk/docs/decisions/ADR-052-configuration-system-architecture.md)
- [Performance Monitoring](/home/phil/Projects/starpunk/docs/decisions/ADR-053-performance-monitoring-strategy.md)
- [Structured Logging](/home/phil/Projects/starpunk/docs/decisions/ADR-054-structured-logging-architecture.md)
- [Error Handling](/home/phil/Projects/starpunk/docs/decisions/ADR-055-error-handling-philosophy.md)
- [Implementation Guide](/home/phil/Projects/starpunk/docs/design/v1.1.1/implementation-guide.md)

---

This architecture maintains StarPunk's commitment to simplicity while adding production-grade capabilities. Every addition has been carefully considered to ensure it provides value without unnecessary complexity.