StarPunk/docs/design/v1.1.1/implementation-guide.md

# v1.1.1 "Polish" Implementation Guide

## Overview
This guide provides the development team with a structured approach to implementing v1.1.1 features. The release focuses on production readiness, performance visibility, and bug fixes without breaking changes.

## Implementation Order

The features should be implemented in this order to manage dependencies:

### Phase 1: Foundation (Day 1-2)
1. **Configuration System** (2 hours)
   - Create `starpunk/config.py` module
   - Implement configuration loading
   - Add validation and defaults
   - Update existing code to use config

2. **Structured Logging** (2 hours)
   - Create `starpunk/logging.py` module
   - Replace print statements with logger calls
   - Add request correlation IDs
   - Configure log levels

3. **Error Handling Framework** (1 hour)
   - Create `starpunk/errors.py` module
   - Define error hierarchy
   - Implement error middleware
   - Add user-friendly messages

### Phase 2: Core Improvements (Day 3-5)
4. **Database Connection Pooling** (2 hours)
   - Create `starpunk/database/pool.py`
   - Implement connection pool
   - Update database access layer
   - Add pool monitoring

5. **Fix Test Race Conditions** (1 hour)
   - Update test fixtures
   - Add database isolation
   - Fix migration locking
   - Verify test stability

6. **Unicode Slug Handling** (1 hour)
   - Update `starpunk/utils/slugify.py`
   - Add Unicode normalization
   - Handle edge cases
   - Add comprehensive tests

### Phase 3: Search Enhancements (Day 6-7)
7. **Search Configuration** (2 hours)
   - Add search configuration options
   - Implement FTS5 detection
   - Create fallback search
   - Add result highlighting

8. **Search UI Updates** (1 hour)
   - Update search templates
   - Add relevance scoring display
   - Implement highlighting CSS
   - Make search optional in UI

### Phase 4: Performance Monitoring (Day 8-10)
9. **Monitoring Infrastructure** (3 hours)
   - Create `starpunk/monitoring/` package
   - Implement metrics collector
   - Add timing instrumentation
   - Create memory monitor

10. **Performance Dashboard** (2 hours)
    - Create dashboard route
    - Design dashboard template
    - Add real-time metrics display
    - Implement data aggregation

### Phase 5: Production Readiness (Day 11-12)
11. **Health Check Enhancements** (1 hour)
    - Update health endpoints
    - Add component checks
    - Implement readiness probe
    - Add detailed status

12. **Session Management** (1 hour)
    - Fix session timeout
    - Add cleanup thread
    - Implement extension logic
    - Update session handling

13. **RSS Optimization** (1 hour)
    - Implement streaming RSS
    - Add feed caching
    - Optimize memory usage
    - Add configuration limits

### Phase 6: Testing & Documentation (Day 13-14)
14. **Testing** (2 hours)
    - Run full test suite
    - Performance benchmarks
    - Load testing
    - Security review

15. **Documentation** (1 hour)
    - Update deployment guide
    - Document configuration
    - Update API documentation
    - Create upgrade guide

## Key Files to Modify

### New Files to Create
```
starpunk/
├── config.py                    # Configuration management
├── errors.py                    # Error handling framework
├── logging.py                   # Logging setup
├── database/
│   └── pool.py                  # Connection pooling
├── monitoring/
│   ├── __init__.py
│   ├── collector.py             # Metrics collection
│   ├── db_monitor.py            # Database monitoring
│   ├── memory.py                # Memory tracking
│   └── http.py                  # HTTP monitoring
├── testing/
│   ├── fixtures.py              # Test fixtures
│   ├── stability.py             # Stability helpers
│   └── unicode.py               # Unicode test suite
└── templates/admin/
    ├── performance.html         # Performance dashboard
    └── performance_disabled.html
```

### Files to Update
```
starpunk/
├── __init__.py                  # Add version 1.1.1
├── app.py                       # Add middleware, routes
├── auth/
│   └── session.py               # Session management fixes
├── utils/
│   └── slugify.py               # Unicode handling
├── search/
│   ├── engine.py                # FTS5 detection, fallback
│   └── highlighting.py          # Result highlighting
├── feeds/
│   └── rss.py                   # Memory optimization
├── web/
│   └── routes.py                # Health checks, dashboard
└── templates/
    ├── search.html              # Search UI updates
    └── base.html                # Conditional search UI
```

## Configuration Variables

All new configuration uses environment variables with `STARPUNK_` prefix:

```bash
# Search Configuration
STARPUNK_SEARCH_ENABLED=true
STARPUNK_SEARCH_TITLE_LENGTH=100
STARPUNK_SEARCH_HIGHLIGHT_CLASS=highlight
STARPUNK_SEARCH_MIN_SCORE=0.0

# Performance Monitoring
STARPUNK_PERF_MONITORING_ENABLED=false
STARPUNK_PERF_SLOW_QUERY_THRESHOLD=1.0
STARPUNK_PERF_LOG_QUERIES=false
STARPUNK_PERF_MEMORY_TRACKING=false

# Database Configuration
STARPUNK_DB_CONNECTION_POOL_SIZE=5
STARPUNK_DB_CONNECTION_TIMEOUT=10.0
STARPUNK_DB_WAL_MODE=true
STARPUNK_DB_BUSY_TIMEOUT=5000

# Logging Configuration
STARPUNK_LOG_LEVEL=INFO
STARPUNK_LOG_FORMAT=json

# Production Configuration
STARPUNK_SESSION_TIMEOUT=86400
STARPUNK_HEALTH_CHECK_DETAILED=false
STARPUNK_ERROR_DETAILS_IN_RESPONSE=false
```

## Testing Requirements

### Unit Test Coverage
- Configuration loading and validation
- Error handling for all error types
- Slug generation with Unicode inputs
- Connection pool operations
- Session timeout logic
- Search with/without FTS5

### Integration Test Coverage
- End-to-end search functionality
- Performance dashboard access
- Health check endpoints
- RSS feed generation
- Session management flow

### Performance Tests
```python
# Required performance benchmarks
def test_search_performance():
    """Search should complete in <500ms"""

def test_rss_memory_usage():
    """RSS should use <10MB for 10k notes"""

def test_monitoring_overhead():
    """Monitoring should add <1% overhead"""

def test_connection_pool_concurrency():
    """Pool should handle 20 concurrent requests"""
```

## Database Migrations

### New Migration: v1.1.1_sessions.sql
```sql
-- Add session management improvements
CREATE TABLE IF NOT EXISTS sessions_new (
    id TEXT PRIMARY KEY,
    user_id TEXT NOT NULL,
    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP NOT NULL,
    last_activity TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    remember BOOLEAN DEFAULT FALSE
);

-- Migrate existing sessions if any
INSERT INTO sessions_new (id, user_id, created_at, expires_at)
SELECT id, user_id, created_at,
       datetime(created_at, '+1 day') as expires_at
FROM sessions WHERE EXISTS (SELECT 1 FROM sessions LIMIT 1);

-- Swap tables
DROP TABLE IF EXISTS sessions;
ALTER TABLE sessions_new RENAME TO sessions;

-- Add index for cleanup
CREATE INDEX idx_sessions_expires ON sessions(expires_at);
CREATE INDEX idx_sessions_user ON sessions(user_id);
```

## Backward Compatibility Checklist

Ensure NO breaking changes:

- [ ] All configuration has sensible defaults
- [ ] Existing deployments work without changes
- [ ] Database migrations are non-destructive
- [ ] API responses maintain same format
- [ ] URL structure unchanged
- [ ] RSS/ATOM feeds compatible
- [ ] IndieAuth flow unmodified
- [ ] Micropub endpoint unchanged

## Deployment Validation

After implementation, verify:

1. **Fresh Install**
   ```bash
   # Clean install works
   pip install starpunk==1.1.1
   starpunk init
   starpunk serve
   ```

2. **Upgrade Path**
   ```bash
   # Upgrade from 1.1.0 works
   pip install --upgrade starpunk==1.1.1
   starpunk migrate
   starpunk serve
   ```

3. **Configuration**
   ```bash
   # All config options work
   export STARPUNK_SEARCH_ENABLED=false
   starpunk serve  # Search should be disabled
   ```

4. **Performance**
   ```bash
   # Run performance tests
   pytest tests/performance/
   ```

## Common Pitfalls to Avoid

1. **Don't Break Existing Features**
   - Test with existing data
   - Verify Micropub compatibility
   - Check RSS feed format

2. **Handle Missing FTS5 Gracefully**
   - Don't crash if FTS5 unavailable
   - Provide clear warnings
   - Fallback must work correctly

3. **Maintain Thread Safety**
   - Connection pool must be thread-safe
   - Metrics collection must be thread-safe
   - Use proper locking

4. **Avoid Memory Leaks**
   - Circular buffer for metrics
   - Stream RSS generation
   - Clean up expired sessions

5. **Configuration Validation**
   - Validate all config at startup
   - Use sensible defaults
   - Log configuration errors clearly

## Success Criteria

The implementation is complete when:

1. All tests pass (including new ones)
2. Performance benchmarks met
3. No breaking changes verified
4. Documentation updated
5. Changelog updated to v1.1.1
6. Version number updated
7. All features configurable
8. Production deployment tested

## Support Resources

- Architecture Decisions: `/docs/decisions/ADR-052-055`
- Feature Specifications: `/docs/design/v1.1.1/`
- Test Suite: `/tests/`
- Original Requirements: User request for v1.1.1

## Timeline

- **Total Effort**: 12-18 hours
- **Calendar Time**: 2 weeks
- **Daily Commitment**: 1-2 hours
- **Buffer**: 20% for unexpected issues

## Risk Mitigation

| Risk | Mitigation |
|------|------------|
| FTS5 compatibility issues | Comprehensive fallback, clear docs |
| Performance regression | Benchmark before/after each change |
| Test instability | Fix race conditions first |
| Memory issues | Profile RSS generation, limit buffers |
| Configuration complexity | Sensible defaults, validation |

## Questions to Answer Before Starting

1. Is the current test suite passing reliably?
2. Do we have performance baselines measured?
3. Is the deployment environment documented?
4. Are there any pending v1.1.0 issues to address?
5. Is the version control branching strategy clear?

## Post-Implementation Checklist

- [ ] All features implemented
- [ ] Tests written and passing
- [ ] Performance validated
- [ ] Documentation complete
- [ ] Changelog updated
- [ ] Version bumped to 1.1.1
- [ ] Migration tested
- [ ] Production deployment successful
- [ ] Announcement prepared

---

This guide should be treated as a living document. Update it as implementation proceeds and lessons are learned.