phil/sneakyklaus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add Participant and MagicToken models with automatic migrations

Browse Source 
Implements Phase 2 infrastructure for participant registration and authentication:

Database Models:
- Add Participant model with exchange scoping and soft deletes
- Add MagicToken model for passwordless authentication
- Add participants relationship to Exchange model
- Include proper indexes and foreign key constraints

Migration Infrastructure:
- Generate Alembic migration for new models
- Create entrypoint.sh script for automatic migrations on container startup
- Update Containerfile to use entrypoint script and include uv binary
- Remove db.create_all() in favor of migration-based schema management

This establishes the foundation for implementing stories 4.1-4.3, 5.1-5.3, and 10.1.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Phil Skentelbery
2025-12-22 16:23:47 -07:00
parent 5201b2f036
commit eaafa78cf3
22 changed files with 10459 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

395
docs/designs/v0.2.0/automatic-migration-implementation.md Normal file
View File

@@ -0,0 +1,395 @@
# Automatic Migration Implementation Guide
**Version**: 0.2.0
**Date**: 2025-12-22
**Status**: Implementation Specification
## Overview
This document specifies how to implement automatic database migrations for containerized deployments of Sneaky Klaus. When self-hosted users pull a new container image with schema changes, migrations must be applied automatically without manual intervention.
## Context
- **Current State**: Container runs gunicorn directly; no automatic migrations
- **Problem**: Users must manually run `alembic upgrade head` after pulling new images
- **Solution**: Container entrypoint script applies migrations before starting app
- **Reference**: ADR-0005 "Automatic Migrations for Self-Hosted Deployments"
## Implementation Steps
### Step 1: Create Entrypoint Script
Create `/home/phil/Projects/sneaky-klaus/entrypoint.sh`:
```bash
#!/bin/bash
set -e # Exit on any error
echo "Running database migrations..."
if uv run alembic upgrade head; then
echo "Database migrations completed successfully"
else
echo "ERROR: Database migration failed!"
echo "Please check the logs above for details."
exit 1
fi
echo "Starting application server..."
exec gunicorn --bind 0.0.0.0:8000 --workers 2 --threads 4 main:app
```
**Key Points**:
- `set -e`: Ensures script exits immediately if any command fails
- Migrations run before gunicorn starts
- `exec` replaces the shell process with gunicorn (PID 1 becomes gunicorn for proper signal handling)
- Clear logging for debugging
- Exit code 1 on migration failure prevents container from starting
### Step 2: Update Containerfile
Modify `/home/phil/Projects/sneaky-klaus/Containerfile`:
**Add after line 29 (after copying application code)**:
```dockerfile
# Copy entrypoint script
COPY entrypoint.sh ./
RUN chmod +x entrypoint.sh
```
**Replace line 51 (CMD line)**:
From:
```dockerfile
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "2", "--threads", "4", "main:app"]
```
To:
```dockerfile
CMD ["./entrypoint.sh"]
```
**Full context** (lines 27-52 after changes):
```dockerfile
# Copy application code
COPY src/ ./src/
COPY migrations/ ./migrations/
COPY alembic.ini main.py ./
# Copy entrypoint script
COPY entrypoint.sh ./
RUN chmod +x entrypoint.sh
# Set environment variables
ENV PATH="/app/.venv/bin:$PATH"
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENV FLASK_ENV=production
# Create data directory and set permissions
RUN mkdir -p /app/data && chown -R appuser:appuser /app
# Switch to non-root user
USER appuser
# Expose port
EXPOSE 8000
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1
# Run with entrypoint script
CMD ["./entrypoint.sh"]
```
### Step 3: Remove db.create_all() from Application
Modify `/home/phil/Projects/sneaky-klaus/src/app.py`:
**Replace lines 74-77**:
From:
```python
# Import models to ensure they're registered with SQLAlchemy
with app.app_context():
from src.models import Admin, Exchange, RateLimit # noqa: F401
db.create_all()
```
To:
```python
# Import models to ensure they're registered with SQLAlchemy
with app.app_context():
from src.models import Admin, Exchange, RateLimit # noqa: F401
# Schema managed by Alembic migrations (applied via entrypoint script)
```
**Rationale**:
- Migrations are now handled by entrypoint script
- `db.create_all()` conflicts with migration-based schema management
- `db.create_all()` cannot handle schema changes (only creates missing tables)
- ADR-0005 explicitly requires migrations for production
### Step 4: Copy uv to Runtime Stage
The entrypoint script uses `uv run alembic`, so we need `uv` available in the runtime stage.
Modify `/home/phil/Projects/sneaky-klaus/Containerfile`:
**Add after line 23 (after copying .venv from builder)**:
```dockerfile
# Copy uv for running alembic in entrypoint
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
```
**Full context** (lines 15-30 after change):
```dockerfile
# Runtime stage - minimal image
FROM python:3.12-slim AS runtime
WORKDIR /app
# Create non-root user
RUN useradd --create-home --shell /bin/bash appuser
# Copy virtual environment from builder
COPY --from=builder /app/.venv /app/.venv
# Copy uv for running alembic in entrypoint
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
# Copy application code
COPY src/ ./src/
COPY migrations/ ./migrations/
COPY alembic.ini main.py ./
```
## Testing
### Test 1: First-Run Scenario
Verify migrations run when starting with no database:
```bash
# Build new image
podman build -t sneaky-klaus:test .
# Remove any existing data volume
podman volume rm sneaky-klaus-data || true
# Start container
podman run -p 8000:8000 \
-e SECRET_KEY=test-secret-key \
-e APP_URL=http://localhost:8000 \
-v sneaky-klaus-data:/app/data \
--name sneaky-klaus-test \
sneaky-klaus:test
# Check logs - should see migration messages
podman logs sneaky-klaus-test
# Expected output:
# Running database migrations...
# INFO [alembic.runtime.migration] Context impl SQLiteImpl.
# INFO [alembic.runtime.migration] Will assume non-transactional DDL.
# INFO [alembic.runtime.migration] Running upgrade -> eeff6e1a89cd, initial schema with Admin and Exchange models
# Database migrations completed successfully
# Starting application server...
```
### Test 2: Update Scenario
Verify incremental migrations run when updating:
```bash
# With container already running from Test 1, rebuild with new migration
# (After developer adds new migration in Phase 2)
podman build -t sneaky-klaus:test .
# Stop and remove old container
podman stop sneaky-klaus-test
podman rm sneaky-klaus-test
# Start new container with same data volume
podman run -p 8000:8000 \
-e SECRET_KEY=test-secret-key \
-e APP_URL=http://localhost:8000 \
-v sneaky-klaus-data:/app/data \
--name sneaky-klaus-test \
sneaky-klaus:test
# Check logs - should see new migration applied
podman logs sneaky-klaus-test
# Expected output:
# Running database migrations...
# INFO [alembic.runtime.migration] Context impl SQLiteImpl.
# INFO [alembic.runtime.migration] Will assume non-transactional DDL.
# INFO [alembic.runtime.migration] Running upgrade eeff6e1a89cd -> abc123def456, Add Participant and MagicToken models
# Database migrations completed successfully
# Starting application server...
```
### Test 3: Migration Failure Scenario
Verify container fails to start if migration fails:
```bash
# Manually corrupt the database or create an invalid migration
# Container should exit with error code 1
podman logs sneaky-klaus-test
# Expected output:
# Running database migrations...
# ERROR: Database migration failed!
# Please check the logs above for details.
# Container should not be running
podman ps -a | grep sneaky-klaus-test
# Should show "Exited (1)"
```
### Test 4: Development Workflow
Verify developers can still run migrations manually:
```bash
# Local development (no container)
uv run alembic upgrade head
# Should work as before
```
## Rollout Plan
### Phase 1: Update Current Release (v0.1.0)
Since v0.1.0 is already released, we should add automatic migrations to ensure users have this feature before Phase 2 ships:
1. Create entrypoint script
2. Update Containerfile
3. Remove `db.create_all()` from app.py
4. Test with existing v0.1.0 schema
5. Merge to `release/v0.1.0` branch
6. Build and tag new v0.1.0 patch release (v0.1.1)
7. Document in release notes
### Phase 2: Available for New Migrations (v0.2.0)
When Phase 2 development starts:
1. New Participant/MagicToken migrations will use automatic migration system
2. Test that migrations apply automatically on container startup
3. Document behavior in Phase 2 release notes
## Error Scenarios
### Scenario 1: Migration Fails
**Symptom**: Container exits immediately after startup
**User Action**:
1. Check logs: `podman logs sneaky-klaus`
2. Identify failing migration from Alembic output
3. Report issue or fix database manually
4. Restart container
**Prevention**: Thorough testing of migrations before release
### Scenario 2: Database File Permissions
**Symptom**: Alembic cannot write to database file
**Root Cause**: Volume mount permissions mismatch
**User Action**: Ensure data volume is writable by container user (UID 1000)
### Scenario 3: Missing uv Binary
**Symptom**: `/bin/bash: uv: command not found`
**Root Cause**: uv not copied to runtime stage
**Fix**: Verify Containerfile includes uv COPY step (Step 4 above)
## Documentation Updates
### User Documentation
Update deployment documentation to explain:
- Migrations run automatically on container startup
- How to check migration logs
- What to do if migrations fail
- No manual migration commands needed
### Developer Documentation
Update CLAUDE.md and development guide:
- Container entrypoint runs migrations
- Local development still uses manual `uv run alembic upgrade head`
- How to test migrations in container
## Future Enhancements
Potential improvements for future phases:
### Backup Before Migration
Add automatic backup before running migrations:
```bash
#!/bin/bash
set -e
echo "Backing up database..."
if [ -f /app/data/sneaky-klaus.db ]; then
cp /app/data/sneaky-klaus.db /app/data/sneaky-klaus.db.backup-$(date +%Y%m%d-%H%M%S)
fi
echo "Running database migrations..."
# ... rest of script
```
### Migration Dry-Run Mode
Add environment variable for dry-run mode:
```bash
if [ "$MIGRATION_DRY_RUN" = "true" ]; then
echo "DRY RUN MODE: Showing pending migrations..."
uv run alembic current
uv run alembic heads
exit 0
fi
```
### Migration Timeout
Add timeout to prevent hanging:
```bash
timeout 300 uv run alembic upgrade head || {
echo "ERROR: Migration timed out after 5 minutes"
exit 1
}
```
These enhancements are not required for Phase 2 but could be considered for Phase 5 (Operations) or later.
## References
- ADR-0005: Database Migrations with Alembic
- Phase 2 Implementation Decisions (Section 9.2)
- Alembic Documentation: https://alembic.sqlalchemy.org/
- Docker/Podman Entrypoint Best Practices
## Acceptance Criteria
This implementation is complete when:
- [ ] `entrypoint.sh` created and executable
- [ ] Containerfile updated to use entrypoint script
- [ ] `uv` binary available in runtime stage
- [ ] `db.create_all()` removed from `src/app.py`
- [ ] First-run test passes (fresh database)
- [ ] Update test passes (existing database with new migration)
- [ ] Failure test passes (container exits on migration error)
- [ ] Documentation updated
- [ ] Changes merged to release branch