sneakyklaus/docs/designs/v0.2.0/automatic-migration-implementation.md

Automatic Migration Implementation Guide

Version: 0.2.0 Date: 2025-12-22 Status: Implementation Specification

Overview

This document specifies how to implement automatic database migrations for containerized deployments of Sneaky Klaus. When self-hosted users pull a new container image with schema changes, migrations must be applied automatically without manual intervention.

Context

• Current State: Container runs gunicorn directly; no automatic migrations
• Problem: Users must manually run alembic upgrade head after pulling new images
• Solution: Container entrypoint script applies migrations before starting app
• Reference: ADR-0005 "Automatic Migrations for Self-Hosted Deployments"

Implementation Steps

Step 1: Create Entrypoint Script

Create /home/phil/Projects/sneaky-klaus/entrypoint.sh:

#!/bin/bash
set -e  # Exit on any error

echo "Running database migrations..."
if uv run alembic upgrade head; then
    echo "Database migrations completed successfully"
else
    echo "ERROR: Database migration failed!"
    echo "Please check the logs above for details."
    exit 1
fi

echo "Starting application server..."
exec gunicorn --bind 0.0.0.0:8000 --workers 2 --threads 4 main:app

Key Points:

• set -e: Ensures script exits immediately if any command fails
• Migrations run before gunicorn starts
• exec replaces the shell process with gunicorn (PID 1 becomes gunicorn for proper signal handling)
• Clear logging for debugging
• Exit code 1 on migration failure prevents container from starting

Step 2: Update Containerfile

Modify /home/phil/Projects/sneaky-klaus/Containerfile:

Add after line 29 (after copying application code):

# Copy entrypoint script
COPY entrypoint.sh ./
RUN chmod +x entrypoint.sh

Replace line 51 (CMD line):

From:

CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "2", "--threads", "4", "main:app"]

To:

CMD ["./entrypoint.sh"]

Full context (lines 27-52 after changes):

# Copy application code
COPY src/ ./src/
COPY migrations/ ./migrations/
COPY alembic.ini main.py ./

# Copy entrypoint script
COPY entrypoint.sh ./
RUN chmod +x entrypoint.sh

# Set environment variables
ENV PATH="/app/.venv/bin:$PATH"
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENV FLASK_ENV=production

# Create data directory and set permissions
RUN mkdir -p /app/data && chown -R appuser:appuser /app

# Switch to non-root user
USER appuser

# Expose port
EXPOSE 8000

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1

# Run with entrypoint script
CMD ["./entrypoint.sh"]

Step 3: Remove db.create_all() from Application

Modify /home/phil/Projects/sneaky-klaus/src/app.py:

Replace lines 74-77:

From:

# Import models to ensure they're registered with SQLAlchemy
with app.app_context():
    from src.models import Admin, Exchange, RateLimit  # noqa: F401

    db.create_all()

To:

# Import models to ensure they're registered with SQLAlchemy
with app.app_context():
    from src.models import Admin, Exchange, RateLimit  # noqa: F401
    # Schema managed by Alembic migrations (applied via entrypoint script)

Rationale:

• Migrations are now handled by entrypoint script
• db.create_all() conflicts with migration-based schema management
• db.create_all() cannot handle schema changes (only creates missing tables)
• ADR-0005 explicitly requires migrations for production

Step 4: Copy uv to Runtime Stage

The entrypoint script uses uv run alembic, so we need uv available in the runtime stage.

Modify /home/phil/Projects/sneaky-klaus/Containerfile:

Add after line 23 (after copying .venv from builder):

# Copy uv for running alembic in entrypoint
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv

Full context (lines 15-30 after change):

# Runtime stage - minimal image
FROM python:3.12-slim AS runtime

WORKDIR /app

# Create non-root user
RUN useradd --create-home --shell /bin/bash appuser

# Copy virtual environment from builder
COPY --from=builder /app/.venv /app/.venv

# Copy uv for running alembic in entrypoint
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv

# Copy application code
COPY src/ ./src/
COPY migrations/ ./migrations/
COPY alembic.ini main.py ./

Testing

Test 1: First-Run Scenario

Verify migrations run when starting with no database:

# Build new image
podman build -t sneaky-klaus:test .

# Remove any existing data volume
podman volume rm sneaky-klaus-data || true

# Start container
podman run -p 8000:8000 \
  -e SECRET_KEY=test-secret-key \
  -e APP_URL=http://localhost:8000 \
  -v sneaky-klaus-data:/app/data \
  --name sneaky-klaus-test \
  sneaky-klaus:test

# Check logs - should see migration messages
podman logs sneaky-klaus-test

# Expected output:
# Running database migrations...
# INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
# INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
# INFO  [alembic.runtime.migration] Running upgrade  -> eeff6e1a89cd, initial schema with Admin and Exchange models
# Database migrations completed successfully
# Starting application server...

Test 2: Update Scenario

Verify incremental migrations run when updating:

# With container already running from Test 1, rebuild with new migration
# (After developer adds new migration in Phase 2)
podman build -t sneaky-klaus:test .

# Stop and remove old container
podman stop sneaky-klaus-test
podman rm sneaky-klaus-test

# Start new container with same data volume
podman run -p 8000:8000 \
  -e SECRET_KEY=test-secret-key \
  -e APP_URL=http://localhost:8000 \
  -v sneaky-klaus-data:/app/data \
  --name sneaky-klaus-test \
  sneaky-klaus:test

# Check logs - should see new migration applied
podman logs sneaky-klaus-test

# Expected output:
# Running database migrations...
# INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
# INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
# INFO  [alembic.runtime.migration] Running upgrade eeff6e1a89cd -> abc123def456, Add Participant and MagicToken models
# Database migrations completed successfully
# Starting application server...

Test 3: Migration Failure Scenario

Verify container fails to start if migration fails:

# Manually corrupt the database or create an invalid migration
# Container should exit with error code 1
podman logs sneaky-klaus-test

# Expected output:
# Running database migrations...
# ERROR: Database migration failed!
# Please check the logs above for details.

# Container should not be running
podman ps -a | grep sneaky-klaus-test
# Should show "Exited (1)"

Test 4: Development Workflow

Verify developers can still run migrations manually:

# Local development (no container)
uv run alembic upgrade head

# Should work as before

Rollout Plan

Phase 1: Update Current Release (v0.1.0)

Since v0.1.0 is already released, we should add automatic migrations to ensure users have this feature before Phase 2 ships:

1. Create entrypoint script
2. Update Containerfile
3. Remove db.create_all() from app.py
4. Test with existing v0.1.0 schema
5. Merge to release/v0.1.0 branch
6. Build and tag new v0.1.0 patch release (v0.1.1)
7. Document in release notes

Phase 2: Available for New Migrations (v0.2.0)

When Phase 2 development starts:

1. New Participant/MagicToken migrations will use automatic migration system
2. Test that migrations apply automatically on container startup
3. Document behavior in Phase 2 release notes

Error Scenarios

Scenario 1: Migration Fails

Symptom: Container exits immediately after startup User Action:

1. Check logs: podman logs sneaky-klaus
2. Identify failing migration from Alembic output
3. Report issue or fix database manually
4. Restart container

Prevention: Thorough testing of migrations before release

Scenario 2: Database File Permissions

Symptom: Alembic cannot write to database file Root Cause: Volume mount permissions mismatch User Action: Ensure data volume is writable by container user (UID 1000)

Scenario 3: Missing uv Binary

Symptom: /bin/bash: uv: command not found Root Cause: uv not copied to runtime stage Fix: Verify Containerfile includes uv COPY step (Step 4 above)

Documentation Updates

User Documentation

Update deployment documentation to explain:

• Migrations run automatically on container startup
• How to check migration logs
• What to do if migrations fail
• No manual migration commands needed

Developer Documentation

Update CLAUDE.md and development guide:

• Container entrypoint runs migrations
• Local development still uses manual uv run alembic upgrade head
• How to test migrations in container

Future Enhancements

Potential improvements for future phases:

Backup Before Migration

Add automatic backup before running migrations:

#!/bin/bash
set -e

echo "Backing up database..."
if [ -f /app/data/sneaky-klaus.db ]; then
    cp /app/data/sneaky-klaus.db /app/data/sneaky-klaus.db.backup-$(date +%Y%m%d-%H%M%S)
fi

echo "Running database migrations..."
# ... rest of script

Migration Dry-Run Mode

Add environment variable for dry-run mode:

if [ "$MIGRATION_DRY_RUN" = "true" ]; then
    echo "DRY RUN MODE: Showing pending migrations..."
    uv run alembic current
    uv run alembic heads
    exit 0
fi

Migration Timeout

Add timeout to prevent hanging:

timeout 300 uv run alembic upgrade head || {
    echo "ERROR: Migration timed out after 5 minutes"
    exit 1
}

These enhancements are not required for Phase 2 but could be considered for Phase 5 (Operations) or later.

References

• ADR-0005: Database Migrations with Alembic
• Phase 2 Implementation Decisions (Section 9.2)
• Alembic Documentation: https://alembic.sqlalchemy.org/
• Docker/Podman Entrypoint Best Practices

Acceptance Criteria

This implementation is complete when:

• entrypoint.sh created and executable
• Containerfile updated to use entrypoint script
• uv binary available in runtime stage
• db.create_all() removed from src/app.py
• First-run test passes (fresh database)
• Update test passes (existing database with new migration)
• Failure test passes (container exits on migration error)
• Documentation updated
• Changes merged to release branch