# Database Migration Flow ## Container Startup Sequence ``` ┌─────────────────────────────────────────────────────────────────┐ │ User runs: podman run sneaky-klaus │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ Container starts entrypoint.sh │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ Step 1: Run "uv run alembic upgrade head" │ │ │ │ Alembic checks /app/data/sneaky-klaus.db │ └─────────────────────────────────────────────────────────────────┘ │ ┌───────────────┴───────────────┐ │ │ ▼ ▼ ┌──────────────────────┐ ┌──────────────────────┐ │ Database exists │ │ No database │ │ (update scenario) │ │ (first run) │ └──────────────────────┘ └──────────────────────┘ │ │ ▼ ▼ ┌──────────────────────┐ ┌──────────────────────┐ │ Read alembic_version │ │ Create database file │ │ table │ │ │ └──────────────────────┘ └──────────────────────┘ │ │ ▼ ▼ ┌──────────────────────┐ ┌──────────────────────┐ │ Apply only new │ │ Run all migrations │ │ migrations │ │ from scratch │ │ (incremental) │ │ │ └──────────────────────┘ └──────────────────────┘ │ │ └───────────────┬───────────────┘ │ ┌───────────┴───────────┐ │ │ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ │ Success │ │ Failure │ └──────────────────┘ └──────────────────┘ │ │ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ │ Log success │ │ Log error │ │ message │ │ Exit code 1 │ └──────────────────┘ └──────────────────┘ │ │ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ │ Step 2: Start │ │ Container stops │ │ gunicorn │ │ (failed state) │ │ │ │ │ │ Application │ │ User checks logs │ │ ready to serve │ │ to debug │ └──────────────────┘ └──────────────────┘ ``` ## Migration Scenarios ### Scenario 1: Fresh Installation (First Run) ``` User action: podman run sneaky-klaus:v0.2.0 Container startup: 1. entrypoint.sh executes 2. alembic upgrade head runs - No database file exists - Creates /app/data/sneaky-klaus.db - Creates alembic_version table - Runs migration eeff6e1a89cd (Admin, Exchange) - Runs migration abc123def456 (Participant, MagicToken) - Sets current version: abc123def456 3. gunicorn starts 4. Application ready Result: Fresh database with all tables ``` ### Scenario 2: Update from v0.1.0 to v0.2.0 ``` User action: 1. podman pull sneaky-klaus:v0.2.0 2. podman stop sneaky-klaus 3. podman rm sneaky-klaus 4. podman run sneaky-klaus:v0.2.0 (same volume) Container startup: 1. entrypoint.sh executes 2. alembic upgrade head runs - Database file exists - Reads alembic_version table - Current version: eeff6e1a89cd - Detects new migration: abc123def456 - Runs migration abc123def456 (adds Participant, MagicToken) - Updates current version: abc123def456 3. gunicorn starts 4. Application ready Result: Updated database with new tables, existing data preserved ``` ### Scenario 3: Already Up-to-Date ``` User action: podman restart sneaky-klaus Container startup: 1. entrypoint.sh executes 2. alembic upgrade head runs - Database file exists - Reads alembic_version table - Current version: abc123def456 - No new migrations to apply - Logs "Already at head" 3. gunicorn starts 4. Application ready Result: No changes, fast startup ``` ### Scenario 4: Migration Failure ``` User action: podman run sneaky-klaus:v0.3.0 (hypothetical buggy migration) Container startup: 1. entrypoint.sh executes 2. alembic upgrade head runs - Database file exists - Current version: abc123def456 - Attempts new migration: xyz789bad000 - Migration fails (SQL error, constraint violation, etc.) - Alembic rolls back transaction - Returns exit code 1 3. entrypoint.sh detects failure - Logs error message - Exits with code 1 4. Container stops (failed state) Result: Database unchanged, container not running User action: Check logs, report bug, or fix database manually ``` ## Comparison: Manual vs Automatic Migrations ### Manual Migration Workflow (without this feature) ``` User workflow: 1. podman pull sneaky-klaus:v0.2.0 2. podman stop sneaky-klaus 3. podman exec sneaky-klaus bash ← Extra step 4. uv run alembic upgrade head ← Manual command 5. exit ← Extra step 6. podman start sneaky-klaus Problems: - Requires command-line knowledge - Easy to forget - Error-prone - Not friendly for non-technical users ``` ### Automatic Migration Workflow (with this feature) ``` User workflow: 1. podman pull sneaky-klaus:v0.2.0 2. podman stop sneaky-klaus 3. podman rm sneaky-klaus 4. podman run sneaky-klaus:v0.2.0 Benefits: - Simple, standard container workflow - Cannot forget to run migrations - Migrations guaranteed to run before app starts - Self-hosted friendly ``` ## File Structure After Implementation ``` sneaky-klaus/ ├── entrypoint.sh ← NEW: Migration + startup script ├── Containerfile ← MODIFIED: Use entrypoint ├── src/ │ └── app.py ← MODIFIED: Remove db.create_all() ├── migrations/ │ ├── env.py ← (existing) │ └── versions/ │ ├── eeff6e1a89cd_....py ← (existing) │ └── abc123def456_....py ← NEW: Phase 2 migration └── alembic.ini ← (existing) ``` ## Developer vs Production Workflows ### Developer Workflow (Local Development) ```bash # Developer makes schema change 1. Edit src/models/participant.py 2. uv run alembic revision --autogenerate -m "Add Participant model" 3. Review generated migration file 4. uv run alembic upgrade head ← Manual migration 5. uv run pytest ← Test 6. git add migrations/versions/... 7. git commit -m "feat: add Participant model" ``` Developers retain explicit control over when migrations run. ### Production Workflow (Container Deployment) ```bash # Self-hosted user updates to new version 1. podman pull sneaky-klaus:v0.2.0 2. podman-compose down 3. podman-compose up -d ← Migrations run automatically # No manual migration step needed ``` Users get automatic, safe migrations without extra commands. ## Security & Safety Considerations ### Why This is Safe 1. **Atomic migrations**: Alembic uses transactions (rollback on failure) 2. **Version tracking**: alembic_version table prevents re-running migrations 3. **Fail-safe**: Container won't start if migration fails 4. **No data loss**: Migrations are additive (add tables/columns) 5. **Tested**: All migrations tested before release ### What Could Go Wrong 1. **Migration bug**: Bad migration could fail or corrupt data - **Mitigation**: Thorough testing of migrations before release - **Recovery**: Database backup (future enhancement) 2. **Permission issue**: Container can't write to database file - **Mitigation**: Volume permissions documentation - **Recovery**: Fix volume permissions and restart 3. **Disk full**: No space for database changes - **Mitigation**: Health checks and monitoring - **Recovery**: Free up disk space and restart ## References - ADR-0005: Database Migrations with Alembic - Automatic Migration Implementation Guide - Phase 2 Implementation Decisions (Section 9.2)