Phil Skentelbery
eaafa78cf3
Implements Phase 2 infrastructure for participant registration and authentication: Database Models: - Add Participant model with exchange scoping and soft deletes - Add MagicToken model for passwordless authentication - Add participants relationship to Exchange model - Include proper indexes and foreign key constraints Migration Infrastructure: - Generate Alembic migration for new models - Create entrypoint.sh script for automatic migrations on container startup - Update Containerfile to use entrypoint script and include uv binary - Remove db.create_all() in favor of migration-based schema management This establishes the foundation for implementing stories 4.1-4.3, 5.1-5.3, and 10.1. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
267 lines
11 KiB
Markdown
267 lines
11 KiB
Markdown
# Database Migration Flow
|
|
|
|
## Container Startup Sequence
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ User runs: podman run sneaky-klaus │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Container starts entrypoint.sh │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Step 1: Run "uv run alembic upgrade head" │
|
|
│ │
|
|
│ Alembic checks /app/data/sneaky-klaus.db │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌───────────────┴───────────────┐
|
|
│ │
|
|
▼ ▼
|
|
┌──────────────────────┐ ┌──────────────────────┐
|
|
│ Database exists │ │ No database │
|
|
│ (update scenario) │ │ (first run) │
|
|
└──────────────────────┘ └──────────────────────┘
|
|
│ │
|
|
▼ ▼
|
|
┌──────────────────────┐ ┌──────────────────────┐
|
|
│ Read alembic_version │ │ Create database file │
|
|
│ table │ │ │
|
|
└──────────────────────┘ └──────────────────────┘
|
|
│ │
|
|
▼ ▼
|
|
┌──────────────────────┐ ┌──────────────────────┐
|
|
│ Apply only new │ │ Run all migrations │
|
|
│ migrations │ │ from scratch │
|
|
│ (incremental) │ │ │
|
|
└──────────────────────┘ └──────────────────────┘
|
|
│ │
|
|
└───────────────┬───────────────┘
|
|
│
|
|
┌───────────┴───────────┐
|
|
│ │
|
|
▼ ▼
|
|
┌──────────────────┐ ┌──────────────────┐
|
|
│ Success │ │ Failure │
|
|
└──────────────────┘ └──────────────────┘
|
|
│ │
|
|
▼ ▼
|
|
┌──────────────────┐ ┌──────────────────┐
|
|
│ Log success │ │ Log error │
|
|
│ message │ │ Exit code 1 │
|
|
└──────────────────┘ └──────────────────┘
|
|
│ │
|
|
▼ ▼
|
|
┌──────────────────┐ ┌──────────────────┐
|
|
│ Step 2: Start │ │ Container stops │
|
|
│ gunicorn │ │ (failed state) │
|
|
│ │ │ │
|
|
│ Application │ │ User checks logs │
|
|
│ ready to serve │ │ to debug │
|
|
└──────────────────┘ └──────────────────┘
|
|
```
|
|
|
|
## Migration Scenarios
|
|
|
|
### Scenario 1: Fresh Installation (First Run)
|
|
|
|
```
|
|
User action: podman run sneaky-klaus:v0.2.0
|
|
|
|
Container startup:
|
|
1. entrypoint.sh executes
|
|
2. alembic upgrade head runs
|
|
- No database file exists
|
|
- Creates /app/data/sneaky-klaus.db
|
|
- Creates alembic_version table
|
|
- Runs migration eeff6e1a89cd (Admin, Exchange)
|
|
- Runs migration abc123def456 (Participant, MagicToken)
|
|
- Sets current version: abc123def456
|
|
3. gunicorn starts
|
|
4. Application ready
|
|
|
|
Result: Fresh database with all tables
|
|
```
|
|
|
|
### Scenario 2: Update from v0.1.0 to v0.2.0
|
|
|
|
```
|
|
User action:
|
|
1. podman pull sneaky-klaus:v0.2.0
|
|
2. podman stop sneaky-klaus
|
|
3. podman rm sneaky-klaus
|
|
4. podman run sneaky-klaus:v0.2.0 (same volume)
|
|
|
|
Container startup:
|
|
1. entrypoint.sh executes
|
|
2. alembic upgrade head runs
|
|
- Database file exists
|
|
- Reads alembic_version table
|
|
- Current version: eeff6e1a89cd
|
|
- Detects new migration: abc123def456
|
|
- Runs migration abc123def456 (adds Participant, MagicToken)
|
|
- Updates current version: abc123def456
|
|
3. gunicorn starts
|
|
4. Application ready
|
|
|
|
Result: Updated database with new tables, existing data preserved
|
|
```
|
|
|
|
### Scenario 3: Already Up-to-Date
|
|
|
|
```
|
|
User action: podman restart sneaky-klaus
|
|
|
|
Container startup:
|
|
1. entrypoint.sh executes
|
|
2. alembic upgrade head runs
|
|
- Database file exists
|
|
- Reads alembic_version table
|
|
- Current version: abc123def456
|
|
- No new migrations to apply
|
|
- Logs "Already at head"
|
|
3. gunicorn starts
|
|
4. Application ready
|
|
|
|
Result: No changes, fast startup
|
|
```
|
|
|
|
### Scenario 4: Migration Failure
|
|
|
|
```
|
|
User action: podman run sneaky-klaus:v0.3.0 (hypothetical buggy migration)
|
|
|
|
Container startup:
|
|
1. entrypoint.sh executes
|
|
2. alembic upgrade head runs
|
|
- Database file exists
|
|
- Current version: abc123def456
|
|
- Attempts new migration: xyz789bad000
|
|
- Migration fails (SQL error, constraint violation, etc.)
|
|
- Alembic rolls back transaction
|
|
- Returns exit code 1
|
|
3. entrypoint.sh detects failure
|
|
- Logs error message
|
|
- Exits with code 1
|
|
4. Container stops (failed state)
|
|
|
|
Result: Database unchanged, container not running
|
|
User action: Check logs, report bug, or fix database manually
|
|
```
|
|
|
|
## Comparison: Manual vs Automatic Migrations
|
|
|
|
### Manual Migration Workflow (without this feature)
|
|
|
|
```
|
|
User workflow:
|
|
1. podman pull sneaky-klaus:v0.2.0
|
|
2. podman stop sneaky-klaus
|
|
3. podman exec sneaky-klaus bash ← Extra step
|
|
4. uv run alembic upgrade head ← Manual command
|
|
5. exit ← Extra step
|
|
6. podman start sneaky-klaus
|
|
|
|
Problems:
|
|
- Requires command-line knowledge
|
|
- Easy to forget
|
|
- Error-prone
|
|
- Not friendly for non-technical users
|
|
```
|
|
|
|
### Automatic Migration Workflow (with this feature)
|
|
|
|
```
|
|
User workflow:
|
|
1. podman pull sneaky-klaus:v0.2.0
|
|
2. podman stop sneaky-klaus
|
|
3. podman rm sneaky-klaus
|
|
4. podman run sneaky-klaus:v0.2.0
|
|
|
|
Benefits:
|
|
- Simple, standard container workflow
|
|
- Cannot forget to run migrations
|
|
- Migrations guaranteed to run before app starts
|
|
- Self-hosted friendly
|
|
```
|
|
|
|
## File Structure After Implementation
|
|
|
|
```
|
|
sneaky-klaus/
|
|
├── entrypoint.sh ← NEW: Migration + startup script
|
|
├── Containerfile ← MODIFIED: Use entrypoint
|
|
├── src/
|
|
│ └── app.py ← MODIFIED: Remove db.create_all()
|
|
├── migrations/
|
|
│ ├── env.py ← (existing)
|
|
│ └── versions/
|
|
│ ├── eeff6e1a89cd_....py ← (existing)
|
|
│ └── abc123def456_....py ← NEW: Phase 2 migration
|
|
└── alembic.ini ← (existing)
|
|
```
|
|
|
|
## Developer vs Production Workflows
|
|
|
|
### Developer Workflow (Local Development)
|
|
|
|
```bash
|
|
# Developer makes schema change
|
|
1. Edit src/models/participant.py
|
|
2. uv run alembic revision --autogenerate -m "Add Participant model"
|
|
3. Review generated migration file
|
|
4. uv run alembic upgrade head ← Manual migration
|
|
5. uv run pytest ← Test
|
|
6. git add migrations/versions/...
|
|
7. git commit -m "feat: add Participant model"
|
|
```
|
|
|
|
Developers retain explicit control over when migrations run.
|
|
|
|
### Production Workflow (Container Deployment)
|
|
|
|
```bash
|
|
# Self-hosted user updates to new version
|
|
1. podman pull sneaky-klaus:v0.2.0
|
|
2. podman-compose down
|
|
3. podman-compose up -d ← Migrations run automatically
|
|
|
|
# No manual migration step needed
|
|
```
|
|
|
|
Users get automatic, safe migrations without extra commands.
|
|
|
|
## Security & Safety Considerations
|
|
|
|
### Why This is Safe
|
|
|
|
1. **Atomic migrations**: Alembic uses transactions (rollback on failure)
|
|
2. **Version tracking**: alembic_version table prevents re-running migrations
|
|
3. **Fail-safe**: Container won't start if migration fails
|
|
4. **No data loss**: Migrations are additive (add tables/columns)
|
|
5. **Tested**: All migrations tested before release
|
|
|
|
### What Could Go Wrong
|
|
|
|
1. **Migration bug**: Bad migration could fail or corrupt data
|
|
- **Mitigation**: Thorough testing of migrations before release
|
|
- **Recovery**: Database backup (future enhancement)
|
|
|
|
2. **Permission issue**: Container can't write to database file
|
|
- **Mitigation**: Volume permissions documentation
|
|
- **Recovery**: Fix volume permissions and restart
|
|
|
|
3. **Disk full**: No space for database changes
|
|
- **Mitigation**: Health checks and monitoring
|
|
- **Recovery**: Free up disk space and restart
|
|
|
|
## References
|
|
|
|
- ADR-0005: Database Migrations with Alembic
|
|
- Automatic Migration Implementation Guide
|
|
- Phase 2 Implementation Decisions (Section 9.2)
|