fix: Resolve database migration failure on existing databases
Fixes critical issue where migration 002 indexes already existed in SCHEMA_SQL, causing 'index already exists' errors on databases created before v1.0.0-rc.1. Changes: - Removed duplicate index definitions from SCHEMA_SQL (database.py) - Enhanced migration system to detect and handle indexes properly - Added comprehensive documentation of the fix Version bumped to 1.0.0-rc.2 with full changelog entry. Refs: docs/reports/2025-11-24-migration-fix-v1.0.0-rc.2.md
This commit is contained in:
124
docs/design/initial-schema-quick-reference.md
Normal file
124
docs/design/initial-schema-quick-reference.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# INITIAL_SCHEMA_SQL Quick Reference
|
||||
|
||||
## What You're Building
|
||||
Implementing Phase 2 of the database migration system redesign (ADR-031/032) by adding INITIAL_SCHEMA_SQL to represent the v0.1.0 baseline schema.
|
||||
|
||||
## Why It's Critical
|
||||
Current system fails on production upgrades because SCHEMA_SQL represents current schema, not initial. This causes index creation on non-existent columns.
|
||||
|
||||
## Key Files to Modify
|
||||
|
||||
1. `/home/phil/Projects/starpunk/starpunk/database.py`
|
||||
- Add INITIAL_SCHEMA_SQL constant (v0.1.0 schema)
|
||||
- Rename SCHEMA_SQL to CURRENT_SCHEMA_SQL
|
||||
- Add database_exists_with_tables() helper
|
||||
- Update init_db() logic
|
||||
|
||||
2. `/home/phil/Projects/starpunk/tests/test_migrations.py`
|
||||
- Add test_fresh_database_initialization()
|
||||
- Add test_existing_database_upgrade()
|
||||
|
||||
## The INITIAL_SCHEMA_SQL Content
|
||||
|
||||
```sql
|
||||
-- EXACTLY as it was in v0.1.0 (commit a68fd57)
|
||||
-- Key differences from current:
|
||||
-- 1. sessions: has 'session_token' not 'session_token_hash'
|
||||
-- 2. tokens: plain text PRIMARY KEY, no token_hash column
|
||||
-- 3. auth_state: no code_verifier column
|
||||
-- 4. NO authorization_codes table at all
|
||||
|
||||
CREATE TABLE notes (...) -- with 4 indexes
|
||||
CREATE TABLE sessions (...) -- with session_token (plain)
|
||||
CREATE TABLE tokens (...) -- with token as PRIMARY KEY (plain)
|
||||
CREATE TABLE auth_state (...) -- without code_verifier
|
||||
```
|
||||
|
||||
## The New init_db() Logic
|
||||
|
||||
```python
|
||||
def init_db(app=None):
|
||||
if database_exists_with_tables(db_path):
|
||||
# Existing DB: Skip schema, run migrations only
|
||||
logger.info("Existing database found")
|
||||
else:
|
||||
# Fresh DB: Create v0.1.0 schema first
|
||||
conn.executescript(INITIAL_SCHEMA_SQL)
|
||||
logger.info("Created initial v0.1.0 schema")
|
||||
|
||||
# Always run migrations (brings everything current)
|
||||
run_migrations(db_path, logger)
|
||||
```
|
||||
|
||||
## Migration Path from INITIAL_SCHEMA_SQL
|
||||
|
||||
1. **Start**: v0.1.0 schema (INITIAL_SCHEMA_SQL)
|
||||
2. **Migration 001**: Adds code_verifier to auth_state
|
||||
3. **Migration 002**: Rebuilds tokens table (secure), adds authorization_codes
|
||||
4. **Result**: Current schema (CURRENT_SCHEMA_SQL)
|
||||
|
||||
## Testing Commands
|
||||
|
||||
```bash
|
||||
# Test fresh database
|
||||
rm data/starpunk.db
|
||||
uv run python app.py
|
||||
# Should see: "Created initial v0.1.0 database schema"
|
||||
# Should see: "Applied migration: 001_..."
|
||||
# Should see: "Applied migration: 002_..."
|
||||
|
||||
# Test existing database
|
||||
# (with backup of existing database)
|
||||
uv run python app.py
|
||||
# Should see: "Existing database found"
|
||||
# Should see: "All migrations up to date"
|
||||
|
||||
# Verify schema
|
||||
sqlite3 data/starpunk.db
|
||||
.tables # Should show all tables including authorization_codes
|
||||
SELECT * FROM schema_migrations; # Should show 2 migrations
|
||||
```
|
||||
|
||||
## Success Indicators
|
||||
|
||||
✅ Fresh database creates without errors
|
||||
✅ Existing database upgrades without "no such column" errors
|
||||
✅ No "index already exists" errors
|
||||
✅ Both migrations show in schema_migrations table
|
||||
✅ authorization_codes table exists after migrations
|
||||
✅ tokens table has token_hash column after migrations
|
||||
✅ All tests pass
|
||||
|
||||
## Common Pitfalls to Avoid
|
||||
|
||||
❌ Don't use current schema for INITIAL_SCHEMA_SQL
|
||||
❌ Don't forget to check database existence before schema creation
|
||||
❌ Don't modify migration files (they're historical record)
|
||||
❌ Don't skip testing both fresh and existing database paths
|
||||
|
||||
## If Something Goes Wrong
|
||||
|
||||
1. Check that INITIAL_SCHEMA_SQL matches commit a68fd57 exactly
|
||||
2. Verify database_exists_with_tables() returns correct boolean
|
||||
3. Ensure migrations/ directory is accessible
|
||||
4. Check SQLite version supports all features
|
||||
5. Review logs for specific error messages
|
||||
|
||||
## Time Estimate
|
||||
|
||||
- Implementation: 1-2 hours
|
||||
- Testing: 2-3 hours
|
||||
- Documentation updates: 1 hour
|
||||
- **Total**: 4-6 hours
|
||||
|
||||
## References
|
||||
|
||||
- **Design**: /home/phil/Projects/starpunk/docs/decisions/ADR-032-initial-schema-sql-implementation.md
|
||||
- **Context**: /home/phil/Projects/starpunk/docs/decisions/ADR-031-database-migration-system-redesign.md
|
||||
- **Priority**: /home/phil/Projects/starpunk/docs/projectplan/v1.1/priority-work.md
|
||||
- **Full Guide**: /home/phil/Projects/starpunk/docs/design/initial-schema-implementation-guide.md
|
||||
- **Original Schema**: Git commit a68fd57
|
||||
|
||||
---
|
||||
|
||||
**Remember**: This is CRITICAL for v1.1.0. Without this fix, production databases cannot upgrade properly.
|
||||
Reference in New Issue
Block a user