Phil Skentelbery
3ed77fd45f
Fixes critical issue where migration 002 indexes already existed in SCHEMA_SQL, causing 'index already exists' errors on databases created before v1.0.0-rc.1. Changes: - Removed duplicate index definitions from SCHEMA_SQL (database.py) - Enhanced migration system to detect and handle indexes properly - Added comprehensive documentation of the fix Version bumped to 1.0.0-rc.2 with full changelog entry. Refs: docs/reports/2025-11-24-migration-fix-v1.0.0-rc.2.md
124 lines
4.2 KiB
Markdown
124 lines
4.2 KiB
Markdown
# INITIAL_SCHEMA_SQL Quick Reference
|
|
|
|
## What You're Building
|
|
Implementing Phase 2 of the database migration system redesign (ADR-031/032) by adding INITIAL_SCHEMA_SQL to represent the v0.1.0 baseline schema.
|
|
|
|
## Why It's Critical
|
|
Current system fails on production upgrades because SCHEMA_SQL represents current schema, not initial. This causes index creation on non-existent columns.
|
|
|
|
## Key Files to Modify
|
|
|
|
1. `/home/phil/Projects/starpunk/starpunk/database.py`
|
|
- Add INITIAL_SCHEMA_SQL constant (v0.1.0 schema)
|
|
- Rename SCHEMA_SQL to CURRENT_SCHEMA_SQL
|
|
- Add database_exists_with_tables() helper
|
|
- Update init_db() logic
|
|
|
|
2. `/home/phil/Projects/starpunk/tests/test_migrations.py`
|
|
- Add test_fresh_database_initialization()
|
|
- Add test_existing_database_upgrade()
|
|
|
|
## The INITIAL_SCHEMA_SQL Content
|
|
|
|
```sql
|
|
-- EXACTLY as it was in v0.1.0 (commit a68fd57)
|
|
-- Key differences from current:
|
|
-- 1. sessions: has 'session_token' not 'session_token_hash'
|
|
-- 2. tokens: plain text PRIMARY KEY, no token_hash column
|
|
-- 3. auth_state: no code_verifier column
|
|
-- 4. NO authorization_codes table at all
|
|
|
|
CREATE TABLE notes (...) -- with 4 indexes
|
|
CREATE TABLE sessions (...) -- with session_token (plain)
|
|
CREATE TABLE tokens (...) -- with token as PRIMARY KEY (plain)
|
|
CREATE TABLE auth_state (...) -- without code_verifier
|
|
```
|
|
|
|
## The New init_db() Logic
|
|
|
|
```python
|
|
def init_db(app=None):
|
|
if database_exists_with_tables(db_path):
|
|
# Existing DB: Skip schema, run migrations only
|
|
logger.info("Existing database found")
|
|
else:
|
|
# Fresh DB: Create v0.1.0 schema first
|
|
conn.executescript(INITIAL_SCHEMA_SQL)
|
|
logger.info("Created initial v0.1.0 schema")
|
|
|
|
# Always run migrations (brings everything current)
|
|
run_migrations(db_path, logger)
|
|
```
|
|
|
|
## Migration Path from INITIAL_SCHEMA_SQL
|
|
|
|
1. **Start**: v0.1.0 schema (INITIAL_SCHEMA_SQL)
|
|
2. **Migration 001**: Adds code_verifier to auth_state
|
|
3. **Migration 002**: Rebuilds tokens table (secure), adds authorization_codes
|
|
4. **Result**: Current schema (CURRENT_SCHEMA_SQL)
|
|
|
|
## Testing Commands
|
|
|
|
```bash
|
|
# Test fresh database
|
|
rm data/starpunk.db
|
|
uv run python app.py
|
|
# Should see: "Created initial v0.1.0 database schema"
|
|
# Should see: "Applied migration: 001_..."
|
|
# Should see: "Applied migration: 002_..."
|
|
|
|
# Test existing database
|
|
# (with backup of existing database)
|
|
uv run python app.py
|
|
# Should see: "Existing database found"
|
|
# Should see: "All migrations up to date"
|
|
|
|
# Verify schema
|
|
sqlite3 data/starpunk.db
|
|
.tables # Should show all tables including authorization_codes
|
|
SELECT * FROM schema_migrations; # Should show 2 migrations
|
|
```
|
|
|
|
## Success Indicators
|
|
|
|
✅ Fresh database creates without errors
|
|
✅ Existing database upgrades without "no such column" errors
|
|
✅ No "index already exists" errors
|
|
✅ Both migrations show in schema_migrations table
|
|
✅ authorization_codes table exists after migrations
|
|
✅ tokens table has token_hash column after migrations
|
|
✅ All tests pass
|
|
|
|
## Common Pitfalls to Avoid
|
|
|
|
❌ Don't use current schema for INITIAL_SCHEMA_SQL
|
|
❌ Don't forget to check database existence before schema creation
|
|
❌ Don't modify migration files (they're historical record)
|
|
❌ Don't skip testing both fresh and existing database paths
|
|
|
|
## If Something Goes Wrong
|
|
|
|
1. Check that INITIAL_SCHEMA_SQL matches commit a68fd57 exactly
|
|
2. Verify database_exists_with_tables() returns correct boolean
|
|
3. Ensure migrations/ directory is accessible
|
|
4. Check SQLite version supports all features
|
|
5. Review logs for specific error messages
|
|
|
|
## Time Estimate
|
|
|
|
- Implementation: 1-2 hours
|
|
- Testing: 2-3 hours
|
|
- Documentation updates: 1 hour
|
|
- **Total**: 4-6 hours
|
|
|
|
## References
|
|
|
|
- **Design**: /home/phil/Projects/starpunk/docs/decisions/ADR-032-initial-schema-sql-implementation.md
|
|
- **Context**: /home/phil/Projects/starpunk/docs/decisions/ADR-031-database-migration-system-redesign.md
|
|
- **Priority**: /home/phil/Projects/starpunk/docs/projectplan/v1.1/priority-work.md
|
|
- **Full Guide**: /home/phil/Projects/starpunk/docs/design/initial-schema-implementation-guide.md
|
|
- **Original Schema**: Git commit a68fd57
|
|
|
|
---
|
|
|
|
**Remember**: This is CRITICAL for v1.1.0. Without this fix, production databases cannot upgrade properly. |