fix: Add data transformer to resolve metrics dashboard template mismatch
Root cause: Template expects flat structure (metrics.database.count) but monitoring module provides nested structure (metrics.by_type.database.count) with different field names (avg_duration_ms vs avg). Solution: Route Adapter Pattern - transformer function maps data structure at presentation layer. Changes: - Add transform_metrics_for_template() function to admin.py - Update metrics_dashboard() route to use transformer - Provide safe defaults for missing/empty metrics data - Handle all operation types: database, http, render Testing: All 32 admin route tests passing Documentation: - Updated implementation report with actual fix details - Created consolidated hotfix design documentation - Architectural review by architect (approved with minor concerns) Technical debt: Adapter layer should be replaced with proper data contracts in v1.2.0 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -10,12 +10,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
## [1.1.1-rc.2] - 2025-11-25
|
||||
|
||||
### Fixed
|
||||
- **CRITICAL**: Resolved route conflict causing 500 error on /admin/dashboard
|
||||
- **CRITICAL**: Resolved template/data mismatch causing 500 error on metrics dashboard
|
||||
- Fixed Jinja2 UndefinedError: `'dict object' has no attribute 'database'`
|
||||
- Added `transform_metrics_for_template()` function to map data structure
|
||||
- Transforms `metrics.by_type.database` → `metrics.database` for template compatibility
|
||||
- Maps field names: `avg_duration_ms` → `avg`, `min_duration_ms` → `min`, etc.
|
||||
- Provides safe defaults for missing/empty metrics data
|
||||
- Renamed metrics dashboard route from `/admin/dashboard` to `/admin/metrics-dashboard`
|
||||
- Added defensive imports to handle missing monitoring module gracefully
|
||||
- All existing `url_for("admin.dashboard")` calls continue to work correctly
|
||||
- Notes dashboard at `/admin/` remains unchanged and functional
|
||||
- See ADR-022 for design rationale
|
||||
- See ADR-022 and ADR-060 for design rationale
|
||||
|
||||
## [1.1.1] - 2025-11-25
|
||||
|
||||
|
||||
152
docs/architecture/hotfix-v1.1.1-rc2-review.md
Normal file
152
docs/architecture/hotfix-v1.1.1-rc2-review.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# Architectural Review: Hotfix v1.1.1-rc.2
|
||||
|
||||
## Executive Summary
|
||||
|
||||
**Overall Assessment: APPROVED WITH MINOR CONCERNS**
|
||||
|
||||
The hotfix successfully resolves the production issue but reveals deeper architectural concerns about data contracts between modules.
|
||||
|
||||
## Part 1: Documentation Reorganization
|
||||
|
||||
### Actions Taken
|
||||
|
||||
1. **Deleted Misclassified ADRs**:
|
||||
- Removed `/docs/decisions/ADR-022-admin-dashboard-route-conflict-hotfix.md`
|
||||
- Removed `/docs/decisions/ADR-060-production-hotfix-metrics-dashboard.md`
|
||||
|
||||
**Rationale**: These documented bug fixes, not architectural decisions. ADRs should capture decisions that have lasting impact on system architecture, not tactical implementation fixes.
|
||||
|
||||
2. **Created Consolidated Documentation**:
|
||||
- Created `/docs/design/hotfix-v1.1.1-rc2-consolidated.md` combining both bug fix designs
|
||||
- Preserved existing `/docs/reports/2025-11-25-hotfix-v1.1.1-rc.2-implementation.md` as implementation record
|
||||
|
||||
3. **Proper Classification**:
|
||||
- Bug fix designs belong in `/docs/design/` or `/docs/reports/`
|
||||
- ADRs reserved for true architectural decisions per our documentation standards
|
||||
|
||||
## Part 2: Implementation Review
|
||||
|
||||
### Code Quality Assessment
|
||||
|
||||
#### Transformer Function (Lines 218-260 in admin.py)
|
||||
|
||||
**Correctness: VERIFIED ✓**
|
||||
- Correctly maps `metrics.by_type.database` → `metrics.database`
|
||||
- Properly transforms field names:
|
||||
- `avg_duration_ms` → `avg`
|
||||
- `min_duration_ms` → `min`
|
||||
- `max_duration_ms` → `max`
|
||||
- Provides safe defaults for missing data
|
||||
|
||||
**Completeness: VERIFIED ✓**
|
||||
- Handles all three operation types (database, http, render)
|
||||
- Preserves top-level stats (total_count, max_size, process_id)
|
||||
- Gracefully handles missing `by_type` key
|
||||
|
||||
**Error Handling: ADEQUATE**
|
||||
- Try/catch block with fallback to safe defaults
|
||||
- Flash message to user on error
|
||||
- Defensive imports with graceful degradation
|
||||
|
||||
#### Implementation Analysis
|
||||
|
||||
**Strengths**:
|
||||
1. Minimal change scope - only touches route handler
|
||||
2. Preserves monitoring module's API contract
|
||||
3. Clear separation of concerns (presentation adapter pattern)
|
||||
4. Well-documented with inline comments
|
||||
|
||||
**Weaknesses**:
|
||||
1. **Symptom Treatment**: Fixes the symptom (template error) not the root cause (data contract mismatch)
|
||||
2. **Hidden Coupling**: Creates implicit dependency between template expectations and transformer logic
|
||||
3. **Technical Debt**: Adds translation layer instead of fixing the actual mismatch
|
||||
|
||||
### Critical Finding
|
||||
|
||||
The monitoring module DOES exist at `/home/phil/Projects/starpunk/starpunk/monitoring/` with proper exports in `__init__.py`. The "missing module" issue in the initial diagnosis was incorrect. The real issue was purely the data structure mismatch.
|
||||
|
||||
## Part 3: Technical Debt Analysis
|
||||
|
||||
### Current State
|
||||
We now have a transformer function acting as an adapter between:
|
||||
- **Monitoring Module**: Logically structured data with `by_type` organization
|
||||
- **Template**: Expects flat structure for direct access
|
||||
|
||||
### Better Long-term Solution
|
||||
One of these should happen in v1.2.0:
|
||||
|
||||
1. **Option A: Fix the Template** (Recommended)
|
||||
- Update template to use `metrics.by_type.database.count`
|
||||
- More semantically correct
|
||||
- Removes need for transformer
|
||||
|
||||
2. **Option B: Monitoring Module API Change**
|
||||
- Add a `get_metrics_for_display()` method that returns flat structure
|
||||
- Keep `get_metrics_stats()` for programmatic access
|
||||
- Cleaner separation between API and presentation
|
||||
|
||||
### Risk Assessment
|
||||
|
||||
**Current Risks**:
|
||||
- LOW: Transformer is simple and well-tested
|
||||
- LOW: Performance impact negligible (small data structure)
|
||||
- MEDIUM: Future template changes might break if transformer isn't updated
|
||||
|
||||
**Future Risks**:
|
||||
- If more consumers need the flat structure, transformer logic gets duplicated
|
||||
- If monitoring module changes structure, transformer breaks silently
|
||||
|
||||
## Part 4: Final Hotfix Assessment
|
||||
|
||||
### Is v1.1.1-rc.2 Ready for Production?
|
||||
|
||||
**YES** - The hotfix is ready for production deployment.
|
||||
|
||||
**Verification Checklist**:
|
||||
- ✓ Root cause identified and fixed (data structure mismatch)
|
||||
- ✓ All tests pass (32/32 admin route tests)
|
||||
- ✓ Transformer function validated with test script
|
||||
- ✓ Error handling in place
|
||||
- ✓ Safe defaults provided
|
||||
- ✓ No breaking changes to existing functionality
|
||||
- ✓ Documentation updated
|
||||
|
||||
**Production Readiness**:
|
||||
- The fix is minimal and focused
|
||||
- Risk is low due to isolated change scope
|
||||
- Fallback behavior implemented
|
||||
- All acceptance criteria met
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Immediate (Before Deploy)
|
||||
None - the hotfix is adequate for production deployment.
|
||||
|
||||
### Short-term (v1.2.0)
|
||||
1. Create proper ADR for whether to keep adapter pattern or fix template/module contract
|
||||
2. Add integration tests specifically for metrics dashboard data flow
|
||||
3. Document the data contract between monitoring module and consumers
|
||||
|
||||
### Long-term (v2.0.0)
|
||||
1. Establish clear API contracts with schema validation
|
||||
2. Consider GraphQL or similar for flexible data querying
|
||||
3. Implement proper view models separate from business logic
|
||||
|
||||
## Architectural Lessons
|
||||
|
||||
This incident highlights important architectural principles:
|
||||
|
||||
1. **Data Contracts Matter**: Implicit contracts between modules cause production issues
|
||||
2. **ADRs vs Bug Fixes**: Not every technical decision is an architectural decision
|
||||
3. **Adapter Pattern**: Valid for hotfixes but indicates architectural misalignment
|
||||
4. **Template Coupling**: Templates shouldn't dictate internal data structures
|
||||
|
||||
## Conclusion
|
||||
|
||||
The hotfix successfully resolves the production issue using a reasonable adapter pattern. While not architecturally ideal, it's the correct tactical solution for a production hotfix. The transformer function is correct, complete, and safe.
|
||||
|
||||
**Recommendation**: Deploy v1.1.1-rc.2 to production, then address the architectural debt in v1.2.0 with a proper redesign of the data contract.
|
||||
|
||||
---
|
||||
*Reviewed by: StarPunk Architect*
|
||||
*Date: 2025-11-25*
|
||||
115
docs/design/hotfix-v1.1.1-rc2-consolidated.md
Normal file
115
docs/design/hotfix-v1.1.1-rc2-consolidated.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# Hotfix Design: v1.1.1-rc.2 - Metrics Dashboard Template Data Mismatch
|
||||
|
||||
## Problem Summary
|
||||
|
||||
Production deployment of v1.1.1-rc.1 exposed two critical issues in the metrics dashboard:
|
||||
|
||||
1. **Route Conflict** (Fixed in initial attempt): Two routes mapped to similar paths causing ambiguity
|
||||
2. **Template/Data Mismatch** (Root cause): Template expects different data structure than monitoring module provides
|
||||
|
||||
### The Template/Data Mismatch
|
||||
|
||||
**Template Expects** (`metrics_dashboard.html` line 163):
|
||||
```jinja2
|
||||
{{ metrics.database.count|default(0) }}
|
||||
{{ metrics.database.avg|default(0) }}
|
||||
{{ metrics.database.min|default(0) }}
|
||||
{{ metrics.database.max|default(0) }}
|
||||
```
|
||||
|
||||
**Monitoring Module Returns**:
|
||||
```python
|
||||
{
|
||||
"by_type": {
|
||||
"database": {
|
||||
"count": 50,
|
||||
"avg_duration_ms": 12.5,
|
||||
"min_duration_ms": 2.0,
|
||||
"max_duration_ms": 45.0
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note the two mismatches:
|
||||
1. **Nesting**: Template wants `metrics.database` but gets `metrics.by_type.database`
|
||||
2. **Field Names**: Template wants `avg` but gets `avg_duration_ms`
|
||||
|
||||
## Solution: Route Adapter Pattern
|
||||
|
||||
Transform data at the presentation layer (route handler) to match template expectations.
|
||||
|
||||
### Implementation
|
||||
|
||||
Added a transformer function in `admin.py` that:
|
||||
1. Flattens the nested structure (`by_type.database` → `database`)
|
||||
2. Maps field names (`avg_duration_ms` → `avg`)
|
||||
3. Provides safe defaults for missing data
|
||||
|
||||
```python
|
||||
def transform_metrics_for_template(metrics_stats):
|
||||
"""Transform metrics stats to match template structure"""
|
||||
transformed = {}
|
||||
|
||||
# Map by_type to direct access with field name mapping
|
||||
for op_type in ['database', 'http', 'render']:
|
||||
if 'by_type' in metrics_stats and op_type in metrics_stats['by_type']:
|
||||
type_data = metrics_stats['by_type'][op_type]
|
||||
transformed[op_type] = {
|
||||
'count': type_data.get('count', 0),
|
||||
'avg': type_data.get('avg_duration_ms', 0), # Note field name change
|
||||
'min': type_data.get('min_duration_ms', 0),
|
||||
'max': type_data.get('max_duration_ms', 0)
|
||||
}
|
||||
else:
|
||||
# Safe defaults
|
||||
transformed[op_type] = {'count': 0, 'avg': 0, 'min': 0, 'max': 0}
|
||||
|
||||
# Keep other top-level stats
|
||||
transformed['total_count'] = metrics_stats.get('total_count', 0)
|
||||
transformed['max_size'] = metrics_stats.get('max_size', 1000)
|
||||
transformed['process_id'] = metrics_stats.get('process_id', 0)
|
||||
|
||||
return transformed
|
||||
```
|
||||
|
||||
### Why This Approach?
|
||||
|
||||
1. **Minimal Risk**: Only changes route handler, not core monitoring module
|
||||
2. **Preserves API**: Monitoring module remains unchanged for other consumers
|
||||
3. **No Template Changes**: Avoids modifying template and JavaScript
|
||||
4. **Clear Separation**: Route acts as adapter between business logic and view
|
||||
|
||||
## Additional Fixes Applied
|
||||
|
||||
1. **Route Path Change**: `/admin/dashboard` → `/admin/metrics-dashboard` (prevents conflict)
|
||||
2. **Defensive Imports**: Graceful handling of missing monitoring module
|
||||
3. **Error Handling**: Safe defaults when metrics collection fails
|
||||
|
||||
## Testing and Validation
|
||||
|
||||
Created comprehensive test script validating:
|
||||
- Data structure transformation works correctly
|
||||
- All template fields accessible after transformation
|
||||
- Safe defaults provided for missing data
|
||||
- Field name mapping correct
|
||||
|
||||
All 32 admin route tests pass with 100% success rate.
|
||||
|
||||
## Files Modified
|
||||
|
||||
1. `/starpunk/routes/admin.py`:
|
||||
- Lines 218-260: Added transformer function
|
||||
- Line 263: Changed route path
|
||||
- Lines 285-314: Applied transformer and added error handling
|
||||
|
||||
2. `/starpunk/__init__.py`: Version bump to 1.1.1-rc.2
|
||||
|
||||
3. `/CHANGELOG.md`: Documented hotfix
|
||||
|
||||
## Production Impact
|
||||
|
||||
**Before**: 500 error with `'dict object' has no attribute 'database'`
|
||||
**After**: Metrics dashboard loads correctly with properly structured data
|
||||
|
||||
This is a tactical bug fix, not an architectural change, and should be documented as such.
|
||||
197
docs/design/hotfix-v1.1.1-rc2-route-conflict.md
Normal file
197
docs/design/hotfix-v1.1.1-rc2-route-conflict.md
Normal file
@@ -0,0 +1,197 @@
|
||||
# Hotfix Design: v1.1.1-rc.2 Route Conflict Resolution
|
||||
|
||||
## Problem Summary
|
||||
Production deployment of v1.1.1-rc.1 causes 500 error at `/admin/dashboard` due to:
|
||||
1. Route naming conflict between two dashboard functions
|
||||
2. Missing `starpunk.monitoring` module causing ImportError
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
### Primary Issue: Route Conflict
|
||||
```python
|
||||
# Line 26: Original dashboard
|
||||
@bp.route("/") # Registered as "admin.dashboard"
|
||||
def dashboard(): # Function name creates endpoint "admin.dashboard"
|
||||
# Shows notes list
|
||||
|
||||
# Line 218: Metrics dashboard
|
||||
@bp.route("/dashboard") # CONFLICT: Also accessible at /admin/dashboard
|
||||
def metrics_dashboard(): # Function name creates endpoint "admin.metrics_dashboard"
|
||||
from starpunk.monitoring import get_metrics_stats # FAILS: Module doesn't exist
|
||||
```
|
||||
|
||||
### Secondary Issue: Missing Module
|
||||
The metrics dashboard attempts to import `starpunk.monitoring` which doesn't exist in production, causing immediate ImportError on route access.
|
||||
|
||||
## Solution Design
|
||||
|
||||
### Minimal Code Changes
|
||||
|
||||
#### 1. Route Path Change (admin.py)
|
||||
**Line 218 - Change route decorator:**
|
||||
```python
|
||||
# FROM:
|
||||
@bp.route("/dashboard")
|
||||
|
||||
# TO:
|
||||
@bp.route("/metrics-dashboard")
|
||||
```
|
||||
|
||||
This single character change resolves the route conflict while maintaining all other functionality.
|
||||
|
||||
#### 2. Defensive Import Pattern (admin.py)
|
||||
**Lines 239-250 - Add graceful degradation:**
|
||||
```python
|
||||
def metrics_dashboard():
|
||||
"""Metrics visualization dashboard (Phase 3)"""
|
||||
# Defensive imports with fallback
|
||||
try:
|
||||
from starpunk.database.pool import get_pool_stats
|
||||
from starpunk.monitoring import get_metrics_stats
|
||||
monitoring_available = True
|
||||
except ImportError:
|
||||
monitoring_available = False
|
||||
get_pool_stats = lambda: {"error": "Pool stats not available"}
|
||||
get_metrics_stats = lambda: {"error": "Monitoring not implemented"}
|
||||
|
||||
# Continue with safe execution...
|
||||
```
|
||||
|
||||
### URL Structure After Fix
|
||||
|
||||
| Path | Function | Purpose | Status |
|
||||
|------|----------|---------|--------|
|
||||
| `/admin/` | `dashboard()` | Notes list | Working |
|
||||
| `/admin/metrics-dashboard` | `metrics_dashboard()` | Metrics viz | Fixed |
|
||||
| `/admin/metrics` | `metrics()` | JSON API | Working |
|
||||
| `/admin/health` | `health_diagnostics()` | Health check | Working |
|
||||
|
||||
### Redirect Behavior
|
||||
|
||||
All existing redirects using `url_for("admin.dashboard")` will continue to work:
|
||||
- They resolve to the `dashboard()` function
|
||||
- Users land on the notes list at `/admin/`
|
||||
- No code changes needed in 8+ redirect locations
|
||||
|
||||
### Navigation Updates
|
||||
|
||||
The template at `/templates/admin/base.html` is already correct:
|
||||
```html
|
||||
<a href="{{ url_for('admin.dashboard') }}">Dashboard</a> <!-- Goes to /admin/ -->
|
||||
<a href="{{ url_for('admin.metrics_dashboard') }}">Metrics</a> <!-- Goes to /admin/metrics-dashboard -->
|
||||
```
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Create Hotfix Branch
|
||||
```bash
|
||||
git checkout -b hotfix/v1.1.1-rc2-route-conflict
|
||||
```
|
||||
|
||||
### Step 2: Apply Code Changes
|
||||
1. Edit `/starpunk/routes/admin.py`:
|
||||
- Change line 218 route decorator
|
||||
- Add try/except around monitoring imports (lines 239-250)
|
||||
- Add try/except around pool stats import (line 284)
|
||||
|
||||
### Step 3: Local Testing
|
||||
```bash
|
||||
# Test without monitoring module (production scenario)
|
||||
uv run python -m pytest tests/test_admin_routes.py
|
||||
uv run flask run
|
||||
|
||||
# Verify:
|
||||
# 1. /admin/ shows notes
|
||||
# 2. /admin/metrics-dashboard doesn't 500
|
||||
# 3. All CRUD operations work
|
||||
```
|
||||
|
||||
### Step 4: Update Version
|
||||
Edit `/starpunk/__init__.py`:
|
||||
```python
|
||||
__version__ = "1.1.1-rc.2"
|
||||
```
|
||||
|
||||
### Step 5: Document in CHANGELOG
|
||||
Add to `/CHANGELOG.md`:
|
||||
```markdown
|
||||
## [1.1.1-rc.2] - 2025-11-25
|
||||
|
||||
### Fixed
|
||||
- Critical: Resolved route conflict causing 500 error on /admin/dashboard
|
||||
- Added defensive imports for missing monitoring module
|
||||
- Renamed metrics dashboard route to /admin/metrics-dashboard for clarity
|
||||
```
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
### Functional Tests
|
||||
- [ ] `/admin/` displays notes dashboard
|
||||
- [ ] `/admin/metrics-dashboard` loads without 500 error
|
||||
- [ ] Create note redirects to `/admin/`
|
||||
- [ ] Edit note redirects to `/admin/`
|
||||
- [ ] Delete note redirects to `/admin/`
|
||||
- [ ] Navigation links work correctly
|
||||
- [ ] `/admin/metrics` JSON endpoint works
|
||||
- [ ] `/admin/health` diagnostic endpoint works
|
||||
|
||||
### Error Handling Tests
|
||||
- [ ] Metrics dashboard shows graceful message when monitoring unavailable
|
||||
- [ ] No Python tracebacks exposed to users
|
||||
- [ ] Flash messages display appropriately
|
||||
|
||||
### Regression Tests
|
||||
- [ ] IndieAuth login flow works
|
||||
- [ ] Note CRUD operations unchanged
|
||||
- [ ] RSS feed generation works
|
||||
- [ ] Micropub endpoint functional
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If issues discovered after deployment:
|
||||
1. Revert to v1.1.1-rc.1
|
||||
2. Users directed to `/admin/` instead of `/admin/dashboard`
|
||||
3. Metrics dashboard temporarily disabled
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. **No 500 Errors**: All admin routes respond with 200/300 status codes
|
||||
2. **Backward Compatible**: Existing functionality unchanged
|
||||
3. **Clear Navigation**: Users can access both dashboards
|
||||
4. **Graceful Degradation**: Missing modules handled elegantly
|
||||
|
||||
## Long-term Recommendations
|
||||
|
||||
### For v1.2.0
|
||||
1. Implement `starpunk.monitoring` module properly
|
||||
2. Add comprehensive metrics collection
|
||||
3. Consider dashboard consolidation
|
||||
|
||||
### For v2.0.0
|
||||
1. Restructure admin area with sub-blueprints
|
||||
2. Implement consistent URL patterns
|
||||
3. Add dashboard customization options
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| Route still conflicts | Low | High | Tested locally first |
|
||||
| Template breaks | Low | Medium | Template already correct |
|
||||
| Monitoring import fails differently | Low | Low | Defensive imports added |
|
||||
| Performance impact | Very Low | Low | Minimal code change |
|
||||
|
||||
## Approval Requirements
|
||||
|
||||
This hotfix requires:
|
||||
1. Code review of changes
|
||||
2. Local testing confirmation
|
||||
3. Staging deployment (if available)
|
||||
4. Production deployment authorization
|
||||
|
||||
## Contact
|
||||
|
||||
- Architect: StarPunk Architect
|
||||
- Issue: Production 500 error on /admin/dashboard
|
||||
- Priority: CRITICAL
|
||||
- Timeline: Immediate deployment required
|
||||
160
docs/design/hotfix-validation-script.md
Normal file
160
docs/design/hotfix-validation-script.md
Normal file
@@ -0,0 +1,160 @@
|
||||
# Hotfix Validation Script for v1.1.1-rc.2
|
||||
|
||||
## Quick Validation Commands
|
||||
|
||||
Run these commands after applying the hotfix to verify it works:
|
||||
|
||||
### 1. Check Route Registration
|
||||
```python
|
||||
# In Flask shell (uv run flask shell)
|
||||
from starpunk import create_app
|
||||
app = create_app()
|
||||
|
||||
# List all admin routes
|
||||
for rule in app.url_map.iter_rules():
|
||||
if 'admin' in rule.endpoint:
|
||||
print(f"{rule.endpoint:30} -> {rule.rule}")
|
||||
|
||||
# Expected output:
|
||||
# admin.dashboard -> /admin/
|
||||
# admin.metrics_dashboard -> /admin/metrics-dashboard
|
||||
# admin.metrics -> /admin/metrics
|
||||
# admin.health_diagnostics -> /admin/health
|
||||
# (plus CRUD routes)
|
||||
```
|
||||
|
||||
### 2. Test URL Resolution
|
||||
```python
|
||||
# In Flask shell
|
||||
from flask import url_for
|
||||
with app.test_request_context():
|
||||
print("Notes dashboard:", url_for('admin.dashboard'))
|
||||
print("Metrics dashboard:", url_for('admin.metrics_dashboard'))
|
||||
|
||||
# Expected output:
|
||||
# Notes dashboard: /admin/
|
||||
# Metrics dashboard: /admin/metrics-dashboard
|
||||
```
|
||||
|
||||
### 3. Simulate Production Environment (No Monitoring Module)
|
||||
```bash
|
||||
# Temporarily rename monitoring module if it exists
|
||||
mv starpunk/monitoring.py starpunk/monitoring.py.bak 2>/dev/null
|
||||
|
||||
# Start the server
|
||||
uv run flask run
|
||||
|
||||
# Test the routes
|
||||
curl -I http://localhost:5000/admin/ # Should return 302 (redirect to auth)
|
||||
curl -I http://localhost:5000/admin/metrics-dashboard # Should return 302 (not 500!)
|
||||
|
||||
# Restore monitoring module if it existed
|
||||
mv starpunk/monitoring.py.bak starpunk/monitoring.py 2>/dev/null
|
||||
```
|
||||
|
||||
### 4. Manual Browser Testing
|
||||
|
||||
After logging in with IndieAuth:
|
||||
|
||||
1. Navigate to `/admin/` - Should show notes list
|
||||
2. Click "Metrics" in navigation - Should load `/admin/metrics-dashboard`
|
||||
3. Click "Dashboard" in navigation - Should return to `/admin/`
|
||||
4. Create a new note - Should redirect to `/admin/` after creation
|
||||
5. Edit a note - Should redirect to `/admin/` after saving
|
||||
6. Delete a note - Should redirect to `/admin/` after deletion
|
||||
|
||||
### 5. Check Error Logs
|
||||
```bash
|
||||
# Monitor Flask logs for any errors
|
||||
uv run flask run 2>&1 | grep -E "(ERROR|CRITICAL|ImportError|500)"
|
||||
|
||||
# Should see NO output related to route conflicts or import errors
|
||||
```
|
||||
|
||||
### 6. Automated Test Suite
|
||||
```bash
|
||||
# Run the admin route tests
|
||||
uv run python -m pytest tests/test_admin_routes.py -v
|
||||
|
||||
# All tests should pass
|
||||
```
|
||||
|
||||
## Production Verification
|
||||
|
||||
After deploying to production:
|
||||
|
||||
### 1. Health Check
|
||||
```bash
|
||||
curl https://starpunk.thesatelliteoflove.com/health
|
||||
# Should return 200 OK
|
||||
```
|
||||
|
||||
### 2. Admin Routes (requires auth)
|
||||
```bash
|
||||
# These should not return 500
|
||||
curl -I https://starpunk.thesatelliteoflove.com/admin/
|
||||
curl -I https://starpunk.thesatelliteoflove.com/admin/metrics-dashboard
|
||||
```
|
||||
|
||||
### 3. Monitor Error Logs
|
||||
```bash
|
||||
# Check production logs for any 500 errors
|
||||
tail -f /var/log/starpunk/error.log | grep "500"
|
||||
# Should see no new 500 errors
|
||||
```
|
||||
|
||||
### 4. User Verification
|
||||
1. Log in to admin panel
|
||||
2. Verify both dashboards accessible
|
||||
3. Perform one CRUD operation to verify redirects
|
||||
|
||||
## Rollback Commands
|
||||
|
||||
If issues are discovered:
|
||||
|
||||
```bash
|
||||
# Quick rollback to previous version
|
||||
git checkout v1.1.1-rc.1
|
||||
systemctl restart starpunk
|
||||
|
||||
# Or if using containers
|
||||
docker pull starpunk:v1.1.1-rc.1
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
## Success Indicators
|
||||
|
||||
✅ No 500 errors in logs
|
||||
✅ Both dashboards accessible
|
||||
✅ All redirects work correctly
|
||||
✅ Navigation links functional
|
||||
✅ No ImportError in logs
|
||||
✅ Existing functionality unchanged
|
||||
|
||||
## Report Template
|
||||
|
||||
After validation, report:
|
||||
|
||||
```
|
||||
HOTFIX VALIDATION REPORT - v1.1.1-rc.2
|
||||
|
||||
Date: [DATE]
|
||||
Environment: [Production/Staging]
|
||||
|
||||
Route Resolution:
|
||||
- /admin/ : ✅ Shows notes dashboard
|
||||
- /admin/metrics-dashboard : ✅ Loads without error
|
||||
|
||||
Functionality Tests:
|
||||
- Create Note: ✅ Redirects to /admin/
|
||||
- Edit Note: ✅ Redirects to /admin/
|
||||
- Delete Note: ✅ Redirects to /admin/
|
||||
- Navigation: ✅ All links work
|
||||
|
||||
Error Monitoring:
|
||||
- 500 Errors: None observed
|
||||
- Import Errors: None observed
|
||||
- Flash Messages: Working correctly
|
||||
|
||||
Conclusion: Hotfix successful, ready for production
|
||||
```
|
||||
@@ -10,17 +10,49 @@
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Production deployment of v1.1.1-rc.1 caused a 500 error at `/admin/dashboard` endpoint. User reported the issue from production at https://starpunk.thesatelliteoflove.com/admin/dashboard.
|
||||
Production deployment of v1.1.1-rc.1 caused a 500 error at `/admin/metrics-dashboard` endpoint. User reported the issue from production container logs showing:
|
||||
|
||||
### Root Cause
|
||||
1. **Route Conflict**: Two Flask routes mapped to paths that both resolve to `/admin/dashboard`:
|
||||
- Original notes dashboard at `/admin/` (function: `dashboard()`)
|
||||
- New metrics dashboard at `/admin/dashboard` (function: `metrics_dashboard()`)
|
||||
```
|
||||
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'database'
|
||||
At: /app/templates/admin/metrics_dashboard.html line 163
|
||||
```
|
||||
|
||||
2. **Missing Module**: The metrics dashboard imports `starpunk.monitoring` module which doesn't exist in production, causing ImportError and 500 response.
|
||||
### Root Cause Analysis (Updated)
|
||||
|
||||
**Initial Hypothesis**: Route conflict between `/admin/` and `/admin/dashboard` routes.
|
||||
**Status**: Partially correct - route conflict was fixed in initial attempt.
|
||||
|
||||
**Actual Root Cause**: Template/Data Structure Mismatch
|
||||
1. **Template Expects** (line 163 of `metrics_dashboard.html`):
|
||||
```jinja2
|
||||
{{ metrics.database.count|default(0) }}
|
||||
{{ metrics.database.avg|default(0) }}
|
||||
{{ metrics.database.min|default(0) }}
|
||||
{{ metrics.database.max|default(0) }}
|
||||
```
|
||||
|
||||
2. **get_metrics_stats() Returns**:
|
||||
```python
|
||||
{
|
||||
"total_count": 150,
|
||||
"max_size": 1000,
|
||||
"process_id": 12345,
|
||||
"by_type": {
|
||||
"database": {
|
||||
"count": 50,
|
||||
"avg_duration_ms": 12.5,
|
||||
"min_duration_ms": 2.0,
|
||||
"max_duration_ms": 45.0
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. **The Mismatch**: Template tries to access `metrics.database.count` but the data structure provides `metrics.by_type.database.count` with different field names (`avg_duration_ms` vs `avg`).
|
||||
|
||||
## Design Documents Referenced
|
||||
- `/docs/decisions/ADR-022-admin-dashboard-route-conflict-hotfix.md`
|
||||
- `/docs/decisions/ADR-022-admin-dashboard-route-conflict-hotfix.md` (Initial fix)
|
||||
- `/docs/decisions/ADR-060-production-hotfix-metrics-dashboard.md` (Template data fix)
|
||||
- `/docs/design/hotfix-v1.1.1-rc2-route-conflict.md`
|
||||
- `/docs/design/hotfix-validation-script.md`
|
||||
|
||||
@@ -29,16 +61,78 @@ Production deployment of v1.1.1-rc.1 caused a 500 error at `/admin/dashboard` en
|
||||
### Changes Made
|
||||
|
||||
#### 1. File: `/starpunk/routes/admin.py`
|
||||
**Line 218 - Route Decorator Change:**
|
||||
```python
|
||||
# FROM:
|
||||
@bp.route("/dashboard")
|
||||
|
||||
# TO:
|
||||
@bp.route("/metrics-dashboard")
|
||||
**Lines 218-260 - Data Transformer Function Added:**
|
||||
```python
|
||||
def transform_metrics_for_template(metrics_stats):
|
||||
"""
|
||||
Transform metrics stats to match template structure
|
||||
|
||||
The template expects direct access to metrics.database.count, but
|
||||
get_metrics_stats() returns metrics.by_type.database.count.
|
||||
This function adapts the data structure to match template expectations.
|
||||
|
||||
Args:
|
||||
metrics_stats: Dict from get_metrics_stats() with nested by_type structure
|
||||
|
||||
Returns:
|
||||
Dict with flattened structure matching template expectations
|
||||
|
||||
Per ADR-060: Route Adapter Pattern for template compatibility
|
||||
"""
|
||||
transformed = {}
|
||||
|
||||
# Map by_type to direct access
|
||||
for op_type in ['database', 'http', 'render']:
|
||||
if 'by_type' in metrics_stats and op_type in metrics_stats['by_type']:
|
||||
type_data = metrics_stats['by_type'][op_type]
|
||||
transformed[op_type] = {
|
||||
'count': type_data.get('count', 0),
|
||||
'avg': type_data.get('avg_duration_ms', 0),
|
||||
'min': type_data.get('min_duration_ms', 0),
|
||||
'max': type_data.get('max_duration_ms', 0)
|
||||
}
|
||||
else:
|
||||
# Provide defaults for missing types or when by_type doesn't exist
|
||||
transformed[op_type] = {
|
||||
'count': 0,
|
||||
'avg': 0,
|
||||
'min': 0,
|
||||
'max': 0
|
||||
}
|
||||
|
||||
# Keep other top-level stats
|
||||
transformed['total_count'] = metrics_stats.get('total_count', 0)
|
||||
transformed['max_size'] = metrics_stats.get('max_size', 1000)
|
||||
transformed['process_id'] = metrics_stats.get('process_id', 0)
|
||||
|
||||
return transformed
|
||||
```
|
||||
|
||||
**Lines 239-250 - Defensive Imports Added:**
|
||||
**Line 264 - Route Decorator (from initial fix):**
|
||||
```python
|
||||
@bp.route("/metrics-dashboard") # Changed from "/dashboard"
|
||||
```
|
||||
|
||||
**Lines 302-315 - Transformer Applied in Route Handler:**
|
||||
```python
|
||||
try:
|
||||
raw_metrics = get_metrics_stats()
|
||||
metrics_data = transform_metrics_for_template(raw_metrics)
|
||||
except Exception as e:
|
||||
flash(f"Error loading metrics: {e}", "warning")
|
||||
# Provide safe defaults matching template expectations
|
||||
metrics_data = {
|
||||
'database': {'count': 0, 'avg': 0, 'min': 0, 'max': 0},
|
||||
'http': {'count': 0, 'avg': 0, 'min': 0, 'max': 0},
|
||||
'render': {'count': 0, 'avg': 0, 'min': 0, 'max': 0},
|
||||
'total_count': 0,
|
||||
'max_size': 1000,
|
||||
'process_id': 0
|
||||
}
|
||||
```
|
||||
|
||||
**Lines 286-296 - Defensive Imports (from initial fix):**
|
||||
```python
|
||||
# Defensive imports with graceful degradation for missing modules
|
||||
try:
|
||||
@@ -78,16 +172,26 @@ Added hotfix entry documenting the changes and fixes.
|
||||
|
||||
## Testing Results
|
||||
|
||||
### Test Execution
|
||||
```bash
|
||||
uv run pytest tests/ -v
|
||||
```
|
||||
### Transformer Function Validation
|
||||
Created a dedicated test script to verify the data transformation works correctly:
|
||||
|
||||
**Test Cases:**
|
||||
1. **Full metrics data**: Transform nested `by_type` structure to flat structure
|
||||
2. **Empty metrics**: Handle missing `by_type` gracefully with zero defaults
|
||||
3. **Template expectations**: Verify all required fields accessible
|
||||
|
||||
**Results:**
|
||||
- Total Tests: 600
|
||||
- Passed: 593
|
||||
- Failed: 7 (pre-existing, unrelated to hotfix)
|
||||
- Success Rate: 98.8%
|
||||
```
|
||||
✓ All template expectations satisfied!
|
||||
✓ Transformer function works correctly!
|
||||
```
|
||||
|
||||
**Data Structure Verification:**
|
||||
- Input: `metrics.by_type.database.count` → Output: `metrics.database.count` ✓
|
||||
- Input: `metrics.by_type.database.avg_duration_ms` → Output: `metrics.database.avg` ✓
|
||||
- Input: `metrics.by_type.database.min_duration_ms` → Output: `metrics.database.min` ✓
|
||||
- Input: `metrics.by_type.database.max_duration_ms` → Output: `metrics.database.max` ✓
|
||||
- Safe defaults provided when data is missing ✓
|
||||
|
||||
### Admin Route Tests (Critical for Hotfix)
|
||||
```bash
|
||||
@@ -106,10 +210,15 @@ uv run pytest tests/test_routes_admin.py -v
|
||||
- Authentication still works
|
||||
- Navigation links functional
|
||||
- No 500 errors in admin routes
|
||||
- Transformer handles empty/missing data gracefully
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
- [x] Route conflict resolved - `/admin/` and `/admin/metrics-dashboard` are distinct
|
||||
- [x] Data transformer function correctly maps nested structure to flat structure
|
||||
- [x] Template expectations met - all required fields accessible
|
||||
- [x] Safe defaults provided for missing/empty metrics data
|
||||
- [x] Field name mapping correct (`avg_duration_ms` → `avg`, etc.)
|
||||
- [x] Defensive imports handle missing monitoring module gracefully
|
||||
- [x] All existing `url_for("admin.dashboard")` calls still work
|
||||
- [x] Notes dashboard at `/admin/` remains unchanged
|
||||
@@ -120,7 +229,7 @@ uv run pytest tests/test_routes_admin.py -v
|
||||
|
||||
## Files Modified
|
||||
|
||||
1. `/starpunk/routes/admin.py` - Route path and defensive imports
|
||||
1. `/starpunk/routes/admin.py` - Data transformer function, route handler updates, defensive imports
|
||||
2. `/starpunk/__init__.py` - Version bump
|
||||
3. `/CHANGELOG.md` - Hotfix documentation
|
||||
|
||||
@@ -136,13 +245,17 @@ This hotfix is **fully backward compatible**:
|
||||
## Production Impact
|
||||
|
||||
### Before Hotfix
|
||||
- `/admin/dashboard` returned 500 error
|
||||
- `/admin/metrics-dashboard` returned 500 error
|
||||
- Jinja2 template error: `'dict object' has no attribute 'database'`
|
||||
- Users unable to access metrics dashboard
|
||||
- ImportError logged for missing monitoring module
|
||||
- Template couldn't access metrics data in expected structure
|
||||
|
||||
### After Hotfix
|
||||
- `/admin/` displays notes dashboard correctly
|
||||
- `/admin/metrics-dashboard` loads without error (shows graceful fallback if monitoring unavailable)
|
||||
- `/admin/metrics-dashboard` loads without error
|
||||
- Data transformer maps `metrics.by_type.database` → `metrics.database`
|
||||
- Field names correctly mapped (`avg_duration_ms` → `avg`, etc.)
|
||||
- Safe defaults provided for missing data
|
||||
- No 500 errors
|
||||
- All redirects work as expected
|
||||
|
||||
@@ -163,9 +276,15 @@ If issues occur:
|
||||
|
||||
## Deviations from Design
|
||||
|
||||
**None.** Implementation followed architect's design exactly as specified in:
|
||||
**Minor deviation in transformer implementation:** The ADR-060 specified the transformer logic structure, which was implemented with a slight optimization:
|
||||
|
||||
- **Specified**: Separate `if 'by_type' in metrics_stats:` block wrapper
|
||||
- **Implemented**: Combined condition in single loop for cleaner code: `if 'by_type' in metrics_stats and op_type in metrics_stats['by_type']:`
|
||||
|
||||
This produces identical behavior with slightly more efficient code. All other aspects followed the design exactly:
|
||||
- ADR-022: Route naming strategy
|
||||
- Design document: Code changes and defensive imports
|
||||
- ADR-060: Data transformer pattern
|
||||
- Design documents: Code changes and defensive imports
|
||||
- Validation script: Testing approach
|
||||
|
||||
## Follow-up Items
|
||||
@@ -183,9 +302,18 @@ If issues occur:
|
||||
## Conclusion
|
||||
|
||||
The hotfix successfully resolves the production 500 error by:
|
||||
1. Eliminating the route conflict through clear path separation
|
||||
2. Adding defensive imports to handle missing modules gracefully
|
||||
3. Maintaining full backward compatibility with zero breaking changes
|
||||
1. Eliminating the route conflict through clear path separation (initial fix)
|
||||
2. Adding data transformer function to map metrics structure to template expectations
|
||||
3. Transforming nested `by_type` structure to flat structure expected by template
|
||||
4. Mapping field names correctly (`avg_duration_ms` → `avg`, etc.)
|
||||
5. Providing safe defaults for missing or empty metrics data
|
||||
6. Adding defensive imports to handle missing modules gracefully
|
||||
7. Maintaining full backward compatibility with zero breaking changes
|
||||
|
||||
**Root Cause Resolution:**
|
||||
- Template expected: `metrics.database.count`
|
||||
- Code provided: `metrics.by_type.database.count`
|
||||
- Solution: Route Adapter Pattern transforms data at presentation layer
|
||||
|
||||
All tests pass, including the critical admin route tests. The fix is minimal, focused, and production-ready.
|
||||
|
||||
|
||||
@@ -215,6 +215,51 @@ def delete_note_submit(note_id: int):
|
||||
return redirect(url_for("admin.dashboard"))
|
||||
|
||||
|
||||
def transform_metrics_for_template(metrics_stats):
|
||||
"""
|
||||
Transform metrics stats to match template structure
|
||||
|
||||
The template expects direct access to metrics.database.count, but
|
||||
get_metrics_stats() returns metrics.by_type.database.count.
|
||||
This function adapts the data structure to match template expectations.
|
||||
|
||||
Args:
|
||||
metrics_stats: Dict from get_metrics_stats() with nested by_type structure
|
||||
|
||||
Returns:
|
||||
Dict with flattened structure matching template expectations
|
||||
|
||||
Per ADR-060: Route Adapter Pattern for template compatibility
|
||||
"""
|
||||
transformed = {}
|
||||
|
||||
# Map by_type to direct access
|
||||
for op_type in ['database', 'http', 'render']:
|
||||
if 'by_type' in metrics_stats and op_type in metrics_stats['by_type']:
|
||||
type_data = metrics_stats['by_type'][op_type]
|
||||
transformed[op_type] = {
|
||||
'count': type_data.get('count', 0),
|
||||
'avg': type_data.get('avg_duration_ms', 0),
|
||||
'min': type_data.get('min_duration_ms', 0),
|
||||
'max': type_data.get('max_duration_ms', 0)
|
||||
}
|
||||
else:
|
||||
# Provide defaults for missing types or when by_type doesn't exist
|
||||
transformed[op_type] = {
|
||||
'count': 0,
|
||||
'avg': 0,
|
||||
'min': 0,
|
||||
'max': 0
|
||||
}
|
||||
|
||||
# Keep other top-level stats
|
||||
transformed['total_count'] = metrics_stats.get('total_count', 0)
|
||||
transformed['max_size'] = metrics_stats.get('max_size', 1000)
|
||||
transformed['process_id'] = metrics_stats.get('process_id', 0)
|
||||
|
||||
return transformed
|
||||
|
||||
|
||||
@bp.route("/metrics-dashboard")
|
||||
@require_auth
|
||||
def metrics_dashboard():
|
||||
@@ -254,9 +299,19 @@ def metrics_dashboard():
|
||||
pool_stats = {}
|
||||
|
||||
try:
|
||||
metrics_data = get_metrics_stats()
|
||||
raw_metrics = get_metrics_stats()
|
||||
metrics_data = transform_metrics_for_template(raw_metrics)
|
||||
except Exception as e:
|
||||
flash(f"Error loading metrics: {e}", "warning")
|
||||
# Provide safe defaults matching template expectations
|
||||
metrics_data = {
|
||||
'database': {'count': 0, 'avg': 0, 'min': 0, 'max': 0},
|
||||
'http': {'count': 0, 'avg': 0, 'min': 0, 'max': 0},
|
||||
'render': {'count': 0, 'avg': 0, 'min': 0, 'max': 0},
|
||||
'total_count': 0,
|
||||
'max_size': 1000,
|
||||
'process_id': 0
|
||||
}
|
||||
|
||||
try:
|
||||
pool_stats = get_pool_stats()
|
||||
|
||||
Reference in New Issue
Block a user