feat(tags): Add database schema and tags module (v1.3.0 Phase 1)
Implements tag/category system backend following microformats2 p-category specification. Database changes: - Migration 008: Add tags and note_tags tables - Normalized tag storage (case-insensitive lookup, display name preserved) - Indexes for performance New module: - starpunk/tags.py: Tag management functions - normalize_tag: Normalize tag strings - get_or_create_tag: Get or create tag records - add_tags_to_note: Associate tags with notes (replaces existing) - get_note_tags: Retrieve note tags (alphabetically ordered) - get_tag_by_name: Lookup tag by normalized name - get_notes_by_tag: Get all notes with specific tag - parse_tag_input: Parse comma-separated tag input Model updates: - Note.tags property (lazy-loaded, prefer pre-loading in routes) - Note.to_dict() add include_tags parameter CRUD updates: - create_note() accepts tags parameter - update_note() accepts tags parameter (None = no change, [] = remove all) Micropub integration: - Pass tags to create_note() (tags already extracted by extract_tags()) - Return tags in q=source response Per design doc: docs/design/v1.3.0/microformats-tags-design.md Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
289
docs/design/v1.1.2/2025-11-28-v1.1.2-rc.2-fixes.md
Normal file
289
docs/design/v1.1.2/2025-11-28-v1.1.2-rc.2-fixes.md
Normal file
@@ -0,0 +1,289 @@
|
||||
# v1.1.2-rc.2 Production Bug Fixes - Implementation Report
|
||||
|
||||
**Date:** 2025-11-28
|
||||
**Developer:** Developer Agent
|
||||
**Version:** 1.1.2-rc.2
|
||||
**Status:** Fixes Complete, Tests Passed
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Successfully implemented fixes for two production issues found in v1.1.2-rc.1:
|
||||
|
||||
1. **CRITICAL (Issue 1)**: Static files returning 500 errors - site completely unusable
|
||||
2. **HIGH (Issue 2)**: Database metrics showing zero due to config mismatch
|
||||
|
||||
Both fixes implemented according to architect specifications. All 28 monitoring tests pass. Ready for production deployment.
|
||||
|
||||
---
|
||||
|
||||
## Issue 1: Static Files Return 500 Error (CRITICAL)
|
||||
|
||||
### Problem
|
||||
HTTP middleware's `after_request` hook accessed `response.data` on streaming responses (used by Flask's `send_from_directory` for static files), causing:
|
||||
```
|
||||
RuntimeError: Attempted implicit sequence conversion but the response object is in direct passthrough mode.
|
||||
```
|
||||
|
||||
### Impact
|
||||
- ALL static files (CSS, JS, images) returned HTTP 500
|
||||
- Site completely unusable without stylesheets
|
||||
- Affected every page load
|
||||
|
||||
### Root Cause
|
||||
The HTTP metrics middleware in `starpunk/monitoring/http.py:74-78` was checking `response.data` to calculate response size for metrics. Streaming responses cannot have their `.data` accessed without triggering an error.
|
||||
|
||||
### Solution Implemented
|
||||
**File:** `starpunk/monitoring/http.py:73-86`
|
||||
|
||||
Added check for `direct_passthrough` mode before accessing response data:
|
||||
|
||||
```python
|
||||
# Get response size
|
||||
response_size = 0
|
||||
|
||||
# Check if response is in direct passthrough mode (streaming)
|
||||
if hasattr(response, 'direct_passthrough') and response.direct_passthrough:
|
||||
# For streaming responses, use content_length if available
|
||||
if hasattr(response, 'content_length') and response.content_length:
|
||||
response_size = response.content_length
|
||||
# Otherwise leave as 0 (unknown size for streaming)
|
||||
elif response.data:
|
||||
# For buffered responses, we can safely get the data
|
||||
response_size = len(response.data)
|
||||
elif hasattr(response, 'content_length') and response.content_length:
|
||||
response_size = response.content_length
|
||||
```
|
||||
|
||||
### Verification
|
||||
- Monitoring tests: 28/28 passed (including HTTP metrics tests)
|
||||
- Static files now load without errors
|
||||
- Metrics still recorded for static files (with size when available)
|
||||
- Graceful fallback for unknown sizes (records as 0)
|
||||
|
||||
---
|
||||
|
||||
## Issue 2: Database Metrics Showing Zero (HIGH)
|
||||
|
||||
### Problem
|
||||
Admin dashboard showed 0 for all database metrics despite metrics being enabled and database operations occurring.
|
||||
|
||||
### Impact
|
||||
- Database performance monitoring feature incomplete
|
||||
- No visibility into database operation performance
|
||||
- Database pool statistics worked, but operation metrics didn't
|
||||
|
||||
### Root Cause
|
||||
Configuration key mismatch:
|
||||
- **`starpunk/config.py:92`**: Sets `METRICS_SAMPLING_RATE` (singular) = 1.0 (100%)
|
||||
- **`starpunk/monitoring/metrics.py:337`**: Reads `METRICS_SAMPLING_RATES` (plural) expecting dict
|
||||
- **Result**: Always returned `None`, fell back to hardcoded 10% sampling
|
||||
- **Consequence**: Low traffic + 10% sampling = no metrics recorded
|
||||
|
||||
### Solution Implemented
|
||||
|
||||
#### Part 1: Updated MetricsBuffer to Accept Float or Dict
|
||||
**File:** `starpunk/monitoring/metrics.py:87-125`
|
||||
|
||||
Modified `MetricsBuffer.__init__` to handle both formats:
|
||||
|
||||
```python
|
||||
def __init__(
|
||||
self,
|
||||
max_size: int = 1000,
|
||||
sampling_rates: Optional[Union[Dict[OperationType, float], float]] = None
|
||||
):
|
||||
"""
|
||||
Initialize metrics buffer
|
||||
|
||||
Args:
|
||||
max_size: Maximum number of metrics to store
|
||||
sampling_rates: Either:
|
||||
- float: Global sampling rate for all operation types (0.0-1.0)
|
||||
- dict: Mapping operation type to sampling rate
|
||||
Default: 1.0 (100% sampling)
|
||||
"""
|
||||
self.max_size = max_size
|
||||
self._buffer: Deque[Metric] = deque(maxlen=max_size)
|
||||
self._lock = Lock()
|
||||
self._process_id = os.getpid()
|
||||
|
||||
# Handle different sampling_rates types
|
||||
if sampling_rates is None:
|
||||
# Default to 100% sampling for all types
|
||||
self._sampling_rates = {
|
||||
"database": 1.0,
|
||||
"http": 1.0,
|
||||
"render": 1.0,
|
||||
}
|
||||
elif isinstance(sampling_rates, (int, float)):
|
||||
# Global rate for all types
|
||||
rate = float(sampling_rates)
|
||||
self._sampling_rates = {
|
||||
"database": rate,
|
||||
"http": rate,
|
||||
"render": rate,
|
||||
}
|
||||
else:
|
||||
# Dict with per-type rates
|
||||
self._sampling_rates = sampling_rates
|
||||
```
|
||||
|
||||
#### Part 2: Fixed Configuration Reading
|
||||
**File:** `starpunk/monitoring/metrics.py:349-361`
|
||||
|
||||
Changed from plural to singular config key:
|
||||
|
||||
```python
|
||||
# Get configuration from Flask app if available
|
||||
try:
|
||||
from flask import current_app
|
||||
max_size = current_app.config.get('METRICS_BUFFER_SIZE', 1000)
|
||||
sampling_rate = current_app.config.get('METRICS_SAMPLING_RATE', 1.0) # Singular!
|
||||
except (ImportError, RuntimeError):
|
||||
# Flask not available or no app context
|
||||
max_size = 1000
|
||||
sampling_rate = 1.0 # Default to 100%
|
||||
|
||||
_metrics_buffer = MetricsBuffer(
|
||||
max_size=max_size,
|
||||
sampling_rates=sampling_rate # Pass float directly
|
||||
)
|
||||
```
|
||||
|
||||
#### Part 3: Updated Documentation
|
||||
**File:** `starpunk/monitoring/metrics.py:76-79`
|
||||
|
||||
Updated class docstring to reflect 100% default:
|
||||
```python
|
||||
Per developer Q&A Q12:
|
||||
- Configurable sampling rates per operation type
|
||||
- Default 100% sampling (suitable for low-traffic sites) # Changed from 10%
|
||||
- Slow queries always logged regardless of sampling
|
||||
```
|
||||
|
||||
### Design Decision: 100% Default Sampling
|
||||
Per architect review, changed default from 10% to 100% because:
|
||||
- StarPunk targets single-user, low-traffic deployments
|
||||
- 100% sampling has negligible overhead for typical usage
|
||||
- Ensures metrics are always visible (better UX)
|
||||
- Power users can reduce via `METRICS_SAMPLING_RATE` environment variable
|
||||
|
||||
### Verification
|
||||
- Monitoring tests: 28/28 passed (including sampling rate tests)
|
||||
- Database metrics now appear immediately
|
||||
- Backwards compatible (still accepts dict for per-type rates)
|
||||
- Config environment variable works correctly
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
### Core Fixes
|
||||
1. **`starpunk/monitoring/http.py`** (lines 73-86)
|
||||
- Added streaming response detection
|
||||
- Graceful fallback for response size calculation
|
||||
|
||||
2. **`starpunk/monitoring/metrics.py`** (multiple locations)
|
||||
- Added `Union` to type imports (line 29)
|
||||
- Updated `MetricsBuffer.__init__` signature (lines 87-125)
|
||||
- Updated class docstring (lines 76-79)
|
||||
- Fixed config key in `get_buffer()` (lines 349-361)
|
||||
|
||||
### Version & Documentation
|
||||
3. **`starpunk/__init__.py`** (line 301)
|
||||
- Updated version: `1.1.2-rc.1` → `1.1.2-rc.2`
|
||||
|
||||
4. **`CHANGELOG.md`**
|
||||
- Added v1.1.2-rc.2 section with fixes and changes
|
||||
|
||||
5. **`docs/reports/2025-11-28-v1.1.2-rc.2-fixes.md`** (this file)
|
||||
- Comprehensive implementation report
|
||||
|
||||
---
|
||||
|
||||
## Test Results
|
||||
|
||||
### Targeted Testing
|
||||
```bash
|
||||
uv run pytest tests/test_monitoring.py -v
|
||||
```
|
||||
**Result:** 28 passed in 18.13s
|
||||
|
||||
All monitoring-related tests passed, including:
|
||||
- HTTP metrics recording
|
||||
- Database metrics recording
|
||||
- Sampling rate configuration
|
||||
- Memory monitoring
|
||||
- Business metrics tracking
|
||||
|
||||
### Key Tests Verified
|
||||
- `test_setup_http_metrics` - HTTP middleware setup
|
||||
- `test_execute_records_metric` - Database metrics recording
|
||||
- `test_sampling_rate_configurable` - Config key fix
|
||||
- `test_slow_query_always_recorded` - Force recording bypass
|
||||
- All HTTP, database, and memory monitor tests
|
||||
|
||||
---
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
- [x] Issue 1 (Static Files) fixed - streaming response handling
|
||||
- [x] Issue 2 (Database Metrics) fixed - config key mismatch
|
||||
- [x] Version number updated to 1.1.2-rc.2
|
||||
- [x] CHANGELOG.md updated with fixes
|
||||
- [x] All monitoring tests pass (28/28)
|
||||
- [x] Backwards compatible (dict sampling rates still work)
|
||||
- [x] Default sampling changed from 10% to 100%
|
||||
- [x] Implementation report created
|
||||
|
||||
---
|
||||
|
||||
## Production Deployment Notes
|
||||
|
||||
### Expected Behavior After Deployment
|
||||
1. **Static files will load immediately** - no more 500 errors
|
||||
2. **Database metrics will show non-zero values immediately** - 100% sampling
|
||||
3. **Existing config still works** - backwards compatible
|
||||
|
||||
### Configuration
|
||||
Users can adjust sampling if needed:
|
||||
```bash
|
||||
# Reduce sampling for high-traffic sites
|
||||
METRICS_SAMPLING_RATE=0.1 # 10% sampling
|
||||
|
||||
# Or disable metrics entirely
|
||||
METRICS_ENABLED=false
|
||||
```
|
||||
|
||||
### Rollback Plan
|
||||
If issues arise:
|
||||
1. Revert to v1.1.2-rc.1 (will restore static file error)
|
||||
2. Or revert to v1.1.1 (stable, no metrics features)
|
||||
|
||||
---
|
||||
|
||||
## Architect Review Required
|
||||
|
||||
Per architect review protocol, this implementation follows exact specifications from:
|
||||
- Investigation Report: `docs/reports/2025-11-28-v1.1.2-rc.1-production-issues.md`
|
||||
- Architect Review: `docs/reviews/2025-11-28-v1.1.2-rc.1-architect-review.md`
|
||||
|
||||
All fixes implemented as specified. No design decisions made independently.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Deploy v1.1.2-rc.2 to production**
|
||||
2. **Monitor for 24 hours** - verify both fixes work
|
||||
3. **If stable, tag as v1.1.2** (remove -rc suffix)
|
||||
4. **Update deployment documentation** with new sampling rate defaults
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- Investigation Report: `docs/reports/2025-11-28-v1.1.2-rc.1-production-issues.md`
|
||||
- Architect Review: `docs/reviews/2025-11-28-v1.1.2-rc.1-architect-review.md`
|
||||
- ADR-053: Performance Monitoring System
|
||||
- v1.1.2 Implementation Plan: `docs/projectplan/v1.1.2-implementation-plan.md`
|
||||
Reference in New Issue
Block a user