feat(tags): Add database schema and tags module (v1.3.0 Phase 1)
Implements tag/category system backend following microformats2 p-category specification. Database changes: - Migration 008: Add tags and note_tags tables - Normalized tag storage (case-insensitive lookup, display name preserved) - Indexes for performance New module: - starpunk/tags.py: Tag management functions - normalize_tag: Normalize tag strings - get_or_create_tag: Get or create tag records - add_tags_to_note: Associate tags with notes (replaces existing) - get_note_tags: Retrieve note tags (alphabetically ordered) - get_tag_by_name: Lookup tag by normalized name - get_notes_by_tag: Get all notes with specific tag - parse_tag_input: Parse comma-separated tag input Model updates: - Note.tags property (lazy-loaded, prefer pre-loading in routes) - Note.to_dict() add include_tags parameter CRUD updates: - create_note() accepts tags parameter - update_note() accepts tags parameter (None = no change, [] = remove all) Micropub integration: - Pass tags to create_note() (tags already extracted by extract_tags()) - Return tags in q=source response Per design doc: docs/design/v1.3.0/microformats-tags-design.md Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
315
docs/design/v1.1.1/upgrade-to-v1.1.1.md
Normal file
315
docs/design/v1.1.1/upgrade-to-v1.1.1.md
Normal file
@@ -0,0 +1,315 @@
|
||||
# Upgrade Guide: StarPunk v1.1.1 "Polish"
|
||||
|
||||
**Release Date**: 2025-11-25
|
||||
**Previous Version**: v1.1.0
|
||||
**Target Version**: v1.1.1
|
||||
|
||||
## Overview
|
||||
|
||||
StarPunk v1.1.1 "Polish" is a maintenance release focused on production readiness, performance optimization, and operational improvements. This release is **100% backward compatible** with v1.1.0 - no breaking changes.
|
||||
|
||||
### Key Improvements
|
||||
|
||||
- **RSS Memory Optimization**: Streaming feed generation for large feeds
|
||||
- **Performance Monitoring**: MetricsBuffer with database pool statistics
|
||||
- **Enhanced Health Checks**: Three-tier health check system
|
||||
- **Search Improvements**: FTS5 fallback and result highlighting
|
||||
- **Unicode Slug Support**: Better international character handling
|
||||
- **Admin Dashboard**: Visual metrics and monitoring interface
|
||||
- **Memory Monitoring**: Background thread for system metrics
|
||||
- **Logging Improvements**: Proper log rotation verification
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before upgrading:
|
||||
|
||||
1. **Backup your data**:
|
||||
```bash
|
||||
# Backup database
|
||||
cp data/starpunk.db data/starpunk.db.backup
|
||||
|
||||
# Backup notes
|
||||
cp -r data/notes data/notes.backup
|
||||
```
|
||||
|
||||
2. **Check current version**:
|
||||
```bash
|
||||
uv run python -c "import starpunk; print(starpunk.__version__)"
|
||||
```
|
||||
|
||||
3. **Review changelog**: Read `CHANGELOG.md` for detailed changes
|
||||
|
||||
## Upgrade Steps
|
||||
|
||||
### Step 1: Stop StarPunk
|
||||
|
||||
If running in production:
|
||||
|
||||
```bash
|
||||
# For systemd service
|
||||
sudo systemctl stop starpunk
|
||||
|
||||
# For container deployment
|
||||
podman stop starpunk # or docker stop starpunk
|
||||
```
|
||||
|
||||
### Step 2: Pull Latest Code
|
||||
|
||||
```bash
|
||||
# From git repository
|
||||
git fetch origin
|
||||
git checkout v1.1.1
|
||||
|
||||
# Or download release tarball
|
||||
wget https://github.com/YOUR_USERNAME/starpunk/archive/v1.1.1.tar.gz
|
||||
tar xzf v1.1.1.tar.gz
|
||||
cd starpunk-1.1.1
|
||||
```
|
||||
|
||||
### Step 3: Update Dependencies
|
||||
|
||||
```bash
|
||||
# Update Python dependencies with uv
|
||||
uv sync
|
||||
```
|
||||
|
||||
### Step 4: Verify Configuration
|
||||
|
||||
No new required configuration variables in v1.1.1, but you can optionally configure new features:
|
||||
|
||||
```bash
|
||||
# Optional: Adjust feed caching (default: 300 seconds)
|
||||
export FEED_CACHE_SECONDS=300
|
||||
|
||||
# Optional: Adjust database pool size (default: 5)
|
||||
export DB_POOL_SIZE=5
|
||||
|
||||
# Optional: Adjust metrics sampling rates
|
||||
export METRICS_SAMPLING_DATABASE=1.0
|
||||
export METRICS_SAMPLING_HTTP=0.1
|
||||
export METRICS_SAMPLING_RENDER=0.1
|
||||
```
|
||||
|
||||
### Step 5: Run Database Migrations
|
||||
|
||||
StarPunk uses automatic migrations - no manual SQL needed:
|
||||
|
||||
```bash
|
||||
# Migrations run automatically on startup
|
||||
# Verify migration status:
|
||||
uv run python -c "from starpunk.database import init_db; init_db()"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
INFO [init]: Database initialized: data/starpunk.db
|
||||
INFO [init]: No pending migrations
|
||||
INFO [init]: Database connection pool initialized (size=5)
|
||||
```
|
||||
|
||||
### Step 6: Verify Installation
|
||||
|
||||
Run the test suite to ensure everything works:
|
||||
|
||||
```bash
|
||||
# Run tests (should see 600+ tests passing)
|
||||
uv run pytest
|
||||
```
|
||||
|
||||
### Step 7: Restart StarPunk
|
||||
|
||||
```bash
|
||||
# For systemd service
|
||||
sudo systemctl start starpunk
|
||||
sudo systemctl status starpunk
|
||||
|
||||
# For container deployment
|
||||
podman start starpunk # or docker start starpunk
|
||||
podman logs -f starpunk
|
||||
```
|
||||
|
||||
### Step 8: Verify Upgrade
|
||||
|
||||
1. **Check version**:
|
||||
```bash
|
||||
curl https://your-domain.com/health
|
||||
```
|
||||
Should show version "1.1.1"
|
||||
|
||||
2. **Test admin dashboard**:
|
||||
- Log in to admin interface
|
||||
- Navigate to "Metrics" tab
|
||||
- Verify charts and statistics display correctly
|
||||
|
||||
3. **Test RSS feed**:
|
||||
```bash
|
||||
curl https://your-domain.com/feed.xml | head -20
|
||||
```
|
||||
Should return valid XML with streaming response
|
||||
|
||||
4. **Check logs**:
|
||||
```bash
|
||||
tail -f data/logs/starpunk.log
|
||||
```
|
||||
Should show clean startup with no errors
|
||||
|
||||
## New Features
|
||||
|
||||
### Admin Metrics Dashboard
|
||||
|
||||
Access the new metrics dashboard at `/admin/dashboard`:
|
||||
|
||||
- Real-time performance metrics
|
||||
- Database connection pool statistics
|
||||
- Auto-refresh every 10 seconds (requires JavaScript)
|
||||
- Progressive enhancement (works without JavaScript)
|
||||
- Charts powered by Chart.js
|
||||
|
||||
### RSS Feed Optimization
|
||||
|
||||
The RSS feed now uses streaming for better memory efficiency:
|
||||
|
||||
- Memory usage reduced from O(n) to O(1)
|
||||
- Lower time-to-first-byte for large feeds
|
||||
- Cache stores note list, not full XML
|
||||
- Transparent to clients (no API changes)
|
||||
|
||||
### Enhanced Health Checks
|
||||
|
||||
Three tiers of health checks available:
|
||||
|
||||
1. **Basic** (`/health`): Public, minimal response
|
||||
2. **Detailed** (`/health?detailed=true`): Authenticated, comprehensive
|
||||
3. **Full Diagnostics** (`/admin/health`): Authenticated, includes metrics
|
||||
|
||||
### Search Improvements
|
||||
|
||||
- FTS5 detection at startup
|
||||
- Graceful fallback to LIKE queries if FTS5 unavailable
|
||||
- Search result highlighting with XSS prevention
|
||||
|
||||
### Unicode Slug Support
|
||||
|
||||
- Unicode normalization (NFKD) for international characters
|
||||
- Timestamp-based fallback for untranslatable text
|
||||
- Never fails Micropub requests due to slug issues
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
### No Breaking Changes
|
||||
|
||||
All existing configuration continues to work. New optional variables:
|
||||
|
||||
```bash
|
||||
# Performance tuning (all optional)
|
||||
FEED_CACHE_SECONDS=300 # RSS feed cache duration
|
||||
DB_POOL_SIZE=5 # Database connection pool size
|
||||
METRICS_SAMPLING_DATABASE=1.0 # Sample 100% of DB operations
|
||||
METRICS_SAMPLING_HTTP=0.1 # Sample 10% of HTTP requests
|
||||
METRICS_SAMPLING_RENDER=0.1 # Sample 10% of template renders
|
||||
```
|
||||
|
||||
### Removed Configuration
|
||||
|
||||
None. All v1.1.0 configuration variables continue to work.
|
||||
|
||||
## Rollback Procedure
|
||||
|
||||
If you encounter issues, rollback to v1.1.0:
|
||||
|
||||
### Step 1: Stop StarPunk
|
||||
|
||||
```bash
|
||||
sudo systemctl stop starpunk # or podman/docker stop
|
||||
```
|
||||
|
||||
### Step 2: Restore Previous Version
|
||||
|
||||
```bash
|
||||
# Restore from git
|
||||
git checkout v1.1.0
|
||||
|
||||
# Or restore from backup
|
||||
cd /path/to/backup
|
||||
cp -r starpunk-1.1.0/* /path/to/starpunk/
|
||||
```
|
||||
|
||||
### Step 3: Restore Database (if needed)
|
||||
|
||||
```bash
|
||||
# Only if database issues occurred
|
||||
cp data/starpunk.db.backup data/starpunk.db
|
||||
```
|
||||
|
||||
### Step 4: Restart
|
||||
|
||||
```bash
|
||||
sudo systemctl start starpunk
|
||||
```
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Issue: Log Rotation Not Working
|
||||
|
||||
**Symptom**: Log files growing unbounded
|
||||
|
||||
**Solution**:
|
||||
1. Check log file permissions
|
||||
2. Verify `data/logs/` directory exists
|
||||
3. Check `LOG_LEVEL` configuration
|
||||
4. See `docs/operations/troubleshooting.md`
|
||||
|
||||
### Issue: Metrics Dashboard Not Loading
|
||||
|
||||
**Symptom**: 404 or blank metrics page
|
||||
|
||||
**Solution**:
|
||||
1. Clear browser cache
|
||||
2. Verify you're logged in as admin
|
||||
3. Check browser console for JavaScript errors
|
||||
4. Verify htmx and Chart.js CDN accessible
|
||||
|
||||
### Issue: RSS Feed Validation Errors
|
||||
|
||||
**Symptom**: Feed validators report errors
|
||||
|
||||
**Solution**:
|
||||
1. Streaming implementation is RSS 2.0 compliant
|
||||
2. Verify XML structure with validator
|
||||
3. Check for special characters in note content
|
||||
4. See `docs/operations/troubleshooting.md`
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
See `docs/operations/performance-tuning.md` for detailed guidance on:
|
||||
|
||||
- Database pool sizing
|
||||
- Metrics sampling rates
|
||||
- Cache configuration
|
||||
- Log rotation settings
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues:
|
||||
|
||||
1. Check `docs/operations/troubleshooting.md`
|
||||
2. Review logs in `data/logs/starpunk.log`
|
||||
3. Run health checks: `curl /admin/health`
|
||||
4. File issue on GitHub with logs and configuration
|
||||
|
||||
## Next Steps
|
||||
|
||||
After upgrading:
|
||||
|
||||
1. **Review new metrics**: Check `/admin/dashboard` regularly
|
||||
2. **Adjust sampling**: Tune metrics sampling for your workload
|
||||
3. **Monitor performance**: Use health endpoints for monitoring
|
||||
4. **Update documentation**: Review operational guides
|
||||
5. **Plan for v1.2.0**: Review roadmap for upcoming features
|
||||
|
||||
## Version History
|
||||
|
||||
- **v1.1.1 (2025-11-25)**: Polish release (current)
|
||||
- **v1.1.0 (2025-11-25)**: Search and custom slugs
|
||||
- **v1.0.1 (2025-11-25)**: Bug fixes
|
||||
- **v1.0.0 (2025-11-24)**: First production release
|
||||
Reference in New Issue
Block a user