StarPunk/docs/operations/troubleshooting.md

StarPunk Troubleshooting Guide

Version: 1.1.1 Last Updated: 2025-11-25

This guide helps diagnose and resolve common issues with StarPunk.

Quick Diagnostics

Check System Health

# Basic health check
curl http://localhost:5000/health

# Detailed health check (requires authentication)
curl -H "Authorization: Bearer YOUR_TOKEN" \
  http://localhost:5000/health?detailed=true

# Full diagnostics
curl -H "Authorization: Bearer YOUR_TOKEN" \
  http://localhost:5000/admin/health

Check Logs

# View recent logs
tail -f data/logs/starpunk.log

# Search for errors
grep ERROR data/logs/starpunk.log | tail -20

# Search for warnings
grep WARNING data/logs/starpunk.log | tail -20

Check Database

# Verify database exists and is accessible
ls -lh data/starpunk.db

# Check database integrity
sqlite3 data/starpunk.db "PRAGMA integrity_check;"

# Check migrations
sqlite3 data/starpunk.db "SELECT * FROM schema_migrations;"

Common Issues

Application Won't Start

Symptom

StarPunk fails to start or crashes immediately.

Possible Causes

1. Missing configuration

   # Check required environment variables
   echo $SITE_URL
   echo $SITE_NAME
   echo $ADMIN_ME
   

   Solution: Set all required variables in .env:

   SITE_URL=https://your-domain.com/
   SITE_NAME=Your Site Name
   ADMIN_ME=https://your-domain.com/
   

2. Database locked

   # Check for other processes
   lsof data/starpunk.db
   

   Solution: Stop other StarPunk instances or wait for lock release

3. Permission issues

   # Check permissions
   ls -ld data/
   ls -l data/starpunk.db
   

   Solution: Fix permissions:

   chmod 755 data/
   chmod 644 data/starpunk.db
   

4. Missing dependencies

   # Re-sync dependencies
   uv sync
   

Database Connection Errors

Symptom

Errors like "database is locked" or "unable to open database file"

Solutions

1. Check database path

   # Verify DATABASE_PATH in config
   echo $DATABASE_PATH
   ls -l $DATABASE_PATH
   

2. Check file permissions

   # Database file needs write permission
   chmod 644 data/starpunk.db
   chmod 755 data/
   

3. Check disk space

   df -h
   

4. Check connection pool

   # View pool statistics
   curl http://localhost:5000/admin/metrics | jq '.database.pool'
   

   If pool is exhausted, increase DB_POOL_SIZE:

   export DB_POOL_SIZE=10
   

IndieAuth Login Fails

Symptom

Cannot log in to admin interface, redirects fail, or authentication errors.

Solutions

1. Check ADMIN_ME configuration

   echo $ADMIN_ME
   

   Must be a valid URL that matches your identity.

2. Check IndieAuth endpoints

   # Verify endpoints are discoverable
   curl -I $ADMIN_ME | grep Link
   

   Should show authorization_endpoint and token_endpoint.

3. Check callback URL

   • Verify /auth/callback is accessible
   • Check for HTTPS in production
   • Verify no trailing slash issues

4. Check session secret

   echo $SESSION_SECRET
   

   Must be set and persistent across restarts.

RSS Feed Issues

Symptom

Feed not displaying, validation errors, or empty feed.

Solutions

1. Check feed endpoint

   curl http://localhost:5000/feed.xml | head -50
   

2. Verify published notes

   sqlite3 data/starpunk.db \
     "SELECT COUNT(*) FROM notes WHERE published=1;"
   

3. Check feed cache

   # Clear cache by restarting
   # Cache duration controlled by FEED_CACHE_SECONDS
   

4. Validate feed

   curl http://localhost:5000/feed.xml | \
     xmllint --format - | head -100
   

Search Not Working

Symptom

Search returns no results or errors.

Solutions

1. Check FTS5 availability

   sqlite3 data/starpunk.db \
     "SELECT COUNT(*) FROM notes_fts;"
   

2. Rebuild search index

   uv run python -c "from starpunk.search import rebuild_fts_index; \
     rebuild_fts_index('data/starpunk.db', 'data')"
   

3. Check for FTS5 support

   sqlite3 data/starpunk.db \
     "PRAGMA compile_options;" | grep FTS5
   

   If not available, StarPunk will fall back to LIKE queries automatically.

Performance Issues

Symptom

Slow response times, high memory usage, or timeouts.

Diagnostics

1. Check performance metrics

   curl http://localhost:5000/admin/metrics | jq '.performance'
   

2. Check database pool

   curl http://localhost:5000/admin/metrics | jq '.database.pool'
   

3. Check system resources

   # Memory usage
   ps aux | grep starpunk
   
   # Disk usage
   df -h
   
   # Open files
   lsof -p $(pgrep -f starpunk)
   

Solutions

1. Increase connection pool

   export DB_POOL_SIZE=10
   

2. Adjust metrics sampling

   # Reduce sampling for high-traffic sites
   export METRICS_SAMPLING_HTTP=0.01  # 1% sampling
   export METRICS_SAMPLING_RENDER=0.01
   

3. Increase cache duration

   export FEED_CACHE_SECONDS=600  # 10 minutes
   

4. Check slow queries

   grep "SLOW" data/logs/starpunk.log
   

Log Rotation Not Working

Symptom

Log files growing unbounded, disk space issues.

Solutions

1. Check log directory

   ls -lh data/logs/
   

2. Verify log rotation configuration

   • RotatingFileHandler configured for 10MB files
   • Keeps 10 backup files
   • Automatic rotation on size limit

3. Manual log rotation

   # Backup and truncate
   mv data/logs/starpunk.log data/logs/starpunk.log.old
   touch data/logs/starpunk.log
   chmod 644 data/logs/starpunk.log
   

4. Check permissions

   ls -l data/logs/
   chmod 755 data/logs/
   chmod 644 data/logs/*.log
   

Metrics Dashboard Not Loading

Symptom

Blank dashboard, 404 errors, or JavaScript errors.

Solutions

1. Check authentication

   • Must be logged in as admin
   • Navigate to /admin/dashboard

2. Check JavaScript console

   • Open browser developer tools
   • Look for CDN loading errors
   • Verify htmx and Chart.js load

3. Check network connectivity

   # Test CDN access
   curl -I https://unpkg.com/htmx.org@1.9.10
   curl -I https://cdn.jsdelivr.net/npm/chart.js@4.4.0/dist/chart.umd.min.js
   

4. Test metrics endpoint

   curl http://localhost:5000/admin/metrics
   

Log File Locations

• Application logs: data/logs/starpunk.log
• Rotated logs: data/logs/starpunk.log.1 through starpunk.log.10
• Container logs: podman logs starpunk or docker logs starpunk
• System logs: /var/log/syslog or journalctl -u starpunk

Health Check Interpretation

Basic Health (/health)

{
  "status": "healthy"
}

• healthy: All systems operational
• unhealthy: Critical issues detected

Detailed Health (/health?detailed=true)

{
  "status": "healthy",
  "version": "1.1.1",
  "checks": {
    "database": {"status": "healthy"},
    "filesystem": {"status": "healthy"},
    "fts_index": {"status": "healthy"}
  }
}

Check each component status individually.

Full Diagnostics (/admin/health)

Includes all above plus:

• Performance metrics
• Database pool statistics
• System resource usage
• Error budget status

Performance Monitoring Tips

Normal Metrics

• Database queries: avg < 50ms
• HTTP requests: avg < 200ms
• Template rendering: avg < 50ms
• Pool usage: < 80% connections active

Warning Signs

• Database: avg > 100ms consistently
• HTTP: avg > 500ms
• Pool: 100% connections active
• Memory: continuous growth

Metrics Sampling

Adjust sampling rates based on traffic:

# Low traffic (< 100 req/day)
METRICS_SAMPLING_DATABASE=1.0
METRICS_SAMPLING_HTTP=1.0
METRICS_SAMPLING_RENDER=1.0

# Medium traffic (100-1000 req/day)
METRICS_SAMPLING_DATABASE=1.0
METRICS_SAMPLING_HTTP=0.1
METRICS_SAMPLING_RENDER=0.1

# High traffic (> 1000 req/day)
METRICS_SAMPLING_DATABASE=0.1
METRICS_SAMPLING_HTTP=0.01
METRICS_SAMPLING_RENDER=0.01

Database Pool Issues

Pool Exhaustion

Symptom: "No available connections" errors

Solution:

# Increase pool size
export DB_POOL_SIZE=10

# Or reduce request concurrency

Pool Leaks

Symptom: Connections not returned to pool

Check:

curl http://localhost:5000/admin/metrics | \
  jq '.database.pool'

Look for high active_connections that don't decrease.

Solution: Restart application to reset pool

Getting Help

Before Filing an Issue

1. Check this troubleshooting guide
2. Review logs for specific errors
3. Run health checks
4. Try with minimal configuration
5. Search existing issues

Information to Include

When filing an issue, include:

1. Version: uv run python -c "import starpunk; print(starpunk.__version__)"
2. Environment: Development or production
3. Configuration: Sanitized .env (remove secrets)
4. Logs: Recent errors from data/logs/starpunk.log
5. Health check: Output from /admin/health
6. Steps to reproduce: Exact commands that trigger the issue

Debug Mode

Enable verbose logging:

export LOG_LEVEL=DEBUG
# Restart StarPunk

WARNING: Debug logs may contain sensitive information. Don't share publicly.

Emergency Recovery

Complete Reset (DESTRUCTIVE)

WARNING: This deletes all data.

# Stop StarPunk
sudo systemctl stop starpunk

# Backup everything
cp -r data data.backup.$(date +%Y%m%d)

# Remove database
rm data/starpunk.db

# Remove logs
rm -rf data/logs/

# Restart (will reinitialize)
sudo systemctl start starpunk

Restore from Backup

# Stop StarPunk
sudo systemctl stop starpunk

# Restore database
cp data.backup/starpunk.db data/

# Restore notes
cp -r data.backup/notes/* data/notes/

# Restart
sudo systemctl start starpunk

Related Documentation

• /docs/operations/upgrade-to-v1.1.1.md - Upgrade procedures
• /docs/operations/performance-tuning.md - Optimization guide
• /docs/architecture/overview.md - System architecture
• CHANGELOG.md - Version history and changes