that initial commit

2025-11-18 19:21:31 -07:00
commit a68fd570c7
69 changed files with 31070 additions and 0 deletions
--- a/docs/standards/development-setup.md
+++ b/docs/standards/development-setup.md
@@ -0,0 +1,814 @@
+# Development Setup Standards
+
+## Purpose
+
+This document defines the standard development setup procedure for StarPunk. It provides step-by-step instructions for developer agents and humans to create a fully functional development environment from scratch.
+
+## Prerequisites
+
+### System Requirements
+
+- **Operating System**: Linux, macOS, or Windows with WSL2
+- **Python**: 3.11 or higher
+- **Disk Space**: 500MB minimum
+- **Network**: Internet connection for package downloads
+
+### Required Tools
+
+- **Python 3.11+**: System Python installation
+- **uv**: Python package and environment manager
+- **git**: Version control (for cloning repository)
+
+## Installation Steps
+
+### Step 1: Verify Python Installation
+
+```bash
+# Check Python version
+python3 --version
+
+# Expected output: Python 3.11.x or higher
+```
+
+**If Python 3.11+ is not installed**:
+
+**Ubuntu/Debian**:
+```bash
+sudo apt update
+sudo apt install python3.11 python3.11-venv
+```
+
+**macOS**:
+```bash
+brew install python@3.11
+```
+
+**Arch Linux**:
+```bash
+sudo pacman -S python
+```
+
+---
+
+### Step 2: Install uv
+
+**Linux/macOS (Recommended)**:
+```bash
+# Install via official installer
+curl -LsSf https://astral.sh/uv/install.sh | sh
+
+# Verify installation
+uv --version
+```
+
+**Alternative (via pip)**:
+```bash
+# Install uv using pip
+pip install --user uv
+
+# Verify installation
+uv --version
+```
+
+**Windows**:
+```powershell
+# Using PowerShell
+powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
+
+# Verify installation
+uv --version
+```
+
+**Expected Output**: `uv 0.x.x` (any recent version)
+
+See [ADR-006](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md) for full uv standards.
+
+---
+
+### Step 3: Clone Repository
+
+```bash
+# Clone the repository
+git clone https://github.com/YOUR_USERNAME/starpunk.git
+cd starpunk
+
+# Verify you're in the correct directory
+pwd
+# Expected: /path/to/starpunk
+
+# List files to confirm
+ls -la
+# Should see: app.py, requirements.txt, starpunk/, etc.
+```
+
+**If starting from scratch without git**:
+Create the project structure manually following [Project Structure Design](/home/phil/Projects/starpunk/docs/design/project-structure.md).
+
+---
+
+### Step 4: Create Virtual Environment
+
+```bash
+# Create virtual environment using uv
+uv venv .venv --python 3.11
+
+# Verify creation
+ls -la .venv
+
+# Expected: bin/, lib/, include/, pyvenv.cfg
+```
+
+**Verification**:
+```bash
+# Check Python executable exists
+.venv/bin/python --version
+
+# Expected: Python 3.11.x
+```
+
+**Important**: Virtual environment is created in `.venv/` directory at project root.
+
+---
+
+### Step 5: Install Dependencies
+
+```bash
+# Install all production dependencies
+uv pip install -r requirements.txt
+
+# Verify installation
+uv pip list
+
+# Expected output should include:
+# - Flask (3.0.x)
+# - markdown (3.5.x)
+# - feedgen (1.0.x)
+# - httpx (0.27.x)
+# - python-dotenv (1.0.x)
+# - pytest (8.0.x)
+```
+
+**For Development** (optional):
+```bash
+# Install development dependencies
+uv pip install -r requirements-dev.txt
+
+# Includes: pytest-cov, black, flake8, mypy
+```
+
+**Troubleshooting**:
+- If installation fails, check internet connection
+- Verify uv is installed correctly: `which uv`
+- Check Python version in venv: `.venv/bin/python --version`
+
+---
+
+### Step 6: Configure Environment
+
+```bash
+# Copy environment template
+cp .env.example .env
+
+# Edit .env file with your settings
+nano .env  # or vim, code, etc.
+```
+
+**Required Configuration** (edit `.env`):
+
+```bash
+# Site Configuration
+SITE_URL=http://localhost:5000
+SITE_NAME=My StarPunk Site
+SITE_AUTHOR=Your Name
+
+# Admin Authentication
+ADMIN_ME=https://your-website.com
+# ^ Replace with YOUR IndieWeb identity URL
+
+# Session Security
+SESSION_SECRET=GENERATE_RANDOM_SECRET_HERE
+# ^ Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
+
+# Data Paths
+DATA_PATH=./data
+NOTES_PATH=./data/notes
+DATABASE_PATH=./data/starpunk.db
+
+# Flask Configuration
+FLASK_ENV=development
+FLASK_DEBUG=1
+```
+
+**Generate SESSION_SECRET**:
+```bash
+# Generate cryptographically secure secret
+python3 -c "import secrets; print(secrets.token_hex(32))"
+
+# Copy output and paste into .env as SESSION_SECRET value
+```
+
+**Important**:
+- **NEVER commit `.env` file** to git
+- `.env.example` is a template and SHOULD be committed
+- Each developer/deployment has their own `.env`
+
+---
+
+### Step 7: Initialize Database
+
+```bash
+# Create data directory structure
+mkdir -p data/notes
+
+# Initialize database schema
+.venv/bin/python -c "from starpunk.database import init_db; init_db()"
+
+# Verify database created
+ls -la data/
+
+# Expected: starpunk.db file should exist
+```
+
+**Alternative** (if Flask CLI is set up):
+```bash
+# Initialize via Flask CLI
+.venv/bin/flask db init
+```
+
+**Verify Database Schema**:
+```bash
+# Check tables exist
+sqlite3 data/starpunk.db ".tables"
+
+# Expected output: notes, sessions, tokens, auth_state
+```
+
+---
+
+### Step 8: Verify Installation
+
+```bash
+# Run all verification checks
+.venv/bin/python -c "
+from starpunk.config import load_config
+from starpunk.database import verify_db
+import flask
+print('✓ Flask version:', flask.__version__)
+print('✓ Config loads successfully')
+print('✓ Database schema verified')
+print('✓ Installation complete!')
+"
+```
+
+**Manual Verification Checklist**:
+```bash
+# Check virtual environment
+[ -d ".venv" ] && echo "✓ Virtual environment exists" || echo "✗ Missing .venv"
+
+# Check Python version
+.venv/bin/python --version | grep "3.11" && echo "✓ Python 3.11+" || echo "✗ Wrong Python version"
+
+# Check Flask installed
+.venv/bin/python -c "import flask" && echo "✓ Flask installed" || echo "✗ Flask missing"
+
+# Check .env file exists
+[ -f ".env" ] && echo "✓ .env configured" || echo "✗ Missing .env"
+
+# Check data directory
+[ -d "data/notes" ] && echo "✓ Data directory exists" || echo "✗ Missing data/"
+
+# Check database
+[ -f "data/starpunk.db" ] && echo "✓ Database initialized" || echo "✗ Missing database"
+```
+
+---
+
+### Step 9: Run Development Server
+
+```bash
+# Run Flask development server
+.venv/bin/flask --app app.py run --debug
+
+# Alternative: Run directly
+.venv/bin/python app.py
+```
+
+**Expected Output**:
+```
+ * Serving Flask app 'app.py'
+ * Debug mode: on
+WARNING: This is a development server. Do not use it in a production deployment.
+ * Running on http://127.0.0.1:5000
+Press CTRL+C to quit
+```
+
+**Access Application**:
+- Open browser to: `http://localhost:5000`
+- Should see StarPunk homepage
+- No notes will exist on first run
+
+**Stop Server**:
+- Press `Ctrl+C` in terminal
+
+---
+
+### Step 10: Run Tests
+
+```bash
+# Run all tests
+.venv/bin/pytest
+
+# Run with coverage report
+.venv/bin/pytest --cov=starpunk tests/
+
+# Run specific test file
+.venv/bin/pytest tests/test_database.py
+
+# Run with verbose output
+.venv/bin/pytest -v
+```
+
+**Expected Output** (once tests are written):
+```
+======================== test session starts =========================
+collected 45 items
+
+tests/test_auth.py ........                                    [ 17%]
+tests/test_database.py .......                                 [ 33%]
+tests/test_feed.py .....                                       [ 44%]
+tests/test_micropub.py ..........                              [ 66%]
+tests/test_notes.py ..........                                 [ 88%]
+tests/test_utils.py .....                                      [100%]
+
+========================= 45 passed in 2.34s =========================
+```
+
+---
+
+## Configuration Management
+
+### Environment Variables
+
+All configuration is managed via environment variables loaded from `.env` file.
+
+**Configuration Categories**:
+
+1. **Site Identity**
+   - `SITE_URL` - Public URL of site
+   - `SITE_NAME` - Site title
+   - `SITE_AUTHOR` - Author name
+   - `SITE_DESCRIPTION` - Site description (for RSS)
+
+2. **Authentication**
+   - `ADMIN_ME` - Admin's IndieWeb identity URL
+   - `SESSION_SECRET` - Secret key for session signing
+   - `SESSION_LIFETIME` - Session duration in days (default: 30)
+
+3. **Data Storage**
+   - `DATA_PATH` - Base data directory (default: `./data`)
+   - `NOTES_PATH` - Notes directory (default: `./data/notes`)
+   - `DATABASE_PATH` - SQLite database path (default: `./data/starpunk.db`)
+
+4. **Flask Settings**
+   - `FLASK_ENV` - Environment: `development` or `production`
+   - `FLASK_DEBUG` - Debug mode: `1` (on) or `0` (off)
+   - `FLASK_SECRET_KEY` - Falls back to `SESSION_SECRET`
+
+5. **External Services**
+   - `INDIELOGIN_URL` - IndieLogin service URL (default: `https://indielogin.com`)
+
+### Configuration Loading
+
+Configuration is loaded in this priority order:
+
+1. **Environment variables** (highest priority)
+2. **`.env` file** (loaded via python-dotenv)
+3. **Default values** (in `config.py`)
+
+**Example** (`starpunk/config.py`):
+```python
+import os
+from pathlib import Path
+from dotenv import load_dotenv
+
+# Load .env file
+load_dotenv()
+
+class Config:
+    # Site
+    SITE_URL = os.getenv('SITE_URL', 'http://localhost:5000')
+    SITE_NAME = os.getenv('SITE_NAME', 'StarPunk')
+
+    # Security
+    SECRET_KEY = os.getenv('SESSION_SECRET')
+    if not SECRET_KEY:
+        raise ValueError("SESSION_SECRET must be set in .env")
+
+    # Data paths
+    DATA_PATH = Path(os.getenv('DATA_PATH', './data'))
+    NOTES_PATH = Path(os.getenv('NOTES_PATH', './data/notes'))
+    DATABASE_PATH = Path(os.getenv('DATABASE_PATH', './data/starpunk.db'))
+
+    # Flask
+    DEBUG = os.getenv('FLASK_DEBUG', '1') == '1'
+    ENV = os.getenv('FLASK_ENV', 'development')
+```
+
+### Development vs Production Settings
+
+**Development** (`.env`):
+```bash
+FLASK_ENV=development
+FLASK_DEBUG=1
+SITE_URL=http://localhost:5000
+```
+
+**Production** (`.env`):
+```bash
+FLASK_ENV=production
+FLASK_DEBUG=0
+SITE_URL=https://your-domain.com
+```
+
+**Key Differences**:
+- **Debug mode**: Development has detailed errors, production hides them
+- **URL**: Development uses localhost, production uses actual domain
+- **HTTPS**: Production requires HTTPS, development doesn't
+- **Session cookies**: Production sets `Secure` flag, development doesn't
+
+---
+
+## Common Development Tasks
+
+### Starting Development
+
+```bash
+# 1. Activate virtual environment context (via uv)
+cd /path/to/starpunk
+
+# 2. Run development server
+.venv/bin/flask --app app.py run --debug
+
+# Server runs on http://localhost:5000
+```
+
+### Running Tests
+
+```bash
+# All tests
+.venv/bin/pytest
+
+# Specific module
+.venv/bin/pytest tests/test_notes.py
+
+# Specific test
+.venv/bin/pytest tests/test_notes.py::test_create_note
+
+# With coverage
+.venv/bin/pytest --cov=starpunk --cov-report=html tests/
+# Open htmlcov/index.html to view coverage report
+```
+
+### Code Formatting
+
+```bash
+# Format all Python code
+.venv/bin/black starpunk/ tests/
+
+# Check formatting without changing
+.venv/bin/black --check starpunk/ tests/
+
+# Format specific file
+.venv/bin/black starpunk/notes.py
+```
+
+### Linting
+
+```bash
+# Lint all code
+.venv/bin/flake8 starpunk/ tests/
+
+# Lint specific file
+.venv/bin/flake8 starpunk/notes.py
+
+# Type checking (if mypy installed)
+.venv/bin/mypy starpunk/
+```
+
+### Adding Dependencies
+
+```bash
+# 1. Install new package
+uv pip install package-name
+
+# 2. Update requirements.txt
+uv pip freeze | sort > requirements.txt
+
+# 3. Commit updated requirements.txt
+git add requirements.txt
+git commit -m "Add package-name dependency"
+```
+
+### Database Operations
+
+```bash
+# Reset database (WARNING: deletes all data)
+rm data/starpunk.db
+.venv/bin/python -c "from starpunk.database import init_db; init_db()"
+
+# Backup database
+cp data/starpunk.db data/starpunk.db.backup
+
+# Inspect database
+sqlite3 data/starpunk.db
+# SQL> .tables
+# SQL> SELECT * FROM notes;
+# SQL> .quit
+```
+
+### Creating a Note Manually
+
+```bash
+# 1. Create directory for current month
+mkdir -p data/notes/$(date +%Y)/$(date +%m)
+
+# 2. Create markdown file
+cat > data/notes/$(date +%Y)/$(date +%m)/test-note.md << 'EOF'
+This is a test note created manually.
+
+It has **markdown** formatting and `code`.
+EOF
+
+# 3. Add to database
+.venv/bin/python -c "
+from starpunk.notes import scan_and_import_notes
+scan_and_import_notes()
+"
+```
+
+---
+
+## Troubleshooting
+
+### Issue: Virtual Environment Not Found
+
+**Symptom**: `bash: .venv/bin/python: No such file or directory`
+
+**Solution**:
+```bash
+# Check if .venv exists
+ls -la .venv
+
+# If not, create it
+uv venv .venv --python 3.11
+```
+
+---
+
+### Issue: Import Errors
+
+**Symptom**: `ModuleNotFoundError: No module named 'flask'`
+
+**Solution**:
+```bash
+# Install dependencies
+uv pip install -r requirements.txt
+
+# Verify Flask is installed
+.venv/bin/python -c "import flask; print(flask.__version__)"
+```
+
+---
+
+### Issue: Database Not Found
+
+**Symptom**: `sqlite3.OperationalError: unable to open database file`
+
+**Solution**:
+```bash
+# Create data directory
+mkdir -p data/notes
+
+# Initialize database
+.venv/bin/python -c "from starpunk.database import init_db; init_db()"
+
+# Verify
+ls -la data/starpunk.db
+```
+
+---
+
+### Issue: Port Already in Use
+
+**Symptom**: `OSError: [Errno 98] Address already in use`
+
+**Solution**:
+```bash
+# Find process using port 5000
+lsof -i :5000
+
+# Kill the process
+kill -9 <PID>
+
+# Or use different port
+.venv/bin/flask run --port 5001
+```
+
+---
+
+### Issue: Missing SESSION_SECRET
+
+**Symptom**: `ValueError: SESSION_SECRET must be set in .env`
+
+**Solution**:
+```bash
+# Generate secret
+python3 -c "import secrets; print(secrets.token_hex(32))"
+
+# Add to .env file
+echo "SESSION_SECRET=<generated-secret>" >> .env
+```
+
+---
+
+### Issue: Permission Denied on data/
+
+**Symptom**: `PermissionError: [Errno 13] Permission denied: 'data/notes'`
+
+**Solution**:
+```bash
+# Fix permissions
+chmod -R 755 data/
+chmod 644 data/starpunk.db  # If exists
+
+# Verify
+ls -la data/
+```
+
+---
+
+## Agent-Specific Standards
+
+### For AI Developer Agents
+
+When setting up development environment, agents MUST:
+
+1. **Use absolute paths** in all bash commands
+   ```bash
+   # CORRECT
+   /home/phil/Projects/starpunk/.venv/bin/python
+
+   # WRONG
+   .venv/bin/python
+   ```
+
+2. **Check before creating** virtual environment
+   ```bash
+   if [ ! -d "/home/phil/Projects/starpunk/.venv" ]; then
+     uv venv /home/phil/Projects/starpunk/.venv --python 3.11
+   fi
+   ```
+
+3. **Verify each step** before proceeding
+   ```bash
+   # Example verification
+   [ -f "/home/phil/Projects/starpunk/.env" ] || echo "ERROR: .env missing"
+   ```
+
+4. **Never modify global Python**
+   ```bash
+   # FORBIDDEN
+   pip install flask
+
+   # CORRECT
+   uv pip install flask  # Uses active venv
+   ```
+
+See [ADR-006](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md) for complete agent standards.
+
+---
+
+## Security Considerations
+
+### Secrets Management
+
+**DO**:
+- Store all secrets in `.env` file
+- Generate strong random secrets
+- Use different secrets for dev/production
+- Never log secrets
+
+**DON'T**:
+- Commit `.env` to version control
+- Hardcode secrets in code
+- Share secrets via email/chat
+- Reuse secrets across projects
+
+### File Permissions
+
+**Recommended Permissions**:
+```bash
+# Application code
+chmod 644 *.py
+chmod 755 starpunk/
+
+# Data directory
+chmod 755 data/
+chmod 755 data/notes/
+chmod 644 data/starpunk.db
+chmod 644 data/notes/**/*.md
+
+# Configuration
+chmod 600 .env  # Only owner can read
+```
+
+### Development-Only Features
+
+In development mode (FLASK_DEBUG=1):
+- Detailed error pages with stack traces
+- Auto-reload on code changes
+- No HTTPS requirement
+- More verbose logging
+
+**NEVER run production with FLASK_DEBUG=1**
+
+---
+
+## Verification Checklist
+
+After completing setup, verify:
+
+- [ ] Python 3.11+ installed
+- [ ] uv installed and working
+- [ ] Virtual environment created in `.venv/`
+- [ ] All dependencies installed
+- [ ] `.env` file configured with SESSION_SECRET
+- [ ] `data/` directory exists and is gitignored
+- [ ] Database initialized with all tables
+- [ ] Development server starts successfully
+- [ ] Can access http://localhost:5000
+- [ ] Tests run successfully
+- [ ] Code formatting works (black)
+- [ ] Linting works (flake8)
+
+---
+
+## Quick Reference
+
+### Essential Commands
+
+```bash
+# Start development server
+.venv/bin/flask --app app.py run --debug
+
+# Run tests
+.venv/bin/pytest
+
+# Format code
+.venv/bin/black starpunk/ tests/
+
+# Install dependency
+uv pip install <package>
+uv pip freeze | sort > requirements.txt
+
+# Database reset
+rm data/starpunk.db
+.venv/bin/python -c "from starpunk.database import init_db; init_db()"
+
+# Generate secret
+python3 -c "import secrets; print(secrets.token_hex(32))"
+```
+
+### File Locations
+
+```
+.venv/              # Virtual environment
+.env                # Configuration (secret, gitignored)
+.env.example        # Configuration template
+requirements.txt    # Production dependencies
+data/               # User data (gitignored)
+starpunk/           # Application code
+tests/              # Test suite
+```
+
+---
+
+## Next Steps
+
+After setup is complete:
+
+1. Read [Python Coding Standards](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
+2. Review [Project Structure](/home/phil/Projects/starpunk/docs/design/project-structure.md)
+3. Explore [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
+4. Start implementing features following [API Contracts](/home/phil/Projects/starpunk/docs/design/api-contracts.md)
+
+---
+
+## References
+
+- [ADR-006: Python Virtual Environment with uv](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md)
+- [Project Structure Design](/home/phil/Projects/starpunk/docs/design/project-structure.md)
+- [Python Coding Standards](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
+- [Flask Configuration Handling](https://flask.palletsprojects.com/en/3.0.x/config/)
+- [python-dotenv Documentation](https://pypi.org/project/python-dotenv/)
+- [uv Documentation](https://docs.astral.sh/uv/)