StarPunk/docs/standards/development-setup.md

# Development Setup Standards

## Purpose

This document defines the standard development setup procedure for StarPunk. It provides step-by-step instructions for developer agents and humans to create a fully functional development environment from scratch.

## Prerequisites

### System Requirements

- **Operating System**: Linux, macOS, or Windows with WSL2
- **Python**: 3.11 or higher
- **Disk Space**: 500MB minimum
- **Network**: Internet connection for package downloads

### Required Tools

- **Python 3.11+**: System Python installation
- **uv**: Python package and environment manager
- **git**: Version control (for cloning repository)

## Installation Steps

### Step 1: Verify Python Installation

```bash
# Check Python version
python3 --version

# Expected output: Python 3.11.x or higher
```

**If Python 3.11+ is not installed**:

**Ubuntu/Debian**:
```bash
sudo apt update
sudo apt install python3.11 python3.11-venv
```

**macOS**:
```bash
brew install python@3.11
```

**Arch Linux**:
```bash
sudo pacman -S python
```

---

### Step 2: Install uv

**Linux/macOS (Recommended)**:
```bash
# Install via official installer
curl -LsSf https://astral.sh/uv/install.sh | sh

# Verify installation
uv --version
```

**Alternative (via pip)**:
```bash
# Install uv using pip
pip install --user uv

# Verify installation
uv --version
```

**Windows**:
```powershell
# Using PowerShell
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# Verify installation
uv --version
```

**Expected Output**: `uv 0.x.x` (any recent version)

See [ADR-006](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md) for full uv standards.

---

### Step 3: Clone Repository

```bash
# Clone the repository
git clone https://github.com/YOUR_USERNAME/starpunk.git
cd starpunk

# Verify you're in the correct directory
pwd
# Expected: /path/to/starpunk

# List files to confirm
ls -la
# Should see: app.py, requirements.txt, starpunk/, etc.
```

**If starting from scratch without git**:
Create the project structure manually following [Project Structure Design](/home/phil/Projects/starpunk/docs/design/project-structure.md).

---

### Step 4: Create Virtual Environment

```bash
# Create virtual environment using uv
uv venv .venv --python 3.11

# Verify creation
ls -la .venv

# Expected: bin/, lib/, include/, pyvenv.cfg
```

**Verification**:
```bash
# Check Python executable exists
.venv/bin/python --version

# Expected: Python 3.11.x
```

**Important**: Virtual environment is created in `.venv/` directory at project root.

---

### Step 5: Install Dependencies

```bash
# Install all production dependencies
uv pip install -r requirements.txt

# Verify installation
uv pip list

# Expected output should include:
# - Flask (3.0.x)
# - markdown (3.5.x)
# - feedgen (1.0.x)
# - httpx (0.27.x)
# - python-dotenv (1.0.x)
# - pytest (8.0.x)
```

**For Development** (optional):
```bash
# Install development dependencies
uv pip install -r requirements-dev.txt

# Includes: pytest-cov, black, flake8, mypy
```

**Troubleshooting**:
- If installation fails, check internet connection
- Verify uv is installed correctly: `which uv`
- Check Python version in venv: `.venv/bin/python --version`

---

### Step 6: Configure Environment

```bash
# Copy environment template
cp .env.example .env

# Edit .env file with your settings
nano .env  # or vim, code, etc.
```

**Required Configuration** (edit `.env`):

```bash
# Site Configuration
SITE_URL=http://localhost:5000
SITE_NAME=My StarPunk Site
SITE_AUTHOR=Your Name

# Admin Authentication
ADMIN_ME=https://your-website.com
# ^ Replace with YOUR IndieWeb identity URL

# Session Security
SESSION_SECRET=GENERATE_RANDOM_SECRET_HERE
# ^ Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"

# Data Paths
DATA_PATH=./data
NOTES_PATH=./data/notes
DATABASE_PATH=./data/starpunk.db

# Flask Configuration
FLASK_ENV=development
FLASK_DEBUG=1
```

**Generate SESSION_SECRET**:
```bash
# Generate cryptographically secure secret
python3 -c "import secrets; print(secrets.token_hex(32))"

# Copy output and paste into .env as SESSION_SECRET value
```

**Important**:
- **NEVER commit `.env` file** to git
- `.env.example` is a template and SHOULD be committed
- Each developer/deployment has their own `.env`

---

### Step 7: Initialize Database

```bash
# Create data directory structure
mkdir -p data/notes

# Initialize database schema
.venv/bin/python -c "from starpunk.database import init_db; init_db()"

# Verify database created
ls -la data/

# Expected: starpunk.db file should exist
```

**Alternative** (if Flask CLI is set up):
```bash
# Initialize via Flask CLI
.venv/bin/flask db init
```

**Verify Database Schema**:
```bash
# Check tables exist
sqlite3 data/starpunk.db ".tables"

# Expected output: notes, sessions, tokens, auth_state
```

---

### Step 8: Verify Installation

```bash
# Run all verification checks
.venv/bin/python -c "
from starpunk.config import load_config
from starpunk.database import verify_db
import flask
print('✓ Flask version:', flask.__version__)
print('✓ Config loads successfully')
print('✓ Database schema verified')
print('✓ Installation complete!')
"
```

**Manual Verification Checklist**:
```bash
# Check virtual environment
[ -d ".venv" ] && echo "✓ Virtual environment exists" || echo "✗ Missing .venv"

# Check Python version
.venv/bin/python --version | grep "3.11" && echo "✓ Python 3.11+" || echo "✗ Wrong Python version"

# Check Flask installed
.venv/bin/python -c "import flask" && echo "✓ Flask installed" || echo "✗ Flask missing"

# Check .env file exists
[ -f ".env" ] && echo "✓ .env configured" || echo "✗ Missing .env"

# Check data directory
[ -d "data/notes" ] && echo "✓ Data directory exists" || echo "✗ Missing data/"

# Check database
[ -f "data/starpunk.db" ] && echo "✓ Database initialized" || echo "✗ Missing database"
```

---

### Step 9: Run Development Server

```bash
# Run Flask development server
.venv/bin/flask --app app.py run --debug

# Alternative: Run directly
.venv/bin/python app.py
```

**Expected Output**:
```
 * Serving Flask app 'app.py'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit
```

**Access Application**:
- Open browser to: `http://localhost:5000`
- Should see StarPunk homepage
- No notes will exist on first run

**Stop Server**:
- Press `Ctrl+C` in terminal

---

### Step 10: Run Tests

```bash
# Run all tests
.venv/bin/pytest

# Run with coverage report
.venv/bin/pytest --cov=starpunk tests/

# Run specific test file
.venv/bin/pytest tests/test_database.py

# Run with verbose output
.venv/bin/pytest -v
```

**Expected Output** (once tests are written):
```
======================== test session starts =========================
collected 45 items

tests/test_auth.py ........                                    [ 17%]
tests/test_database.py .......                                 [ 33%]
tests/test_feed.py .....                                       [ 44%]
tests/test_micropub.py ..........                              [ 66%]
tests/test_notes.py ..........                                 [ 88%]
tests/test_utils.py .....                                      [100%]

========================= 45 passed in 2.34s =========================
```

---

## Configuration Management

### Environment Variables

All configuration is managed via environment variables loaded from `.env` file.

**Configuration Categories**:

1. **Site Identity**
   - `SITE_URL` - Public URL of site
   - `SITE_NAME` - Site title
   - `SITE_AUTHOR` - Author name
   - `SITE_DESCRIPTION` - Site description (for RSS)

2. **Authentication**
   - `ADMIN_ME` - Admin's IndieWeb identity URL
   - `SESSION_SECRET` - Secret key for session signing
   - `SESSION_LIFETIME` - Session duration in days (default: 30)

3. **Data Storage**
   - `DATA_PATH` - Base data directory (default: `./data`)
   - `NOTES_PATH` - Notes directory (default: `./data/notes`)
   - `DATABASE_PATH` - SQLite database path (default: `./data/starpunk.db`)

4. **Flask Settings**
   - `FLASK_ENV` - Environment: `development` or `production`
   - `FLASK_DEBUG` - Debug mode: `1` (on) or `0` (off)
   - `FLASK_SECRET_KEY` - Falls back to `SESSION_SECRET`

5. **External Services**
   - `INDIELOGIN_URL` - IndieLogin service URL (default: `https://indielogin.com`)

### Configuration Loading

Configuration is loaded in this priority order:

1. **Environment variables** (highest priority)
2. **`.env` file** (loaded via python-dotenv)
3. **Default values** (in `config.py`)

**Example** (`starpunk/config.py`):
```python
import os
from pathlib import Path
from dotenv import load_dotenv

# Load .env file
load_dotenv()

class Config:
    # Site
    SITE_URL = os.getenv('SITE_URL', 'http://localhost:5000')
    SITE_NAME = os.getenv('SITE_NAME', 'StarPunk')

    # Security
    SECRET_KEY = os.getenv('SESSION_SECRET')
    if not SECRET_KEY:
        raise ValueError("SESSION_SECRET must be set in .env")

    # Data paths
    DATA_PATH = Path(os.getenv('DATA_PATH', './data'))
    NOTES_PATH = Path(os.getenv('NOTES_PATH', './data/notes'))
    DATABASE_PATH = Path(os.getenv('DATABASE_PATH', './data/starpunk.db'))

    # Flask
    DEBUG = os.getenv('FLASK_DEBUG', '1') == '1'
    ENV = os.getenv('FLASK_ENV', 'development')
```

### Development vs Production Settings

**Development** (`.env`):
```bash
FLASK_ENV=development
FLASK_DEBUG=1
SITE_URL=http://localhost:5000
```

**Production** (`.env`):
```bash
FLASK_ENV=production
FLASK_DEBUG=0
SITE_URL=https://your-domain.com
```

**Key Differences**:
- **Debug mode**: Development has detailed errors, production hides them
- **URL**: Development uses localhost, production uses actual domain
- **HTTPS**: Production requires HTTPS, development doesn't
- **Session cookies**: Production sets `Secure` flag, development doesn't

---

## Common Development Tasks

### Starting Development

```bash
# 1. Activate virtual environment context (via uv)
cd /path/to/starpunk

# 2. Run development server
.venv/bin/flask --app app.py run --debug

# Server runs on http://localhost:5000
```

### Running Tests

```bash
# All tests
.venv/bin/pytest

# Specific module
.venv/bin/pytest tests/test_notes.py

# Specific test
.venv/bin/pytest tests/test_notes.py::test_create_note

# With coverage
.venv/bin/pytest --cov=starpunk --cov-report=html tests/
# Open htmlcov/index.html to view coverage report
```

### Code Formatting

```bash
# Format all Python code
.venv/bin/black starpunk/ tests/

# Check formatting without changing
.venv/bin/black --check starpunk/ tests/

# Format specific file
.venv/bin/black starpunk/notes.py
```

### Linting

```bash
# Lint all code
.venv/bin/flake8 starpunk/ tests/

# Lint specific file
.venv/bin/flake8 starpunk/notes.py

# Type checking (if mypy installed)
.venv/bin/mypy starpunk/
```

### Adding Dependencies

```bash
# 1. Install new package
uv pip install package-name

# 2. Update requirements.txt
uv pip freeze | sort > requirements.txt

# 3. Commit updated requirements.txt
git add requirements.txt
git commit -m "Add package-name dependency"
```

### Database Operations

```bash
# Reset database (WARNING: deletes all data)
rm data/starpunk.db
.venv/bin/python -c "from starpunk.database import init_db; init_db()"

# Backup database
cp data/starpunk.db data/starpunk.db.backup

# Inspect database
sqlite3 data/starpunk.db
# SQL> .tables
# SQL> SELECT * FROM notes;
# SQL> .quit
```

### Creating a Note Manually

```bash
# 1. Create directory for current month
mkdir -p data/notes/$(date +%Y)/$(date +%m)

# 2. Create markdown file
cat > data/notes/$(date +%Y)/$(date +%m)/test-note.md << 'EOF'
This is a test note created manually.

It has **markdown** formatting and `code`.
EOF

# 3. Add to database
.venv/bin/python -c "
from starpunk.notes import scan_and_import_notes
scan_and_import_notes()
"
```

---

## Troubleshooting

### Issue: Virtual Environment Not Found

**Symptom**: `bash: .venv/bin/python: No such file or directory`

**Solution**:
```bash
# Check if .venv exists
ls -la .venv

# If not, create it
uv venv .venv --python 3.11
```

---

### Issue: Import Errors

**Symptom**: `ModuleNotFoundError: No module named 'flask'`

**Solution**:
```bash
# Install dependencies
uv pip install -r requirements.txt

# Verify Flask is installed
.venv/bin/python -c "import flask; print(flask.__version__)"
```

---

### Issue: Database Not Found

**Symptom**: `sqlite3.OperationalError: unable to open database file`

**Solution**:
```bash
# Create data directory
mkdir -p data/notes

# Initialize database
.venv/bin/python -c "from starpunk.database import init_db; init_db()"

# Verify
ls -la data/starpunk.db
```

---

### Issue: Port Already in Use

**Symptom**: `OSError: [Errno 98] Address already in use`

**Solution**:
```bash
# Find process using port 5000
lsof -i :5000

# Kill the process
kill -9 <PID>

# Or use different port
.venv/bin/flask run --port 5001
```

---

### Issue: Missing SESSION_SECRET

**Symptom**: `ValueError: SESSION_SECRET must be set in .env`

**Solution**:
```bash
# Generate secret
python3 -c "import secrets; print(secrets.token_hex(32))"

# Add to .env file
echo "SESSION_SECRET=<generated-secret>" >> .env
```

---

### Issue: Permission Denied on data/

**Symptom**: `PermissionError: [Errno 13] Permission denied: 'data/notes'`

**Solution**:
```bash
# Fix permissions
chmod -R 755 data/
chmod 644 data/starpunk.db  # If exists

# Verify
ls -la data/
```

---

## Agent-Specific Standards

### For AI Developer Agents

When setting up development environment, agents MUST:

1. **Use absolute paths** in all bash commands
   ```bash
   # CORRECT
   /home/phil/Projects/starpunk/.venv/bin/python

   # WRONG
   .venv/bin/python
   ```

2. **Check before creating** virtual environment
   ```bash
   if [ ! -d "/home/phil/Projects/starpunk/.venv" ]; then
     uv venv /home/phil/Projects/starpunk/.venv --python 3.11
   fi
   ```

3. **Verify each step** before proceeding
   ```bash
   # Example verification
   [ -f "/home/phil/Projects/starpunk/.env" ] || echo "ERROR: .env missing"
   ```

4. **Never modify global Python**
   ```bash
   # FORBIDDEN
   pip install flask

   # CORRECT
   uv pip install flask  # Uses active venv
   ```

See [ADR-006](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md) for complete agent standards.

---

## Security Considerations

### Secrets Management

**DO**:
- Store all secrets in `.env` file
- Generate strong random secrets
- Use different secrets for dev/production
- Never log secrets

**DON'T**:
- Commit `.env` to version control
- Hardcode secrets in code
- Share secrets via email/chat
- Reuse secrets across projects

### File Permissions

**Recommended Permissions**:
```bash
# Application code
chmod 644 *.py
chmod 755 starpunk/

# Data directory
chmod 755 data/
chmod 755 data/notes/
chmod 644 data/starpunk.db
chmod 644 data/notes/**/*.md

# Configuration
chmod 600 .env  # Only owner can read
```

### Development-Only Features

In development mode (FLASK_DEBUG=1):
- Detailed error pages with stack traces
- Auto-reload on code changes
- No HTTPS requirement
- More verbose logging

**NEVER run production with FLASK_DEBUG=1**

---

## Verification Checklist

After completing setup, verify:

- [ ] Python 3.11+ installed
- [ ] uv installed and working
- [ ] Virtual environment created in `.venv/`
- [ ] All dependencies installed
- [ ] `.env` file configured with SESSION_SECRET
- [ ] `data/` directory exists and is gitignored
- [ ] Database initialized with all tables
- [ ] Development server starts successfully
- [ ] Can access http://localhost:5000
- [ ] Tests run successfully
- [ ] Code formatting works (black)
- [ ] Linting works (flake8)

---

## Quick Reference

### Essential Commands

```bash
# Start development server
.venv/bin/flask --app app.py run --debug

# Run tests
.venv/bin/pytest

# Format code
.venv/bin/black starpunk/ tests/

# Install dependency
uv pip install <package>
uv pip freeze | sort > requirements.txt

# Database reset
rm data/starpunk.db
.venv/bin/python -c "from starpunk.database import init_db; init_db()"

# Generate secret
python3 -c "import secrets; print(secrets.token_hex(32))"
```

### File Locations

```
.venv/              # Virtual environment
.env                # Configuration (secret, gitignored)
.env.example        # Configuration template
requirements.txt    # Production dependencies
data/               # User data (gitignored)
starpunk/           # Application code
tests/              # Test suite
```

---

## Next Steps

After setup is complete:

1. Read [Python Coding Standards](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
2. Review [Project Structure](/home/phil/Projects/starpunk/docs/design/project-structure.md)
3. Explore [Architecture Overview](/home/phil/Projects/starpunk/docs/architecture/overview.md)
4. Start implementing features following [API Contracts](/home/phil/Projects/starpunk/docs/design/api-contracts.md)

---

## References

- [ADR-006: Python Virtual Environment with uv](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md)
- [Project Structure Design](/home/phil/Projects/starpunk/docs/design/project-structure.md)
- [Python Coding Standards](/home/phil/Projects/starpunk/docs/standards/python-coding-standards.md)
- [Flask Configuration Handling](https://flask.palletsprojects.com/en/3.0.x/config/)
- [python-dotenv Documentation](https://pypi.org/project/python-dotenv/)
- [uv Documentation](https://docs.astral.sh/uv/)