phil/Gondulf
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8dddc73826402fd6eff1c46fd309eb098999942f
Gondulf/docs/designs/phase-4-5-critical-components.md
Phil Skentelbery 115e733604 feat(phase-4a): complete Phase 3 implementation and gap analysis 
Merges Phase 4a work including:

Implementation:
- Metadata discovery endpoint (/api/.well-known/oauth-authorization-server)
- h-app microformat parser service
- Enhanced authorization endpoint with client info display
- Configuration management system
- Dependency injection framework

Documentation:
- Comprehensive gap analysis for v1.0.0 compliance
- Phase 4a clarifications on development approach
- Phase 4-5 critical components breakdown

Testing:
- Unit tests for h-app parser (308 lines, comprehensive coverage)
- Unit tests for metadata endpoint (134 lines)
- Unit tests for configuration system (18 lines)
- Integration test updates

All tests passing with high coverage. Ready for Phase 4b security hardening.
2025-11-20 17:16:11 -07:00

3234 lines
92 KiB
Markdown
Raw Blame History

# Phase 4-5: Critical Components for v1.0.0 Release
**Architect**: Claude (Architect Agent)
**Date**: 2025-11-20
**Status**: Design Complete
**Design References**:
- `/docs/roadmap/v1.0.0.md` - Original release plan
- `/docs/reports/2025-11-20-gap-analysis-v1.0.0.md` - Gap analysis identifying missing components
- `/docs/architecture/security.md` - Security architecture
- `/docs/architecture/indieauth-protocol.md` - Protocol implementation details
## Overview
This design document addresses the 7 critical missing components identified in the v1.0.0 gap analysis that are blocking release. These components span Phase 3 completion (metadata endpoint, client metadata fetching), Phase 4 (security hardening), and Phase 5 (deployment, testing, documentation).
**Current Status**: Phases 1-2 are complete (100%). Phase 3 is 75% complete (missing metadata endpoint and h-app parsing). Phases 4-5 have not been started.
**Estimated Remaining Effort**: 10-15 days to reach v1.0.0 release readiness.
**Design Philosophy**: Maintain simplicity while meeting all P0 requirements. Reuse existing infrastructure where possible. Focus on production readiness and W3C IndieAuth compliance.
---
## Component 1: Metadata Endpoint
### Purpose
Provide OAuth 2.0 Authorization Server Metadata endpoint per RFC 8414 to enable IndieAuth client discovery of server capabilities and endpoints.
### Specification References
- **W3C IndieAuth**: Section on Discovery (metadata endpoint)
- **RFC 8414**: OAuth 2.0 Authorization Server Metadata
- **v1.0.0 Roadmap**: Line 62 (P0 feature), Phase 3 lines 162, 168
### Design Overview
Create a static metadata endpoint at `/.well-known/oauth-authorization-server` that returns server capabilities in JSON format. This endpoint requires no authentication and should be publicly cacheable.
### API Specification
**Endpoint**: `GET /.well-known/oauth-authorization-server`
**Request**: No parameters, no authentication required
**Response** (HTTP 200 OK):
```json
{
"issuer": "https://auth.example.com",
"authorization_endpoint": "https://auth.example.com/authorize",
"token_endpoint": "https://auth.example.com/token",
"response_types_supported": ["code"],
"grant_types_supported": ["authorization_code"],
"code_challenge_methods_supported": [],
"token_endpoint_auth_methods_supported": ["none"],
"revocation_endpoint_auth_methods_supported": ["none"],
"scopes_supported": []
}
```
**Response Headers**:
```http
Content-Type: application/json
Cache-Control: public, max-age=86400
```
### Field Definitions
| Field | Value | Rationale |
|-------|-------|-----------|
| `issuer` | Server base URL (from config) | Identifies this authorization server |
| `authorization_endpoint` | `{base_url}/authorize` | Where clients initiate auth flow |
| `token_endpoint` | `{base_url}/token` | Where clients exchange codes for tokens |
| `response_types_supported` | `["code"]` | Only authorization code flow supported |
| `grant_types_supported` | `["authorization_code"]` | Only grant type in v1.0.0 |
| `code_challenge_methods_supported` | `[]` | PKCE not supported in v1.0.0 (ADR-003) |
| `token_endpoint_auth_methods_supported` | `["none"]` | Public clients, no client secrets |
| `revocation_endpoint_auth_methods_supported` | `["none"]` | No revocation endpoint in v1.0.0 |
| `scopes_supported` | `[]` | Authentication-only, no scopes in v1.0.0 |
### Implementation Approach
**File**: `/src/gondulf/routers/metadata.py`
**Implementation Strategy**: Static JSON response generated from configuration at startup.
```python
from fastapi import APIRouter, Response
from gondulf.config import get_config
router = APIRouter()
@router.get("/.well-known/oauth-authorization-server")
async def get_metadata():
"""
OAuth 2.0 Authorization Server Metadata (RFC 8414).
Returns server capabilities for IndieAuth client discovery.
"""
config = get_config()
metadata = {
"issuer": config.BASE_URL,
"authorization_endpoint": f"{config.BASE_URL}/authorize",
"token_endpoint": f"{config.BASE_URL}/token",
"response_types_supported": ["code"],
"grant_types_supported": ["authorization_code"],
"code_challenge_methods_supported": [],
"token_endpoint_auth_methods_supported": ["none"],
"revocation_endpoint_auth_methods_supported": ["none"],
"scopes_supported": []
}
return Response(
content=json.dumps(metadata, indent=2),
media_type="application/json",
headers={
"Cache-Control": "public, max-age=86400"
}
)
```
**Configuration Change**: Add `BASE_URL` to config (e.g., `GONDULF_BASE_URL=https://auth.example.com`).
**Registration**: Add router to main.py:
```python
from gondulf.routers import metadata
app.include_router(metadata.router)
```
### Error Handling
No error conditions - endpoint always returns metadata. If configuration is invalid, application fails at startup (fail-fast principle).
### Security Considerations
- **Public Endpoint**: No authentication required (per RFC 8414)
- **Cache-Control**: Set public cache for 24 hours to reduce load
- **No Secrets**: Metadata contains no sensitive information
- **HTTPS**: Served over HTTPS in production (enforced by middleware)
### Testing Requirements
**Unit Tests** (`tests/unit/test_metadata.py`):
1. Test metadata endpoint returns 200 OK
2. Test all required fields present
3. Test correct values for each field
4. Test Cache-Control header present
5. Test Content-Type is application/json
6. Test issuer matches BASE_URL configuration
**Integration Test** (`tests/integration/test_metadata_integration.py`):
1. Test endpoint accessible via FastAPI TestClient
2. Test response can be parsed as valid JSON
3. Test endpoints in metadata are valid URLs
**Compliance Test**:
1. Verify metadata response matches RFC 8414 format
### Acceptance Criteria
- [ ] `/.well-known/oauth-authorization-server` endpoint returns valid JSON
- [ ] All required fields present per RFC 8414
- [ ] Endpoint values match actual server configuration
- [ ] Cache-Control header set to public, max-age=86400
- [ ] All tests pass (unit, integration)
- [ ] Endpoint accessible without authentication
---
## Component 2: Client Metadata Fetching (h-app Microformat)
### Purpose
Fetch and parse client application metadata from `client_id` URL to display application name, icon, and URL on the consent screen. This improves user experience by showing what application they're authorizing.
### Specification References
- **W3C IndieAuth**: Client Information Discovery
- **Microformats h-app**: http://microformats.org/wiki/h-app
- **v1.0.0 Roadmap**: Success criteria line 27, Phase 3 deliverables line 169
### Design Overview
Create a service that fetches the `client_id` URL, parses HTML for h-app microformat data, extracts application metadata, and caches results. Integrate with authorization endpoint to display app information on consent screen.
### h-app Microformat Structure
Example HTML from client application:
```html
<div class="h-app">
<img src="/icon.png" class="u-logo" alt="App Icon">
<a href="/" class="u-url p-name">My IndieAuth Client</a>
</div>
```
**Properties to Extract**:
- `p-name`: Application name (required)
- `u-logo`: Application icon URL (optional)
- `u-url`: Application URL (optional, usually same as client_id)
### Data Model
**ClientMetadata** (in-memory cache):
```python
@dataclass
class ClientMetadata:
"""Client application metadata from h-app microformat."""
client_id: str
name: str # Extracted from p-name, or domain fallback
logo_url: Optional[str] = None # Extracted from u-logo
url: Optional[str] = None # Extracted from u-url
fetched_at: datetime = None # Timestamp for cache expiry
```
### Service Design
**File**: `/src/gondulf/services/h_app_parser.py`
**Dependencies**:
- `html_fetcher.py` (already exists from Phase 2)
- `mf2py` library for microformat parsing
**Service Interface**:
```python
class HAppParser:
"""Parse h-app microformat from client_id URL."""
def __init__(self, html_fetcher: HTMLFetcher):
self.html_fetcher = html_fetcher
self.cache: Dict[str, ClientMetadata] = {}
self.cache_ttl = timedelta(hours=24)
def parse_client_metadata(self, client_id: str) -> ClientMetadata:
"""
Fetch and parse client metadata from client_id URL.
Returns ClientMetadata with name (always populated), and
optional logo_url and url.
Caches results for 24 hours to reduce HTTP requests.
"""
pass
def _parse_h_app(self, html: str, client_id: str) -> ClientMetadata:
"""
Parse h-app microformat from HTML.
Returns ClientMetadata with extracted values, or fallback
to domain name if no h-app found.
"""
pass
def _extract_domain_name(self, client_id: str) -> str:
"""Extract domain name from client_id for fallback display."""
pass
```
### Implementation Details
**Parsing Strategy**:
1. Check cache for `client_id` (if cached and not expired, return cached)
2. Fetch HTML from `client_id` using `HTMLFetcher` (reuse Phase 2 infrastructure)
3. Parse HTML with `mf2py` library to extract h-app microformat
4. Extract `p-name`, `u-logo`, `u-url` properties
5. If h-app not found, fallback to domain name extraction
6. Store in cache with 24-hour TTL
7. Return `ClientMetadata` object
**mf2py Usage**:
```python
import mf2py
from urllib.parse import urlparse, urljoin
def _parse_h_app(self, html: str, client_id: str) -> ClientMetadata:
"""Parse h-app microformat from HTML."""
# Parse microformats
parsed = mf2py.parse(doc=html, url=client_id)
# Find h-app items
h_apps = [item for item in parsed.get('items', [])
if 'h-app' in item.get('type', [])]
if not h_apps:
# Fallback: no h-app found
return ClientMetadata(
client_id=client_id,
name=self._extract_domain_name(client_id),
fetched_at=datetime.utcnow()
)
# Use first h-app
h_app = h_apps[0]
properties = h_app.get('properties', {})
# Extract properties
name = properties.get('name', [None])[0] or self._extract_domain_name(client_id)
logo_url = properties.get('logo', [None])[0]
url = properties.get('url', [None])[0] or client_id
# Resolve relative URLs
if logo_url and not logo_url.startswith('http'):
logo_url = urljoin(client_id, logo_url)
return ClientMetadata(
client_id=client_id,
name=name,
logo_url=logo_url,
url=url,
fetched_at=datetime.utcnow()
)
```
**Fallback Strategy**:
```python
def _extract_domain_name(self, client_id: str) -> str:
"""Extract domain name for fallback display."""
parsed = urlparse(client_id)
domain = parsed.hostname or client_id
# Remove 'www.' prefix if present
if domain.startswith('www.'):
domain = domain[4:]
return domain
```
**Cache Management**:
```python
def parse_client_metadata(self, client_id: str) -> ClientMetadata:
"""Fetch and parse with caching."""
# Check cache
if client_id in self.cache:
cached = self.cache[client_id]
age = datetime.utcnow() - cached.fetched_at
if age < self.cache_ttl:
logger.debug(f"Cache hit for client_id: {client_id}")
return cached
# Fetch HTML
try:
html = self.html_fetcher.fetch(client_id)
if not html:
raise ValueError("Failed to fetch client_id URL")
# Parse h-app
metadata = self._parse_h_app(html, client_id)
# Cache result
self.cache[client_id] = metadata
return metadata
except Exception as e:
logger.warning(f"Failed to parse client metadata for {client_id}: {e}")
# Return fallback metadata
return ClientMetadata(
client_id=client_id,
name=self._extract_domain_name(client_id),
fetched_at=datetime.utcnow()
)
```
### Integration with Authorization Endpoint
**Update**: `/src/gondulf/routers/authorization.py`
**Change**: Inject `HAppParser` and fetch client metadata before rendering consent screen.
```python
from gondulf.services.h_app_parser import HAppParser, ClientMetadata
@router.get("/authorize")
async def authorize_get(
# ... existing parameters ...
h_app_parser: HAppParser = Depends(get_h_app_parser)
):
# ... existing validation ...
# Fetch client metadata
client_metadata = h_app_parser.parse_client_metadata(client_id)
# Render consent screen with client metadata
return templates.TemplateResponse(
"authorize.html",
{
"request": request,
"me": me,
"client_name": client_metadata.name,
"client_logo_url": client_metadata.logo_url,
"client_url": client_metadata.url or client_id,
"client_id": client_id,
"redirect_uri": redirect_uri,
"state": state
}
)
```
**Template Update**: `/src/gondulf/templates/authorize.html`
Add client metadata display:
```html
<div class="client-info">
{% if client_logo_url %}
<img src="{{ client_logo_url }}" alt="{{ client_name }} icon" class="client-logo">
{% endif %}
<h2>{{ client_name }}</h2>
<p class="client-url">{{ client_url }}</p>
</div>
<p>This application wants to sign you in as:</p>
<p class="user-identity"><strong>{{ me }}</strong></p>
<form method="post" action="/authorize">
<!-- ... existing form fields ... -->
<button type="submit" name="authorized" value="true">Authorize</button>
<button type="submit" name="authorized" value="false">Deny</button>
</form>
```
### Dependency Injection
**File**: `/src/gondulf/dependencies.py`
Add `get_h_app_parser()`:
```python
from gondulf.services.h_app_parser import HAppParser
@lru_cache()
def get_h_app_parser() -> HAppParser:
"""Get HAppParser singleton."""
html_fetcher = get_html_fetcher()
return HAppParser(html_fetcher)
```
### Error Handling
**Failure Modes**:
1. **HTTP fetch fails**: Return fallback metadata with domain name
2. **HTML parse fails**: Return fallback metadata with domain name
3. **h-app not found**: Return fallback metadata with domain name
4. **Invalid URLs in h-app**: Skip invalid fields, use available data
**Logging**:
- Log INFO when h-app successfully parsed
- Log WARNING when fallback used
- Log ERROR only on unexpected exceptions
### Security Considerations
- **HTTPS Only**: Reuse `HTMLFetcher` which enforces HTTPS
- **Timeout**: 5-second timeout from `HTMLFetcher`
- **Size Limit**: 5MB limit from `HTMLFetcher`
- **XSS Prevention**: HTML escape all client metadata in templates (Jinja2 auto-escaping)
- **Logo URL Validation**: Only display logo if HTTPS URL
- **Cache Poisoning**: Cache keyed by client_id, no user input
### Testing Requirements
**Unit Tests** (`tests/unit/test_h_app_parser.py`):
1. Test h-app parsing with complete metadata
2. Test h-app parsing with missing logo
3. Test h-app parsing with missing url
4. Test h-app not found (fallback to domain)
5. Test relative logo URL resolution
6. Test domain name extraction fallback
7. Test cache hit (no HTTP request)
8. Test cache expiry (new HTTP request)
9. Test HTML fetch failure (fallback)
10. Test multiple h-app items (use first)
**Integration Tests** (`tests/integration/test_authorization_with_client_metadata.py`):
1. Test authorization endpoint displays client name
2. Test authorization endpoint displays client logo
3. Test authorization endpoint with fallback metadata
### Acceptance Criteria
- [ ] `HAppParser` service created with caching
- [ ] h-app microformat parsing working with mf2py
- [ ] Fallback to domain name when h-app not found
- [ ] Cache working with 24-hour TTL
- [ ] Integration with authorization endpoint complete
- [ ] Consent screen displays client name, logo, URL
- [ ] All tests pass (unit, integration)
- [ ] HTML escaping prevents XSS
---
## Component 3: Security Hardening
### Purpose
Implement security best practices required for production deployment: security headers, HTTPS enforcement, input sanitization audit, and PII logging audit.
### Specification References
- **v1.0.0 Roadmap**: Line 65 (P0 feature), Phase 4 lines 198-203
- **OWASP Top 10**: Security header recommendations
- **OAuth 2.0 Security Best Practices**: HTTPS enforcement
### Design Overview
Create security middleware to add HTTP security headers to all responses. Implement HTTPS enforcement for production environments. Conduct comprehensive audit of input sanitization and PII logging practices.
### 3.1: Security Headers Middleware
**File**: `/src/gondulf/middleware/security_headers.py`
**Headers to Implement**:
| Header | Value | Purpose |
|--------|-------|---------|
| `X-Frame-Options` | `DENY` | Prevent clickjacking attacks |
| `X-Content-Type-Options` | `nosniff` | Prevent MIME sniffing |
| `X-XSS-Protection` | `1; mode=block` | Enable XSS filter (legacy browsers) |
| `Referrer-Policy` | `strict-origin-when-cross-origin` | Control referrer information |
| `Strict-Transport-Security` | `max-age=31536000; includeSubDomains` | Force HTTPS (production only) |
| `Content-Security-Policy` | `default-src 'self'; style-src 'self' 'unsafe-inline'` | Restrict resource loading |
| `Cache-Control` | `no-store, no-cache, must-revalidate` | Prevent caching of sensitive pages (auth endpoints only) |
| `Pragma` | `no-cache` | HTTP/1.0 cache control |
**Implementation**:
```python
from fastapi import Request, Response
from starlette.middleware.base import BaseHTTPMiddleware
from gondolf.config import get_config
class SecurityHeadersMiddleware(BaseHTTPMiddleware):
"""Add security headers to all responses."""
async def dispatch(self, request: Request, call_next):
response = await call_next(request)
config = get_config()
# Always add these headers
response.headers["X-Frame-Options"] = "DENY"
response.headers["X-Content-Type-Options"] = "nosniff"
response.headers["X-XSS-Protection"] = "1; mode=block"
response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
# CSP: Allow self and inline styles (for minimal CSS in templates)
response.headers["Content-Security-Policy"] = (
"default-src 'self'; "
"style-src 'self' 'unsafe-inline'; "
"img-src 'self' https:; " # Allow HTTPS images (client logos)
"frame-ancestors 'none'" # Equivalent to X-Frame-Options: DENY
)
# HSTS: Only in production with HTTPS
if not config.DEBUG and request.url.scheme == "https":
response.headers["Strict-Transport-Security"] = (
"max-age=31536000; includeSubDomains"
)
# Cache control for sensitive endpoints
if request.url.path in ["/authorize", "/token", "/api/verify/code"]:
response.headers["Cache-Control"] = "no-store, no-cache, must-revalidate"
response.headers["Pragma"] = "no-cache"
return response
```
**Registration**: Add to `main.py`:
```python
from gondulf.middleware.security_headers import SecurityHeadersMiddleware
app.add_middleware(SecurityHeadersMiddleware)
```
### 3.2: HTTPS Enforcement Middleware
**File**: `/src/gondulf/middleware/https_enforcement.py`
**Implementation**:
```python
from fastapi import Request, Response
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.responses import RedirectResponse
from gondolf.config import get_config
class HTTPSEnforcementMiddleware(BaseHTTPMiddleware):
"""Enforce HTTPS in production (redirect HTTP to HTTPS)."""
async def dispatch(self, request: Request, call_next):
config = get_config()
# Only enforce in production
if not config.DEBUG:
# Allow localhost HTTP for local testing
if request.url.hostname not in ["localhost", "127.0.0.1"]:
# Check if HTTP (not HTTPS)
if request.url.scheme != "https":
# Redirect to HTTPS
https_url = request.url.replace(scheme="https")
return RedirectResponse(url=str(https_url), status_code=301)
return await call_next(request)
```
**Registration**: Add to `main.py` BEFORE security headers middleware:
```python
from gondulf.middleware.https_enforcement import HTTPSEnforcementMiddleware
# Add HTTPS enforcement first (before security headers)
app.add_middleware(HTTPSEnforcementMiddleware)
app.add_middleware(SecurityHeadersMiddleware)
```
**Configuration**: Add `DEBUG` flag to config:
```python
# config.py
DEBUG = os.getenv("GONDULF_DEBUG", "false").lower() == "true"
```
### 3.3: Input Sanitization Audit
**Scope**: Review all user input handling for proper validation and sanitization.
**Audit Checklist**:
| Input Source | Current Validation | Additional Sanitization Needed |
|--------------|-------------------|-------------------------------|
| `me` parameter | Pydantic HttpUrl, custom validation | ✅ Adequate (URL validation comprehensive) |
| `client_id` parameter | Pydantic HttpUrl | ✅ Adequate |
| `redirect_uri` parameter | Pydantic HttpUrl, domain validation | ✅ Adequate |
| `state` parameter | Pydantic str, max_length=512 | ✅ Adequate (opaque, not interpreted) |
| `code` parameter | str, checked against storage | ✅ Adequate (constant-time hash comparison) |
| Email addresses | Regex validation in `validation.py` | ✅ Adequate (from rel=me, not user input) |
| HTML templates | Jinja2 auto-escaping | ✅ Adequate (all {{ }} escaped by default) |
| SQL queries | SQLAlchemy parameterized queries | ✅ Adequate (no string interpolation) |
| DNS queries | dnspython library | ✅ Adequate (library handles escaping) |
**Action Items**:
1. **Add HTML escaping test**: Verify Jinja2 auto-escaping works
2. **Add SQL injection test**: Verify parameterized queries prevent injection
3. **Document validation patterns**: Update security.md with validation approach
**No Code Changes Required**: Existing validation is adequate.
### 3.4: PII Logging Audit
**Scope**: Ensure no Personally Identifiable Information (PII) is logged.
**PII Definition**:
- Email addresses
- Full tokens (access tokens, authorization codes, verification codes)
- IP addresses (in production)
**Audit Checklist**:
| Service | Logs Email? | Logs Tokens? | Logs IP? | Action Required |
|---------|-------------|--------------|----------|-----------------|
| `email.py` | ❌ No (only domain) | N/A | ❌ No | ✅ Compliant |
| `token_service.py` | N/A | ⚠️ Token prefix only (8 chars) | ❌ No | ✅ Compliant (prefix OK) |
| `domain_verification.py` | ⚠️ **May log email** | ⚠️ **May log code** | ❌ No | 🔍 **Review Required** |
| `authorization.py` | ❌ No | ⚠️ Code prefix only | ⚠️ **May log IP in errors** | 🔍 **Review Required** |
| `verification.py` | ❌ No | ⚠️ Code in errors? | ❌ No | 🔍 **Review Required** |
**Action Items**:
1. **Review all logger.info/warning/error calls** in services and routers
2. **Remove email addresses** from log messages (use domain only)
3. **Remove full codes/tokens** from log messages (use prefix or hash)
4. **Remove IP addresses** from production logs (OK in DEBUG mode)
5. **Add logging best practices** to coding standards
**Example Fix**:
```python
# BAD: Logs email (PII)
logger.info(f"Verification sent to {email}")
# GOOD: Logs domain only
logger.info(f"Verification sent to user at domain {domain}")
# BAD: Logs full token
logger.debug(f"Generated token: {token}")
# GOOD: Logs token prefix for correlation
logger.debug(f"Generated token: {token[:8]}...")
```
### 3.5: Security Configuration
**Add to config.py**:
```python
# Security settings
DEBUG = os.getenv("GONDULF_DEBUG", "false").lower() == "true"
HSTS_MAX_AGE = int(os.getenv("GONDULF_HSTS_MAX_AGE", "31536000")) # 1 year
CSP_POLICY = os.getenv(
"GONDULF_CSP_POLICY",
"default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' https:; frame-ancestors 'none'"
)
```
**Add to .env.example**:
```bash
# Security Settings (Production)
GONDULF_DEBUG=false
GONDULF_HSTS_MAX_AGE=31536000 # HSTS max-age in seconds (1 year default)
# GONDULF_CSP_POLICY=... # Custom CSP policy (optional)
```
### Error Handling
**Middleware Errors**:
- Log errors but DO NOT block requests
- If middleware fails, continue without headers (fail-open for availability)
- Log ERROR level for middleware failures
### Security Considerations
- **HSTS Preloading**: Consider submitting domain to HSTS preload list (future)
- **CSP Reporting**: Consider adding CSP report-uri (future)
- **Security.txt**: Consider adding /.well-known/security.txt (future)
### Testing Requirements
**Unit Tests** (`tests/unit/test_security_middleware.py`):
1. Test security headers present on all responses
2. Test HSTS header only in production
3. Test HSTS header not present in DEBUG mode
4. Test HTTPS enforcement redirects HTTP to HTTPS
5. Test HTTPS enforcement allows localhost HTTP
6. Test Cache-Control headers on sensitive endpoints
7. Test CSP header allows self and inline styles
**Integration Tests** (`tests/integration/test_security_integration.py`):
1. Test authorization endpoint has security headers
2. Test token endpoint has cache control headers
3. Test metadata endpoint does NOT have cache control (should be cacheable)
**Security Tests** (`tests/security/test_input_validation.py`):
1. Test HTML escaping in templates (XSS prevention)
2. Test SQL injection prevention (parameterized queries)
3. Test URL validation rejects malicious URLs
**PII Tests** (`tests/security/test_pii_logging.py`):
1. Test no email addresses in logs (mock logger, verify calls)
2. Test no full tokens in logs
3. Test no full codes in logs
### Acceptance Criteria
- [ ] Security headers middleware implemented and registered
- [ ] HTTPS enforcement middleware implemented and registered (production only)
- [ ] All security headers present on responses
- [ ] HSTS header only in production
- [ ] Input sanitization audit complete (documented)
- [ ] PII logging audit complete (issues fixed)
- [ ] All tests pass (unit, integration, security)
- [ ] No email addresses in logs
- [ ] No full tokens/codes in logs
---
## Component 4: Deployment Configuration
### Purpose
Provide production-ready deployment artifacts: Dockerfile, docker-compose.yml, database backup script, and comprehensive environment variable documentation.
### Specification References
- **v1.0.0 Roadmap**: Line 66 (P0 feature), Phase 5 lines 233-236
- **Docker Best Practices**: Multi-stage builds, non-root user, minimal base images
### Design Overview
Create Docker deployment configuration using multi-stage build for minimal image size. Provide docker-compose.yml for easy local testing. Create SQLite backup script with GPG encryption support. Document all environment variables comprehensively.
### 4.1: Dockerfile
**File**: `/Dockerfile`
**Strategy**: Multi-stage build to minimize final image size.
**Implementation**:
```dockerfile
# Stage 1: Builder
FROM python:3.11-slim AS builder
# Install uv for fast dependency resolution
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
# Set working directory
WORKDIR /app
# Copy dependency files
COPY pyproject.toml uv.lock ./
# Install dependencies to a virtual environment
RUN uv sync --frozen --no-dev
# Stage 2: Runtime
FROM python:3.11-slim
# Create non-root user
RUN useradd --create-home --shell /bin/bash gondulf
# Set working directory
WORKDIR /app
# Copy virtual environment from builder
COPY --from=builder /app/.venv /app/.venv
# Copy application code
COPY src/ /app/src/
COPY migrations/ /app/migrations/
# Create data directory for SQLite
RUN mkdir -p /app/data && chown gondulf:gondulf /app/data
# Switch to non-root user
USER gondulf
# Set environment variables
ENV PATH="/app/.venv/bin:$PATH"
ENV PYTHONPATH="/app/src"
ENV GONDULF_DATABASE_URL="sqlite:////app/data/gondulf.db"
# Expose port
EXPOSE 8000
# Health check
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
# Run application
CMD ["uvicorn", "gondulf.main:app", "--host", "0.0.0.0", "--port", "8000"]
```
**Build Instructions**:
```bash
# Build image
docker build -t gondulf:1.0.0 .
# Tag as latest
docker tag gondulf:1.0.0 gondulf:latest
```
**Image Properties**:
- Base: `python:3.11-slim` (Debian-based, ~150MB)
- User: Non-root `gondulf` user
- Port: 8000 (Uvicorn default)
- Health check: `/health` endpoint every 30 seconds
- Data volume: `/app/data` (for SQLite database)
### 4.2: docker-compose.yml
**File**: `/docker-compose.yml`
**Purpose**: Local testing and development deployment.
**Implementation**:
```yaml
version: "3.8"
services:
gondulf:
build: .
image: gondulf:latest
container_name: gondulf
restart: unless-stopped
ports:
- "8000:8000"
volumes:
- ./data:/app/data
- ./logs:/app/logs
environment:
# Required
- GONDULF_SECRET_KEY=${GONDULF_SECRET_KEY}
- GONDULF_BASE_URL=${GONDULF_BASE_URL:-http://localhost:8000}
# SMTP Configuration (required for email verification)
- GONDULF_SMTP_HOST=${GONDULF_SMTP_HOST}
- GONDULF_SMTP_PORT=${GONDULF_SMTP_PORT:-587}
- GONDULF_SMTP_USERNAME=${GONDULF_SMTP_USERNAME}
- GONDULF_SMTP_PASSWORD=${GONDULF_SMTP_PASSWORD}
- GONDULF_SMTP_FROM_EMAIL=${GONDULF_SMTP_FROM_EMAIL}
- GONDULF_SMTP_USE_TLS=${GONDULF_SMTP_USE_TLS:-true}
# Optional Configuration
- GONDULF_DEBUG=${GONDULF_DEBUG:-false}
- GONDULF_LOG_LEVEL=${GONDULF_LOG_LEVEL:-INFO}
- GONDULF_TOKEN_TTL=${GONDULF_TOKEN_TTL:-3600}
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
networks:
- gondulf-network
networks:
gondulf-network:
driver: bridge
volumes:
data:
logs:
```
**Usage**:
```bash
# Create .env file with required variables
cp .env.example .env
nano .env # Edit with your values
# Start service
docker-compose up -d
# View logs
docker-compose logs -f gondulf
# Stop service
docker-compose down
# Restart service
docker-compose restart gondulf
```
### 4.3: Backup Script
**File**: `/scripts/backup_database.sh`
**Purpose**: Backup SQLite database with optional GPG encryption.
**Implementation**:
```bash
#!/bin/bash
set -euo pipefail
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
DATA_DIR="${DATA_DIR:-$PROJECT_ROOT/data}"
BACKUP_DIR="${BACKUP_DIR:-$PROJECT_ROOT/backups}"
DB_FILE="${DB_FILE:-$DATA_DIR/gondulf.db}"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="$BACKUP_DIR/gondulf_${TIMESTAMP}.db"
RETENTION_DAYS="${RETENTION_DAYS:-30}"
# GPG encryption (optional)
GPG_RECIPIENT="${GPG_RECIPIENT:-}"
# Colors for output
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color
echo "========================================="
echo "Gondulf Database Backup"
echo "========================================="
echo
# Check if database exists
if [ ! -f "$DB_FILE" ]; then
echo -e "${RED}ERROR: Database file not found: $DB_FILE${NC}"
exit 1
fi
# Create backup directory
mkdir -p "$BACKUP_DIR"
# SQLite backup (using .backup command for consistency)
echo -e "${YELLOW}Backing up database...${NC}"
sqlite3 "$DB_FILE" ".backup $BACKUP_FILE"
if [ $? -eq 0 ]; then
echo -e "${GREEN}✓ Database backed up to: $BACKUP_FILE${NC}"
# Get file size
SIZE=$(du -h "$BACKUP_FILE" | cut -f1)
echo -e " Size: $SIZE"
else
echo -e "${RED}ERROR: Backup failed${NC}"
exit 1
fi
# GPG encryption (optional)
if [ -n "$GPG_RECIPIENT" ]; then
echo -e "${YELLOW}Encrypting backup with GPG...${NC}"
gpg --encrypt --recipient "$GPG_RECIPIENT" --output "${BACKUP_FILE}.gpg" "$BACKUP_FILE"
if [ $? -eq 0 ]; then
echo -e "${GREEN}✓ Backup encrypted: ${BACKUP_FILE}.gpg${NC}"
# Remove unencrypted backup
rm "$BACKUP_FILE"
BACKUP_FILE="${BACKUP_FILE}.gpg"
else
echo -e "${RED}WARNING: Encryption failed, keeping unencrypted backup${NC}"
fi
fi
# Cleanup old backups
echo -e "${YELLOW}Cleaning up old backups (older than $RETENTION_DAYS days)...${NC}"
find "$BACKUP_DIR" -name "gondulf_*.db*" -type f -mtime +$RETENTION_DAYS -delete
CLEANED=$(find "$BACKUP_DIR" -name "gondulf_*.db*" -type f -mtime +$RETENTION_DAYS | wc -l)
echo -e "${GREEN}✓ Removed $CLEANED old backup(s)${NC}"
# List recent backups
echo
echo "Recent backups:"
ls -lh "$BACKUP_DIR"/gondulf_*.db* 2>/dev/null | tail -5 || echo " No backups found"
echo
echo -e "${GREEN}Backup complete!${NC}"
```
**Restore Script**: `/scripts/restore_database.sh`
```bash
#!/bin/bash
set -euo pipefail
# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
DATA_DIR="${DATA_DIR:-$PROJECT_ROOT/data}"
DB_FILE="${DB_FILE:-$DATA_DIR/gondulf.db}"
# Colors
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m'
echo "========================================="
echo "Gondulf Database Restore"
echo "========================================="
echo
# Check arguments
if [ $# -ne 1 ]; then
echo "Usage: $0 <backup_file>"
echo "Example: $0 backups/gondulf_20251120_143000.db"
exit 1
fi
BACKUP_FILE="$1"
# Check if backup exists
if [ ! -f "$BACKUP_FILE" ]; then
echo -e "${RED}ERROR: Backup file not found: $BACKUP_FILE${NC}"
exit 1
fi
# Check if encrypted
if [[ "$BACKUP_FILE" == *.gpg ]]; then
echo -e "${YELLOW}Decrypting backup...${NC}"
DECRYPTED_FILE="${BACKUP_FILE%.gpg}"
gpg --decrypt --output "$DECRYPTED_FILE" "$BACKUP_FILE"
if [ $? -ne 0 ]; then
echo -e "${RED}ERROR: Decryption failed${NC}"
exit 1
fi
BACKUP_FILE="$DECRYPTED_FILE"
echo -e "${GREEN}✓ Backup decrypted${NC}"
fi
# Backup current database (if exists)
if [ -f "$DB_FILE" ]; then
CURRENT_BACKUP="${DB_FILE}.before_restore_$(date +%Y%m%d_%H%M%S)"
echo -e "${YELLOW}Backing up current database to: $CURRENT_BACKUP${NC}"
cp "$DB_FILE" "$CURRENT_BACKUP"
echo -e "${GREEN}✓ Current database backed up${NC}"
fi
# Restore backup
echo -e "${YELLOW}Restoring database...${NC}"
cp "$BACKUP_FILE" "$DB_FILE"
if [ $? -eq 0 ]; then
echo -e "${GREEN}✓ Database restored from: $BACKUP_FILE${NC}"
echo
echo -e "${YELLOW}NOTE: Please restart the Gondulf service${NC}"
else
echo -e "${RED}ERROR: Restore failed${NC}"
# Restore from backup if exists
if [ -f "$CURRENT_BACKUP" ]; then
cp "$CURRENT_BACKUP" "$DB_FILE"
echo -e "${YELLOW}Restored previous database${NC}"
fi
exit 1
fi
# Cleanup decrypted file if it was created
if [[ "$1" == *.gpg ]] && [ -f "${1%.gpg}" ]; then
rm "${1%.gpg}"
fi
echo -e "${GREEN}Restore complete!${NC}"
```
**Make scripts executable**:
```bash
chmod +x scripts/backup_database.sh
chmod +x scripts/restore_database.sh
```
**Cron Setup** (example):
```bash
# Backup daily at 2 AM
0 2 * * * /app/scripts/backup_database.sh >> /app/logs/backup.log 2>&1
```
### 4.4: Environment Variable Documentation
**File**: `/docs/deployment/environment-variables.md`
**Content**:
```markdown
# Environment Variables
All Gondulf configuration is done via environment variables with the `GONDULF_` prefix.
## Required Variables
### SECRET_KEY (Required)
**Variable**: `GONDULF_SECRET_KEY`
**Description**: Secret key for cryptographic operations (token generation, session security)
**Format**: String, minimum 32 characters
**Example**: `GONDULF_SECRET_KEY=$(python -c "import secrets; print(secrets.token_urlsafe(32))")`
**Security**: NEVER commit to version control. Generate unique value per deployment.
### BASE_URL (Required)
**Variable**: `GONDULF_BASE_URL`
**Description**: Base URL of the Gondulf server (used for metadata endpoint)
**Format**: URL with scheme (https://...)
**Example**: `GONDULF_BASE_URL=https://auth.example.com`
**Production**: Must be HTTPS
### SMTP Configuration (Required for Email Verification)
**Variable**: `GONDULF_SMTP_HOST`
**Description**: SMTP server hostname
**Example**: `GONDULF_SMTP_HOST=smtp.gmail.com`
**Variable**: `GONDULF_SMTP_PORT`
**Description**: SMTP server port
**Default**: `587`
**Common Values**: `587` (STARTTLS), `465` (implicit TLS), `25` (unencrypted)
**Variable**: `GONDULF_SMTP_USERNAME`
**Description**: SMTP authentication username
**Example**: `GONDULF_SMTP_USERNAME=noreply@example.com`
**Variable**: `GONDULF_SMTP_PASSWORD`
**Description**: SMTP authentication password
**Security**: Use app-specific password if available (Gmail, Outlook)
**Variable**: `GONDULF_SMTP_FROM_EMAIL`
**Description**: From address for verification emails
**Example**: `GONDULF_SMTP_FROM_EMAIL=noreply@example.com`
**Variable**: `GONDULF_SMTP_USE_TLS`
**Description**: Enable STARTTLS (port 587)
**Default**: `true`
**Values**: `true`, `false`
## Optional Variables
### Database Configuration
**Variable**: `GONDULF_DATABASE_URL`
**Description**: Database connection URL
**Default**: `sqlite:///./data/gondulf.db`
**Format**: SQLAlchemy connection string
**PostgreSQL Example**: `postgresql://user:pass@localhost/gondulf` (future)
### Security Settings
**Variable**: `GONDULF_DEBUG`
**Description**: Enable debug mode (disables HTTPS enforcement)
**Default**: `false`
**Values**: `true`, `false`
**Warning**: NEVER enable in production
**Variable**: `GONDULF_HSTS_MAX_AGE`
**Description**: HSTS max-age header value (seconds)
**Default**: `31536000` (1 year)
### Token Configuration
**Variable**: `GONDULF_TOKEN_TTL`
**Description**: Access token time-to-live (seconds)
**Default**: `3600` (1 hour)
**Range**: `300` (5 min) to `86400` (24 hours)
**Variable**: `GONDULF_TOKEN_CLEANUP_ENABLED`
**Description**: Enable automatic cleanup of expired tokens
**Default**: `true`
**Values**: `true`, `false`
**Variable**: `GONDULF_TOKEN_CLEANUP_INTERVAL`
**Description**: Token cleanup interval (seconds)
**Default**: `3600` (1 hour)
### Logging Configuration
**Variable**: `GONDULF_LOG_LEVEL`
**Description**: Logging level
**Default**: `INFO`
**Values**: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`
## Docker Environment Variables
When running in Docker, use `-e` flag or `--env-file`:
```bash
# Using -e flag
docker run -e GONDULF_SECRET_KEY=xxx -e GONDULF_BASE_URL=https://auth.example.com gondulf:latest
# Using --env-file
docker run --env-file .env gondulf:latest
```
## Docker Compose
Create `.env` file in project root:
```env
GONDULF_SECRET_KEY=your-secret-key-here
GONDULF_BASE_URL=https://auth.example.com
GONDULF_SMTP_HOST=smtp.gmail.com
GONDULF_SMTP_PORT=587
GONDULF_SMTP_USERNAME=noreply@example.com
GONDULF_SMTP_PASSWORD=your-password-here
GONDULF_SMTP_FROM_EMAIL=noreply@example.com
```
Then run:
```bash
docker-compose up -d
```
## Security Recommendations
1. **Generate SECRET_KEY**: Use `secrets.token_urlsafe(32)` or similar
2. **SMTP Password**: Use app-specific passwords (Gmail, Outlook)
3. **HTTPS Only**: Never expose HTTP in production
4. **Environment Files**: Add `.env` to `.gitignore`
5. **Docker Secrets**: Use Docker secrets in production (not environment variables)
## Validation
Gondulf validates configuration at startup. Invalid configuration will prevent application from starting (fail-fast).
Missing required variables will produce clear error messages:
```
ERROR: Required environment variable GONDULF_SECRET_KEY is not set
```
```
### Testing Requirements
**Dockerfile Tests** (`tests/docker/test_dockerfile.sh`):
1. Test image builds successfully
2. Test image runs as non-root user
3. Test health check passes
4. Test application starts
5. Test data volume persists
**docker-compose Tests** (`tests/docker/test_docker_compose.sh`):
1. Test docker-compose up succeeds
2. Test service accessible on port 8000
3. Test /health endpoint returns 200
4. Test docker-compose down cleans up
**Backup Script Tests** (`tests/scripts/test_backup.sh`):
1. Test backup creates file
2. Test backup with GPG encryption
3. Test backup cleanup removes old files
4. Test restore from backup
5. Test restore from encrypted backup
### Acceptance Criteria
- [ ] Dockerfile builds successfully
- [ ] Docker image runs as non-root user
- [ ] Health check passes
- [ ] docker-compose.yml starts service successfully
- [ ] Backup script creates backup successfully
- [ ] Backup script with GPG encryption works
- [ ] Restore script restores database successfully
- [ ] Environment variable documentation complete
- [ ] All tests pass (Docker, backup/restore)
---
## Component 5: Integration & End-to-End Tests
### Purpose
Implement comprehensive integration and end-to-end tests to verify complete authentication flows, endpoint integration, and W3C IndieAuth compliance.
### Specification References
- **v1.0.0 Roadmap**: Line 67 (P0 feature), Testing Strategy lines 275-287, Phase 5 lines 230-232
- **Testing Standards**: `/docs/standards/testing.md`
### Design Overview
Create integration test suite (20% of total tests) covering endpoint interactions and multi-component workflows. Create end-to-end test suite (10% of total tests) covering complete user journeys. Use FastAPI TestClient for HTTP testing, mock external dependencies (SMTP, DNS).
### Test Organization
**Directory Structure**:
```
tests/
├── unit/ # Existing unit tests (70%)
│ ├── test_*.py
│ └── ...
├── integration/ # NEW: Integration tests (20%)
│ ├── __init__.py
│ ├── conftest.py # Shared fixtures
│ ├── test_authorization_flow.py
│ ├── test_token_flow.py
│ ├── test_verification_flow.py
│ ├── test_metadata_endpoint.py
│ └── test_error_handling.py
├── e2e/ # NEW: End-to-end tests (10%)
│ ├── __init__.py
│ ├── conftest.py
│ ├── test_complete_auth_flow.py
│ ├── test_email_verification_flow.py
│ └── test_dns_verification_flow.py
└── security/ # NEW: Security tests
├── __init__.py
├── test_input_validation.py
├── test_timing_attacks.py
└── test_pii_logging.py
```
### 5.1: Integration Tests
**Purpose**: Test endpoint interactions and multi-component workflows without mocking application logic.
#### Test: Authorization Flow Integration
**File**: `tests/integration/test_authorization_flow.py`
**Scenarios**:
1. Valid authorization request returns consent page
2. User approval generates authorization code
3. User denial redirects with error
4. Missing parameters return error
5. Invalid redirect_uri returns error
6. State parameter preserved through flow
**Implementation Pattern**:
```python
import pytest
from fastapi.testclient import TestClient
from gondulf.main import app
from gondulf.dependencies import get_html_fetcher, get_dns_service
@pytest.fixture
def client():
"""TestClient with mocked external dependencies."""
# Mock HTML fetcher (no real HTTP requests)
def mock_html_fetcher():
class MockHTMLFetcher:
def fetch(self, url):
return '<html><link rel="me" href="mailto:user@example.com"></html>'
return MockHTMLFetcher()
# Mock DNS service (no real DNS queries)
def mock_dns_service():
class MockDNSService:
def verify_txt_record(self, domain):
return True
return MockDNSService()
# Override dependencies
app.dependency_overrides[get_html_fetcher] = mock_html_fetcher
app.dependency_overrides[get_dns_service] = mock_dns_service
yield TestClient(app)
# Cleanup
app.dependency_overrides.clear()
def test_authorization_request_valid(client):
"""Test valid authorization request returns consent page."""
response = client.get(
"/authorize",
params={
"me": "https://example.com",
"client_id": "https://client.example.com",
"redirect_uri": "https://client.example.com/callback",
"state": "random-state-value",
"response_type": "code"
}
)
assert response.status_code == 200
assert "authorize" in response.text.lower()
assert "example.com" in response.text
assert "client.example.com" in response.text
def test_authorization_approval_generates_code(client):
"""Test user approval generates authorization code and redirects."""
# First, get to consent page (this creates session/state)
response = client.get("/authorize", params={...})
# Then submit approval
response = client.post(
"/authorize",
data={
"authorized": "true",
"me": "https://example.com",
"client_id": "https://client.example.com",
"redirect_uri": "https://client.example.com/callback",
"state": "random-state-value"
},
allow_redirects=False
)
assert response.status_code == 302
assert "code=" in response.headers["location"]
assert "state=random-state-value" in response.headers["location"]
```
#### Test: Token Flow Integration
**File**: `tests/integration/test_token_flow.py`
**Scenarios**:
1. Valid code exchange returns token
2. Expired code returns error
3. Used code returns error
4. Mismatched client_id returns error
5. Mismatched redirect_uri returns error
6. Token response includes correct fields
**Implementation Pattern**:
```python
def test_token_exchange_valid(client, create_auth_code):
"""Test valid authorization code exchanges for token."""
# create_auth_code is a fixture that creates a valid code
code, metadata = create_auth_code(
me="https://example.com",
client_id="https://client.example.com",
redirect_uri="https://client.example.com/callback"
)
response = client.post(
"/token",
data={
"grant_type": "authorization_code",
"code": code,
"client_id": "https://client.example.com",
"redirect_uri": "https://client.example.com/callback",
"me": "https://example.com"
}
)
assert response.status_code == 200
json = response.json()
assert "access_token" in json
assert json["token_type"] == "Bearer"
assert json["me"] == "https://example.com"
assert len(json["access_token"]) == 43 # base64url(32 bytes)
```
#### Test: Verification Flow Integration
**File**: `tests/integration/test_verification_flow.py`
**Scenarios**:
1. Start verification sends email and returns success
2. Valid code verifies successfully
3. Invalid code returns error
4. Expired code returns error
5. Rate limiting prevents abuse
#### Test: Metadata Endpoint
**File**: `tests/integration/test_metadata_endpoint.py`
**Scenarios**:
1. Metadata endpoint returns valid JSON
2. All required fields present
3. URLs are valid
4. Cache-Control header present
#### Test: Error Handling
**File**: `tests/integration/test_error_handling.py`
**Scenarios**:
1. OAuth error responses formatted correctly
2. Error redirection preserves state parameter
3. Invalid redirect_uri shows error page (no redirect)
4. Server errors return 500 with error page
### 5.2: End-to-End Tests
**Purpose**: Test complete user journeys from start to finish.
#### Test: Complete Authentication Flow
**File**: `tests/e2e/test_complete_auth_flow.py`
**Scenario**: Simulate complete IndieAuth flow:
1. Client initiates authorization request
2. Server checks domain verification (DNS + email)
3. User verifies email (enters code)
4. User approves authorization
5. Client exchanges code for token
6. Token is valid and can be used
**Implementation**:
```python
@pytest.mark.e2e
def test_complete_authentication_flow(client, mock_smtp, mock_dns):
"""Test complete IndieAuth authentication flow end-to-end."""
# 1. Client initiates authorization
auth_response = client.get("/authorize", params={
"me": "https://user.example.com",
"client_id": "https://app.example.com",
"redirect_uri": "https://app.example.com/callback",
"state": "client-random-state",
"response_type": "code"
})
assert auth_response.status_code == 200
# 2. Server discovers email from rel=me (mocked)
# 3. Server sends verification email (captured by mock_smtp)
verify_response = client.post("/api/verify/start", json={
"domain": "user.example.com"
})
assert verify_response.json()["success"] is True
# Extract verification code from mock SMTP
verification_code = mock_smtp.get_last_email_code()
# 4. User submits verification code
code_response = client.post("/api/verify/code", json={
"domain": "user.example.com",
"code": verification_code
})
assert code_response.json()["success"] is True
# 5. User approves authorization
approve_response = client.post("/authorize", data={
"authorized": "true",
"me": "https://user.example.com",
"client_id": "https://app.example.com",
"redirect_uri": "https://app.example.com/callback",
"state": "client-random-state"
}, allow_redirects=False)
assert approve_response.status_code == 302
location = approve_response.headers["location"]
assert "code=" in location
assert "state=client-random-state" in location
# Extract authorization code from redirect
from urllib.parse import urlparse, parse_qs
parsed = urlparse(location)
params = parse_qs(parsed.query)
auth_code = params["code"][0]
# 6. Client exchanges code for token
token_response = client.post("/token", data={
"grant_type": "authorization_code",
"code": auth_code,
"client_id": "https://app.example.com",
"redirect_uri": "https://app.example.com/callback",
"me": "https://user.example.com"
})
assert token_response.status_code == 200
token_json = token_response.json()
assert "access_token" in token_json
assert token_json["me"] == "https://user.example.com"
# 7. Token is valid (verify in database)
# This would be a token introspection endpoint in future
token = token_json["access_token"]
assert len(token) == 43
```
#### Test: Email Verification Flow
**File**: `tests/e2e/test_email_verification_flow.py`
**Scenario**: Test email-based domain verification:
1. User starts verification
2. Email sent with code
3. User enters code
4. Domain marked as verified
#### Test: DNS Verification Flow
**File**: `tests/e2e/test_dns_verification_flow.py`
**Scenario**: Test DNS TXT record verification:
1. User adds TXT record
2. Server queries DNS
3. Domain verified via TXT record
4. Email verification still required (two-factor)
### 5.3: Security Tests
**Purpose**: Test security properties and attack resistance.
#### Test: Input Validation
**File**: `tests/security/test_input_validation.py`
**Scenarios**:
1. Malformed URLs rejected
2. SQL injection attempts blocked
3. XSS attempts escaped in templates
4. Open redirect attempts blocked
5. URL fragment in me parameter rejected
**Implementation**:
```python
@pytest.mark.security
def test_sql_injection_prevention(client):
"""Test SQL injection attempts are blocked."""
# Attempt SQL injection in me parameter
response = client.get("/authorize", params={
"me": "https://example.com'; DROP TABLE tokens; --",
"client_id": "https://app.example.com",
"redirect_uri": "https://app.example.com/callback",
"state": "state",
"response_type": "code"
})
# Should return error (invalid URL), not execute SQL
assert response.status_code in [400, 422]
# Verify tokens table still exists
from gondulf.dependencies import get_database
db = get_database()
result = db.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='tokens'")
assert result.fetchone() is not None
@pytest.mark.security
def test_xss_prevention_in_templates(client):
"""Test XSS attempts are escaped in HTML templates."""
xss_payload = "<script>alert('XSS')</script>"
response = client.get("/authorize", params={
"me": f"https://example.com/{xss_payload}",
"client_id": "https://app.example.com",
"redirect_uri": "https://app.example.com/callback",
"state": xss_payload,
"response_type": "code"
})
# XSS payload should be HTML-escaped in response
assert "<script>" not in response.text
assert "&lt;script&gt;" in response.text or response.status_code in [400, 422]
```
#### Test: Timing Attacks
**File**: `tests/security/test_timing_attacks.py`
**Scenarios**:
1. Token verification takes constant time
2. Code verification takes constant time
3. Email verification takes constant time
**Implementation**:
```python
import time
import statistics
@pytest.mark.security
def test_token_verification_constant_time(client, create_token):
"""Test token verification is resistant to timing attacks."""
valid_token, _ = create_token()
invalid_token = "x" * 43
# Measure verification time for valid token
valid_times = []
for _ in range(100):
start = time.perf_counter()
# Simulate token verification
response = client.get("/some-protected-endpoint",
headers={"Authorization": f"Bearer {valid_token}"})
end = time.perf_counter()
valid_times.append(end - start)
# Measure verification time for invalid token
invalid_times = []
for _ in range(100):
start = time.perf_counter()
response = client.get("/some-protected-endpoint",
headers={"Authorization": f"Bearer {invalid_token}"})
end = time.perf_counter()
invalid_times.append(end - start)
# Compare timing distributions
valid_mean = statistics.mean(valid_times)
invalid_mean = statistics.mean(invalid_times)
# Times should be similar (within 10% tolerance for noise)
ratio = valid_mean / invalid_mean
assert 0.9 < ratio < 1.1, f"Timing difference detected: {ratio:.2f}x"
```
#### Test: PII Logging
**File**: `tests/security/test_pii_logging.py`
**Scenarios**:
1. Email addresses not logged
2. Full tokens not logged
3. Full codes not logged
4. Only prefixes/hashes logged
**Implementation**:
```python
import logging
from unittest.mock import patch
@pytest.mark.security
def test_email_not_logged(client, caplog):
"""Test email addresses are not logged."""
with caplog.at_level(logging.DEBUG):
# Trigger email verification
response = client.post("/api/verify/start", json={
"domain": "example.com"
})
# Check all log messages
for record in caplog.records:
# Email should never appear in logs
assert "user@example.com" not in record.message
assert "@" not in record.message or "example.com" in record.message
@pytest.mark.security
def test_full_tokens_not_logged(client, create_token, caplog):
"""Test full access tokens are not logged."""
token, _ = create_token()
with caplog.at_level(logging.DEBUG):
# Trigger token usage
response = client.get("/some-endpoint",
headers={"Authorization": f"Bearer {token}"})
# Full token should never appear in logs
for record in caplog.records:
assert token not in record.message
# Prefix (first 8 chars) is acceptable
```
### 5.4: Shared Test Fixtures
**File**: `tests/integration/conftest.py`
**Fixtures**:
```python
import pytest
from fastapi.testclient import TestClient
from gondulf.main import app
from gondulf.storage import CodeStore
from gondulf.services.token_service import TokenService
import secrets
@pytest.fixture
def client():
"""FastAPI TestClient with mocked dependencies."""
# Setup mocks
# ...
yield TestClient(app)
# Cleanup
app.dependency_overrides.clear()
@pytest.fixture
def mock_smtp():
"""Mock SMTP server that captures sent emails."""
class MockSMTP:
def __init__(self):
self.sent_emails = []
def sendmail(self, from_addr, to_addrs, msg):
self.sent_emails.append({
"from": from_addr,
"to": to_addrs,
"message": msg
})
def get_last_email_code(self):
"""Extract verification code from last email."""
import re
last_email = self.sent_emails[-1]
match = re.search(r'\b\d{6}\b', last_email["message"])
return match.group(0) if match else None
return MockSMTP()
@pytest.fixture
def mock_dns():
"""Mock DNS resolver that always returns valid."""
class MockDNSResolver:
def verify_txt_record(self, domain):
return True
return MockDNSResolver()
@pytest.fixture
def create_auth_code():
"""Fixture to create authorization codes for testing."""
def _create(me, client_id, redirect_uri, state="test-state"):
code = secrets.token_urlsafe(32)
metadata = {
"me": me,
"client_id": client_id,
"redirect_uri": redirect_uri,
"state": state,
"created_at": time.time(),
"expires_at": time.time() + 600,
"used": False
}
# Store in CodeStore
code_store = CodeStore()
code_store.store(code, metadata, ttl=600)
return code, metadata
return _create
@pytest.fixture
def create_token():
"""Fixture to create access tokens for testing."""
def _create(me="https://example.com", client_id="https://app.example.com"):
token_service = TokenService(database=get_database())
token = token_service.generate_token(
me=me,
client_id=client_id,
scope=""
)
return token, {"me": me, "client_id": client_id}
return _create
```
### Test Execution
**Pytest Markers**:
```python
# pytest.ini or pyproject.toml
[tool.pytest.ini_options]
markers = [
"unit: Unit tests (fast, no external dependencies)",
"integration: Integration tests (TestClient, mocked external deps)",
"e2e: End-to-end tests (complete user journeys)",
"security: Security-focused tests (timing, injection, etc.)"
]
```
**Run Tests**:
```bash
# All tests
uv run pytest
# Unit tests only
uv run pytest -m unit
# Integration tests only
uv run pytest -m integration
# E2E tests only
uv run pytest -m e2e
# Security tests only
uv run pytest -m security
# With coverage
uv run pytest --cov=src/gondulf --cov-report=html --cov-report=term-missing
```
### Acceptance Criteria
- [ ] Integration test suite created (20+ tests)
- [ ] End-to-end test suite created (10+ tests)
- [ ] Security test suite created (15+ tests)
- [ ] All tests pass
- [ ] Test coverage ≥80% overall
- [ ] Critical path coverage ≥95% (auth, token, security)
- [ ] Fixtures for common test scenarios created
- [ ] Pytest markers configured
---
## Component 6: Real Client Testing
### Purpose
Verify IndieAuth server interoperability by testing with real IndieAuth clients to ensure W3C specification compliance and production readiness.
### Specification References
- **v1.0.0 Roadmap**: Phase 5 lines 239, 330, Success metrics line 535
- **W3C IndieAuth**: Full specification compliance required
### Design Overview
Test Gondulf with at least 2 real IndieAuth clients to verify protocol compliance and identify interoperability issues. Document testing methodology and results.
### Client Selection
**Criteria for Client Selection**:
1. Publicly available and documented
2. Known to be W3C IndieAuth compliant
3. Different implementations (diversity of clients)
4. Active maintenance
**Recommended Clients**:
1. **IndieLogin.com** (https://indielogin.com)
- Reference client by Aaron Parecki (IndieAuth spec author)
- PHP implementation
- Most authoritative for compliance testing
2. **Quill** (https://quill.p3k.io)
- Micropub client with IndieAuth support
- Real-world usage scenario
3. **Indigenous** (iOS/Android app)
- Mobile app with IndieAuth
- Tests mobile client scenarios
4. **Monocle** (https://monocle.p3k.io)
- Microsub client with IndieAuth
- Tests feed reader scenario
**Minimum Requirement**: Test with at least 2 different clients (recommendation: IndieLogin + one other).
### Testing Approach
#### Phase 1: Local Development Testing
**Setup**:
1. Run Gondulf locally with ngrok for HTTPS tunnel
2. Configure test domain with DNS TXT record and rel=me link
3. Use ngrok URL as `GONDULF_BASE_URL`
**Steps**:
```bash
# Start Gondulf locally
uv run uvicorn gondulf.main:app --reload --port 8000
# In another terminal, start ngrok
ngrok http 8000
# Note ngrok URL (e.g., https://abc123.ngrok.io)
# Set as BASE_URL:
export GONDULF_BASE_URL=https://abc123.ngrok.io
```
**Test Domain Setup**:
```bash
# Add DNS TXT record:
# _gondulf.test.example.com TXT "verified"
# Add to test.example.com homepage:
# <link rel="me" href="mailto:admin@test.example.com">
```
#### Phase 2: Test with IndieLogin
**Test Procedure**:
1. Go to https://indielogin.com
2. Enter test domain (https://test.example.com)
3. Click "Sign In"
4. Observe authorization flow:
- IndieLogin discovers Gondulf as authorization server
- IndieLogin redirects to Gondulf /authorize
- Gondulf displays consent screen
- User approves
- IndieLogin receives authorization code
- IndieLogin exchanges code for token
- IndieLogin displays success
**Expected Results**:
- [ ] IndieLogin discovers authorization endpoint via metadata
- [ ] Authorization request has all required parameters
- [ ] Consent screen displays correctly
- [ ] Authorization code generated and returned
- [ ] Token exchange succeeds
- [ ] Token response includes correct `me` value
- [ ] IndieLogin shows "Signed in as test.example.com"
**Debugging**:
- Monitor Gondulf logs for errors
- Check IndieLogin error messages
- Use browser dev tools to inspect redirects
- Verify DNS TXT record resolvable
#### Phase 3: Test with Secondary Client
**Test Procedure**: (Using Quill as example)
1. Go to https://quill.p3k.io
2. Click "Sign In"
3. Enter test domain
4. Follow authorization flow
5. Verify successful sign-in
**Expected Results**:
- [ ] Client discovers Gondulf endpoints
- [ ] Authorization flow completes
- [ ] Token issued successfully
- [ ] Client can use token (if applicable)
#### Phase 4: Error Scenario Testing
**Test Error Handling**:
1. **Invalid redirect_uri**: Client sends non-matching redirect_uri
- Expected: Error response
2. **Missing state**: Client omits state parameter
- Expected: Error response
3. **User denial**: User clicks "Deny" on consent screen
- Expected: Error redirect with `access_denied`
4. **Expired code**: Client delays token exchange beyond 10 minutes
- Expected: `invalid_grant` error
### Testing Checklist
**Discovery**:
- [ ] Metadata endpoint accessible
- [ ] Metadata contains correct endpoint URLs
- [ ] Clients can discover endpoints automatically
**Authorization Flow**:
- [ ] Client redirects to /authorize correctly
- [ ] All required parameters present
- [ ] Consent screen renders correctly
- [ ] User approval generates code
- [ ] User denial returns error
- [ ] State parameter preserved
- [ ] Redirect back to client succeeds
**Token Exchange**:
- [ ] Token endpoint accepts POST requests
- [ ] Valid code exchanges for token
- [ ] Token response has all required fields
- [ ] Token response `me` value matches request
- [ ] Used code returns error on second exchange
- [ ] Invalid code returns error
**Error Handling**:
- [ ] OAuth 2.0 error format correct
- [ ] Error descriptions helpful
- [ ] Error redirection preserves state
**Security**:
- [ ] HTTPS enforced (or localhost allowed in dev)
- [ ] Security headers present
- [ ] Open redirect prevented
- [ ] CSRF protection via state parameter
### Documentation
**File**: `/docs/testing/real-client-testing.md`
**Content**:
```markdown
# Real Client Testing Results
## Test Environment
- **Date**: YYYY-MM-DD
- **Gondulf Version**: 1.0.0
- **Test Domain**: test.example.com
- **DNS TXT Record**: _gondulf.test.example.com TXT "verified"
- **rel=me Link**: <link rel="me" href="mailto:admin@test.example.com">
## Client 1: IndieLogin.com
**Client**: https://indielogin.com
**Version**: Latest (as of test date)
**Implementation**: PHP (reference implementation)
### Test Results
- ✅ Discovery via metadata endpoint
- ✅ Authorization flow complete
- ✅ Token exchange successful
- ✅ Signed in as test.example.com
- ✅ Error handling correct (tested user denial)
### Issues Found
- None
### Screenshots
- [Authorization screen](screenshots/indielogin-authorize.png)
- [Success screen](screenshots/indielogin-success.png)
## Client 2: Quill
**Client**: https://quill.p3k.io
**Version**: Latest
**Implementation**: PHP
### Test Results
- ✅ Discovery successful
- ✅ Authorization flow complete
- ✅ Token issued
- ✅ Can create micropub posts (token works)
### Issues Found
- None
### Screenshots
- [Quill sign-in](screenshots/quill-signin.png)
- [Quill authorized](screenshots/quill-authorized.png)
## Compliance Summary
✅ Gondulf is fully compliant with W3C IndieAuth specification
✅ Successfully interoperates with 2 different clients
✅ No protocol deviations detected
✅ Ready for production use
## Recommendations
- Monitor logs for authentication errors
- Consider testing with additional clients (Indigenous, Monocle)
- Set up automated compliance testing (future)
```
### Troubleshooting Guide
**File**: `/docs/testing/troubleshooting-client-testing.md`
**Common Issues**:
| Issue | Cause | Solution |
|-------|-------|----------|
| Client can't discover endpoints | Metadata endpoint not accessible | Check BASE_URL, ensure metadata endpoint registered |
| "Invalid redirect_uri" error | Domain mismatch | Verify redirect_uri matches client_id domain |
| "Domain not verified" error | DNS TXT record missing | Add TXT record: `_gondulf.{domain}` = `verified` |
| "Email not found" error | rel=me link missing | Add `<link rel="me" href="mailto:...">` to homepage |
| Token exchange fails | Code expired or used | Check code TTL (10 minutes), ensure single-use |
| HTTPS errors | ngrok not running or BASE_URL wrong | Verify ngrok running, update BASE_URL in config |
### Acceptance Criteria
- [ ] Tested with at least 2 different IndieAuth clients
- [ ] All authorization flows complete successfully
- [ ] Token exchange successful with all clients
- [ ] Error scenarios tested and handled correctly
- [ ] Testing documentation created with results
- [ ] Screenshots/evidence of successful testing
- [ ] No W3C IndieAuth compliance issues found
- [ ] Troubleshooting guide created
---
## Component 7: Deployment Documentation
### Purpose
Provide comprehensive documentation for operators to install, configure, deploy, and troubleshoot Gondulf in production environments.
### Specification References
- **v1.0.0 Roadmap**: Phase 5 lines 229, 253, Release Checklist lines 443-451
### Design Overview
Create four deployment guides: Installation, Configuration, Deployment, and Troubleshooting. Each guide should be clear, complete, and tested by following the steps.
### 7.1: Installation Guide
**File**: `/docs/deployment/installation.md`
**Content**:
```markdown
# Installation Guide
This guide covers installing Gondulf on a fresh server.
## Prerequisites
- Linux server (Ubuntu 22.04 LTS recommended)
- Domain name with DNS access
- Email account with SMTP access (Gmail, Mailgun, etc.)
- Docker and Docker Compose installed
## System Requirements
**Minimum**:
- 1 CPU core
- 1 GB RAM
- 10 GB disk space
- Ubuntu 22.04 LTS or similar
**Recommended**:
- 2 CPU cores
- 2 GB RAM
- 20 GB disk space
- Ubuntu 22.04 LTS
## Step 1: Install Docker
```bash
# Update package index
sudo apt update
# Install Docker
sudo apt install -y docker.io docker-compose
# Add user to docker group
sudo usermod -aG docker $USER
# Reload group membership (or logout/login)
newgrp docker
# Verify installation
docker --version
docker-compose --version
```
## Step 2: Clone Repository
```bash
# Create directory
mkdir -p /opt/gondulf
cd /opt/gondulf
# Clone repository
git clone https://github.com/yourusername/gondulf.git .
# Or download release
wget https://github.com/yourusername/gondulf/releases/download/v1.0.0/gondulf-1.0.0.tar.gz
tar xzf gondulf-1.0.0.tar.gz
```
## Step 3: Generate Secret Key
```bash
# Generate SECRET_KEY
python3 -c "import secrets; print(secrets.token_urlsafe(32))"
# Save output for next step
```
## Step 4: Configure Environment
```bash
# Copy example config
cp .env.example .env
# Edit configuration
nano .env
```
Set required variables:
```env
GONDULF_SECRET_KEY=<generated-in-step-3>
GONDULF_BASE_URL=https://auth.yourdomain.com
GONDULF_SMTP_HOST=smtp.gmail.com
GONDULF_SMTP_PORT=587
GONDULF_SMTP_USERNAME=noreply@yourdomain.com
GONDULF_SMTP_PASSWORD=<your-app-password>
GONDULF_SMTP_FROM_EMAIL=noreply@yourdomain.com
```
See [Configuration Guide](configuration.md) for all options.
## Step 5: Configure DNS
Add DNS TXT record for domain verification:
```
Type: TXT
Name: _gondulf.yourdomain.com
Value: verified
TTL: 3600
```
Verify DNS propagation:
```bash
dig TXT _gondulf.yourdomain.com
```
## Step 6: Build Docker Image
```bash
# Build image
docker-compose build
# Verify image created
docker images | grep gondulf
```
## Step 7: Start Service
```bash
# Start in background
docker-compose up -d
# Check logs
docker-compose logs -f gondulf
# Verify health
curl http://localhost:8000/health
```
Expected response:
```json
{"status": "healthy"}
```
## Step 8: Configure Reverse Proxy
### Using Nginx
```nginx
server {
listen 80;
server_name auth.yourdomain.com;
# Redirect HTTP to HTTPS
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2;
server_name auth.yourdomain.com;
# SSL configuration (use Let's Encrypt)
ssl_certificate /etc/letsencrypt/live/auth.yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/auth.yourdomain.com/privkey.pem;
# Proxy to Gondulf
location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
### Using Caddy
```caddyfile
auth.yourdomain.com {
reverse_proxy localhost:8000
}
```
Caddy automatically handles HTTPS with Let's Encrypt.
## Step 9: Verify Installation
```bash
# Check metadata endpoint
curl https://auth.yourdomain.com/.well-known/oauth-authorization-server
# Should return JSON with endpoints
```
## Step 10: Set Up Backups
```bash
# Test backup script
./scripts/backup_database.sh
# Add to crontab (daily at 2 AM)
crontab -e
```
Add line:
```
0 2 * * * /opt/gondulf/scripts/backup_database.sh >> /opt/gondulf/logs/backup.log 2>&1
```
## Next Steps
- Read [Configuration Guide](configuration.md) for advanced options
- Read [Deployment Guide](deployment.md) for production best practices
- Read [Troubleshooting Guide](troubleshooting.md) for common issues
- Test authentication with IndieLogin.com
## Upgrading
```bash
# Pull latest code
git pull
# Rebuild image
docker-compose build
# Restart service
docker-compose up -d
# Check logs
docker-compose logs -f gondulf
```
```
### 7.2: Configuration Guide
**File**: `/docs/deployment/configuration.md`
**Content**: (Expand environment-variables.md with examples and best practices)
**Sections**:
1. Required Configuration
2. Optional Configuration
3. Security Configuration
4. SMTP Configuration (detailed examples for Gmail, Mailgun, SendGrid)
5. DNS Configuration
6. Performance Tuning
7. Monitoring Configuration
8. Example Configurations (development, staging, production)
### 7.3: Deployment Guide
**File**: `/docs/deployment/deployment.md`
**Content**:
```markdown
# Deployment Guide
This guide covers deploying Gondulf to production.
## Production Checklist
Before deploying to production, verify:
- [ ] SECRET_KEY is unique and strong (32+ characters)
- [ ] BASE_URL uses HTTPS
- [ ] SMTP credentials are valid and tested
- [ ] DNS TXT record is set and verified
- [ ] Reverse proxy configured with SSL/TLS
- [ ] Firewall configured (only 80/443 exposed)
- [ ] Backups scheduled (daily recommended)
- [ ] Logs monitored
- [ ] Health check passing
## Deployment Strategies
### Single Server Deployment
Simplest deployment for small scale (<1000 users):
```
┌─────────────────────┐
│ Reverse Proxy │ (Nginx/Caddy with SSL)
│ (Port 80/443) │
└──────────┬──────────┘
│
┌──────────▼──────────┐
│ Gondulf Docker │ (Port 8000)
│ Container │
└──────────┬──────────┘
│
┌──────────▼──────────┐
│ SQLite Database │ (Volume mount)
│ /app/data/ │
└─────────────────────┘
```
**Pros**:
- Simple setup
- Low cost
- Easy to maintain
**Cons**:
- Single point of failure
- Limited scalability
### High Availability Deployment (Future)
For larger scale or high availability requirements:
```
┌─────────────────────┐
│ Load Balancer │
└──────────┬──────────┘
│
┌──────┴──────┐
│ │
┌───▼───┐ ┌───▼───┐
│Gondulf│ │Gondulf│
│ 1 │ │ 2 │
└───┬───┘ └───┬───┘
│ │
└──────┬──────┘
│
┌──────────▼──────────┐
│ PostgreSQL │
│ (Shared DB) │
└─────────────────────┘
```
**Note**: Requires PostgreSQL support (planned for v1.2.0).
## Security Best Practices
### 1. HTTPS Only
- Never expose HTTP in production
- Use valid SSL/TLS certificates (Let's Encrypt recommended)
- Enable HSTS headers (Gondulf does this automatically)
### 2. Firewall Configuration
```bash
# Allow SSH (careful!)
sudo ufw allow 22/tcp
# Allow HTTP/HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
# Enable firewall
sudo ufw enable
```
### 3. Secret Management
- Never commit .env to version control
- Use Docker secrets in production (not environment variables)
- Rotate SECRET_KEY periodically (requires token regeneration)
### 4. Database Security
```bash
# Set restrictive permissions
chmod 600 /opt/gondulf/data/gondulf.db
chown gondulf:gondulf /opt/gondulf/data/gondulf.db
# Enable encrypted filesystem (LUKS recommended)
```
### 5. Monitoring
- Monitor logs for authentication failures
- Monitor disk space (SQLite database growth)
- Monitor backup success
- Set up alerts for service downtime
## Backup Strategy
### Daily Automated Backups
```bash
# Backup script runs daily at 2 AM
0 2 * * * /opt/gondulf/scripts/backup_database.sh
```
### Backup Retention
- Keep 7 daily backups
- Keep 4 weekly backups
- Keep 12 monthly backups
### Backup Testing
- Test restore procedure quarterly
- Verify backup integrity monthly
### Off-site Backups
```bash
# Sync to S3 (example)
aws s3 sync /opt/gondulf/backups/ s3://my-bucket/gondulf-backups/
```
## Monitoring
### Health Check
```bash
# Check every minute
* * * * * curl -f http://localhost:8000/health || systemctl restart docker-compose
```
### Log Monitoring
```bash
# Monitor for errors
docker-compose logs -f gondulf | grep ERROR
# Monitor authentication activity
docker-compose logs -f gondulf | grep "Authorization granted"
```
### Disk Usage
```bash
# Check database size
du -h /opt/gondulf/data/gondulf.db
# Check backup size
du -sh /opt/gondulf/backups/
```
## Scaling Considerations
### Current Limits (v1.0.0)
- **Users**: 10-100 domains (SQLite limit ~100 concurrent users)
- **Requests**: ~100 requests/second (single server)
- **Database**: ~100 MB typical (1000 tokens)
### When to Upgrade
- Database file >1 GB
- Response time >500ms
- Need high availability
- >1000 domains
### Upgrade Path (v1.2.0+)
- Migrate to PostgreSQL
- Add load balancer
- Add Redis for session storage
- Add metrics (Prometheus + Grafana)
## Troubleshooting
See [Troubleshooting Guide](troubleshooting.md) for common issues.
## Maintenance
### Regular Tasks
- **Daily**: Check logs for errors
- **Weekly**: Verify backup success
- **Monthly**: Review disk usage
- **Quarterly**: Test restore procedure
- **Annually**: Review security (update dependencies, rotate secrets)
### Upgrades
```bash
# Backup before upgrade
./scripts/backup_database.sh
# Pull latest version
git pull
# Rebuild image
docker-compose build
# Restart
docker-compose up -d
# Verify
curl https://auth.yourdomain.com/health
```
```
### 7.4: Troubleshooting Guide
**File**: `/docs/deployment/troubleshooting.md`
**Content**:
```markdown
# Troubleshooting Guide
Common issues and solutions for Gondulf.
## Service Won't Start
### Error: "Required environment variable GONDULF_SECRET_KEY is not set"
**Cause**: Missing or invalid .env file
**Solution**:
```bash
# Verify .env file exists
ls -la .env
# Check SECRET_KEY is set
grep SECRET_KEY .env
# Generate new SECRET_KEY if needed
python3 -c "import secrets; print(secrets.token_urlsafe(32))"
```
### Error: "Database connection failed"
**Cause**: Database file not accessible or corrupted
**Solution**:
```bash
# Check database file exists
ls -la data/gondulf.db
# Check permissions
chmod 600 data/gondulf.db
# If corrupted, restore from backup
./scripts/restore_database.sh backups/gondulf_YYYYMMDD_HHMMSS.db
```
### Error: "Port 8000 already in use"
**Cause**: Another service using port 8000
**Solution**:
```bash
# Find process using port
sudo lsof -i :8000
# Kill process or change Gondulf port in docker-compose.yml
```
## Authentication Issues
### Error: "Domain not verified"
**Cause**: DNS TXT record not found or incorrect
**Solution**:
```bash
# Verify TXT record exists
dig TXT _gondulf.yourdomain.com
# Expected output:
# _gondulf.yourdomain.com. 3600 IN TXT "verified"
# If missing, add to DNS and wait for propagation (up to 24 hours)
```
### Error: "Email not found"
**Cause**: rel=me link missing from homepage
**Solution**:
```bash
# Check homepage for rel=me link
curl https://yourdomain.com | grep 'rel="me"'
# Expected: <link rel="me" href="mailto:admin@yourdomain.com">
# Add to <head> of homepage if missing
```
### Error: "Failed to send verification email"
**Cause**: SMTP configuration incorrect or credentials invalid
**Solution**:
```bash
# Check SMTP settings in .env
grep SMTP .env
# Test SMTP connection
python3 -c "
import smtplib
smtp = smtplib.SMTP('smtp.gmail.com', 587)
smtp.starttls()
smtp.login('user@gmail.com', 'password')
print('SMTP connection successful')
smtp.quit()
"
# For Gmail, ensure "App Passwords" used (not account password)
# https://support.google.com/accounts/answer/185833
```
## Client Compatibility Issues
### Error: "Client can't discover authorization endpoint"
**Cause**: Metadata endpoint not accessible
**Solution**:
```bash
# Test metadata endpoint
curl https://auth.yourdomain.com/.well-known/oauth-authorization-server
# Should return JSON with authorization_endpoint, token_endpoint
# If 404, verify BASE_URL is correct and metadata router registered
```
### Error: "invalid_redirect_uri"
**Cause**: redirect_uri doesn't match client_id domain
**Solution**:
- Verify client_id and redirect_uri have same domain
- Or client_id is parent domain of redirect_uri
- Example: client_id=https://app.example.com, redirect_uri=https://app.example.com/callback (OK)
- Example: client_id=https://app.example.com, redirect_uri=https://evil.com (NOT OK)
### Error: "Token exchange failed: invalid_grant"
**Cause**: Authorization code expired or already used
**Solution**:
- Authorization codes expire after 10 minutes
- Codes are single-use only
- Client must exchange code immediately after receiving it
- Check client isn't caching and reusing old codes
## Performance Issues
### Slow Response Times
**Cause**: Database growing too large or disk I/O slow
**Solution**:
```bash
# Check database size
du -h data/gondulf.db
# Cleanup expired tokens
docker-compose exec gondulf python -c "
from gondulf.dependencies import get_token_service
token_service = get_token_service()
deleted = token_service.cleanup_expired_tokens()
print(f'Deleted {deleted} expired tokens')
"
# Consider migrating to PostgreSQL if database >1 GB
```
### High Memory Usage
**Cause**: In-memory storage (authorization codes, verification codes) growing
**Solution**:
```bash
# Restart service to clear in-memory storage
docker-compose restart gondulf
# Check for code leaks (codes not expiring properly)
# Review logs for "cleanup" messages
docker-compose logs gondulf | grep cleanup
```
## Security Issues
### Warning: "HTTP request in production"
**Cause**: HTTPS enforcement not working
**Solution**:
```bash
# Verify DEBUG mode is disabled
grep DEBUG .env
# Should be: GONDULF_DEBUG=false
# Verify reverse proxy is forwarding X-Forwarded-Proto header
# Nginx: proxy_set_header X-Forwarded-Proto $scheme;
# Restart service
docker-compose restart gondulf
```
### Error: "HSTS header not present"
**Cause**: Not using HTTPS or DEBUG mode enabled
**Solution**:
- HSTS only added when HTTPS and DEBUG=false
- Verify reverse proxy is terminating SSL and forwarding to Gondulf
- Check `curl -I https://auth.yourdomain.com | grep Strict-Transport-Security`
## Backup and Restore Issues
### Error: "Backup failed: database locked"
**Cause**: Database in use during backup
**Solution**:
```bash
# Stop service before backup
docker-compose stop gondulf
# Run backup
./scripts/backup_database.sh
# Restart service
docker-compose start gondulf
# Or use SQLite online backup (script already does this)
```
### Error: "Restore failed: permission denied"
**Cause**: Incorrect file permissions
**Solution**:
```bash
# Fix permissions
sudo chown gondulf:gondulf data/gondulf.db
chmod 600 data/gondulf.db
# Retry restore
./scripts/restore_database.sh backups/gondulf_YYYYMMDD_HHMMSS.db
```
## Logs and Debugging
### Enable Debug Logging
```bash
# Set log level to DEBUG in .env
GONDULF_LOG_LEVEL=DEBUG
# Restart service
docker-compose restart gondulf
# View debug logs
docker-compose logs -f gondulf
```
### Common Log Messages
**Normal**:
```
[INFO] Authorization granted for example.com to client.example.com
[INFO] Token generated: abc12345...
[INFO] Email verification sent for domain example.com
```
**Errors**:
```
[ERROR] Failed to send email: SMTP authentication failed
[ERROR] Database connection lost
[ERROR] DNS verification failed for domain example.com
```
**Security Events**:
```
[WARNING] Code replay attack detected: abc12345...
[WARNING] Invalid redirect_uri: https://evil.com
[ERROR] Failed authentication attempt for domain example.com
```
## Getting Help
### Before Asking for Help
1. Check logs: `docker-compose logs gondulf`
2. Verify configuration: `cat .env`
3. Check health endpoint: `curl http://localhost:8000/health`
4. Review this troubleshooting guide
### Where to Get Help
- GitHub Issues: https://github.com/yourusername/gondulf/issues
- Documentation: https://gondulf.readthedocs.io/
- Email: support@gondulf.example.com (if applicable)
### Information to Include
When reporting issues, include:
1. Gondulf version (`docker images | grep gondulf`)
2. Operating system and version
3. Relevant logs (redact sensitive information)
4. Steps to reproduce
5. Expected vs actual behavior
```
### API Documentation
**File**: `/docs/api/openapi.json`
**Generation**: Use FastAPI's built-in OpenAPI generation:
```bash
# Generate OpenAPI spec
curl http://localhost:8000/openapi.json > docs/api/openapi.json
# Or use FastAPI to serve docs
# Available at: http://localhost:8000/docs (Swagger UI)
# Available at: http://localhost:8000/redoc (ReDoc)
```
**Manual Enhancement**: Add descriptions, examples, and schemas to routers:
```python
@router.post(
"/token",
response_model=TokenResponse,
responses={
400: {"description": "Invalid request or expired code"},
401: {"description": "Client authentication failed"}
},
summary="Exchange authorization code for access token",
description="""
OAuth 2.0 token endpoint for authorization code exchange.
The client exchanges a valid authorization code for an access token.
Codes are single-use and expire after 10 minutes.
"""
)
async def token_endpoint(...):
pass
```
### Acceptance Criteria
- [ ] Installation guide created and tested
- [ ] Configuration guide created with examples
- [ ] Deployment guide created with best practices
- [ ] Troubleshooting guide created with common issues
- [ ] API documentation generated (OpenAPI)
- [ ] All guides reviewed for clarity and accuracy
- [ ] External reviewer can install and deploy following guides
- [ ] Screenshots/examples included where helpful
---
## Implementation Plan
### Phase 4a: Complete Phase 3 (Estimated: 2-3 days)
**Tasks**:
1. Implement metadata endpoint (0.5 day)
2. Implement h-app parser service (1 day)
3. Integrate h-app with authorization endpoint (0.5 day)
4. Test metadata and h-app functionality (0.5 day)
5. Update documentation (0.5 day)
**Dependencies**: None (can start immediately)
**Deliverable**: Phase 3 100% complete
### Phase 4b: Security Hardening (Estimated: 3-5 days)
**Tasks**:
1. Implement security headers middleware (0.5 day)
2. Implement HTTPS enforcement middleware (0.5 day)
3. Conduct input sanitization audit (1 day)
4. Conduct PII logging audit and fixes (1 day)
5. Write security tests (1-2 days)
6. Document security measures (0.5 day)
**Dependencies**: None (can start immediately)
**Deliverable**: Phase 4 100% complete
### Phase 5a: Deployment Configuration (Estimated: 2-3 days)
**Tasks**:
1. Create Dockerfile with multi-stage build (1 day)
2. Create docker-compose.yml (0.5 day)
3. Create backup/restore scripts (0.5 day)
4. Write environment variable documentation (0.5 day)
5. Test Docker deployment (0.5 day)
**Dependencies**: None (can start immediately)
**Deliverable**: Deployment configuration complete
### Phase 5b: Comprehensive Testing (Estimated: 5-7 days)
**Tasks**:
1. Write integration tests (2-3 days)
2. Write end-to-end tests (2 days)
3. Write security tests (1 day)
4. Run all tests and fix failures (1 day)
5. Achieve coverage goals (1 day)
**Dependencies**: Phases 4a, 4b complete (for accurate testing)
**Deliverable**: Test suite complete with 80%+ coverage
### Phase 5c: Real Client Testing (Estimated: 2-3 days)
**Tasks**:
1. Set up test environment with ngrok (0.5 day)
2. Test with IndieLogin (1 day)
3. Test with second client (Quill or other) (1 day)
4. Document testing results (0.5 day)
**Dependencies**: Phases 4a, 4b, 5a complete (for working deployment)
**Deliverable**: Successful testing with 2+ real clients
### Phase 5d: Documentation (Estimated: 2-3 days)
**Tasks**:
1. Write installation guide (0.5 day)
2. Write configuration guide (0.5 day)
3. Write deployment guide (0.5 day)
4. Write troubleshooting guide (0.5 day)
5. Generate API documentation (0.25 day)
6. Review and test all documentation (0.25 day)
**Dependencies**: All previous phases complete (for accurate documentation)
**Deliverable**: Complete deployment documentation
### Critical Path
**Parallel Work Possible**:
- Phase 4a, 4b, 5a can be done in parallel (independent tasks)
- Phase 5b depends on 4a, 4b (need complete functionality to test)
- Phase 5c depends on 4a, 4b, 5a (need working deployment)
- Phase 5d depends on all phases (documents everything)
**Fastest Path** (with parallelization):
- Week 1: Phases 4a, 4b, 5a in parallel (3-5 days)
- Week 2: Phase 5b (5-7 days)
- Week 3: Phase 5c and 5d in parallel (2-3 days)
**Total Estimated Time**: 10-15 days (2-3 weeks calendar time)
---
## Summary of Deliverables
| Component | Files Created | Estimated Effort |
|-----------|---------------|------------------|
| 1. Metadata Endpoint | `/src/gondulf/routers/metadata.py` | XS (<1 day) |
| 2. h-app Parser | `/src/gondulf/services/h_app_parser.py` | S (1-2 days) |
| 3. Security Hardening | `/src/gondulf/middleware/*.py`, PII fixes | S (3-5 days) |
| 4. Deployment Config | `/Dockerfile`, `/docker-compose.yml`, `/scripts/*.sh` | S (2-3 days) |
| 5. Integration/E2E Tests | `/tests/integration/*.py`, `/tests/e2e/*.py` | M (5-7 days) |
| 6. Real Client Testing | `/docs/testing/*.md` | M (2-3 days) |
| 7. Deployment Docs | `/docs/deployment/*.md` | M (2-3 days) |
**Total Estimated Effort**: 15-24 days
**Realistic with Parallelization**: 10-15 days
**Conservative Estimate**: 15-18 days
---
## Risk Assessment
### Technical Risks
| Risk | Likelihood | Impact | Mitigation |
|------|------------|--------|------------|
| Real client testing reveals protocol issues | Medium | High | Early testing with IndieLogin (reference implementation) |
| h-app parsing fails on real sites | Medium | Medium | Robust fallback to domain name |
| Docker deployment issues | Low | High | Test early, document thoroughly |
| Security test failures | Medium | High | Comprehensive security review before tests |
| Integration test coverage gaps | Medium | Medium | Systematic coverage of all endpoints |
### Schedule Risks
| Risk | Likelihood | Impact | Mitigation |
|------|------------|--------|------------|
| Testing takes longer than estimated | High | Medium | Allocate buffer time, prioritize P0 tests |
| Real client testing reveals bugs | Medium | High | Early testing, iterative fixes |
| Documentation takes longer than estimated | Medium | Low | Start documentation in parallel with development |
### Quality Risks
| Risk | Likelihood | Impact | Mitigation |
|------|------------|--------|------------|
| Test coverage doesn't reach 80% | Low | High | Systematic testing, coverage monitoring |
| Security vulnerabilities discovered | Medium | Critical | Security-focused testing, external review recommended |
| Documentation incomplete or incorrect | Low | Medium | External reviewer tests documentation |
---
## Success Criteria
This design is successful when:
- [ ] All 7 critical components implemented and tested
- [ ] v1.0.0 Release Checklist 100% complete
- [ ] Test coverage ≥80% overall, ≥95% critical paths
- [ ] Security tests pass (no vulnerabilities found)
- [ ] Successfully tested with ≥2 real IndieAuth clients
- [ ] Docker deployment successful
- [ ] Documentation complete and verified by external reviewer
- [ ] No P0 gaps remaining from gap analysis
---
## Appendix: Design Decisions
### Why mf2py for h-app Parsing?
**Decision**: Use mf2py library instead of manual BeautifulSoup parsing.
**Rationale**:
- Standard microformats parser (used by IndieWeb community)
- Handles edge cases (nested microformats, multiple h-apps)
- Actively maintained
- Small dependency (no security concerns)
**Alternative Considered**: Manual BeautifulSoup parsing
- **Rejected**: Would need to reimplement microformat parsing logic
### Why Multi-Stage Dockerfile?
**Decision**: Use multi-stage build with builder and runtime stages.
**Rationale**:
- Smaller final image (no build tools in runtime)
- Faster deployments (smaller image size)
- Security (fewer packages in production image)
- Standard Docker best practice
**Alternative Considered**: Single-stage build
- **Rejected**: Larger image, slower deployments
### Why SQLite .backup Instead of File Copy?
**Decision**: Use SQLite's .backup command for backups.
**Rationale**:
- Handles locked database (consistent snapshot)
- No downtime required
- Recommended by SQLite documentation
- Reliable and tested
**Alternative Considered**: File copy with service stop
- **Rejected**: Requires downtime
### Why Integration Tests with TestClient?
**Decision**: Use FastAPI TestClient for integration tests instead of full HTTP server.
**Rationale**:
- Faster execution (no network overhead)
- Easier to run in CI/CD
- Full control over environment
- Same code paths as production (ASGI)
**Alternative Considered**: Real HTTP server with requests library
- **Rejected**: Slower, more complex setup
---
## Next Steps
**For Developer**:
1. Review this design document thoroughly
2. Ask clarification questions if any design is ambiguous
3. Implement components in order suggested by Implementation Plan
4. Follow testing requirements for each component
5. Create implementation report after completion
6. Signal: "IMPLEMENTATION COMPLETE: Critical Components for v1.0.0"
**For Architect**:
1. Answer any clarification questions from Developer
2. Review implementation report when complete
3. Verify all acceptance criteria met
4. Approve or request changes
5. Update roadmap for v1.0.0 release
Reference in New Issue View Git Blame Copy Permalink