Phil Skentelbery
a7e0af9c2c
Complete architectural documentation for: - Migration race condition fix with database locking - IndieAuth endpoint discovery implementation - Security considerations and migration guides New documentation: - ADR-030-CORRECTED: IndieAuth endpoint discovery decision - ADR-031: Endpoint discovery implementation details - Architecture docs on endpoint discovery - Migration guide for removed TOKEN_ENDPOINT - Security analysis of endpoint discovery - Implementation and analysis reports
492 lines
13 KiB
Markdown
492 lines
13 KiB
Markdown
# Migration Guide: Fixing Hardcoded IndieAuth Endpoints
|
|
|
|
## Overview
|
|
|
|
This guide explains how to migrate from the **incorrect** hardcoded endpoint implementation to the **correct** dynamic endpoint discovery implementation that actually follows the IndieAuth specification.
|
|
|
|
## The Problem We're Fixing
|
|
|
|
### What's Currently Wrong
|
|
|
|
```python
|
|
# WRONG - auth_external.py (hypothetical incorrect implementation)
|
|
class ExternalTokenVerifier:
|
|
def __init__(self):
|
|
# FATAL FLAW: Hardcoded endpoint
|
|
self.token_endpoint = "https://tokens.indieauth.com/token"
|
|
|
|
def verify_token(self, token):
|
|
# Uses hardcoded endpoint for ALL users
|
|
response = requests.get(
|
|
self.token_endpoint,
|
|
headers={'Authorization': f'Bearer {token}'}
|
|
)
|
|
return response.json()
|
|
```
|
|
|
|
### Why It's Wrong
|
|
|
|
1. **Not IndieAuth**: This completely violates the IndieAuth specification
|
|
2. **No User Choice**: Forces all users to use the same provider
|
|
3. **Security Risk**: Single point of failure for all authentications
|
|
4. **No Flexibility**: Users can't change or choose providers
|
|
|
|
## The Correct Implementation
|
|
|
|
### Step 1: Remove Hardcoded Configuration
|
|
|
|
**Remove from config files:**
|
|
|
|
```ini
|
|
# DELETE THESE LINES - They are wrong!
|
|
TOKEN_ENDPOINT=https://tokens.indieauth.com/token
|
|
AUTHORIZATION_ENDPOINT=https://indieauth.com/auth
|
|
```
|
|
|
|
**Keep only:**
|
|
|
|
```ini
|
|
# CORRECT - Only the admin's identity URL
|
|
ADMIN_ME=https://admin.example.com/
|
|
```
|
|
|
|
### Step 2: Implement Endpoint Discovery
|
|
|
|
**Create `endpoint_discovery.py`:**
|
|
|
|
```python
|
|
"""
|
|
IndieAuth Endpoint Discovery
|
|
Implements: https://www.w3.org/TR/indieauth/#discovery-by-clients
|
|
"""
|
|
|
|
import re
|
|
from typing import Dict, Optional
|
|
from urllib.parse import urljoin, urlparse
|
|
import httpx
|
|
from bs4 import BeautifulSoup
|
|
|
|
class EndpointDiscovery:
|
|
"""Discovers IndieAuth endpoints from profile URLs"""
|
|
|
|
def __init__(self, timeout: int = 5):
|
|
self.timeout = timeout
|
|
self.client = httpx.Client(
|
|
timeout=timeout,
|
|
follow_redirects=True,
|
|
limits=httpx.Limits(max_redirects=5)
|
|
)
|
|
|
|
def discover(self, profile_url: str) -> Dict[str, str]:
|
|
"""
|
|
Discover IndieAuth endpoints from a profile URL
|
|
|
|
Args:
|
|
profile_url: The user's profile URL (their identity)
|
|
|
|
Returns:
|
|
Dictionary with 'authorization_endpoint' and 'token_endpoint'
|
|
|
|
Raises:
|
|
DiscoveryError: If discovery fails
|
|
"""
|
|
# Ensure HTTPS in production
|
|
if not self._is_development() and not profile_url.startswith('https://'):
|
|
raise DiscoveryError("Profile URL must use HTTPS")
|
|
|
|
try:
|
|
response = self.client.get(profile_url)
|
|
response.raise_for_status()
|
|
except Exception as e:
|
|
raise DiscoveryError(f"Failed to fetch profile: {e}")
|
|
|
|
endpoints = {}
|
|
|
|
# 1. Check HTTP Link headers (highest priority)
|
|
link_header = response.headers.get('Link', '')
|
|
if link_header:
|
|
endpoints.update(self._parse_link_header(link_header, profile_url))
|
|
|
|
# 2. Check HTML link elements
|
|
if 'text/html' in response.headers.get('Content-Type', ''):
|
|
endpoints.update(self._extract_from_html(
|
|
response.text,
|
|
profile_url
|
|
))
|
|
|
|
# Validate we found required endpoints
|
|
if 'token_endpoint' not in endpoints:
|
|
raise DiscoveryError("No token endpoint found in profile")
|
|
|
|
return endpoints
|
|
|
|
def _parse_link_header(self, header: str, base_url: str) -> Dict[str, str]:
|
|
"""Parse HTTP Link header for endpoints"""
|
|
endpoints = {}
|
|
|
|
# Parse Link: <url>; rel="relation"
|
|
pattern = r'<([^>]+)>;\s*rel="([^"]+)"'
|
|
matches = re.findall(pattern, header)
|
|
|
|
for url, rel in matches:
|
|
if rel == 'authorization_endpoint':
|
|
endpoints['authorization_endpoint'] = urljoin(base_url, url)
|
|
elif rel == 'token_endpoint':
|
|
endpoints['token_endpoint'] = urljoin(base_url, url)
|
|
|
|
return endpoints
|
|
|
|
def _extract_from_html(self, html: str, base_url: str) -> Dict[str, str]:
|
|
"""Extract endpoints from HTML link elements"""
|
|
endpoints = {}
|
|
soup = BeautifulSoup(html, 'html.parser')
|
|
|
|
# Find <link rel="authorization_endpoint" href="...">
|
|
auth_link = soup.find('link', rel='authorization_endpoint')
|
|
if auth_link and auth_link.get('href'):
|
|
endpoints['authorization_endpoint'] = urljoin(
|
|
base_url,
|
|
auth_link['href']
|
|
)
|
|
|
|
# Find <link rel="token_endpoint" href="...">
|
|
token_link = soup.find('link', rel='token_endpoint')
|
|
if token_link and token_link.get('href'):
|
|
endpoints['token_endpoint'] = urljoin(
|
|
base_url,
|
|
token_link['href']
|
|
)
|
|
|
|
return endpoints
|
|
|
|
def _is_development(self) -> bool:
|
|
"""Check if running in development mode"""
|
|
# Implementation depends on your config system
|
|
return False
|
|
|
|
|
|
class DiscoveryError(Exception):
|
|
"""Raised when endpoint discovery fails"""
|
|
pass
|
|
```
|
|
|
|
### Step 3: Update Token Verification
|
|
|
|
**Update `auth_external.py`:**
|
|
|
|
```python
|
|
"""
|
|
External Token Verification with Dynamic Discovery
|
|
"""
|
|
|
|
import hashlib
|
|
import time
|
|
from typing import Dict, Optional
|
|
import httpx
|
|
|
|
from .endpoint_discovery import EndpointDiscovery, DiscoveryError
|
|
|
|
|
|
class ExternalTokenVerifier:
|
|
"""Verifies tokens using discovered IndieAuth endpoints"""
|
|
|
|
def __init__(self, admin_me: str, cache_ttl: int = 300):
|
|
self.admin_me = admin_me
|
|
self.discovery = EndpointDiscovery()
|
|
self.cache = TokenCache(ttl=cache_ttl)
|
|
|
|
def verify_token(self, token: str) -> Dict:
|
|
"""
|
|
Verify a token using endpoint discovery
|
|
|
|
Args:
|
|
token: Bearer token to verify
|
|
|
|
Returns:
|
|
Token info dict with 'me', 'scope', 'client_id'
|
|
|
|
Raises:
|
|
TokenVerificationError: If verification fails
|
|
"""
|
|
# Check cache first
|
|
token_hash = self._hash_token(token)
|
|
cached = self.cache.get(token_hash)
|
|
if cached:
|
|
return cached
|
|
|
|
# Discover endpoints for admin
|
|
try:
|
|
endpoints = self.discovery.discover(self.admin_me)
|
|
except DiscoveryError as e:
|
|
raise TokenVerificationError(f"Endpoint discovery failed: {e}")
|
|
|
|
# Verify with discovered endpoint
|
|
token_endpoint = endpoints['token_endpoint']
|
|
|
|
try:
|
|
response = httpx.get(
|
|
token_endpoint,
|
|
headers={'Authorization': f'Bearer {token}'},
|
|
timeout=5.0
|
|
)
|
|
response.raise_for_status()
|
|
except Exception as e:
|
|
raise TokenVerificationError(f"Token verification failed: {e}")
|
|
|
|
token_info = response.json()
|
|
|
|
# Validate response
|
|
if 'me' not in token_info:
|
|
raise TokenVerificationError("Invalid token response: missing 'me'")
|
|
|
|
# Ensure token is for our admin
|
|
if self._normalize_url(token_info['me']) != self._normalize_url(self.admin_me):
|
|
raise TokenVerificationError(
|
|
f"Token is for {token_info['me']}, expected {self.admin_me}"
|
|
)
|
|
|
|
# Check scope
|
|
scopes = token_info.get('scope', '').split()
|
|
if 'create' not in scopes:
|
|
raise TokenVerificationError("Token missing 'create' scope")
|
|
|
|
# Cache successful verification
|
|
self.cache.store(token_hash, token_info)
|
|
|
|
return token_info
|
|
|
|
def _hash_token(self, token: str) -> str:
|
|
"""Hash token for secure caching"""
|
|
return hashlib.sha256(token.encode()).hexdigest()
|
|
|
|
def _normalize_url(self, url: str) -> str:
|
|
"""Normalize URL for comparison"""
|
|
# Add trailing slash if missing
|
|
if not url.endswith('/'):
|
|
url += '/'
|
|
return url.lower()
|
|
|
|
|
|
class TokenCache:
|
|
"""Simple in-memory cache for token verifications"""
|
|
|
|
def __init__(self, ttl: int = 300):
|
|
self.ttl = ttl
|
|
self.cache = {}
|
|
|
|
def get(self, token_hash: str) -> Optional[Dict]:
|
|
"""Get cached token info if still valid"""
|
|
if token_hash in self.cache:
|
|
info, expiry = self.cache[token_hash]
|
|
if time.time() < expiry:
|
|
return info
|
|
else:
|
|
del self.cache[token_hash]
|
|
return None
|
|
|
|
def store(self, token_hash: str, info: Dict):
|
|
"""Cache token info"""
|
|
expiry = time.time() + self.ttl
|
|
self.cache[token_hash] = (info, expiry)
|
|
|
|
|
|
class TokenVerificationError(Exception):
|
|
"""Raised when token verification fails"""
|
|
pass
|
|
```
|
|
|
|
### Step 4: Update Micropub Integration
|
|
|
|
**Update Micropub to use discovery-based verification:**
|
|
|
|
```python
|
|
# micropub.py
|
|
from ..auth.auth_external import ExternalTokenVerifier
|
|
|
|
class MicropubEndpoint:
|
|
def __init__(self, config):
|
|
self.verifier = ExternalTokenVerifier(
|
|
admin_me=config['ADMIN_ME'],
|
|
cache_ttl=config.get('TOKEN_CACHE_TTL', 300)
|
|
)
|
|
|
|
def handle_request(self, request):
|
|
# Extract token
|
|
auth_header = request.headers.get('Authorization', '')
|
|
if not auth_header.startswith('Bearer '):
|
|
return error_response(401, "No bearer token provided")
|
|
|
|
token = auth_header[7:] # Remove 'Bearer ' prefix
|
|
|
|
# Verify using discovery
|
|
try:
|
|
token_info = self.verifier.verify_token(token)
|
|
except TokenVerificationError as e:
|
|
return error_response(403, str(e))
|
|
|
|
# Process Micropub request
|
|
# ...
|
|
```
|
|
|
|
## Migration Steps
|
|
|
|
### Phase 1: Preparation
|
|
|
|
1. **Review current implementation**
|
|
- Identify all hardcoded endpoint references
|
|
- Document current configuration
|
|
|
|
2. **Set up test environment**
|
|
- Create test profile with IndieAuth links
|
|
- Set up test IndieAuth provider
|
|
|
|
3. **Write tests for new implementation**
|
|
- Unit tests for discovery
|
|
- Integration tests for verification
|
|
|
|
### Phase 2: Implementation
|
|
|
|
1. **Implement discovery module**
|
|
- Create endpoint_discovery.py
|
|
- Add comprehensive error handling
|
|
- Include logging for debugging
|
|
|
|
2. **Update token verification**
|
|
- Remove hardcoded endpoints
|
|
- Integrate discovery module
|
|
- Add caching layer
|
|
|
|
3. **Update configuration**
|
|
- Remove TOKEN_ENDPOINT from config
|
|
- Ensure ADMIN_ME is set correctly
|
|
|
|
### Phase 3: Testing
|
|
|
|
1. **Test discovery with various providers**
|
|
- indieauth.com
|
|
- Self-hosted IndieAuth
|
|
- Custom implementations
|
|
|
|
2. **Test error conditions**
|
|
- Profile URL unreachable
|
|
- No endpoints in profile
|
|
- Invalid token responses
|
|
|
|
3. **Performance testing**
|
|
- Measure discovery latency
|
|
- Verify cache effectiveness
|
|
- Test under load
|
|
|
|
### Phase 4: Deployment
|
|
|
|
1. **Update documentation**
|
|
- Explain endpoint discovery
|
|
- Provide setup instructions
|
|
- Include troubleshooting guide
|
|
|
|
2. **Deploy to staging**
|
|
- Test with real IndieAuth providers
|
|
- Monitor for issues
|
|
- Verify performance
|
|
|
|
3. **Deploy to production**
|
|
- Clear any existing caches
|
|
- Monitor closely for first 24 hours
|
|
- Be ready to roll back if needed
|
|
|
|
## Verification Checklist
|
|
|
|
After migration, verify:
|
|
|
|
- [ ] No hardcoded endpoints remain in code
|
|
- [ ] Discovery works with test profiles
|
|
- [ ] Token verification uses discovered endpoints
|
|
- [ ] Cache improves performance
|
|
- [ ] Error messages are clear
|
|
- [ ] Logs contain useful debugging info
|
|
- [ ] Documentation is updated
|
|
- [ ] Tests pass
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
#### "No token endpoint found"
|
|
|
|
**Cause**: Profile URL doesn't have IndieAuth links
|
|
|
|
**Solution**:
|
|
1. Check profile URL returns HTML
|
|
2. Verify link elements are present
|
|
3. Check for typos in rel attributes
|
|
|
|
#### "Token verification failed"
|
|
|
|
**Cause**: Various issues with endpoint or token
|
|
|
|
**Solution**:
|
|
1. Check endpoint is reachable
|
|
2. Verify token hasn't expired
|
|
3. Ensure 'me' URL matches expected
|
|
|
|
#### "Discovery timeout"
|
|
|
|
**Cause**: Profile URL slow or unreachable
|
|
|
|
**Solution**:
|
|
1. Increase timeout if needed
|
|
2. Check network connectivity
|
|
3. Verify profile URL is correct
|
|
|
|
## Rollback Plan
|
|
|
|
If issues arise:
|
|
|
|
1. **Keep old code available**
|
|
- Tag release before migration
|
|
- Keep backup of old implementation
|
|
|
|
2. **Quick rollback procedure**
|
|
```bash
|
|
# Revert to previous version
|
|
git checkout tags/pre-discovery-migration
|
|
|
|
# Restore old configuration
|
|
cp config.ini.backup config.ini
|
|
|
|
# Restart application
|
|
systemctl restart starpunk
|
|
```
|
|
|
|
3. **Document issues for retry**
|
|
- What failed?
|
|
- Error messages
|
|
- Affected users
|
|
|
|
## Success Criteria
|
|
|
|
Migration is successful when:
|
|
|
|
1. All token verifications use discovered endpoints
|
|
2. No hardcoded endpoints remain
|
|
3. Performance is acceptable (< 500ms uncached)
|
|
4. All tests pass
|
|
5. Documentation is complete
|
|
6. Users can authenticate successfully
|
|
|
|
## Long-term Benefits
|
|
|
|
After this migration:
|
|
|
|
1. **True IndieAuth Compliance**: Finally following the specification
|
|
2. **User Freedom**: Users control their authentication
|
|
3. **Better Security**: No single point of failure
|
|
4. **Future Proof**: Ready for new IndieAuth providers
|
|
5. **Maintainable**: Cleaner, spec-compliant code
|
|
|
|
---
|
|
|
|
**Document Version**: 1.0
|
|
**Created**: 2024-11-24
|
|
**Purpose**: Fix critical IndieAuth implementation error
|
|
**Priority**: CRITICAL - Must be fixed before V1 release |