docs: Add complete documentation for v1.0.0-rc.5 hotfix
Complete architectural documentation for: - Migration race condition fix with database locking - IndieAuth endpoint discovery implementation - Security considerations and migration guides New documentation: - ADR-030-CORRECTED: IndieAuth endpoint discovery decision - ADR-031: Endpoint discovery implementation details - Architecture docs on endpoint discovery - Migration guide for removed TOKEN_ENDPOINT - Security analysis of endpoint discovery - Implementation and analysis reports
This commit is contained in:
492
docs/migration/fix-hardcoded-endpoints.md
Normal file
492
docs/migration/fix-hardcoded-endpoints.md
Normal file
@@ -0,0 +1,492 @@
|
||||
# Migration Guide: Fixing Hardcoded IndieAuth Endpoints
|
||||
|
||||
## Overview
|
||||
|
||||
This guide explains how to migrate from the **incorrect** hardcoded endpoint implementation to the **correct** dynamic endpoint discovery implementation that actually follows the IndieAuth specification.
|
||||
|
||||
## The Problem We're Fixing
|
||||
|
||||
### What's Currently Wrong
|
||||
|
||||
```python
|
||||
# WRONG - auth_external.py (hypothetical incorrect implementation)
|
||||
class ExternalTokenVerifier:
|
||||
def __init__(self):
|
||||
# FATAL FLAW: Hardcoded endpoint
|
||||
self.token_endpoint = "https://tokens.indieauth.com/token"
|
||||
|
||||
def verify_token(self, token):
|
||||
# Uses hardcoded endpoint for ALL users
|
||||
response = requests.get(
|
||||
self.token_endpoint,
|
||||
headers={'Authorization': f'Bearer {token}'}
|
||||
)
|
||||
return response.json()
|
||||
```
|
||||
|
||||
### Why It's Wrong
|
||||
|
||||
1. **Not IndieAuth**: This completely violates the IndieAuth specification
|
||||
2. **No User Choice**: Forces all users to use the same provider
|
||||
3. **Security Risk**: Single point of failure for all authentications
|
||||
4. **No Flexibility**: Users can't change or choose providers
|
||||
|
||||
## The Correct Implementation
|
||||
|
||||
### Step 1: Remove Hardcoded Configuration
|
||||
|
||||
**Remove from config files:**
|
||||
|
||||
```ini
|
||||
# DELETE THESE LINES - They are wrong!
|
||||
TOKEN_ENDPOINT=https://tokens.indieauth.com/token
|
||||
AUTHORIZATION_ENDPOINT=https://indieauth.com/auth
|
||||
```
|
||||
|
||||
**Keep only:**
|
||||
|
||||
```ini
|
||||
# CORRECT - Only the admin's identity URL
|
||||
ADMIN_ME=https://admin.example.com/
|
||||
```
|
||||
|
||||
### Step 2: Implement Endpoint Discovery
|
||||
|
||||
**Create `endpoint_discovery.py`:**
|
||||
|
||||
```python
|
||||
"""
|
||||
IndieAuth Endpoint Discovery
|
||||
Implements: https://www.w3.org/TR/indieauth/#discovery-by-clients
|
||||
"""
|
||||
|
||||
import re
|
||||
from typing import Dict, Optional
|
||||
from urllib.parse import urljoin, urlparse
|
||||
import httpx
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
class EndpointDiscovery:
|
||||
"""Discovers IndieAuth endpoints from profile URLs"""
|
||||
|
||||
def __init__(self, timeout: int = 5):
|
||||
self.timeout = timeout
|
||||
self.client = httpx.Client(
|
||||
timeout=timeout,
|
||||
follow_redirects=True,
|
||||
limits=httpx.Limits(max_redirects=5)
|
||||
)
|
||||
|
||||
def discover(self, profile_url: str) -> Dict[str, str]:
|
||||
"""
|
||||
Discover IndieAuth endpoints from a profile URL
|
||||
|
||||
Args:
|
||||
profile_url: The user's profile URL (their identity)
|
||||
|
||||
Returns:
|
||||
Dictionary with 'authorization_endpoint' and 'token_endpoint'
|
||||
|
||||
Raises:
|
||||
DiscoveryError: If discovery fails
|
||||
"""
|
||||
# Ensure HTTPS in production
|
||||
if not self._is_development() and not profile_url.startswith('https://'):
|
||||
raise DiscoveryError("Profile URL must use HTTPS")
|
||||
|
||||
try:
|
||||
response = self.client.get(profile_url)
|
||||
response.raise_for_status()
|
||||
except Exception as e:
|
||||
raise DiscoveryError(f"Failed to fetch profile: {e}")
|
||||
|
||||
endpoints = {}
|
||||
|
||||
# 1. Check HTTP Link headers (highest priority)
|
||||
link_header = response.headers.get('Link', '')
|
||||
if link_header:
|
||||
endpoints.update(self._parse_link_header(link_header, profile_url))
|
||||
|
||||
# 2. Check HTML link elements
|
||||
if 'text/html' in response.headers.get('Content-Type', ''):
|
||||
endpoints.update(self._extract_from_html(
|
||||
response.text,
|
||||
profile_url
|
||||
))
|
||||
|
||||
# Validate we found required endpoints
|
||||
if 'token_endpoint' not in endpoints:
|
||||
raise DiscoveryError("No token endpoint found in profile")
|
||||
|
||||
return endpoints
|
||||
|
||||
def _parse_link_header(self, header: str, base_url: str) -> Dict[str, str]:
|
||||
"""Parse HTTP Link header for endpoints"""
|
||||
endpoints = {}
|
||||
|
||||
# Parse Link: <url>; rel="relation"
|
||||
pattern = r'<([^>]+)>;\s*rel="([^"]+)"'
|
||||
matches = re.findall(pattern, header)
|
||||
|
||||
for url, rel in matches:
|
||||
if rel == 'authorization_endpoint':
|
||||
endpoints['authorization_endpoint'] = urljoin(base_url, url)
|
||||
elif rel == 'token_endpoint':
|
||||
endpoints['token_endpoint'] = urljoin(base_url, url)
|
||||
|
||||
return endpoints
|
||||
|
||||
def _extract_from_html(self, html: str, base_url: str) -> Dict[str, str]:
|
||||
"""Extract endpoints from HTML link elements"""
|
||||
endpoints = {}
|
||||
soup = BeautifulSoup(html, 'html.parser')
|
||||
|
||||
# Find <link rel="authorization_endpoint" href="...">
|
||||
auth_link = soup.find('link', rel='authorization_endpoint')
|
||||
if auth_link and auth_link.get('href'):
|
||||
endpoints['authorization_endpoint'] = urljoin(
|
||||
base_url,
|
||||
auth_link['href']
|
||||
)
|
||||
|
||||
# Find <link rel="token_endpoint" href="...">
|
||||
token_link = soup.find('link', rel='token_endpoint')
|
||||
if token_link and token_link.get('href'):
|
||||
endpoints['token_endpoint'] = urljoin(
|
||||
base_url,
|
||||
token_link['href']
|
||||
)
|
||||
|
||||
return endpoints
|
||||
|
||||
def _is_development(self) -> bool:
|
||||
"""Check if running in development mode"""
|
||||
# Implementation depends on your config system
|
||||
return False
|
||||
|
||||
|
||||
class DiscoveryError(Exception):
|
||||
"""Raised when endpoint discovery fails"""
|
||||
pass
|
||||
```
|
||||
|
||||
### Step 3: Update Token Verification
|
||||
|
||||
**Update `auth_external.py`:**
|
||||
|
||||
```python
|
||||
"""
|
||||
External Token Verification with Dynamic Discovery
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import time
|
||||
from typing import Dict, Optional
|
||||
import httpx
|
||||
|
||||
from .endpoint_discovery import EndpointDiscovery, DiscoveryError
|
||||
|
||||
|
||||
class ExternalTokenVerifier:
|
||||
"""Verifies tokens using discovered IndieAuth endpoints"""
|
||||
|
||||
def __init__(self, admin_me: str, cache_ttl: int = 300):
|
||||
self.admin_me = admin_me
|
||||
self.discovery = EndpointDiscovery()
|
||||
self.cache = TokenCache(ttl=cache_ttl)
|
||||
|
||||
def verify_token(self, token: str) -> Dict:
|
||||
"""
|
||||
Verify a token using endpoint discovery
|
||||
|
||||
Args:
|
||||
token: Bearer token to verify
|
||||
|
||||
Returns:
|
||||
Token info dict with 'me', 'scope', 'client_id'
|
||||
|
||||
Raises:
|
||||
TokenVerificationError: If verification fails
|
||||
"""
|
||||
# Check cache first
|
||||
token_hash = self._hash_token(token)
|
||||
cached = self.cache.get(token_hash)
|
||||
if cached:
|
||||
return cached
|
||||
|
||||
# Discover endpoints for admin
|
||||
try:
|
||||
endpoints = self.discovery.discover(self.admin_me)
|
||||
except DiscoveryError as e:
|
||||
raise TokenVerificationError(f"Endpoint discovery failed: {e}")
|
||||
|
||||
# Verify with discovered endpoint
|
||||
token_endpoint = endpoints['token_endpoint']
|
||||
|
||||
try:
|
||||
response = httpx.get(
|
||||
token_endpoint,
|
||||
headers={'Authorization': f'Bearer {token}'},
|
||||
timeout=5.0
|
||||
)
|
||||
response.raise_for_status()
|
||||
except Exception as e:
|
||||
raise TokenVerificationError(f"Token verification failed: {e}")
|
||||
|
||||
token_info = response.json()
|
||||
|
||||
# Validate response
|
||||
if 'me' not in token_info:
|
||||
raise TokenVerificationError("Invalid token response: missing 'me'")
|
||||
|
||||
# Ensure token is for our admin
|
||||
if self._normalize_url(token_info['me']) != self._normalize_url(self.admin_me):
|
||||
raise TokenVerificationError(
|
||||
f"Token is for {token_info['me']}, expected {self.admin_me}"
|
||||
)
|
||||
|
||||
# Check scope
|
||||
scopes = token_info.get('scope', '').split()
|
||||
if 'create' not in scopes:
|
||||
raise TokenVerificationError("Token missing 'create' scope")
|
||||
|
||||
# Cache successful verification
|
||||
self.cache.store(token_hash, token_info)
|
||||
|
||||
return token_info
|
||||
|
||||
def _hash_token(self, token: str) -> str:
|
||||
"""Hash token for secure caching"""
|
||||
return hashlib.sha256(token.encode()).hexdigest()
|
||||
|
||||
def _normalize_url(self, url: str) -> str:
|
||||
"""Normalize URL for comparison"""
|
||||
# Add trailing slash if missing
|
||||
if not url.endswith('/'):
|
||||
url += '/'
|
||||
return url.lower()
|
||||
|
||||
|
||||
class TokenCache:
|
||||
"""Simple in-memory cache for token verifications"""
|
||||
|
||||
def __init__(self, ttl: int = 300):
|
||||
self.ttl = ttl
|
||||
self.cache = {}
|
||||
|
||||
def get(self, token_hash: str) -> Optional[Dict]:
|
||||
"""Get cached token info if still valid"""
|
||||
if token_hash in self.cache:
|
||||
info, expiry = self.cache[token_hash]
|
||||
if time.time() < expiry:
|
||||
return info
|
||||
else:
|
||||
del self.cache[token_hash]
|
||||
return None
|
||||
|
||||
def store(self, token_hash: str, info: Dict):
|
||||
"""Cache token info"""
|
||||
expiry = time.time() + self.ttl
|
||||
self.cache[token_hash] = (info, expiry)
|
||||
|
||||
|
||||
class TokenVerificationError(Exception):
|
||||
"""Raised when token verification fails"""
|
||||
pass
|
||||
```
|
||||
|
||||
### Step 4: Update Micropub Integration
|
||||
|
||||
**Update Micropub to use discovery-based verification:**
|
||||
|
||||
```python
|
||||
# micropub.py
|
||||
from ..auth.auth_external import ExternalTokenVerifier
|
||||
|
||||
class MicropubEndpoint:
|
||||
def __init__(self, config):
|
||||
self.verifier = ExternalTokenVerifier(
|
||||
admin_me=config['ADMIN_ME'],
|
||||
cache_ttl=config.get('TOKEN_CACHE_TTL', 300)
|
||||
)
|
||||
|
||||
def handle_request(self, request):
|
||||
# Extract token
|
||||
auth_header = request.headers.get('Authorization', '')
|
||||
if not auth_header.startswith('Bearer '):
|
||||
return error_response(401, "No bearer token provided")
|
||||
|
||||
token = auth_header[7:] # Remove 'Bearer ' prefix
|
||||
|
||||
# Verify using discovery
|
||||
try:
|
||||
token_info = self.verifier.verify_token(token)
|
||||
except TokenVerificationError as e:
|
||||
return error_response(403, str(e))
|
||||
|
||||
# Process Micropub request
|
||||
# ...
|
||||
```
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### Phase 1: Preparation
|
||||
|
||||
1. **Review current implementation**
|
||||
- Identify all hardcoded endpoint references
|
||||
- Document current configuration
|
||||
|
||||
2. **Set up test environment**
|
||||
- Create test profile with IndieAuth links
|
||||
- Set up test IndieAuth provider
|
||||
|
||||
3. **Write tests for new implementation**
|
||||
- Unit tests for discovery
|
||||
- Integration tests for verification
|
||||
|
||||
### Phase 2: Implementation
|
||||
|
||||
1. **Implement discovery module**
|
||||
- Create endpoint_discovery.py
|
||||
- Add comprehensive error handling
|
||||
- Include logging for debugging
|
||||
|
||||
2. **Update token verification**
|
||||
- Remove hardcoded endpoints
|
||||
- Integrate discovery module
|
||||
- Add caching layer
|
||||
|
||||
3. **Update configuration**
|
||||
- Remove TOKEN_ENDPOINT from config
|
||||
- Ensure ADMIN_ME is set correctly
|
||||
|
||||
### Phase 3: Testing
|
||||
|
||||
1. **Test discovery with various providers**
|
||||
- indieauth.com
|
||||
- Self-hosted IndieAuth
|
||||
- Custom implementations
|
||||
|
||||
2. **Test error conditions**
|
||||
- Profile URL unreachable
|
||||
- No endpoints in profile
|
||||
- Invalid token responses
|
||||
|
||||
3. **Performance testing**
|
||||
- Measure discovery latency
|
||||
- Verify cache effectiveness
|
||||
- Test under load
|
||||
|
||||
### Phase 4: Deployment
|
||||
|
||||
1. **Update documentation**
|
||||
- Explain endpoint discovery
|
||||
- Provide setup instructions
|
||||
- Include troubleshooting guide
|
||||
|
||||
2. **Deploy to staging**
|
||||
- Test with real IndieAuth providers
|
||||
- Monitor for issues
|
||||
- Verify performance
|
||||
|
||||
3. **Deploy to production**
|
||||
- Clear any existing caches
|
||||
- Monitor closely for first 24 hours
|
||||
- Be ready to roll back if needed
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
After migration, verify:
|
||||
|
||||
- [ ] No hardcoded endpoints remain in code
|
||||
- [ ] Discovery works with test profiles
|
||||
- [ ] Token verification uses discovered endpoints
|
||||
- [ ] Cache improves performance
|
||||
- [ ] Error messages are clear
|
||||
- [ ] Logs contain useful debugging info
|
||||
- [ ] Documentation is updated
|
||||
- [ ] Tests pass
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### "No token endpoint found"
|
||||
|
||||
**Cause**: Profile URL doesn't have IndieAuth links
|
||||
|
||||
**Solution**:
|
||||
1. Check profile URL returns HTML
|
||||
2. Verify link elements are present
|
||||
3. Check for typos in rel attributes
|
||||
|
||||
#### "Token verification failed"
|
||||
|
||||
**Cause**: Various issues with endpoint or token
|
||||
|
||||
**Solution**:
|
||||
1. Check endpoint is reachable
|
||||
2. Verify token hasn't expired
|
||||
3. Ensure 'me' URL matches expected
|
||||
|
||||
#### "Discovery timeout"
|
||||
|
||||
**Cause**: Profile URL slow or unreachable
|
||||
|
||||
**Solution**:
|
||||
1. Increase timeout if needed
|
||||
2. Check network connectivity
|
||||
3. Verify profile URL is correct
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If issues arise:
|
||||
|
||||
1. **Keep old code available**
|
||||
- Tag release before migration
|
||||
- Keep backup of old implementation
|
||||
|
||||
2. **Quick rollback procedure**
|
||||
```bash
|
||||
# Revert to previous version
|
||||
git checkout tags/pre-discovery-migration
|
||||
|
||||
# Restore old configuration
|
||||
cp config.ini.backup config.ini
|
||||
|
||||
# Restart application
|
||||
systemctl restart starpunk
|
||||
```
|
||||
|
||||
3. **Document issues for retry**
|
||||
- What failed?
|
||||
- Error messages
|
||||
- Affected users
|
||||
|
||||
## Success Criteria
|
||||
|
||||
Migration is successful when:
|
||||
|
||||
1. All token verifications use discovered endpoints
|
||||
2. No hardcoded endpoints remain
|
||||
3. Performance is acceptable (< 500ms uncached)
|
||||
4. All tests pass
|
||||
5. Documentation is complete
|
||||
6. Users can authenticate successfully
|
||||
|
||||
## Long-term Benefits
|
||||
|
||||
After this migration:
|
||||
|
||||
1. **True IndieAuth Compliance**: Finally following the specification
|
||||
2. **User Freedom**: Users control their authentication
|
||||
3. **Better Security**: No single point of failure
|
||||
4. **Future Proof**: Ready for new IndieAuth providers
|
||||
5. **Maintainable**: Cleaner, spec-compliant code
|
||||
|
||||
---
|
||||
|
||||
**Document Version**: 1.0
|
||||
**Created**: 2024-11-24
|
||||
**Purpose**: Fix critical IndieAuth implementation error
|
||||
**Priority**: CRITICAL - Must be fixed before V1 release
|
||||
Reference in New Issue
Block a user