Files
StarPunk/docs/design/hotfix/2025-12-17-indieauth-pkce-endpoint-discovery-hotfix.md
Phil Skentelbery 2bd971f3d6 fix(auth): Implement IndieAuth endpoint discovery per W3C spec
BREAKING: Removes INDIELOGIN_URL config - endpoints are now properly
discovered from user's profile URL as required by W3C IndieAuth spec.

- auth.py: Uses discover_endpoints() to find authorization_endpoint
- config.py: Deprecation warning for obsolete INDIELOGIN_URL setting
- auth_external.py: Relaxed validation (allows auth-only flows)
- tests: Updated to mock endpoint discovery

This fixes a regression where admin login was hardcoded to use
indielogin.com instead of respecting the user's declared endpoints.

Version: 1.5.0-hotfix.1

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-17 13:52:36 -07:00

609 lines
22 KiB
Markdown

# IndieAuth Endpoint Discovery Hotfix
**Date:** 2025-12-17
**Type:** Production Hotfix
**Priority:** Critical
**Status:** Ready for Implementation
## Problem Summary
Users cannot log in to StarPunk. The root cause is that the authentication code ignores endpoint discovery and hardcodes `INDIELOGIN_URL` instead of discovering the authorization and token endpoints from the user's profile URL.
**Root Cause:** The `starpunk/auth.py` module uses `INDIELOGIN_URL` config instead of discovering endpoints from the user's profile URL as required by the IndieAuth specification. This is a regression - the system used to respect discovered endpoints.
**Note:** The PKCE error message in the callback is a symptom, not the cause. Once we use the correct discovered endpoints, PKCE will not be required (since the user's actual IndieAuth server doesn't require it).
## Specification Requirements
### W3C IndieAuth Spec (https://www.w3.org/TR/indieauth/)
- Clients MUST discover `authorization_endpoint` from user's profile URL
- Clients MUST discover `token_endpoint` from user's profile URL
- Discovery via HTTP Link headers (highest priority) or HTML `<link>` elements
## Implementation Plan
### Overview
The fix reuses the existing `discover_endpoints()` function from `auth_external.py` in the login flow. Changes are minimal and focused:
1. Use `discover_endpoints()` in `initiate_login()` to get the `authorization_endpoint`
2. Use `discover_endpoints()` in `handle_callback()` to get the `token_endpoint`
3. Remove `INDIELOGIN_URL` config (with deprecation warning)
---
### Step 1: Update config.py - Remove INDIELOGIN_URL
In `/home/phil/Projects/starpunk/starpunk/config.py`:
**Change 1:** Remove the INDIELOGIN_URL config line (line 37):
```python
# DELETE THIS LINE:
app.config["INDIELOGIN_URL"] = os.getenv("INDIELOGIN_URL", "https://indielogin.com")
```
**Change 2:** Add deprecation warning for INDIELOGIN_URL (add after the TOKEN_ENDPOINT warning, around line 47):
```python
# DEPRECATED: INDIELOGIN_URL no longer used (hotfix 2025-12-17)
# Authorization endpoint is now discovered from ADMIN_ME profile per IndieAuth spec
if 'INDIELOGIN_URL' in os.environ:
app.logger.warning(
"INDIELOGIN_URL is deprecated and will be ignored. "
"Remove it from your configuration. "
"The authorization endpoint is now discovered automatically from your ADMIN_ME profile."
)
```
---
### Step 2: Update auth.py - Use Endpoint Discovery
In `/home/phil/Projects/starpunk/starpunk/auth.py`:
**Change 1:** Add import for endpoint discovery (after line 42):
```python
from starpunk.auth_external import discover_endpoints, DiscoveryError, normalize_url
```
> **Note:** The `normalize_url` import is at the top level (not inside `handle_callback()`) for consistency with the existing code style.
**Change 2:** Update `initiate_login()` to use discovered authorization_endpoint (replace lines 251-318):
```python
def initiate_login(me_url: str) -> str:
"""
Initiate IndieAuth authentication flow with endpoint discovery.
Per W3C IndieAuth spec, discovers authorization_endpoint from user's
profile URL rather than using a hardcoded service.
Args:
me_url: User's IndieWeb identity URL
Returns:
Redirect URL to discovered authorization endpoint
Raises:
ValueError: Invalid me_url format
DiscoveryError: Failed to discover endpoints from profile
"""
# Validate URL format
if not is_valid_url(me_url):
raise ValueError(f"Invalid URL format: {me_url}")
current_app.logger.debug(f"Auth: Validating me URL: {me_url}")
# Discover authorization endpoint from user's profile URL
# Per IndieAuth spec: clients MUST discover endpoints, not hardcode them
try:
endpoints = discover_endpoints(me_url)
except DiscoveryError as e:
current_app.logger.error(f"Auth: Endpoint discovery failed for {me_url}: {e}")
raise
auth_endpoint = endpoints.get('authorization_endpoint')
if not auth_endpoint:
raise DiscoveryError(
f"No authorization_endpoint found at {me_url}. "
"Ensure your profile has IndieAuth link elements or headers."
)
current_app.logger.info(f"Auth: Discovered authorization_endpoint: {auth_endpoint}")
# Generate CSRF state token
state = _generate_state_token()
current_app.logger.debug(f"Auth: Generated state token: {_redact_token(state, 8)}")
# Store state in database (5-minute expiry)
db = get_db(current_app)
expires_at = datetime.utcnow() + timedelta(minutes=5)
redirect_uri = f"{current_app.config['SITE_URL']}auth/callback"
db.execute(
"""
INSERT INTO auth_state (state, expires_at, redirect_uri)
VALUES (?, ?, ?)
""",
(state, expires_at, redirect_uri),
)
db.commit()
# Build authorization URL
params = {
"me": me_url,
"client_id": current_app.config["SITE_URL"],
"redirect_uri": redirect_uri,
"state": state,
"response_type": "code",
}
current_app.logger.debug(
f"Auth: Building authorization URL with params:\n"
f" me: {me_url}\n"
f" client_id: {current_app.config['SITE_URL']}\n"
f" redirect_uri: {redirect_uri}\n"
f" state: {_redact_token(state, 8)}\n"
f" response_type: code"
)
auth_url = f"{auth_endpoint}?{urlencode(params)}"
current_app.logger.debug(f"Auth: Complete authorization URL: {auth_url}")
current_app.logger.info(f"Auth: Authentication initiated for {me_url}")
return auth_url
```
**Change 3:** Update `handle_callback()` to use discovered authorization_endpoint (replace lines 321-474):
> **Important:** Per IndieAuth spec, authentication-only flows (identity verification without access tokens) POST to the **authorization_endpoint**, NOT the token_endpoint. The `grant_type` parameter is NOT included for authentication-only flows.
```python
def handle_callback(code: str, state: str, iss: Optional[str] = None) -> Optional[str]:
"""
Handle IndieAuth callback with endpoint discovery.
Discovers authorization_endpoint from ADMIN_ME profile and exchanges
authorization code for identity verification.
Per IndieAuth spec: Authentication-only flows POST to the authorization
endpoint (not token endpoint) and do not include grant_type.
Args:
code: Authorization code from authorization server
state: CSRF state token
iss: Issuer identifier (optional, for security validation)
Returns:
Session token if successful, None otherwise
Raises:
InvalidStateError: State token validation failed
UnauthorizedError: User not authorized as admin
IndieLoginError: Code exchange failed
"""
current_app.logger.debug(f"Auth: Verifying state token: {_redact_token(state, 8)}")
# Verify state token (CSRF protection)
if not _verify_state_token(state):
current_app.logger.warning(
"Auth: Invalid state token received (possible CSRF or expired token)"
)
raise InvalidStateError("Invalid or expired state token")
current_app.logger.debug("Auth: State token valid")
# Discover authorization endpoint from ADMIN_ME profile
admin_me = current_app.config.get("ADMIN_ME")
if not admin_me:
current_app.logger.error("Auth: ADMIN_ME not configured")
raise IndieLoginError("ADMIN_ME not configured")
try:
endpoints = discover_endpoints(admin_me)
except DiscoveryError as e:
current_app.logger.error(f"Auth: Endpoint discovery failed: {e}")
raise IndieLoginError(f"Failed to discover endpoints: {e}")
# Use authorization_endpoint for authentication-only flow (identity verification)
# Per IndieAuth spec: auth-only flows POST to authorization_endpoint, not token_endpoint
auth_endpoint = endpoints.get('authorization_endpoint')
if not auth_endpoint:
raise IndieLoginError(
f"No authorization_endpoint found at {admin_me}. "
"Ensure your profile has IndieAuth endpoints configured."
)
current_app.logger.debug(f"Auth: Using authorization_endpoint: {auth_endpoint}")
# Verify issuer if provided (security check - optional)
if iss:
current_app.logger.debug(f"Auth: Issuer provided: {iss}")
# Prepare code verification request
# Note: grant_type is NOT included for authentication-only flows per IndieAuth spec
token_exchange_data = {
"code": code,
"client_id": current_app.config["SITE_URL"],
"redirect_uri": f"{current_app.config['SITE_URL']}auth/callback",
}
# Log the request
_log_http_request(
method="POST",
url=auth_endpoint,
data=token_exchange_data,
)
current_app.logger.debug(
"Auth: Sending code verification request to authorization endpoint:\n"
" Method: POST\n"
" URL: %s\n"
" Data: code=%s, client_id=%s, redirect_uri=%s",
auth_endpoint,
_redact_token(code),
token_exchange_data["client_id"],
token_exchange_data["redirect_uri"],
)
# Exchange code for identity at authorization endpoint (authentication-only flow)
try:
response = httpx.post(
auth_endpoint,
data=token_exchange_data,
timeout=10.0,
)
current_app.logger.debug(
"Auth: Received code verification response:\n"
" Status: %d\n"
" Headers: %s\n"
" Body: %s",
response.status_code,
{k: v for k, v in dict(response.headers).items()
if k.lower() not in ["set-cookie", "authorization"]},
_redact_token(response.text) if response.text else "(empty)",
)
_log_http_response(
status_code=response.status_code,
headers=dict(response.headers),
body=response.text,
)
response.raise_for_status()
except httpx.RequestError as e:
current_app.logger.error(f"Auth: Authorization endpoint request failed: {e}")
raise IndieLoginError(f"Failed to verify code: {e}")
except httpx.HTTPStatusError as e:
current_app.logger.error(
f"Auth: Authorization endpoint returned error: {e.response.status_code} - {e.response.text}"
)
raise IndieLoginError(
f"Authorization endpoint returned error: {e.response.status_code}"
)
# Parse response
try:
data = response.json()
except Exception as e:
current_app.logger.error(f"Auth: Failed to parse authorization endpoint response: {e}")
raise IndieLoginError("Invalid JSON response from authorization endpoint")
me = data.get("me")
if not me:
current_app.logger.error("Auth: No identity returned from authorization endpoint")
raise IndieLoginError("No identity returned from authorization endpoint")
current_app.logger.debug(f"Auth: Received identity: {me}")
# Verify this is the admin user
current_app.logger.info(f"Auth: Verifying admin authorization for me={me}")
# Normalize URLs for comparison (handles trailing slashes and case differences)
# This is correct per IndieAuth spec - the returned 'me' is the canonical form
if normalize_url(me) != normalize_url(admin_me):
current_app.logger.warning(
f"Auth: Unauthorized login attempt: {me} (expected {admin_me})"
)
raise UnauthorizedError(f"User {me} is not authorized")
current_app.logger.debug("Auth: Admin verification passed")
# Create session
session_token = create_session(me)
# Trigger author profile discovery (v1.2.0 Phase 2)
# Per Q14: Never block login, always allow fallback
try:
from starpunk.author_discovery import get_author_profile
author_profile = get_author_profile(me, refresh=True)
current_app.logger.info(f"Author profile refreshed for {me}")
except Exception as e:
current_app.logger.warning(f"Author discovery failed: {e}")
# Continue login anyway - never block per Q14
return session_token
```
---
### Step 3: Update auth_external.py - Relax Endpoint Validation
The existing `_fetch_and_parse()` function requires `token_endpoint` to be present. We need to relax this since some profiles may only have `authorization_endpoint` (for authentication-only flows).
In `/home/phil/Projects/starpunk/starpunk/auth_external.py`, update the validation in `_fetch_and_parse()` (around lines 302-307):
**Change:** Make token_endpoint not strictly required (allow authentication-only profiles):
```python
# Validate we found at least one endpoint
# - authorization_endpoint: Required for authentication-only flows (admin login)
# - token_endpoint: Required for Micropub token verification
# Having at least one allows the appropriate flow to work
if 'token_endpoint' not in endpoints and 'authorization_endpoint' not in endpoints:
raise DiscoveryError(
f"No IndieAuth endpoints found at {profile_url}. "
"Ensure your profile has authorization_endpoint or token_endpoint configured."
)
```
---
### Step 4: Update routes/auth.py - Handle DiscoveryError
In `/home/phil/Projects/starpunk/starpunk/routes/auth.py`:
**Change 1:** Add import for DiscoveryError (update lines 20-29):
```python
from starpunk.auth import (
IndieLoginError,
InvalidStateError,
UnauthorizedError,
destroy_session,
handle_callback,
initiate_login,
require_auth,
verify_session,
)
from starpunk.auth_external import DiscoveryError
```
**Change 2:** Handle DiscoveryError in login_initiate() (update lines 79-85):
> **Note:** The user-facing error message is kept simple. Technical details are logged but not shown to users.
```python
try:
# Initiate IndieAuth flow
auth_url = initiate_login(me_url)
return redirect(auth_url)
except ValueError as e:
flash(str(e), "error")
return redirect(url_for("auth.login_form"))
except DiscoveryError as e:
current_app.logger.error(f"Endpoint discovery failed for {me_url}: {e}")
flash("Unable to verify your profile URL. Please check that it's correct and try again.", "error")
return redirect(url_for("auth.login_form"))
```
---
## File Summary
| File | Change Type | Description |
|------|-------------|-------------|
| `starpunk/config.py` | Edit | Remove INDIELOGIN_URL, add deprecation warning |
| `starpunk/auth.py` | Edit | Use endpoint discovery instead of hardcoded URL |
| `starpunk/auth_external.py` | Edit | Relax endpoint validation (allow auth-only flow) |
| `starpunk/routes/auth.py` | Edit | Handle DiscoveryError exception |
---
## Testing Requirements
### Manual Testing
1. **Login Flow Test**
- Navigate to `/auth/login`
- Enter ADMIN_ME URL
- Verify redirect goes to discovered authorization_endpoint (not hardcoded indielogin.com)
- Complete login and verify session is created
2. **Endpoint Discovery Test**
- Test with profile that declares custom endpoints
- Verify discovered endpoints are used, not defaults
### Existing Test Updates
**Update test fixture in `tests/test_auth.py`:**
Remove `INDIELOGIN_URL` from the app fixture (line 51):
```python
@pytest.fixture
def app(tmp_path):
"""Create Flask app for testing"""
from starpunk import create_app
test_data_dir = tmp_path / "data"
test_data_dir.mkdir(parents=True, exist_ok=True)
app = create_app(
{
"TESTING": True,
"SITE_URL": "http://localhost:5000/",
"ADMIN_ME": "https://example.com",
"SESSION_SECRET": secrets.token_hex(32),
"SESSION_LIFETIME": 30,
# REMOVED: "INDIELOGIN_URL": "https://indielogin.com",
"DATA_PATH": test_data_dir,
"NOTES_PATH": test_data_dir / "notes",
"DATABASE_PATH": test_data_dir / "starpunk.db",
}
)
return app
```
**Update existing tests that use `httpx.post` mock:**
Tests in `TestInitiateLogin` and `TestHandleCallback` need to mock `discover_endpoints()` in addition to `httpx.post`. Example pattern:
```python
@patch("starpunk.auth.discover_endpoints")
@patch("starpunk.auth.httpx.post")
def test_handle_callback_success(self, mock_post, mock_discover, app, db, client):
"""Test successful callback handling"""
# Mock endpoint discovery
mock_discover.return_value = {
'authorization_endpoint': 'https://auth.example.com/authorize',
'token_endpoint': 'https://auth.example.com/token'
}
# Rest of test remains the same...
```
**Update `TestInitiateLogin.test_initiate_login_success`:**
The assertion checking for `indielogin.com` needs to change to check for the mocked endpoint:
```python
@patch("starpunk.auth.discover_endpoints")
def test_initiate_login_success(self, mock_discover, app, db):
"""Test successful login initiation"""
mock_discover.return_value = {
'authorization_endpoint': 'https://auth.example.com/authorize',
'token_endpoint': 'https://auth.example.com/token'
}
with app.app_context():
me_url = "https://example.com"
auth_url = initiate_login(me_url)
# Changed: Check for discovered endpoint instead of indielogin.com
assert "auth.example.com/authorize" in auth_url
assert "me=https%3A%2F%2Fexample.com" in auth_url
# ... rest of assertions
```
### New Automated Tests to Add
```python
# tests/test_auth_endpoint_discovery.py
def test_initiate_login_uses_endpoint_discovery(client, mocker):
"""Verify login uses discovered endpoint, not hardcoded"""
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
mock_discover.return_value = {
'authorization_endpoint': 'https://custom-auth.example.com/authorize',
'token_endpoint': 'https://custom-auth.example.com/token'
}
response = client.post('/auth/login', data={'me': 'https://example.com'})
assert response.status_code == 302
assert 'custom-auth.example.com' in response.headers['Location']
def test_callback_uses_discovered_authorization_endpoint(client, mocker):
"""Verify callback uses discovered authorization endpoint (not token endpoint)"""
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
mock_discover.return_value = {
'authorization_endpoint': 'https://custom-auth.example.com/authorize',
'token_endpoint': 'https://custom-auth.example.com/token'
}
mock_post = mocker.patch('starpunk.auth.httpx.post')
# Setup state token and mock httpx response
# Verify code exchange POSTs to authorization_endpoint, not token_endpoint
pass
def test_discovery_error_shows_user_friendly_message(client, mocker):
"""Verify discovery failures show helpful error"""
mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
mock_discover.side_effect = DiscoveryError("No endpoints found")
response = client.post('/auth/login', data={'me': 'https://example.com'})
assert response.status_code == 302
# Should redirect back to login form with flash message
def test_url_normalization_handles_trailing_slash(app, mocker):
"""Verify URL normalization allows trailing slash differences"""
# ADMIN_ME without trailing slash, auth server returns with trailing slash
# Should still authenticate successfully
pass
def test_url_normalization_handles_case_differences(app, mocker):
"""Verify URL normalization is case-insensitive"""
# ADMIN_ME: https://Example.com, auth server returns: https://example.com
# Should still authenticate successfully
pass
```
---
## Rollback Plan
If issues occur after deployment:
1. **Code:** Revert to previous commit
2. **Config:** Re-add INDIELOGIN_URL to .env if needed
---
## Post-Deployment Verification
1. Verify login works with the user's actual profile URL
2. Check logs for "Discovered authorization_endpoint" message
3. Test logout and re-login cycle
---
## Architect Q&A (2025-12-17)
Developer questions answered by the architect prior to implementation:
### Q1: Import Placement
**Q:** Should `normalize_url` import be inside the function or at top level?
**A:** Move to top level with other imports for consistency. The design has been updated.
### Q2: URL Normalization Behavior Change
**Q:** Is the URL normalization change intentional?
**A:** Yes, this is an intentional bugfix. The current exact-match behavior is incorrect per IndieAuth spec. URLs differing only in trailing slashes or case should be considered equivalent for identity purposes. The `normalize_url()` function already exists in `auth_external.py` and is used by `verify_external_token()`.
### Q3: Which Endpoint for Authentication Flow?
**Q:** Should we use token_endpoint or authorization_endpoint?
**A:** Use **authorization_endpoint** for authentication-only flows. Per IndieAuth spec: "the client makes a POST request to the authorization endpoint to verify the authorization code and retrieve the final user profile URL." The design has been corrected.
### Q4: Endpoint Validation Relaxation
**Q:** Is relaxed endpoint validation acceptable?
**A:** Yes. Login requires `authorization_endpoint`, Micropub requires `token_endpoint`. Requiring at least one is correct. If only auth endpoint exists, login works but Micropub fails gracefully (401).
### Q5: Test Update Strategy
**Q:** Remove INDIELOGIN_URL and/or mock discover_endpoints()?
**A:** Both. Remove `INDIELOGIN_URL` from fixtures, add `discover_endpoints()` mocking to existing tests. Detailed guidance added to Testing Requirements section.
### Q6: grant_type Parameter
**Q:** Should we include grant_type in the code exchange?
**A:** No. Authentication-only flows do not include `grant_type`. This parameter is only required when POSTing to the token_endpoint for access tokens. The design has been corrected.
### Q7: Error Message Verbosity
**Q:** Should we simplify the user-facing error message?
**A:** Yes. User-facing message should be simple: "Unable to verify your profile URL. Please check that it's correct and try again." Technical details are logged at ERROR level. The design has been updated.
---
## References
- W3C IndieAuth Specification: https://www.w3.org/TR/indieauth/
- IndieAuth Endpoint Discovery: https://www.w3.org/TR/indieauth/#discovery-by-clients