StarPunk/docs/design/v1.5.0/2025-12-17-indieauth-discovery-fix.md

# IndieAuth Endpoint Discovery Regression Analysis

**Date**: 2025-12-17
**Author**: Agent-Architect
**Status**: Analysis Complete - Design Proposed
**Version**: v1.5.0 (or v1.6.0 depending on scope decision)

## Executive Summary

StarPunk has a **critical authentication bug** where admin login always redirects to `indielogin.com` instead of the user's declared IndieAuth endpoints. This violates the W3C IndieAuth specification and prevents users with self-hosted IndieAuth servers from logging in.

**Root Cause**: The `initiate_login()` function in `starpunk/auth.py` hardcodes `INDIELOGIN_URL` for authorization, bypassing endpoint discovery entirely. While `auth_external.py` correctly implements endpoint discovery for Micropub token verification, the admin login flow was never updated to use it.

**Impact**: Users whose site declares their own authorization endpoint (e.g., `https://gondulf.thesatelliteoflove.com/authorize`) cannot log in because StarPunk sends them to `https://indielogin.com/authorize` instead. The error message `code_challenge is required (PKCE)` occurs because indielogin.com requires PKCE but the user's server may not.

---

## 1. Git History Investigation

### Timeline of Changes

| Commit | Date | Description | Relevant? |
|--------|------|-------------|-----------|
| `d4f1bfb` | Early Nov 2025 | Initial auth module with IndieLogin support | **Origin of bug** |
| `a3bac86` | 2025-11-24 | Complete IndieAuth server removal (Phases 2-4) | Added `auth_external.py` |
| `80bd51e` | 2025-11-24 | Implement IndieAuth endpoint discovery (v1.0.0-rc.5) | Fixed discovery for **tokens only** |

### Key Finding

The endpoint discovery implemented in commit `80bd51e` (ADR-044) only applies to **Micropub token verification** via `auth_external.py`. The **admin login flow** in `auth.py` was never updated to discover endpoints.

### Code Locations

**Problem Location** - `starpunk/auth.py` lines 306-307:
```python
# CORRECT ENDPOINT: /authorize (not /auth)
auth_url = f"{current_app.config['INDIELOGIN_URL']}/authorize?{urlencode(params)}"
```

**Working Implementation** - `starpunk/auth_external.py` lines 195-247:
The `discover_endpoints()` function correctly implements W3C IndieAuth discovery but is only used for token verification, not login.

### Why This Was Missed

1. **Two separate auth flows**: StarPunk has two distinct authentication contexts:
   - **Admin Login**: Delegates to external authorization server (currently hardcoded to indielogin.com)
   - **Micropub Token Verification**: Validates bearer tokens against user's token endpoint (correctly discovers endpoints)

2. **ADR-044 scope**: The endpoint discovery ADR focused on token verification for Micropub, not the admin login flow.

3. **Working configuration**: indielogin.com works for users who delegate to it, masking the bug for those use cases.

---

## 2. W3C IndieAuth Specification Requirements

Per the [W3C IndieAuth Specification](https://www.w3.org/TR/indieauth/) and the [IndieAuth Living Spec](https://indieauth.spec.indieweb.org/):

### 2.1 Discovery Process (Section 4.2)

Clients MUST:

1. **Fetch the user's profile URL** (the "me" URL they enter at login)
2. **Look for `indieauth-metadata` link relation first** (modern approach)
3. **Fall back to `authorization_endpoint` and `token_endpoint` link relations** (legacy)
4. **Use the DISCOVERED endpoints**, not hardcoded values

### 2.2 Discovery Priority

1. HTTP Link header with `rel="indieauth-metadata"` (preferred)
2. HTML `<link rel="indieauth-metadata">` element
3. Fall back: HTTP Link header with `rel="authorization_endpoint"`
4. Fall back: HTML `<link rel="authorization_endpoint">` element

### 2.3 Metadata Document

When `indieauth-metadata` is found, fetch the JSON document which contains:
```json
{
  "issuer": "https://auth.example.com/",
  "authorization_endpoint": "https://auth.example.com/authorize",
  "token_endpoint": "https://auth.example.com/token",
  "code_challenge_methods_supported": ["S256"]
}
```

### 2.4 Key Quote from Spec

> "Clients MUST start by making a GET or HEAD request to fetch the user's profile URL to discover the necessary values."

---

## 3. Design for v1.5.0/v1.6.0

### 3.1 Architecture Decision

**Recommendation**: Reuse the existing `discover_endpoints()` function from `auth_external.py` for the login flow.

This function already:
- Implements W3C-compliant discovery
- Handles HTTP Link headers and HTML link elements
- Resolves relative URLs
- Validates HTTPS in production
- Caches results (1-hour TTL)
- Has comprehensive test coverage (35 tests)

### 3.2 Proposed Changes

#### 3.2.1 Extend `auth_external.py`

Add support for `indieauth-metadata` discovery (currently missing):

```python
def discover_endpoints(profile_url: str) -> Dict[str, str]:
    """
    Discover IndieAuth endpoints from a profile URL

    Implements IndieAuth endpoint discovery per W3C spec:
    https://www.w3.org/TR/indieauth/#discovery-by-clients

    Discovery priority:
    1. indieauth-metadata link (modern approach)
    2. HTTP Link headers for individual endpoints (legacy)
    3. HTML link elements (legacy)
    """
    # Check cache first
    cached_endpoints = _cache.get_endpoints()
    if cached_endpoints:
        return cached_endpoints

    # Validate and fetch profile
    _validate_profile_url(profile_url)

    try:
        endpoints = _fetch_and_parse(profile_url)
        _cache.set_endpoints(endpoints)
        return endpoints
    except Exception as e:
        # Graceful fallback to expired cache
        cached = _cache.get_endpoints(ignore_expiry=True)
        if cached:
            return cached
        raise DiscoveryError(f"Endpoint discovery failed: {e}")
```

**New function needed**:
```python
def _fetch_metadata(metadata_url: str) -> Dict[str, str]:
    """
    Fetch IndieAuth metadata document and extract endpoints.

    Args:
        metadata_url: URL of the indieauth-metadata document

    Returns:
        Dict with authorization_endpoint and token_endpoint
    """
    response = httpx.get(metadata_url, timeout=DISCOVERY_TIMEOUT)
    response.raise_for_status()
    metadata = response.json()

    return {
        'authorization_endpoint': metadata.get('authorization_endpoint'),
        'token_endpoint': metadata.get('token_endpoint'),
        'issuer': metadata.get('issuer'),
    }
```

#### 3.2.2 Update `auth.py::initiate_login()`

Replace hardcoded INDIELOGIN_URL with discovered endpoint:

**Before**:
```python
def initiate_login(me_url: str) -> str:
    # ... validation ...

    params = {
        "me": me_url,
        "client_id": current_app.config["SITE_URL"],
        "redirect_uri": redirect_uri,
        "state": state,
        "response_type": "code",
    }

    # HARDCODED - THE BUG
    auth_url = f"{current_app.config['INDIELOGIN_URL']}/authorize?{urlencode(params)}"
    return auth_url
```

**After**:
```python
from starpunk.auth_external import discover_endpoints, DiscoveryError

def initiate_login(me_url: str) -> str:
    # ... validation ...

    # Discover endpoints from user's profile URL
    try:
        endpoints = discover_endpoints(me_url)
    except DiscoveryError as e:
        current_app.logger.error(f"Endpoint discovery failed for {me_url}: {e}")
        raise ValueError(f"Could not discover IndieAuth endpoints for {me_url}")

    authorization_endpoint = endpoints.get('authorization_endpoint')
    if not authorization_endpoint:
        raise ValueError(f"No authorization endpoint found at {me_url}")

    params = {
        "me": me_url,
        "client_id": current_app.config["SITE_URL"],
        "redirect_uri": redirect_uri,
        "state": state,
        "response_type": "code",
    }

    # Use discovered endpoint
    auth_url = f"{authorization_endpoint}?{urlencode(params)}"
    return auth_url
```

#### 3.2.3 Update `auth.py::handle_callback()`

The callback handler also references INDIELOGIN_URL for:
1. Issuer validation (line 350)
2. Token exchange (line 371)

These need to use the discovered endpoints instead. Store discovered endpoints in the auth_state table:

**Schema change** - Add column to auth_state:
```sql
ALTER TABLE auth_state ADD COLUMN discovered_endpoints TEXT;
```

Or use session storage (simpler, no migration needed):
```python
session['discovered_endpoints'] = endpoints
```

#### 3.2.4 Configuration Changes

- **Deprecate**: `INDIELOGIN_URL` configuration variable
- **Keep as fallback**: If discovery fails and user has configured INDIELOGIN_URL, use it as a last resort
- **Add deprecation warning**: Log warning if INDIELOGIN_URL is set

### 3.3 Edge Cases

| Scenario | Behavior |
|----------|----------|
| No endpoints found | Raise clear error, log details |
| Only authorization_endpoint found | Use it (token_endpoint not needed for login-only flow) |
| Discovery timeout | Use cached endpoints if available, else fail |
| Invalid endpoints (HTTP in production) | Reject with security error |
| Relative URLs | Resolve against profile URL |
| Redirected profile URL | Follow redirects, use final URL for resolution |

### 3.4 Backward Compatibility

Users who currently rely on indielogin.com via `INDIELOGIN_URL`:
- Their profiles likely don't declare endpoints (they delegate to indielogin.com)
- Discovery will fail, then fall back to INDIELOGIN_URL if configured
- Log deprecation warning recommending they add endpoint declarations to their profile

### 3.5 PKCE Considerations

The error message mentions PKCE (`code_challenge is required`). Current flow:
- StarPunk does NOT send PKCE parameters in `initiate_login()`
- indielogin.com requires PKCE
- User's server (gondulf) may or may not require PKCE

**Design Decision**: Add optional PKCE support based on metadata discovery:
1. If `indieauth-metadata` is found, check `code_challenge_methods_supported`
2. If S256 is supported, include PKCE parameters
3. If not discoverable, default to sending PKCE (most servers support it)

---

## 4. File Changes Summary

| File | Change Type | Description |
|------|-------------|-------------|
| `starpunk/auth_external.py` | Modify | Add `indieauth-metadata` discovery support |
| `starpunk/auth.py` | Modify | Use discovered endpoints instead of INDIELOGIN_URL |
| `starpunk/config.py` | Modify | Deprecate INDIELOGIN_URL with warning |
| `tests/test_auth.py` | Modify | Update tests for endpoint discovery |
| `tests/test_auth_external.py` | Modify | Add metadata discovery tests |
| `.env.example` | Modify | Update INDIELOGIN_URL documentation |

---

## 5. Testing Strategy

### 5.1 Unit Tests

```python
class TestEndpointDiscoveryForLogin:
    """Test endpoint discovery in login flow"""

    def test_discover_from_link_header(self):
        """Authorization endpoint discovered from Link header"""

    def test_discover_from_html_link(self):
        """Authorization endpoint discovered from HTML link element"""

    def test_discover_from_metadata(self):
        """Endpoints discovered from indieauth-metadata document"""

    def test_metadata_priority_over_link(self):
        """Metadata takes priority over individual link relations"""

    def test_discovery_failure_clear_error(self):
        """Clear error message when no endpoints found"""

    def test_fallback_to_indielogin_url(self):
        """Falls back to INDIELOGIN_URL if configured and discovery fails"""

    def test_deprecation_warning_for_indielogin_url(self):
        """Logs deprecation warning when INDIELOGIN_URL used"""
```

### 5.2 Integration Tests

```python
class TestLoginWithDiscovery:
    """End-to-end login flow with endpoint discovery"""

    def test_login_to_self_hosted_indieauth(self):
        """Login works with user's declared authorization endpoint"""

    def test_login_callback_uses_discovered_token_endpoint(self):
        """Callback exchanges code at discovered endpoint"""
```

---

## 6. Open Questions for User

1. **Version Scope**: Should this fix be part of v1.5.0 (quality-focused release) or v1.6.0 (new feature)?
   - **Argument for v1.5.0**: This is a critical bug preventing login
   - **Argument for v1.6.0**: Significant changes to authentication flow

2. **PKCE Implementation**: Should we add full PKCE support as part of this fix?
   - Current: No PKCE in login flow
   - indielogin.com requires PKCE
   - Many IndieAuth servers support PKCE
   - W3C spec recommends PKCE

3. **Migration Path**: How should existing deployments handle this change?
   - Remove INDIELOGIN_URL from config?
   - Add endpoint declarations to their profile?
   - Both (with fallback period)?

---

## 7. Recommended Implementation Order

1. **Phase A**: Extend `discover_endpoints()` to support `indieauth-metadata`
2. **Phase B**: Update `initiate_login()` to use discovered authorization_endpoint
3. **Phase C**: Update `handle_callback()` to use discovered token_endpoint
4. **Phase D**: Add PKCE support (if approved)
5. **Phase E**: Deprecate INDIELOGIN_URL configuration
6. **Phase F**: Update documentation and migration guide

---

## 8. References

- [W3C IndieAuth Specification](https://www.w3.org/TR/indieauth/)
- [IndieAuth Living Spec](https://indieauth.spec.indieweb.org/)
- ADR-044: Endpoint Discovery Implementation Details
- ADR-030: External Token Verification Architecture
- Commit `80bd51e`: Implement IndieAuth endpoint discovery (v1.0.0-rc.5)

---

## 9. Conclusion

The bug is a straightforward oversight: endpoint discovery was implemented for Micropub token verification but not for admin login. The fix involves:

1. Reusing the existing `discover_endpoints()` function
2. Adding `indieauth-metadata` support
3. Replacing hardcoded INDIELOGIN_URL references
4. Optionally adding PKCE support

**Estimated effort**: 8-12 hours including tests and documentation.

**Risk level**: Medium - changes to authentication flow require careful testing.