StarPunk/docs/decisions/ADR-030-CORRECTED-indieauth-endpoint-discovery.md

# ADR-030-CORRECTED: IndieAuth Endpoint Discovery Architecture

## Status
Accepted (Replaces incorrect understanding in ADR-030)

## Context

I fundamentally misunderstood IndieAuth endpoint discovery. I incorrectly recommended hardcoding token endpoints like `https://tokens.indieauth.com/token` in configuration. This violates the core principle of IndieAuth: **user sovereignty over authentication endpoints**.

IndieAuth uses **dynamic endpoint discovery** - endpoints are NEVER hardcoded. They are discovered from the user's profile URL at runtime.

## The Correct IndieAuth Flow

### How IndieAuth Actually Works

1. **User Identity**: A user is identified by their URL (e.g., `https://alice.example.com/`)
2. **Endpoint Discovery**: Endpoints are discovered FROM that URL
3. **Provider Choice**: The user chooses their provider by linking to it from their profile
4. **Dynamic Verification**: Token verification uses the discovered endpoint, not a hardcoded one

### Example Flow

When alice authenticates:
```
1. Alice tries to sign in with: https://alice.example.com/
2. Client fetches https://alice.example.com/
3. Client finds: <link rel="authorization_endpoint" href="https://auth.alice.net/auth">
4. Client finds: <link rel="token_endpoint" href="https://auth.alice.net/token">
5. Client uses THOSE endpoints for alice's authentication
```

When bob authenticates:
```
1. Bob tries to sign in with: https://bob.example.org/
2. Client fetches https://bob.example.org/
3. Client finds: <link rel="authorization_endpoint" href="https://indieauth.com/auth">
4. Client finds: <link rel="token_endpoint" href="https://indieauth.com/token">
5. Client uses THOSE endpoints for bob's authentication
```

**Alice and Bob use different providers, discovered from their URLs!**

## Decision: Correct Token Verification Architecture

### Token Verification Flow

```python
def verify_token(token: str) -> dict:
    """
    Verify a token using IndieAuth endpoint discovery

    1. Get claimed 'me' URL (from token introspection or previous knowledge)
    2. Discover token endpoint from 'me' URL
    3. Verify token with discovered endpoint
    4. Validate response
    """

    # Step 1: Initial token introspection (if needed)
    # Some flows provide 'me' in Authorization header or token itself

    # Step 2: Discover endpoints from user's profile URL
    endpoints = discover_endpoints(me_url)
    if not endpoints.get('token_endpoint'):
        raise Error("No token endpoint found for user")

    # Step 3: Verify with discovered endpoint
    response = verify_with_endpoint(
        token=token,
        endpoint=endpoints['token_endpoint']
    )

    # Step 4: Validate response
    if response['me'] != me_url:
        raise Error("Token 'me' doesn't match claimed identity")

    return response
```

### Endpoint Discovery Implementation

```python
def discover_endpoints(profile_url: str) -> dict:
    """
    Discover IndieAuth endpoints from a profile URL
    Per https://www.w3.org/TR/indieauth/#discovery-by-clients

    Priority order:
    1. HTTP Link headers
    2. HTML <link> elements
    3. IndieAuth metadata endpoint
    """

    # Fetch the profile URL
    response = http_get(profile_url, headers={'Accept': 'text/html'})

    endpoints = {}

    # 1. Check HTTP Link headers (highest priority)
    link_header = response.headers.get('Link')
    if link_header:
        endpoints.update(parse_link_header(link_header))

    # 2. Check HTML <link> elements
    if 'text/html' in response.headers.get('Content-Type', ''):
        soup = parse_html(response.text)

        # Find authorization endpoint
        auth_link = soup.find('link', rel='authorization_endpoint')
        if auth_link and not endpoints.get('authorization_endpoint'):
            endpoints['authorization_endpoint'] = urljoin(
                profile_url,
                auth_link.get('href')
            )

        # Find token endpoint
        token_link = soup.find('link', rel='token_endpoint')
        if token_link and not endpoints.get('token_endpoint'):
            endpoints['token_endpoint'] = urljoin(
                profile_url,
                token_link.get('href')
            )

    # 3. Check IndieAuth metadata endpoint (if supported)
    # Look for rel="indieauth-metadata"

    return endpoints
```

### Caching Strategy

```python
class EndpointCache:
    """
    Cache discovered endpoints for performance
    Key insight: User's chosen endpoints rarely change
    """

    def __init__(self, ttl=3600):  # 1 hour default
        self.cache = {}  # profile_url -> (endpoints, expiry)
        self.ttl = ttl

    def get_endpoints(self, profile_url: str) -> dict:
        """Get endpoints, using cache if valid"""

        if profile_url in self.cache:
            endpoints, expiry = self.cache[profile_url]
            if time.time() < expiry:
                return endpoints

        # Discovery needed
        endpoints = discover_endpoints(profile_url)

        # Cache for future use
        self.cache[profile_url] = (
            endpoints,
            time.time() + self.ttl
        )

        return endpoints
```

## Why This Is Correct

### User Sovereignty
- Users control their authentication by choosing their provider
- Users can switch providers by updating their profile links
- No vendor lock-in to specific auth servers

### Decentralization
- No central authority for authentication
- Any server can be an IndieAuth provider
- Users can self-host their auth if desired

### Security
- Provider changes are immediately reflected
- Compromised providers can be switched instantly
- Users maintain control of their identity

## What Was Wrong Before

### The Fatal Flaw
```ini
# WRONG - This violates IndieAuth!
TOKEN_ENDPOINT=https://tokens.indieauth.com/token
```

This assumes ALL users use the same token endpoint. This is fundamentally incorrect because:

1. **Breaks user choice**: Forces everyone to use indieauth.com
2. **Violates spec**: IndieAuth requires endpoint discovery
3. **Security risk**: If indieauth.com is compromised, all users affected
4. **No flexibility**: Users can't switch providers
5. **Not IndieAuth**: This is just OAuth with a hardcoded provider

### The Correct Approach
```ini
# CORRECT - Only store the admin's identity URL
ADMIN_ME=https://admin.example.com/

# Endpoints are discovered from ADMIN_ME at runtime!
```

## Implementation Requirements

### 1. HTTP Client Requirements
- Follow redirects (up to a limit)
- Parse Link headers correctly
- Handle HTML parsing
- Respect Content-Type
- Implement timeouts

### 2. URL Resolution
- Properly resolve relative URLs
- Handle different URL schemes
- Normalize URLs correctly

### 3. Error Handling
- Profile URL unreachable
- No endpoints discovered
- Invalid HTML
- Malformed Link headers
- Network timeouts

### 4. Security Considerations
- Validate HTTPS for endpoints
- Prevent redirect loops
- Limit redirect chains
- Validate discovered URLs
- Cache poisoning prevention

## Configuration Changes

### Remove (WRONG)
```ini
TOKEN_ENDPOINT=https://tokens.indieauth.com/token
AUTHORIZATION_ENDPOINT=https://indieauth.com/auth
```

### Keep (CORRECT)
```ini
ADMIN_ME=https://admin.example.com/
# Endpoints discovered from ADMIN_ME automatically!
```

## Micropub Token Verification Flow

```
1. Micropub receives request with Bearer token
2. Extract token from Authorization header
3. Need to verify token, but with which endpoint?
4. Option A: If we have cached token info, use cached 'me' URL
5. Option B: Try verification with last known endpoint for similar tokens
6. Option C: Require 'me' parameter in Micropub request
7. Discover token endpoint from 'me' URL
8. Verify token with discovered endpoint
9. Cache the verification result and endpoint
10. Process Micropub request if valid
```

## Testing Requirements

### Unit Tests
- Endpoint discovery from HTML
- Link header parsing
- URL resolution
- Cache behavior

### Integration Tests
- Discovery from real IndieAuth providers
- Different HTML structures
- Various Link header formats
- Redirect handling

### Test Cases
```python
# Test different profile configurations
test_profiles = [
    {
        'url': 'https://user1.example.com/',
        'html': '<link rel="token_endpoint" href="https://auth.example.com/token">',
        'expected': 'https://auth.example.com/token'
    },
    {
        'url': 'https://user2.example.com/',
        'html': '<link rel="token_endpoint" href="/auth/token">',  # Relative URL
        'expected': 'https://user2.example.com/auth/token'
    },
    {
        'url': 'https://user3.example.com/',
        'link_header': '<https://indieauth.com/token>; rel="token_endpoint"',
        'expected': 'https://indieauth.com/token'
    }
]
```

## Documentation Requirements

### User Documentation
- Explain how to set up profile URLs
- Show examples of link elements
- List compatible providers
- Troubleshooting guide

### Developer Documentation
- Endpoint discovery algorithm
- Cache implementation details
- Error handling strategies
- Security considerations

## Consequences

### Positive
- **Spec Compliant**: Correctly implements IndieAuth
- **User Freedom**: Users choose their providers
- **Decentralized**: No hardcoded central authority
- **Flexible**: Supports any IndieAuth provider
- **Secure**: Provider changes take effect immediately

### Negative
- **Complexity**: More complex than hardcoded endpoints
- **Performance**: Discovery adds latency (mitigated by caching)
- **Reliability**: Depends on profile URL availability
- **Testing**: More complex test scenarios

## Alternatives Considered

### Alternative 1: Hardcoded Endpoints (REJECTED)
**Why it's wrong**: Violates IndieAuth specification fundamentally

### Alternative 2: Configuration Per User
**Why it's wrong**: Still not dynamic discovery, doesn't follow spec

### Alternative 3: Only Support One Provider
**Why it's wrong**: Defeats the purpose of IndieAuth's decentralization

## References

- [IndieAuth Spec Section 4.2: Discovery](https://www.w3.org/TR/indieauth/#discovery-by-clients)
- [IndieAuth Spec Section 6: Token Verification](https://www.w3.org/TR/indieauth/#token-verification)
- [Link Header RFC 8288](https://tools.ietf.org/html/rfc8288)
- [HTML Link Element Spec](https://html.spec.whatwg.org/multipage/semantics.html#the-link-element)

## Acknowledgment of Error

This ADR corrects a fundamental misunderstanding in the original ADR-030. The error was:
- Recommending hardcoded token endpoints
- Not understanding endpoint discovery
- Missing the core principle of user sovereignty

The architect acknowledges this critical error and has:
1. Re-read the IndieAuth specification thoroughly
2. Understood the importance of endpoint discovery
3. Designed the correct implementation
4. Documented the proper architecture

---

**Document Version**: 2.0 (Complete Correction)
**Created**: 2024-11-24
**Author**: StarPunk Architecture Team
**Note**: This completely replaces the incorrect understanding in ADR-030