StarPunk/docs/decisions/ADR-030-CORRECTED-indieauth-endpoint-discovery.md

ADR-030-CORRECTED: IndieAuth Endpoint Discovery Architecture

Status

Accepted (Replaces incorrect understanding in ADR-030)

Context

I fundamentally misunderstood IndieAuth endpoint discovery. I incorrectly recommended hardcoding token endpoints like https://tokens.indieauth.com/token in configuration. This violates the core principle of IndieAuth: user sovereignty over authentication endpoints.

IndieAuth uses dynamic endpoint discovery - endpoints are NEVER hardcoded. They are discovered from the user's profile URL at runtime.

The Correct IndieAuth Flow

How IndieAuth Actually Works

1. User Identity: A user is identified by their URL (e.g., https://alice.example.com/)
2. Endpoint Discovery: Endpoints are discovered FROM that URL
3. Provider Choice: The user chooses their provider by linking to it from their profile
4. Dynamic Verification: Token verification uses the discovered endpoint, not a hardcoded one

Example Flow

When alice authenticates:

1. Alice tries to sign in with: https://alice.example.com/
2. Client fetches https://alice.example.com/
3. Client finds: <link rel="authorization_endpoint" href="https://auth.alice.net/auth">
4. Client finds: <link rel="token_endpoint" href="https://auth.alice.net/token">
5. Client uses THOSE endpoints for alice's authentication

When bob authenticates:

1. Bob tries to sign in with: https://bob.example.org/
2. Client fetches https://bob.example.org/
3. Client finds: <link rel="authorization_endpoint" href="https://indieauth.com/auth">
4. Client finds: <link rel="token_endpoint" href="https://indieauth.com/token">
5. Client uses THOSE endpoints for bob's authentication

Alice and Bob use different providers, discovered from their URLs!

Decision: Correct Token Verification Architecture

Token Verification Flow

def verify_token(token: str) -> dict:
    """
    Verify a token using IndieAuth endpoint discovery

    1. Get claimed 'me' URL (from token introspection or previous knowledge)
    2. Discover token endpoint from 'me' URL
    3. Verify token with discovered endpoint
    4. Validate response
    """

    # Step 1: Initial token introspection (if needed)
    # Some flows provide 'me' in Authorization header or token itself

    # Step 2: Discover endpoints from user's profile URL
    endpoints = discover_endpoints(me_url)
    if not endpoints.get('token_endpoint'):
        raise Error("No token endpoint found for user")

    # Step 3: Verify with discovered endpoint
    response = verify_with_endpoint(
        token=token,
        endpoint=endpoints['token_endpoint']
    )

    # Step 4: Validate response
    if response['me'] != me_url:
        raise Error("Token 'me' doesn't match claimed identity")

    return response

Endpoint Discovery Implementation

def discover_endpoints(profile_url: str) -> dict:
    """
    Discover IndieAuth endpoints from a profile URL
    Per https://www.w3.org/TR/indieauth/#discovery-by-clients

    Priority order:
    1. HTTP Link headers
    2. HTML <link> elements
    3. IndieAuth metadata endpoint
    """

    # Fetch the profile URL
    response = http_get(profile_url, headers={'Accept': 'text/html'})

    endpoints = {}

    # 1. Check HTTP Link headers (highest priority)
    link_header = response.headers.get('Link')
    if link_header:
        endpoints.update(parse_link_header(link_header))

    # 2. Check HTML <link> elements
    if 'text/html' in response.headers.get('Content-Type', ''):
        soup = parse_html(response.text)

        # Find authorization endpoint
        auth_link = soup.find('link', rel='authorization_endpoint')
        if auth_link and not endpoints.get('authorization_endpoint'):
            endpoints['authorization_endpoint'] = urljoin(
                profile_url,
                auth_link.get('href')
            )

        # Find token endpoint
        token_link = soup.find('link', rel='token_endpoint')
        if token_link and not endpoints.get('token_endpoint'):
            endpoints['token_endpoint'] = urljoin(
                profile_url,
                token_link.get('href')
            )

    # 3. Check IndieAuth metadata endpoint (if supported)
    # Look for rel="indieauth-metadata"

    return endpoints

Caching Strategy

class EndpointCache:
    """
    Cache discovered endpoints for performance
    Key insight: User's chosen endpoints rarely change
    """

    def __init__(self, ttl=3600):  # 1 hour default
        self.cache = {}  # profile_url -> (endpoints, expiry)
        self.ttl = ttl

    def get_endpoints(self, profile_url: str) -> dict:
        """Get endpoints, using cache if valid"""

        if profile_url in self.cache:
            endpoints, expiry = self.cache[profile_url]
            if time.time() < expiry:
                return endpoints

        # Discovery needed
        endpoints = discover_endpoints(profile_url)

        # Cache for future use
        self.cache[profile_url] = (
            endpoints,
            time.time() + self.ttl
        )

        return endpoints

Why This Is Correct

User Sovereignty

• Users control their authentication by choosing their provider
• Users can switch providers by updating their profile links
• No vendor lock-in to specific auth servers

Decentralization

• No central authority for authentication
• Any server can be an IndieAuth provider
• Users can self-host their auth if desired

Security

• Provider changes are immediately reflected
• Compromised providers can be switched instantly
• Users maintain control of their identity

What Was Wrong Before

The Fatal Flaw

# WRONG - This violates IndieAuth!
TOKEN_ENDPOINT=https://tokens.indieauth.com/token

This assumes ALL users use the same token endpoint. This is fundamentally incorrect because:

1. Breaks user choice: Forces everyone to use indieauth.com
2. Violates spec: IndieAuth requires endpoint discovery
3. Security risk: If indieauth.com is compromised, all users affected
4. No flexibility: Users can't switch providers
5. Not IndieAuth: This is just OAuth with a hardcoded provider

The Correct Approach

# CORRECT - Only store the admin's identity URL
ADMIN_ME=https://admin.example.com/

# Endpoints are discovered from ADMIN_ME at runtime!

Implementation Requirements

1. HTTP Client Requirements

• Follow redirects (up to a limit)
• Parse Link headers correctly
• Handle HTML parsing
• Respect Content-Type
• Implement timeouts

2. URL Resolution

• Properly resolve relative URLs
• Handle different URL schemes
• Normalize URLs correctly

3. Error Handling

• Profile URL unreachable
• No endpoints discovered
• Invalid HTML
• Malformed Link headers
• Network timeouts

4. Security Considerations

• Validate HTTPS for endpoints
• Prevent redirect loops
• Limit redirect chains
• Validate discovered URLs
• Cache poisoning prevention

Configuration Changes

Remove (WRONG)

TOKEN_ENDPOINT=https://tokens.indieauth.com/token
AUTHORIZATION_ENDPOINT=https://indieauth.com/auth

Keep (CORRECT)

ADMIN_ME=https://admin.example.com/
# Endpoints discovered from ADMIN_ME automatically!

Micropub Token Verification Flow

1. Micropub receives request with Bearer token
2. Extract token from Authorization header
3. Need to verify token, but with which endpoint?
4. Option A: If we have cached token info, use cached 'me' URL
5. Option B: Try verification with last known endpoint for similar tokens
6. Option C: Require 'me' parameter in Micropub request
7. Discover token endpoint from 'me' URL
8. Verify token with discovered endpoint
9. Cache the verification result and endpoint
10. Process Micropub request if valid

Testing Requirements

Unit Tests

• Endpoint discovery from HTML
• Link header parsing
• URL resolution
• Cache behavior

Integration Tests

• Discovery from real IndieAuth providers
• Different HTML structures
• Various Link header formats
• Redirect handling

Test Cases

# Test different profile configurations
test_profiles = [
    {
        'url': 'https://user1.example.com/',
        'html': '<link rel="token_endpoint" href="https://auth.example.com/token">',
        'expected': 'https://auth.example.com/token'
    },
    {
        'url': 'https://user2.example.com/',
        'html': '<link rel="token_endpoint" href="/auth/token">',  # Relative URL
        'expected': 'https://user2.example.com/auth/token'
    },
    {
        'url': 'https://user3.example.com/',
        'link_header': '<https://indieauth.com/token>; rel="token_endpoint"',
        'expected': 'https://indieauth.com/token'
    }
]

Documentation Requirements

User Documentation

• Explain how to set up profile URLs
• Show examples of link elements
• List compatible providers
• Troubleshooting guide

Developer Documentation

• Endpoint discovery algorithm
• Cache implementation details
• Error handling strategies
• Security considerations

Consequences

Positive

• Spec Compliant: Correctly implements IndieAuth
• User Freedom: Users choose their providers
• Decentralized: No hardcoded central authority
• Flexible: Supports any IndieAuth provider
• Secure: Provider changes take effect immediately

Negative

• Complexity: More complex than hardcoded endpoints
• Performance: Discovery adds latency (mitigated by caching)
• Reliability: Depends on profile URL availability
• Testing: More complex test scenarios

Alternatives Considered

Alternative 1: Hardcoded Endpoints (REJECTED)

Why it's wrong: Violates IndieAuth specification fundamentally

Alternative 2: Configuration Per User

Why it's wrong: Still not dynamic discovery, doesn't follow spec

Alternative 3: Only Support One Provider

Why it's wrong: Defeats the purpose of IndieAuth's decentralization

References

• IndieAuth Spec Section 4.2: Discovery
• IndieAuth Spec Section 6: Token Verification
• Link Header RFC 8288
• HTML Link Element Spec

Acknowledgment of Error

This ADR corrects a fundamental misunderstanding in the original ADR-030. The error was:

• Recommending hardcoded token endpoints
• Not understanding endpoint discovery
• Missing the core principle of user sovereignty

The architect acknowledges this critical error and has:

1. Re-read the IndieAuth specification thoroughly
2. Understood the importance of endpoint discovery
3. Designed the correct implementation
4. Documented the proper architecture

---

Document Version: 2.0 (Complete Correction) Created: 2024-11-24 Author: StarPunk Architecture Team Note: This completely replaces the incorrect understanding in ADR-030