StarPunk/docs/design/hotfix/2025-12-17-indieauth-pkce-endpoint-discovery-hotfix.md

IndieAuth Endpoint Discovery Hotfix

Date: 2025-12-17 Type: Production Hotfix Priority: Critical Status: Ready for Implementation

Problem Summary

Users cannot log in to StarPunk. The root cause is that the authentication code ignores endpoint discovery and hardcodes INDIELOGIN_URL instead of discovering the authorization and token endpoints from the user's profile URL.

Root Cause: The starpunk/auth.py module uses INDIELOGIN_URL config instead of discovering endpoints from the user's profile URL as required by the IndieAuth specification. This is a regression - the system used to respect discovered endpoints.

Note: The PKCE error message in the callback is a symptom, not the cause. Once we use the correct discovered endpoints, PKCE will not be required (since the user's actual IndieAuth server doesn't require it).

Specification Requirements

W3C IndieAuth Spec (https://www.w3.org/TR/indieauth/)

• Clients MUST discover authorization_endpoint from user's profile URL
• Clients MUST discover token_endpoint from user's profile URL
• Discovery via HTTP Link headers (highest priority) or HTML <link> elements

Implementation Plan

Overview

The fix reuses the existing discover_endpoints() function from auth_external.py in the login flow. Changes are minimal and focused:

1. Use discover_endpoints() in initiate_login() to get the authorization_endpoint
2. Use discover_endpoints() in handle_callback() to get the token_endpoint
3. Remove INDIELOGIN_URL config (with deprecation warning)

---

Step 1: Update config.py - Remove INDIELOGIN_URL

In /home/phil/Projects/starpunk/starpunk/config.py:

Change 1: Remove the INDIELOGIN_URL config line (line 37):

# DELETE THIS LINE:
app.config["INDIELOGIN_URL"] = os.getenv("INDIELOGIN_URL", "https://indielogin.com")

Change 2: Add deprecation warning for INDIELOGIN_URL (add after the TOKEN_ENDPOINT warning, around line 47):

# DEPRECATED: INDIELOGIN_URL no longer used (hotfix 2025-12-17)
# Authorization endpoint is now discovered from ADMIN_ME profile per IndieAuth spec
if 'INDIELOGIN_URL' in os.environ:
    app.logger.warning(
        "INDIELOGIN_URL is deprecated and will be ignored. "
        "Remove it from your configuration. "
        "The authorization endpoint is now discovered automatically from your ADMIN_ME profile."
    )

---

Step 2: Update auth.py - Use Endpoint Discovery

In /home/phil/Projects/starpunk/starpunk/auth.py:

Change 1: Add import for endpoint discovery (after line 42):

from starpunk.auth_external import discover_endpoints, DiscoveryError, normalize_url

Note: The normalize_url import is at the top level (not inside handle_callback()) for consistency with the existing code style.

Change 2: Update initiate_login() to use discovered authorization_endpoint (replace lines 251-318):

def initiate_login(me_url: str) -> str:
    """
    Initiate IndieAuth authentication flow with endpoint discovery.

    Per W3C IndieAuth spec, discovers authorization_endpoint from user's
    profile URL rather than using a hardcoded service.

    Args:
        me_url: User's IndieWeb identity URL

    Returns:
        Redirect URL to discovered authorization endpoint

    Raises:
        ValueError: Invalid me_url format
        DiscoveryError: Failed to discover endpoints from profile
    """
    # Validate URL format
    if not is_valid_url(me_url):
        raise ValueError(f"Invalid URL format: {me_url}")

    current_app.logger.debug(f"Auth: Validating me URL: {me_url}")

    # Discover authorization endpoint from user's profile URL
    # Per IndieAuth spec: clients MUST discover endpoints, not hardcode them
    try:
        endpoints = discover_endpoints(me_url)
    except DiscoveryError as e:
        current_app.logger.error(f"Auth: Endpoint discovery failed for {me_url}: {e}")
        raise

    auth_endpoint = endpoints.get('authorization_endpoint')
    if not auth_endpoint:
        raise DiscoveryError(
            f"No authorization_endpoint found at {me_url}. "
            "Ensure your profile has IndieAuth link elements or headers."
        )

    current_app.logger.info(f"Auth: Discovered authorization_endpoint: {auth_endpoint}")

    # Generate CSRF state token
    state = _generate_state_token()
    current_app.logger.debug(f"Auth: Generated state token: {_redact_token(state, 8)}")

    # Store state in database (5-minute expiry)
    db = get_db(current_app)
    expires_at = datetime.utcnow() + timedelta(minutes=5)
    redirect_uri = f"{current_app.config['SITE_URL']}auth/callback"

    db.execute(
        """
        INSERT INTO auth_state (state, expires_at, redirect_uri)
        VALUES (?, ?, ?)
        """,
        (state, expires_at, redirect_uri),
    )
    db.commit()

    # Build authorization URL
    params = {
        "me": me_url,
        "client_id": current_app.config["SITE_URL"],
        "redirect_uri": redirect_uri,
        "state": state,
        "response_type": "code",
    }

    current_app.logger.debug(
        f"Auth: Building authorization URL with params:\n"
        f"  me: {me_url}\n"
        f"  client_id: {current_app.config['SITE_URL']}\n"
        f"  redirect_uri: {redirect_uri}\n"
        f"  state: {_redact_token(state, 8)}\n"
        f"  response_type: code"
    )

    auth_url = f"{auth_endpoint}?{urlencode(params)}"

    current_app.logger.debug(f"Auth: Complete authorization URL: {auth_url}")
    current_app.logger.info(f"Auth: Authentication initiated for {me_url}")

    return auth_url

Change 3: Update handle_callback() to use discovered authorization_endpoint (replace lines 321-474):

Important: Per IndieAuth spec, authentication-only flows (identity verification without access tokens) POST to the authorization_endpoint, NOT the token_endpoint. The grant_type parameter is NOT included for authentication-only flows.

def handle_callback(code: str, state: str, iss: Optional[str] = None) -> Optional[str]:
    """
    Handle IndieAuth callback with endpoint discovery.

    Discovers authorization_endpoint from ADMIN_ME profile and exchanges
    authorization code for identity verification.

    Per IndieAuth spec: Authentication-only flows POST to the authorization
    endpoint (not token endpoint) and do not include grant_type.

    Args:
        code: Authorization code from authorization server
        state: CSRF state token
        iss: Issuer identifier (optional, for security validation)

    Returns:
        Session token if successful, None otherwise

    Raises:
        InvalidStateError: State token validation failed
        UnauthorizedError: User not authorized as admin
        IndieLoginError: Code exchange failed
    """
    current_app.logger.debug(f"Auth: Verifying state token: {_redact_token(state, 8)}")

    # Verify state token (CSRF protection)
    if not _verify_state_token(state):
        current_app.logger.warning(
            "Auth: Invalid state token received (possible CSRF or expired token)"
        )
        raise InvalidStateError("Invalid or expired state token")

    current_app.logger.debug("Auth: State token valid")

    # Discover authorization endpoint from ADMIN_ME profile
    admin_me = current_app.config.get("ADMIN_ME")
    if not admin_me:
        current_app.logger.error("Auth: ADMIN_ME not configured")
        raise IndieLoginError("ADMIN_ME not configured")

    try:
        endpoints = discover_endpoints(admin_me)
    except DiscoveryError as e:
        current_app.logger.error(f"Auth: Endpoint discovery failed: {e}")
        raise IndieLoginError(f"Failed to discover endpoints: {e}")

    # Use authorization_endpoint for authentication-only flow (identity verification)
    # Per IndieAuth spec: auth-only flows POST to authorization_endpoint, not token_endpoint
    auth_endpoint = endpoints.get('authorization_endpoint')
    if not auth_endpoint:
        raise IndieLoginError(
            f"No authorization_endpoint found at {admin_me}. "
            "Ensure your profile has IndieAuth endpoints configured."
        )

    current_app.logger.debug(f"Auth: Using authorization_endpoint: {auth_endpoint}")

    # Verify issuer if provided (security check - optional)
    if iss:
        current_app.logger.debug(f"Auth: Issuer provided: {iss}")

    # Prepare code verification request
    # Note: grant_type is NOT included for authentication-only flows per IndieAuth spec
    token_exchange_data = {
        "code": code,
        "client_id": current_app.config["SITE_URL"],
        "redirect_uri": f"{current_app.config['SITE_URL']}auth/callback",
    }

    # Log the request
    _log_http_request(
        method="POST",
        url=auth_endpoint,
        data=token_exchange_data,
    )

    current_app.logger.debug(
        "Auth: Sending code verification request to authorization endpoint:\n"
        "  Method: POST\n"
        "  URL: %s\n"
        "  Data: code=%s, client_id=%s, redirect_uri=%s",
        auth_endpoint,
        _redact_token(code),
        token_exchange_data["client_id"],
        token_exchange_data["redirect_uri"],
    )

    # Exchange code for identity at authorization endpoint (authentication-only flow)
    try:
        response = httpx.post(
            auth_endpoint,
            data=token_exchange_data,
            timeout=10.0,
        )

        current_app.logger.debug(
            "Auth: Received code verification response:\n"
            "  Status: %d\n"
            "  Headers: %s\n"
            "  Body: %s",
            response.status_code,
            {k: v for k, v in dict(response.headers).items()
             if k.lower() not in ["set-cookie", "authorization"]},
            _redact_token(response.text) if response.text else "(empty)",
        )

        _log_http_response(
            status_code=response.status_code,
            headers=dict(response.headers),
            body=response.text,
        )

        response.raise_for_status()
    except httpx.RequestError as e:
        current_app.logger.error(f"Auth: Authorization endpoint request failed: {e}")
        raise IndieLoginError(f"Failed to verify code: {e}")
    except httpx.HTTPStatusError as e:
        current_app.logger.error(
            f"Auth: Authorization endpoint returned error: {e.response.status_code} - {e.response.text}"
        )
        raise IndieLoginError(
            f"Authorization endpoint returned error: {e.response.status_code}"
        )

    # Parse response
    try:
        data = response.json()
    except Exception as e:
        current_app.logger.error(f"Auth: Failed to parse authorization endpoint response: {e}")
        raise IndieLoginError("Invalid JSON response from authorization endpoint")

    me = data.get("me")

    if not me:
        current_app.logger.error("Auth: No identity returned from authorization endpoint")
        raise IndieLoginError("No identity returned from authorization endpoint")

    current_app.logger.debug(f"Auth: Received identity: {me}")

    # Verify this is the admin user
    current_app.logger.info(f"Auth: Verifying admin authorization for me={me}")

    # Normalize URLs for comparison (handles trailing slashes and case differences)
    # This is correct per IndieAuth spec - the returned 'me' is the canonical form
    if normalize_url(me) != normalize_url(admin_me):
        current_app.logger.warning(
            f"Auth: Unauthorized login attempt: {me} (expected {admin_me})"
        )
        raise UnauthorizedError(f"User {me} is not authorized")

    current_app.logger.debug("Auth: Admin verification passed")

    # Create session
    session_token = create_session(me)

    # Trigger author profile discovery (v1.2.0 Phase 2)
    # Per Q14: Never block login, always allow fallback
    try:
        from starpunk.author_discovery import get_author_profile
        author_profile = get_author_profile(me, refresh=True)
        current_app.logger.info(f"Author profile refreshed for {me}")
    except Exception as e:
        current_app.logger.warning(f"Author discovery failed: {e}")
        # Continue login anyway - never block per Q14

    return session_token

---

Step 3: Update auth_external.py - Relax Endpoint Validation

The existing _fetch_and_parse() function requires token_endpoint to be present. We need to relax this since some profiles may only have authorization_endpoint (for authentication-only flows).

In /home/phil/Projects/starpunk/starpunk/auth_external.py, update the validation in _fetch_and_parse() (around lines 302-307):

Change: Make token_endpoint not strictly required (allow authentication-only profiles):

# Validate we found at least one endpoint
# - authorization_endpoint: Required for authentication-only flows (admin login)
# - token_endpoint: Required for Micropub token verification
# Having at least one allows the appropriate flow to work
if 'token_endpoint' not in endpoints and 'authorization_endpoint' not in endpoints:
    raise DiscoveryError(
        f"No IndieAuth endpoints found at {profile_url}. "
        "Ensure your profile has authorization_endpoint or token_endpoint configured."
    )

---

Step 4: Update routes/auth.py - Handle DiscoveryError

In /home/phil/Projects/starpunk/starpunk/routes/auth.py:

Change 1: Add import for DiscoveryError (update lines 20-29):

from starpunk.auth import (
    IndieLoginError,
    InvalidStateError,
    UnauthorizedError,
    destroy_session,
    handle_callback,
    initiate_login,
    require_auth,
    verify_session,
)
from starpunk.auth_external import DiscoveryError

Change 2: Handle DiscoveryError in login_initiate() (update lines 79-85):

Note: The user-facing error message is kept simple. Technical details are logged but not shown to users.

    try:
        # Initiate IndieAuth flow
        auth_url = initiate_login(me_url)
        return redirect(auth_url)
    except ValueError as e:
        flash(str(e), "error")
        return redirect(url_for("auth.login_form"))
    except DiscoveryError as e:
        current_app.logger.error(f"Endpoint discovery failed for {me_url}: {e}")
        flash("Unable to verify your profile URL. Please check that it's correct and try again.", "error")
        return redirect(url_for("auth.login_form"))

---

File Summary

File	Change Type	Description
starpunk/config.py	Edit	Remove INDIELOGIN_URL, add deprecation warning
starpunk/auth.py	Edit	Use endpoint discovery instead of hardcoded URL
starpunk/auth_external.py	Edit	Relax endpoint validation (allow auth-only flow)
starpunk/routes/auth.py	Edit	Handle DiscoveryError exception

---

Testing Requirements

Manual Testing

1. Login Flow Test

   • Navigate to /auth/login
   • Enter ADMIN_ME URL
   • Verify redirect goes to discovered authorization_endpoint (not hardcoded indielogin.com)
   • Complete login and verify session is created

2. Endpoint Discovery Test

   • Test with profile that declares custom endpoints
   • Verify discovered endpoints are used, not defaults

Existing Test Updates

Update test fixture in tests/test_auth.py:

Remove INDIELOGIN_URL from the app fixture (line 51):

@pytest.fixture
def app(tmp_path):
    """Create Flask app for testing"""
    from starpunk import create_app

    test_data_dir = tmp_path / "data"
    test_data_dir.mkdir(parents=True, exist_ok=True)

    app = create_app(
        {
            "TESTING": True,
            "SITE_URL": "http://localhost:5000/",
            "ADMIN_ME": "https://example.com",
            "SESSION_SECRET": secrets.token_hex(32),
            "SESSION_LIFETIME": 30,
            # REMOVED: "INDIELOGIN_URL": "https://indielogin.com",
            "DATA_PATH": test_data_dir,
            "NOTES_PATH": test_data_dir / "notes",
            "DATABASE_PATH": test_data_dir / "starpunk.db",
        }
    )
    return app

Update existing tests that use httpx.post mock:

Tests in TestInitiateLogin and TestHandleCallback need to mock discover_endpoints() in addition to httpx.post. Example pattern:

@patch("starpunk.auth.discover_endpoints")
@patch("starpunk.auth.httpx.post")
def test_handle_callback_success(self, mock_post, mock_discover, app, db, client):
    """Test successful callback handling"""
    # Mock endpoint discovery
    mock_discover.return_value = {
        'authorization_endpoint': 'https://auth.example.com/authorize',
        'token_endpoint': 'https://auth.example.com/token'
    }

    # Rest of test remains the same...

Update TestInitiateLogin.test_initiate_login_success:

The assertion checking for indielogin.com needs to change to check for the mocked endpoint:

@patch("starpunk.auth.discover_endpoints")
def test_initiate_login_success(self, mock_discover, app, db):
    """Test successful login initiation"""
    mock_discover.return_value = {
        'authorization_endpoint': 'https://auth.example.com/authorize',
        'token_endpoint': 'https://auth.example.com/token'
    }

    with app.app_context():
        me_url = "https://example.com"
        auth_url = initiate_login(me_url)

        # Changed: Check for discovered endpoint instead of indielogin.com
        assert "auth.example.com/authorize" in auth_url
        assert "me=https%3A%2F%2Fexample.com" in auth_url
        # ... rest of assertions

New Automated Tests to Add

# tests/test_auth_endpoint_discovery.py

def test_initiate_login_uses_endpoint_discovery(client, mocker):
    """Verify login uses discovered endpoint, not hardcoded"""
    mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
    mock_discover.return_value = {
        'authorization_endpoint': 'https://custom-auth.example.com/authorize',
        'token_endpoint': 'https://custom-auth.example.com/token'
    }

    response = client.post('/auth/login', data={'me': 'https://example.com'})

    assert response.status_code == 302
    assert 'custom-auth.example.com' in response.headers['Location']


def test_callback_uses_discovered_authorization_endpoint(client, mocker):
    """Verify callback uses discovered authorization endpoint (not token endpoint)"""
    mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
    mock_discover.return_value = {
        'authorization_endpoint': 'https://custom-auth.example.com/authorize',
        'token_endpoint': 'https://custom-auth.example.com/token'
    }
    mock_post = mocker.patch('starpunk.auth.httpx.post')
    # Setup state token and mock httpx response
    # Verify code exchange POSTs to authorization_endpoint, not token_endpoint
    pass


def test_discovery_error_shows_user_friendly_message(client, mocker):
    """Verify discovery failures show helpful error"""
    mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
    mock_discover.side_effect = DiscoveryError("No endpoints found")

    response = client.post('/auth/login', data={'me': 'https://example.com'})

    assert response.status_code == 302
    # Should redirect back to login form with flash message


def test_url_normalization_handles_trailing_slash(app, mocker):
    """Verify URL normalization allows trailing slash differences"""
    # ADMIN_ME without trailing slash, auth server returns with trailing slash
    # Should still authenticate successfully
    pass


def test_url_normalization_handles_case_differences(app, mocker):
    """Verify URL normalization is case-insensitive"""
    # ADMIN_ME: https://Example.com, auth server returns: https://example.com
    # Should still authenticate successfully
    pass

---

Rollback Plan

If issues occur after deployment:

1. Code: Revert to previous commit
2. Config: Re-add INDIELOGIN_URL to .env if needed

---

Post-Deployment Verification

1. Verify login works with the user's actual profile URL
2. Check logs for "Discovered authorization_endpoint" message
3. Test logout and re-login cycle

---

Architect Q&A (2025-12-17)

Developer questions answered by the architect prior to implementation:

Q1: Import Placement

Q: Should normalize_url import be inside the function or at top level? A: Move to top level with other imports for consistency. The design has been updated.

Q2: URL Normalization Behavior Change

Q: Is the URL normalization change intentional? A: Yes, this is an intentional bugfix. The current exact-match behavior is incorrect per IndieAuth spec. URLs differing only in trailing slashes or case should be considered equivalent for identity purposes. The normalize_url() function already exists in auth_external.py and is used by verify_external_token().

Q3: Which Endpoint for Authentication Flow?

Q: Should we use token_endpoint or authorization_endpoint? A: Use authorization_endpoint for authentication-only flows. Per IndieAuth spec: "the client makes a POST request to the authorization endpoint to verify the authorization code and retrieve the final user profile URL." The design has been corrected.

Q4: Endpoint Validation Relaxation

Q: Is relaxed endpoint validation acceptable? A: Yes. Login requires authorization_endpoint, Micropub requires token_endpoint. Requiring at least one is correct. If only auth endpoint exists, login works but Micropub fails gracefully (401).

Q5: Test Update Strategy

Q: Remove INDIELOGIN_URL and/or mock discover_endpoints()? A: Both. Remove INDIELOGIN_URL from fixtures, add discover_endpoints() mocking to existing tests. Detailed guidance added to Testing Requirements section.

Q6: grant_type Parameter

Q: Should we include grant_type in the code exchange? A: No. Authentication-only flows do not include grant_type. This parameter is only required when POSTing to the token_endpoint for access tokens. The design has been corrected.

Q7: Error Message Verbosity

Q: Should we simplify the user-facing error message? A: Yes. User-facing message should be simple: "Unable to verify your profile URL. Please check that it's correct and try again." Technical details are logged at ERROR level. The design has been updated.

---

References

• W3C IndieAuth Specification: https://www.w3.org/TR/indieauth/
• IndieAuth Endpoint Discovery: https://www.w3.org/TR/indieauth/#discovery-by-clients