# IndieAuth Endpoint Discovery Hotfix

**Date:** 2025-12-17
**Type:** Production Hotfix
**Priority:** Critical
**Status:** Ready for Implementation

## Problem Summary

Users cannot log in to StarPunk. The root cause is that the authentication code ignores endpoint discovery and hardcodes `INDIELOGIN_URL` instead of discovering the authorization and token endpoints from the user's profile URL.

**Root Cause:** The `starpunk/auth.py` module uses `INDIELOGIN_URL` config instead of discovering endpoints from the user's profile URL as required by the IndieAuth specification. This is a regression - the system used to respect discovered endpoints.

**Note:** The PKCE error message in the callback is a symptom, not the cause. Once we use the correct discovered endpoints, PKCE will not be required (since the user's actual IndieAuth server doesn't require it).

## Specification Requirements

### W3C IndieAuth Spec (https://www.w3.org/TR/indieauth/)

- Clients MUST discover `authorization_endpoint` from user's profile URL
- Clients MUST discover `token_endpoint` from user's profile URL
- Discovery via HTTP Link headers (highest priority) or HTML `<link>` elements

## Implementation Plan

### Overview

The fix reuses the existing `discover_endpoints()` function from `auth_external.py` in the login flow. Changes are minimal and focused:

1. Use `discover_endpoints()` in `initiate_login()` to get the `authorization_endpoint`
2. Use `discover_endpoints()` in `handle_callback()` to get the `token_endpoint`
3. Remove `INDIELOGIN_URL` config (with deprecation warning)

---

### Step 1: Update config.py - Remove INDIELOGIN_URL

In `/home/phil/Projects/starpunk/starpunk/config.py`:

**Change 1:** Remove the INDIELOGIN_URL config line (line 37):

```python
# DELETE THIS LINE:
app.config["INDIELOGIN_URL"] = os.getenv("INDIELOGIN_URL", "https://indielogin.com")
```

**Change 2:** Add deprecation warning for INDIELOGIN_URL (add after the TOKEN_ENDPOINT warning, around line 47):

```python
# DEPRECATED: INDIELOGIN_URL no longer used (hotfix 2025-12-17)
# Authorization endpoint is now discovered from ADMIN_ME profile per IndieAuth spec
if 'INDIELOGIN_URL' in os.environ:
    app.logger.warning(
        "INDIELOGIN_URL is deprecated and will be ignored. "
        "Remove it from your configuration. "
        "The authorization endpoint is now discovered automatically from your ADMIN_ME profile."
    )
```

---

### Step 2: Update auth.py - Use Endpoint Discovery

In `/home/phil/Projects/starpunk/starpunk/auth.py`:

**Change 1:** Add import for endpoint discovery (after line 42):

```python
from starpunk.auth_external import discover_endpoints, DiscoveryError, normalize_url
```

> **Note:** The `normalize_url` import is at the top level (not inside `handle_callback()`) for consistency with the existing code style.

**Change 2:** Update `initiate_login()` to use discovered authorization_endpoint (replace lines 251-318):

```python
def initiate_login(me_url: str) -> str:
    """
    Initiate IndieAuth authentication flow with endpoint discovery.

    Per W3C IndieAuth spec, discovers authorization_endpoint from user's
    profile URL rather than using a hardcoded service.

    Args:
        me_url: User's IndieWeb identity URL

    Returns:
        Redirect URL to discovered authorization endpoint

    Raises:
        ValueError: Invalid me_url format
        DiscoveryError: Failed to discover endpoints from profile
    """
    # Validate URL format
    if not is_valid_url(me_url):
        raise ValueError(f"Invalid URL format: {me_url}")

    current_app.logger.debug(f"Auth: Validating me URL: {me_url}")

    # Discover authorization endpoint from user's profile URL
    # Per IndieAuth spec: clients MUST discover endpoints, not hardcode them
    try:
        endpoints = discover_endpoints(me_url)
    except DiscoveryError as e:
        current_app.logger.error(f"Auth: Endpoint discovery failed for {me_url}: {e}")
        raise

    auth_endpoint = endpoints.get('authorization_endpoint')
    if not auth_endpoint:
        raise DiscoveryError(
            f"No authorization_endpoint found at {me_url}. "
            "Ensure your profile has IndieAuth link elements or headers."
        )

    current_app.logger.info(f"Auth: Discovered authorization_endpoint: {auth_endpoint}")

    # Generate CSRF state token
    state = _generate_state_token()
    current_app.logger.debug(f"Auth: Generated state token: {_redact_token(state, 8)}")

    # Store state in database (5-minute expiry)
    db = get_db(current_app)
    expires_at = datetime.utcnow() + timedelta(minutes=5)
    redirect_uri = f"{current_app.config['SITE_URL']}auth/callback"

    db.execute(
        """
        INSERT INTO auth_state (state, expires_at, redirect_uri)
        VALUES (?, ?, ?)
        """,
        (state, expires_at, redirect_uri),
    )
    db.commit()

    # Build authorization URL
    params = {
        "me": me_url,
        "client_id": current_app.config["SITE_URL"],
        "redirect_uri": redirect_uri,
        "state": state,
        "response_type": "code",
    }

    current_app.logger.debug(
        f"Auth: Building authorization URL with params:\n"
        f"  me: {me_url}\n"
        f"  client_id: {current_app.config['SITE_URL']}\n"
        f"  redirect_uri: {redirect_uri}\n"
        f"  state: {_redact_token(state, 8)}\n"
        f"  response_type: code"
    )

    auth_url = f"{auth_endpoint}?{urlencode(params)}"

    current_app.logger.debug(f"Auth: Complete authorization URL: {auth_url}")
    current_app.logger.info(f"Auth: Authentication initiated for {me_url}")

    return auth_url
```

**Change 3:** Update `handle_callback()` to use discovered authorization_endpoint (replace lines 321-474):

> **Important:** Per IndieAuth spec, authentication-only flows (identity verification without access tokens) POST to the **authorization_endpoint**, NOT the token_endpoint. The `grant_type` parameter is NOT included for authentication-only flows.

```python
def handle_callback(code: str, state: str, iss: Optional[str] = None) -> Optional[str]:
    """
    Handle IndieAuth callback with endpoint discovery.

    Discovers authorization_endpoint from ADMIN_ME profile and exchanges
    authorization code for identity verification.

    Per IndieAuth spec: Authentication-only flows POST to the authorization
    endpoint (not token endpoint) and do not include grant_type.

    Args:
        code: Authorization code from authorization server
        state: CSRF state token
        iss: Issuer identifier (optional, for security validation)

    Returns:
        Session token if successful, None otherwise

    Raises:
        InvalidStateError: State token validation failed
        UnauthorizedError: User not authorized as admin
        IndieLoginError: Code exchange failed
    """
    current_app.logger.debug(f"Auth: Verifying state token: {_redact_token(state, 8)}")

    # Verify state token (CSRF protection)
    if not _verify_state_token(state):
        current_app.logger.warning(
            "Auth: Invalid state token received (possible CSRF or expired token)"
        )
        raise InvalidStateError("Invalid or expired state token")

    current_app.logger.debug("Auth: State token valid")

    # Discover authorization endpoint from ADMIN_ME profile
    admin_me = current_app.config.get("ADMIN_ME")
    if not admin_me:
        current_app.logger.error("Auth: ADMIN_ME not configured")
        raise IndieLoginError("ADMIN_ME not configured")

    try:
        endpoints = discover_endpoints(admin_me)
    except DiscoveryError as e:
        current_app.logger.error(f"Auth: Endpoint discovery failed: {e}")
        raise IndieLoginError(f"Failed to discover endpoints: {e}")

    # Use authorization_endpoint for authentication-only flow (identity verification)
    # Per IndieAuth spec: auth-only flows POST to authorization_endpoint, not token_endpoint
    auth_endpoint = endpoints.get('authorization_endpoint')
    if not auth_endpoint:
        raise IndieLoginError(
            f"No authorization_endpoint found at {admin_me}. "
            "Ensure your profile has IndieAuth endpoints configured."
        )

    current_app.logger.debug(f"Auth: Using authorization_endpoint: {auth_endpoint}")

    # Verify issuer if provided (security check - optional)
    if iss:
        current_app.logger.debug(f"Auth: Issuer provided: {iss}")

    # Prepare code verification request
    # Note: grant_type is NOT included for authentication-only flows per IndieAuth spec
    token_exchange_data = {
        "code": code,
        "client_id": current_app.config["SITE_URL"],
        "redirect_uri": f"{current_app.config['SITE_URL']}auth/callback",
    }

    # Log the request
    _log_http_request(
        method="POST",
        url=auth_endpoint,
        data=token_exchange_data,
    )

    current_app.logger.debug(
        "Auth: Sending code verification request to authorization endpoint:\n"
        "  Method: POST\n"
        "  URL: %s\n"
        "  Data: code=%s, client_id=%s, redirect_uri=%s",
        auth_endpoint,
        _redact_token(code),
        token_exchange_data["client_id"],
        token_exchange_data["redirect_uri"],
    )

    # Exchange code for identity at authorization endpoint (authentication-only flow)
    try:
        response = httpx.post(
            auth_endpoint,
            data=token_exchange_data,
            timeout=10.0,
        )

        current_app.logger.debug(
            "Auth: Received code verification response:\n"
            "  Status: %d\n"
            "  Headers: %s\n"
            "  Body: %s",
            response.status_code,
            {k: v for k, v in dict(response.headers).items()
             if k.lower() not in ["set-cookie", "authorization"]},
            _redact_token(response.text) if response.text else "(empty)",
        )

        _log_http_response(
            status_code=response.status_code,
            headers=dict(response.headers),
            body=response.text,
        )

        response.raise_for_status()
    except httpx.RequestError as e:
        current_app.logger.error(f"Auth: Authorization endpoint request failed: {e}")
        raise IndieLoginError(f"Failed to verify code: {e}")
    except httpx.HTTPStatusError as e:
        current_app.logger.error(
            f"Auth: Authorization endpoint returned error: {e.response.status_code} - {e.response.text}"
        )
        raise IndieLoginError(
            f"Authorization endpoint returned error: {e.response.status_code}"
        )

    # Parse response
    try:
        data = response.json()
    except Exception as e:
        current_app.logger.error(f"Auth: Failed to parse authorization endpoint response: {e}")
        raise IndieLoginError("Invalid JSON response from authorization endpoint")

    me = data.get("me")

    if not me:
        current_app.logger.error("Auth: No identity returned from authorization endpoint")
        raise IndieLoginError("No identity returned from authorization endpoint")

    current_app.logger.debug(f"Auth: Received identity: {me}")

    # Verify this is the admin user
    current_app.logger.info(f"Auth: Verifying admin authorization for me={me}")

    # Normalize URLs for comparison (handles trailing slashes and case differences)
    # This is correct per IndieAuth spec - the returned 'me' is the canonical form
    if normalize_url(me) != normalize_url(admin_me):
        current_app.logger.warning(
            f"Auth: Unauthorized login attempt: {me} (expected {admin_me})"
        )
        raise UnauthorizedError(f"User {me} is not authorized")

    current_app.logger.debug("Auth: Admin verification passed")

    # Create session
    session_token = create_session(me)

    # Trigger author profile discovery (v1.2.0 Phase 2)
    # Per Q14: Never block login, always allow fallback
    try:
        from starpunk.author_discovery import get_author_profile
        author_profile = get_author_profile(me, refresh=True)
        current_app.logger.info(f"Author profile refreshed for {me}")
    except Exception as e:
        current_app.logger.warning(f"Author discovery failed: {e}")
        # Continue login anyway - never block per Q14

    return session_token
```

---

### Step 3: Update auth_external.py - Relax Endpoint Validation

The existing `_fetch_and_parse()` function requires `token_endpoint` to be present. We need to relax this since some profiles may only have `authorization_endpoint` (for authentication-only flows).

In `/home/phil/Projects/starpunk/starpunk/auth_external.py`, update the validation in `_fetch_and_parse()` (around lines 302-307):

**Change:** Make token_endpoint not strictly required (allow authentication-only profiles):

```python
# Validate we found at least one endpoint
# - authorization_endpoint: Required for authentication-only flows (admin login)
# - token_endpoint: Required for Micropub token verification
# Having at least one allows the appropriate flow to work
if 'token_endpoint' not in endpoints and 'authorization_endpoint' not in endpoints:
    raise DiscoveryError(
        f"No IndieAuth endpoints found at {profile_url}. "
        "Ensure your profile has authorization_endpoint or token_endpoint configured."
    )
```

---

### Step 4: Update routes/auth.py - Handle DiscoveryError

In `/home/phil/Projects/starpunk/starpunk/routes/auth.py`:

**Change 1:** Add import for DiscoveryError (update lines 20-29):

```python
from starpunk.auth import (
    IndieLoginError,
    InvalidStateError,
    UnauthorizedError,
    destroy_session,
    handle_callback,
    initiate_login,
    require_auth,
    verify_session,
)
from starpunk.auth_external import DiscoveryError
```

**Change 2:** Handle DiscoveryError in login_initiate() (update lines 79-85):

> **Note:** The user-facing error message is kept simple. Technical details are logged but not shown to users.

```python
    try:
        # Initiate IndieAuth flow
        auth_url = initiate_login(me_url)
        return redirect(auth_url)
    except ValueError as e:
        flash(str(e), "error")
        return redirect(url_for("auth.login_form"))
    except DiscoveryError as e:
        current_app.logger.error(f"Endpoint discovery failed for {me_url}: {e}")
        flash("Unable to verify your profile URL. Please check that it's correct and try again.", "error")
        return redirect(url_for("auth.login_form"))
```

---

## File Summary

| File | Change Type | Description |
|------|-------------|-------------|
| `starpunk/config.py` | Edit | Remove INDIELOGIN_URL, add deprecation warning |
| `starpunk/auth.py` | Edit | Use endpoint discovery instead of hardcoded URL |
| `starpunk/auth_external.py` | Edit | Relax endpoint validation (allow auth-only flow) |
| `starpunk/routes/auth.py` | Edit | Handle DiscoveryError exception |

---

## Testing Requirements

### Manual Testing

1. **Login Flow Test**
   - Navigate to `/auth/login`
   - Enter ADMIN_ME URL
   - Verify redirect goes to discovered authorization_endpoint (not hardcoded indielogin.com)
   - Complete login and verify session is created

2. **Endpoint Discovery Test**
   - Test with profile that declares custom endpoints
   - Verify discovered endpoints are used, not defaults

### Existing Test Updates

**Update test fixture in `tests/test_auth.py`:**

Remove `INDIELOGIN_URL` from the app fixture (line 51):

```python
@pytest.fixture
def app(tmp_path):
    """Create Flask app for testing"""
    from starpunk import create_app

    test_data_dir = tmp_path / "data"
    test_data_dir.mkdir(parents=True, exist_ok=True)

    app = create_app(
        {
            "TESTING": True,
            "SITE_URL": "http://localhost:5000/",
            "ADMIN_ME": "https://example.com",
            "SESSION_SECRET": secrets.token_hex(32),
            "SESSION_LIFETIME": 30,
            # REMOVED: "INDIELOGIN_URL": "https://indielogin.com",
            "DATA_PATH": test_data_dir,
            "NOTES_PATH": test_data_dir / "notes",
            "DATABASE_PATH": test_data_dir / "starpunk.db",
        }
    )
    return app
```

**Update existing tests that use `httpx.post` mock:**

Tests in `TestInitiateLogin` and `TestHandleCallback` need to mock `discover_endpoints()` in addition to `httpx.post`. Example pattern:

```python
@patch("starpunk.auth.discover_endpoints")
@patch("starpunk.auth.httpx.post")
def test_handle_callback_success(self, mock_post, mock_discover, app, db, client):
    """Test successful callback handling"""
    # Mock endpoint discovery
    mock_discover.return_value = {
        'authorization_endpoint': 'https://auth.example.com/authorize',
        'token_endpoint': 'https://auth.example.com/token'
    }

    # Rest of test remains the same...
```

**Update `TestInitiateLogin.test_initiate_login_success`:**

The assertion checking for `indielogin.com` needs to change to check for the mocked endpoint:

```python
@patch("starpunk.auth.discover_endpoints")
def test_initiate_login_success(self, mock_discover, app, db):
    """Test successful login initiation"""
    mock_discover.return_value = {
        'authorization_endpoint': 'https://auth.example.com/authorize',
        'token_endpoint': 'https://auth.example.com/token'
    }

    with app.app_context():
        me_url = "https://example.com"
        auth_url = initiate_login(me_url)

        # Changed: Check for discovered endpoint instead of indielogin.com
        assert "auth.example.com/authorize" in auth_url
        assert "me=https%3A%2F%2Fexample.com" in auth_url
        # ... rest of assertions
```

### New Automated Tests to Add

```python
# tests/test_auth_endpoint_discovery.py

def test_initiate_login_uses_endpoint_discovery(client, mocker):
    """Verify login uses discovered endpoint, not hardcoded"""
    mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
    mock_discover.return_value = {
        'authorization_endpoint': 'https://custom-auth.example.com/authorize',
        'token_endpoint': 'https://custom-auth.example.com/token'
    }

    response = client.post('/auth/login', data={'me': 'https://example.com'})

    assert response.status_code == 302
    assert 'custom-auth.example.com' in response.headers['Location']


def test_callback_uses_discovered_authorization_endpoint(client, mocker):
    """Verify callback uses discovered authorization endpoint (not token endpoint)"""
    mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
    mock_discover.return_value = {
        'authorization_endpoint': 'https://custom-auth.example.com/authorize',
        'token_endpoint': 'https://custom-auth.example.com/token'
    }
    mock_post = mocker.patch('starpunk.auth.httpx.post')
    # Setup state token and mock httpx response
    # Verify code exchange POSTs to authorization_endpoint, not token_endpoint
    pass


def test_discovery_error_shows_user_friendly_message(client, mocker):
    """Verify discovery failures show helpful error"""
    mock_discover = mocker.patch('starpunk.auth.discover_endpoints')
    mock_discover.side_effect = DiscoveryError("No endpoints found")

    response = client.post('/auth/login', data={'me': 'https://example.com'})

    assert response.status_code == 302
    # Should redirect back to login form with flash message


def test_url_normalization_handles_trailing_slash(app, mocker):
    """Verify URL normalization allows trailing slash differences"""
    # ADMIN_ME without trailing slash, auth server returns with trailing slash
    # Should still authenticate successfully
    pass


def test_url_normalization_handles_case_differences(app, mocker):
    """Verify URL normalization is case-insensitive"""
    # ADMIN_ME: https://Example.com, auth server returns: https://example.com
    # Should still authenticate successfully
    pass
```

---

## Rollback Plan

If issues occur after deployment:

1. **Code:** Revert to previous commit
2. **Config:** Re-add INDIELOGIN_URL to .env if needed

---

## Post-Deployment Verification

1. Verify login works with the user's actual profile URL
2. Check logs for "Discovered authorization_endpoint" message
3. Test logout and re-login cycle

---

## Architect Q&A (2025-12-17)

Developer questions answered by the architect prior to implementation:

### Q1: Import Placement
**Q:** Should `normalize_url` import be inside the function or at top level?
**A:** Move to top level with other imports for consistency. The design has been updated.

### Q2: URL Normalization Behavior Change
**Q:** Is the URL normalization change intentional?
**A:** Yes, this is an intentional bugfix. The current exact-match behavior is incorrect per IndieAuth spec. URLs differing only in trailing slashes or case should be considered equivalent for identity purposes. The `normalize_url()` function already exists in `auth_external.py` and is used by `verify_external_token()`.

### Q3: Which Endpoint for Authentication Flow?
**Q:** Should we use token_endpoint or authorization_endpoint?
**A:** Use **authorization_endpoint** for authentication-only flows. Per IndieAuth spec: "the client makes a POST request to the authorization endpoint to verify the authorization code and retrieve the final user profile URL." The design has been corrected.

### Q4: Endpoint Validation Relaxation
**Q:** Is relaxed endpoint validation acceptable?
**A:** Yes. Login requires `authorization_endpoint`, Micropub requires `token_endpoint`. Requiring at least one is correct. If only auth endpoint exists, login works but Micropub fails gracefully (401).

### Q5: Test Update Strategy
**Q:** Remove INDIELOGIN_URL and/or mock discover_endpoints()?
**A:** Both. Remove `INDIELOGIN_URL` from fixtures, add `discover_endpoints()` mocking to existing tests. Detailed guidance added to Testing Requirements section.

### Q6: grant_type Parameter
**Q:** Should we include grant_type in the code exchange?
**A:** No. Authentication-only flows do not include `grant_type`. This parameter is only required when POSTing to the token_endpoint for access tokens. The design has been corrected.

### Q7: Error Message Verbosity
**Q:** Should we simplify the user-facing error message?
**A:** Yes. User-facing message should be simple: "Unable to verify your profile URL. Please check that it's correct and try again." Technical details are logged at ERROR level. The design has been updated.

---

## References

- W3C IndieAuth Specification: https://www.w3.org/TR/indieauth/
- IndieAuth Endpoint Discovery: https://www.w3.org/TR/indieauth/#discovery-by-clients