Gondulf/docs/standards/testing.md

# Testing Standard

## Overview

Testing is mandatory for all code. This project maintains high quality through comprehensive automated testing at multiple levels.

## Testing Philosophy

- Write tests first when possible (TDD approach)
- Test behavior, not implementation
- Keep tests simple and focused
- Fast tests are better tests
- Clear test names that describe what is being tested

## Test Coverage Requirements

### Minimum Coverage
- Overall: 80% code coverage minimum
- Critical paths (auth, token, security): 95% coverage
- New code: 90% coverage required

### Coverage Exclusions
Acceptable to exclude from coverage:
- Logging statements
- Debug utilities
- Main entry points (if trivial)
- Third-party integration boilerplate

## Testing Pyramid

### Unit Tests (70% of tests)
- Test individual functions and methods
- No external dependencies (use mocks)
- Should run in milliseconds
- Located in `tests/unit/`

### Integration Tests (20% of tests)
- Test component interactions
- May use test database
- Test API endpoints with real HTTP requests
- Located in `tests/integration/`

### End-to-End Tests (10% of tests)
- Test complete IndieAuth flows
- Verify compliance with W3C specification
- Include client registration and authentication flows
- Located in `tests/e2e/`

## Test Structure

### Test File Organization
```
tests/
├── unit/
│   ├── test_auth.py
│   ├── test_token.py
│   └── test_client.py
├── integration/
│   ├── test_auth_endpoint.py
│   ├── test_token_endpoint.py
│   └── test_client_registration.py
├── e2e/
│   ├── test_full_auth_flow.py
│   └── test_client_self_registration.py
├── fixtures/
│   └── test_data.py
└── conftest.py  # pytest configuration
```

### Test Naming Convention
- Test files: `test_*.py`
- Test classes: `Test*`
- Test methods: `test_*`

Examples:
```python
# Good test names
def test_token_expires_after_configured_duration():
def test_client_registration_requires_redirect_uri():
def test_authorization_code_is_single_use():

# Bad test names
def test_auth():  # Too vague
def test_1():     # Meaningless
```

## Python Testing Tools

### Core Testing Framework
- **pytest**: Primary testing framework
  - Use fixtures for test setup
  - Use parametrize for testing multiple cases
  - Use markers for test categorization

### Required Testing Libraries
```python
# requirements-test.txt
pytest>=7.0.0
pytest-cov>=4.0.0      # Coverage reporting
pytest-asyncio>=0.20.0  # Async test support
pytest-mock>=3.10.0     # Mocking utilities
freezegun>=1.2.0        # Time mocking
factory-boy>=3.2.0      # Test data factories
responses>=0.22.0       # HTTP response mocking
```

### Test Fixtures
```python
# Example fixture structure
@pytest.fixture
def auth_client():
    """Returns authenticated test client."""

@pytest.fixture
def test_user():
    """Returns test user with domain."""

@pytest.fixture
def registered_client():
    """Returns pre-registered OAuth client."""
```

## Writing Tests

### Test Structure (AAA Pattern)
```python
def test_authorization_code_exchange():
    # Arrange - Set up test data and conditions
    code = generate_auth_code()
    client = create_test_client()

    # Act - Execute the behavior being tested
    token = exchange_code_for_token(code, client)

    # Assert - Verify the outcome
    assert token.is_valid()
    assert token.client_id == client.id
```

### Mocking Guidelines
- Mock external services (HTTP calls, databases in unit tests)
- Don't mock what you're testing
- Prefer dependency injection over patching

### Testing IndieAuth Compliance
Special test suite for W3C specification compliance:
```python
class TestIndieAuthCompliance:
    """Tests that verify W3C IndieAuth specification compliance."""

    def test_authorization_endpoint_supports_response_type_code(self):
    def test_token_endpoint_requires_code_verifier_with_pkce(self):
    def test_client_id_must_be_valid_url(self):
```

## Test Execution

### Running Tests
```bash
# Run all tests
pytest

# Run with coverage
pytest --cov=indieauth --cov-report=html

# Run specific test level
pytest tests/unit/
pytest tests/integration/
pytest tests/e2e/

# Run tests matching pattern
pytest -k "test_token"

# Run with verbose output
pytest -vv
```

### Continuous Integration
All tests must pass before merging:
```yaml
# Example CI configuration
test:
  - pytest tests/unit/ --cov=indieauth
  - pytest tests/integration/
  - pytest tests/e2e/
  - coverage report --fail-under=80
```

## Performance Testing

### Response Time Requirements
- Authorization endpoint: < 200ms
- Token endpoint: < 100ms
- Client registration: < 500ms

### Load Testing
- Support 100 concurrent authentications
- Handle 1000 registered clients
- Token validation: 10,000 requests/minute

## Security Testing

### Required Security Tests
- Test for timing attacks in token validation
- Test rate limiting on all endpoints
- Test PKCE validation
- Test redirect URI validation
- Test for open redirect vulnerabilities
- Test token entropy and uniqueness

## Test Data Management

### Test Database
- Use in-memory SQLite for unit tests
- Use PostgreSQL/MySQL for integration tests (same as production)
- Reset database between test runs
- Use transactions for test isolation

### Test Secrets
- Never use production secrets in tests
- Generate test keys and tokens dynamically
- Use consistent test data for reproducibility

## Documentation of Tests

Each test should be self-documenting:
```python
def test_expired_token_is_rejected():
    """
    Verify that tokens past their expiration time are rejected.

    This prevents replay attacks and ensures tokens have limited lifetime
    as required by OAuth 2.0 security best practices.
    """
```

## Test Maintenance

- Review and update tests when requirements change
- Remove obsolete tests
- Refactor tests that become brittle
- Keep test execution time under 1 minute for unit tests
- Keep full test suite under 5 minutes