Gondulf/docs/standards/testing.md

Testing Standard

Overview

Testing is mandatory for all code. This project maintains high quality through comprehensive automated testing at multiple levels.

Testing Philosophy

• Write tests first when possible (TDD approach)
• Test behavior, not implementation
• Keep tests simple and focused
• Fast tests are better tests
• Clear test names that describe what is being tested

Test Coverage Requirements

Minimum Coverage

• Overall: 80% code coverage minimum
• Critical paths (auth, token, security): 95% coverage
• New code: 90% coverage required

Coverage Exclusions

Acceptable to exclude from coverage:

• Logging statements
• Debug utilities
• Main entry points (if trivial)
• Third-party integration boilerplate

Testing Pyramid

Unit Tests (70% of tests)

• Test individual functions and methods
• No external dependencies (use mocks)
• Should run in milliseconds
• Located in tests/unit/

Integration Tests (20% of tests)

• Test component interactions
• May use test database
• Test API endpoints with real HTTP requests
• Located in tests/integration/

End-to-End Tests (10% of tests)

• Test complete IndieAuth flows
• Verify compliance with W3C specification
• Include client registration and authentication flows
• Located in tests/e2e/

Test Structure

Test File Organization

tests/
├── unit/
│   ├── test_auth.py
│   ├── test_token.py
│   └── test_client.py
├── integration/
│   ├── test_auth_endpoint.py
│   ├── test_token_endpoint.py
│   └── test_client_registration.py
├── e2e/
│   ├── test_full_auth_flow.py
│   └── test_client_self_registration.py
├── fixtures/
│   └── test_data.py
└── conftest.py  # pytest configuration

Test Naming Convention

• Test files: test_*.py
• Test classes: Test*
• Test methods: test_*

Examples:

# Good test names
def test_token_expires_after_configured_duration():
def test_client_registration_requires_redirect_uri():
def test_authorization_code_is_single_use():

# Bad test names
def test_auth():  # Too vague
def test_1():     # Meaningless

Python Testing Tools

Core Testing Framework

• pytest: Primary testing framework
  • Use fixtures for test setup
  • Use parametrize for testing multiple cases
  • Use markers for test categorization

Required Testing Libraries

# requirements-test.txt
pytest>=7.0.0
pytest-cov>=4.0.0      # Coverage reporting
pytest-asyncio>=0.20.0  # Async test support
pytest-mock>=3.10.0     # Mocking utilities
freezegun>=1.2.0        # Time mocking
factory-boy>=3.2.0      # Test data factories
responses>=0.22.0       # HTTP response mocking

Test Fixtures

# Example fixture structure
@pytest.fixture
def auth_client():
    """Returns authenticated test client."""

@pytest.fixture
def test_user():
    """Returns test user with domain."""

@pytest.fixture
def registered_client():
    """Returns pre-registered OAuth client."""

Writing Tests

Test Structure (AAA Pattern)

def test_authorization_code_exchange():
    # Arrange - Set up test data and conditions
    code = generate_auth_code()
    client = create_test_client()

    # Act - Execute the behavior being tested
    token = exchange_code_for_token(code, client)

    # Assert - Verify the outcome
    assert token.is_valid()
    assert token.client_id == client.id

Mocking Guidelines

• Mock external services (HTTP calls, databases in unit tests)
• Don't mock what you're testing
• Prefer dependency injection over patching

Testing IndieAuth Compliance

Special test suite for W3C specification compliance:

class TestIndieAuthCompliance:
    """Tests that verify W3C IndieAuth specification compliance."""

    def test_authorization_endpoint_supports_response_type_code(self):
    def test_token_endpoint_requires_code_verifier_with_pkce(self):
    def test_client_id_must_be_valid_url(self):

Test Execution

Running Tests

# Run all tests
pytest

# Run with coverage
pytest --cov=indieauth --cov-report=html

# Run specific test level
pytest tests/unit/
pytest tests/integration/
pytest tests/e2e/

# Run tests matching pattern
pytest -k "test_token"

# Run with verbose output
pytest -vv

Continuous Integration

All tests must pass before merging:

# Example CI configuration
test:
  - pytest tests/unit/ --cov=indieauth
  - pytest tests/integration/
  - pytest tests/e2e/
  - coverage report --fail-under=80

Performance Testing

Response Time Requirements

• Authorization endpoint: < 200ms
• Token endpoint: < 100ms
• Client registration: < 500ms

Load Testing

• Support 100 concurrent authentications
• Handle 1000 registered clients
• Token validation: 10,000 requests/minute

Security Testing

Required Security Tests

• Test for timing attacks in token validation
• Test rate limiting on all endpoints
• Test PKCE validation
• Test redirect URI validation
• Test for open redirect vulnerabilities
• Test token entropy and uniqueness

Test Data Management

Test Database

• Use in-memory SQLite for unit tests
• Use PostgreSQL/MySQL for integration tests (same as production)
• Reset database between test runs
• Use transactions for test isolation

Test Secrets

• Never use production secrets in tests
• Generate test keys and tokens dynamically
• Use consistent test data for reproducibility

Documentation of Tests

Each test should be self-documenting:

def test_expired_token_is_rejected():
    """
    Verify that tokens past their expiration time are rejected.

    This prevents replay attacks and ensures tokens have limited lifetime
    as required by OAuth 2.0 security best practices.
    """

Test Maintenance

• Review and update tests when requirements change
• Remove obsolete tests
• Refactor tests that become brittle
• Keep test execution time under 1 minute for unit tests
• Keep full test suite under 5 minutes