diff --git a/docs/decisions/ADR-028-micropub-implementation.md b/docs/decisions/ADR-028-micropub-implementation.md new file mode 100644 index 0000000..9fc452b --- /dev/null +++ b/docs/decisions/ADR-028-micropub-implementation.md @@ -0,0 +1,224 @@ +# ADR-028: Micropub Implementation Strategy + +## Status + +Proposed + +## Context + +StarPunk needs a Micropub endpoint to achieve V1 release. Micropub is a W3C standard that allows external clients to create, update, and delete posts on a website. This is a critical IndieWeb building block that enables users to post from various apps and services. + +### Current State +- StarPunk has working IndieAuth authentication (authorization endpoint with PKCE) +- Note CRUD operations exist in `starpunk/notes.py` +- File-based storage with SQLite metadata is implemented +- **Missing**: Micropub endpoint for external posting +- **Missing**: Token endpoint for API authentication + +### Requirements Analysis + +Based on the W3C Micropub specification review, we identified: + +**Minimum Required Features:** +- Bearer token authentication (header or form parameter) +- Create posts via form-encoded requests +- HTTP 201 Created response with Location header +- Proper error responses with JSON error bodies + +**Recommended Features:** +- JSON request support for complex operations +- Update and delete operations +- Query endpoints (config, source, syndicate-to) + +**Optional Features (Not for V1):** +- Media endpoint for file uploads +- Syndication targets +- Complex post types beyond notes + +## Decision + +We will implement a **minimal but complete Micropub server** for V1, focusing on core functionality that enables real-world usage while deferring advanced features. + +### Implementation Approach + +1. **Token Management System** + - New token endpoint (`/auth/token`) for IndieAuth code exchange + - Secure token storage using SHA256 hashing + - 90-day token expiry with scope validation + - Database schema updates for token management + +2. **Micropub Endpoint Architecture** + - Single endpoint (`/micropub`) handling all operations + - Support both form-encoded and JSON content types + - Delegate to existing `notes.py` CRUD functions + - Proper error handling and status codes + +3. **V1 Feature Scope** + - ✅ Create posts (form-encoded and JSON) + - ✅ Query endpoints (config, source) + - ✅ Bearer token authentication + - ✅ Scope-based authorization + - ❌ Media endpoint (deferred) + - ❌ Update/delete operations (deferred) + - ❌ Syndication (deferred) + +### Technology Choices + +| Component | Technology | Rationale | +|-----------|------------|-----------| +| Token Storage | SQLite with SHA256 hashing | Secure, consistent with existing database | +| Token Format | Random URL-safe strings | Simple, secure, no JWT complexity | +| Request Parsing | Flask built-in + custom normalization | Handles both form and JSON naturally | +| Response Format | JSON for errors, headers for success | Follows Micropub spec exactly | + +## Rationale + +### Why Minimal V1 Scope? + +1. **Get to V1 Faster**: Core create functionality enables 90% of use cases +2. **Real Usage Feedback**: Deploy and learn from actual usage patterns +3. **Reduced Complexity**: Fewer edge cases and error conditions +4. **Clear Foundation**: Establish patterns before adding complexity + +### Why Not JWT Tokens? + +1. **Unnecessary Complexity**: JWT adds libraries and complexity +2. **No Distributed Validation**: Single-server system doesn't need it +3. **Simpler Revocation**: Database tokens are easily revoked +4. **Consistent with IndieAuth**: Random tokens match the pattern + +### Why Reuse Existing CRUD? + +1. **Proven Code**: `notes.py` already handles file/database sync +2. **Consistency**: Same validation and error handling +3. **Maintainability**: Single source of truth for note operations +4. **Atomic Operations**: Existing transaction handling + +### Security Considerations + +1. **Token Hashing**: Never store plaintext tokens +2. **Scope Enforcement**: Each operation checks required scopes +3. **HTTPS Required**: Enforce in production configuration +4. **Token Expiry**: 90-day lifetime limits exposure +5. **Single-Use Auth Codes**: Prevent replay attacks + +## Consequences + +### Positive + +✅ **Enables V1 Release**: Removes the last blocker for V1 +✅ **Real IndieWeb Participation**: Can post from standard clients +✅ **Clean Architecture**: Clear separation of concerns +✅ **Extensible Design**: Easy to add features later +✅ **Security First**: Proper token handling from day one + +### Negative + +⚠️ **Limited Initial Features**: No media uploads in V1 +⚠️ **Database Migration Required**: Token schema changes needed +⚠️ **Client Testing Needed**: Must verify with real Micropub clients +⚠️ **Additional Complexity**: New endpoints and token management + +### Neutral + +- **8-10 Day Implementation**: Reasonable timeline for critical feature +- **New Dependencies**: None required (using existing libraries) +- **Documentation Burden**: Must document API for users + +## Implementation Plan + +### Phase 1: Token Infrastructure (Days 1-3) +- Token database schema and migration +- Token generation and storage functions +- Token endpoint for code exchange +- Scope validation helpers + +### Phase 2: Micropub Core (Days 4-7) +- Main endpoint handler +- Property normalization for form/JSON +- Create post functionality +- Error response formatting + +### Phase 3: Queries & Polish (Days 8-10) +- Config and source query endpoints +- Authorization endpoint scope handling +- Discovery headers and links +- Client testing and documentation + +## Alternatives Considered + +### Alternative 1: Full Micropub Implementation +**Rejected**: Too complex for V1, would delay release by weeks + +### Alternative 2: Custom API Instead of Micropub +**Rejected**: Breaks IndieWeb compatibility, requires custom clients + +### Alternative 3: JWT-Based Tokens +**Rejected**: Unnecessary complexity for single-server system + +### Alternative 4: Separate Media Endpoint First +**Rejected**: Not required for text posts, can add later + +## Compliance + +### Standards Compliance +- ✅ W3C Micropub specification +- ✅ IndieAuth specification for tokens +- ✅ OAuth 2.0 Bearer Token usage + +### Project Principles +- ✅ Minimal code (reuses existing CRUD) +- ✅ Standards-first (follows W3C spec) +- ✅ No lock-in (standard protocols) +- ✅ Progressive enhancement (can add features) + +## Risks and Mitigations + +| Risk | Impact | Probability | Mitigation | +|------|--------|-------------|------------| +| Token security breach | High | Low | SHA256 hashing, HTTPS required | +| Client incompatibility | Medium | Medium | Test with 3+ clients before release | +| Scope creep | Medium | High | Strict V1 feature list | +| Performance issues | Low | Low | Simple operations, indexed database | + +## Success Metrics + +1. **Functional Success** + - Posts can be created from Indigenous app + - Posts can be created from Quill + - Token endpoint works with IndieAuth flow + +2. **Performance Targets** + - Post creation < 500ms + - Token validation < 50ms + - Query responses < 200ms + +3. **Security Requirements** + - All tokens hashed in database + - Expired tokens rejected + - Invalid scopes return 403 + +## References + +- [W3C Micropub Specification](https://www.w3.org/TR/micropub/) +- [IndieAuth Specification](https://indieauth.spec.indieweb.org/) +- [OAuth 2.0 Bearer Token Usage](https://tools.ietf.org/html/rfc6750) +- [Micropub Rocks Validator](https://micropub.rocks/) + +## Related ADRs + +- ADR-004: File-based Note Storage (storage layer) +- ADR-019: IndieAuth Implementation (authentication foundation) +- ADR-025: PKCE Authentication (security pattern) + +## Version Impact + +**Version Change**: 0.9.5 → 1.0.0 (V1 Release!) + +This change represents the final feature for V1 release, warranting the major version increment to 1.0.0. + +--- + +**Date**: 2024-11-24 +**Author**: StarPunk Architecture Team +**Status**: Proposed \ No newline at end of file diff --git a/docs/design/micropub-endpoint-design.md b/docs/design/micropub-endpoint-design.md new file mode 100644 index 0000000..bf0f774 --- /dev/null +++ b/docs/design/micropub-endpoint-design.md @@ -0,0 +1,1065 @@ +# Micropub Endpoint Design for StarPunk V1 + +## Executive Summary + +This document defines the architecture and implementation plan for adding Micropub support to StarPunk, the **critical blocker for V1 release**. The Micropub endpoint will enable external clients (like Indigenous, Quill, or Micropublish.net) to create, update, and delete posts on StarPunk installations. + +**Current State:** +- ✅ IndieAuth authentication working (authorization endpoint with PKCE) +- ✅ Note CRUD operations via `notes.py` +- ✅ Markdown file storage with YAML frontmatter +- ✅ SQLite metadata database +- ❌ **No Micropub endpoint** (V1 blocker) +- ❌ **No token endpoint** (required for Micropub auth) + +**Target State:** +- Full Micropub server implementation +- IndieAuth token endpoint for issuing access tokens +- Support for creating, updating, and deleting posts +- Minimal V1 implementation (no media endpoint initially) + +## Architecture Overview + +### System Components + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Micropub Client │ +│ (Indigenous, Quill, etc.) │ +└──────────────────────┬──────────────────────────────────────┘ + │ + │ HTTP Requests with Bearer Token + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ StarPunk Server │ +│ │ +│ ┌────────────────────────────────────────────────────┐ │ +│ │ Micropub Endpoint │ │ +│ │ /micropub (Blueprint) │ │ +│ ├────────────────────────────────────────────────────┤ │ +│ │ • Token Validation (Bearer auth) │ │ +│ │ • Request Parsing (form/JSON) │ │ +│ │ • Action Routing (create/update/delete/query) │ │ +│ │ • Response Formatting │ │ +│ └──────────────┬─────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────┐ │ +│ │ Token Management │ │ +│ │ /auth/token (endpoint) │ │ +│ ├────────────────────────────────────────────────────┤ │ +│ │ • Authorization code exchange │ │ +│ │ • Access token generation │ │ +│ │ • Scope validation │ │ +│ │ • Token storage in database │ │ +│ └──────────────┬─────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────┐ │ +│ │ Existing Notes CRUD │ │ +│ │ starpunk/notes.py │ │ +│ ├────────────────────────────────────────────────────┤ │ +│ │ • create_note() │ │ +│ │ • update_note() │ │ +│ │ • delete_note() │ │ +│ │ • get_note() │ │ +│ └──────────────┬─────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────┐ │ +│ │ Storage Layer │ │ +│ ├──────────────────────┬──────────────────────────────┤ │ +│ │ SQLite Database │ Markdown Files │ │ +│ │ • notes metadata │ /data/notes/YYYY/MM/ │ │ +│ │ • tokens │ • YAML frontmatter │ │ +│ │ • sessions │ • Markdown content │ │ +│ └──────────────────────┴──────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +## Detailed Component Design + +### 1. Token Endpoint (`/auth/token`) + +The token endpoint is required for IndieAuth authorization code flow, converting authorization codes into access tokens for API access. + +#### Endpoint Specification + +``` +POST /auth/token +Content-Type: application/x-www-form-urlencoded + +grant_type=authorization_code& +code={authorization_code}& +client_id={client_url}& +redirect_uri={redirect_url}& +code_verifier={pkce_verifier} +``` + +#### Response Format + +Success (200 OK): +```json +{ + "access_token": "eyJ0eXAiOiJKV1QiLCJhbGc...", + "token_type": "Bearer", + "scope": "create update delete", + "me": "https://example.com/" +} +``` + +Error (400 Bad Request): +```json +{ + "error": "invalid_grant", + "error_description": "The authorization code is invalid or expired" +} +``` + +#### Implementation Details + +```python +# New file: starpunk/tokens.py + +def generate_access_token() -> str: + """Generate cryptographically secure access token""" + return secrets.token_urlsafe(32) + +def create_token(me: str, client_id: str, scope: str) -> str: + """ + Create and store access token in database + + Args: + me: User's identity URL + client_id: Client application URL + scope: Space-separated list of scopes + + Returns: + Access token string + """ + token = generate_access_token() + token_hash = hashlib.sha256(token.encode()).hexdigest() + + db = get_db() + db.execute(""" + INSERT INTO tokens (token_hash, me, client_id, scope, created_at, expires_at) + VALUES (?, ?, ?, ?, datetime('now'), datetime('now', '+90 days')) + """, (token_hash, me, client_id, scope)) + db.commit() + + return token + +def verify_token(token: str) -> Optional[dict]: + """ + Verify access token and return token info + + Returns dict with: me, client_id, scope, or None if invalid + """ + token_hash = hashlib.sha256(token.encode()).hexdigest() + + db = get_db() + row = db.execute(""" + SELECT me, client_id, scope + FROM tokens + WHERE token_hash = ? + AND expires_at > datetime('now') + """, (token_hash,)).fetchone() + + if row: + return dict(row) + return None +``` + +#### Database Schema Update + +```sql +-- Update existing tokens table to use token_hash +ALTER TABLE tokens RENAME TO tokens_old; + +CREATE TABLE tokens ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + token_hash TEXT UNIQUE NOT NULL, -- SHA256 hash of token + me TEXT NOT NULL, + client_id TEXT, + scope TEXT, + created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, + expires_at TIMESTAMP, + last_used_at TIMESTAMP, + revoked_at TIMESTAMP +); + +CREATE INDEX idx_tokens_hash ON tokens(token_hash); +CREATE INDEX idx_tokens_me ON tokens(me); +CREATE INDEX idx_tokens_expires ON tokens(expires_at); +``` + +### 2. Micropub Endpoint (`/micropub`) + +The main Micropub endpoint handles all post operations. + +#### Route Structure + +```python +# New file: starpunk/routes/micropub.py + +from flask import Blueprint, request, jsonify, current_app +from starpunk.tokens import verify_token +from starpunk.micropub import ( + handle_create, + handle_update, + handle_delete, + handle_query, + MicropubError +) + +bp = Blueprint("micropub", __name__) + +@bp.route("/micropub", methods=["GET", "POST"]) +def micropub_endpoint(): + """Main Micropub endpoint""" + + # Extract token from Authorization header or form + token = extract_bearer_token(request) + if not token: + return error_response("unauthorized", "No access token provided"), 401 + + # Verify token + token_info = verify_token(token) + if not token_info: + return error_response("unauthorized", "Invalid access token"), 401 + + # Handle query endpoints (GET requests) + if request.method == "GET": + return handle_query(request.args, token_info) + + # Handle action endpoints (POST requests) + content_type = request.headers.get("Content-Type", "") + + if "application/json" in content_type: + data = request.get_json() + action = data.get("action", "create") + else: + # Form-encoded or multipart + data = request.form.to_dict(flat=False) + action = data.get("action", ["create"])[0] + + try: + if action == "create": + return handle_create(data, token_info) + elif action == "update": + return handle_update(data, token_info) + elif action == "delete": + return handle_delete(data, token_info) + else: + return error_response("invalid_request", f"Unknown action: {action}"), 400 + except MicropubError as e: + return error_response("invalid_request", str(e)), 400 +``` + +#### Content Creation Handler + +```python +# New file: starpunk/micropub.py + +def handle_create(data: dict, token_info: dict) -> tuple: + """ + Handle post creation + + Args: + data: Micropub request data (form or JSON) + token_info: Authenticated user info + + Returns: + Response tuple (body, status, headers) + """ + # Check scope + if "create" not in token_info.get("scope", ""): + return error_response("insufficient_scope", "Token lacks create scope"), 403 + + # Extract properties + properties = normalize_properties(data) + + # Map Micropub properties to StarPunk note format + title = extract_title(properties) + content = extract_content(properties) + tags = properties.get("category", []) + + # Create note using existing CRUD + from starpunk.notes import create_note + + try: + note = create_note( + title=title, + content=content, + tags=tags, + author=token_info["me"], + published=True # Micropub posts are published by default + ) + + # Build permalink URL + permalink = f"{current_app.config['SITE_URL']}/notes/{note.slug}" + + # Return 201 Created with Location header + return "", 201, {"Location": permalink} + + except Exception as e: + current_app.logger.error(f"Failed to create note via Micropub: {e}") + return error_response("server_error", "Failed to create post"), 500 + +def normalize_properties(data: dict) -> dict: + """ + Normalize Micropub properties from both form and JSON formats + + Handles: + - Form: property[]=value arrays + - JSON: {"properties": {"property": ["value"]}} + """ + if "properties" in data: + # JSON format + return data["properties"] + + # Form format - convert to properties dict + properties = {} + for key, value in data.items(): + if key.startswith("mp-") or key in ["action", "url", "access_token"]: + continue # Skip reserved properties + + # Handle array notation: property[] -> property + clean_key = key.rstrip("[]") + + # Ensure value is always a list + if not isinstance(value, list): + value = [value] + + properties[clean_key] = value + + return properties +``` + +#### Update and Delete Handlers + +```python +def handle_update(data: dict, token_info: dict) -> tuple: + """Handle post updates""" + if "update" not in token_info.get("scope", ""): + return error_response("insufficient_scope", "Token lacks update scope"), 403 + + url = data.get("url") + if not url: + return error_response("invalid_request", "No URL provided"), 400 + + # Extract slug from URL + slug = extract_slug_from_url(url) + + # Get existing note + from starpunk.notes import get_note, update_note + note = get_note(slug) + + if not note: + return error_response("invalid_request", "Post not found"), 400 + + # Apply updates based on operation type + if "replace" in data: + # Replace properties + for prop, values in data["replace"].items(): + apply_property_replace(note, prop, values) + + if "add" in data: + # Add to properties + for prop, values in data["add"].items(): + apply_property_add(note, prop, values) + + if "delete" in data: + # Delete properties or values + for prop, values in data["delete"].items(): + apply_property_delete(note, prop, values) + + # Save changes + update_note(note) + + return "", 204 # No Content + +def handle_delete(data: dict, token_info: dict) -> tuple: + """Handle post deletion""" + if "delete" not in token_info.get("scope", ""): + return error_response("insufficient_scope", "Token lacks delete scope"), 403 + + url = data.get("url") + if not url: + return error_response("invalid_request", "No URL provided"), 400 + + # Extract slug and delete + slug = extract_slug_from_url(url) + from starpunk.notes import delete_note + + try: + delete_note(slug, soft=True) # Soft delete for safety + return "", 204 + except NoteNotFoundError: + return error_response("invalid_request", "Post not found"), 400 +``` + +### 3. Query Endpoints + +Micropub defines several query endpoints for client discovery: + +```python +def handle_query(args: dict, token_info: dict) -> tuple: + """Handle Micropub query endpoints""" + q = args.get("q") + + if q == "config": + # Return server configuration + config = { + "media-endpoint": None, # No media endpoint in V1 + "syndicate-to": [], # No syndication targets in V1 + "post-types": [ + { + "type": "note", + "name": "Note", + "properties": ["content"] + } + ] + } + return jsonify(config), 200 + + elif q == "source": + # Return source of a specific post + url = args.get("url") + if not url: + return error_response("invalid_request", "No URL provided"), 400 + + slug = extract_slug_from_url(url) + from starpunk.notes import get_note + + note = get_note(slug) + if not note: + return error_response("invalid_request", "Post not found"), 400 + + # Convert note to Micropub format + mf2 = { + "type": ["h-entry"], + "properties": { + "content": [note.content], + "name": [note.title] if note.title else [], + "category": note.tags if note.tags else [], + "published": [note.created_at.isoformat()], + "url": [f"{current_app.config['SITE_URL']}/notes/{note.slug}"] + } + } + + return jsonify(mf2), 200 + + elif q == "syndicate-to": + # Return syndication targets (none for V1) + return jsonify({"syndicate-to": []}), 200 + + else: + return error_response("invalid_request", f"Unknown query: {q}"), 400 +``` + +### 4. Authorization Flow Integration + +The complete authorization flow for Micropub: + +``` +┌──────────┐ +│ Client │ +└─────┬────┘ + │ + │ 1. GET /auth/authorization? + │ response_type=code& + │ client_id=https://client.example& + │ redirect_uri=https://client.example/callback& + │ state=1234567890& + │ scope=create+update+delete& + │ code_challenge=XXXXX& + │ code_challenge_method=S256 + ▼ +┌──────────────┐ +│ StarPunk │ +│ (Authorize) │ +└─────┬────────┘ + │ + │ 2. Redirect to IndieLogin.com for authentication + │ (existing PKCE flow) + │ + │ 3. After successful auth, redirect back with code + │ + ▼ +┌──────────┐ +│ Client │ +└─────┬────┘ + │ + │ 4. POST /auth/token + │ grant_type=authorization_code& + │ code=XXXXX& + │ client_id=https://client.example& + │ redirect_uri=https://client.example/callback& + │ code_verifier=XXXXX + ▼ +┌──────────────┐ +│ StarPunk │ +│ (Token) │ +└─────┬────────┘ + │ + │ 5. Return access token + │ { + │ "access_token": "XXXXX", + │ "token_type": "Bearer", + │ "scope": "create update delete", + │ "me": "https://user.example" + │ } + ▼ +┌──────────┐ +│ Client │ +└─────┬────┘ + │ + │ 6. POST /micropub + │ Authorization: Bearer XXXXX + │ Content-Type: application/x-www-form-urlencoded + │ + │ h=entry&content=Hello+world + ▼ +┌──────────────┐ +│ StarPunk │ +│ (Micropub) │ +└──────────────┘ +``` + +## Implementation Plan + +### Phase 1: Token Management (2-3 days) + +1. **Database Migration** + - Update tokens table schema + - Add indexes for performance + - Create migration script + +2. **Token Module** (`starpunk/tokens.py`) + - Token generation functions + - Token storage with hashing + - Token verification + - Scope validation helpers + +3. **Token Endpoint** (`/auth/token`) + - Authorization code validation + - PKCE verification + - Token generation + - Response formatting + +4. **Testing** + - Unit tests for token functions + - Integration tests for token endpoint + - Security tests (token expiry, revocation) + +### Phase 2: Micropub Core (3-4 days) + +1. **Micropub Module** (`starpunk/micropub.py`) + - Property normalization + - Content extraction + - Note format conversion + - Error handling + +2. **Micropub Routes** (`starpunk/routes/micropub.py`) + - Main endpoint handler + - Bearer token extraction + - Content-type handling + - Action routing + +3. **Create Handler** + - Form-encoded parsing + - JSON parsing + - Note creation via CRUD + - Location header generation + +4. **Testing** + - Create post via form-encoded + - Create post via JSON + - Invalid token handling + - Scope validation + +### Phase 3: Extended Operations (2-3 days) + +1. **Update Handler** + - Replace operations + - Add operations + - Delete operations + - Property mapping + +2. **Delete Handler** + - URL parsing + - Soft deletion + - Response handling + +3. **Query Endpoints** + - Config endpoint + - Source endpoint + - Syndicate-to endpoint + +4. **Testing** + - Update operations + - Delete operations + - Query responses + +### Phase 4: Integration & Polish (1-2 days) + +1. **Authorization Endpoint Updates** + - Add scope parameter handling + - Store requested scopes with auth state + - Pass scopes to token creation + +2. **Discovery Headers** + - Add `Link: ; rel="micropub"` header + - Add `Link: ; rel="authorization_endpoint"` header + - Add `Link: ; rel="token_endpoint"` header + +3. **Error Handling** + - Consistent error responses + - Proper HTTP status codes + - Detailed error descriptions + +4. **Documentation** + - API documentation + - Client configuration guide + - Testing with real clients + +## V1 Minimal Feature Set + +### In Scope for V1 + +✅ **Required for Micropub Compliance:** +- Bearer token authentication +- Create posts (form-encoded) +- Create posts (JSON) +- Return 201 Created with Location header +- Configuration query endpoint +- Source query endpoint + +✅ **Required for Functionality:** +- Token endpoint for IndieAuth +- Token storage and validation +- Scope enforcement +- Integration with existing notes CRUD + +### Out of Scope for V1 (Future) + +❌ **Nice to Have but Not Required:** +- Media endpoint (file uploads) +- Photo/video properties (without media endpoint) +- Syndication targets +- Complex post types (articles, replies, etc.) +- Batch operations +- Websub notifications +- Token introspection endpoint +- Token revocation endpoint + +## Security Considerations + +### Token Security + +1. **Storage** + - Tokens stored as SHA256 hashes + - Original token never logged + - Secure random generation (32 bytes) + +2. **Validation** + - Check token exists + - Check not expired (90 days) + - Check not revoked + - Verify scope for operation + +3. **Transport** + - Require HTTPS in production + - Support both header and form parameter + - Never include in URLs + +### Scope Enforcement + +```python +SCOPE_REQUIREMENTS = { + "create": ["create"], + "update": ["update"], + "delete": ["delete"], + "query": [] # Queries don't require specific scopes +} + +def check_scope(required: str, granted: str) -> bool: + """Check if granted scopes include required scope""" + granted_scopes = set(granted.split()) + return required in granted_scopes +``` + +### Input Validation + +1. **Content Validation** + - Sanitize HTML content + - Validate URLs + - Limit content size + - Check required properties + +2. **URL Validation** + - Verify URL belongs to this site + - Extract valid slugs + - Prevent directory traversal + +3. **Rate Limiting** + - Limit requests per token + - Limit failed authentication attempts + - Implement exponential backoff + +## Testing Strategy + +### Unit Tests + +```python +# tests/test_micropub.py + +def test_normalize_properties_form(): + """Test form-encoded property normalization""" + data = { + "content": ["Hello world"], + "category[]": ["tag1", "tag2"], + "mp-slug": ["my-post"] + } + + properties = normalize_properties(data) + + assert properties["content"] == ["Hello world"] + assert properties["category"] == ["tag1", "tag2"] + assert "mp-slug" not in properties + +def test_normalize_properties_json(): + """Test JSON property normalization""" + data = { + "type": ["h-entry"], + "properties": { + "content": ["Hello world"], + "category": ["tag1", "tag2"] + } + } + + properties = normalize_properties(data) + + assert properties["content"] == ["Hello world"] + assert properties["category"] == ["tag1", "tag2"] + +def test_bearer_token_extraction(): + """Test token extraction from header and form""" + # Test header + request = Mock(headers={"Authorization": "Bearer abc123"}) + token = extract_bearer_token(request) + assert token == "abc123" + + # Test form parameter + request = Mock(headers={}, form={"access_token": "xyz789"}) + token = extract_bearer_token(request) + assert token == "xyz789" +``` + +### Integration Tests + +```python +# tests/test_micropub_integration.py + +def test_create_post_form_encoded(client, auth_token): + """Test creating post with form-encoded request""" + response = client.post("/micropub", + headers={"Authorization": f"Bearer {auth_token}"}, + data={ + "h": "entry", + "content": "Test post", + "category[]": ["test", "micropub"] + } + ) + + assert response.status_code == 201 + assert "Location" in response.headers + assert "/notes/" in response.headers["Location"] + +def test_create_post_json(client, auth_token): + """Test creating post with JSON request""" + response = client.post("/micropub", + headers={ + "Authorization": f"Bearer {auth_token}", + "Content-Type": "application/json" + }, + json={ + "type": ["h-entry"], + "properties": { + "content": ["Test post"], + "category": ["test", "micropub"] + } + } + ) + + assert response.status_code == 201 + assert "Location" in response.headers + +def test_unauthorized_request(client): + """Test request without token""" + response = client.post("/micropub", + data={"h": "entry", "content": "Test"} + ) + + assert response.status_code == 401 + assert response.json["error"] == "unauthorized" + +def test_insufficient_scope(client, read_only_token): + """Test request with insufficient scope""" + response = client.post("/micropub", + headers={"Authorization": f"Bearer {read_only_token}"}, + data={"h": "entry", "content": "Test"} + ) + + assert response.status_code == 403 + assert response.json["error"] == "insufficient_scope" +``` + +### Client Testing + +Test with real Micropub clients: + +1. **Indigenous** (iOS/Android) + - Configure with StarPunk URLs + - Test post creation + - Test photo uploads (when media endpoint added) + +2. **Quill** (Web) + - Test at https://quill.p3k.io + - Enter StarPunk domain + - Complete authentication + - Create test posts + +3. **Micropublish.net** (Web) + - Test at https://micropublish.net + - Full-featured client + - Test all operations + +## Migration Path + +### Database Migrations + +```sql +-- Migration: Add Micropub support +-- Version: 0.10.0 + +-- 1. Update tokens table for better security +ALTER TABLE tokens RENAME TO tokens_old; + +CREATE TABLE tokens ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + token_hash TEXT UNIQUE NOT NULL, + me TEXT NOT NULL, + client_id TEXT, + scope TEXT, + created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, + expires_at TIMESTAMP, + last_used_at TIMESTAMP, + revoked_at TIMESTAMP +); + +-- Migrate existing tokens (if any) +-- Note: Existing tokens will be invalid as we don't have original values + +DROP TABLE tokens_old; + +-- 2. Add authorization codes table for token exchange +CREATE TABLE IF NOT EXISTS authorization_codes ( + code TEXT PRIMARY KEY, + me TEXT NOT NULL, + client_id TEXT NOT NULL, + redirect_uri TEXT NOT NULL, + scope TEXT, + code_challenge TEXT, + code_challenge_method TEXT, + created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, + expires_at TIMESTAMP NOT NULL, + used_at TIMESTAMP +); + +CREATE INDEX idx_auth_codes_expires ON authorization_codes(expires_at); +``` + +### Configuration Updates + +```python +# config.py additions + +# Micropub settings +MICROPUB_ENDPOINT = "/micropub" +TOKEN_ENDPOINT = "/auth/token" +TOKEN_EXPIRY_DAYS = 90 +AUTHORIZATION_CODE_EXPIRY_MINUTES = 10 + +# Supported scopes +SUPPORTED_SCOPES = ["create", "update", "delete", "read"] +DEFAULT_SCOPE = "create" +``` + +## Effort Estimates + +### Total Effort: 8-10 days + +| Phase | Task | Effort | Priority | +|-------|------|--------|----------| +| 1 | Token Management | 2-3 days | Critical | +| 2 | Micropub Core | 3-4 days | Critical | +| 3 | Extended Operations | 2-3 days | High | +| 4 | Integration & Polish | 1-2 days | High | + +### Resource Requirements + +- **Developer**: 1 person full-time for 2 weeks +- **Testing**: Access to Micropub clients +- **Documentation**: 1 day included in estimates + +### Risk Factors + +1. **Token Security** (High Impact, Low Probability) + - Mitigation: Follow security best practices, hash tokens + +2. **Client Compatibility** (Medium Impact, Medium Probability) + - Mitigation: Test with multiple clients early + +3. **Scope Creep** (Medium Impact, High Probability) + - Mitigation: Strict V1 feature set, defer nice-to-haves + +## Success Criteria + +### Functional Requirements + +✅ Micropub endpoint responds to POST requests +✅ Bearer token authentication works +✅ Posts can be created via form-encoded requests +✅ Posts can be created via JSON requests +✅ Location header returned on creation +✅ Query endpoints return valid responses +✅ Token endpoint exchanges codes for tokens +✅ Scopes are enforced correctly + +### Performance Requirements + +✅ Post creation < 500ms +✅ Token validation < 50ms +✅ Query responses < 200ms + +### Security Requirements + +✅ Tokens stored as hashes +✅ Expired tokens rejected +✅ Invalid tokens return 401 +✅ Insufficient scope returns 403 +✅ HTTPS required in production + +### Compatibility Requirements + +✅ Works with Indigenous app +✅ Works with Quill web client +✅ Works with Micropublish.net +✅ Passes Micropub Rocks validator + +## Conclusion + +The Micropub implementation is straightforward given StarPunk's existing architecture: + +1. **Existing CRUD operations** in `notes.py` handle the storage +2. **Existing authentication** provides the foundation for tokens +3. **Clear specification** from W3C defines exact requirements +4. **Minimal V1 scope** focuses on core functionality + +The main work is: +- Building the token endpoint for IndieAuth +- Creating the Micropub request handler +- Mapping between Micropub and StarPunk formats +- Proper error handling and responses + +With 8-10 days of focused development, StarPunk will have a fully functional Micropub endpoint, unblocking the V1 release. + +## Appendix: Example Requests + +### Creating a Note (Form-Encoded) + +```http +POST /micropub HTTP/1.1 +Host: starpunk.example.com +Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc... +Content-Type: application/x-www-form-urlencoded + +h=entry& +content=Just+had+coffee+at+the+new+place+downtown.+Really+good!& +category[]=coffee& +category[]=portland +``` + +### Creating a Note (JSON) + +```http +POST /micropub HTTP/1.1 +Host: starpunk.example.com +Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc... +Content-Type: application/json + +{ + "type": ["h-entry"], + "properties": { + "content": ["Just had coffee at the new place downtown. Really good!"], + "category": ["coffee", "portland"] + } +} +``` + +### Updating a Note + +```http +POST /micropub HTTP/1.1 +Host: starpunk.example.com +Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc... +Content-Type: application/json + +{ + "action": "update", + "url": "https://starpunk.example.com/notes/2024/11/coffee-downtown", + "add": { + "category": ["recommended"] + }, + "replace": { + "content": ["Just had coffee at the new place downtown. Really good! Will definitely return."] + } +} +``` + +### Deleting a Note + +```http +POST /micropub HTTP/1.1 +Host: starpunk.example.com +Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc... +Content-Type: application/x-www-form-urlencoded + +action=delete& +url=https://starpunk.example.com/notes/2024/11/test-post +``` + +### Querying Configuration + +```http +GET /micropub?q=config HTTP/1.1 +Host: starpunk.example.com +Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc... +``` + +Response: +```json +{ + "media-endpoint": null, + "syndicate-to": [], + "post-types": [ + { + "type": "note", + "name": "Note", + "properties": ["content"] + } + ] +} +``` \ No newline at end of file