Implements tag/category system backend following microformats2 p-category specification. Database changes: - Migration 008: Add tags and note_tags tables - Normalized tag storage (case-insensitive lookup, display name preserved) - Indexes for performance New module: - starpunk/tags.py: Tag management functions - normalize_tag: Normalize tag strings - get_or_create_tag: Get or create tag records - add_tags_to_note: Associate tags with notes (replaces existing) - get_note_tags: Retrieve note tags (alphabetically ordered) - get_tag_by_name: Lookup tag by normalized name - get_notes_by_tag: Get all notes with specific tag - parse_tag_input: Parse comma-separated tag input Model updates: - Note.tags property (lazy-loaded, prefer pre-loading in routes) - Note.to_dict() add include_tags parameter CRUD updates: - create_note() accepts tags parameter - update_note() accepts tags parameter (None = no change, [] = remove all) Micropub integration: - Pass tags to create_note() (tags already extracted by extract_tags()) - Return tags in q=source response Per design doc: docs/design/v1.3.0/microformats-tags-design.md Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
1087 lines
33 KiB
Markdown
1087 lines
33 KiB
Markdown
# Micropub Endpoint Design for StarPunk V1
|
|
|
|
## Executive Summary
|
|
|
|
This document defines the architecture and implementation plan for adding Micropub support to StarPunk, the **critical blocker for V1 release**. The Micropub endpoint will enable external clients (like Indigenous, Quill, or Micropublish.net) to create posts on StarPunk installations.
|
|
|
|
**Current State:**
|
|
- ✅ IndieAuth authentication working (authorization endpoint with PKCE)
|
|
- ✅ Note CRUD operations via `notes.py`
|
|
- ✅ Markdown file storage with YAML frontmatter
|
|
- ✅ SQLite metadata database
|
|
- ❌ **No Micropub endpoint** (V1 blocker)
|
|
- ❌ **No token endpoint** (required for Micropub auth)
|
|
|
|
**Target State for V1:**
|
|
- Micropub server implementation (create-only for V1)
|
|
- IndieAuth token endpoint for issuing access tokens
|
|
- Support for creating posts via form-encoded and JSON
|
|
- Minimal V1 implementation (no media endpoint, no update/delete)
|
|
|
|
## Architecture Overview
|
|
|
|
### System Components
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Micropub Client │
|
|
│ (Indigenous, Quill, etc.) │
|
|
└──────────────────────┬──────────────────────────────────────┘
|
|
│
|
|
│ HTTP Requests with Bearer Token
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ StarPunk Server │
|
|
│ │
|
|
│ ┌────────────────────────────────────────────────────┐ │
|
|
│ │ Micropub Endpoint │ │
|
|
│ │ /micropub (Blueprint) │ │
|
|
│ ├────────────────────────────────────────────────────┤ │
|
|
│ │ • Token Validation (Bearer auth) │ │
|
|
│ │ • Request Parsing (form/JSON) │ │
|
|
│ │ • Action Routing (create/update/delete/query) │ │
|
|
│ │ • Response Formatting │ │
|
|
│ └──────────────┬─────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌────────────────────────────────────────────────────┐ │
|
|
│ │ Token Management │ │
|
|
│ │ /auth/token (endpoint) │ │
|
|
│ ├────────────────────────────────────────────────────┤ │
|
|
│ │ • Authorization code exchange │ │
|
|
│ │ • Access token generation │ │
|
|
│ │ • Scope validation │ │
|
|
│ │ • Token storage in database │ │
|
|
│ └──────────────┬─────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌────────────────────────────────────────────────────┐ │
|
|
│ │ Existing Notes CRUD │ │
|
|
│ │ starpunk/notes.py │ │
|
|
│ ├────────────────────────────────────────────────────┤ │
|
|
│ │ • create_note() │ │
|
|
│ │ • update_note() │ │
|
|
│ │ • delete_note() │ │
|
|
│ │ • get_note() │ │
|
|
│ └──────────────┬─────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌────────────────────────────────────────────────────┐ │
|
|
│ │ Storage Layer │ │
|
|
│ ├──────────────────────┬──────────────────────────────┤ │
|
|
│ │ SQLite Database │ Markdown Files │ │
|
|
│ │ • notes metadata │ /data/notes/YYYY/MM/ │ │
|
|
│ │ • tokens │ • YAML frontmatter │ │
|
|
│ │ • sessions │ • Markdown content │ │
|
|
│ └──────────────────────┴──────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Detailed Component Design
|
|
|
|
### 1. Token Endpoint (`/auth/token`)
|
|
|
|
The token endpoint is required for IndieAuth authorization code flow, converting authorization codes into access tokens for API access.
|
|
|
|
#### Endpoint Specification
|
|
|
|
```
|
|
POST /auth/token
|
|
Content-Type: application/x-www-form-urlencoded
|
|
|
|
grant_type=authorization_code&
|
|
code={authorization_code}&
|
|
client_id={client_url}&
|
|
redirect_uri={redirect_url}&
|
|
me={user_profile_url}&
|
|
code_verifier={pkce_verifier} # Optional (if PKCE was used)
|
|
```
|
|
|
|
#### Response Format
|
|
|
|
Success (200 OK):
|
|
```json
|
|
{
|
|
"access_token": "eyJ0eXAiOiJKV1QiLCJhbGc...",
|
|
"token_type": "Bearer",
|
|
"scope": "create update delete",
|
|
"me": "https://example.com/"
|
|
}
|
|
```
|
|
|
|
Error (400 Bad Request):
|
|
```json
|
|
{
|
|
"error": "invalid_grant",
|
|
"error_description": "The authorization code is invalid or expired"
|
|
}
|
|
```
|
|
|
|
#### Implementation Details
|
|
|
|
```python
|
|
# New file: starpunk/tokens.py
|
|
|
|
def generate_access_token() -> str:
|
|
"""Generate cryptographically secure access token"""
|
|
return secrets.token_urlsafe(32)
|
|
|
|
def create_token(me: str, client_id: str, scope: str) -> str:
|
|
"""
|
|
Create and store access token in database
|
|
|
|
Args:
|
|
me: User's identity URL
|
|
client_id: Client application URL
|
|
scope: Space-separated list of scopes
|
|
|
|
Returns:
|
|
Access token string
|
|
"""
|
|
token = generate_access_token()
|
|
token_hash = hashlib.sha256(token.encode()).hexdigest()
|
|
|
|
db = get_db()
|
|
db.execute("""
|
|
INSERT INTO tokens (token_hash, me, client_id, scope, created_at, expires_at)
|
|
VALUES (?, ?, ?, ?, datetime('now'), datetime('now', '+90 days'))
|
|
""", (token_hash, me, client_id, scope))
|
|
db.commit()
|
|
|
|
return token
|
|
|
|
def verify_token(token: str) -> Optional[dict]:
|
|
"""
|
|
Verify access token and return token info
|
|
|
|
Returns dict with: me, client_id, scope, or None if invalid
|
|
"""
|
|
token_hash = hashlib.sha256(token.encode()).hexdigest()
|
|
|
|
db = get_db()
|
|
row = db.execute("""
|
|
SELECT me, client_id, scope
|
|
FROM tokens
|
|
WHERE token_hash = ?
|
|
AND expires_at > datetime('now')
|
|
""", (token_hash,)).fetchone()
|
|
|
|
if row:
|
|
return dict(row)
|
|
return None
|
|
```
|
|
|
|
#### Database Schema Update
|
|
|
|
```sql
|
|
-- SECURITY FIX: Migrate to hashed token storage
|
|
-- Note: This will invalidate all existing tokens
|
|
|
|
-- Step 1: Create new secure tokens table
|
|
CREATE TABLE tokens_secure (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
token_hash TEXT UNIQUE NOT NULL, -- SHA256 hash of token
|
|
me TEXT NOT NULL,
|
|
client_id TEXT,
|
|
scope TEXT DEFAULT 'create', -- Default scope for V1
|
|
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
expires_at TIMESTAMP NOT NULL,
|
|
last_used_at TIMESTAMP,
|
|
revoked_at TIMESTAMP
|
|
);
|
|
|
|
-- Step 2: Drop old insecure table (invalidates existing tokens)
|
|
DROP TABLE IF EXISTS tokens;
|
|
|
|
-- Step 3: Rename to final name
|
|
ALTER TABLE tokens_secure RENAME TO tokens;
|
|
|
|
-- Step 4: Create indexes
|
|
CREATE INDEX idx_tokens_hash ON tokens(token_hash);
|
|
CREATE INDEX idx_tokens_me ON tokens(me);
|
|
CREATE INDEX idx_tokens_expires ON tokens(expires_at);
|
|
|
|
-- Step 5: Create authorization_codes table
|
|
CREATE TABLE authorization_codes (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
code_hash TEXT UNIQUE NOT NULL, -- SHA256 hash for security
|
|
me TEXT NOT NULL,
|
|
client_id TEXT NOT NULL,
|
|
redirect_uri TEXT NOT NULL,
|
|
scope TEXT,
|
|
state TEXT,
|
|
code_challenge TEXT, -- Optional PKCE
|
|
code_challenge_method TEXT,
|
|
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
expires_at TIMESTAMP NOT NULL,
|
|
used_at TIMESTAMP -- Prevent replay attacks
|
|
);
|
|
|
|
CREATE INDEX idx_auth_codes_hash ON authorization_codes(code_hash);
|
|
CREATE INDEX idx_auth_codes_expires ON authorization_codes(expires_at);
|
|
```
|
|
|
|
### 2. Micropub Endpoint (`/micropub`)
|
|
|
|
The main Micropub endpoint handles all post operations.
|
|
|
|
#### Route Structure
|
|
|
|
```python
|
|
# New file: starpunk/routes/micropub.py
|
|
|
|
from flask import Blueprint, request, jsonify, current_app
|
|
from starpunk.tokens import verify_token
|
|
from starpunk.micropub import (
|
|
handle_create,
|
|
handle_update,
|
|
handle_delete,
|
|
handle_query,
|
|
MicropubError
|
|
)
|
|
|
|
bp = Blueprint("micropub", __name__)
|
|
|
|
@bp.route("/micropub", methods=["GET", "POST"])
|
|
def micropub_endpoint():
|
|
"""Main Micropub endpoint"""
|
|
|
|
# Extract token from Authorization header or form
|
|
token = extract_bearer_token(request)
|
|
if not token:
|
|
return error_response("unauthorized", "No access token provided"), 401
|
|
|
|
# Verify token
|
|
token_info = verify_token(token)
|
|
if not token_info:
|
|
return error_response("unauthorized", "Invalid access token"), 401
|
|
|
|
# Handle query endpoints (GET requests)
|
|
if request.method == "GET":
|
|
return handle_query(request.args, token_info)
|
|
|
|
# Handle action endpoints (POST requests)
|
|
content_type = request.headers.get("Content-Type", "")
|
|
|
|
if "application/json" in content_type:
|
|
data = request.get_json()
|
|
action = data.get("action", "create")
|
|
else:
|
|
# Form-encoded or multipart
|
|
data = request.form.to_dict(flat=False)
|
|
action = data.get("action", ["create"])[0]
|
|
|
|
try:
|
|
if action == "create":
|
|
return handle_create(data, token_info)
|
|
elif action == "update":
|
|
# V1: Update not supported
|
|
return error_response("invalid_request", "Update action not supported in V1"), 400
|
|
elif action == "delete":
|
|
# V1: Delete not supported
|
|
return error_response("invalid_request", "Delete action not supported in V1"), 400
|
|
else:
|
|
return error_response("invalid_request", f"Unknown action: {action}"), 400
|
|
except MicropubError as e:
|
|
return error_response("invalid_request", str(e)), 400
|
|
```
|
|
|
|
#### Content Creation Handler
|
|
|
|
```python
|
|
# New file: starpunk/micropub.py
|
|
|
|
def handle_create(data: dict, token_info: dict) -> tuple:
|
|
"""
|
|
Handle post creation
|
|
|
|
Args:
|
|
data: Micropub request data (form or JSON)
|
|
token_info: Authenticated user info
|
|
|
|
Returns:
|
|
Response tuple (body, status, headers)
|
|
"""
|
|
# Check scope
|
|
if "create" not in token_info.get("scope", ""):
|
|
return error_response("insufficient_scope", "Token lacks create scope"), 403
|
|
|
|
# Extract properties
|
|
properties = normalize_properties(data)
|
|
|
|
# Map Micropub properties to StarPunk note format
|
|
title = extract_title(properties)
|
|
content = extract_content(properties)
|
|
tags = properties.get("category", [])
|
|
|
|
# Create note using existing CRUD
|
|
from starpunk.notes import create_note
|
|
|
|
try:
|
|
note = create_note(
|
|
title=title,
|
|
content=content,
|
|
tags=tags,
|
|
author=token_info["me"],
|
|
published=True # Micropub posts are published by default
|
|
)
|
|
|
|
# Build permalink URL
|
|
permalink = f"{current_app.config['SITE_URL']}/notes/{note.slug}"
|
|
|
|
# Return 201 Created with Location header
|
|
return "", 201, {"Location": permalink}
|
|
|
|
except Exception as e:
|
|
current_app.logger.error(f"Failed to create note via Micropub: {e}")
|
|
return error_response("server_error", "Failed to create post"), 500
|
|
|
|
def normalize_properties(data: dict) -> dict:
|
|
"""
|
|
Normalize Micropub properties from both form and JSON formats
|
|
|
|
Handles:
|
|
- Form: property[]=value arrays
|
|
- JSON: {"properties": {"property": ["value"]}}
|
|
"""
|
|
if "properties" in data:
|
|
# JSON format
|
|
return data["properties"]
|
|
|
|
# Form format - convert to properties dict
|
|
properties = {}
|
|
for key, value in data.items():
|
|
if key.startswith("mp-") or key in ["action", "url", "access_token"]:
|
|
continue # Skip reserved properties
|
|
|
|
# Handle array notation: property[] -> property
|
|
clean_key = key.rstrip("[]")
|
|
|
|
# Ensure value is always a list
|
|
if not isinstance(value, list):
|
|
value = [value]
|
|
|
|
properties[clean_key] = value
|
|
|
|
return properties
|
|
|
|
def extract_content(properties: dict) -> str:
|
|
"""Extract content from Micropub properties"""
|
|
content = properties.get('content', [''])[0]
|
|
if not content:
|
|
raise MicropubError("Content is required")
|
|
return content
|
|
|
|
def extract_title(properties: dict) -> str:
|
|
"""Extract title from Micropub properties"""
|
|
# Use 'name' property if provided
|
|
title = properties.get('name', [''])[0]
|
|
|
|
# If no name, extract from content (first line, max 50 chars)
|
|
if not title:
|
|
content = properties.get('content', [''])[0]
|
|
if content:
|
|
first_line = content.split('\n')[0]
|
|
title = first_line[:50] + ('...' if len(first_line) > 50 else '')
|
|
|
|
return title
|
|
```
|
|
|
|
#### V1 Limitations
|
|
|
|
In V1, only the `create` action is supported. Update and delete operations will return an error response and are planned for a post-V1 release.
|
|
|
|
### 3. Query Endpoints
|
|
|
|
Micropub defines several query endpoints for client discovery:
|
|
|
|
```python
|
|
def handle_query(args: dict, token_info: dict) -> tuple:
|
|
"""Handle Micropub query endpoints"""
|
|
q = args.get("q")
|
|
|
|
if q == "config":
|
|
# Return server configuration
|
|
config = {
|
|
"media-endpoint": None, # No media endpoint in V1
|
|
"syndicate-to": [], # No syndication targets in V1
|
|
"post-types": [
|
|
{
|
|
"type": "note",
|
|
"name": "Note",
|
|
"properties": ["content"]
|
|
}
|
|
]
|
|
}
|
|
return jsonify(config), 200
|
|
|
|
elif q == "source":
|
|
# Return source of a specific post
|
|
url = args.get("url")
|
|
if not url:
|
|
return error_response("invalid_request", "No URL provided"), 400
|
|
|
|
slug = extract_slug_from_url(url)
|
|
from starpunk.notes import get_note
|
|
|
|
note = get_note(slug)
|
|
if not note:
|
|
return error_response("invalid_request", "Post not found"), 400
|
|
|
|
# Convert note to Micropub format
|
|
mf2 = {
|
|
"type": ["h-entry"],
|
|
"properties": {
|
|
"content": [note.content],
|
|
"name": [note.title] if note.title else [],
|
|
"category": note.tags if note.tags else [],
|
|
"published": [note.created_at.isoformat()],
|
|
"url": [f"{current_app.config['SITE_URL']}/notes/{note.slug}"]
|
|
}
|
|
}
|
|
|
|
return jsonify(mf2), 200
|
|
|
|
elif q == "syndicate-to":
|
|
# Return syndication targets (none for V1)
|
|
return jsonify({"syndicate-to": []}), 200
|
|
|
|
else:
|
|
return error_response("invalid_request", f"Unknown query: {q}"), 400
|
|
```
|
|
|
|
### 4. Authorization Flow Integration
|
|
|
|
The complete authorization flow for Micropub:
|
|
|
|
```
|
|
┌──────────┐
|
|
│ Client │
|
|
└─────┬────┘
|
|
│
|
|
│ 1. GET /auth/authorization?
|
|
│ response_type=code&
|
|
│ client_id=https://client.example&
|
|
│ redirect_uri=https://client.example/callback&
|
|
│ state=1234567890&
|
|
│ scope=create& # V1: create only
|
|
│ me=https://user.example&
|
|
│ code_challenge=XXXXX& # Optional PKCE
|
|
│ code_challenge_method=S256
|
|
▼
|
|
┌──────────────┐
|
|
│ StarPunk │
|
|
│ (Authorize) │
|
|
└─────┬────────┘
|
|
│
|
|
│ 2. Check admin session
|
|
│ If not logged in: redirect to /auth/login
|
|
│ If logged in: show authorization form
|
|
│
|
|
│ 3. User approves, generate authorization code
|
|
│
|
|
▼
|
|
┌──────────┐
|
|
│ Client │
|
|
└─────┬────┘
|
|
│
|
|
│ 4. POST /auth/token
|
|
│ grant_type=authorization_code&
|
|
│ code=XXXXX&
|
|
│ client_id=https://client.example&
|
|
│ redirect_uri=https://client.example/callback&
|
|
│ me=https://user.example&
|
|
│ code_verifier=XXXXX # If PKCE was used
|
|
▼
|
|
┌──────────────┐
|
|
│ StarPunk │
|
|
│ (Token) │
|
|
└─────┬────────┘
|
|
│
|
|
│ 5. Return access token
|
|
│ {
|
|
│ "access_token": "XXXXX",
|
|
│ "token_type": "Bearer",
|
|
│ "scope": "create", # V1: create only
|
|
│ "me": "https://user.example"
|
|
│ }
|
|
▼
|
|
┌──────────┐
|
|
│ Client │
|
|
└─────┬────┘
|
|
│
|
|
│ 6. POST /micropub
|
|
│ Authorization: Bearer XXXXX
|
|
│ Content-Type: application/x-www-form-urlencoded
|
|
│
|
|
│ h=entry&content=Hello+world
|
|
▼
|
|
┌──────────────┐
|
|
│ StarPunk │
|
|
│ (Micropub) │
|
|
└──────────────┘
|
|
```
|
|
|
|
## Implementation Plan
|
|
|
|
### Phase 1: Token Management (2-3 days)
|
|
|
|
1. **Database Migration**
|
|
- Update tokens table schema
|
|
- Add indexes for performance
|
|
- Create migration script
|
|
|
|
2. **Token Module** (`starpunk/tokens.py`)
|
|
- Token generation functions
|
|
- Token storage with hashing
|
|
- Token verification
|
|
- Scope validation helpers
|
|
|
|
3. **Token Endpoint** (`/auth/token`)
|
|
- Authorization code validation
|
|
- PKCE verification
|
|
- Token generation
|
|
- Response formatting
|
|
|
|
4. **Testing**
|
|
- Unit tests for token functions
|
|
- Integration tests for token endpoint
|
|
- Security tests (token expiry, revocation)
|
|
|
|
### Phase 2: Micropub Core (3-4 days)
|
|
|
|
1. **Micropub Module** (`starpunk/micropub.py`)
|
|
- Property normalization
|
|
- Content extraction
|
|
- Note format conversion
|
|
- Error handling
|
|
|
|
2. **Micropub Routes** (`starpunk/routes/micropub.py`)
|
|
- Main endpoint handler
|
|
- Bearer token extraction
|
|
- Content-type handling
|
|
- Action routing
|
|
|
|
3. **Create Handler**
|
|
- Form-encoded parsing
|
|
- JSON parsing
|
|
- Note creation via CRUD
|
|
- Location header generation
|
|
|
|
4. **Testing**
|
|
- Create post via form-encoded
|
|
- Create post via JSON
|
|
- Invalid token handling
|
|
- Scope validation
|
|
|
|
### Phase 3: Query Endpoints & Validation (1-2 days)
|
|
|
|
1. **Query Endpoints**
|
|
- Config endpoint
|
|
- Source endpoint
|
|
- Syndicate-to endpoint (empty for V1)
|
|
|
|
2. **Property Validation**
|
|
- Content extraction and validation
|
|
- Title generation from content
|
|
- Tag/category mapping
|
|
|
|
3. **Testing**
|
|
- Query endpoint responses
|
|
- Property mapping edge cases
|
|
- Error response formatting
|
|
|
|
### Phase 4: Integration & Polish (1-2 days)
|
|
|
|
1. **Authorization Endpoint Updates**
|
|
- Add scope parameter handling
|
|
- Store requested scopes with auth state
|
|
- Pass scopes to token creation
|
|
|
|
2. **Discovery Headers**
|
|
- Add `Link: <https://site/micropub>; rel="micropub"` header
|
|
- Add `Link: <https://site/auth/authorization>; rel="authorization_endpoint"` header
|
|
- Add `Link: <https://site/auth/token>; rel="token_endpoint"` header
|
|
|
|
3. **Error Handling**
|
|
- Consistent error responses
|
|
- Proper HTTP status codes
|
|
- Detailed error descriptions
|
|
|
|
4. **Documentation**
|
|
- API documentation
|
|
- Client configuration guide
|
|
- Testing with real clients
|
|
|
|
## V1 Minimal Feature Set
|
|
|
|
### In Scope for V1
|
|
|
|
✅ **Required for Micropub Compliance:**
|
|
- Bearer token authentication
|
|
- Create posts (form-encoded)
|
|
- Create posts (JSON)
|
|
- Return 201 Created with Location header
|
|
- Configuration query endpoint
|
|
- Source query endpoint
|
|
- Authorization endpoint (`/auth/authorization`)
|
|
- Token endpoint (`/auth/token`)
|
|
|
|
✅ **Required for Functionality:**
|
|
- Authorization code generation and storage
|
|
- Token generation with SHA256 hashing
|
|
- Scope enforcement (create only for V1)
|
|
- Integration with existing notes CRUD
|
|
- Property mapping (content, title, tags)
|
|
- Optional PKCE support
|
|
|
|
### Out of Scope for V1 (Post-V1 Roadmap)
|
|
|
|
❌ **Deferred to Post-V1:**
|
|
- Update operations (action=update)
|
|
- Delete operations (action=delete)
|
|
- Media endpoint (file uploads)
|
|
- Photo/video properties
|
|
- Syndication targets
|
|
- Complex post types (articles, replies, etc.)
|
|
- Batch operations
|
|
- Websub notifications
|
|
- Token introspection endpoint
|
|
- Token revocation endpoint
|
|
- Multiple scopes beyond "create"
|
|
|
|
## Security Considerations
|
|
|
|
### Token Security
|
|
|
|
1. **Storage**
|
|
- Tokens stored as SHA256 hashes
|
|
- Original token never logged
|
|
- Secure random generation (32 bytes)
|
|
|
|
2. **Validation**
|
|
- Check token exists
|
|
- Check not expired (90 days)
|
|
- Check not revoked
|
|
- Verify scope for operation
|
|
|
|
3. **Transport**
|
|
- Require HTTPS in production
|
|
- Support both header and form parameter
|
|
- Never include in URLs
|
|
|
|
### Scope Enforcement
|
|
|
|
```python
|
|
# V1 only supports "create" scope
|
|
SUPPORTED_SCOPES = ["create"]
|
|
DEFAULT_SCOPE = "create"
|
|
|
|
def check_scope(required: str, granted: str) -> bool:
|
|
"""Check if granted scopes include required scope"""
|
|
if not granted:
|
|
# IndieAuth spec: no scope means no access
|
|
return False
|
|
granted_scopes = set(granted.split())
|
|
return required in granted_scopes
|
|
|
|
def validate_requested_scope(scope: str) -> str:
|
|
"""Validate and filter requested scopes to supported ones"""
|
|
if not scope:
|
|
return "" # Empty scope allowed during authorization
|
|
|
|
requested = set(scope.split())
|
|
supported = set(SUPPORTED_SCOPES)
|
|
valid_scopes = requested & supported
|
|
|
|
return " ".join(valid_scopes) if valid_scopes else ""
|
|
```
|
|
|
|
### Input Validation
|
|
|
|
1. **Content Validation**
|
|
- Sanitize HTML content
|
|
- Validate URLs
|
|
- Limit content size
|
|
- Check required properties
|
|
|
|
2. **URL Validation**
|
|
- Verify URL belongs to this site
|
|
- Extract valid slugs
|
|
- Prevent directory traversal
|
|
|
|
3. **Rate Limiting**
|
|
- Limit requests per token
|
|
- Limit failed authentication attempts
|
|
- Implement exponential backoff
|
|
|
|
## Testing Strategy
|
|
|
|
### Unit Tests
|
|
|
|
```python
|
|
# tests/test_micropub.py
|
|
|
|
def test_normalize_properties_form():
|
|
"""Test form-encoded property normalization"""
|
|
data = {
|
|
"content": ["Hello world"],
|
|
"category[]": ["tag1", "tag2"],
|
|
"mp-slug": ["my-post"]
|
|
}
|
|
|
|
properties = normalize_properties(data)
|
|
|
|
assert properties["content"] == ["Hello world"]
|
|
assert properties["category"] == ["tag1", "tag2"]
|
|
assert "mp-slug" not in properties
|
|
|
|
def test_normalize_properties_json():
|
|
"""Test JSON property normalization"""
|
|
data = {
|
|
"type": ["h-entry"],
|
|
"properties": {
|
|
"content": ["Hello world"],
|
|
"category": ["tag1", "tag2"]
|
|
}
|
|
}
|
|
|
|
properties = normalize_properties(data)
|
|
|
|
assert properties["content"] == ["Hello world"]
|
|
assert properties["category"] == ["tag1", "tag2"]
|
|
|
|
def test_bearer_token_extraction():
|
|
"""Test token extraction from header and form"""
|
|
# Test header
|
|
request = Mock(headers={"Authorization": "Bearer abc123"})
|
|
token = extract_bearer_token(request)
|
|
assert token == "abc123"
|
|
|
|
# Test form parameter
|
|
request = Mock(headers={}, form={"access_token": "xyz789"})
|
|
token = extract_bearer_token(request)
|
|
assert token == "xyz789"
|
|
```
|
|
|
|
### Integration Tests
|
|
|
|
```python
|
|
# tests/test_micropub_integration.py
|
|
|
|
def test_create_post_form_encoded(client, auth_token):
|
|
"""Test creating post with form-encoded request"""
|
|
response = client.post("/micropub",
|
|
headers={"Authorization": f"Bearer {auth_token}"},
|
|
data={
|
|
"h": "entry",
|
|
"content": "Test post",
|
|
"category[]": ["test", "micropub"]
|
|
}
|
|
)
|
|
|
|
assert response.status_code == 201
|
|
assert "Location" in response.headers
|
|
assert "/notes/" in response.headers["Location"]
|
|
|
|
def test_create_post_json(client, auth_token):
|
|
"""Test creating post with JSON request"""
|
|
response = client.post("/micropub",
|
|
headers={
|
|
"Authorization": f"Bearer {auth_token}",
|
|
"Content-Type": "application/json"
|
|
},
|
|
json={
|
|
"type": ["h-entry"],
|
|
"properties": {
|
|
"content": ["Test post"],
|
|
"category": ["test", "micropub"]
|
|
}
|
|
}
|
|
)
|
|
|
|
assert response.status_code == 201
|
|
assert "Location" in response.headers
|
|
|
|
def test_unauthorized_request(client):
|
|
"""Test request without token"""
|
|
response = client.post("/micropub",
|
|
data={"h": "entry", "content": "Test"}
|
|
)
|
|
|
|
assert response.status_code == 401
|
|
assert response.json["error"] == "unauthorized"
|
|
|
|
def test_insufficient_scope(client, read_only_token):
|
|
"""Test request with insufficient scope"""
|
|
response = client.post("/micropub",
|
|
headers={"Authorization": f"Bearer {read_only_token}"},
|
|
data={"h": "entry", "content": "Test"}
|
|
)
|
|
|
|
assert response.status_code == 403
|
|
assert response.json["error"] == "insufficient_scope"
|
|
```
|
|
|
|
### Client Testing
|
|
|
|
Test with real Micropub clients:
|
|
|
|
1. **Indigenous** (iOS/Android)
|
|
- Configure with StarPunk URLs
|
|
- Test post creation
|
|
- Test photo uploads (when media endpoint added)
|
|
|
|
2. **Quill** (Web)
|
|
- Test at https://quill.p3k.io
|
|
- Enter StarPunk domain
|
|
- Complete authentication
|
|
- Create test posts
|
|
|
|
3. **Micropublish.net** (Web)
|
|
- Test at https://micropublish.net
|
|
- Full-featured client
|
|
- Test all operations
|
|
|
|
## Migration Path
|
|
|
|
### Database Migrations
|
|
|
|
```sql
|
|
-- Migration: Add Micropub support
|
|
-- Version: 0.10.0
|
|
|
|
-- 1. Update tokens table for better security
|
|
ALTER TABLE tokens RENAME TO tokens_old;
|
|
|
|
CREATE TABLE tokens (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
token_hash TEXT UNIQUE NOT NULL,
|
|
me TEXT NOT NULL,
|
|
client_id TEXT,
|
|
scope TEXT,
|
|
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
expires_at TIMESTAMP,
|
|
last_used_at TIMESTAMP,
|
|
revoked_at TIMESTAMP
|
|
);
|
|
|
|
-- Migrate existing tokens (if any)
|
|
-- Note: Existing tokens will be invalid as we don't have original values
|
|
|
|
DROP TABLE tokens_old;
|
|
|
|
-- 2. Add authorization codes table for token exchange
|
|
CREATE TABLE IF NOT EXISTS authorization_codes (
|
|
code TEXT PRIMARY KEY,
|
|
me TEXT NOT NULL,
|
|
client_id TEXT NOT NULL,
|
|
redirect_uri TEXT NOT NULL,
|
|
scope TEXT,
|
|
code_challenge TEXT,
|
|
code_challenge_method TEXT,
|
|
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
expires_at TIMESTAMP NOT NULL,
|
|
used_at TIMESTAMP
|
|
);
|
|
|
|
CREATE INDEX idx_auth_codes_expires ON authorization_codes(expires_at);
|
|
```
|
|
|
|
### Configuration Updates
|
|
|
|
```python
|
|
# config.py additions
|
|
|
|
# Micropub settings
|
|
MICROPUB_ENDPOINT = "/micropub"
|
|
TOKEN_ENDPOINT = "/auth/token"
|
|
TOKEN_EXPIRY_DAYS = 90
|
|
AUTHORIZATION_CODE_EXPIRY_MINUTES = 10
|
|
|
|
# Supported scopes (V1 only supports create)
|
|
SUPPORTED_SCOPES = ["create"]
|
|
DEFAULT_SCOPE = "create"
|
|
```
|
|
|
|
## Effort Estimates
|
|
|
|
### Total Effort: 6-8 days (Reduced for V1 scope)
|
|
|
|
| Phase | Task | Effort | Priority |
|
|
|-------|------|--------|----------|
|
|
| 1 | Token Management | 2-3 days | Critical |
|
|
| 2 | Micropub Core (Create Only) | 2-3 days | Critical |
|
|
| 3 | Query Endpoints & Validation | 1-2 days | High |
|
|
| 4 | Integration & Polish | 1-2 days | High |
|
|
|
|
### Resource Requirements
|
|
|
|
- **Developer**: 1 person full-time for 2 weeks
|
|
- **Testing**: Access to Micropub clients
|
|
- **Documentation**: 1 day included in estimates
|
|
|
|
### Risk Factors
|
|
|
|
1. **Token Security** (High Impact, Low Probability)
|
|
- Mitigation: Follow security best practices, hash tokens
|
|
|
|
2. **Client Compatibility** (Medium Impact, Medium Probability)
|
|
- Mitigation: Test with multiple clients early
|
|
|
|
3. **Scope Creep** (Medium Impact, High Probability)
|
|
- Mitigation: Strict V1 feature set, defer nice-to-haves
|
|
|
|
## Success Criteria
|
|
|
|
### Functional Requirements
|
|
|
|
✅ Micropub endpoint responds to POST requests
|
|
✅ Bearer token authentication works
|
|
✅ Posts can be created via form-encoded requests
|
|
✅ Posts can be created via JSON requests
|
|
✅ Location header returned on creation
|
|
✅ Query endpoints return valid responses
|
|
✅ Token endpoint exchanges codes for tokens
|
|
✅ Scopes are enforced correctly
|
|
|
|
### Performance Requirements
|
|
|
|
✅ Post creation < 500ms
|
|
✅ Token validation < 50ms
|
|
✅ Query responses < 200ms
|
|
|
|
### Security Requirements
|
|
|
|
✅ Tokens stored as hashes
|
|
✅ Expired tokens rejected
|
|
✅ Invalid tokens return 401
|
|
✅ Insufficient scope returns 403
|
|
✅ HTTPS required in production
|
|
|
|
### Compatibility Requirements
|
|
|
|
✅ Works with Indigenous app
|
|
✅ Works with Quill web client
|
|
✅ Works with Micropublish.net
|
|
✅ Passes Micropub Rocks validator
|
|
|
|
## Conclusion
|
|
|
|
The Micropub implementation is straightforward given StarPunk's existing architecture:
|
|
|
|
1. **Existing CRUD operations** in `notes.py` handle the storage
|
|
2. **Existing authentication** provides the foundation for tokens
|
|
3. **Clear specification** from W3C defines exact requirements
|
|
4. **Minimal V1 scope** focuses on core functionality
|
|
|
|
The main work is:
|
|
- Building the token endpoint for IndieAuth
|
|
- Creating the Micropub request handler
|
|
- Mapping between Micropub and StarPunk formats
|
|
- Proper error handling and responses
|
|
|
|
With 8-10 days of focused development, StarPunk will have a fully functional Micropub endpoint, unblocking the V1 release.
|
|
|
|
## Appendix: Example Requests
|
|
|
|
### Creating a Note (Form-Encoded)
|
|
|
|
```http
|
|
POST /micropub HTTP/1.1
|
|
Host: starpunk.example.com
|
|
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
|
|
Content-Type: application/x-www-form-urlencoded
|
|
|
|
h=entry&
|
|
content=Just+had+coffee+at+the+new+place+downtown.+Really+good!&
|
|
category[]=coffee&
|
|
category[]=portland
|
|
```
|
|
|
|
### Creating a Note (JSON)
|
|
|
|
```http
|
|
POST /micropub HTTP/1.1
|
|
Host: starpunk.example.com
|
|
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
|
|
Content-Type: application/json
|
|
|
|
```
|
|
|
|
### V1 Limitation Examples
|
|
|
|
#### Attempting Update (Returns Error)
|
|
|
|
```http
|
|
POST /micropub HTTP/1.1
|
|
Host: starpunk.example.com
|
|
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
|
|
Content-Type: application/json
|
|
|
|
```
|
|
|
|
Response (400 Bad Request):
|
|
```json
|
|
{
|
|
"error": "invalid_request",
|
|
"error_description": "Update action not supported in V1"
|
|
}
|
|
```
|
|
|
|
#### Attempting Delete (Returns Error)
|
|
|
|
```http
|
|
POST /micropub HTTP/1.1
|
|
Host: starpunk.example.com
|
|
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
|
|
Content-Type: application/x-www-form-urlencoded
|
|
|
|
action=delete&
|
|
url=https://starpunk.example.com/notes/2024/11/test-post
|
|
```
|
|
|
|
Response (400 Bad Request):
|
|
```json
|
|
{
|
|
"error": "invalid_request",
|
|
"error_description": "Delete action not supported in V1"
|
|
}
|
|
```
|
|
|
|
### Querying Configuration
|
|
|
|
```http
|
|
GET /micropub?q=config HTTP/1.1
|
|
Host: starpunk.example.com
|
|
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"media-endpoint": null,
|
|
"syndicate-to": [],
|
|
"post-types": [
|
|
{
|
|
"type": "note",
|
|
"name": "Note",
|
|
"properties": ["content"]
|
|
}
|
|
]
|
|
}
|
|
``` |