phil/StarPunk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Address Micropub design issues and clarify V1 scope

Browse Source 
- Create ADR-029 for IndieAuth/Micropub integration strategy
- Address all critical issues from developer review:
  - Add missing 'me' parameter to token endpoint
  - Clarify PKCE as optional extension
  - Define token security migration strategy
  - Add authorization_codes table schema
  - Define property mapping rules
  - Clarify two authentication flows
- Simplify V1 scope per user decision:
  - Remove update/delete operations from V1
  - Focus on create-only functionality
  - Reduce timeline from 8-10 to 6-8 days
- Update project plan with post-V1 roadmap:
  - Phase 11: Update/delete operations (V1.1)
  - Phase 12: Media endpoint (V1.2)
  - Phase 13: Advanced IndieWeb features (V2.0)
  - Phase 14: Enhanced features (V2.0+)
- Create token security migration documentation
- Update ADR-028 for consistency with simplified scope

BREAKING CHANGE: Token migration will invalidate all existing tokens for security
This commit is contained in:
Phil Skentelbery
2025-11-24 11:39:13 -07:00
parent 5bbecad01d
commit dca9604746
6 changed files with 1127 additions and 129 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
CLAUDE.md
View File

@@ -8,6 +8,19 @@ This file contains operational instructions for Claude agents working on this pr
- All Python commands must be run with `uv run` prefix
- Example: `uv run pytest`, `uv run flask run`
## Agent-Architect Protocol
When invoking the agent-architect, always remind it to:
1. Review documentation in docs/ before working on the task it is given
- docs/architecture, docs/decisions, docs/standards are of particular interest
2. Give it the map of the documentation folder as described in the "Understanding the docs/ Structure" section below
3. Search for authoritative documentation for any web standard it is implementing on https://www.w3.org/
4. If it is reviewing a developers implementation report and it is accepts the completed work it should go back and update the project plan to reflect the completed work
## Agent-Developer Protocol
When invoking the agent-developer, always remind it to:

17
docs/decisions/ADR-028-micropub-implementation.md
View File

@@ -53,14 +53,15 @@ We will implement a **minimal but complete Micropub server** for V1, focusing on
- Delegate to existing `notes.py` CRUD functions
- Proper error handling and status codes
3. **V1 Feature Scope**
3. **V1 Feature Scope** (Simplified per user decision)
- ✅ Create posts (form-encoded and JSON)
- ✅ Query endpoints (config, source)
- ✅ Bearer token authentication
- ✅ Scope-based authorization
- ❌ Media endpoint (deferred)
- ❌ Update/delete operations (deferred)
- ❌ Syndication (deferred)
- ✅ Scope-based authorization (create only)
- ❌ Media endpoint (post-V1)
- ❌ Update operations (post-V1)
- ❌ Delete operations (post-V1)
- ❌ Syndication (post-V1)
### Technology Choices
@@ -139,12 +140,14 @@ We will implement a **minimal but complete Micropub server** for V1, focusing on
- Create post functionality
- Error response formatting
### Phase 3: Queries & Polish (Days 8-10)
### Phase 3: Queries & Polish (Days 6-8)
- Config and source query endpoints
- Authorization endpoint scope handling
- Authorization endpoint with admin session check
- Discovery headers and links
- Client testing and documentation
**Note**: Timeline reduced from 8-10 days to 6-8 days due to V1 scope simplification (no update/delete)
## Alternatives Considered
### Alternative 1: Full Micropub Implementation

537
docs/decisions/ADR-029-micropub-indieauth-integration.md Normal file
View File

@@ -0,0 +1,537 @@
# ADR-029: Micropub IndieAuth Integration Strategy
## Status
Accepted
## Context
The developer review of our Micropub design (ADR-028) revealed critical issues and questions about how IndieAuth and Micropub integrate. This ADR addresses all architectural decisions needed to proceed with implementation.
### Critical Issues Identified
1. **Token endpoint missing required `me` parameter** in the IndieAuth spec
2. **PKCE confusion** - it's not part of IndieAuth spec, but StarPunk uses it with IndieLogin.com
3. **Database security issue** - tokens stored in plain text
4. **Missing `authorization_codes` table** for token exchange
5. **Property mapping rules** undefined for Micropub to StarPunk conversion
6. **Authorization endpoint location** unclear
7. **Two authentication flows** need clarification
### V1 Scope Decision
The user has agreed to **simplify V1** by:
- ✅ Omitting update operations from V1
- ✅ Omitting delete operations from V1
- ✅ Focusing on create-only for V1 release
- Post-V1 features will be tracked separately
## Decision
We will implement a **hybrid IndieAuth architecture** that clearly separates admin authentication from Micropub authorization.
### Architectural Decisions
#### 1. Token Endpoint `me` Parameter (RESOLVED)
**Issue**: IndieAuth spec requires `me` parameter in token exchange, but our design missed it.
**Decision**: Add `me` parameter validation to token endpoint.
**Implementation**:
```python
# Token exchange request MUST include:
POST /auth/token
grant_type=authorization_code
code={code}
client_id={client_url}
redirect_uri={redirect_url}
me={user_profile_url} # REQUIRED by IndieAuth spec
```
**Validation**:
- Verify `me` matches the value stored with the authorization code
- Return error if mismatch (prevents code hijacking)
#### 2. PKCE Strategy (RESOLVED)
**Issue**: PKCE is not part of IndieAuth spec, but StarPunk uses it with IndieLogin.com.
**Decision**: Make PKCE **optional but recommended**.
**Implementation**:
- Check for `code_challenge` in authorization request
- If present, require `code_verifier` in token exchange
- If absent, proceed without PKCE (spec-compliant)
- Document as security enhancement beyond spec
**Rationale**:
- IndieLogin.com supports PKCE as an extension
- Other IndieAuth providers may not support it
- Making it optional ensures broader compatibility
#### 3. Token Storage Security (RESOLVED)
**Issue**: Current `tokens` table stores tokens in plain text (major security vulnerability).
**Decision**: Implement **immediate migration** to hashed token storage.
**Migration Strategy**:
```sql
-- Step 1: Create new secure tokens table
CREATE TABLE tokens_secure (
id INTEGER PRIMARY KEY AUTOINCREMENT,
token_hash TEXT UNIQUE NOT NULL, -- SHA256 hash
me TEXT NOT NULL,
client_id TEXT,
scope TEXT DEFAULT 'create',
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
last_used_at TIMESTAMP,
revoked_at TIMESTAMP
);
-- Step 2: Invalidate all existing tokens (security breach recovery)
-- Since we can't hash plain text tokens retroactively, all must be revoked
DROP TABLE IF EXISTS tokens;
-- Step 3: Rename secure table
ALTER TABLE tokens_secure RENAME TO tokens;
-- Step 4: Create indexes
CREATE INDEX idx_tokens_hash ON tokens(token_hash);
CREATE INDEX idx_tokens_me ON tokens(me);
CREATE INDEX idx_tokens_expires ON tokens(expires_at);
```
**Security Notice**: All existing tokens will be invalidated. Users must re-authenticate.
#### 4. Authorization Codes Table (RESOLVED)
**Issue**: Design references `authorization_codes` table that doesn't exist.
**Decision**: Create the table as part of Micropub implementation.
**Schema**:
```sql
CREATE TABLE authorization_codes (
code TEXT PRIMARY KEY,
code_hash TEXT UNIQUE NOT NULL, -- SHA256 hash for security
me TEXT NOT NULL,
client_id TEXT NOT NULL,
redirect_uri TEXT NOT NULL,
scope TEXT DEFAULT 'create',
code_challenge TEXT, -- Optional PKCE
code_challenge_method TEXT, -- S256 if PKCE used
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
used_at TIMESTAMP -- Prevent replay attacks
);
CREATE INDEX idx_auth_codes_hash ON authorization_codes(code_hash);
CREATE INDEX idx_auth_codes_expires ON authorization_codes(expires_at);
```
#### 5. Property Mapping Rules (RESOLVED)
**Issue**: Functions like `extract_title()` and `extract_content()` are undefined.
**Decision**: Define explicit mapping rules for V1.
**Micropub → StarPunk Mapping**:
```python
# Content mapping (required)
content = properties.get('content', [''])[0] # First content value
if not content:
return error_response("invalid_request", "Content is required")
# Title mapping (optional)
# Option 1: Use 'name' property if provided
title = properties.get('name', [''])[0]
# Option 2: If no name, extract from content (first line up to 50 chars)
if not title and content:
first_line = content.split('\n')[0]
title = first_line[:50] + ('...' if len(first_line) > 50 else '')
# Tags mapping
tags = properties.get('category', []) # All category values become tags
# Published date (respect if provided, otherwise use current time)
published = properties.get('published', [''])[0]
if published:
# Parse ISO 8601 date
created_at = parse_iso8601(published)
else:
created_at = datetime.now()
# Slug generation
mp_slug = properties.get('mp-slug', [''])[0]
if mp_slug:
slug = slugify(mp_slug)
else:
slug = generate_slug(title or content[:30])
```
### Q1: Authorization Endpoint Location (RESOLVED)
**Issue**: Design mentions `/auth/authorization` but it doesn't exist.
**Decision**: Create **NEW** `/auth/authorization` endpoint for Micropub clients.
**Rationale**:
- Keep admin login (`/auth/login`) separate from Micropub authorization
- Clear separation of concerns
- Follows IndieAuth spec naming conventions
**Implementation**:
```python
@bp.route("/auth/authorization", methods=["GET", "POST"])
def authorization_endpoint():
"""
IndieAuth authorization endpoint for Micropub clients
GET: Display authorization form
POST: Process authorization and redirect with code
"""
if request.method == "GET":
# Parse IndieAuth parameters
response_type = request.args.get('response_type')
client_id = request.args.get('client_id')
redirect_uri = request.args.get('redirect_uri')
state = request.args.get('state')
scope = request.args.get('scope', 'create')
me = request.args.get('me')
code_challenge = request.args.get('code_challenge')
# Validate parameters
if response_type != 'code':
return error_response("unsupported_response_type")
# Check if user is logged in (via admin session)
if not verify_admin_session():
# Redirect to login, then back here
session['pending_auth'] = request.url
return redirect(url_for('auth.login_form'))
# Display authorization form
return render_template('auth/authorize.html',
client_id=client_id,
scope=scope,
redirect_uri=redirect_uri)
else: # POST
# User approved/denied authorization
# Generate authorization code
# Store in authorization_codes table
# Redirect to client with code
```
### Q2: Two Authentication Flows Integration (RESOLVED)
**Decision**: Maintain **two separate flows** with clear boundaries.
**Flow 1: Admin Login** (Existing)
- Purpose: Admin access to StarPunk interface
- Path: `/auth/login` → IndieLogin.com → `/auth/callback`
- Result: Session cookie for admin panel
- No changes needed
**Flow 2: Micropub Authorization** (New)
- Purpose: Micropub client authorization
- Path: `/auth/authorization` → `/auth/token`
- Result: Bearer token for API access
**Integration Point**: The authorization endpoint checks for admin session:
```python
def authorization_endpoint():
# Check if admin is logged in
if not has_admin_session():
# Store authorization request
# Redirect to admin login
# After login, return to authorization
return redirect_to_login_with_return()
# Admin is logged in, show authorization form
return show_authorization_form()
```
**Key Design Choice**: We act as our **own authorization server** for Micropub, not delegating to IndieLogin.com for this flow. This is because:
1. IndieLogin.com doesn't issue access tokens
2. We need to control scopes and token lifetime
3. We already have admin authentication to verify the user
### Q3: Scope Validation Rules (RESOLVED)
**Issue**: What happens when client requests no scopes?
**Decision**: Implement **Option C** - Allow empty scope during authorization, reject at token endpoint.
**Rationale**: This matches the IndieAuth spec requirement exactly.
**Implementation**:
```python
def handle_authorization():
scope = request.args.get('scope', '')
# Store whatever scope was requested (even empty)
authorization_code = create_authorization_code(
scope=scope, # Can be empty string
# ... other parameters
)
def handle_token_exchange():
auth_code = get_authorization_code(code)
# IndieAuth spec: MUST NOT issue token if no scope
if not auth_code.scope:
return error_response("invalid_scope",
"Authorization code was issued without scope")
# Issue token with the authorized scope
token = create_access_token(scope=auth_code.scope)
```
### Q4: V1 Scope - Update/Delete Operations (RESOLVED)
**Decision**: Remove update/delete from V1 completely.
**Changes Required**:
1. Remove `handle_update()` and `handle_delete()` from design doc
2. Remove update/delete from supported scopes in V1
3. Return "invalid_request" if action=update or action=delete
4. Document in project plan for post-V1
**V1 Supported Actions**:
- ✅ action=create (or no action - default)
- ❌ action=update → error response
- ❌ action=delete → error response
### Q5: Token Storage Security Fix (RESOLVED)
**Decision**: Fix the security issue as part of Micropub implementation.
**Implementation Plan**:
1. Create migration to new secure schema
2. Hash all new tokens before storage
3. Document that existing tokens will be invalidated
4. Add security notice to changelog
## Implementation Architecture
### Complete Authorization Flow
```
┌─────────────────────────────────────────────────────────┐
│ Micropub Client │
└────────────────────┬────────────────────────────────────┘
│
│ 1. GET /auth/authorization?
│ response_type=code&
│ client_id=https://app.example&
│ redirect_uri=...&
│ state=...&
│ scope=create&
│ me=https://user.example
▼
┌─────────────────────────────────────────────────────────┐
│ StarPunk Authorization Endpoint │
│ /auth/authorization │
├─────────────────────────────────────────────────────────┤
│ if not admin_logged_in: │
│ redirect_to_login() │
│ else: │
│ show_authorization_form() │
└────────────────────┬────────────────────────────────────┘
│
│ 2. User approves
│ POST /auth/authorization
│
│ 3. Redirect with code
│ https://app.example/callback?
│ code=xxx&state=yyy
▼
┌─────────────────────────────────────────────────────────┐
│ Micropub Client │
└────────────────────┬────────────────────────────────────┘
│
│ 4. POST /auth/token
│ grant_type=authorization_code&
│ code=xxx&
│ client_id=https://app.example&
│ redirect_uri=...&
│ me=https://user.example&
│ code_verifier=... (if PKCE)
▼
┌─────────────────────────────────────────────────────────┐
│ StarPunk Token Endpoint │
│ /auth/token │
├─────────────────────────────────────────────────────────┤
│ 1. Verify authorization code │
│ 2. Check code not used │
│ 3. Verify client_id matches │
│ 4. Verify redirect_uri matches │
│ 5. Verify me matches │
│ 6. Verify PKCE if present │
│ 7. Check scope not empty │
│ 8. Generate access token │
│ 9. Store hashed token │
│ 10. Return token response │
└────────────────────┬────────────────────────────────────┘
│
│ 5. Response:
│ {
│ "access_token": "xxx",
│ "token_type": "Bearer",
│ "scope": "create",
│ "me": "https://user.example"
│ }
▼
┌─────────────────────────────────────────────────────────┐
│ Micropub Client │
└────────────────────┬────────────────────────────────────┘
│
│ 6. POST /micropub
│ Authorization: Bearer xxx
│ h=entry&content=Hello
▼
┌─────────────────────────────────────────────────────────┐
│ StarPunk Micropub Endpoint │
│ /micropub │
├─────────────────────────────────────────────────────────┤
│ 1. Extract bearer token │
│ 2. Hash token and lookup │
│ 3. Verify not expired │
│ 4. Check scope includes "create" │
│ 5. Parse Micropub properties │
│ 6. Create note via notes.py │
│ 7. Return 201 with Location header │
└─────────────────────────────────────────────────────────┘
```
## Consequences
### Positive
- ✅ All spec compliance issues resolved
- ✅ Clear separation between admin auth and Micropub auth
- ✅ Security vulnerability in token storage fixed
- ✅ Simplified V1 scope (create-only)
- ✅ PKCE optional for compatibility
- ✅ Clear property mapping rules
### Negative
- ⚠️ Existing tokens will be invalidated (security fix)
- ⚠️ More complex than initially designed
- ⚠️ Two authorization flows to maintain
### Neutral
- We become our own authorization server (for Micropub only)
- Admin must be logged in to authorize Micropub clients
- Update/delete deferred to post-V1
## Migration Requirements
### Database Migration Script
```sql
-- Migration: Fix token security and add authorization codes
-- Version: 0.10.0
-- 1. Create secure tokens table
CREATE TABLE tokens_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
token_hash TEXT UNIQUE NOT NULL,
me TEXT NOT NULL,
client_id TEXT,
scope TEXT DEFAULT 'create',
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
last_used_at TIMESTAMP,
revoked_at TIMESTAMP
);
-- 2. Drop insecure table (invalidates all tokens)
DROP TABLE IF EXISTS tokens;
-- 3. Rename to final name
ALTER TABLE tokens_new RENAME TO tokens;
-- 4. Create authorization codes table
CREATE TABLE authorization_codes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
code_hash TEXT UNIQUE NOT NULL,
me TEXT NOT NULL,
client_id TEXT NOT NULL,
redirect_uri TEXT NOT NULL,
scope TEXT,
state TEXT,
code_challenge TEXT,
code_challenge_method TEXT,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
used_at TIMESTAMP
);
-- 5. Create indexes
CREATE INDEX idx_tokens_hash ON tokens(token_hash);
CREATE INDEX idx_tokens_expires ON tokens(expires_at);
CREATE INDEX idx_auth_codes_hash ON authorization_codes(code_hash);
CREATE INDEX idx_auth_codes_expires ON authorization_codes(expires_at);
-- 6. Clean up expired auth state
DELETE FROM auth_state WHERE expires_at < datetime('now');
```
## Implementation Checklist
### Phase 1: Security & Database
- [ ] Create database migration script
- [ ] Implement token hashing functions
- [ ] Add authorization_codes table
- [ ] Update database.py schema
### Phase 2: Authorization Endpoint
- [ ] Create `/auth/authorization` route
- [ ] Implement authorization form template
- [ ] Add scope approval UI
- [ ] Generate and store authorization codes
### Phase 3: Token Endpoint
- [ ] Create `/auth/token` route
- [ ] Implement code exchange logic
- [ ] Add `me` parameter validation
- [ ] Optional PKCE verification
- [ ] Generate and store hashed tokens
### Phase 4: Micropub Endpoint (Create Only)
- [ ] Create `/micropub` route
- [ ] Bearer token extraction
- [ ] Token verification (hash lookup)
- [ ] Property normalization
- [ ] Content/title/tags mapping
- [ ] Note creation via notes.py
- [ ] Location header response
### Phase 5: Testing & Documentation
- [ ] Test with Indigenous app
- [ ] Test with Quill
- [ ] Update API documentation
- [ ] Security audit
- [ ] Performance testing
## References
- [IndieAuth Spec - Token Endpoint](https://indieauth.spec.indieweb.org/#token-endpoint)
- [IndieAuth Spec - Authorization Code](https://indieauth.spec.indieweb.org/#authorization-code)
- [Micropub Spec - Authentication](https://www.w3.org/TR/micropub/#authentication)
- [OAuth 2.0 Security Best Practices](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-security-topics)
## Related ADRs
- ADR-021: IndieAuth Provider Strategy (understanding flows)
- ADR-028: Micropub Implementation Strategy (original design)
- ADR-005: IndieLogin Authentication (admin auth flow)
---
**Date**: 2024-11-24
**Author**: StarPunk Architecture Team
**Status**: Accepted
**Version Impact**: Requires 0.10.0 (breaking change - token invalidation)

266
docs/design/micropub-endpoint-design.md
View File

@@ -2,7 +2,7 @@
## Executive Summary
This document defines the architecture and implementation plan for adding Micropub support to StarPunk, the **critical blocker for V1 release**. The Micropub endpoint will enable external clients (like Indigenous, Quill, or Micropublish.net) to create, update, and delete posts on StarPunk installations.
This document defines the architecture and implementation plan for adding Micropub support to StarPunk, the **critical blocker for V1 release**. The Micropub endpoint will enable external clients (like Indigenous, Quill, or Micropublish.net) to create posts on StarPunk installations.
**Current State:**
- ✅ IndieAuth authentication working (authorization endpoint with PKCE)
@@ -12,11 +12,11 @@ This document defines the architecture and implementation plan for adding Microp
- ❌ **No Micropub endpoint** (V1 blocker)
- ❌ **No token endpoint** (required for Micropub auth)
**Target State:**
- Full Micropub server implementation
**Target State for V1:**
- Micropub server implementation (create-only for V1)
- IndieAuth token endpoint for issuing access tokens
- Support for creating, updating, and deleting posts
- Minimal V1 implementation (no media endpoint initially)
- Support for creating posts via form-encoded and JSON
- Minimal V1 implementation (no media endpoint, no update/delete)
## Architecture Overview
@@ -93,7 +93,8 @@ grant_type=authorization_code&
code={authorization_code}&
client_id={client_url}&
redirect_uri={redirect_url}&
code_verifier={pkce_verifier}
me={user_profile_url}&
code_verifier={pkce_verifier} # Optional (if PKCE was used)
```
#### Response Format
@@ -173,24 +174,51 @@ def verify_token(token: str) -> Optional[dict]:
#### Database Schema Update
```sql
-- Update existing tokens table to use token_hash
ALTER TABLE tokens RENAME TO tokens_old;
-- SECURITY FIX: Migrate to hashed token storage
-- Note: This will invalidate all existing tokens
CREATE TABLE tokens (
-- Step 1: Create new secure tokens table
CREATE TABLE tokens_secure (
id INTEGER PRIMARY KEY AUTOINCREMENT,
token_hash TEXT UNIQUE NOT NULL, -- SHA256 hash of token
me TEXT NOT NULL,
client_id TEXT,
scope TEXT,
scope TEXT DEFAULT 'create', -- Default scope for V1
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
last_used_at TIMESTAMP,
revoked_at TIMESTAMP
);
-- Step 2: Drop old insecure table (invalidates existing tokens)
DROP TABLE IF EXISTS tokens;
-- Step 3: Rename to final name
ALTER TABLE tokens_secure RENAME TO tokens;
-- Step 4: Create indexes
CREATE INDEX idx_tokens_hash ON tokens(token_hash);
CREATE INDEX idx_tokens_me ON tokens(me);
CREATE INDEX idx_tokens_expires ON tokens(expires_at);
-- Step 5: Create authorization_codes table
CREATE TABLE authorization_codes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
code_hash TEXT UNIQUE NOT NULL, -- SHA256 hash for security
me TEXT NOT NULL,
client_id TEXT NOT NULL,
redirect_uri TEXT NOT NULL,
scope TEXT,
state TEXT,
code_challenge TEXT, -- Optional PKCE
code_challenge_method TEXT,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
used_at TIMESTAMP -- Prevent replay attacks
);
CREATE INDEX idx_auth_codes_hash ON authorization_codes(code_hash);
CREATE INDEX idx_auth_codes_expires ON authorization_codes(expires_at);
```
### 2. Micropub Endpoint (`/micropub`)
@@ -247,9 +275,11 @@ def micropub_endpoint():
if action == "create":
return handle_create(data, token_info)
elif action == "update":
return handle_update(data, token_info)
# V1: Update not supported
return error_response("invalid_request", "Update action not supported in V1"), 400
elif action == "delete":
return handle_delete(data, token_info)
# V1: Delete not supported
return error_response("invalid_request", "Delete action not supported in V1"), 400
else:
return error_response("invalid_request", f"Unknown action: {action}"), 400
except MicropubError as e:
@@ -334,70 +364,32 @@ def normalize_properties(data: dict) -> dict:
properties[clean_key] = value
return properties
def extract_content(properties: dict) -> str:
"""Extract content from Micropub properties"""
content = properties.get('content', [''])[0]
if not content:
raise MicropubError("Content is required")
return content
def extract_title(properties: dict) -> str:
"""Extract title from Micropub properties"""
# Use 'name' property if provided
title = properties.get('name', [''])[0]
# If no name, extract from content (first line, max 50 chars)
if not title:
content = properties.get('content', [''])[0]
if content:
first_line = content.split('\n')[0]
title = first_line[:50] + ('...' if len(first_line) > 50 else '')
return title
```
#### Update and Delete Handlers
#### V1 Limitations
```python
def handle_update(data: dict, token_info: dict) -> tuple:
"""Handle post updates"""
if "update" not in token_info.get("scope", ""):
return error_response("insufficient_scope", "Token lacks update scope"), 403
url = data.get("url")
if not url:
return error_response("invalid_request", "No URL provided"), 400
# Extract slug from URL
slug = extract_slug_from_url(url)
# Get existing note
from starpunk.notes import get_note, update_note
note = get_note(slug)
if not note:
return error_response("invalid_request", "Post not found"), 400
# Apply updates based on operation type
if "replace" in data:
# Replace properties
for prop, values in data["replace"].items():
apply_property_replace(note, prop, values)
if "add" in data:
# Add to properties
for prop, values in data["add"].items():
apply_property_add(note, prop, values)
if "delete" in data:
# Delete properties or values
for prop, values in data["delete"].items():
apply_property_delete(note, prop, values)
# Save changes
update_note(note)
return "", 204 # No Content
def handle_delete(data: dict, token_info: dict) -> tuple:
"""Handle post deletion"""
if "delete" not in token_info.get("scope", ""):
return error_response("insufficient_scope", "Token lacks delete scope"), 403
url = data.get("url")
if not url:
return error_response("invalid_request", "No URL provided"), 400
# Extract slug and delete
slug = extract_slug_from_url(url)
from starpunk.notes import delete_note
try:
delete_note(slug, soft=True) # Soft delete for safety
return "", 204
except NoteNotFoundError:
return error_response("invalid_request", "Post not found"), 400
```
In V1, only the `create` action is supported. Update and delete operations will return an error response and are planned for a post-V1 release.
### 3. Query Endpoints
@@ -472,8 +464,9 @@ The complete authorization flow for Micropub:
│ client_id=https://client.example&
│ redirect_uri=https://client.example/callback&
│ state=1234567890&
│ scope=create+update+delete&
│ code_challenge=XXXXX&
│ scope=create& # V1: create only
│ me=https://user.example&
│ code_challenge=XXXXX& # Optional PKCE
│ code_challenge_method=S256
▼
┌──────────────┐
@@ -481,10 +474,11 @@ The complete authorization flow for Micropub:
│ (Authorize) │
└─────┬────────┘
│
│ 2. Redirect to IndieLogin.com for authentication
│ (existing PKCE flow)
│ 2. Check admin session
│ If not logged in: redirect to /auth/login
│ If logged in: show authorization form
│
│ 3. After successful auth, redirect back with code
│ 3. User approves, generate authorization code
│
▼
┌──────────┐
@@ -496,7 +490,8 @@ The complete authorization flow for Micropub:
│ code=XXXXX&
│ client_id=https://client.example&
│ redirect_uri=https://client.example/callback&
│ code_verifier=XXXXX
│ me=https://user.example&
│ code_verifier=XXXXX # If PKCE was used
▼
┌──────────────┐
│ StarPunk │
@@ -507,7 +502,7 @@ The complete authorization flow for Micropub:
│ {
│ "access_token": "XXXXX",
│ "token_type": "Bearer",
│ "scope": "create update delete",
│ "scope": "create", # V1: create only
│ "me": "https://user.example"
│ }
▼
@@ -579,28 +574,22 @@ The complete authorization flow for Micropub:
- Invalid token handling
- Scope validation
### Phase 3: Extended Operations (2-3 days)
### Phase 3: Query Endpoints & Validation (1-2 days)
1. **Update Handler**
- Replace operations
- Add operations
- Delete operations
- Property mapping
2. **Delete Handler**
- URL parsing
- Soft deletion
- Response handling
3. **Query Endpoints**
1. **Query Endpoints**
- Config endpoint
- Source endpoint
- Syndicate-to endpoint
- Syndicate-to endpoint (empty for V1)
4. **Testing**
- Update operations
- Delete operations
- Query responses
2. **Property Validation**
- Content extraction and validation
- Title generation from content
- Tag/category mapping
3. **Testing**
- Query endpoint responses
- Property mapping edge cases
- Error response formatting
### Phase 4: Integration & Polish (1-2 days)
@@ -635,24 +624,31 @@ The complete authorization flow for Micropub:
- Return 201 Created with Location header
- Configuration query endpoint
- Source query endpoint
- Authorization endpoint (`/auth/authorization`)
- Token endpoint (`/auth/token`)
✅ **Required for Functionality:**
- Token endpoint for IndieAuth
- Token storage and validation
- Scope enforcement
- Authorization code generation and storage
- Token generation with SHA256 hashing
- Scope enforcement (create only for V1)
- Integration with existing notes CRUD
- Property mapping (content, title, tags)
- Optional PKCE support
### Out of Scope for V1 (Future)
### Out of Scope for V1 (Post-V1 Roadmap)
❌ **Nice to Have but Not Required:**
❌ **Deferred to Post-V1:**
- Update operations (action=update)
- Delete operations (action=delete)
- Media endpoint (file uploads)
- Photo/video properties (without media endpoint)
- Photo/video properties
- Syndication targets
- Complex post types (articles, replies, etc.)
- Batch operations
- Websub notifications
- Token introspection endpoint
- Token revocation endpoint
- Multiple scopes beyond "create"
## Security Considerations
@@ -677,17 +673,28 @@ The complete authorization flow for Micropub:
### Scope Enforcement
```python
SCOPE_REQUIREMENTS = {
"create": ["create"],
"update": ["update"],
"delete": ["delete"],
"query": [] # Queries don't require specific scopes
}
# V1 only supports "create" scope
SUPPORTED_SCOPES = ["create"]
DEFAULT_SCOPE = "create"
def check_scope(required: str, granted: str) -> bool:
"""Check if granted scopes include required scope"""
if not granted:
# IndieAuth spec: no scope means no access
return False
granted_scopes = set(granted.split())
return required in granted_scopes
def validate_requested_scope(scope: str) -> str:
"""Validate and filter requested scopes to supported ones"""
if not scope:
return "" # Empty scope allowed during authorization
requested = set(scope.split())
supported = set(SUPPORTED_SCOPES)
valid_scopes = requested & supported
return " ".join(valid_scopes) if valid_scopes else ""
```
### Input Validation
@@ -892,20 +899,20 @@ TOKEN_ENDPOINT = "/auth/token"
TOKEN_EXPIRY_DAYS = 90
AUTHORIZATION_CODE_EXPIRY_MINUTES = 10
# Supported scopes
SUPPORTED_SCOPES = ["create", "update", "delete", "read"]
# Supported scopes (V1 only supports create)
SUPPORTED_SCOPES = ["create"]
DEFAULT_SCOPE = "create"
```
## Effort Estimates
### Total Effort: 8-10 days
### Total Effort: 6-8 days (Reduced for V1 scope)
| Phase | Task | Effort | Priority |
|-------|------|--------|----------|
| 1 | Token Management | 2-3 days | Critical |
| 2 | Micropub Core | 3-4 days | Critical |
| 3 | Extended Operations | 2-3 days | High |
| 2 | Micropub Core (Create Only) | 2-3 days | Critical |
| 3 | Query Endpoints & Validation | 1-2 days | High |
| 4 | Integration & Polish | 1-2 days | High |
### Resource Requirements
@@ -1009,7 +1016,9 @@ Content-Type: application/json
```http
POST /micropub HTTP/1.1
Host: starpunk.example.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
Content-Type: application/json
```
Response (400 Bad Request):
@@ -1020,16 +1029,21 @@ Content-Type: application/json
}
```
action=delete&
url=https://starpunk.example.com/notes/2024/11/test-post
```
#### Attempting Delete (Returns Error)
### Querying Configuration
```http
POST /micropub HTTP/1.1
Host: starpunk.example.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
Content-Type: application/x-www-form-urlencoded
action=delete&
url=https://starpunk.example.com/notes/2024/11/test-post
```
Response (400 Bad Request):
```json
{
"error": "invalid_request",
"error_description": "Delete action not supported in V1"
}
@@ -1041,6 +1055,14 @@ action=delete&
GET /micropub?q=config HTTP/1.1
Host: starpunk.example.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
```
Response:
```json
{
"media-endpoint": null,
"syndicate-to": [],
"post-types": [
{
"type": "note",
"name": "Note",

307
docs/design/token-security-migration.md Normal file
View File

@@ -0,0 +1,307 @@
# Token Security Migration Strategy
## Overview
This document outlines the migration strategy for fixing the critical security issue where access tokens are stored in plain text in the database. This migration will invalidate all existing tokens as a necessary security measure.
## Security Issue
**Current State**: The `tokens` table stores tokens in plain text, which is a major security vulnerability. If the database is compromised, all tokens are immediately usable by an attacker.
**Target State**: Store only SHA256 hashes of tokens, making stolen database contents useless without the original tokens.
## Migration Plan
### Phase 1: Database Schema Migration
#### Migration Script (`migrations/005_token_security.sql`)
```sql
-- Migration: Fix token security and add Micropub support
-- Version: 0.10.0
-- Breaking Change: This will invalidate all existing tokens
-- Step 1: Create new secure tokens table
CREATE TABLE tokens_secure (
id INTEGER PRIMARY KEY AUTOINCREMENT,
token_hash TEXT UNIQUE NOT NULL, -- SHA256 hash of token
me TEXT NOT NULL, -- User identity URL
client_id TEXT, -- Client application URL
scope TEXT DEFAULT 'create', -- Granted scopes
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL, -- Token expiration
last_used_at TIMESTAMP, -- Track usage
revoked_at TIMESTAMP -- Soft revocation
);
-- Step 2: Create indexes for performance
CREATE INDEX idx_tokens_secure_hash ON tokens_secure(token_hash);
CREATE INDEX idx_tokens_secure_me ON tokens_secure(me);
CREATE INDEX idx_tokens_secure_expires ON tokens_secure(expires_at);
-- Step 3: Create authorization_codes table for Micropub
CREATE TABLE authorization_codes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
code_hash TEXT UNIQUE NOT NULL, -- SHA256 hash of code
me TEXT NOT NULL, -- User identity
client_id TEXT NOT NULL, -- Client application
redirect_uri TEXT NOT NULL, -- Callback URL
scope TEXT, -- Requested scopes
state TEXT, -- CSRF state
code_challenge TEXT, -- PKCE challenge
code_challenge_method TEXT, -- PKCE method
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL, -- 10 minute expiry
used_at TIMESTAMP -- Prevent replay
);
-- Step 4: Create indexes for authorization codes
CREATE INDEX idx_auth_codes_hash ON authorization_codes(code_hash);
CREATE INDEX idx_auth_codes_expires ON authorization_codes(expires_at);
-- Step 5: Drop old insecure tokens table
-- WARNING: This will invalidate all existing tokens
DROP TABLE IF EXISTS tokens;
-- Step 6: Rename secure table to final name
ALTER TABLE tokens_secure RENAME TO tokens;
-- Step 7: Clean up expired auth state
DELETE FROM auth_state WHERE expires_at < datetime('now');
```
### Phase 2: Code Implementation
#### Token Generation and Storage
```python
# starpunk/tokens.py
import hashlib
import secrets
from datetime import datetime, timedelta
def generate_token() -> str:
"""Generate cryptographically secure random token"""
return secrets.token_urlsafe(32)
def hash_token(token: str) -> str:
"""Create SHA256 hash of token"""
return hashlib.sha256(token.encode()).hexdigest()
def create_access_token(me: str, client_id: str, scope: str, db) -> str:
"""
Create new access token and store hash in database
Returns:
Plain text token (only returned once, never stored)
"""
token = generate_token()
token_hash = hash_token(token)
expires_at = datetime.now() + timedelta(days=90)
db.execute("""
INSERT INTO tokens (token_hash, me, client_id, scope, expires_at)
VALUES (?, ?, ?, ?, ?)
""", (token_hash, me, client_id, scope, expires_at))
db.commit()
return token # Return plain text to user ONCE
def verify_token(token: str, db) -> dict:
"""
Verify token by comparing hash
Returns:
Token info if valid, None if invalid/expired
"""
token_hash = hash_token(token)
row = db.execute("""
SELECT me, client_id, scope
FROM tokens
WHERE token_hash = ?
AND expires_at > datetime('now')
AND revoked_at IS NULL
""", (token_hash,)).fetchone()
if row:
# Update last used timestamp
db.execute("""
UPDATE tokens
SET last_used_at = datetime('now')
WHERE token_hash = ?
""", (token_hash,))
db.commit()
return dict(row)
return None
```
### Phase 3: Migration Execution
#### Step-by-Step Process
1. **Backup Database**
```bash
cp data/starpunk.db data/starpunk.db.backup-$(date +%Y%m%d)
```
2. **Notify Users** (if applicable)
- Email or announcement about token invalidation
- Explain security improvement
- Provide re-authentication instructions
3. **Apply Migration**
```python
# In starpunk/migrations.py
def run_migration_005(conn):
"""Apply token security migration"""
with open('migrations/005_token_security.sql', 'r') as f:
conn.executescript(f.read())
conn.commit()
```
4. **Update Code**
- Deploy new token handling code
- Update all token verification points
- Add proper error messages
5. **Test Migration**
```python
# Verify new schema
cursor = conn.execute("PRAGMA table_info(tokens)")
columns = {col[1] for col in cursor.fetchall()}
assert 'token_hash' in columns
assert 'token' not in columns # Old column gone
# Test token operations
token = create_access_token("https://user.example", "app", "create", conn)
assert verify_token(token, conn) is not None
assert verify_token("invalid", conn) is None
```
### Phase 4: Post-Migration Validation
#### Security Checklist
- [ ] Verify no plain text tokens in database
- [ ] Confirm all tokens are hashed with SHA256
- [ ] Test token creation returns plain text once
- [ ] Test token verification works with hash
- [ ] Verify expired tokens are rejected
- [ ] Check revoked tokens are rejected
- [ ] Audit logs show migration completed
#### Functional Testing
- [ ] Micropub client can obtain new token
- [ ] New tokens work for API requests
- [ ] Invalid tokens return 401 Unauthorized
- [ ] Token expiry is enforced
- [ ] Last used timestamp updates
## Rollback Plan
If critical issues arise:
1. **Restore Database**
```bash
cp data/starpunk.db.backup-YYYYMMDD data/starpunk.db
```
2. **Revert Code**
```bash
git revert <migration-commit>
```
3. **Investigate Issues**
- Review migration logs
- Test in development environment
- Fix issues before retry
## User Communication
### Pre-Migration Notice
```
Subject: Important Security Update - Token Re-authentication Required
Dear StarPunk User,
We're implementing an important security update that will require you to
re-authenticate any Micropub clients you use with StarPunk.
What's Changing:
- Enhanced token security (SHA256 hashing)
- All existing access tokens will be invalidated
- You'll need to re-authorize Micropub clients
When:
- [Date and time of migration]
What You Need to Do:
1. After the update, go to your Micropub client
2. Remove and re-add your StarPunk site
3. Complete the authorization flow again
This change significantly improves the security of your StarPunk installation.
Thank you for your understanding.
```
### Post-Migration Notice
```
Subject: Security Update Complete - Please Re-authenticate
The security update has been completed successfully. All previous access
tokens have been invalidated for security reasons.
To continue using Micropub clients:
1. Open your Micropub client (Quill, Indigenous, etc.)
2. Remove your StarPunk site if listed
3. Add it again and complete authorization
4. You're ready to post!
If you experience any issues, please contact support.
```
## Timeline
| Phase | Duration | Description |
|-------|----------|-------------|
| Preparation | 1 day | Create migration scripts, test in dev |
| Communication | 1 day | Notify users of upcoming change |
| Migration | 2 hours | Apply migration, deploy code |
| Validation | 2 hours | Test and verify success |
| Support | 1 week | Help users re-authenticate |
## Risk Assessment
| Risk | Impact | Mitigation |
|------|--------|------------|
| Data loss | High | Full backup before migration |
| User disruption | Medium | Clear communication, documentation |
| Migration failure | Low | Test in dev, have rollback plan |
| Performance impact | Low | Indexes on hash columns |
## Long-term Benefits
1. **Security**: Compromised database doesn't expose usable tokens
2. **Compliance**: Follows security best practices
3. **Auditability**: Can track token usage via last_used_at
4. **Revocability**: Can revoke tokens without deletion
5. **Foundation**: Proper structure for OAuth/IndieAuth
## Conclusion
While this migration will cause temporary disruption by invalidating existing tokens, it's a necessary security improvement that brings StarPunk in line with security best practices. The migration is straightforward, well-tested, and includes comprehensive rollback procedures if needed.
---
**Document Version**: 1.0
**Created**: 2024-11-24
**Author**: StarPunk Architecture Team
**Related**: ADR-029 (IndieAuth Integration)

116
docs/projectplan/v1/implementation-plan.md
View File

@@ -1278,6 +1278,122 @@ Final steps before V1 release.
---
## Post-V1 Roadmap
### Phase 11: Micropub Extended Operations (V1.1)
**Priority**: HIGH for V1.1 release
**Estimated Effort**: 4-6 hours
**Dependencies**: Phase 6 (Micropub Core) must be complete
#### 11.1 Update Operations
- [ ] Implement `action=update` handler in `/micropub`
- Support replace operations (replace entire property)
- Support add operations (append to array properties)
- Support delete operations (remove from array properties)
- Map Micropub properties to StarPunk note fields
- Validate URL belongs to this StarPunk instance
- **Acceptance Criteria**: Can update posts via Micropub clients
#### 11.2 Delete Operations
- [ ] Implement `action=delete` handler in `/micropub`
- Soft delete implementation (set deleted_at timestamp)
- URL validation and slug extraction
- Authorization check (delete scope required)
- Proper 204 No Content response
- **Acceptance Criteria**: Can delete posts via Micropub clients
#### 11.3 Extended Scopes
- [ ] Add "update" and "delete" to SUPPORTED_SCOPES
- [ ] Update authorization form to display requested scopes
- [ ] Implement scope-specific permission checks
- [ ] Update token endpoint to validate extended scopes
- [ ] **Acceptance Criteria**: Fine-grained permission control
### Phase 12: Media Endpoint (V1.2)
**Priority**: MEDIUM for V1.2 release
**Estimated Effort**: 6-8 hours
**Dependencies**: Micropub core functionality
#### 12.1 Media Upload Endpoint
- [ ] Create `/micropub/media` endpoint
- [ ] Handle multipart/form-data file uploads
- [ ] Store files in `/data/media/YYYY/MM/` structure
- [ ] Generate unique filenames to prevent collisions
- [ ] Image optimization (resize, compress)
- [ ] Return 201 Created with Location header
- [ ] **Acceptance Criteria**: Can upload images via Micropub clients
#### 12.2 Media in Posts
- [ ] Support photo property in Micropub create/update
- [ ] Embed images in Markdown content
- [ ] Update templates to display images properly
- [ ] Add media-endpoint to Micropub config query
- [ ] **Acceptance Criteria**: Posts can include images
### Phase 13: Advanced IndieWeb Features (V2.0)
**Priority**: LOW - Future enhancement
**Estimated Effort**: 10-15 hours per feature
**Dependencies**: All V1.x features complete
#### 13.1 Webmentions
- [ ] Receive webmentions at `/webmention` endpoint
- [ ] Verify source links to target
- [ ] Extract microformats from source
- [ ] Store webmentions in database
- [ ] Display webmentions on posts
- [ ] Send webmentions on publish
- [ ] Moderation interface in admin
#### 13.2 Syndication (POSSE)
- [ ] Add syndication targets configuration
- [ ] Support mp-syndicate-to in Micropub
- [ ] Implement Mastodon syndication
- [ ] Implement Twitter/X syndication (if API available)
- [ ] Store syndication URLs in post metadata
- [ ] Display syndication links on posts
#### 13.3 IndieAuth Server
- [ ] Implement full authorization server
- [ ] Allow StarPunk to be identity provider
- [ ] Profile URL verification
- [ ] Client registration/discovery
- [ ] Token introspection endpoint
- [ ] Token revocation endpoint
- [ ] Refresh tokens support
### Phase 14: Enhanced Features (V2.0+)
**Priority**: LOW - Long-term vision
**Estimated Effort**: Variable
#### 14.1 Multiple Post Types
- [ ] Articles (long-form with title)
- [ ] Replies (in-reply-to support)
- [ ] Likes (like-of property)
- [ ] Bookmarks (bookmark-of property)
- [ ] Events (h-event microformat)
- [ ] Check-ins (location data)
#### 14.2 Multi-User Support
- [ ] User registration system
- [ ] Per-user permissions and roles
- [ ] Separate author feeds (/author/username)
- [ ] Multi-author Micropub (me verification)
- [ ] User profile pages
#### 14.3 Advanced UI Features
- [ ] WYSIWYG Markdown editor
- [ ] Draft/schedule posts
- [ ] Batch operations interface
- [ ] Analytics dashboard
- [ ] Theme customization
- [ ] Plugin system
---
## Summary Checklist
### Core Features (Must Have)