phil/StarPunk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Address Micropub design issues and clarify V1 scope

Browse Source 
- Create ADR-029 for IndieAuth/Micropub integration strategy
- Address all critical issues from developer review:
  - Add missing 'me' parameter to token endpoint
  - Clarify PKCE as optional extension
  - Define token security migration strategy
  - Add authorization_codes table schema
  - Define property mapping rules
  - Clarify two authentication flows
- Simplify V1 scope per user decision:
  - Remove update/delete operations from V1
  - Focus on create-only functionality
  - Reduce timeline from 8-10 to 6-8 days
- Update project plan with post-V1 roadmap:
  - Phase 11: Update/delete operations (V1.1)
  - Phase 12: Media endpoint (V1.2)
  - Phase 13: Advanced IndieWeb features (V2.0)
  - Phase 14: Enhanced features (V2.0+)
- Create token security migration documentation
- Update ADR-028 for consistency with simplified scope

BREAKING CHANGE: Token migration will invalidate all existing tokens for security
This commit is contained in:
Phil Skentelbery
2025-11-24 11:39:13 -07:00
parent 5bbecad01d
commit dca9604746
6 changed files with 1127 additions and 129 deletions
Download Patch File Download Diff File Expand all files Collapse all files

266
docs/design/micropub-endpoint-design.md
View File

@@ -2,7 +2,7 @@
## Executive Summary
This document defines the architecture and implementation plan for adding Micropub support to StarPunk, the **critical blocker for V1 release**. The Micropub endpoint will enable external clients (like Indigenous, Quill, or Micropublish.net) to create, update, and delete posts on StarPunk installations.
This document defines the architecture and implementation plan for adding Micropub support to StarPunk, the **critical blocker for V1 release**. The Micropub endpoint will enable external clients (like Indigenous, Quill, or Micropublish.net) to create posts on StarPunk installations.
**Current State:**
- ✅ IndieAuth authentication working (authorization endpoint with PKCE)
@@ -12,11 +12,11 @@ This document defines the architecture and implementation plan for adding Microp
- ❌ **No Micropub endpoint** (V1 blocker)
- ❌ **No token endpoint** (required for Micropub auth)
**Target State:**
- Full Micropub server implementation
**Target State for V1:**
- Micropub server implementation (create-only for V1)
- IndieAuth token endpoint for issuing access tokens
- Support for creating, updating, and deleting posts
- Minimal V1 implementation (no media endpoint initially)
- Support for creating posts via form-encoded and JSON
- Minimal V1 implementation (no media endpoint, no update/delete)
## Architecture Overview
@@ -93,7 +93,8 @@ grant_type=authorization_code&
code={authorization_code}&
client_id={client_url}&
redirect_uri={redirect_url}&
code_verifier={pkce_verifier}
me={user_profile_url}&
code_verifier={pkce_verifier} # Optional (if PKCE was used)
```
#### Response Format
@@ -173,24 +174,51 @@ def verify_token(token: str) -> Optional[dict]:
#### Database Schema Update
```sql
-- Update existing tokens table to use token_hash
ALTER TABLE tokens RENAME TO tokens_old;
-- SECURITY FIX: Migrate to hashed token storage
-- Note: This will invalidate all existing tokens
CREATE TABLE tokens (
-- Step 1: Create new secure tokens table
CREATE TABLE tokens_secure (
id INTEGER PRIMARY KEY AUTOINCREMENT,
token_hash TEXT UNIQUE NOT NULL, -- SHA256 hash of token
me TEXT NOT NULL,
client_id TEXT,
scope TEXT,
scope TEXT DEFAULT 'create', -- Default scope for V1
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
last_used_at TIMESTAMP,
revoked_at TIMESTAMP
);
-- Step 2: Drop old insecure table (invalidates existing tokens)
DROP TABLE IF EXISTS tokens;
-- Step 3: Rename to final name
ALTER TABLE tokens_secure RENAME TO tokens;
-- Step 4: Create indexes
CREATE INDEX idx_tokens_hash ON tokens(token_hash);
CREATE INDEX idx_tokens_me ON tokens(me);
CREATE INDEX idx_tokens_expires ON tokens(expires_at);
-- Step 5: Create authorization_codes table
CREATE TABLE authorization_codes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
code_hash TEXT UNIQUE NOT NULL, -- SHA256 hash for security
me TEXT NOT NULL,
client_id TEXT NOT NULL,
redirect_uri TEXT NOT NULL,
scope TEXT,
state TEXT,
code_challenge TEXT, -- Optional PKCE
code_challenge_method TEXT,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL,
used_at TIMESTAMP -- Prevent replay attacks
);
CREATE INDEX idx_auth_codes_hash ON authorization_codes(code_hash);
CREATE INDEX idx_auth_codes_expires ON authorization_codes(expires_at);
```
### 2. Micropub Endpoint (`/micropub`)
@@ -247,9 +275,11 @@ def micropub_endpoint():
if action == "create":
return handle_create(data, token_info)
elif action == "update":
return handle_update(data, token_info)
# V1: Update not supported
return error_response("invalid_request", "Update action not supported in V1"), 400
elif action == "delete":
return handle_delete(data, token_info)
# V1: Delete not supported
return error_response("invalid_request", "Delete action not supported in V1"), 400
else:
return error_response("invalid_request", f"Unknown action: {action}"), 400
except MicropubError as e:
@@ -334,70 +364,32 @@ def normalize_properties(data: dict) -> dict:
properties[clean_key] = value
return properties
def extract_content(properties: dict) -> str:
"""Extract content from Micropub properties"""
content = properties.get('content', [''])[0]
if not content:
raise MicropubError("Content is required")
return content
def extract_title(properties: dict) -> str:
"""Extract title from Micropub properties"""
# Use 'name' property if provided
title = properties.get('name', [''])[0]
# If no name, extract from content (first line, max 50 chars)
if not title:
content = properties.get('content', [''])[0]
if content:
first_line = content.split('\n')[0]
title = first_line[:50] + ('...' if len(first_line) > 50 else '')
return title
```
#### Update and Delete Handlers
#### V1 Limitations
```python
def handle_update(data: dict, token_info: dict) -> tuple:
"""Handle post updates"""
if "update" not in token_info.get("scope", ""):
return error_response("insufficient_scope", "Token lacks update scope"), 403
url = data.get("url")
if not url:
return error_response("invalid_request", "No URL provided"), 400
# Extract slug from URL
slug = extract_slug_from_url(url)
# Get existing note
from starpunk.notes import get_note, update_note
note = get_note(slug)
if not note:
return error_response("invalid_request", "Post not found"), 400
# Apply updates based on operation type
if "replace" in data:
# Replace properties
for prop, values in data["replace"].items():
apply_property_replace(note, prop, values)
if "add" in data:
# Add to properties
for prop, values in data["add"].items():
apply_property_add(note, prop, values)
if "delete" in data:
# Delete properties or values
for prop, values in data["delete"].items():
apply_property_delete(note, prop, values)
# Save changes
update_note(note)
return "", 204 # No Content
def handle_delete(data: dict, token_info: dict) -> tuple:
"""Handle post deletion"""
if "delete" not in token_info.get("scope", ""):
return error_response("insufficient_scope", "Token lacks delete scope"), 403
url = data.get("url")
if not url:
return error_response("invalid_request", "No URL provided"), 400
# Extract slug and delete
slug = extract_slug_from_url(url)
from starpunk.notes import delete_note
try:
delete_note(slug, soft=True) # Soft delete for safety
return "", 204
except NoteNotFoundError:
return error_response("invalid_request", "Post not found"), 400
```
In V1, only the `create` action is supported. Update and delete operations will return an error response and are planned for a post-V1 release.
### 3. Query Endpoints
@@ -472,8 +464,9 @@ The complete authorization flow for Micropub:
│ client_id=https://client.example&
│ redirect_uri=https://client.example/callback&
│ state=1234567890&
│ scope=create+update+delete&
│ code_challenge=XXXXX&
│ scope=create& # V1: create only
│ me=https://user.example&
│ code_challenge=XXXXX& # Optional PKCE
│ code_challenge_method=S256
▼
┌──────────────┐
@@ -481,10 +474,11 @@ The complete authorization flow for Micropub:
│ (Authorize) │
└─────┬────────┘
│
│ 2. Redirect to IndieLogin.com for authentication
│ (existing PKCE flow)
│ 2. Check admin session
│ If not logged in: redirect to /auth/login
│ If logged in: show authorization form
│
│ 3. After successful auth, redirect back with code
│ 3. User approves, generate authorization code
│
▼
┌──────────┐
@@ -496,7 +490,8 @@ The complete authorization flow for Micropub:
│ code=XXXXX&
│ client_id=https://client.example&
│ redirect_uri=https://client.example/callback&
│ code_verifier=XXXXX
│ me=https://user.example&
│ code_verifier=XXXXX # If PKCE was used
▼
┌──────────────┐
│ StarPunk │
@@ -507,7 +502,7 @@ The complete authorization flow for Micropub:
│ {
│ "access_token": "XXXXX",
│ "token_type": "Bearer",
│ "scope": "create update delete",
│ "scope": "create", # V1: create only
│ "me": "https://user.example"
│ }
▼
@@ -579,28 +574,22 @@ The complete authorization flow for Micropub:
- Invalid token handling
- Scope validation
### Phase 3: Extended Operations (2-3 days)
### Phase 3: Query Endpoints & Validation (1-2 days)
1. **Update Handler**
- Replace operations
- Add operations
- Delete operations
- Property mapping
2. **Delete Handler**
- URL parsing
- Soft deletion
- Response handling
3. **Query Endpoints**
1. **Query Endpoints**
- Config endpoint
- Source endpoint
- Syndicate-to endpoint
- Syndicate-to endpoint (empty for V1)
4. **Testing**
- Update operations
- Delete operations
- Query responses
2. **Property Validation**
- Content extraction and validation
- Title generation from content
- Tag/category mapping
3. **Testing**
- Query endpoint responses
- Property mapping edge cases
- Error response formatting
### Phase 4: Integration & Polish (1-2 days)
@@ -635,24 +624,31 @@ The complete authorization flow for Micropub:
- Return 201 Created with Location header
- Configuration query endpoint
- Source query endpoint
- Authorization endpoint (`/auth/authorization`)
- Token endpoint (`/auth/token`)
✅ **Required for Functionality:**
- Token endpoint for IndieAuth
- Token storage and validation
- Scope enforcement
- Authorization code generation and storage
- Token generation with SHA256 hashing
- Scope enforcement (create only for V1)
- Integration with existing notes CRUD
- Property mapping (content, title, tags)
- Optional PKCE support
### Out of Scope for V1 (Future)
### Out of Scope for V1 (Post-V1 Roadmap)
❌ **Nice to Have but Not Required:**
❌ **Deferred to Post-V1:**
- Update operations (action=update)
- Delete operations (action=delete)
- Media endpoint (file uploads)
- Photo/video properties (without media endpoint)
- Photo/video properties
- Syndication targets
- Complex post types (articles, replies, etc.)
- Batch operations
- Websub notifications
- Token introspection endpoint
- Token revocation endpoint
- Multiple scopes beyond "create"
## Security Considerations
@@ -677,17 +673,28 @@ The complete authorization flow for Micropub:
### Scope Enforcement
```python
SCOPE_REQUIREMENTS = {
"create": ["create"],
"update": ["update"],
"delete": ["delete"],
"query": [] # Queries don't require specific scopes
}
# V1 only supports "create" scope
SUPPORTED_SCOPES = ["create"]
DEFAULT_SCOPE = "create"
def check_scope(required: str, granted: str) -> bool:
"""Check if granted scopes include required scope"""
if not granted:
# IndieAuth spec: no scope means no access
return False
granted_scopes = set(granted.split())
return required in granted_scopes
def validate_requested_scope(scope: str) -> str:
"""Validate and filter requested scopes to supported ones"""
if not scope:
return "" # Empty scope allowed during authorization
requested = set(scope.split())
supported = set(SUPPORTED_SCOPES)
valid_scopes = requested & supported
return " ".join(valid_scopes) if valid_scopes else ""
```
### Input Validation
@@ -892,20 +899,20 @@ TOKEN_ENDPOINT = "/auth/token"
TOKEN_EXPIRY_DAYS = 90
AUTHORIZATION_CODE_EXPIRY_MINUTES = 10
# Supported scopes
SUPPORTED_SCOPES = ["create", "update", "delete", "read"]
# Supported scopes (V1 only supports create)
SUPPORTED_SCOPES = ["create"]
DEFAULT_SCOPE = "create"
```
## Effort Estimates
### Total Effort: 8-10 days
### Total Effort: 6-8 days (Reduced for V1 scope)
| Phase | Task | Effort | Priority |
|-------|------|--------|----------|
| 1 | Token Management | 2-3 days | Critical |
| 2 | Micropub Core | 3-4 days | Critical |
| 3 | Extended Operations | 2-3 days | High |
| 2 | Micropub Core (Create Only) | 2-3 days | Critical |
| 3 | Query Endpoints & Validation | 1-2 days | High |
| 4 | Integration & Polish | 1-2 days | High |
### Resource Requirements
@@ -1009,7 +1016,9 @@ Content-Type: application/json
```http
POST /micropub HTTP/1.1
Host: starpunk.example.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
Content-Type: application/json
```
Response (400 Bad Request):
@@ -1020,16 +1029,21 @@ Content-Type: application/json
}
```
action=delete&
url=https://starpunk.example.com/notes/2024/11/test-post
```
#### Attempting Delete (Returns Error)
### Querying Configuration
```http
POST /micropub HTTP/1.1
Host: starpunk.example.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
Content-Type: application/x-www-form-urlencoded
action=delete&
url=https://starpunk.example.com/notes/2024/11/test-post
```
Response (400 Bad Request):
```json
{
"error": "invalid_request",
"error_description": "Delete action not supported in V1"
}
@@ -1041,6 +1055,14 @@ action=delete&
GET /micropub?q=config HTTP/1.1
Host: starpunk.example.com
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc...
```
Response:
```json
{
"media-endpoint": null,
"syndicate-to": [],
"post-types": [
{
"type": "note",
"name": "Note",

307
docs/design/token-security-migration.md Normal file
View File

@@ -0,0 +1,307 @@
# Token Security Migration Strategy
## Overview
This document outlines the migration strategy for fixing the critical security issue where access tokens are stored in plain text in the database. This migration will invalidate all existing tokens as a necessary security measure.
## Security Issue
**Current State**: The `tokens` table stores tokens in plain text, which is a major security vulnerability. If the database is compromised, all tokens are immediately usable by an attacker.
**Target State**: Store only SHA256 hashes of tokens, making stolen database contents useless without the original tokens.
## Migration Plan
### Phase 1: Database Schema Migration
#### Migration Script (`migrations/005_token_security.sql`)
```sql
-- Migration: Fix token security and add Micropub support
-- Version: 0.10.0
-- Breaking Change: This will invalidate all existing tokens
-- Step 1: Create new secure tokens table
CREATE TABLE tokens_secure (
id INTEGER PRIMARY KEY AUTOINCREMENT,
token_hash TEXT UNIQUE NOT NULL, -- SHA256 hash of token
me TEXT NOT NULL, -- User identity URL
client_id TEXT, -- Client application URL
scope TEXT DEFAULT 'create', -- Granted scopes
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL, -- Token expiration
last_used_at TIMESTAMP, -- Track usage
revoked_at TIMESTAMP -- Soft revocation
);
-- Step 2: Create indexes for performance
CREATE INDEX idx_tokens_secure_hash ON tokens_secure(token_hash);
CREATE INDEX idx_tokens_secure_me ON tokens_secure(me);
CREATE INDEX idx_tokens_secure_expires ON tokens_secure(expires_at);
-- Step 3: Create authorization_codes table for Micropub
CREATE TABLE authorization_codes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
code_hash TEXT UNIQUE NOT NULL, -- SHA256 hash of code
me TEXT NOT NULL, -- User identity
client_id TEXT NOT NULL, -- Client application
redirect_uri TEXT NOT NULL, -- Callback URL
scope TEXT, -- Requested scopes
state TEXT, -- CSRF state
code_challenge TEXT, -- PKCE challenge
code_challenge_method TEXT, -- PKCE method
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL, -- 10 minute expiry
used_at TIMESTAMP -- Prevent replay
);
-- Step 4: Create indexes for authorization codes
CREATE INDEX idx_auth_codes_hash ON authorization_codes(code_hash);
CREATE INDEX idx_auth_codes_expires ON authorization_codes(expires_at);
-- Step 5: Drop old insecure tokens table
-- WARNING: This will invalidate all existing tokens
DROP TABLE IF EXISTS tokens;
-- Step 6: Rename secure table to final name
ALTER TABLE tokens_secure RENAME TO tokens;
-- Step 7: Clean up expired auth state
DELETE FROM auth_state WHERE expires_at < datetime('now');
```
### Phase 2: Code Implementation
#### Token Generation and Storage
```python
# starpunk/tokens.py
import hashlib
import secrets
from datetime import datetime, timedelta
def generate_token() -> str:
"""Generate cryptographically secure random token"""
return secrets.token_urlsafe(32)
def hash_token(token: str) -> str:
"""Create SHA256 hash of token"""
return hashlib.sha256(token.encode()).hexdigest()
def create_access_token(me: str, client_id: str, scope: str, db) -> str:
"""
Create new access token and store hash in database
Returns:
Plain text token (only returned once, never stored)
"""
token = generate_token()
token_hash = hash_token(token)
expires_at = datetime.now() + timedelta(days=90)
db.execute("""
INSERT INTO tokens (token_hash, me, client_id, scope, expires_at)
VALUES (?, ?, ?, ?, ?)
""", (token_hash, me, client_id, scope, expires_at))
db.commit()
return token # Return plain text to user ONCE
def verify_token(token: str, db) -> dict:
"""
Verify token by comparing hash
Returns:
Token info if valid, None if invalid/expired
"""
token_hash = hash_token(token)
row = db.execute("""
SELECT me, client_id, scope
FROM tokens
WHERE token_hash = ?
AND expires_at > datetime('now')
AND revoked_at IS NULL
""", (token_hash,)).fetchone()
if row:
# Update last used timestamp
db.execute("""
UPDATE tokens
SET last_used_at = datetime('now')
WHERE token_hash = ?
""", (token_hash,))
db.commit()
return dict(row)
return None
```
### Phase 3: Migration Execution
#### Step-by-Step Process
1. **Backup Database**
```bash
cp data/starpunk.db data/starpunk.db.backup-$(date +%Y%m%d)
```
2. **Notify Users** (if applicable)
- Email or announcement about token invalidation
- Explain security improvement
- Provide re-authentication instructions
3. **Apply Migration**
```python
# In starpunk/migrations.py
def run_migration_005(conn):
"""Apply token security migration"""
with open('migrations/005_token_security.sql', 'r') as f:
conn.executescript(f.read())
conn.commit()
```
4. **Update Code**
- Deploy new token handling code
- Update all token verification points
- Add proper error messages
5. **Test Migration**
```python
# Verify new schema
cursor = conn.execute("PRAGMA table_info(tokens)")
columns = {col[1] for col in cursor.fetchall()}
assert 'token_hash' in columns
assert 'token' not in columns # Old column gone
# Test token operations
token = create_access_token("https://user.example", "app", "create", conn)
assert verify_token(token, conn) is not None
assert verify_token("invalid", conn) is None
```
### Phase 4: Post-Migration Validation
#### Security Checklist
- [ ] Verify no plain text tokens in database
- [ ] Confirm all tokens are hashed with SHA256
- [ ] Test token creation returns plain text once
- [ ] Test token verification works with hash
- [ ] Verify expired tokens are rejected
- [ ] Check revoked tokens are rejected
- [ ] Audit logs show migration completed
#### Functional Testing
- [ ] Micropub client can obtain new token
- [ ] New tokens work for API requests
- [ ] Invalid tokens return 401 Unauthorized
- [ ] Token expiry is enforced
- [ ] Last used timestamp updates
## Rollback Plan
If critical issues arise:
1. **Restore Database**
```bash
cp data/starpunk.db.backup-YYYYMMDD data/starpunk.db
```
2. **Revert Code**
```bash
git revert <migration-commit>
```
3. **Investigate Issues**
- Review migration logs
- Test in development environment
- Fix issues before retry
## User Communication
### Pre-Migration Notice
```
Subject: Important Security Update - Token Re-authentication Required
Dear StarPunk User,
We're implementing an important security update that will require you to
re-authenticate any Micropub clients you use with StarPunk.
What's Changing:
- Enhanced token security (SHA256 hashing)
- All existing access tokens will be invalidated
- You'll need to re-authorize Micropub clients
When:
- [Date and time of migration]
What You Need to Do:
1. After the update, go to your Micropub client
2. Remove and re-add your StarPunk site
3. Complete the authorization flow again
This change significantly improves the security of your StarPunk installation.
Thank you for your understanding.
```
### Post-Migration Notice
```
Subject: Security Update Complete - Please Re-authenticate
The security update has been completed successfully. All previous access
tokens have been invalidated for security reasons.
To continue using Micropub clients:
1. Open your Micropub client (Quill, Indigenous, etc.)
2. Remove your StarPunk site if listed
3. Add it again and complete authorization
4. You're ready to post!
If you experience any issues, please contact support.
```
## Timeline
| Phase | Duration | Description |
|-------|----------|-------------|
| Preparation | 1 day | Create migration scripts, test in dev |
| Communication | 1 day | Notify users of upcoming change |
| Migration | 2 hours | Apply migration, deploy code |
| Validation | 2 hours | Test and verify success |
| Support | 1 week | Help users re-authenticate |
## Risk Assessment
| Risk | Impact | Mitigation |
|------|--------|------------|
| Data loss | High | Full backup before migration |
| User disruption | Medium | Clear communication, documentation |
| Migration failure | Low | Test in dev, have rollback plan |
| Performance impact | Low | Indexes on hash columns |
## Long-term Benefits
1. **Security**: Compromised database doesn't expose usable tokens
2. **Compliance**: Follows security best practices
3. **Auditability**: Can track token usage via last_used_at
4. **Revocability**: Can revoke tokens without deletion
5. **Foundation**: Proper structure for OAuth/IndieAuth
## Conclusion
While this migration will cause temporary disruption by invalidating existing tokens, it's a necessary security improvement that brings StarPunk in line with security best practices. The migration is straightforward, well-tested, and includes comprehensive rollback procedures if needed.
---
**Document Version**: 1.0
**Created**: 2024-11-24
**Author**: StarPunk Architecture Team
**Related**: ADR-029 (IndieAuth Integration)