StarPunk/docs/design/project-structure.md

# Project Structure Design

## Purpose

This document defines the complete directory and file structure for the StarPunk project. It provides the exact layout that developer agents should create, including all directories, their purposes, file organization, and naming conventions.

## Philosophy

The project structure follows these principles:
- **Flat is better than nested**: Avoid deep directory hierarchies
- **Conventional is better than clever**: Use standard Python project layout
- **Obvious is better than hidden**: Clear directory names over abbreviations
- **Data is sacred**: User data isolated in dedicated directory

## Complete Directory Structure

```
starpunk/
├── .venv/                          # Python virtual environment (gitignored)
├── .env                            # Environment configuration (gitignored)
├── .env.example                    # Configuration template (committed)
├── .gitignore                      # Git ignore rules
├── README.md                       # Project documentation
├── CLAUDE.MD                       # AI agent requirements document
├── LICENSE                         # Project license
├── requirements.txt                # Python dependencies
├── requirements-dev.txt            # Development dependencies
│
├── app.py                          # Main application entry point
│
├── starpunk/                       # Application package
│   ├── __init__.py                # Package initialization
│   ├── config.py                  # Configuration management
│   ├── database.py                # Database operations
│   ├── models.py                  # Data models
│   ├── auth.py                    # Authentication logic
│   ├── micropub.py                # Micropub endpoint
│   ├── feed.py                    # RSS feed generation
│   ├── notes.py                   # Note management
│   └── utils.py                   # Helper functions
│
├── static/                         # Static assets
│   ├── css/
│   │   └── style.css              # Main stylesheet (~200 lines)
│   └── js/
│       └── preview.js             # Optional markdown preview
│
├── templates/                      # Jinja2 templates
│   ├── base.html                  # Base layout template
│   ├── index.html                 # Homepage (note list)
│   ├── note.html                  # Single note view
│   ├── feed.xml                   # RSS feed template
│   └── admin/
│       ├── base.html              # Admin base layout
│       ├── login.html             # Login form
│       ├── dashboard.html         # Admin dashboard
│       ├── new.html               # Create note form
│       └── edit.html              # Edit note form
│
├── data/                           # User data directory (gitignored)
│   ├── notes/                     # Markdown note files
│   │   ├── 2024/
│   │   │   ├── 11/
│   │   │   │   ├── first-note.md
│   │   │   │   └── second-note.md
│   │   │   └── 12/
│   │   │       └── december-note.md
│   │   └── 2025/
│   │       └── 01/
│   │           └── new-year.md
│   └── starpunk.db                # SQLite database
│
├── tests/                          # Test suite
│   ├── __init__.py
│   ├── conftest.py                # Pytest configuration and fixtures
│   ├── test_auth.py               # Authentication tests
│   ├── test_database.py           # Database tests
│   ├── test_micropub.py           # Micropub endpoint tests
│   ├── test_feed.py               # RSS feed tests
│   ├── test_notes.py              # Note management tests
│   └── test_utils.py              # Utility function tests
│
└── docs/                           # Architecture documentation
    ├── architecture/
    │   ├── overview.md
    │   ├── components.md
    │   ├── data-flow.md
    │   ├── security.md
    │   ├── deployment.md
    │   └── technology-stack.md
    ├── decisions/
    │   ├── ADR-001-python-web-framework.md
    │   ├── ADR-002-flask-extensions.md
    │   ├── ADR-003-frontend-technology.md
    │   ├── ADR-004-file-based-note-storage.md
    │   ├── ADR-005-indielogin-authentication.md
    │   └── ADR-006-python-virtual-environment-uv.md
    ├── design/
    │   ├── project-structure.md (this file)
    │   ├── database-schema.md
    │   ├── api-contracts.md
    │   ├── initial-files.md
    │   └── component-interfaces.md
    └── standards/
        ├── documentation-organization.md
        ├── python-coding-standards.md
        ├── development-setup.md
        └── testing-strategy.md
```

## Directory Purposes

### Root Directory (`/`)

**Purpose**: Project root containing configuration and entry point.

**Key Files**:
- `app.py` - Main Flask application (import app from starpunk package)
- `.env` - Environment variables (NEVER commit)
- `.env.example` - Template for configuration
- `requirements.txt` - Production dependencies
- `requirements-dev.txt` - Development tools
- `README.md` - User-facing documentation
- `CLAUDE.MD` - AI agent instructions
- `LICENSE` - Open source license

**Rationale**: Flat root with minimal files makes project easy to understand at a glance.

---

### Application Package (`starpunk/`)

**Purpose**: Core application code organized as Python package.

**Structure**: Flat module layout (no sub-packages in V1)

**Modules**:

| Module | Purpose | Approximate LOC |
|--------|---------|-----------------|
| `__init__.py` | Package init, create Flask app | 50 |
| `config.py` | Load configuration from .env | 75 |
| `database.py` | SQLite operations, schema management | 200 |
| `models.py` | Data models (Note, Session, Token) | 150 |
| `auth.py` | IndieAuth flow, session management | 200 |
| `micropub.py` | Micropub endpoint implementation | 250 |
| `feed.py` | RSS feed generation | 100 |
| `notes.py` | Note CRUD, file operations | 300 |
| `utils.py` | Slug generation, hashing, helpers | 100 |

**Total Estimated**: ~1,425 LOC for core application

**Naming Convention**:
- Lowercase with underscores
- Singular nouns for single-purpose modules (`config.py`, not `configs.py`)
- Plural for collections (`notes.py` manages many notes)
- Descriptive names over abbreviations (`database.py`, not `db.py`)

**Import Strategy**:
```python
# In app.py
from starpunk import create_app

app = create_app()
```

**Rationale**: Flat package structure is simpler than nested sub-packages. All modules are peers. No circular dependency issues.

---

### Static Assets (`static/`)

**Purpose**: CSS, JavaScript, and other static files served directly.

**Organization**:
```
static/
├── css/
│   └── style.css        # Single stylesheet
└── js/
    └── preview.js       # Optional markdown preview
```

**CSS Structure** (`style.css`):
- ~200 lines total
- CSS custom properties for theming
- Mobile-first responsive design
- Semantic HTML with minimal classes

**JavaScript Structure** (`preview.js`):
- Optional enhancement only
- Vanilla JavaScript (no frameworks)
- Markdown preview using marked.js from CDN
- Degrades gracefully if disabled

**URL Pattern**: `/static/{type}/{file}`
- Example: `/static/css/style.css`
- Example: `/static/js/preview.js`

**Future Assets** (V2+):
- Favicon: `static/favicon.ico`
- Icons: `static/icons/`
- Images: `static/images/` (if needed)

**Rationale**: Standard Flask static file convention. Simple, flat structure since we have minimal assets.

---

### Templates (`templates/`)

**Purpose**: Jinja2 HTML templates for server-side rendering.

**Organization**:
```
templates/
├── base.html              # Public site base layout
├── index.html             # Homepage (extends base.html)
├── note.html              # Single note (extends base.html)
├── feed.xml               # RSS feed (no layout)
└── admin/
    ├── base.html          # Admin base layout
    ├── login.html         # Login form
    ├── dashboard.html     # Admin dashboard
    ├── new.html           # Create note
    └── edit.html          # Edit note
```

**Template Hierarchy**:
```
base.html (public)
  ├── index.html (note list)
  └── note.html (single note)

admin/base.html
  ├── admin/dashboard.html
  ├── admin/new.html
  └── admin/edit.html

admin/login.html (no base, standalone)
feed.xml (no base, XML output)
```

**Naming Convention**:
- Lowercase with hyphens for multi-word names
- Descriptive names (`dashboard.html`, not `dash.html`)
- Base templates clearly named (`base.html`)

**Template Features**:
- Microformats2 markup (h-entry, h-card)
- Semantic HTML5
- Accessible markup (ARIA labels)
- Mobile-responsive
- Progressive enhancement

**Rationale**: Standard Flask template convention. Admin templates in subdirectory keeps them organized.

---

### Data Directory (`data/`)

**Purpose**: User-created content and database. This is the sacred directory.

**Structure**:
```
data/
├── notes/
│   └── {YYYY}/
│       └── {MM}/
│           └── {slug}.md
└── starpunk.db
```

**Notes Directory** (`data/notes/`):
- **Pattern**: Year/Month hierarchy (`YYYY/MM/`)
- **Files**: Markdown files with slug names
- **Example**: `data/notes/2024/11/my-first-note.md`

**Database File** (`data/starpunk.db`):
- SQLite database
- Contains metadata, sessions, tokens
- NOT content (content is in .md files)

**Permissions**:
- Directory: 755 (rwxr-xr-x)
- Files: 644 (rw-r--r--)
- Database: 644 (rw-r--r--)

**Backup Strategy**:
```bash
# Simple backup
tar -czf starpunk-backup-$(date +%Y%m%d).tar.gz data/

# Or rsync
rsync -av data/ /backup/starpunk/
```

**Gitignore**: ENTIRE `data/` directory must be gitignored.

**Rationale**: User data is completely isolated. Easy to backup. Portable across systems.

---

### Tests (`tests/`)

**Purpose**: Complete test suite using pytest.

**Organization**:
```
tests/
├── __init__.py           # Empty, marks as package
├── conftest.py           # Pytest fixtures and configuration
├── test_auth.py          # Authentication tests
├── test_database.py      # Database operations tests
├── test_micropub.py      # Micropub endpoint tests
├── test_feed.py          # RSS feed generation tests
├── test_notes.py         # Note management tests
└── test_utils.py         # Utility function tests
```

**Naming Convention**:
- All test files: `test_{module}.py`
- All test functions: `def test_{function_name}():`
- Fixtures in `conftest.py`

**Test Organization**:
- One test file per application module
- Integration tests in same file as unit tests
- Use fixtures for common setup (database, app context)

**Coverage Target**: >80% for V1

**Rationale**: Standard pytest convention. Easy to run all tests or specific modules.

---

### Documentation (`docs/`)

**Purpose**: Architecture, decisions, designs, and standards.

**Organization**: See [Documentation Organization Standard](/home/phil/Projects/starpunk/docs/standards/documentation-organization.md)

```
docs/
├── architecture/        # System-level architecture
├── decisions/          # ADRs (Architecture Decision Records)
├── design/             # Detailed technical designs
└── standards/          # Coding and process standards
```

**Key Documents**:
- `architecture/overview.md` - System architecture
- `architecture/technology-stack.md` - Complete tech stack
- `decisions/ADR-*.md` - All architectural decisions
- `design/project-structure.md` - This document
- `standards/python-coding-standards.md` - Code style

**Rationale**: Clear separation of document types. Easy to find relevant documentation.

---

## File Naming Conventions

### Python Files
- **Modules**: `lowercase_with_underscores.py`
- **Packages**: `lowercase` (no underscores if possible)
- **Classes**: `PascalCase` (in code, not filenames)
- **Functions**: `lowercase_with_underscores` (in code)

**Examples**:
```
starpunk/database.py         # Good
starpunk/Database.py         # Bad
starpunk/db.py              # Bad (use full word)
starpunk/note_manager.py     # Good if needed
```

### Markdown Files (Notes)
- **Pattern**: `{slug}.md`
- **Slug format**: `lowercase-with-hyphens`
- **Valid characters**: `a-z`, `0-9`, `-` (hyphen)
- **No spaces, no underscores, no special characters**

**Examples**:
```
first-note.md               # Good
my-thoughts-on-python.md    # Good
2024-11-18-daily-note.md    # Good (date prefix okay)
First Note.md               # Bad (spaces)
first_note.md               # Bad (underscores)
first-note.markdown         # Bad (use .md)
```

### Template Files
- **Pattern**: `lowercase.html` or `lowercase-name.html`
- **XML**: `.xml` extension for RSS feed

**Examples**:
```
base.html                   # Good
note.html                   # Good
admin/dashboard.html        # Good
admin/new-note.html         # Good (if multi-word)
admin/NewNote.html          # Bad
```

### Documentation Files
- **Pattern**: `lowercase-with-hyphens.md`
- **ADRs**: `ADR-{NNN}-{short-title}.md`

**Examples**:
```
project-structure.md        # Good
database-schema.md          # Good
ADR-001-python-web-framework.md  # Good
ProjectStructure.md         # Bad
project_structure.md        # Bad (use hyphens)
```

### Configuration Files
- **Pattern**: Standard conventions (`.env`, `requirements.txt`, etc.)

**Examples**:
```
.env                        # Good
.env.example                # Good
requirements.txt            # Good
requirements-dev.txt        # Good
.gitignore                  # Good
```

---

## Gitignore Requirements

The `.gitignore` file MUST include the following:

```gitignore
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python

# Virtual Environment
.venv/
venv/
env/
ENV/
.venv.*

# Environment Configuration
.env
*.env
!.env.example

# User Data (CRITICAL - NEVER COMMIT)
data/
*.db
*.sqlite
*.sqlite3

# IDE
.vscode/
.idea/
*.swp
*.swo
*~
.DS_Store

# Testing
.pytest_cache/
.coverage
htmlcov/
*.cover
.hypothesis/

# Distribution
dist/
build/
*.egg-info/
*.egg

# Logs
*.log
logs/
```

**Critical Rules**:
1. **NEVER commit `data/` directory** - contains user data
2. **NEVER commit `.env` file** - contains secrets
3. **NEVER commit `.venv/`** - virtual environment is local
4. **ALWAYS commit `.env.example`** - template is safe

---

## Directory Creation Order

When initializing project, create in this order:

1. **Root files**:
   ```bash
   touch .gitignore
   touch .env.example
   touch README.md
   touch CLAUDE.MD
   touch LICENSE
   touch requirements.txt
   touch requirements-dev.txt
   ```

2. **Application package**:
   ```bash
   mkdir -p starpunk
   touch starpunk/__init__.py
   ```

3. **Static assets**:
   ```bash
   mkdir -p static/css static/js
   touch static/css/style.css
   touch static/js/preview.js
   ```

4. **Templates**:
   ```bash
   mkdir -p templates/admin
   touch templates/base.html
   touch templates/index.html
   touch templates/note.html
   touch templates/feed.xml
   touch templates/admin/base.html
   touch templates/admin/login.html
   touch templates/admin/dashboard.html
   touch templates/admin/new.html
   touch templates/admin/edit.html
   ```

5. **Data directory** (created on first run):
   ```bash
   mkdir -p data/notes
   ```

6. **Tests**:
   ```bash
   mkdir -p tests
   touch tests/__init__.py
   touch tests/conftest.py
   ```

7. **Documentation** (mostly exists):
   ```bash
   mkdir -p docs/architecture docs/decisions docs/design docs/standards
   ```

---

## Path Standards

### Absolute vs Relative Paths

**In Code**:
- Use relative paths from project root
- Let Flask handle path resolution
- Use `pathlib.Path` for file operations

**In Configuration**:
- Support both absolute and relative paths
- Relative paths are relative to project root
- Document clearly in `.env.example`

**In Documentation**:
- Use absolute paths for clarity
- Example: `/home/phil/Projects/starpunk/data/notes`

**In Agent Operations**:
- ALWAYS use absolute paths
- Never rely on current working directory
- See ADR-006 for details

### Path Construction Examples

```python
from pathlib import Path

# Project root
PROJECT_ROOT = Path(__file__).parent.parent

# Data directory
DATA_DIR = PROJECT_ROOT / "data"
NOTES_DIR = DATA_DIR / "notes"
DB_PATH = DATA_DIR / "starpunk.db"

# Static files
STATIC_DIR = PROJECT_ROOT / "static"
CSS_DIR = STATIC_DIR / "css"

# Templates (handled by Flask)
TEMPLATE_DIR = PROJECT_ROOT / "templates"
```

---

## File Size Guidelines

### Target Sizes

| File Type | Target Size | Maximum Size | Rationale |
|-----------|-------------|--------------|-----------|
| Python module | 100-300 LOC | 500 LOC | Keep modules focused |
| Template | 50-100 lines | 200 lines | Use template inheritance |
| CSS | 200 LOC total | 300 LOC | Minimal styling only |
| JavaScript | 100 LOC | 200 LOC | Optional feature only |
| Markdown note | 50-500 words | No limit | User content |
| Test file | 100-200 LOC | 400 LOC | One module per file |

**If file exceeds maximum**: Consider splitting into multiple modules or refactoring.

---

## Module Organization

### starpunk/__init__.py

**Purpose**: Package initialization and Flask app factory.

**Contents**:
- `create_app()` function
- Blueprint registration
- Configuration loading

**Example**:
```python
from flask import Flask

def create_app():
    app = Flask(__name__)

    # Load configuration
    from starpunk.config import load_config
    load_config(app)

    # Initialize database
    from starpunk.database import init_db
    init_db(app)

    # Register blueprints
    from starpunk.routes import public, admin, api
    app.register_blueprint(public.bp)
    app.register_blueprint(admin.bp)
    app.register_blueprint(api.bp)

    return app
```

### app.py (Root)

**Purpose**: Application entry point for Flask.

**Contents**:
```python
from starpunk import create_app

app = create_app()

if __name__ == '__main__':
    app.run(debug=True)
```

**Rationale**: Minimal entry point. All logic in package.

---

## Import Organization

Follow PEP 8 import ordering:

1. Standard library imports
2. Third-party imports
3. Local application imports

**Example**:
```python
# Standard library
import os
import sqlite3
from pathlib import Path
from datetime import datetime

# Third-party
from flask import Flask, render_template
import markdown
import httpx

# Local
from starpunk.config import load_config
from starpunk.database import get_db
from starpunk.models import Note
```

---

## Future Structure Extensions (V2+)

### Potential Additions

**Media Uploads**:
```
data/
├── notes/
├── media/
│   └── {YYYY}/{MM}/
│       └── {filename}
└── starpunk.db
```

**Migrations**:
```
migrations/
├── 001_initial_schema.sql
├── 002_add_media_table.sql
└── 003_add_tags.sql
```

**Deployment**:
```
deploy/
├── systemd/
│   └── starpunk.service
├── nginx/
│   └── starpunk.conf
└── docker/
    └── Dockerfile
```

**Do NOT create these in V1** - only when needed.

---

## Verification Checklist

After creating project structure, verify:

- [ ] All directories exist
- [ ] `.gitignore` is configured correctly
- [ ] `.env.example` exists (`.env` does not)
- [ ] `data/` directory is gitignored
- [ ] All `__init__.py` files exist in Python packages
- [ ] Template hierarchy is correct
- [ ] Static files are in correct locations
- [ ] Tests directory mirrors application structure
- [ ] Documentation is organized correctly
- [ ] No placeholder files committed (except templates)

---

## Anti-Patterns to Avoid

**Don't**:
- Create deeply nested directories (max 3 levels)
- Use abbreviations in directory names (`tpl/` instead of `templates/`)
- Mix code and data (keep `data/` separate)
- Put configuration in code (use `.env`)
- Create empty placeholder directories
- Use both `src/` and package name (pick one - we chose package)
- Create `scripts/`, `bin/`, `tools/` directories in V1 (YAGNI)
- Put tests inside application package (keep separate)

**Do**:
- Keep structure flat where possible
- Use standard conventions (Flask, Python, pytest)
- Separate concerns (code, data, tests, docs, static)
- Make structure obvious to newcomers
- Follow principle: "Every directory must justify its existence"

---

## Summary

**Total Directories**: 12 top-level directories/files
**Total Python Modules**: ~9 in starpunk package
**Total Templates**: 9 HTML/XML files
**Total LOC Estimate**: ~1,500 LOC application + 500 LOC tests = 2,000 total

**Philosophy**: Simple, flat, conventional structure. Every file and directory has a clear purpose. User data is isolated and portable. Documentation is comprehensive and organized.

## References

- [Python Packaging Guide](https://packaging.python.org/)
- [Flask Project Layout](https://flask.palletsprojects.com/en/3.0.x/tutorial/layout/)
- [Pytest Good Practices](https://docs.pytest.org/en/stable/goodpractices.html)
- [PEP 8 Package Layout](https://peps.python.org/pep-0008/)
- [ADR-004: File-Based Note Storage](/home/phil/Projects/starpunk/docs/decisions/ADR-004-file-based-note-storage.md)
- [ADR-006: Python Virtual Environment](/home/phil/Projects/starpunk/docs/decisions/ADR-006-python-virtual-environment-uv.md)
- [Documentation Organization Standard](/home/phil/Projects/starpunk/docs/standards/documentation-organization.md)