Phil 68f0276ac0 feat: complete infrastructure cleanup and optimization
This comprehensive update improves maintainability and removes unused services:

## Major Changes
- Remove 5 unused services (beaver, grist, stirlingpdf, tasksmd, redlib)
- Convert remaining static compose files to Jinja2 templates
- Clean up Caddyfile removing orphaned proxy configurations
- Align DNS records with active services

## Service Cleanup
- Remove habits.thesatelliteoflove.com DNS record (beaver service)
- Add missing DNS records for active services:
  - post.thesatelliteoflove.com (Postiz)
  - files.thesatelliteoflove.com (Pingvin Share)
  - bookmarks.thesatelliteoflove.com (Hoarder)

## Template Standardization
- Convert caddy-compose.yml to template
- Convert dockge-compose.yml to template
- Convert hoarder-compose.yml to template
- All services now use consistent template-driven approach

## Documentation Updates
- Update CLAUDE.md with new service organization
- Update README.md files with category-based deployment examples
- Update todo.md with completed work summary
- Service count updated to 22+ active services

Infrastructure is now fully organized, cleaned up, and ready for future enhancements.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-06 12:16:44 -06:00
..

Docker Role

Purpose

Deploys and manages a comprehensive self-hosted infrastructure with 22+ containerized services organized into logical categories, transforming a server into a personal cloud platform with authentication, media management, productivity tools, and development services.

Architecture Overview

Network Configuration

  • External Network: All services connect to shared lava network (172.20.0.0/24)
  • Reverse Proxy: Caddy handles all ingress traffic with automatic HTTPS
  • Service Discovery: Container-to-container communication using service names
  • Firewall Integration: UFW-Docker script properly configures firewall rules

Security Features

  • Centralized SSO: Authentik provides OIDC authentication for most services
  • Network Isolation: Services restricted to appropriate network segments
  • Container Hardening: Non-root users, capability dropping, security options
  • Secret Management: Ansible vault for sensitive configuration

Services Deployed (Organized by Category)

Infrastructure (infrastructure/)

  • Caddy - Reverse proxy with automatic HTTPS (static IP: 172.20.0.5)
  • Authentik - Enterprise authentication server (OIDC/SAML SSO)
  • Dockge - Docker compose stack management UI

Development (development/)

  • Gitea - Self-hosted Git with CI/CD runners
  • Code Server - VS Code in the browser
  • Conduit - Matrix homeserver for communication

Media (media/)

  • Audiobookshelf - Audiobook and podcast server
  • Calibre - E-book management and conversion
  • Ghost - Modern blogging platform
  • Pinchflat - YouTube video archiving
  • Pinry - Pinterest-like image board
  • Hoarder - Bookmark management with AI tagging
  • Manyfold - 3D model file organization

Productivity (productivity/)

  • Paperless-ngx - Document management with OCR
  • MMDL - Task and calendar management with CalDAV integration
  • Baikal - CalDAV/CardDAV server
  • Syncthing - Decentralized file sync
  • Heyform - Form builder and surveys
  • Dawarich - Location tracking
  • Pingvin Share - File sharing service

Communication (communication/)

  • GoToSocial - Lightweight ActivityPub server
  • Postiz - Social media management

Monitoring (monitoring/)

  • Glance - Customizable dashboard with monitoring
  • Change Detection - Website monitoring
  • Apprise API - Unified notifications

Deployment Patterns

Standardized Service Deployment

Each service follows a consistent pattern:

  1. Creates /opt/stacks/[service-name] directory structure
  2. Generates Docker Compose file from Jinja2 template
  3. Deploys using community.docker.docker_compose_v2
  4. Configures environment variables from vault secrets

Template System

  • Compose Templates: .j2 files in templates/ for dynamic configuration
  • Environment Templates: Separate .env.j2 files for services requiring environment variables
  • Variable Substitution: Uses Ansible vault variables for secrets and configuration

Shell Environment Setup

The role also configures the shell environment:

  • Zsh Installation: Sets zsh as default shell
  • Atuin: Command history sync and search
  • Bat: Enhanced cat command with syntax highlighting

File Organization

roles/docker/
├── tasks/
│   ├── main.yml                    # Orchestrates all deployments
│   ├── shell.yml                   # Shell environment setup
│   ├── infrastructure/
│   │   ├── main.yml               # Infrastructure category orchestrator
│   │   ├── caddy.yml              # Reverse proxy
│   │   └── authentik.yml          # Authentication
│   ├── development/
│   │   ├── main.yml               # Development category orchestrator
│   │   ├── gitea.yml              # Git hosting
│   │   ├── codeserver.yml         # VS Code server
│   │   └── conduit.yml            # Matrix server
│   ├── media/                     # Media services (7 services)
│   ├── productivity/              # Productivity services (7 services)
│   ├── communication/             # Communication services (2 services)
│   └── monitoring/                # Monitoring services (3 services)
├── templates/
│   ├── [service]-compose.yml.j2   # Docker Compose templates (all templated)
│   ├── [service]-env.j2           # Environment variable templates
│   └── [service]-*.j2             # Service-specific templates
├── files/
│   ├── Caddyfile                  # Caddy configuration
│   ├── ufw-docker.sh              # Firewall integration script
│   ├── client                     # Matrix well-known client file
│   └── server                     # Matrix well-known server file
└── handlers/
    └── main.yml                   # Service restart handlers

Usage

Deploy All Services

ansible-playbook site.yml -i hosts.yml --tags docker

Deploy by Service Category

# Deploy entire service categories
ansible-playbook site.yml -i hosts.yml --tags infrastructure
ansible-playbook site.yml -i hosts.yml --tags development
ansible-playbook site.yml -i hosts.yml --tags media
ansible-playbook site.yml -i hosts.yml --tags productivity
ansible-playbook site.yml -i hosts.yml --tags communication
ansible-playbook site.yml -i hosts.yml --tags monitoring

# Deploy multiple categories
ansible-playbook site.yml -i hosts.yml --tags infrastructure,monitoring

Deploy Individual Services

# Deploy specific services
ansible-playbook site.yml -i hosts.yml --tags authentik
ansible-playbook site.yml -i hosts.yml --tags gitea,codeserver
ansible-playbook site.yml -i hosts.yml --tags mmdl

Service-Specific Notes

MMDL (Task Management)

  • URL: https://tasks.thesatelliteoflove.com
  • Initial Setup: Visit /install endpoint first to run database migrations
  • Authentication: Integrates with Authentik OIDC provider
  • Database: Uses MySQL 8.0 with automatic schema migration
  • Features: CalDAV integration, multiple account support, task management

Dependencies

System Requirements

  • Docker CE installed and running
  • UFW firewall configured
  • DNS records pointing to the server
  • Valid SSL certificates (handled automatically by Caddy)

External Services

  • DNS: Requires subdomains configured for each service
  • Email: Gitea uses Resend for notifications
  • Storage: All services persist data to /opt/stacks/[service]/

Configuration

Required Variables (in vault)

  • Authentication credentials for various services
  • API keys for external integrations
  • OAuth client secrets for SSO integration
  • Database passwords and connection strings

Network Configuration

Services expect to be accessible via subdomains of configured domains:

  • auth.thesatelliteoflove.com - Authentik
  • git.thesatelliteoflove.com - Gitea
  • books.thesatelliteoflove.com - Calibre
  • tasks.thesatelliteoflove.com - MMDL
  • (and many more...)

Monitoring & Management

Glance Dashboard Integration

All services include Glance labels for dashboard monitoring:

  • Service health status
  • Container resource usage
  • Parent-child relationships for multi-container services

Operational Features

  • Automatic container restart policies
  • Health checks for database services
  • Centralized logging and monitoring
  • Backup-ready data structure in /opt/stacks/

Security Considerations

Network Security

  • UFW-Docker integration for proper firewall rules
  • Services isolated to appropriate network segments
  • Restricted access for sensitive tools (Stirling PDF)

Authentication

  • Centralized SSO through Authentik for most services
  • OAuth integration where supported
  • Secure secret management through Ansible vault

Container Security

  • Non-root container execution (UID/GID 1000:1000)
  • Security options: no-new-privileges: true
  • Capability dropping and minimal permissions

Troubleshooting

Common Issues

  • Database Connection: Ensure MySQL containers use proper authentication plugins
  • OAuth Discovery: Check issuer URLs don't have trailing slashes
  • Migration Failures: Visit service /install endpoints for database setup
  • Network Issues: Verify containers are on the same Docker network