Phil
dad7e887a8
- Update remaining Docker Compose templates with centralized variables
- Fix service tag isolation to deploy individual services only
- Update all README files with variable management architecture
- Document variable hierarchy in DEPLOYMENT_LEARNINGS.md
- Add comprehensive variable usage patterns to CLAUDE.md
- Standardize domain references using {{ subdomains.* }} pattern
- Replace hardcoded network names with {{ docker.network_name }}
- Update hairpinning configuration to use variables
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
227 lines
8.9 KiB
Markdown
227 lines
8.9 KiB
Markdown
# Docker Role
|
|
|
|
## Purpose
|
|
Deploys and manages a comprehensive self-hosted infrastructure with 22+ containerized services organized into logical categories, transforming a server into a personal cloud platform with authentication, media management, productivity tools, and development services.
|
|
|
|
## Architecture Overview
|
|
|
|
### Network Configuration
|
|
- **External Network**: All services connect to shared Docker network (configurable)
|
|
- **Reverse Proxy**: Caddy handles all ingress traffic with automatic HTTPS
|
|
- **Service Discovery**: Container-to-container communication using service names
|
|
- **Firewall Integration**: UFW-Docker script properly configures firewall rules
|
|
|
|
### Security Features
|
|
- **Centralized SSO**: Authentik provides OIDC authentication for most services
|
|
- **Network Isolation**: Services restricted to appropriate network segments
|
|
- **Container Hardening**: Non-root users, capability dropping, security options
|
|
- **Secret Management**: Ansible vault for sensitive configuration
|
|
- **Variable Management**: Centralized variable hierarchy using group_vars structure
|
|
|
|
## Services Deployed (Organized by Category)
|
|
|
|
### Infrastructure (`infrastructure/`)
|
|
- **Caddy** - Reverse proxy with automatic HTTPS (static IP: 172.20.0.5)
|
|
- **Authentik** - Enterprise authentication server (OIDC/SAML SSO)
|
|
- **Dockge** - Docker compose stack management UI
|
|
|
|
### Development (`development/`)
|
|
- **Gitea** - Self-hosted Git with CI/CD runners
|
|
- **Code Server** - VS Code in the browser
|
|
- **Conduit** - Matrix homeserver for communication
|
|
|
|
### Media (`media/`)
|
|
- **Audiobookshelf** - Audiobook and podcast server
|
|
- **Calibre** - E-book management and conversion
|
|
- **Ghost** - Modern blogging platform
|
|
- **Pinchflat** - YouTube video archiving
|
|
- **Pinry** - Pinterest-like image board
|
|
- **Karakeep** - Bookmark management with AI tagging
|
|
- **Manyfold** - 3D model file organization
|
|
|
|
### Productivity (`productivity/`)
|
|
- **Paperless-ngx** - Document management with OCR
|
|
- **MMDL** - Task and calendar management with CalDAV integration
|
|
- **Baikal** - CalDAV/CardDAV server
|
|
- **Syncthing** - Decentralized file sync
|
|
- **Heyform** - Form builder and surveys
|
|
- **Dawarich** - Location tracking
|
|
- **Pingvin Share** - File sharing service
|
|
|
|
### Communication (`communication/`)
|
|
- **GoToSocial** - Lightweight ActivityPub server
|
|
- **Postiz** - Social media management
|
|
|
|
### Monitoring (`monitoring/`)
|
|
- **Glance** - Customizable dashboard with monitoring
|
|
- **Change Detection** - Website monitoring
|
|
- **Apprise API** - Unified notifications
|
|
|
|
## Deployment Patterns
|
|
|
|
### Standardized Service Deployment
|
|
Each service follows a consistent pattern:
|
|
1. Creates `/opt/stacks/[service-name]` directory structure
|
|
2. Generates Docker Compose file from Jinja2 template
|
|
3. Deploys using `community.docker.docker_compose_v2`
|
|
4. Configures environment variables from vault secrets
|
|
|
|
### Template System
|
|
- **Compose Templates**: `.j2` files in `templates/` for dynamic configuration
|
|
- **Environment Templates**: Separate `.env.j2` files for services requiring environment variables
|
|
- **Variable Substitution**: Uses centralized variable hierarchy from group_vars structure
|
|
- **Domain Management**: Centralized domain and subdomain configuration
|
|
- **Network Configuration**: Standardized Docker network and IP address management
|
|
|
|
## Shell Environment Setup
|
|
The role also configures the shell environment:
|
|
- **Zsh Installation**: Sets zsh as default shell
|
|
- **Atuin**: Command history sync and search
|
|
- **Bat**: Enhanced `cat` command with syntax highlighting
|
|
|
|
## File Organization
|
|
```
|
|
roles/docker/
|
|
├── tasks/
|
|
│ ├── main.yml # Orchestrates all deployments
|
|
│ ├── shell.yml # Shell environment setup
|
|
│ ├── infrastructure/
|
|
│ │ ├── main.yml # Infrastructure category orchestrator
|
|
│ │ ├── caddy.yml # Reverse proxy
|
|
│ │ └── authentik.yml # Authentication
|
|
│ ├── development/
|
|
│ │ ├── main.yml # Development category orchestrator
|
|
│ │ ├── gitea.yml # Git hosting
|
|
│ │ ├── codeserver.yml # VS Code server
|
|
│ │ └── conduit.yml # Matrix server
|
|
│ ├── media/ # Media services (7 services)
|
|
│ ├── productivity/ # Productivity services (7 services)
|
|
│ ├── communication/ # Communication services (2 services)
|
|
│ └── monitoring/ # Monitoring services (3 services)
|
|
├── templates/
|
|
│ ├── [service]-compose.yml.j2 # Docker Compose templates (all templated)
|
|
│ ├── [service]-env.j2 # Environment variable templates
|
|
│ └── [service]-*.j2 # Service-specific templates
|
|
├── files/
|
|
│ ├── Caddyfile # Caddy configuration
|
|
│ ├── ufw-docker.sh # Firewall integration script
|
|
│ ├── client # Matrix well-known client file
|
|
│ └── server # Matrix well-known server file
|
|
└── handlers/
|
|
└── main.yml # Service restart handlers
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Deploy All Services
|
|
```bash
|
|
ansible-playbook site.yml -i hosts.yml --tags docker
|
|
```
|
|
|
|
### Deploy by Service Category
|
|
```bash
|
|
# Deploy entire service categories
|
|
ansible-playbook site.yml -i hosts.yml --tags infrastructure
|
|
ansible-playbook site.yml -i hosts.yml --tags development
|
|
ansible-playbook site.yml -i hosts.yml --tags media
|
|
ansible-playbook site.yml -i hosts.yml --tags productivity
|
|
ansible-playbook site.yml -i hosts.yml --tags communication
|
|
ansible-playbook site.yml -i hosts.yml --tags monitoring
|
|
|
|
# Deploy multiple categories
|
|
ansible-playbook site.yml -i hosts.yml --tags infrastructure,monitoring
|
|
```
|
|
|
|
### Deploy Individual Services
|
|
```bash
|
|
# Deploy specific services
|
|
ansible-playbook site.yml -i hosts.yml --tags authentik
|
|
ansible-playbook site.yml -i hosts.yml --tags gitea,codeserver
|
|
ansible-playbook site.yml -i hosts.yml --tags mmdl
|
|
```
|
|
|
|
## Service-Specific Notes
|
|
|
|
### MMDL (Task Management)
|
|
- **URL**: https://tasks.thesatelliteoflove.com
|
|
- **Initial Setup**: Visit `/install` endpoint first to run database migrations
|
|
- **Authentication**: Integrates with Authentik OIDC provider
|
|
- **Database**: Uses MySQL 8.0 with automatic schema migration
|
|
- **Features**: CalDAV integration, multiple account support, task management
|
|
|
|
## Dependencies
|
|
|
|
### System Requirements
|
|
- Docker CE installed and running
|
|
- UFW firewall configured
|
|
- DNS records pointing to the server
|
|
- Valid SSL certificates (handled automatically by Caddy)
|
|
|
|
### External Services
|
|
- **DNS**: Requires subdomains configured for each service
|
|
- **Email**: Gitea uses Resend for notifications
|
|
- **Storage**: All services persist data to `/opt/stacks/[service]/`
|
|
|
|
## Configuration
|
|
|
|
### Variable Structure
|
|
The role uses a centralized variable hierarchy in `group_vars/all/`:
|
|
|
|
- **domains.yml**: Domain and subdomain mappings for all services
|
|
- **infrastructure.yml**: Network configuration, Docker settings, and system parameters
|
|
- **vault.yml**: Encrypted secrets including API keys, passwords, and OAuth credentials
|
|
- **services.yml**: Service-specific configuration and feature flags
|
|
|
|
### Required Variables (in vault.yml)
|
|
- Authentication credentials for various services (vault_*)
|
|
- API keys for external integrations
|
|
- OAuth client secrets for SSO integration
|
|
- Database passwords and connection strings
|
|
- SMTP credentials for notifications
|
|
|
|
### Network Configuration
|
|
Services expect to be accessible via subdomains of configured domains:
|
|
- `auth.thesatelliteoflove.com` - Authentik
|
|
- `git.thesatelliteoflove.com` - Gitea
|
|
- `books.thesatelliteoflove.com` - Calibre
|
|
- `tasks.thesatelliteoflove.com` - MMDL
|
|
- (and many more...)
|
|
|
|
## Monitoring & Management
|
|
|
|
### Glance Dashboard Integration
|
|
All services include Glance labels for dashboard monitoring:
|
|
- Service health status
|
|
- Container resource usage
|
|
- Parent-child relationships for multi-container services
|
|
|
|
### Operational Features
|
|
- Automatic container restart policies
|
|
- Health checks for database services
|
|
- Centralized logging and monitoring
|
|
- Backup-ready data structure in `/opt/stacks/`
|
|
|
|
## Security Considerations
|
|
|
|
### Network Security
|
|
- UFW-Docker integration for proper firewall rules
|
|
- Services isolated to appropriate network segments
|
|
- Restricted access for sensitive tools (Stirling PDF)
|
|
|
|
### Authentication
|
|
- Centralized SSO through Authentik for most services
|
|
- OAuth integration where supported
|
|
- Secure secret management through Ansible vault
|
|
|
|
### Container Security
|
|
- Non-root container execution (UID/GID 1000:1000)
|
|
- Security options: `no-new-privileges: true`
|
|
- Capability dropping and minimal permissions
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
- **Database Connection**: Ensure MySQL containers use proper authentication plugins
|
|
- **OAuth Discovery**: Check issuer URLs don't have trailing slashes
|
|
- **Migration Failures**: Visit service `/install` endpoints for database setup
|
|
- **Network Issues**: Verify containers are on the same Docker network |