From 7fdb52e91b31cbc72eacdb1af36861ce5fe7af2a Mon Sep 17 00:00:00 2001 From: Phil Date: Fri, 6 Jun 2025 10:51:39 -0600 Subject: [PATCH] add comprehensive documentation for all Ansible roles MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add main README with infrastructure overview and usage instructions - Document bootstrap role for server initialization and security hardening - Document common role for shared server configuration - Document cron role for scheduled tasks and automation - Document docker role with detailed service descriptions and deployment patterns - Include MMDL service documentation with setup requirements - Add troubleshooting guides and security considerations 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- README.md | 131 ++++++++++++++++++++++++ roles/bootstrap/README.md | 41 ++++++++ roles/common/README.md | 23 +++++ roles/cron/README.md | 37 +++++++ roles/docker/README.md | 202 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 434 insertions(+) create mode 100644 README.md create mode 100644 roles/bootstrap/README.md create mode 100644 roles/common/README.md create mode 100644 roles/cron/README.md create mode 100644 roles/docker/README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..629ca20 --- /dev/null +++ b/README.md @@ -0,0 +1,131 @@ +# Personal Infrastructure Ansible Playbook + +This Ansible playbook automates the setup and management of a personal self-hosted infrastructure running Docker containers for various services. + +## Overview + +The playbook manages two main environments: +- **Bootstrap server** (`netcup`): Initial server setup with Tailscale VPN +- **Docker server** (`docker-01`): Main application server running containerized services + +## Services Deployed + +The Docker role deploys and manages the following self-hosted services: + +- **Authentication**: Authentik (SSO/Identity Provider) +- **Media**: Audiobookshelf, Calibre, Pinchflat +- **Productivity**: Ghost blog, Gitea, Code Server, Grist, TasksMD, Stirling PDF, MMDL (Task Management) +- **Communication**: GoToSocial, Matrix (Conduit) +- **File Management**: Hoarder, Paperless-NGX, Syncthing, Manyfold +- **Monitoring**: Changedetection, Glance dashboard, Dawarich location tracking +- **Utilities**: Baikal (CalDAV/CardDAV), HeyForm, Pingvin Share, Pinry +- **Notifications**: Apprise API +- **Reverse Proxy**: Caddy + +## Structure + +``` +├── site.yml # Main playbook +├── bootstrap.yml # Server bootstrap playbook +├── dns.yml # AWS Route53 DNS management +├── hosts.yml # Inventory file +├── requirements.yml # External role dependencies +└── roles/ + ├── bootstrap/ # Initial server setup + ├── common/ # Common server configuration + ├── cron/ # Scheduled tasks + └── docker/ # Docker services deployment +``` + +## Roles Documentation + +Each role has detailed documentation in its respective directory: + +### [Bootstrap Role](roles/bootstrap/README.md) +Performs initial server setup and hardening: +- Creates user accounts with SSH key authentication +- Configures passwordless sudo and security hardening +- Installs essential packages and configures UFW firewall +- Sets up Tailscale VPN for secure network access + +### [Common Role](roles/common/README.md) +Provides shared configuration for all servers: +- Installs common packages (aptitude) +- Enables UFW firewall with default deny policy +- Ensures consistent base configuration across infrastructure + +### [Cron Role](roles/cron/README.md) +Manages scheduled tasks and automation: +- **Warhammer RSS Feed Updater**: Daily job that generates and updates RSS feeds +- Integrates with Docker services for content generation +- Supports easy addition of new scheduled tasks + +### [Docker Role](roles/docker/README.md) +The most comprehensive role, deploying 25+ containerized services: +- **Core Infrastructure**: Caddy reverse proxy, Authentik SSO, Dockge management +- **Development Tools**: Gitea, Code Server, Matrix communication +- **Media Management**: Audiobookshelf, Calibre, Ghost blog +- **Productivity**: Paperless-NGX, Baikal calendar, Glance dashboard +- **Security Features**: Centralized authentication, network isolation, container hardening +- **Monitoring**: Comprehensive service health monitoring and alerting + +## Usage + +### Prerequisites + +1. Install Ansible and required collections: + ```bash + ansible-galaxy install -r requirements.yml + ``` + +2. Configure your inventory in `hosts.yml` with your server details + +### Bootstrap a New Server + +```bash +ansible-playbook bootstrap.yml -i hosts.yml +``` + +This will: +- Create a user account +- Install and configure Tailscale VPN +- Set up basic security + +### Deploy Docker Services + +```bash +ansible-playbook site.yml -i hosts.yml +``` + +Or deploy specific services using tags: +```bash +# Deploy only Caddy reverse proxy +ansible-playbook site.yml -i hosts.yml --tags caddy + +# Deploy authentication services +ansible-playbook site.yml -i hosts.yml --tags authentik + +# Deploy task management +ansible-playbook site.yml -i hosts.yml --tags mmdl +``` + +### Manage DNS Records + +```bash +ansible-playbook dns.yml -i hosts.yml +``` + +Updates AWS Route53 DNS records for configured domains (`thesatelliteoflove.com` and `nerder.land`). + +## Configuration + +- Service configurations are templated in `roles/docker/templates/` +- Environment variables and secrets should be managed through Ansible Vault +- Docker Compose files are generated from Jinja2 templates + +## Security Notes + +- Uses Tailscale for secure network access +- Caddy provides automatic HTTPS with Let's Encrypt +- Services are containerized for isolation +- UFW firewall rules are managed via Docker integration \ No newline at end of file diff --git a/roles/bootstrap/README.md b/roles/bootstrap/README.md new file mode 100644 index 0000000..1f70cfd --- /dev/null +++ b/roles/bootstrap/README.md @@ -0,0 +1,41 @@ +# Bootstrap Role + +## Purpose +Performs initial server setup and hardening for new Ubuntu/Debian servers. + +## What It Does + +### User Management +- Creates a new user account with sudo privileges (specified by `created_username` variable) +- Configures passwordless sudo for the sudo group +- Sets up SSH key authentication using your local `~/.ssh/id_ed25519.pub` key +- Disables root password authentication + +### System Packages +- Installs `aptitude` for better package management +- Installs essential packages: + - `curl` - HTTP client + - `vim` - Text editor + - `git` - Version control + - `ufw` - Uncomplicated Firewall + +### Security Configuration +- Configures UFW firewall to: + - Allow SSH connections + - Enable firewall with default deny policy +- Hardens SSH configuration + +## Variables Required +- `created_username`: The username to create (typically set in bootstrap.yml) +- `tailscale_key`: Tailscale authentication key (prompted during playbook run) + +## Dependencies +- Requires the `artis3n.tailscale` role for VPN setup +- Requires your SSH public key at `~/.ssh/id_ed25519.pub` + +## Usage +```bash +ansible-playbook bootstrap.yml -i hosts.yml +``` + +This role is designed to be run once on a fresh server before deploying other services. \ No newline at end of file diff --git a/roles/common/README.md b/roles/common/README.md new file mode 100644 index 0000000..ece64b7 --- /dev/null +++ b/roles/common/README.md @@ -0,0 +1,23 @@ +# Common Role + +## Purpose +Provides shared configuration and security setup that applies to all servers in the infrastructure. + +## What It Does + +### System Packages +- Installs `aptitude` for better package management and dependency resolution +- Updates package cache to ensure latest package information + +### Security Configuration +- Enables UFW (Uncomplicated Firewall) with default deny policy +- Provides baseline firewall protection for all managed servers + +## Usage +This role is automatically applied to all servers in the infrastructure as part of the main site.yml playbook. It ensures consistent base configuration across all managed systems. + +## Dependencies +None - this is a foundational role that other roles can depend on. + +## Notes +This role is designed to be lightweight and provide only the most essential common functionality. Server-specific configurations should be handled by dedicated roles like `docker` or `bootstrap`. \ No newline at end of file diff --git a/roles/cron/README.md b/roles/cron/README.md new file mode 100644 index 0000000..898970a --- /dev/null +++ b/roles/cron/README.md @@ -0,0 +1,37 @@ +# Cron Role + +## Purpose +Manages scheduled tasks and automated maintenance jobs for the infrastructure. + +## What It Does + +### Warhammer RSS Feed Updater +- Copies `update_warhammer_feed.sh` script to `/usr/local/bin/` with executable permissions +- Creates a daily cron job that runs at 09:10 AM +- The script performs these actions: + 1. Creates a temporary directory `/tmp/warhammer_feed` + 2. Runs a custom Docker container (`git.thesatelliteoflove.com/phil/rss-warhammer`) to generate RSS feed + 3. Copies the generated `warhammer_rss_feed.xml` to `/opt/stacks/caddy/site/tsol/feeds/` + 4. Restarts the Glance dashboard stack to reflect the updated feed + +## Files Managed +- `/usr/local/bin/update_warhammer_feed.sh` - RSS feed update script +- Cron job: "Update Warhammer RSS Feed" (daily at 09:10) + +## Dependencies +- Requires Docker to be installed and running +- Depends on the following Docker stacks being deployed: + - Custom RSS generator container at `git.thesatelliteoflove.com/phil/rss-warhammer` + - Caddy web server stack at `/opt/stacks/caddy/` + - Glance dashboard stack at `/opt/stacks/glance/` + +## Usage +This role is automatically applied as part of the main site.yml playbook with the `cron` tag. + +```bash +# Deploy only cron jobs +ansible-playbook site.yml -i hosts.yml --tags cron +``` + +## Customization +To add additional cron jobs, create new tasks in the main.yml file following the same pattern as the Warhammer feed updater. \ No newline at end of file diff --git a/roles/docker/README.md b/roles/docker/README.md new file mode 100644 index 0000000..9dfc835 --- /dev/null +++ b/roles/docker/README.md @@ -0,0 +1,202 @@ +# Docker Role + +## Purpose +Deploys and manages a comprehensive self-hosted infrastructure with 25+ containerized services, transforming a server into a personal cloud platform with authentication, media management, productivity tools, and development services. + +## Architecture Overview + +### Network Configuration +- **External Network**: All services connect to shared `lava` network (172.20.0.0/24) +- **Reverse Proxy**: Caddy handles all ingress traffic with automatic HTTPS +- **Service Discovery**: Container-to-container communication using service names +- **Firewall Integration**: UFW-Docker script properly configures firewall rules + +### Security Features +- **Centralized SSO**: Authentik provides OIDC authentication for most services +- **Network Isolation**: Services restricted to appropriate network segments +- **Container Hardening**: Non-root users, capability dropping, security options +- **Secret Management**: Ansible vault for sensitive configuration + +## Services Deployed + +### Core Infrastructure +- **Caddy** - Reverse proxy with automatic HTTPS (static IP: 172.20.0.5) +- **Dockge** - Docker compose stack management UI +- **Authentik** - Enterprise authentication server (OIDC/SAML SSO) + +### Development & Code Management +- **Gitea** - Self-hosted Git with CI/CD runners +- **Code Server** - VS Code in the browser +- **Conduit** - Matrix homeserver for communication + +### Media & Content Management +- **Audiobookshelf** - Audiobook and podcast server +- **Calibre** - E-book management and conversion +- **Ghost** - Modern blogging platform +- **Hoarder** - Bookmark management with AI tagging +- **Pinry** - Pinterest-like image board +- **Pingvin Share** - File sharing service +- **Syncthing** - Decentralized file sync + +### Productivity & Organization +- **Paperless-ngx** - Document management with OCR +- **Baikal** - CalDAV/CardDAV server +- **Glance** - Customizable dashboard with monitoring +- **Heyform** - Form builder and surveys +- **Postiz** - Social media management +- **Dawarich** - Location tracking +- **Change Detection** - Website monitoring +- **Manyfold** - 3D model file organization +- **MMDL** - Task and calendar management with CalDAV integration + +### Utilities & Tools +- **Stirling PDF** - PDF manipulation (internal network only) +- **Pinchflat** - YouTube video archiving +- **Apprise API** - Unified notifications +- **GoToSocial** - Lightweight ActivityPub server + +## Deployment Patterns + +### Standardized Service Deployment +Each service follows a consistent pattern: +1. Creates `/opt/stacks/[service-name]` directory structure +2. Generates Docker Compose file from Jinja2 template +3. Deploys using `community.docker.docker_compose_v2` +4. Configures environment variables from vault secrets + +### Template System +- **Compose Templates**: `.j2` files in `templates/` for dynamic configuration +- **Environment Templates**: Separate `.env.j2` files for services requiring environment variables +- **Variable Substitution**: Uses Ansible vault variables for secrets and configuration + +## Shell Environment Setup +The role also configures the shell environment: +- **Zsh Installation**: Sets zsh as default shell +- **Atuin**: Command history sync and search +- **Bat**: Enhanced `cat` command with syntax highlighting + +## File Organization +``` +roles/docker/ +├── tasks/ +│ ├── main.yml # Orchestrates all deployments +│ ├── shell.yml # Shell environment setup +│ ├── caddy.yml # Reverse proxy +│ ├── authentik.yml # Authentication +│ ├── mmdl.yml # Task management +│ └── [25+ service files] # Individual service deployments +├── templates/ +│ ├── [service]-compose.yml.j2 # Docker Compose templates +│ ├── [service]-env.j2 # Environment variable templates +│ └── mmdl-*.j2 # MMDL-specific templates +├── files/ +│ ├── Caddyfile # Caddy configuration +│ ├── ufw-docker.sh # Firewall integration script +│ └── [various configs] # Static configuration files +└── handlers/ + └── main.yml # Service restart handlers +``` + +## Usage + +### Deploy All Services +```bash +ansible-playbook site.yml -i hosts.yml --tags docker +``` + +### Deploy Specific Services +```bash +# Deploy only authentication stack +ansible-playbook site.yml -i hosts.yml --tags authentik + +# Deploy media services +ansible-playbook site.yml -i hosts.yml --tags audiobookshelf,calibre + +# Deploy development tools +ansible-playbook site.yml -i hosts.yml --tags gitea,codeserver + +# Deploy task management +ansible-playbook site.yml -i hosts.yml --tags mmdl +``` + +### Deploy Core Infrastructure Only +```bash +ansible-playbook site.yml -i hosts.yml --tags caddy,authentik,glance +``` + +## Service-Specific Notes + +### MMDL (Task Management) +- **URL**: https://tasks.thesatelliteoflove.com +- **Initial Setup**: Visit `/install` endpoint first to run database migrations +- **Authentication**: Integrates with Authentik OIDC provider +- **Database**: Uses MySQL 8.0 with automatic schema migration +- **Features**: CalDAV integration, multiple account support, task management + +## Dependencies + +### System Requirements +- Docker CE installed and running +- UFW firewall configured +- DNS records pointing to the server +- Valid SSL certificates (handled automatically by Caddy) + +### External Services +- **DNS**: Requires subdomains configured for each service +- **Email**: Gitea uses Resend for notifications +- **Storage**: All services persist data to `/opt/stacks/[service]/` + +## Configuration + +### Required Variables (in vault) +- Authentication credentials for various services +- API keys for external integrations +- OAuth client secrets for SSO integration +- Database passwords and connection strings + +### Network Configuration +Services expect to be accessible via subdomains of configured domains: +- `auth.thesatelliteoflove.com` - Authentik +- `git.thesatelliteoflove.com` - Gitea +- `books.thesatelliteoflove.com` - Calibre +- `tasks.thesatelliteoflove.com` - MMDL +- (and many more...) + +## Monitoring & Management + +### Glance Dashboard Integration +All services include Glance labels for dashboard monitoring: +- Service health status +- Container resource usage +- Parent-child relationships for multi-container services + +### Operational Features +- Automatic container restart policies +- Health checks for database services +- Centralized logging and monitoring +- Backup-ready data structure in `/opt/stacks/` + +## Security Considerations + +### Network Security +- UFW-Docker integration for proper firewall rules +- Services isolated to appropriate network segments +- Restricted access for sensitive tools (Stirling PDF) + +### Authentication +- Centralized SSO through Authentik for most services +- OAuth integration where supported +- Secure secret management through Ansible vault + +### Container Security +- Non-root container execution (UID/GID 1000:1000) +- Security options: `no-new-privileges: true` +- Capability dropping and minimal permissions + +## Troubleshooting + +### Common Issues +- **Database Connection**: Ensure MySQL containers use proper authentication plugins +- **OAuth Discovery**: Check issuer URLs don't have trailing slashes +- **Migration Failures**: Visit service `/install` endpoints for database setup +- **Network Issues**: Verify containers are on the same Docker network \ No newline at end of file