diff --git a/.gitignore b/.gitignore index 832175e..6dbd9a3 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ .python-version secrets.enc vault_pass -DEPLOYMENT_LEARNINGS.md \ No newline at end of file +DEPLOYMENT_LEARNINGS.md +group_vars/all/vault.yml \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md index 63b8935..26faad6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Overview -This is a personal infrastructure Ansible playbook that automates deployment and management of 22+ self-hosted Docker services across two domains (`thesatelliteoflove.com` and `nerder.land`). The setup uses Tailscale VPN for secure networking and Caddy for reverse proxy with automated HTTPS. +This is a personal infrastructure Ansible playbook that automates deployment and management of 25 self-hosted Docker services across two domains (`thesatelliteoflove.com` and `nerder.land`). The setup uses Tailscale VPN for secure networking and Caddy for reverse proxy with automated HTTPS. **Important**: Always review `DEPLOYMENT_LEARNINGS.md` when working on this repository for lessons learned and troubleshooting guidance. @@ -51,7 +51,7 @@ ansible-playbook site.yml -i hosts.yml --tags common,cron --vault-password-file ### Role Structure - **bootstrap**: Initial server hardening, user creation, Tailscale VPN setup - **common**: Basic system configuration, UFW firewall management -- **docker**: Comprehensive service deployment (22+ containerized applications, organized by category) +- **docker**: Comprehensive service deployment (25 containerized applications, organized by category) - **cron**: Scheduled task management (currently Warhammer RSS feed generation) ### Docker Role Organization (Reorganized into Logical Categories) @@ -62,7 +62,7 @@ The docker role is now organized into logical service groups under `roles/docker - **development/**: Development and collaboration tools - Gitea, Code Server, Matrix (Conduit) - **media/**: Content creation and consumption - - Audiobookshelf, Calibre, Ghost blog, Pinchflat, Pinry, Karakeep, Manyfold + - Audiobookshelf, Calibre, Ghost blog, Pinchflat, Pinry, Karakeep (formerly Hoarder), Manyfold - **productivity/**: Personal organization and document management - Paperless-NGX, MMDL, Baikal (CalDAV/CardDAV), Syncthing, Heyform, Dawarich, Pingvin - **communication/**: Social media and external communication @@ -132,6 +132,9 @@ Common domains requiring hairpinning fixes: ## Service Memories - pingvin is the service that responds on files.thesatelliteoflove.com +- karakeep (formerly called hoarder) is deployed with both 'hoarder' and 'karakeep' tags for backward compatibility +- whenever i ask you what containers need updates, run dockcheck and return a list of containers needing updates +- when i ask for the status container updates i want you to run dockcheck on the docker host https://github.com/mag37/dockcheck?ref=selfh.st ## Variable Management Implementation Notes **Major Infrastructure Update**: Variable management system was implemented to replace all hardcoded values with centralized variables. diff --git a/README.md b/README.md index e9757e5..4666928 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ The playbook manages two main environments: ## Services Deployed -The Docker role deploys and manages 22+ self-hosted services organized into logical categories: +The Docker role deploys and manages 25 self-hosted services organized into logical categories: ### Infrastructure - **Caddy** (Reverse proxy with automatic HTTPS) @@ -28,7 +28,7 @@ The Docker role deploys and manages 22+ self-hosted services organized into logi - **Ghost** (Blog platform) - **Pinchflat** (Media downloader) - **Pinry** (Pinterest-like board) -- **Karakeep** (Bookmark manager) +- **Hoarder** (Bookmark manager) - **Manyfold** (3D model organizer) ### Productivity @@ -88,7 +88,7 @@ Manages scheduled tasks and automation: - Supports easy addition of new scheduled tasks ### [Docker Role](roles/docker/README.md) -The most comprehensive role, deploying 22+ containerized services organized into logical categories: +The most comprehensive role, deploying 25 containerized services organized into logical categories: - **Infrastructure**: Caddy reverse proxy, Authentik SSO, Dockge management - **Development**: Gitea, Code Server, Matrix communication - **Media**: Audiobookshelf, Calibre, Ghost blog, Pinchflat, and more diff --git a/group_vars/all/infrastructure.yml b/group_vars/all/infrastructure.yml index 00851cf..34e60d9 100644 --- a/group_vars/all/infrastructure.yml +++ b/group_vars/all/infrastructure.yml @@ -19,4 +19,8 @@ network: # Paths paths: - stacks: "{{ docker.stacks_path }}" \ No newline at end of file + stacks: "{{ docker.stacks_path }}" + +# Notification services +notifications: + appriseapi_endpoint: "http://apprise:8000/notify/apprise" \ No newline at end of file diff --git a/roles/common/tasks/main.yml b/roles/common/tasks/main.yml index 4f0448d..1992917 100644 --- a/roles/common/tasks/main.yml +++ b/roles/common/tasks/main.yml @@ -1,6 +1,8 @@ -- name: Install aptitude +- name: Install common packages apt: - name: aptitude + name: + - aptitude + - jq state: latest update_cache: true diff --git a/roles/docker/templates/appriseapi-compose.yml.j2 b/roles/docker/templates/appriseapi-compose.yml.j2 index 895b071..2645a57 100644 --- a/roles/docker/templates/appriseapi-compose.yml.j2 +++ b/roles/docker/templates/appriseapi-compose.yml.j2 @@ -11,6 +11,8 @@ services: - plugin:/plugin - attach:/attach image: caronc/apprise:latest + extra_hosts: + - "{{ subdomains.chat }}:{{ docker.hairpin_ip }}" labels: glance.name: Apprise glance.icon: si:imessage diff --git a/todo.md b/todo.md index 680a105..6490062 100644 --- a/todo.md +++ b/todo.md @@ -17,18 +17,18 @@ ``` - **COMPLETED**: All services organized into logical categories with category-level tags -### 2. Standardize variable management +### 2. Standardize variable management ✅ COMPLETED - **Current Issue**: Secrets in single encrypted file, no clear variable hierarchy - **Solution**: Create proper variable structure: ``` group_vars/ ├── all/ - │ ├── common.yml (shared config) - │ └── secrets.yml (vault encrypted) - ├── docker/ - │ ├── services.yml (service configs) - │ └── networking.yml (network settings) + │ ├── domains.yml (domain and subdomain mappings) + │ ├── infrastructure.yml (network config, Docker settings) + │ ├── vault.yml (encrypted secrets with vault_ prefix) + │ └── services.yml (service configuration flags) ``` +- **COMPLETED**: Implemented comprehensive variable hierarchy, updated all templates to use centralized variables, fixed service tag isolation ### 3. Template consolidation ✅ PARTIALLY COMPLETED - **Current Issue**: Many compose templates repeat patterns, some services used static files @@ -43,7 +43,7 @@ - **Solution**: Implement comprehensive health monitoring with standardized healthcheck patterns ### 5. Implement backup strategy -- **Issue**: No automated backups for 25+ services and their data +- **Issue**: No automated backups for 25 services and their data - **Solution**: Add backup role with: - Database dumps for PostgreSQL services - Volume backups for file-based services @@ -99,7 +99,7 @@ - [x] Convert static compose files to templates - [x] Remove unused services (beaver, grist, stirlingpdf, tasksmd, redlib) - [x] Clean up templates and files directories -- [ ] Implement variable hierarchy +- [x] Implement variable hierarchy - [ ] Create reusable template patterns ### Week 2: Security & Monitoring @@ -124,10 +124,12 @@ - **Template Standardization**: Converted all static compose files to Jinja2 templates - **Service Cleanup**: Removed 5 unused/broken services (beaver, grist, stirlingpdf, tasksmd, redlib) - **Category-Based Deployment**: Can now deploy services by category using tags (infrastructure, media, etc.) -- **Documentation Updates**: Updated CLAUDE.md to reflect new architecture +- **Variable Management**: Implemented comprehensive centralized variable hierarchy with proper secret organization +- **Service Tag Isolation**: Fixed service tags to deploy individual services only (not entire categories) +- **Documentation Updates**: Updated all README files and CLAUDE.md to reflect new architecture ### 📊 Current Stats -- **22+ active services** organized into 6 categories +- **25 active services** organized into 6 categories - **100% templated** compose files (no static files) - **6 service directories** for logical organization - **Clean file structure** with only essential static files