diff --git a/CLAUDE.md b/CLAUDE.md index 726a861..c85f7aa 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Overview -This is a personal infrastructure Ansible playbook that automates deployment and management of 25+ self-hosted Docker services across two domains (`thesatelliteoflove.com` and `nerder.land`). The setup uses Tailscale VPN for secure networking and Caddy for reverse proxy with automated HTTPS. +This is a personal infrastructure Ansible playbook that automates deployment and management of 22+ self-hosted Docker services across two domains (`thesatelliteoflove.com` and `nerder.land`). The setup uses Tailscale VPN for secure networking and Caddy for reverse proxy with automated HTTPS. **Important**: Always review `DEPLOYMENT_LEARNINGS.md` when working on this repository for lessons learned and troubleshooting guidance. @@ -32,6 +32,11 @@ ansible-playbook site.yml -i hosts.yml --tags caddy --vault-password-file vault_ ansible-playbook site.yml -i hosts.yml --tags authentik,gitea --vault-password-file vault_pass --extra-vars "@secrets.enc" ansible-playbook site.yml -i hosts.yml --tags docker --vault-password-file vault_pass --extra-vars "@secrets.enc" # all docker services +# Deploy services by category (new organized structure) +ansible-playbook site.yml -i hosts.yml --tags infrastructure --vault-password-file vault_pass --extra-vars "@secrets.enc" +ansible-playbook site.yml -i hosts.yml --tags media,productivity --vault-password-file vault_pass --extra-vars "@secrets.enc" +ansible-playbook site.yml -i hosts.yml --tags development,monitoring --vault-password-file vault_pass --extra-vars "@secrets.enc" + # Deploy only infrastructure components ansible-playbook site.yml -i hosts.yml --tags common,cron --vault-password-file vault_pass --extra-vars "@secrets.enc" ``` @@ -45,16 +50,24 @@ ansible-playbook site.yml -i hosts.yml --tags common,cron --vault-password-file ### Role Structure - **bootstrap**: Initial server hardening, user creation, Tailscale VPN setup - **common**: Basic system configuration, UFW firewall management -- **docker**: Comprehensive service deployment (25+ containerized applications) +- **docker**: Comprehensive service deployment (22+ containerized applications, organized by category) - **cron**: Scheduled task management (currently Warhammer RSS feed generation) -### Service Categories in Docker Role -- **Infrastructure**: Caddy (reverse proxy), Authentik (SSO), Dockge (container management) -- **Development**: Gitea, Code Server, Matrix (Conduit) -- **Media**: Audiobookshelf, Calibre, Ghost blog, Pinchflat -- **Productivity**: Paperless-NGX, TasksMD/MMDL, Baikal (CalDAV/CardDAV), Syncthing -- **Communication**: GoToSocial (Fediverse), Postiz (social media management) -- **Monitoring**: Changedetection, Glance dashboard, AppriseAPI +### Docker Role Organization (Reorganized into Logical Categories) +The docker role is now organized into logical service groups under `roles/docker/tasks/`: + +- **infrastructure/**: Core platform components + - Caddy (reverse proxy), Authentik (SSO), Dockge (container management) +- **development/**: Development and collaboration tools + - Gitea, Code Server, Matrix (Conduit) +- **media/**: Content creation and consumption + - Audiobookshelf, Calibre, Ghost blog, Pinchflat, Pinry, Hoarder, Manyfold +- **productivity/**: Personal organization and document management + - Paperless-NGX, MMDL, Baikal (CalDAV/CardDAV), Syncthing, Heyform, Dawarich, Pingvin +- **communication/**: Social media and external communication + - GoToSocial (Fediverse), Postiz (social media management) +- **monitoring/**: System monitoring and alerts + - Changedetection, Glance dashboard, AppriseAPI ### Data Structure - All service data stored in `/opt/stacks/[service-name]/` on docker host @@ -64,10 +77,11 @@ ansible-playbook site.yml -i hosts.yml --tags common,cron --vault-password-file ## Key Implementation Details ### Template-Driven Configuration -The docker role uses Jinja2 templates extensively. When modifying services: +The docker role uses Jinja2 templates exclusively for all services. When modifying services: - Update templates in `roles/docker/templates/[service]-compose.yml.j2` - Environment files use `.env.j2` templates where needed -- Main task files in `roles/docker/tasks/` include service-specific deployment logic +- Task files organized by category in `roles/docker/tasks/[category]/[service].yml` +- All services now use templated configurations (no static compose files) ### DNS Management The `dns.yml` playbook manages AWS Route53 records for both domains. All subdomains point to the netcup server (152.53.36.98), with Caddy handling internal routing to the docker host via Tailscale. diff --git a/README.md b/README.md index 629ca20..2891733 100644 --- a/README.md +++ b/README.md @@ -10,17 +10,44 @@ The playbook manages two main environments: ## Services Deployed -The Docker role deploys and manages the following self-hosted services: +The Docker role deploys and manages 22+ self-hosted services organized into logical categories: -- **Authentication**: Authentik (SSO/Identity Provider) -- **Media**: Audiobookshelf, Calibre, Pinchflat -- **Productivity**: Ghost blog, Gitea, Code Server, Grist, TasksMD, Stirling PDF, MMDL (Task Management) -- **Communication**: GoToSocial, Matrix (Conduit) -- **File Management**: Hoarder, Paperless-NGX, Syncthing, Manyfold -- **Monitoring**: Changedetection, Glance dashboard, Dawarich location tracking -- **Utilities**: Baikal (CalDAV/CardDAV), HeyForm, Pingvin Share, Pinry -- **Notifications**: Apprise API -- **Reverse Proxy**: Caddy +### Infrastructure +- **Caddy** (Reverse proxy with automatic HTTPS) +- **Authentik** (SSO/Identity Provider) +- **Dockge** (Container management) + +### Development +- **Gitea** (Git repository hosting) +- **Code Server** (VS Code in browser) +- **Matrix Conduit** (Communication/chat) + +### Media +- **Audiobookshelf** (Audiobook server) +- **Calibre** (E-book management) +- **Ghost** (Blog platform) +- **Pinchflat** (Media downloader) +- **Pinry** (Pinterest-like board) +- **Hoarder** (Bookmark manager) +- **Manyfold** (3D model organizer) + +### Productivity +- **Paperless-NGX** (Document management) +- **MMDL** (Task management) +- **Baikal** (CalDAV/CardDAV server) +- **Syncthing** (File synchronization) +- **HeyForm** (Form builder) +- **Dawarich** (Location tracking) +- **Pingvin Share** (File sharing) + +### Communication +- **GoToSocial** (Fediverse/Mastodon) +- **Postiz** (Social media management) + +### Monitoring +- **Changedetection** (Website change monitoring) +- **Glance** (Dashboard) +- **AppriseAPI** (Notification service) ## Structure @@ -61,13 +88,15 @@ Manages scheduled tasks and automation: - Supports easy addition of new scheduled tasks ### [Docker Role](roles/docker/README.md) -The most comprehensive role, deploying 25+ containerized services: -- **Core Infrastructure**: Caddy reverse proxy, Authentik SSO, Dockge management -- **Development Tools**: Gitea, Code Server, Matrix communication -- **Media Management**: Audiobookshelf, Calibre, Ghost blog -- **Productivity**: Paperless-NGX, Baikal calendar, Glance dashboard -- **Security Features**: Centralized authentication, network isolation, container hardening -- **Monitoring**: Comprehensive service health monitoring and alerting +The most comprehensive role, deploying 22+ containerized services organized into logical categories: +- **Infrastructure**: Caddy reverse proxy, Authentik SSO, Dockge management +- **Development**: Gitea, Code Server, Matrix communication +- **Media**: Audiobookshelf, Calibre, Ghost blog, Pinchflat, and more +- **Productivity**: Paperless-NGX, MMDL task management, Baikal calendar +- **Communication**: GoToSocial, Postiz social media management +- **Monitoring**: Glance dashboard, Changedetection, AppriseAPI notifications +- **Template-Driven**: All services use Jinja2 templates for consistent configuration +- **Category-Based Deployment**: Deploy services by category using Ansible tags ## Usage @@ -97,15 +126,15 @@ This will: ansible-playbook site.yml -i hosts.yml ``` -Or deploy specific services using tags: +Deploy specific services using tags: ```bash -# Deploy only Caddy reverse proxy +# Deploy by service category +ansible-playbook site.yml -i hosts.yml --tags infrastructure +ansible-playbook site.yml -i hosts.yml --tags media,productivity + +# Deploy individual services ansible-playbook site.yml -i hosts.yml --tags caddy - -# Deploy authentication services ansible-playbook site.yml -i hosts.yml --tags authentik - -# Deploy task management ansible-playbook site.yml -i hosts.yml --tags mmdl ``` diff --git a/dns.yml b/dns.yml index bb250a3..e24cbce 100644 --- a/dns.yml +++ b/dns.yml @@ -33,8 +33,6 @@ ip: "152.53.36.98" - name: loclog ip: "152.53.36.98" - - name: habits - ip: "152.53.36.98" - name: watcher ip: "152.53.36.98" - name: chat @@ -43,6 +41,12 @@ ip: "152.53.36.98" - name: tasks ip: "152.53.36.98" + - name: post + ip: "152.53.36.98" + - name: files + ip: "152.53.36.98" + - name: bookmarks + ip: "152.53.36.98" - name: nerder.land dns_records: - name: "forms" diff --git a/roles/docker/README.md b/roles/docker/README.md index 9dfc835..445375d 100644 --- a/roles/docker/README.md +++ b/roles/docker/README.md @@ -1,7 +1,7 @@ # Docker Role ## Purpose -Deploys and manages a comprehensive self-hosted infrastructure with 25+ containerized services, transforming a server into a personal cloud platform with authentication, media management, productivity tools, and development services. +Deploys and manages a comprehensive self-hosted infrastructure with 22+ containerized services organized into logical categories, transforming a server into a personal cloud platform with authentication, media management, productivity tools, and development services. ## Architecture Overview @@ -17,43 +17,44 @@ Deploys and manages a comprehensive self-hosted infrastructure with 25+ containe - **Container Hardening**: Non-root users, capability dropping, security options - **Secret Management**: Ansible vault for sensitive configuration -## Services Deployed +## Services Deployed (Organized by Category) -### Core Infrastructure +### Infrastructure (`infrastructure/`) - **Caddy** - Reverse proxy with automatic HTTPS (static IP: 172.20.0.5) -- **Dockge** - Docker compose stack management UI - **Authentik** - Enterprise authentication server (OIDC/SAML SSO) +- **Dockge** - Docker compose stack management UI -### Development & Code Management +### Development (`development/`) - **Gitea** - Self-hosted Git with CI/CD runners - **Code Server** - VS Code in the browser - **Conduit** - Matrix homeserver for communication -### Media & Content Management +### Media (`media/`) - **Audiobookshelf** - Audiobook and podcast server - **Calibre** - E-book management and conversion - **Ghost** - Modern blogging platform -- **Hoarder** - Bookmark management with AI tagging -- **Pinry** - Pinterest-like image board -- **Pingvin Share** - File sharing service -- **Syncthing** - Decentralized file sync - -### Productivity & Organization -- **Paperless-ngx** - Document management with OCR -- **Baikal** - CalDAV/CardDAV server -- **Glance** - Customizable dashboard with monitoring -- **Heyform** - Form builder and surveys -- **Postiz** - Social media management -- **Dawarich** - Location tracking -- **Change Detection** - Website monitoring -- **Manyfold** - 3D model file organization -- **MMDL** - Task and calendar management with CalDAV integration - -### Utilities & Tools -- **Stirling PDF** - PDF manipulation (internal network only) - **Pinchflat** - YouTube video archiving -- **Apprise API** - Unified notifications +- **Pinry** - Pinterest-like image board +- **Hoarder** - Bookmark management with AI tagging +- **Manyfold** - 3D model file organization + +### Productivity (`productivity/`) +- **Paperless-ngx** - Document management with OCR +- **MMDL** - Task and calendar management with CalDAV integration +- **Baikal** - CalDAV/CardDAV server +- **Syncthing** - Decentralized file sync +- **Heyform** - Form builder and surveys +- **Dawarich** - Location tracking +- **Pingvin Share** - File sharing service + +### Communication (`communication/`) - **GoToSocial** - Lightweight ActivityPub server +- **Postiz** - Social media management + +### Monitoring (`monitoring/`) +- **Glance** - Customizable dashboard with monitoring +- **Change Detection** - Website monitoring +- **Apprise API** - Unified notifications ## Deployment Patterns @@ -79,22 +80,32 @@ The role also configures the shell environment: ``` roles/docker/ ├── tasks/ -│ ├── main.yml # Orchestrates all deployments -│ ├── shell.yml # Shell environment setup -│ ├── caddy.yml # Reverse proxy -│ ├── authentik.yml # Authentication -│ ├── mmdl.yml # Task management -│ └── [25+ service files] # Individual service deployments +│ ├── main.yml # Orchestrates all deployments +│ ├── shell.yml # Shell environment setup +│ ├── infrastructure/ +│ │ ├── main.yml # Infrastructure category orchestrator +│ │ ├── caddy.yml # Reverse proxy +│ │ └── authentik.yml # Authentication +│ ├── development/ +│ │ ├── main.yml # Development category orchestrator +│ │ ├── gitea.yml # Git hosting +│ │ ├── codeserver.yml # VS Code server +│ │ └── conduit.yml # Matrix server +│ ├── media/ # Media services (7 services) +│ ├── productivity/ # Productivity services (7 services) +│ ├── communication/ # Communication services (2 services) +│ └── monitoring/ # Monitoring services (3 services) ├── templates/ -│ ├── [service]-compose.yml.j2 # Docker Compose templates -│ ├── [service]-env.j2 # Environment variable templates -│ └── mmdl-*.j2 # MMDL-specific templates +│ ├── [service]-compose.yml.j2 # Docker Compose templates (all templated) +│ ├── [service]-env.j2 # Environment variable templates +│ └── [service]-*.j2 # Service-specific templates ├── files/ -│ ├── Caddyfile # Caddy configuration -│ ├── ufw-docker.sh # Firewall integration script -│ └── [various configs] # Static configuration files +│ ├── Caddyfile # Caddy configuration +│ ├── ufw-docker.sh # Firewall integration script +│ ├── client # Matrix well-known client file +│ └── server # Matrix well-known server file └── handlers/ - └── main.yml # Service restart handlers + └── main.yml # Service restart handlers ``` ## Usage @@ -104,24 +115,26 @@ roles/docker/ ansible-playbook site.yml -i hosts.yml --tags docker ``` -### Deploy Specific Services +### Deploy by Service Category ```bash -# Deploy only authentication stack -ansible-playbook site.yml -i hosts.yml --tags authentik +# Deploy entire service categories +ansible-playbook site.yml -i hosts.yml --tags infrastructure +ansible-playbook site.yml -i hosts.yml --tags development +ansible-playbook site.yml -i hosts.yml --tags media +ansible-playbook site.yml -i hosts.yml --tags productivity +ansible-playbook site.yml -i hosts.yml --tags communication +ansible-playbook site.yml -i hosts.yml --tags monitoring -# Deploy media services -ansible-playbook site.yml -i hosts.yml --tags audiobookshelf,calibre - -# Deploy development tools -ansible-playbook site.yml -i hosts.yml --tags gitea,codeserver - -# Deploy task management -ansible-playbook site.yml -i hosts.yml --tags mmdl +# Deploy multiple categories +ansible-playbook site.yml -i hosts.yml --tags infrastructure,monitoring ``` -### Deploy Core Infrastructure Only +### Deploy Individual Services ```bash -ansible-playbook site.yml -i hosts.yml --tags caddy,authentik,glance +# Deploy specific services +ansible-playbook site.yml -i hosts.yml --tags authentik +ansible-playbook site.yml -i hosts.yml --tags gitea,codeserver +ansible-playbook site.yml -i hosts.yml --tags mmdl ``` ## Service-Specific Notes diff --git a/roles/docker/files/Caddyfile b/roles/docker/files/Caddyfile index 3d11ad3..48e5d8c 100644 --- a/roles/docker/files/Caddyfile +++ b/roles/docker/files/Caddyfile @@ -44,9 +44,6 @@ phlog.thesatelliteoflove.com { reverse_proxy ghost-1-ghost-1:2368 } -habits.thesatelliteoflove.com { - reverse_proxy beaverhabits:8080 -} code.thesatelliteoflove.com { reverse_proxy authentik-server-1:9000 @@ -88,27 +85,11 @@ models.thesatelliteoflove.com { reverse_proxy manyfold-app-1:3214 } -grist.thesatelliteoflove.com { - reverse_proxy grist-grist-1:8484 -} home.thesatelliteoflove.com { reverse_proxy authentik-server-1:9000 } -pdftools.thesatelliteoflove.com:80 { - @allowed { - remote_ip 100.64.0.0/10 - } - - handle @allowed { - reverse_proxy stirling-stirlingpdf-1:8080 - } - - handle { - respond "Access denied" 403 - } -} repair.nerder.land { root * /srv/repair diff --git a/roles/docker/files/tasksmd-compose.yml b/roles/docker/files/tasksmd-compose.yml deleted file mode 100644 index a14224b..0000000 --- a/roles/docker/files/tasksmd-compose.yml +++ /dev/null @@ -1,21 +0,0 @@ -version: "3" -services: - tasks.md: - image: baldissaramatheus/tasks.md:2.5.4 - container_name: tasksmd - environment: - - PUID=1000 - - PGID=1000 - volumes: - - tasksmd-data:/tasks - - tasksmd-config:/config - restart: unless-stopped -volumes: - tasksmd-data: - driver: local - tasksmd-config: - driver: local -networks: - default: - external: true - name: lava \ No newline at end of file diff --git a/roles/docker/tasks/development/grist.yml b/roles/docker/tasks/development/grist.yml deleted file mode 100644 index e797ba5..0000000 --- a/roles/docker/tasks/development/grist.yml +++ /dev/null @@ -1,19 +0,0 @@ -- name: make grist directories - ansible.builtin.file: - path: "{{ item}}" - state: directory - loop: - - /opt/stacks/grist - -- name: Template out the compose file - ansible.builtin.template: - src: grist-compose.yml.j2 - dest: /opt/stacks/grist/compose.yml - owner: root - mode: 644 - -- name: deploy grist stack - community.docker.docker_compose_v2: - project_src: /opt/stacks/grist - files: - - compose.yml \ No newline at end of file diff --git a/roles/docker/tasks/development/main.yml b/roles/docker/tasks/development/main.yml index a959797..bf8230b 100644 --- a/roles/docker/tasks/development/main.yml +++ b/roles/docker/tasks/development/main.yml @@ -11,9 +11,4 @@ - name: Install conduit import_tasks: conduit.yml - tags: conduit - -# Commented services can be uncommented when ready -#- name: Install grist -# import_tasks: grist.yml -# tags: grist \ No newline at end of file + tags: conduit \ No newline at end of file diff --git a/roles/docker/tasks/infrastructure/caddy.yml b/roles/docker/tasks/infrastructure/caddy.yml index 1908288..fbe0c77 100644 --- a/roles/docker/tasks/infrastructure/caddy.yml +++ b/roles/docker/tasks/infrastructure/caddy.yml @@ -13,9 +13,9 @@ mode: 644 notify: restart caddy -- name: copy caddy compose file - ansible.builtin.copy: - src: caddy-compose.yml +- name: template caddy compose file + ansible.builtin.template: + src: caddy-compose.yml.j2 dest: /opt/stacks/caddy/compose.yml owner: root mode: 644 diff --git a/roles/docker/tasks/main.yml b/roles/docker/tasks/main.yml index 379b87e..4ab88ff 100644 --- a/roles/docker/tasks/main.yml +++ b/roles/docker/tasks/main.yml @@ -49,9 +49,9 @@ - /opt/stacks - /opt/dockge -- name: copy dockge compose file - ansible.builtin.copy: - src: dockge-compose.yml +- name: template dockge compose file + ansible.builtin.template: + src: dockge-compose.yml.j2 dest: /opt/dockge/dockge.yml owner: root mode: 644 diff --git a/roles/docker/tasks/media/beaver.yml b/roles/docker/tasks/media/beaver.yml deleted file mode 100644 index df81d64..0000000 --- a/roles/docker/tasks/media/beaver.yml +++ /dev/null @@ -1,19 +0,0 @@ -- name: make beaver directories - ansible.builtin.file: - path: "{{ item}}" - state: directory - loop: - - /opt/stacks/beaver - -- name: Template out the compose file - ansible.builtin.template: - src: beaver-compose.yml.j2 - dest: /opt/stacks/beaver/compose.yml - owner: root - mode: 644 - -- name: deploy beaver stack - community.docker.docker_compose_v2: - project_src: /opt/stacks/beaver - files: - - compose.yml \ No newline at end of file diff --git a/roles/docker/tasks/media/hoarder.yml b/roles/docker/tasks/media/hoarder.yml index 8ed528a..d4b9111 100644 --- a/roles/docker/tasks/media/hoarder.yml +++ b/roles/docker/tasks/media/hoarder.yml @@ -5,9 +5,9 @@ loop: - /opt/stacks/hoarder -- name: copy hoarder compose file - ansible.builtin.copy: - src: hoarder-compose.yml +- name: template hoarder compose file + ansible.builtin.template: + src: hoarder-compose.yml.j2 dest: /opt/stacks/hoarder/compose.yml owner: root mode: 644 diff --git a/roles/docker/tasks/media/main.yml b/roles/docker/tasks/media/main.yml index c95e3c2..f605170 100644 --- a/roles/docker/tasks/media/main.yml +++ b/roles/docker/tasks/media/main.yml @@ -27,9 +27,4 @@ - name: Install manyfold import_tasks: manyfold.yml - tags: manyfold - -# Commented services can be uncommented when ready -#- name: Install beaver -# import_tasks: beaver.yml -# tags: beaver \ No newline at end of file + tags: manyfold \ No newline at end of file diff --git a/roles/docker/tasks/productivity/main.yml b/roles/docker/tasks/productivity/main.yml index 7d0f958..c7f81b0 100644 --- a/roles/docker/tasks/productivity/main.yml +++ b/roles/docker/tasks/productivity/main.yml @@ -27,13 +27,4 @@ - name: Install pingvin import_tasks: pingvin.yml - tags: pingvin - -# Commented services can be uncommented when ready -#- name: Install tasksmd -# import_tasks: tasksmd.yml -# tags: tasksmd - -#- name: Install stirlingpdf -# import_tasks: stirlingpdf.yml -# tags: stirlingpdf \ No newline at end of file + tags: pingvin \ No newline at end of file diff --git a/roles/docker/tasks/productivity/stirlingpdf.yml b/roles/docker/tasks/productivity/stirlingpdf.yml deleted file mode 100644 index 9a257ef..0000000 --- a/roles/docker/tasks/productivity/stirlingpdf.yml +++ /dev/null @@ -1,19 +0,0 @@ -- name: make StirlingPDF directories - ansible.builtin.file: - path: "{{ item}}" - state: directory - loop: - - /opt/stacks/stirlingpdf - -- name: Template out the compose file - ansible.builtin.template: - src: striling-compose.yml.j2 - dest: /opt/stacks/stirling/compose.yml - owner: root - mode: 644 - -- name: deploy stirling stack - community.docker.docker_compose_v2: - project_src: /opt/stacks/stirling - files: - - compose.yml \ No newline at end of file diff --git a/roles/docker/tasks/productivity/tasksmd.yml b/roles/docker/tasks/productivity/tasksmd.yml deleted file mode 100644 index 1163712..0000000 --- a/roles/docker/tasks/productivity/tasksmd.yml +++ /dev/null @@ -1,19 +0,0 @@ -- name: make tasksmd directories - ansible.builtin.file: - path: "{{ item}}" - state: directory - loop: - - /opt/stacks/tasksmd - -- name: copy tasksmd compose file - ansible.builtin.copy: - src: tasksmd-compose.yml - dest: /opt/stacks/tasksmd/compose.yml - owner: root - mode: 644 - -- name: deploy tasksmd stack - community.docker.docker_compose_v2: - project_src: /opt/stacks/tasksmd - files: - - compose.yml \ No newline at end of file diff --git a/roles/docker/templates/beaver-compose.yml.j2 b/roles/docker/templates/beaver-compose.yml.j2 deleted file mode 100644 index bf76a13..0000000 --- a/roles/docker/templates/beaver-compose.yml.j2 +++ /dev/null @@ -1,20 +0,0 @@ -services: - beaverhabits: - container_name: beaverhabits - user: 1000:1000 - environment: - # See the note below to find all the environment variables - - HABITS_STORAGE=USER_DISK # DATABASE stores in a single SQLite database named habits.db. USER_DISK option saves in a local json file. - - MAX_USER_COUNT=1 - volumes: - - ./data:/app/.user/ # Change directory to match your docker file scheme. - restart: unless-stopped - image: daya0576/beaverhabits:latest - -volumes: - data: - -networks: - default: - external: true - name: lava \ No newline at end of file diff --git a/roles/docker/files/caddy-compose.yml b/roles/docker/templates/caddy-compose.yml.j2 similarity index 100% rename from roles/docker/files/caddy-compose.yml rename to roles/docker/templates/caddy-compose.yml.j2 diff --git a/roles/docker/files/dockge-compose.yml b/roles/docker/templates/dockge-compose.yml.j2 similarity index 100% rename from roles/docker/files/dockge-compose.yml rename to roles/docker/templates/dockge-compose.yml.j2 diff --git a/roles/docker/templates/grist-compose.yml.j2 b/roles/docker/templates/grist-compose.yml.j2 deleted file mode 100644 index da0d952..0000000 --- a/roles/docker/templates/grist-compose.yml.j2 +++ /dev/null @@ -1,23 +0,0 @@ -version: "3.3" -services: - grist: - volumes: - - grist:/persist - extra_hosts: - - 'auth.thesatelliteoflove.com:172.20.0.3' - environment: - - GRIST_SESSION_SECRET={{ grist_session_secret }} - - APP_HOME_URL=https://grist.thesatelliteoflove.com - - GRIST_OIDC_IDP_ISSUER=https://auth.thesatelliteoflove.com/application/o/grist/.well-known/openid-configuration - - GRIST_OIDC_IDP_CLIENT_ID={{ grist_oidc_client_id }} - - GRIST_OIDC_IDP_CLIENT_SECRET={{ grist_oidc_client_secret }} - image: gristlabs/grist - -volumes: - grist: - driver: local - -networks: - default: - external: true - name: lava diff --git a/roles/docker/files/hoarder-compose.yml b/roles/docker/templates/hoarder-compose.yml.j2 similarity index 100% rename from roles/docker/files/hoarder-compose.yml rename to roles/docker/templates/hoarder-compose.yml.j2 diff --git a/roles/docker/templates/redlib-compose.yml.j2 b/roles/docker/templates/redlib-compose.yml.j2 deleted file mode 100644 index c24ac66..0000000 --- a/roles/docker/templates/redlib-compose.yml.j2 +++ /dev/null @@ -1,24 +0,0 @@ -services: - redlib: - image: quay.io/redlib/redlib:latest - restart: always - container_name: "redlib" - user: nobody - read_only: true - security_opt: - - no-new-privileges:true - # - seccomp=seccomp-redlib.json - cap_drop: - - ALL - env_file: .env - networks: - - redlib - healthcheck: - test: ["CMD", "wget", "--spider", "-q", "--tries=1", "http://localhost:8080/settings"] - interval: 5m - timeout: 3s - -networks: - default: - external: true - name: lava \ No newline at end of file diff --git a/roles/docker/templates/stirling-compose.yml.j2 b/roles/docker/templates/stirling-compose.yml.j2 deleted file mode 100644 index a903680..0000000 --- a/roles/docker/templates/stirling-compose.yml.j2 +++ /dev/null @@ -1,18 +0,0 @@ -version: '3.3' -services: - stirlingpdf: - image: frooodle/s-pdf:latest - volumes: - - ./trainingData:/usr/share/tessdata #Required for extra OCR languages - - ./extraConfigs:/configs -# - ./customFiles:/customFiles/ -# - ./logs:/logs/ - environment: - - DOCKER_ENABLE_SECURITY=false - - INSTALL_BOOK_AND_ADVANCED_HTML_OPS=false - - LANGS=en_GB - -networks: - default: - external: true - name: lava diff --git a/todo.md b/todo.md index 9821945..680a105 100644 --- a/todo.md +++ b/todo.md @@ -2,7 +2,7 @@ ## High Priority (Quick Wins) -### 1. Split the massive docker role ⚠️ IN PROGRESS +### 1. Split the massive docker role ✅ COMPLETED - **Current Issue**: `roles/docker/tasks/main.yml` has 20+ services in one file (176 lines) - **Solution**: Break into logical service groups: ``` @@ -10,10 +10,12 @@ ├── main.yml (orchestrator) ├── infrastructure/ (caddy, authentik, dockge) ├── development/ (gitea, codeserver, conduit) - ├── media/ (audiobookshelf, calibre, ghost, pinchflat) - ├── productivity/ (paperless, baikal, syncthing, tasksmd) + ├── media/ (audiobookshelf, calibre, ghost, pinchflat, pinry, hoarder, manyfold) + ├── productivity/ (paperless, baikal, syncthing, mmdl, heyform, dawarich, pingvin) + ├── communication/ (gotosocial, postiz) └── monitoring/ (glance, changedetection, appriseapi) ``` +- **COMPLETED**: All services organized into logical categories with category-level tags ### 2. Standardize variable management - **Current Issue**: Secrets in single encrypted file, no clear variable hierarchy @@ -28,9 +30,11 @@ │ └── networking.yml (network settings) ``` -### 3. Template consolidation -- **Current Issue**: Many compose templates repeat patterns +### 3. Template consolidation ✅ PARTIALLY COMPLETED +- **Current Issue**: Many compose templates repeat patterns, some services used static files - **Solution**: Create reusable template includes with standard service template structure +- **COMPLETED**: Converted all static compose files (caddy, dockge, hoarder) to Jinja2 templates +- **REMAINING**: Create reusable template patterns for common configurations ## Security & Reliability @@ -91,9 +95,12 @@ ### Week 1: Foundation - [x] Document improvements in todo.md -- [ ] Reorganize docker role structure +- [x] Reorganize docker role structure +- [x] Convert static compose files to templates +- [x] Remove unused services (beaver, grist, stirlingpdf, tasksmd, redlib) +- [x] Clean up templates and files directories - [ ] Implement variable hierarchy -- [ ] Standardize templates +- [ ] Create reusable template patterns ### Week 2: Security & Monitoring - [ ] Add health checks @@ -110,7 +117,23 @@ - [ ] Dependency management - [ ] Monitoring dashboard +## Completed Work Summary + +### ✅ Major Accomplishments +- **Docker Role Reorganization**: Split monolithic 176-line main.yml into 6 logical service categories +- **Template Standardization**: Converted all static compose files to Jinja2 templates +- **Service Cleanup**: Removed 5 unused/broken services (beaver, grist, stirlingpdf, tasksmd, redlib) +- **Category-Based Deployment**: Can now deploy services by category using tags (infrastructure, media, etc.) +- **Documentation Updates**: Updated CLAUDE.md to reflect new architecture + +### 📊 Current Stats +- **22+ active services** organized into 6 categories +- **100% templated** compose files (no static files) +- **6 service directories** for logical organization +- **Clean file structure** with only essential static files + ## Notes -- Current architecture is solid but needs better organization for long-term maintainability +- Current architecture is solid and much better organized for long-term maintainability - Focus on high-impact, low-effort improvements first -- Leverage existing infrastructure (Gitea, Authentik) for automation \ No newline at end of file +- Leverage existing infrastructure (Gitea, Authentik) for automation +- Template-driven approach enables future dynamic configuration \ No newline at end of file