feat: complete infrastructure cleanup and optimization

This comprehensive update improves maintainability and removes unused services:

## Major Changes
- Remove 5 unused services (beaver, grist, stirlingpdf, tasksmd, redlib)
- Convert remaining static compose files to Jinja2 templates
- Clean up Caddyfile removing orphaned proxy configurations
- Align DNS records with active services

## Service Cleanup
- Remove habits.thesatelliteoflove.com DNS record (beaver service)
- Add missing DNS records for active services:
  - post.thesatelliteoflove.com (Postiz)
  - files.thesatelliteoflove.com (Pingvin Share)
  - bookmarks.thesatelliteoflove.com (Hoarder)

## Template Standardization
- Convert caddy-compose.yml to template
- Convert dockge-compose.yml to template
- Convert hoarder-compose.yml to template
- All services now use consistent template-driven approach

## Documentation Updates
- Update CLAUDE.md with new service organization
- Update README.md files with category-based deployment examples
- Update todo.md with completed work summary
- Service count updated to 22+ active services

Infrastructure is now fully organized, cleaned up, and ready for future enhancements.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

roles/docker/README.md

@@ -1,7 +1,7 @@
 # Docker Role
 
 ## Purpose
-Deploys and manages a comprehensive self-hosted infrastructure with 25+ containerized services, transforming a server into a personal cloud platform with authentication, media management, productivity tools, and development services.
+Deploys and manages a comprehensive self-hosted infrastructure with 22+ containerized services organized into logical categories, transforming a server into a personal cloud platform with authentication, media management, productivity tools, and development services.
 
 ## Architecture Overview
 
@@ -17,43 +17,44 @@ Deploys and manages a comprehensive self-hosted infrastructure with 25+ containe
 - **Container Hardening**: Non-root users, capability dropping, security options
 - **Secret Management**: Ansible vault for sensitive configuration
 
-## Services Deployed
+## Services Deployed (Organized by Category)
 
-### Core Infrastructure
+### Infrastructure (`infrastructure/`)
 - **Caddy** - Reverse proxy with automatic HTTPS (static IP: 172.20.0.5)
-- **Dockge** - Docker compose stack management UI
 - **Authentik** - Enterprise authentication server (OIDC/SAML SSO)
+- **Dockge** - Docker compose stack management UI
 
-### Development & Code Management
+### Development (`development/`)
 - **Gitea** - Self-hosted Git with CI/CD runners
 - **Code Server** - VS Code in the browser
 - **Conduit** - Matrix homeserver for communication
 
-### Media & Content Management
+### Media (`media/`)
 - **Audiobookshelf** - Audiobook and podcast server
 - **Calibre** - E-book management and conversion
 - **Ghost** - Modern blogging platform
-- **Hoarder** - Bookmark management with AI tagging
-- **Pinry** - Pinterest-like image board
-- **Pingvin Share** - File sharing service
-- **Syncthing** - Decentralized file sync
-
-### Productivity & Organization
-- **Paperless-ngx** - Document management with OCR
-- **Baikal** - CalDAV/CardDAV server
-- **Glance** - Customizable dashboard with monitoring
-- **Heyform** - Form builder and surveys
-- **Postiz** - Social media management
-- **Dawarich** - Location tracking
-- **Change Detection** - Website monitoring
-- **Manyfold** - 3D model file organization
-- **MMDL** - Task and calendar management with CalDAV integration
-
-### Utilities & Tools
-- **Stirling PDF** - PDF manipulation (internal network only)
 - **Pinchflat** - YouTube video archiving
-- **Apprise API** - Unified notifications
+- **Pinry** - Pinterest-like image board
+- **Hoarder** - Bookmark management with AI tagging
+- **Manyfold** - 3D model file organization
+
+### Productivity (`productivity/`)
+- **Paperless-ngx** - Document management with OCR
+- **MMDL** - Task and calendar management with CalDAV integration
+- **Baikal** - CalDAV/CardDAV server
+- **Syncthing** - Decentralized file sync
+- **Heyform** - Form builder and surveys
+- **Dawarich** - Location tracking
+- **Pingvin Share** - File sharing service
+
+### Communication (`communication/`)
 - **GoToSocial** - Lightweight ActivityPub server
+- **Postiz** - Social media management
+
+### Monitoring (`monitoring/`)
+- **Glance** - Customizable dashboard with monitoring
+- **Change Detection** - Website monitoring
+- **Apprise API** - Unified notifications
 
 ## Deployment Patterns
 
@@ -79,22 +80,32 @@ The role also configures the shell environment:
 ```
 roles/docker/
 ├── tasks/
-│   ├── main.yml              # Orchestrates all deployments
-│   ├── shell.yml             # Shell environment setup
-│   ├── caddy.yml             # Reverse proxy
-│   ├── authentik.yml         # Authentication
-│   ├── mmdl.yml              # Task management
-│   └── [25+ service files]   # Individual service deployments
+│   ├── main.yml                    # Orchestrates all deployments
+│   ├── shell.yml                   # Shell environment setup
+│   ├── infrastructure/
+│   │   ├── main.yml               # Infrastructure category orchestrator
+│   │   ├── caddy.yml              # Reverse proxy
+│   │   └── authentik.yml          # Authentication
+│   ├── development/
+│   │   ├── main.yml               # Development category orchestrator
+│   │   ├── gitea.yml              # Git hosting
+│   │   ├── codeserver.yml         # VS Code server
+│   │   └── conduit.yml            # Matrix server
+│   ├── media/                     # Media services (7 services)
+│   ├── productivity/              # Productivity services (7 services)
+│   ├── communication/             # Communication services (2 services)
+│   └── monitoring/                # Monitoring services (3 services)
 ├── templates/
-│   ├── [service]-compose.yml.j2  # Docker Compose templates
-│   ├── [service]-env.j2          # Environment variable templates
-│   └── mmdl-*.j2                 # MMDL-specific templates
+│   ├── [service]-compose.yml.j2   # Docker Compose templates (all templated)
+│   ├── [service]-env.j2           # Environment variable templates
+│   └── [service]-*.j2             # Service-specific templates
 ├── files/
-│   ├── Caddyfile             # Caddy configuration
-│   ├── ufw-docker.sh         # Firewall integration script
-│   └── [various configs]     # Static configuration files
+│   ├── Caddyfile                  # Caddy configuration
+│   ├── ufw-docker.sh              # Firewall integration script
+│   ├── client                     # Matrix well-known client file
+│   └── server                     # Matrix well-known server file
 └── handlers/
-    └── main.yml              # Service restart handlers
+    └── main.yml                   # Service restart handlers
 ```
 
 ## Usage
@@ -104,24 +115,26 @@ roles/docker/
 ansible-playbook site.yml -i hosts.yml --tags docker
 ```
 
-### Deploy Specific Services
+### Deploy by Service Category
 ```bash
-# Deploy only authentication stack
-ansible-playbook site.yml -i hosts.yml --tags authentik
+# Deploy entire service categories
+ansible-playbook site.yml -i hosts.yml --tags infrastructure
+ansible-playbook site.yml -i hosts.yml --tags development
+ansible-playbook site.yml -i hosts.yml --tags media
+ansible-playbook site.yml -i hosts.yml --tags productivity
+ansible-playbook site.yml -i hosts.yml --tags communication
+ansible-playbook site.yml -i hosts.yml --tags monitoring
 
-# Deploy media services
-ansible-playbook site.yml -i hosts.yml --tags audiobookshelf,calibre
-
-# Deploy development tools
-ansible-playbook site.yml -i hosts.yml --tags gitea,codeserver
-
-# Deploy task management
-ansible-playbook site.yml -i hosts.yml --tags mmdl
+# Deploy multiple categories
+ansible-playbook site.yml -i hosts.yml --tags infrastructure,monitoring
 ```
 
-### Deploy Core Infrastructure Only
+### Deploy Individual Services
 ```bash
-ansible-playbook site.yml -i hosts.yml --tags caddy,authentik,glance
+# Deploy specific services
+ansible-playbook site.yml -i hosts.yml --tags authentik
+ansible-playbook site.yml -i hosts.yml --tags gitea,codeserver
+ansible-playbook site.yml -i hosts.yml --tags mmdl
 ```
 
 ## Service-Specific Notes

roles/docker/files/Caddyfile

@@ -44,9 +44,6 @@ phlog.thesatelliteoflove.com {
     reverse_proxy ghost-1-ghost-1:2368
 }
 
-habits.thesatelliteoflove.com {
-    reverse_proxy beaverhabits:8080
-}
 
 code.thesatelliteoflove.com {
     reverse_proxy authentik-server-1:9000
@@ -88,27 +85,11 @@ models.thesatelliteoflove.com {
     reverse_proxy manyfold-app-1:3214
 }
 
-grist.thesatelliteoflove.com {
-    reverse_proxy grist-grist-1:8484
-}
 
 home.thesatelliteoflove.com {
     reverse_proxy authentik-server-1:9000
 }
 
-pdftools.thesatelliteoflove.com:80 {
-    @allowed {
-        remote_ip 100.64.0.0/10
-    }
-
-    handle @allowed {
-        reverse_proxy stirling-stirlingpdf-1:8080
-    }
-
-    handle {
-        respond "Access denied" 403
-    }
-}
 
 repair.nerder.land {
     root * /srv/repair

roles/docker/files/tasksmd-compose.yml

@@ -1,21 +0,0 @@
-version: "3"
-services:
-  tasks.md:
-    image: baldissaramatheus/tasks.md:2.5.4
-    container_name: tasksmd
-    environment:
-      - PUID=1000
-      - PGID=1000
-    volumes:
-      - tasksmd-data:/tasks
-      - tasksmd-config:/config
-    restart: unless-stopped
-volumes:
-  tasksmd-data:
-    driver: local
-  tasksmd-config:
-    driver: local
-networks:
-  default:
-    external: true
-    name: lava

roles/docker/tasks/development/grist.yml

@@ -1,19 +0,0 @@
-- name: make grist directories
-  ansible.builtin.file:
-    path: "{{ item}}"
-    state: directory
-  loop:
-    - /opt/stacks/grist
-
-- name: Template out the compose file
-  ansible.builtin.template:
-    src: grist-compose.yml.j2
-    dest: /opt/stacks/grist/compose.yml
-    owner: root
-    mode: 644
-
-- name: deploy grist stack
-  community.docker.docker_compose_v2:
-    project_src: /opt/stacks/grist
-    files:
-    - compose.yml

roles/docker/tasks/development/main.yml

@@ -11,9 +11,4 @@
 
 - name: Install conduit
   import_tasks: conduit.yml
-  tags: conduit
-
-# Commented services can be uncommented when ready
-#- name: Install grist
-#  import_tasks: grist.yml
-#  tags: grist
+  tags: conduit

roles/docker/tasks/infrastructure/caddy.yml

@@ -13,9 +13,9 @@
     mode: 644
   notify: restart caddy
 
-- name: copy caddy compose file
-  ansible.builtin.copy:
-    src: caddy-compose.yml
+- name: template caddy compose file
+  ansible.builtin.template:
+    src: caddy-compose.yml.j2
     dest: /opt/stacks/caddy/compose.yml
     owner: root
     mode: 644

roles/docker/tasks/main.yml

@@ -49,9 +49,9 @@
     - /opt/stacks
     - /opt/dockge
 
-- name: copy dockge compose file
-  ansible.builtin.copy:
-    src: dockge-compose.yml
+- name: template dockge compose file
+  ansible.builtin.template:
+    src: dockge-compose.yml.j2
     dest: /opt/dockge/dockge.yml
     owner: root
     mode: 644

roles/docker/tasks/media/beaver.yml

@@ -1,19 +0,0 @@
-- name: make beaver directories
-  ansible.builtin.file:
-    path: "{{ item}}"
-    state: directory
-  loop:
-    - /opt/stacks/beaver
-
-- name: Template out the compose file
-  ansible.builtin.template:
-    src: beaver-compose.yml.j2
-    dest: /opt/stacks/beaver/compose.yml
-    owner: root
-    mode: 644
-
-- name: deploy beaver stack
-  community.docker.docker_compose_v2:
-    project_src: /opt/stacks/beaver
-    files:
-    - compose.yml

roles/docker/tasks/media/hoarder.yml

@@ -5,9 +5,9 @@
   loop:
     - /opt/stacks/hoarder
 
-- name: copy hoarder compose file
-  ansible.builtin.copy:
-    src: hoarder-compose.yml
+- name: template hoarder compose file
+  ansible.builtin.template:
+    src: hoarder-compose.yml.j2
     dest: /opt/stacks/hoarder/compose.yml
     owner: root
     mode: 644

roles/docker/tasks/media/main.yml

@@ -27,9 +27,4 @@
 
 - name: Install manyfold
   import_tasks: manyfold.yml
-  tags: manyfold
-
-# Commented services can be uncommented when ready
-#- name: Install beaver
-#  import_tasks: beaver.yml
-#  tags: beaver
+  tags: manyfold

roles/docker/tasks/productivity/main.yml

@@ -27,13 +27,4 @@
 
 - name: Install pingvin
   import_tasks: pingvin.yml
-  tags: pingvin
-
-# Commented services can be uncommented when ready
-#- name: Install tasksmd
-#  import_tasks: tasksmd.yml
-#  tags: tasksmd
-
-#- name: Install stirlingpdf
-#  import_tasks: stirlingpdf.yml
-#  tags: stirlingpdf
+  tags: pingvin

roles/docker/tasks/productivity/stirlingpdf.yml

@@ -1,19 +0,0 @@
-- name: make StirlingPDF directories
-  ansible.builtin.file:
-    path: "{{ item}}"
-    state: directory
-  loop:
-    - /opt/stacks/stirlingpdf
-
-- name: Template out the compose file
-  ansible.builtin.template:
-    src: striling-compose.yml.j2
-    dest: /opt/stacks/stirling/compose.yml
-    owner: root
-    mode: 644
-
-- name: deploy stirling stack
-  community.docker.docker_compose_v2:
-    project_src: /opt/stacks/stirling
-    files:
-    - compose.yml

roles/docker/tasks/productivity/tasksmd.yml

@@ -1,19 +0,0 @@
-- name: make tasksmd directories
-  ansible.builtin.file:
-    path: "{{ item}}"
-    state: directory
-  loop:
-    - /opt/stacks/tasksmd
-
-- name: copy tasksmd compose file
-  ansible.builtin.copy:
-    src: tasksmd-compose.yml
-    dest: /opt/stacks/tasksmd/compose.yml
-    owner: root
-    mode: 644
-
-- name: deploy tasksmd stack
-  community.docker.docker_compose_v2:
-    project_src: /opt/stacks/tasksmd
-    files:
-    - compose.yml

roles/docker/templates/beaver-compose.yml.j2

@@ -1,20 +0,0 @@
-services:
-    beaverhabits:
-        container_name: beaverhabits
-        user: 1000:1000
-        environment:
-            # See the note below to find all the environment variables
-            - HABITS_STORAGE=USER_DISK # DATABASE stores in a single SQLite database named habits.db. USER_DISK option saves in a local json file.
-            - MAX_USER_COUNT=1
-        volumes:
-            - ./data:/app/.user/ # Change directory to match your docker file scheme.
-        restart: unless-stopped
-        image: daya0576/beaverhabits:latest
-
-volumes:
-  data:
-
-networks:
-  default:
-    external: true
-    name: lava

roles/docker/templates/grist-compose.yml.j2

@@ -1,23 +0,0 @@
-version: "3.3"
-services:
-  grist:
-    volumes:
-      - grist:/persist
-    extra_hosts:
-      - 'auth.thesatelliteoflove.com:172.20.0.3'
-    environment:
-      - GRIST_SESSION_SECRET={{ grist_session_secret }}
-      - APP_HOME_URL=https://grist.thesatelliteoflove.com
-      - GRIST_OIDC_IDP_ISSUER=https://auth.thesatelliteoflove.com/application/o/grist/.well-known/openid-configuration
-      - GRIST_OIDC_IDP_CLIENT_ID={{ grist_oidc_client_id }}
-      - GRIST_OIDC_IDP_CLIENT_SECRET={{ grist_oidc_client_secret }}
-    image: gristlabs/grist
-
-volumes:
-  grist:
-    driver: local
-
-networks:
-  default:
-    external: true
-    name: lava

roles/docker/templates/redlib-compose.yml.j2

@@ -1,24 +0,0 @@
-services:
-  redlib:
-    image: quay.io/redlib/redlib:latest
-    restart: always
-    container_name: "redlib"
-    user: nobody
-    read_only: true
-    security_opt:
-      - no-new-privileges:true
-      # - seccomp=seccomp-redlib.json
-    cap_drop:
-      - ALL
-    env_file: .env
-    networks:
-      - redlib
-    healthcheck:
-      test: ["CMD", "wget", "--spider", "-q", "--tries=1", "http://localhost:8080/settings"]
-      interval: 5m
-      timeout: 3s
-
-networks:
-  default:
-    external: true
-    name: lava

roles/docker/templates/stirling-compose.yml.j2

@@ -1,18 +0,0 @@
-version: '3.3'
-services:
-  stirlingpdf:
-    image: frooodle/s-pdf:latest
-    volumes:
-      - ./trainingData:/usr/share/tessdata #Required for extra OCR languages
-      - ./extraConfigs:/configs
-#      - ./customFiles:/customFiles/
-#      - ./logs:/logs/
-    environment:
-      - DOCKER_ENABLE_SECURITY=false
-      - INSTALL_BOOK_AND_ADVANCED_HTML_OPS=false
-      - LANGS=en_GB
-
-networks:
-  default:
-    external: true
-    name: lava