Draft the Debian 13 upgrade plan

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

debian-13-upgrade.md

@@ -0,0 +1,99 @@
+# Plan: upgrade cogitator-01 from Debian 12 (bookworm) to 13 (trixie)
+
+Status: draft, started 2026-10-31. Nothing here has been run yet.
+
+Why: Debian 12 has been LTS-only since June 2026, so fewer packages get
+security fixes. Debian 13 is the current stable release.
+
+## Where the server is now (checked 2026-10-31)
+
+- Debian 12.15, kernel 6.1.0-54, Python 3.11, Docker 29.9.2.
+- Disk: 237 GB free of 503 GB on `/`, 819 MB free on `/boot`. RAM 15 GB.
+- `dpkg --audit` is clean; nothing is held back; one package is upgradable.
+- Apt sources: Debian main/updates/security, Docker
+  (`download_docker_com_linux_debian.list`) and Tailscale
+  (`tailscale.list`, set up by hand, not by Ansible). All say `bookworm`.
+- iptables already uses the nftables backend, so ufw and `ufw-docker.sh`
+  should behave the same after the upgrade.
+- `/tmp` is on the root disk, with no fstab entry.
+- Leftovers to tidy: old kernel `linux-image-6.1.0-45`, and config-only
+  kernel packages 6.1.0-9/23/25/37.
+
+## Step 1: repo changes (land and deploy before the upgrade)
+
+1. **Replace `apt_key` in `roles/docker/tasks/main.yml`.** Trixie's apt no
+   longer ships `apt-key`, so this task would fail on the next deploy.
+   Download Docker's key to `/etc/apt/keyrings/docker.asc` with
+   `get_url`, and write the repo with `signed-by=`. Use the Debian key URL
+   (the task currently fetches the Ubuntu one; it's the same key).
+2. **Stop hard-coding `bookworm`.** Use `{{ ansible_distribution_release }}`
+   in the Docker repo line, so the role follows whatever release is
+   installed. `apt_repository` writes a new file, so remove the old
+   `download_docker_com_linux_debian.list` in the same change, to avoid
+   having the repo listed twice.
+3. Optional: manage the Tailscale repo the same way, so all third-party
+   repos come from Ansible.
+4. Deploy with `--tags` for the docker role's base tasks, then check that
+   `apt update` is clean and nothing changed on bookworm.
+
+## Step 2: backups (the day of the upgrade)
+
+1. **Take a netcup snapshot of the whole server** from the netcup control
+   panel. This is the rollback plan. Check that the snapshot has finished
+   before going on.
+2. Trigger the existing S3 backups by hand (Paperless, GoToSocial), as a
+   second copy of the most important data.
+3. Note the running containers (`docker ps`) and the dockcheck output, to
+   compare against afterwards.
+
+## Step 3: the upgrade (allow about an hour of downtime)
+
+Run in `tmux` over Tailscale SSH, so a dropped connection doesn't kill
+apt. Keep the netcup VNC console open as a fallback.
+
+1. Bring bookworm fully up to date: `apt update && apt full-upgrade`,
+   reboot if the kernel changed, then `apt autoremove --purge`.
+2. Stop the containers, so databases aren't written to while their
+   libraries and Docker are replaced: `systemctl stop docker docker.socket`.
+3. Change `bookworm` to `trixie` in `/etc/apt/sources.list` (including
+   `trixie-security` and `trixie-updates`) and in the Tailscale list. The
+   Docker list comes from step 1.
+4. `apt update`, then the minimal upgrade first:
+   `apt upgrade --without-new-pkgs`, then `apt full-upgrade`.
+   - Keep our versions of config files Ansible manages (sshd drop-in,
+     `20auto-upgrades`). For anything else, look at the diff before
+     choosing.
+5. Reboot into kernel 6.12.
+
+## Step 4: checks after the reboot
+
+- `cat /etc/debian_version` shows 13.x; `uname -r` shows 6.12.
+- SSH key login works over both the public IP and Tailscale; passwords
+  are still refused (`sshd -T | grep -i password`).
+- `tailscale status` is up; `ufw status` still has the four Docker
+  forward rules (Caddy 80/443, Gitea SSH).
+- Docker is running and every container came back (`docker ps` against the
+  list from step 2). Check the Postgres and MariaDB/Mongo containers' logs.
+- Public sites load through Caddy; ports 22, 80, 443 and 222 are the only
+  ones open from outside.
+- `/tmp`: trixie can move `/tmp` into RAM (tmpfs) and clean `/tmp` and
+  `/var/tmp` on a timer. Check `findmnt /tmp` and make sure nothing here
+  keeps files there long-term.
+- `unattended-upgrades` picks up `trixie-security` (dry run:
+  `unattended-upgrade --dry-run -d`).
+- Run `bin/deploy --check` against the whole site to confirm Ansible
+  still works with the new Python (3.13) and reports no unexpected changes.
+- Then `apt autoremove --purge`, and purge the `rc` kernel packages.
+
+## Rollback
+
+If the server doesn't come back or something important is broken and
+can't be fixed quickly, restore the netcup snapshot from step 2. Once
+things are stable, delete the snapshot (netcup snapshots slow down the
+disk over time).
+
+## Open questions
+
+- Does the netcup plan include snapshots, and how big a snapshot can it
+  take? (The server uses 246 GB.)
+- When is a good maintenance window?