Adds a small hermes_settings map that Ansible checks with
`hermes config get` and fixes with `hermes config set`, leaving the rest
of config.yaml to Hermes (its updates migrate that file).
First setting: updates.pre_update_backup: full (was false), so every
`hermes update` zips ~/.hermes first. Rollback isn't supported once a new
version has written state.db, so the backup is the way back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dashboard used to be started by hand as `serve --host 0.0.0.0`, so it
was reachable from the LAN and didn't survive a reboot or a Hermes update.
Now, following Hermes's docs:
- launchd agent local.hermes-dashboard runs `hermes dashboard` on
127.0.0.1:9119 (KeepAlive, --skip-build since `hermes update` builds it).
- Tailscale Serve publishes it on https://cogitator-02.porgy-porgy.ts.net,
tailnet only. No module exists for Serve; the task reads
`serve status --json` and only sets it when it differs.
- Login: scrypt password hash and a real signing secret from the vault
(the old secret was the literal text "$(openssl rand -base64 32)"); the
plaintext password line is removed from .env since it overrides the hash.
- After Serve is set, the play checks /api/status demands the basic login;
if not, it turns Serve off and fails.
- Hermes's Python no longer needs a firewall exception, so that's gone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- pmset: never sleep, no disk sleep, restart after a power cut, wake on
network. The Mac had sleep 1 and only stayed up because BlueBubbles runs
caffeinate.
- Application firewall on, keeping signed software allowed automatically,
stealth and block-all off. Hermes's ad hoc signed Python (dashboard on
9119) is allowed explicitly, since an unattended Mac blocks unsigned
programs silently.
- Local id_ed25519.pub in phil's authorized_keys (already there).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cogitator-02 is an Apple Silicon Mac on the tailnet running Hermes Agent.
It gets a `mac` group and mac.yml, kept apart from site.yml so it never
runs with the server's deploys (PLAYBOOK=mac.yml bin/deploy).
The mac role starts by recording the Homebrew taps, formulae and casks
already installed (present only, never upgrades or removes), so the first
run changes nothing.
No passwordless sudo on the Mac: Hermes runs as phil and takes commands
over Telegram. Ansible's sudo password is vault_cogitator02.become_password
and only tasks with become: true use it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ckulka/baikal is stuck on Baikal 0.10.1 (PHP 8.1). dschoepel/baikal is a
drop-in successor of its nginx variant (same volumes, UID 101) with
Baikal 0.13.0 on PHP 8.4. Upgrade tested on a copy of the data.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compose deploys used the default pull policy (missing), so switching a
service to a tag that was already cached deployed a year-old image
(paperless-ngx and heyform :latest). module_defaults now sets pull: always
for every docker_compose_v2 task.
dockcheck pulls the image it updates, but then ran compose up for the
whole stack, which could recreate other containers in the stack from
cached images, including ones it excludes. OnlySpecific=true limits it to
the updated container.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 1.7 -> 1.22 upgrade is done; the app gets the dockcheck label.
Redis and Postgres stay excluded in dockcheck.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1.17.0 background jobs drained cleanly. 1.18.1 (skipping 1.18.0, which
needed the hotfix) converts Areas to Places and re-detects visits across
the whole history; the changes can't be rolled back, so a fresh dump was
taken first (~/backups/dawarich-1.17.0-20261109). Redis goes to 8 as in
upstream's compose since 1.18.0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follows the 1.17.0 upgrade notes and upstream compose: the separate
dawarich_sidekiq service is gone (removed on the host before deploying),
the app runs Sidekiq embedded (SIDEKIQ_EMBEDDED=true, WEB_CONCURRENCY=0,
BACKGROUND_PROCESSING_CONCURRENCY=2), gets stop_grace_period 60s, and the
limits match upstream (1 CPU, 4G).
Fresh dump before this step: ~/backups/dawarich-1.16.0-20261109.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4.4 is end-of-life. Moved by mongodump/mongorestore into a new volume
(heyform_database7); the 4.4 volume stays for rollback.
The restore also fixes a data mix-up. From Nov 2024 the compose file set
MONGO_URI='mongodb://mongo:27017/heyform' in list form, so the quotes
became part of the value and HeyForm used a database literally named
"heyform'". The v3 upgrade (a1a79af) dropped the quotes, which switched
HeyForm to the old "heyform" database from the first week of setup, so
the real Repairs and Volunteer forms (91 submissions) disappeared. The new
volume holds "heyform'" restored as "heyform". Tested first with v3 on a
throwaway Mongo 7.0: both forms load.
Backups: ~/backups/heyform-mongo4.4-20261109 on the server (dump of both
databases + tar of the 4.4 volume).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gitea/gitea:1 stopped moving in August when Gitea renumbered to 28.0, and
1.27 gets no more fixes (28.0 fixed ~20 CVEs). Track :latest so the next
renumber can't freeze it again. The act_runner image was renamed to
gitea/runner; the old one stopped updating in April. The .runner
registration file and config format carry over unchanged.
v28 starts deleting Actions runs older than 400 days; set
RUN_RETENTION_DAYS=0 to keep the history.
Checked for this instance: ROOT_URL/SSH_DOMAIN set, no ALLOWED_HOST_LIST,
no webhooks or custom templates, one pull mirror from a public host.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both earlier failures had one cause: the host still had old images cached
under the :latest tag (paperless-ngx from 2025-09, i.e. 2.x; heyform from
2025-02, i.e. v2), and the compose task only pulls missing images. So the
switch to :latest started year-old code. Pulled the real :latest images on
the host. Paperless 3.4.2 has no pending migrations on this database
(checked on a copy), and HeyForm v3.0.6 listens on 9157 as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- HeyForm v3.0.6 changed the default listen port from 9157 to 8000, so
Caddy got 502s. Set APP_LISTEN_PORT=9157 explicitly.
- Paperless 3.4.2 fails its migration check on this database
(InconsistentMigrationHistory, squashed 1022..1036). Rolled back to 3.4.1
until that is investigated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
First stop is 1.10.3 (drops the legacy lat/lon columns with the 1.10.2
lock fix). Next stops: 1.14.5 (security fix), 1.16.0, 1.17.0 (sidekiq
merged into the app), 1.18.1, 1.22.0, then back to latest.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pinned versions never got updates, because dockcheck only notices a new
image under the same tag.
- paperless-ngx: 3.4.1 -> latest (stable only; beta/dev have own tags)
- paperless redis: 7 -> 8-alpine (reads Redis 7 data as-is)
- gotenberg: 8.39.0 -> 8 (9 changed the API Paperless calls)
- tika: 3.3.1.0 -> 3.3.3.0 (no floating 3.x tag; Tika 4 breaks Paperless)
- heyform: v3.0.5 -> latest (only moved by v* release tags)
- add mag37.dockcheck.update to all of the above
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dockcheck runs as phil (docker group) and couldn't read
/opt/stacks/gokapi (0700 root), so the labelled gokapi container could
never be auto-updated. Paperless's root-only .env (2024 leftover, unused,
same values as docker-compose.env) made docker compose fail for phil.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Install dockcheck as a git clone in ~/.local/share/dockcheck with
AutoSelfUpdate (git pull also updates notify templates and urls.list);
no pinned versions. regctl is refreshed weekly by run_dockcheck.sh.
- Since dockcheck ~v0.7, OnlyLabel skips unlabelled containers entirely, so
they got no update notices. run_dockcheck.sh now runs twice: update
labelled containers quietly, then check everything and notify once.
- Running dockcheck.sh by hand now checks everything and changes nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Pin dockcheck (v0.8.3) and regctl (v0.11.7) and re-download on version
bumps; get_url never replaced existing files, and dockcheck never updates
regctl itself (server had regctl v0.8.3 from June 2025)
- New options: DaysOld=3, BackupForDays=7 (rollback), MaxAsync=10,
MonoMode for the log, PrintReleaseURL with upstream urls.list
- Drop authentik-postgresql-1 from Exclude (container is gone)
- Wrapper logged exit code 0 always ($? read after $(date)); fixed
- Rotate /var/log/dockcheck/dockcheck.log weekly
- Remove stale ~/.local/bin/notify_v2.sh
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pass the Threads token (vault_starpunk.threads_access_token, Meta app
"starpunk") and username philskents. StarPunk refreshes the token daily
and keeps the current one in /data/threads_token.json.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The server now runs Debian 13.7. systemd 257's ssh generator logs a
harmless AF_VSOCK error on netcup VMs, so the common role masks it.
The plan file now records how the upgrade went and what's left.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Debian 13 drops apt-key and software-properties-common, so the old tasks
would fail after the upgrade. The key now lives in /etc/apt/keyrings
(same fingerprint, 9DC8...CD88), the repo is a deb822 file whose release
name follows the installed Debian, and the old .list file is removed.
The apt tasks get a docker_apt tag so they can be deployed on their own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Browsers will use HTTPS for each site for a year. includeSubDomains is
left off so hosts not served by this Caddy are unaffected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v0.1.0 (Feb 2025) has dozens of published security advisories fixed in
v3. Tested against a copy of the database first; forms load unchanged.
v3 listens on 9157, keeps uploads under packages/server, and uses Redis 8
commands, so KeyDB is replaced by redis:8-alpine. The homepage URL is now
https (v3's CORS allowlist is built from it), Caddy is the trusted proxy,
public sign-up is off, and integration credentials get their own key.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StarPunk 2.1.0 copies new notes to GoToSocial and keeps edits and
deletes in step. Token for the app "StarPunk crossposting" (scopes
write:statuses write:media) is in vault_starpunk.gotosocial_token; the
container reaches social. through Caddy (hairpin).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v3 can only be reached from 2.20.15, so stop tracking :latest. Also set
the two settings v3 needs from us: trust Caddy as the proxy (v3's login
rate limiting), and keep rejecting duplicate documents.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StarPunk's Gitea Actions workflow now builds and pushes the image on
vX.Y.Z tags. Pin the tag in starpunk_version (v2.0.2, the first CI build)
and always pull it, so an image built by hand on the server with the
same name can't be deployed instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Karakeep's Chrome was alpine-chrome 123 (April 2024) and it opens every
bookmarked URL; it now uses Karakeep's own Chrome image like upstream.
Meilisearch goes from 1.13.3 to 1.41.0 with a dumpless in-place upgrade.
Gotenberg for Paperless goes from 8.7 to 8.39.0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sshd accepted passwords from the internet (about 12,000 failed logins a
day) and root logins with a key. Keys only now, no root, no X11 forwarding.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jobs run inside the runner container (label docker:host), and with the
host's Docker socket mounted every workflow had root on the server. The
socket was only there for StarPunk's image build, which has never worked
(it logged in to gitea:3000, which the host daemon can't resolve).
Image builds now go to a rootless BuildKit container that shares only a
unix socket with the runner. Site builds keep their /sites mount.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StarPunk 2.0.0 logs in with Pocket ID and issues its own Micropub tokens
(ADR-064 in the StarPunk repo), so the Gondulf IndieAuth server is no
longer used. Removes its task, templates, Caddy route, subdomain and
service entry, and moves its DNS record to removed_records.
The StarPunk template was still on v1.6.0-rc.2 with Gondulf hairpin
hosts; deploying it would have broken login. It now matches production:
v2.0.0, Pocket ID hairpin, OIDC client from vault_starpunk.pocketid and
the admin sub in starpunk_admin_sub.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Karakeep had DISABLE_SIGNUPS unset, so anyone could make an account on
bookmarks. and use the OpenAI key behind it. The server also had no
automatic updates and three security updates waiting; the common role now
installs unattended-upgrades with Debian's default security-only origins.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The READMEs, CLAUDE.md and todo.md still described Authentik, services
removed long ago (Ghost, MMDL, Postiz, Pinchflat...) and two servers. Now
they list the 24 services actually deployed (checked against docker ps),
explain that netcup and docker-01 are one machine, document Gondulf,
StarPunk, Listmonk, Gokapi and dockcheck, and use bin/deploy throughout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--deployment-password sets the password and exits, so the detached run
plus health wait never saw a healthy container.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 'Gokapi started' log line is not always flushed to log.txt, so the
first-run wait timed out even though the super admin had been created.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All of Authentik's applications (including Tailscale's tailnet login)
have moved to Pocket ID, and its only recent sign-ins were those
handovers. Drops its stack, the auth. route and DNS record, the auth
subdomain variables, Gotify's leftover hairpin entries, and the dockcheck
exclude; the docs now name Pocket ID as the SSO provider.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gokapi now keeps its config and data in /opt/stacks/gokapi instead of named
volumes. config.json is templated with Pocket ID as the OIDC provider, so a
fresh install never shows the open /setup wizard. On a new database a one-off
run with --deployment-password creates the super admin, after which only
existing users can log in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Audiobookshelf's OIDC settings (stored in its own database) now use the
Pocket ID client in vault_audiobookshelf.pocketid instead of Authentik, so
the container needs the Pocket ID hairpin entry. The podcasts library was
deleted and its files purged, so its volume goes too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both were hand-deployed to /opt/stacks. Templates match the live setup
and keep the same stack dirs and volume names, so data stays put.
Secrets come from the vault; .env and the StarPunk compose file are now
0640 root:docker instead of the broken 1204 mode. Adds their DNS
records to dns.yml and drops the dead godulf. Caddy route (no DNS).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kavita's KOReader sync API and OPDS feed replace them, and both readers
(Kindle KOReader, Xteink X4) now point at books.thesatelliteoflove.com.
Drops their stacks, the sync. and opds. routes and DNS records, and the
htpasswd tasks that only existed for dir2opds basic auth.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>