Mac: Hermes models (Sol chat, cheap side tasks) and monthly model review

Models, from the Nov 2026 review (Nous Hermes Index, OpenRouter data):
- chat: openai/gpt-6.1-sol (user's pick), reasoning high (was ultra)
- side tasks (compression, approval, titles, mcp, memory query rewrite,
  curator, mail monitor) and unpinned cron: z-ai/glm-5.3-flash
- subagents: deepseek/deepseek-v4.1-flash
- /model aliases: deepseek, haiku
agent.reasoning_effort needs --force (hermes-agent #140221), so settings
can list keys in hermes_settings_force. `config get` exits 1 for unset
keys without a default; that now reads as unset, not a failure.

hermes-model-review: monthly cron job on Sol (pinned, high effort,
continuity) that redoes this research and sends keep/change
recommendations as Ansible keys to Telegram. A script injects the live
model setup and 30-day usage into its prompt. hermes_cron_jobs gains
model/provider, reasoning_effort and continuity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
PhilandClaude Opus 5.5 committed 2026-11-11 08:13:00 -07:00
1 parent 6632729892
commit a0e49f4670
5 files changed
+103 -11

No files matched your search

+68 -2
View File
@@ -51,15 +51,81 @@ hermes_settings:
streaming.enabled: true # replies stream into Telegram as they're written streaming.enabled: true # replies stream into Telegram as they're written
stt.local.model: small # voice notes: more accurate than base stt.local.model: small # voice notes: more accurate than base
# Models (OpenRouter), from the Nov 2026 review of Nous's Hermes Index
# (portal.nousresearch.com/bench): chat on GPT-6.1 Sol (best value of the
# premium models there), background work on cheap models.
model.default: openai/gpt-6.1-sol
agent.reasoning_effort: high # the level Nous benchmarks at (was ultra)
# /model deepseek or /model haiku in Telegram; add --once for one reply.
model_aliases.deepseek: {model: deepseek/deepseek-v4.1-flash, provider: openrouter}
model_aliases.haiku: {model: anthropic/claude-haiku-5.5, provider: openrouter}
# Side tasks would otherwise run on the chat model ("auto"). GLM-5.3 Flash
# is the cheapest model with credible Hermes scores; its slow first token
# doesn't matter here. Vision, voice and /review stay on the chat model.
auxiliary.compression.provider: openrouter
auxiliary.compression.model: z-ai/glm-5.3-flash
auxiliary.approval.provider: openrouter
auxiliary.approval.model: z-ai/glm-5.3-flash
auxiliary.title_generation.provider: openrouter
auxiliary.title_generation.model: z-ai/glm-5.3-flash
auxiliary.mcp.provider: openrouter
auxiliary.mcp.model: z-ai/glm-5.3-flash
auxiliary.memory_query_rewrite.provider: openrouter
auxiliary.memory_query_rewrite.model: z-ai/glm-5.3-flash
auxiliary.curator.provider: openrouter
auxiliary.curator.model: z-ai/glm-5.3-flash
auxiliary.monitor.provider: openrouter
auxiliary.monitor.model: z-ai/glm-5.3-flash
# Unpinned cron jobs. Per-job reasoning effort: set --reasoning-effort low
# when creating agent jobs (there's no global cron setting for it).
cron.model_provider: openrouter
cron.model: z-ai/glm-5.3-flash
# Subagents: DeepSeek V4.1 Flash does shell work better than the other
# cheap models (TerminalBench 17.5 vs GPT-6 Luna 6.3).
delegation.provider: openrouter
delegation.model: deepseek/deepseek-v4.1-flash
# Keys `hermes config set` refuses without --force although Hermes reads them
# (agent.reasoning_effort: hermes-agent #140221).
hermes_settings_force: [agent.reasoning_effort]
# Podman VM that runs Hermes's sandbox containers. # Podman VM that runs Hermes's sandbox containers.
podman_machine_memory: 4096 # MB podman_machine_memory: 4096 # MB
# Hermes cron jobs. `script` is a file in roles/mac/files/hermes-scripts/, # Hermes cron jobs. `script` is a file in roles/mac/files/hermes-scripts/,
# copied to ~/.hermes/scripts/. Jobs are created when missing; to change one, # copied to ~/.hermes/scripts/ (with an agent job its output is injected into
# edit or remove it in Hermes (`hermes cron edit|remove <name>`) and redeploy. # the prompt). Optional: model + provider (pin), reasoning_effort, continuity.
# Jobs are created when missing; to change one, remove it in Hermes
# (`hermes cron remove <name>`) and redeploy.
hermes_cron_jobs: hermes_cron_jobs:
- name: hermes-update-notice - name: hermes-update-notice
schedule: "7 9 * * 1" # Mondays 09:07 schedule: "7 9 * * 1" # Mondays 09:07
script: hermes-update-notice.sh script: hermes-update-notice.sh
no_agent: true # just the script's output, no model no_agent: true # just the script's output, no model
deliver: telegram deliver: telegram
- name: hermes-model-review
schedule: "7 9 1 * *" # 1st of the month, 09:07
script: hermes-model-context.sh # current setup + usage, injected into the prompt
model: openai/gpt-6.1-sol # a report you act on: strong model, pinned
provider: openrouter
reasoning_effort: high
continuity: true # sees last month's report, says what changed
deliver: telegram
prompt: >-
Monthly review of which AI models this Hermes install should use. The
current setup and last month's usage are above. Research what changed
since last month, using primary sources where possible:
(1) Nous Research's Hermes Index, https://portal.nousresearch.com/bench
(agent tasks run inside Hermes; note it is run by Nous, who also sell
model access); (2) OpenRouter's Hermes Agent app rankings,
https://openrouter.ai/apps/hermes-agent, remembering that token volume
mostly reflects cheap automated workloads, not chat quality;
(3) current prices from https://openrouter.ai/api/v1/models; (4) notable
model releases and Hermes release notes from the past month that affect
model choice. Then, for each slot (chat model and reasoning effort,
side tasks, cron, subagents, /model aliases), say keep or change, with
the evidence, a link, and the rough monthly cost difference at the
current usage. Only recommend a change when the evidence is clearly
better, not just newer. Don't change any settings: they are managed with
Ansible, so end with the exact keys and values to change, if any. Keep
it under 300 words. If nothing should change, say so in two sentences.
+9 -4
View File
@@ -27,9 +27,11 @@ Manages cogitator-02, an Apple Silicon Mac on the tailnet (`mac.yml`,
`hermes config get` and fixed with `hermes config set`; any change restarts the gateway (`hermes gateway restart`) `hermes config get` and fixed with `hermes config set`; any change restarts the gateway (`hermes gateway restart`)
and the dashboard. and the dashboard.
Covers update backups, timezone, not borrowing Claude Code's login, sandbox Covers update backups, timezone, not borrowing Claude Code's login, sandbox
memory, disabled toolsets, skill-write approval, deny rules, streaming and memory, disabled toolsets, skill-write approval, deny rules, streaming,
voice-note transcription. Leave `agent.reasoning_effort` out: `config set` voice-note transcription, and models: chat on GPT-6.1 Sol, side tasks and
refuses it without `--force` (hermes-agent #140221). cron on GLM-5.3 Flash, subagents on DeepSeek V4.1 Flash, `/model` aliases.
Keys in `hermes_settings_force` are set with `--force` (Hermes reads
`agent.reasoning_effort` but `config set` refuses it, hermes-agent #140221).
- **Podman VM** (tag `podman`): a launchd job (`local.podman-machine`) that - **Podman VM** (tag `podman`): a launchd job (`local.podman-machine`) that
starts the VM running Hermes's sandbox at login, and `podman_machine_memory` starts the VM running Hermes's sandbox at login, and `podman_machine_memory`
for its size. A resize stops the VM and starts it again through that job. for its size. A resize stops the VM and starts it again through that job.
@@ -39,7 +41,10 @@ Manages cogitator-02, an Apple Silicon Mac on the tailnet (`mac.yml`,
`group_vars/mac/main.yml`, with their scripts from `files/hermes-scripts/` `group_vars/mac/main.yml`, with their scripts from `files/hermes-scripts/`
copied to `~/.hermes/scripts/`. Missing jobs are created; existing ones are copied to `~/.hermes/scripts/`. Missing jobs are created; existing ones are
left as they are. Today: `hermes-update-notice`, which messages you on left as they are. Today: `hermes-update-notice`, which messages you on
Telegram on Mondays only when a new stable Hermes release is out. Telegram on Mondays only when a new stable Hermes release is out, and
`hermes-model-review`, a monthly GPT-6.1 Sol research run that recommends
model changes (as Ansible keys) based on the Hermes Index, OpenRouter
rankings and prices, and your usage.
- **Automatic login** (tag `autologin`): phil is logged in automatically after - **Automatic login** (tag `autologin`): phil is logged in automatically after
a reboot, so everything that needs a login session (dashboard, Podman VM, a reboot, so everything that needs a login session (dashboard, Podman VM,
BlueBubbles, Beeper) comes back after a power cut. Uses phil's password from BlueBubbles, Beeper) comes back after a power cut. Uses phil's password from
@@ -0,0 +1,15 @@
#!/bin/bash
# Injected into the monthly model review's prompt: the live model setup and
# last month's usage, which the agent can't read from inside its sandbox.
h="$HOME/.local/bin/hermes"
echo "## Current Hermes model setup (cogitator-02)"
for k in model.provider model.default agent.reasoning_effort model_aliases \
auxiliary.compression.model auxiliary.approval.model \
auxiliary.title_generation.model auxiliary.mcp.model \
auxiliary.memory_query_rewrite.model auxiliary.curator.model \
auxiliary.monitor.model cron.model delegation.model; do
printf -- '- %s: %s\n' "$k" "$("$h" config get "$k" 2>/dev/null | tr '\n' ' ')"
done
echo
echo "## Usage over the last 30 days (hermes insights)"
"$h" insights --days 30 2>/dev/null | sed -n '/Overview/,/Activity Patterns/p' | grep -v -E '^[[:space:]]*$|─'
+3
View File
@@ -20,6 +20,9 @@
{{ [hermes_cli, 'cron', 'create', item.schedule, '--name', item.name, '--deliver', item.deliver] {{ [hermes_cli, 'cron', 'create', item.schedule, '--name', item.name, '--deliver', item.deliver]
+ (['--script', item.script] if item.script is defined else []) + (['--script', item.script] if item.script is defined else [])
+ (['--no-agent'] if item.no_agent | default(false) else []) + (['--no-agent'] if item.no_agent | default(false) else [])
+ (['--model', item.model, '--provider', item.provider] if item.model is defined else [])
+ (['--reasoning-effort', item.reasoning_effort] if item.reasoning_effort is defined else [])
+ (['--continuity'] if item.continuity | default(false) else [])
+ ([item.prompt] if item.prompt is defined else []) }} + ([item.prompt] if item.prompt is defined else []) }}
loop: "{{ hermes_cron_jobs }}" loop: "{{ hermes_cron_jobs }}"
loop_control: loop_control:
+8 -5
View File
@@ -1,24 +1,27 @@
# `hermes config get` prints the value as YAML, so it's compared # `hermes config get` prints the value as YAML, so it's compared
# structurally; `hermes config set` takes JSON for lists and booleans. # structurally; `hermes config set` takes JSON for lists and booleans.
# Settings are always set explicitly: for a key that isn't in config.yaml, # Settings are always set explicitly: for a key that isn't in config.yaml,
# `config get` (non-interactive) prints just the default, so "unset" can't be # `config get` (non-interactive) prints just the default when there is one,
# detected. # so "unset" can't be detected.
- name: Read Hermes settings - name: Read Hermes settings
ansible.builtin.command: ansible.builtin.command:
argv: ["{{ hermes_cli }}", config, get, "{{ item.key }}"] argv: ["{{ hermes_cli }}", config, get, "{{ item.key }}"]
loop: "{{ hermes_settings | dict2items }}" loop: "{{ hermes_settings | dict2items }}"
register: hermes_settings_current register: hermes_settings_current
changed_when: false changed_when: false
# Exit 1 = "Config key not set" (only for keys without a built-in default).
failed_when: hermes_settings_current.rc > 1
check_mode: false check_mode: false
- name: Set Hermes settings - name: Set Hermes settings
ansible.builtin.command: ansible.builtin.command:
argv: >- argv: >-
{{ [hermes_cli, 'config', 'set', item.item.key, {{ [hermes_cli, 'config', 'set']
item.item.value if item.item.value is string else item.item.value | to_json] }} + (['--force'] if item.item.key in hermes_settings_force | default([]) else [])
+ [item.item.key, item.item.value if item.item.value is string else item.item.value | to_json] }}
loop: "{{ hermes_settings_current.results }}" loop: "{{ hermes_settings_current.results }}"
loop_control: loop_control:
label: "{{ item.item.key }} = {{ item.item.value | to_json }}" label: "{{ item.item.key }} = {{ item.item.value | to_json }}"
when: (item.stdout | from_yaml) != item.item.value when: (none if item.rc == 1 else item.stdout | from_yaml) != item.item.value
# The dashboard is a separate process with its own copy of the settings. # The dashboard is a separate process with its own copy of the settings.
notify: [Restart Hermes gateway, Restart Hermes dashboard] notify: [Restart Hermes gateway, Restart Hermes dashboard]