Mac: Hermes models (Sol chat, cheap side tasks) and monthly model review
Models, from the Nov 2026 review (Nous Hermes Index, OpenRouter data): - chat: openai/gpt-6.1-sol (user's pick), reasoning high (was ultra) - side tasks (compression, approval, titles, mcp, memory query rewrite, curator, mail monitor) and unpinned cron: z-ai/glm-5.3-flash - subagents: deepseek/deepseek-v4.1-flash - /model aliases: deepseek, haiku agent.reasoning_effort needs --force (hermes-agent #140221), so settings can list keys in hermes_settings_force. `config get` exits 1 for unset keys without a default; that now reads as unset, not a failure. hermes-model-review: monthly cron job on Sol (pinned, high effort, continuity) that redoes this research and sends keep/change recommendations as Ansible keys to Telegram. A script injects the live model setup and 30-day usage into its prompt. hermes_cron_jobs gains model/provider, reasoning_effort and continuity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
6632729892
commit
a0e49f4670
5 files changed
+103
-11
No files matched your search
+68
-2
@@ -51,15 +51,81 @@ hermes_settings:
|
|||||||
streaming.enabled: true # replies stream into Telegram as they're written
|
streaming.enabled: true # replies stream into Telegram as they're written
|
||||||
stt.local.model: small # voice notes: more accurate than base
|
stt.local.model: small # voice notes: more accurate than base
|
||||||
|
|
||||||
|
# Models (OpenRouter), from the Nov 2026 review of Nous's Hermes Index
|
||||||
|
# (portal.nousresearch.com/bench): chat on GPT-6.1 Sol (best value of the
|
||||||
|
# premium models there), background work on cheap models.
|
||||||
|
model.default: openai/gpt-6.1-sol
|
||||||
|
agent.reasoning_effort: high # the level Nous benchmarks at (was ultra)
|
||||||
|
# /model deepseek or /model haiku in Telegram; add --once for one reply.
|
||||||
|
model_aliases.deepseek: {model: deepseek/deepseek-v4.1-flash, provider: openrouter}
|
||||||
|
model_aliases.haiku: {model: anthropic/claude-haiku-5.5, provider: openrouter}
|
||||||
|
# Side tasks would otherwise run on the chat model ("auto"). GLM-5.3 Flash
|
||||||
|
# is the cheapest model with credible Hermes scores; its slow first token
|
||||||
|
# doesn't matter here. Vision, voice and /review stay on the chat model.
|
||||||
|
auxiliary.compression.provider: openrouter
|
||||||
|
auxiliary.compression.model: z-ai/glm-5.3-flash
|
||||||
|
auxiliary.approval.provider: openrouter
|
||||||
|
auxiliary.approval.model: z-ai/glm-5.3-flash
|
||||||
|
auxiliary.title_generation.provider: openrouter
|
||||||
|
auxiliary.title_generation.model: z-ai/glm-5.3-flash
|
||||||
|
auxiliary.mcp.provider: openrouter
|
||||||
|
auxiliary.mcp.model: z-ai/glm-5.3-flash
|
||||||
|
auxiliary.memory_query_rewrite.provider: openrouter
|
||||||
|
auxiliary.memory_query_rewrite.model: z-ai/glm-5.3-flash
|
||||||
|
auxiliary.curator.provider: openrouter
|
||||||
|
auxiliary.curator.model: z-ai/glm-5.3-flash
|
||||||
|
auxiliary.monitor.provider: openrouter
|
||||||
|
auxiliary.monitor.model: z-ai/glm-5.3-flash
|
||||||
|
# Unpinned cron jobs. Per-job reasoning effort: set --reasoning-effort low
|
||||||
|
# when creating agent jobs (there's no global cron setting for it).
|
||||||
|
cron.model_provider: openrouter
|
||||||
|
cron.model: z-ai/glm-5.3-flash
|
||||||
|
# Subagents: DeepSeek V4.1 Flash does shell work better than the other
|
||||||
|
# cheap models (TerminalBench 17.5 vs GPT-6 Luna 6.3).
|
||||||
|
delegation.provider: openrouter
|
||||||
|
delegation.model: deepseek/deepseek-v4.1-flash
|
||||||
|
|
||||||
|
# Keys `hermes config set` refuses without --force although Hermes reads them
|
||||||
|
# (agent.reasoning_effort: hermes-agent #140221).
|
||||||
|
hermes_settings_force: [agent.reasoning_effort]
|
||||||
|
|
||||||
# Podman VM that runs Hermes's sandbox containers.
|
# Podman VM that runs Hermes's sandbox containers.
|
||||||
podman_machine_memory: 4096 # MB
|
podman_machine_memory: 4096 # MB
|
||||||
|
|
||||||
# Hermes cron jobs. `script` is a file in roles/mac/files/hermes-scripts/,
|
# Hermes cron jobs. `script` is a file in roles/mac/files/hermes-scripts/,
|
||||||
# copied to ~/.hermes/scripts/. Jobs are created when missing; to change one,
|
# copied to ~/.hermes/scripts/ (with an agent job its output is injected into
|
||||||
# edit or remove it in Hermes (`hermes cron edit|remove <name>`) and redeploy.
|
# the prompt). Optional: model + provider (pin), reasoning_effort, continuity.
|
||||||
|
# Jobs are created when missing; to change one, remove it in Hermes
|
||||||
|
# (`hermes cron remove <name>`) and redeploy.
|
||||||
hermes_cron_jobs:
|
hermes_cron_jobs:
|
||||||
- name: hermes-update-notice
|
- name: hermes-update-notice
|
||||||
schedule: "7 9 * * 1" # Mondays 09:07
|
schedule: "7 9 * * 1" # Mondays 09:07
|
||||||
script: hermes-update-notice.sh
|
script: hermes-update-notice.sh
|
||||||
no_agent: true # just the script's output, no model
|
no_agent: true # just the script's output, no model
|
||||||
deliver: telegram
|
deliver: telegram
|
||||||
|
- name: hermes-model-review
|
||||||
|
schedule: "7 9 1 * *" # 1st of the month, 09:07
|
||||||
|
script: hermes-model-context.sh # current setup + usage, injected into the prompt
|
||||||
|
model: openai/gpt-6.1-sol # a report you act on: strong model, pinned
|
||||||
|
provider: openrouter
|
||||||
|
reasoning_effort: high
|
||||||
|
continuity: true # sees last month's report, says what changed
|
||||||
|
deliver: telegram
|
||||||
|
prompt: >-
|
||||||
|
Monthly review of which AI models this Hermes install should use. The
|
||||||
|
current setup and last month's usage are above. Research what changed
|
||||||
|
since last month, using primary sources where possible:
|
||||||
|
(1) Nous Research's Hermes Index, https://portal.nousresearch.com/bench
|
||||||
|
(agent tasks run inside Hermes; note it is run by Nous, who also sell
|
||||||
|
model access); (2) OpenRouter's Hermes Agent app rankings,
|
||||||
|
https://openrouter.ai/apps/hermes-agent, remembering that token volume
|
||||||
|
mostly reflects cheap automated workloads, not chat quality;
|
||||||
|
(3) current prices from https://openrouter.ai/api/v1/models; (4) notable
|
||||||
|
model releases and Hermes release notes from the past month that affect
|
||||||
|
model choice. Then, for each slot (chat model and reasoning effort,
|
||||||
|
side tasks, cron, subagents, /model aliases), say keep or change, with
|
||||||
|
the evidence, a link, and the rough monthly cost difference at the
|
||||||
|
current usage. Only recommend a change when the evidence is clearly
|
||||||
|
better, not just newer. Don't change any settings: they are managed with
|
||||||
|
Ansible, so end with the exact keys and values to change, if any. Keep
|
||||||
|
it under 300 words. If nothing should change, say so in two sentences.
|
||||||
+9
-4
@@ -27,9 +27,11 @@ Manages cogitator-02, an Apple Silicon Mac on the tailnet (`mac.yml`,
|
|||||||
`hermes config get` and fixed with `hermes config set`; any change restarts the gateway (`hermes gateway restart`)
|
`hermes config get` and fixed with `hermes config set`; any change restarts the gateway (`hermes gateway restart`)
|
||||||
and the dashboard.
|
and the dashboard.
|
||||||
Covers update backups, timezone, not borrowing Claude Code's login, sandbox
|
Covers update backups, timezone, not borrowing Claude Code's login, sandbox
|
||||||
memory, disabled toolsets, skill-write approval, deny rules, streaming and
|
memory, disabled toolsets, skill-write approval, deny rules, streaming,
|
||||||
voice-note transcription. Leave `agent.reasoning_effort` out: `config set`
|
voice-note transcription, and models: chat on GPT-6.1 Sol, side tasks and
|
||||||
refuses it without `--force` (hermes-agent #140221).
|
cron on GLM-5.3 Flash, subagents on DeepSeek V4.1 Flash, `/model` aliases.
|
||||||
|
Keys in `hermes_settings_force` are set with `--force` (Hermes reads
|
||||||
|
`agent.reasoning_effort` but `config set` refuses it, hermes-agent #140221).
|
||||||
- **Podman VM** (tag `podman`): a launchd job (`local.podman-machine`) that
|
- **Podman VM** (tag `podman`): a launchd job (`local.podman-machine`) that
|
||||||
starts the VM running Hermes's sandbox at login, and `podman_machine_memory`
|
starts the VM running Hermes's sandbox at login, and `podman_machine_memory`
|
||||||
for its size. A resize stops the VM and starts it again through that job.
|
for its size. A resize stops the VM and starts it again through that job.
|
||||||
@@ -39,7 +41,10 @@ Manages cogitator-02, an Apple Silicon Mac on the tailnet (`mac.yml`,
|
|||||||
`group_vars/mac/main.yml`, with their scripts from `files/hermes-scripts/`
|
`group_vars/mac/main.yml`, with their scripts from `files/hermes-scripts/`
|
||||||
copied to `~/.hermes/scripts/`. Missing jobs are created; existing ones are
|
copied to `~/.hermes/scripts/`. Missing jobs are created; existing ones are
|
||||||
left as they are. Today: `hermes-update-notice`, which messages you on
|
left as they are. Today: `hermes-update-notice`, which messages you on
|
||||||
Telegram on Mondays only when a new stable Hermes release is out.
|
Telegram on Mondays only when a new stable Hermes release is out, and
|
||||||
|
`hermes-model-review`, a monthly GPT-6.1 Sol research run that recommends
|
||||||
|
model changes (as Ansible keys) based on the Hermes Index, OpenRouter
|
||||||
|
rankings and prices, and your usage.
|
||||||
- **Automatic login** (tag `autologin`): phil is logged in automatically after
|
- **Automatic login** (tag `autologin`): phil is logged in automatically after
|
||||||
a reboot, so everything that needs a login session (dashboard, Podman VM,
|
a reboot, so everything that needs a login session (dashboard, Podman VM,
|
||||||
BlueBubbles, Beeper) comes back after a power cut. Uses phil's password from
|
BlueBubbles, Beeper) comes back after a power cut. Uses phil's password from
|
||||||
|
|||||||
@@ -0,0 +1,15 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Injected into the monthly model review's prompt: the live model setup and
|
||||||
|
# last month's usage, which the agent can't read from inside its sandbox.
|
||||||
|
h="$HOME/.local/bin/hermes"
|
||||||
|
echo "## Current Hermes model setup (cogitator-02)"
|
||||||
|
for k in model.provider model.default agent.reasoning_effort model_aliases \
|
||||||
|
auxiliary.compression.model auxiliary.approval.model \
|
||||||
|
auxiliary.title_generation.model auxiliary.mcp.model \
|
||||||
|
auxiliary.memory_query_rewrite.model auxiliary.curator.model \
|
||||||
|
auxiliary.monitor.model cron.model delegation.model; do
|
||||||
|
printf -- '- %s: %s\n' "$k" "$("$h" config get "$k" 2>/dev/null | tr '\n' ' ')"
|
||||||
|
done
|
||||||
|
echo
|
||||||
|
echo "## Usage over the last 30 days (hermes insights)"
|
||||||
|
"$h" insights --days 30 2>/dev/null | sed -n '/Overview/,/Activity Patterns/p' | grep -v -E '^[[:space:]]*$|─'
|
||||||
@@ -20,6 +20,9 @@
|
|||||||
{{ [hermes_cli, 'cron', 'create', item.schedule, '--name', item.name, '--deliver', item.deliver]
|
{{ [hermes_cli, 'cron', 'create', item.schedule, '--name', item.name, '--deliver', item.deliver]
|
||||||
+ (['--script', item.script] if item.script is defined else [])
|
+ (['--script', item.script] if item.script is defined else [])
|
||||||
+ (['--no-agent'] if item.no_agent | default(false) else [])
|
+ (['--no-agent'] if item.no_agent | default(false) else [])
|
||||||
|
+ (['--model', item.model, '--provider', item.provider] if item.model is defined else [])
|
||||||
|
+ (['--reasoning-effort', item.reasoning_effort] if item.reasoning_effort is defined else [])
|
||||||
|
+ (['--continuity'] if item.continuity | default(false) else [])
|
||||||
+ ([item.prompt] if item.prompt is defined else []) }}
|
+ ([item.prompt] if item.prompt is defined else []) }}
|
||||||
loop: "{{ hermes_cron_jobs }}"
|
loop: "{{ hermes_cron_jobs }}"
|
||||||
loop_control:
|
loop_control:
|
||||||
|
|||||||
@@ -1,24 +1,27 @@
|
|||||||
# `hermes config get` prints the value as YAML, so it's compared
|
# `hermes config get` prints the value as YAML, so it's compared
|
||||||
# structurally; `hermes config set` takes JSON for lists and booleans.
|
# structurally; `hermes config set` takes JSON for lists and booleans.
|
||||||
# Settings are always set explicitly: for a key that isn't in config.yaml,
|
# Settings are always set explicitly: for a key that isn't in config.yaml,
|
||||||
# `config get` (non-interactive) prints just the default, so "unset" can't be
|
# `config get` (non-interactive) prints just the default when there is one,
|
||||||
# detected.
|
# so "unset" can't be detected.
|
||||||
- name: Read Hermes settings
|
- name: Read Hermes settings
|
||||||
ansible.builtin.command:
|
ansible.builtin.command:
|
||||||
argv: ["{{ hermes_cli }}", config, get, "{{ item.key }}"]
|
argv: ["{{ hermes_cli }}", config, get, "{{ item.key }}"]
|
||||||
loop: "{{ hermes_settings | dict2items }}"
|
loop: "{{ hermes_settings | dict2items }}"
|
||||||
register: hermes_settings_current
|
register: hermes_settings_current
|
||||||
changed_when: false
|
changed_when: false
|
||||||
|
# Exit 1 = "Config key not set" (only for keys without a built-in default).
|
||||||
|
failed_when: hermes_settings_current.rc > 1
|
||||||
check_mode: false
|
check_mode: false
|
||||||
|
|
||||||
- name: Set Hermes settings
|
- name: Set Hermes settings
|
||||||
ansible.builtin.command:
|
ansible.builtin.command:
|
||||||
argv: >-
|
argv: >-
|
||||||
{{ [hermes_cli, 'config', 'set', item.item.key,
|
{{ [hermes_cli, 'config', 'set']
|
||||||
item.item.value if item.item.value is string else item.item.value | to_json] }}
|
+ (['--force'] if item.item.key in hermes_settings_force | default([]) else [])
|
||||||
|
+ [item.item.key, item.item.value if item.item.value is string else item.item.value | to_json] }}
|
||||||
loop: "{{ hermes_settings_current.results }}"
|
loop: "{{ hermes_settings_current.results }}"
|
||||||
loop_control:
|
loop_control:
|
||||||
label: "{{ item.item.key }} = {{ item.item.value | to_json }}"
|
label: "{{ item.item.key }} = {{ item.item.value | to_json }}"
|
||||||
when: (item.stdout | from_yaml) != item.item.value
|
when: (none if item.rc == 1 else item.stdout | from_yaml) != item.item.value
|
||||||
# The dashboard is a separate process with its own copy of the settings.
|
# The dashboard is a separate process with its own copy of the settings.
|
||||||
notify: [Restart Hermes gateway, Restart Hermes dashboard]
|
notify: [Restart Hermes gateway, Restart Hermes dashboard]
|
||||||
Reference in new issue
Block a user