type
Playbook
title
Runbook — deploy, restart, rotate, provision, recover
description
The operational path for nx-master, including the traps that cost time the first time.
tags
oncall, deploy, operations
status
stable
generated
crm-bot/deepseek-v4.1-flash · Fri Oct 30 2026 10:53:57 GMT+0100 (Central European Standard Time)
verified
process:live-probe · Fri Oct 30 2026 10:53:57 GMT+0100 (Central European Standard Time)
sources
makefile → https://github.com/nextrum-sy/nx-master/blob/main/Makefile

Deploy

cd ~/Documents/work/nx-master
make deploy HOST=82.165.128.237 SSH_KEY=~/.ssh/nx_master_ed25519

deploy rsyncs the tree and runs compose up -d --force-recreate. Two traps:

  1. --force-recreate is not optional. compose up does not recreate a container when only env_file content changed, so an env edit appears to do nothing.
  2. The data volume is separate (nx-master_nx_data → /data). rsync --delete must never be able to reach nx.db or the .jsonl files; if a deploy ever looks like it reset the CRM, that boundary is what broke.

After changing MCP tools

The agent caches tools/list at connect time. Adding a tool is not visible to the model until the gateway restarts:

ssh root@82.165.128.237
hermes mcp test nx-master      # server side: is the tool actually advertised?
hermes gateway restart         # then: make the agent re-read it
sleep 14 && systemctl --user is-active hermes-gateway

Symptom if you skip it: the model insists "I don't have that tool" while /mcp lists it. Verify the server first so you do not debug the wrong half.

Rotating credentials

Credential Where Effect of rotation
MCP_TOKEN /opt/nx-master/.env app restart; the Hermes MCP entry must be updated in step
API_SERVER_KEY /root/.hermes/.env hermes gateway restart; app restart with the new HERMES_API_KEY
OPENCODE_GO_API_KEY /root/.hermes/.env gateway restart only
GITHUB_CLIENT_SECRET /opt/nx-master/.env app recreate; existing sessions survive

Two of these were pasted into a chat during setup and must be treated as burned regardless of current behaviour.

Provisioning a user

docker exec nx-master-app-1 nxmaster useradd --login <gh-login> --role member

The row is an allowlist entry, not an account: provider_id binds on the person's first successful GitHub sign-in. Adding a user must eventually also create their bot — see one bot per user; until that exists, a new user shares the single agent and its single MCP token.

Recovering a silent chat

  1. docker compose -f /opt/nx-master/docker-compose.yml ps — is app healthy?
  2. curl -sk -H 'Host: master.nextrum-sy.de' https://127.0.0.1/healthz
  3. ss -ltn | grep 8642 — the agent API is per-user systemd, not a container.
  4. journalctl --user -u hermes-gateway -n 50 — exit 75 means another gateway already owns this profile; stop it before starting the unit.
  5. ufw status | grep 8642 — the docker bridge must reach the API. ufw deny 8642 once broke the chat while leaving the container healthy.

Certificates

Daily renewal via /etc/cron.d/nx-master-certbot, lineage nx, SAN exactly master.nextrum-sy.de. Do not add a name to the SAN that no longer resolves — the next http-01 challenge fails and takes the live certificate with it.

Rendered from nx-master/runbook.md · OKF bundle · regenerate with npm run render