toknmtr-selfhost/ops

ops/ — capture wiring (hooks + cron)

This directory holds everything needed to make the agent/ JSONL parser run automatically on a machine, instead of being invoked by hand. Two complementary triggers feed the same idempotent ingest pipeline:

Trigger	Script	Fires	Purpose
Claude Code Stop hook	agent/hooks/toknmtr-capture.sh (installed by ops/install-hook.sh)	end of every assistant turn	near-live capture
cron	ops/install-cron.sh	every ~10 min	reconcile sweep / safety net

Both ultimately just run:

node --experimental-strip-types agent/run.ts --once

which is itself idempotent (see "Why both triggers are safe to overlap" below), so having both wired up is never a correctness problem — only a (very cheap) redundancy.

1. One-time setup: ~/.toknmtr/env

Both the hook and the cron sweep read agent config from a file outside ~/.claude/settings.json, specifically so TOKNMTR_TOKEN never ends up in a file that's more likely to be synced, shared, or pasted into a support thread.

Create ~/.toknmtr/env (plain KEY=value lines, shell-sourceable):

mkdir -p ~/.toknmtr
cat > ~/.toknmtr/env <<'EOF'
TOKNMTR_URL=http://<server-host>:3001
TOKNMTR_TOKEN=<the API_TOKEN configured on the server>
EOF
chmod 600 ~/.toknmtr/env

• TOKNMTR_URL — base URL of the deployed server (see .env.example at the repo root for the server-side API_TOKEN / PORT config that this must match).
• TOKNMTR_TOKEN — must equal the server's API_TOKEN env var; sent as Authorization: Bearer <token> on every /api/ingest request (agent/push.ts).

Neither the hook script nor the installers will create this file for you — they treat its absence as "not configured yet" and (for the hook) silently no-op, or (for the installers) proceed anyway since the file only needs to exist by the time a sweep actually runs.

If you'd rather keep config somewhere else, both scripts honor TOKNMTR_ENV_FILE to override the path (see each script's header for the full list of override env vars).

2. Install the live (Stop-hook) capture path

ops/install-hook.sh

This merges a Stop hook entry into ~/.claude/settings.json (via jq, additive — it will not touch any other hooks or settings already there) pointing at agent/hooks/toknmtr-capture.sh. It:

• backs up settings.json (timestamped, alongside the original) before writing,
• is idempotent — matches on the exact command string, so re-running is a no-op,
• is reversible — ops/install-hook.sh --remove deletes just this entry.

Why Stop and not SessionEnd (this is a real tradeoff, documented in full in the script's header comment — short version): Stop fires at the end of every assistant turn, so capture is close to live; SessionEnd only fires once when the CLI process exits, and Claude Code does not guarantee it fires on every exit path (e.g. a killed terminal can skip it). Since agent/hooks/toknmtr-capture.sh is designed to cost the foreground session basically nothing (it backgrounds the real work and returns in milliseconds — verified: ~3ms wall time in testing), Stop's higher firing frequency is nearly free, so it wins.

The exact JSON this installs (for review before running it live) is reproduced in this repo's PR/report — see also ~/.claude/settings.json directly after running the script.

3. Install the cron reconcile sweep

ops/install-cron.sh

Adds a crontab line (default schedule */10 * * * *) that sources ~/.toknmtr/env and runs agent/run.ts --once, logging to ~/.toknmtr/cron.log. Idempotent (matched by a # toknmtr-cron-reconcile marker comment) and reversible (ops/install-cron.sh --remove, or just delete the marked line via crontab -e). Only touches the one marked line — every other line in the user's crontab is preserved verbatim.

4. Backfill (one-time, per machine)

The hook/cron only push new bytes appended to transcripts since the last run (see the per-file cursor logic in agent/cursor.ts / agent/run.ts). To ingest every transcript that already exists on a machine before capture was wired up:

cd ~/claude/projects/toknmtr
node --experimental-strip-types agent/run.ts --backfill

--backfill ignores stored cursors and reparses every transcript from byte 0 (but still updates cursors afterward, so the next run — hook or cron — resumes incrementally from there rather than re-walking everything again). Safe to re-run any time: ingest is an idempotent upsert keyed on host + session_id + uuid (events) and deduped on session_id + message_id + request_id (usage), so a repeated backfill just re-writes the same rows.

Why both triggers are safe to overlap

Every push is an idempotent upsert. The event primary key is host + session_id + uuid (every physical JSONL line has a unique top-level uuid); usage rows are deduped to exactly one canonical row per session_id + message_id + request_id group server-side. So if a hook-triggered sweep and a cron-triggered sweep ever race (or a hook fires twice because of a fast back-to-back turn, or the cron sweep re-reads a tail the hook already pushed), the result is just redundant writes of already-correct rows — never duplicate or conflicting data. Cursors (agent/cursor.ts) are only advanced after a push succeeds, so a failed/unreachable-server push is naturally retried by the next sweep (hook or cron) rather than silently dropping data.

Files in this directory

• install-hook.sh — registers/unregisters the Stop hook in ~/.claude/settings.json.
• install-cron.sh — registers/unregisters the cron reconcile line.
• README.md — this file.

The hook script itself lives at agent/hooks/toknmtr-capture.sh (next to the rest of the agent code, since it's part of the agent's runtime surface, not an ops/deploy concern).

Troubleshooting

• Nothing showing up in the dashboard: check ~/.toknmtr/capture.log (hook) and ~/.toknmtr/cron.log (cron) for errors — most commonly a wrong TOKNMTR_URL/ TOKNMTR_TOKEN in ~/.toknmtr/env, or the server being unreachable.
• Hook seems to never run: confirm it's actually registered — jq '.hooks.Stop' ~/.claude/settings.json — and that agent/hooks/toknmtr-capture.sh is executable (chmod +x).
• Suspect a missed turn / gap: the cron sweep will pick it up within its schedule window regardless of what the hook did or didn't capture; you can also just run node --experimental-strip-types agent/run.ts --once by hand at any time.
• Want to undo everything: ops/install-hook.sh --remove && ops/install-cron.sh --remove.