# ops/ — capture wiring (hooks + cron) This directory holds everything needed to make the `agent/` JSONL parser run automatically on a machine, instead of being invoked by hand. Two complementary triggers feed the same idempotent ingest pipeline: | Trigger | Script | Fires | Purpose | | ----------------------- | --------------------------------------------------------------------- | --------------------------- | ---------------------------- | | Claude Code `Stop` hook | `agent/hooks/toknmtr-capture.sh` (installed by `ops/install-hook.sh`) | end of every assistant turn | near-live capture | | cron | `ops/install-cron.sh` | every ~10 min | reconcile sweep / safety net | Both ultimately just run: ```sh node --experimental-strip-types agent/run.ts --once ``` which is itself idempotent (see "Why both triggers are safe to overlap" below), so having both wired up is never a correctness problem — only a (very cheap) redundancy. ## 1. One-time setup: `~/.toknmtr/env` Both the hook and the cron sweep read agent config from a file **outside** `~/.claude/settings.json`, specifically so `TOKNMTR_TOKEN` never ends up in a file that's more likely to be synced, shared, or pasted into a support thread. Create `~/.toknmtr/env` (plain `KEY=value` lines, shell-sourceable): ```sh mkdir -p ~/.toknmtr cat > ~/.toknmtr/env <<'EOF' TOKNMTR_URL=http://:3001 TOKNMTR_TOKEN= EOF chmod 600 ~/.toknmtr/env ``` - `TOKNMTR_URL` — base URL of the deployed server (see `.env.example` at the repo root for the server-side `API_TOKEN` / `PORT` config that this must match). - `TOKNMTR_TOKEN` — must equal the server's `API_TOKEN` env var; sent as `Authorization: Bearer ` on every `/api/ingest` request (`agent/push.ts`). Neither the hook script nor the installers will create this file for you — they treat its absence as "not configured yet" and (for the hook) silently no-op, or (for the installers) proceed anyway since the file only needs to exist by the time a sweep actually runs. If you'd rather keep config somewhere else, both scripts honor `TOKNMTR_ENV_FILE` to override the path (see each script's header for the full list of override env vars). ## 2. Install the live (`Stop`-hook) capture path ```sh ops/install-hook.sh ``` This merges a `Stop` hook entry into `~/.claude/settings.json` (via `jq`, additive — it will not touch any other hooks or settings already there) pointing at `agent/hooks/toknmtr-capture.sh`. It: - backs up `settings.json` (timestamped, alongside the original) before writing, - is idempotent — matches on the exact command string, so re-running is a no-op, - is reversible — `ops/install-hook.sh --remove` deletes just this entry. **Why `Stop` and not `SessionEnd`** (this is a real tradeoff, documented in full in the script's header comment — short version): `Stop` fires at the end of every assistant turn, so capture is close to live; `SessionEnd` only fires once when the CLI process exits, and Claude Code does not guarantee it fires on every exit path (e.g. a killed terminal can skip it). Since `agent/hooks/toknmtr-capture.sh` is designed to cost the foreground session basically nothing (it backgrounds the real work and returns in milliseconds — verified: ~3ms wall time in testing), `Stop`'s higher firing frequency is nearly free, so it wins. The exact JSON this installs (for review before running it live) is reproduced in this repo's PR/report — see also `~/.claude/settings.json` directly after running the script. ## 3. Install the cron reconcile sweep ```sh ops/install-cron.sh ``` Adds a crontab line (default schedule `*/10 * * * *`) that sources `~/.toknmtr/env` and runs `agent/run.ts --once`, logging to `~/.toknmtr/cron.log`. Idempotent (matched by a `# toknmtr-cron-reconcile` marker comment) and reversible (`ops/install-cron.sh --remove`, or just delete the marked line via `crontab -e`). Only touches the one marked line — every other line in the user's crontab is preserved verbatim. ## 4. Backfill (one-time, per machine) The hook/cron only push _new_ bytes appended to transcripts since the last run (see the per-file cursor logic in `agent/cursor.ts` / `agent/run.ts`). To ingest every transcript that already exists on a machine before capture was wired up: ```sh cd ~/claude/projects/toknmtr node --experimental-strip-types agent/run.ts --backfill ``` `--backfill` ignores stored cursors and reparses every transcript from byte 0 (but still updates cursors afterward, so the _next_ run — hook or cron — resumes incrementally from there rather than re-walking everything again). Safe to re-run any time: ingest is an idempotent upsert keyed on `host + session_id + uuid` (events) and deduped on `session_id + message_id + request_id` (usage), so a repeated backfill just re-writes the same rows. ## Why both triggers are safe to overlap Every push is an idempotent upsert. The event primary key is `host + session_id + uuid` (every physical JSONL line has a unique top-level `uuid`); usage rows are deduped to exactly one canonical row per `session_id + message_id + request_id` group server-side. So if a hook-triggered sweep and a cron-triggered sweep ever race (or a hook fires twice because of a fast back-to-back turn, or the cron sweep re-reads a tail the hook already pushed), the result is just redundant writes of already-correct rows — never duplicate or conflicting data. Cursors (`agent/cursor.ts`) are only advanced _after_ a push succeeds, so a failed/unreachable-server push is naturally retried by the next sweep (hook or cron) rather than silently dropping data. ## Files in this directory - `install-hook.sh` — registers/unregisters the `Stop` hook in `~/.claude/settings.json`. - `install-cron.sh` — registers/unregisters the cron reconcile line. - `README.md` — this file. The hook script itself lives at `agent/hooks/toknmtr-capture.sh` (next to the rest of the agent code, since it's part of the agent's runtime surface, not an ops/deploy concern). ## Troubleshooting - **Nothing showing up in the dashboard:** check `~/.toknmtr/capture.log` (hook) and `~/.toknmtr/cron.log` (cron) for errors — most commonly a wrong `TOKNMTR_URL`/ `TOKNMTR_TOKEN` in `~/.toknmtr/env`, or the server being unreachable. - **Hook seems to never run:** confirm it's actually registered — `jq '.hooks.Stop' ~/.claude/settings.json` — and that `agent/hooks/toknmtr-capture.sh` is executable (`chmod +x`). - **Suspect a missed turn / gap:** the cron sweep will pick it up within its schedule window regardless of what the hook did or didn't capture; you can also just run `node --experimental-strip-types agent/run.ts --once` by hand at any time. - **Want to undo everything:** `ops/install-hook.sh --remove && ops/install-cron.sh --remove`.