toknmtr — self-hostable Claude Code usage & analytics dashboard

Server (SvelteKit + SQLite Docker container) + per-machine agent that parses
Claude Code JSONL transcripts. docker-compose one-command deploy; raw transcript
view + search gated behind SHOW_TRANSCRIPTS (hidden by default).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

ops/README.md

@@ -0,0 +1,137 @@
+# ops/ — capture wiring (hooks + cron)
+
+This directory holds everything needed to make the `agent/` JSONL parser run automatically
+on a machine, instead of being invoked by hand. Two complementary triggers feed the same
+idempotent ingest pipeline:
+
+| Trigger                 | Script                                                                | Fires                       | Purpose                      |
+| ----------------------- | --------------------------------------------------------------------- | --------------------------- | ---------------------------- |
+| Claude Code `Stop` hook | `agent/hooks/toknmtr-capture.sh` (installed by `ops/install-hook.sh`) | end of every assistant turn | near-live capture            |
+| cron                    | `ops/install-cron.sh`                                                 | every ~10 min               | reconcile sweep / safety net |
+
+Both ultimately just run:
+
+```sh
+node --experimental-strip-types agent/run.ts --once
+```
+
+which is itself idempotent (see "Why both triggers are safe to overlap" below), so having
+both wired up is never a correctness problem — only a (very cheap) redundancy.
+
+## 1. One-time setup: `~/.toknmtr/env`
+
+Both the hook and the cron sweep read agent config from a file **outside**
+`~/.claude/settings.json`, specifically so `TOKNMTR_TOKEN` never ends up in a file that's
+more likely to be synced, shared, or pasted into a support thread.
+
+Create `~/.toknmtr/env` (plain `KEY=value` lines, shell-sourceable):
+
+```sh
+mkdir -p ~/.toknmtr
+cat > ~/.toknmtr/env <<'EOF'
+TOKNMTR_URL=http://<server-host>:3001
+TOKNMTR_TOKEN=<the API_TOKEN configured on the server>
+EOF
+chmod 600 ~/.toknmtr/env
+```
+
+- `TOKNMTR_URL` — base URL of the deployed server (see `.env.example` at the repo root for
+  the server-side `API_TOKEN` / `PORT` config that this must match).
+- `TOKNMTR_TOKEN` — must equal the server's `API_TOKEN` env var; sent as
+  `Authorization: Bearer <token>` on every `/api/ingest` request (`agent/push.ts`).
+
+Neither the hook script nor the installers will create this file for you — they treat its
+absence as "not configured yet" and (for the hook) silently no-op, or (for the installers)
+proceed anyway since the file only needs to exist by the time a sweep actually runs.
+
+If you'd rather keep config somewhere else, both scripts honor `TOKNMTR_ENV_FILE` to
+override the path (see each script's header for the full list of override env vars).
+
+## 2. Install the live (`Stop`-hook) capture path
+
+```sh
+ops/install-hook.sh
+```
+
+This merges a `Stop` hook entry into `~/.claude/settings.json` (via `jq`, additive — it
+will not touch any other hooks or settings already there) pointing at
+`agent/hooks/toknmtr-capture.sh`. It:
+
+- backs up `settings.json` (timestamped, alongside the original) before writing,
+- is idempotent — matches on the exact command string, so re-running is a no-op,
+- is reversible — `ops/install-hook.sh --remove` deletes just this entry.
+
+**Why `Stop` and not `SessionEnd`** (this is a real tradeoff, documented in full in the
+script's header comment — short version): `Stop` fires at the end of every assistant turn,
+so capture is close to live; `SessionEnd` only fires once when the CLI process exits, and
+Claude Code does not guarantee it fires on every exit path (e.g. a killed terminal can skip
+it). Since `agent/hooks/toknmtr-capture.sh` is designed to cost the foreground session
+basically nothing (it backgrounds the real work and returns in milliseconds — verified:
+~3ms wall time in testing), `Stop`'s higher firing frequency is nearly free, so it wins.
+
+The exact JSON this installs (for review before running it live) is reproduced in this
+repo's PR/report — see also `~/.claude/settings.json` directly after running the script.
+
+## 3. Install the cron reconcile sweep
+
+```sh
+ops/install-cron.sh
+```
+
+Adds a crontab line (default schedule `*/10 * * * *`) that sources `~/.toknmtr/env` and
+runs `agent/run.ts --once`, logging to `~/.toknmtr/cron.log`. Idempotent (matched by a
+`# toknmtr-cron-reconcile` marker comment) and reversible (`ops/install-cron.sh --remove`,
+or just delete the marked line via `crontab -e`). Only touches the one marked line — every
+other line in the user's crontab is preserved verbatim.
+
+## 4. Backfill (one-time, per machine)
+
+The hook/cron only push _new_ bytes appended to transcripts since the last run (see the
+per-file cursor logic in `agent/cursor.ts` / `agent/run.ts`). To ingest every transcript
+that already exists on a machine before capture was wired up:
+
+```sh
+cd ~/claude/projects/toknmtr
+node --experimental-strip-types agent/run.ts --backfill
+```
+
+`--backfill` ignores stored cursors and reparses every transcript from byte 0 (but still
+updates cursors afterward, so the _next_ run — hook or cron — resumes incrementally from
+there rather than re-walking everything again). Safe to re-run any time: ingest is an
+idempotent upsert keyed on `host + session_id + uuid` (events) and deduped on
+`session_id + message_id + request_id` (usage), so a repeated backfill just re-writes the
+same rows.
+
+## Why both triggers are safe to overlap
+
+Every push is an idempotent upsert. The event primary key is `host + session_id + uuid`
+(every physical JSONL line has a unique top-level `uuid`); usage rows are deduped to
+exactly one canonical row per `session_id + message_id + request_id` group server-side.
+So if a hook-triggered sweep and a cron-triggered sweep ever race (or a hook fires twice
+because of a fast back-to-back turn, or the cron sweep re-reads a tail the hook already
+pushed), the result is just redundant writes of already-correct rows — never duplicate or
+conflicting data. Cursors (`agent/cursor.ts`) are only advanced _after_ a push succeeds, so
+a failed/unreachable-server push is naturally retried by the next sweep (hook or cron)
+rather than silently dropping data.
+
+## Files in this directory
+
+- `install-hook.sh` — registers/unregisters the `Stop` hook in `~/.claude/settings.json`.
+- `install-cron.sh` — registers/unregisters the cron reconcile line.
+- `README.md` — this file.
+
+The hook script itself lives at `agent/hooks/toknmtr-capture.sh` (next to the rest of the
+agent code, since it's part of the agent's runtime surface, not an ops/deploy concern).
+
+## Troubleshooting
+
+- **Nothing showing up in the dashboard:** check `~/.toknmtr/capture.log` (hook) and
+  `~/.toknmtr/cron.log` (cron) for errors — most commonly a wrong `TOKNMTR_URL`/
+  `TOKNMTR_TOKEN` in `~/.toknmtr/env`, or the server being unreachable.
+- **Hook seems to never run:** confirm it's actually registered —
+  `jq '.hooks.Stop' ~/.claude/settings.json` — and that `agent/hooks/toknmtr-capture.sh` is
+  executable (`chmod +x`).
+- **Suspect a missed turn / gap:** the cron sweep will pick it up within its schedule
+  window regardless of what the hook did or didn't capture; you can also just run
+  `node --experimental-strip-types agent/run.ts --once` by hand at any time.
+- **Want to undo everything:** `ops/install-hook.sh --remove && ops/install-cron.sh --remove`.