toknmtr — self-hostable Claude Code usage & analytics dashboard
Server (SvelteKit + SQLite Docker container) + per-machine agent that parses Claude Code JSONL transcripts. docker-compose one-command deploy; raw transcript view + search gated behind SHOW_TRANSCRIPTS (hidden by default). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
commit
71a60ab054
74 changed files with 15613 additions and 0 deletions
184
DEPLOY.md
Normal file
184
DEPLOY.md
Normal file
|
|
@ -0,0 +1,184 @@
|
|||
# Deploying toknmtr — step by step
|
||||
|
||||
This walks you from a clone to a live dashboard with your own Claude Code usage flowing in.
|
||||
|
||||
toknmtr has **two halves**, and you set them up in order:
|
||||
|
||||
1. **Server** — one Docker container (dashboard + ingest API + SQLite). Run it once, anywhere
|
||||
reachable from your machines (a home server, a NAS, or even your laptop).
|
||||
2. **Agent** — a small script you run on **every machine where you use Claude Code**. It reads
|
||||
that machine's transcripts and pushes them to the server. Nothing shows up until at least
|
||||
one agent has run.
|
||||
|
||||
> **The server image holds no data.** The database is created empty in a Docker volume the
|
||||
> first time the container starts. Everything you see in the dashboard came from *your* agent
|
||||
> pushing *your* transcripts to *your* server.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
**For the server:**
|
||||
- A host with **Docker** + the **Compose plugin** (`docker compose version` works).
|
||||
|
||||
**For the agent (on each machine you use Claude Code):**
|
||||
- **Node.js 24+** (`node --version`) — needed for `--experimental-strip-types`.
|
||||
- A clone of this repo.
|
||||
- `jq` (only if you use the auto-capture hook installer).
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Run the server
|
||||
|
||||
### 1.1 Clone and configure
|
||||
|
||||
```sh
|
||||
git clone <this-repo-url> toknmtr
|
||||
cd toknmtr
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Open `.env` and set a strong `API_TOKEN` — this is the shared secret between the server and
|
||||
every agent. Generate one:
|
||||
|
||||
```sh
|
||||
openssl rand -hex 32
|
||||
```
|
||||
|
||||
Paste it as `API_TOKEN=...`. Leave `SHOW_TRANSCRIPTS` commented out for now (see
|
||||
[Security](#security--exposure) below).
|
||||
|
||||
### 1.2 Start it
|
||||
|
||||
```sh
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
The first build takes a few minutes (it compiles `better-sqlite3`). When it finishes, the
|
||||
dashboard is at **http://localhost:3001** (or `http://<server-host>:3001` from another
|
||||
machine on your network).
|
||||
|
||||
### 1.3 Verify
|
||||
|
||||
```sh
|
||||
# health check — should print {"ok":true,...}
|
||||
curl -s http://localhost:3001/api/ingest
|
||||
|
||||
# dashboard should return 200
|
||||
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:3001/
|
||||
```
|
||||
|
||||
The dashboard will be empty until you set up an agent (Part 2). The container restarts
|
||||
automatically unless you stop it; the DB persists in the `toknmtr-data` Docker volume across
|
||||
restarts and rebuilds.
|
||||
|
||||
**Common first-run issues**
|
||||
- `API_TOKEN` error from compose → you didn't set it in `.env`.
|
||||
- Port 3001 already in use → change the host side of the mapping in `docker-compose.yml`
|
||||
(`"3001:3000"` → e.g. `"8099:3000"`), then `docker compose up -d`.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Feed it data (the agent)
|
||||
|
||||
Do this on **each machine** where you run Claude Code. The agent reads
|
||||
`~/.claude/projects/**/*.jsonl` and POSTs to your server.
|
||||
|
||||
### 2.1 Point the agent at your server
|
||||
|
||||
```sh
|
||||
mkdir -p ~/.toknmtr
|
||||
cat > ~/.toknmtr/env <<'EOF'
|
||||
TOKNMTR_URL=http://<server-host>:3001
|
||||
TOKNMTR_TOKEN=<the same API_TOKEN you put on the server>
|
||||
EOF
|
||||
chmod 600 ~/.toknmtr/env
|
||||
```
|
||||
|
||||
Replace `<server-host>` with the server's hostname or LAN IP (use `localhost` if the agent
|
||||
runs on the same box as the server). `TOKNMTR_TOKEN` **must exactly match** the server's
|
||||
`API_TOKEN`.
|
||||
|
||||
### 2.2 Backfill existing history (one time)
|
||||
|
||||
From your clone of this repo on that machine:
|
||||
|
||||
```sh
|
||||
node --experimental-strip-types agent/run.ts --backfill
|
||||
```
|
||||
|
||||
This ingests every transcript already on disk. Refresh the dashboard — data should appear.
|
||||
|
||||
> Claude Code prunes local transcripts after `cleanupPeriodDays` (default 30), so backfill
|
||||
> only reaches as far back as what's still on disk. From here on, the live hook (next step)
|
||||
> captures everything going forward into the server's permanent DB.
|
||||
|
||||
### 2.3 Turn on live capture
|
||||
|
||||
```sh
|
||||
ops/install-hook.sh
|
||||
```
|
||||
|
||||
This registers a Claude Code `Stop` hook that pushes new activity at the end of every turn —
|
||||
near-live, and near-zero cost to your session. It's additive and reversible
|
||||
(`ops/install-hook.sh --remove`).
|
||||
|
||||
Optionally add a periodic reconcile sweep as a safety net (catches anything a missed hook or
|
||||
a server-down window skipped):
|
||||
|
||||
```sh
|
||||
ops/install-cron.sh
|
||||
```
|
||||
|
||||
Full agent details — the `~/.toknmtr/env` format, the hook-vs-cron tradeoff, and
|
||||
troubleshooting — are in **[ops/README.md](ops/README.md)**.
|
||||
|
||||
Re-ingesting is always safe: the server upserts idempotently on `host + session_id + uuid`,
|
||||
so backfills and overlapping hook/cron sweeps never create duplicates.
|
||||
|
||||
---
|
||||
|
||||
## Security & exposure
|
||||
|
||||
**The dashboard has no login.** Treat it as trusted-network-only unless you add auth.
|
||||
|
||||
- **Raw conversation text is hidden by default.** The transcript view and full-text search —
|
||||
the only surfaces that show verbatim prompts/responses — return `403` unless you set
|
||||
`SHOW_TRANSCRIPTS=true`. Charts, KPIs, and session metadata are always visible. Only enable
|
||||
`SHOW_TRANSCRIPTS` once the whole dashboard is behind authentication.
|
||||
- **Don't put it on the public internet as-is.** Keep it on your LAN/VPN, or front it with a
|
||||
reverse proxy that enforces auth (e.g. Caddy/nginx basic-auth, Authelia, Tailscale, etc.).
|
||||
- The `API_TOKEN` gates ingest only, not the dashboard. Keep it secret; it lives in `.env`
|
||||
(server) and `~/.toknmtr/env` (agents), both of which stay off git.
|
||||
|
||||
To reveal transcripts/search after you've added auth: set `SHOW_TRANSCRIPTS=true` in `.env`
|
||||
and `docker compose up -d` again.
|
||||
|
||||
---
|
||||
|
||||
## Updating
|
||||
|
||||
```sh
|
||||
cd toknmtr
|
||||
git pull
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Your data is untouched — it lives in the `toknmtr-data` volume, not the image.
|
||||
|
||||
On agent machines, `git pull` in the clone; the hook/cron run the updated code automatically.
|
||||
|
||||
---
|
||||
|
||||
## Uninstalling
|
||||
|
||||
```sh
|
||||
# server
|
||||
docker compose down # stop + remove the container (keeps the data volume)
|
||||
docker compose down -v # ...and DELETE the database volume too
|
||||
|
||||
# agent (per machine)
|
||||
ops/install-hook.sh --remove
|
||||
ops/install-cron.sh --remove
|
||||
rm -rf ~/.toknmtr
|
||||
```
|
||||
Loading…
Add table
Add a link
Reference in a new issue