toknmtr-selfhost/DEPLOY.md

# Deploying toknmtr — step by step

This walks you from a clone to a live dashboard with your own Claude Code usage flowing in.

toknmtr has **two halves**, and you set them up in order:

1. **Server** — one Docker container (dashboard + ingest API + SQLite). Run it once, anywhere
   reachable from your machines (a home server, a NAS, or even your laptop).
2. **Agent** — a small script you run on **every machine where you use Claude Code**. It reads
   that machine's transcripts and pushes them to the server. Nothing shows up until at least
   one agent has run.

> **The server image holds no data.** The database is created empty in a Docker volume the
> first time the container starts. Everything you see in the dashboard came from *your* agent
> pushing *your* transcripts to *your* server.

---

## Prerequisites

**For the server:**
- A host with **Docker** + the **Compose plugin** (`docker compose version` works).

**For the agent (on each machine you use Claude Code):**
- **Node.js 24+** (`node --version`) — needed for `--experimental-strip-types`.
- A clone of this repo.
- `jq` (only if you use the auto-capture hook installer).

---

## Part 1 — Run the server

### 1.1 Clone and configure

```sh
git clone <this-repo-url> toknmtr
cd toknmtr
cp .env.example .env
```

Open `.env` and set a strong `API_TOKEN` — this is the shared secret between the server and
every agent. Generate one:

```sh
openssl rand -hex 32
```

Paste it as `API_TOKEN=...`. Leave `SHOW_TRANSCRIPTS` commented out for now (see
[Security](#security--exposure) below).

### 1.2 Start it

```sh
docker compose up -d --build
```

The first build takes a few minutes (it compiles `better-sqlite3`). When it finishes, the
dashboard is at **http://localhost:3001** (or `http://<server-host>:3001` from another
machine on your network).

### 1.3 Verify

```sh
# health check — should print {"ok":true,...}
curl -s http://localhost:3001/api/ingest

# dashboard should return 200
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:3001/
```

The dashboard will be empty until you set up an agent (Part 2). The container restarts
automatically unless you stop it; the DB persists in the `toknmtr-data` Docker volume across
restarts and rebuilds.

**Common first-run issues**
- `API_TOKEN` error from compose → you didn't set it in `.env`.
- Port 3001 already in use → change the host side of the mapping in `docker-compose.yml`
  (`"3001:3000"` → e.g. `"8099:3000"`), then `docker compose up -d`.

---

## Part 2 — Feed it data (the agent)

Do this on **each machine** where you run Claude Code. The agent reads
`~/.claude/projects/**/*.jsonl` and POSTs to your server.

### 2.1 Point the agent at your server

```sh
mkdir -p ~/.toknmtr
cat > ~/.toknmtr/env <<'EOF'
TOKNMTR_URL=http://<server-host>:3001
TOKNMTR_TOKEN=<the same API_TOKEN you put on the server>
EOF
chmod 600 ~/.toknmtr/env
```

Replace `<server-host>` with the server's hostname or LAN IP (use `localhost` if the agent
runs on the same box as the server). `TOKNMTR_TOKEN` **must exactly match** the server's
`API_TOKEN`.

### 2.2 Backfill existing history (one time)

From your clone of this repo on that machine:

```sh
node --experimental-strip-types agent/run.ts --backfill
```

This ingests every transcript already on disk. Refresh the dashboard — data should appear.

> Claude Code prunes local transcripts after `cleanupPeriodDays` (default 30), so backfill
> only reaches as far back as what's still on disk. From here on, the live hook (next step)
> captures everything going forward into the server's permanent DB.

### 2.3 Turn on live capture

```sh
ops/install-hook.sh
```

This registers a Claude Code `Stop` hook that pushes new activity at the end of every turn —
near-live, and near-zero cost to your session. It's additive and reversible
(`ops/install-hook.sh --remove`).

Optionally add a periodic reconcile sweep as a safety net (catches anything a missed hook or
a server-down window skipped):

```sh
ops/install-cron.sh
```

Full agent details — the `~/.toknmtr/env` format, the hook-vs-cron tradeoff, and
troubleshooting — are in **[ops/README.md](ops/README.md)**.

Re-ingesting is always safe: the server upserts idempotently on `host + session_id + uuid`,
so backfills and overlapping hook/cron sweeps never create duplicates.

---

## Security & exposure

**The dashboard has no login.** Treat it as trusted-network-only unless you add auth.

- **Raw conversation text is hidden by default.** The transcript view and full-text search —
  the only surfaces that show verbatim prompts/responses — return `403` unless you set
  `SHOW_TRANSCRIPTS=true`. Charts, KPIs, and session metadata are always visible. Only enable
  `SHOW_TRANSCRIPTS` once the whole dashboard is behind authentication.
- **Don't put it on the public internet as-is.** Keep it on your LAN/VPN, or front it with a
  reverse proxy that enforces auth (e.g. Caddy/nginx basic-auth, Authelia, Tailscale, etc.).
- The `API_TOKEN` gates ingest only, not the dashboard. Keep it secret; it lives in `.env`
  (server) and `~/.toknmtr/env` (agents), both of which stay off git.

To reveal transcripts/search after you've added auth: set `SHOW_TRANSCRIPTS=true` in `.env`
and `docker compose up -d` again.

---

## Updating

```sh
cd toknmtr
git pull
docker compose up -d --build
```

Your data is untouched — it lives in the `toknmtr-data` volume, not the image.

On agent machines, `git pull` in the clone; the hook/cron run the updated code automatically.

---

## Uninstalling

```sh
# server
docker compose down            # stop + remove the container (keeps the data volume)
docker compose down -v         # ...and DELETE the database volume too

# agent (per machine)
ops/install-hook.sh --remove
ops/install-cron.sh --remove
rm -rf ~/.toknmtr
```