Complete Phase 3, Phase 5 polish and hardening

Phase 3 — Investments: - Multi-currency support: holdings track purchase currency, FX rates convert to base for totals - Capital gains report using UK Section 104 pool method, grouped by tax year - Capital Gains tab added to Reports page Phase 5 — Polish & Hardening: - Mobile-responsive layout: bottom nav, sidebar hidden on mobile, logo in TopBar, compact header buttons, hover-only actions now always visible on touch - Backup system: encrypted GPG backups via backup.sh, nightly scheduler job, admin API (list/trigger/download/restore), Settings UI with drag-to-restore confirmation - Docker entrypoint with gosu privilege drop to fix bind-mount ownership on fresh deployments - OWASP fixes: refresh token now bound to its session (new refresh_token_hash column + migration), CSRF secure flag tied to environment, IP-level rate limiting on login, TOTPEnableRequest Pydantic schema replaces raw dict - AES-256-GCM key rotation script (rotate_keys.py) with dry-run mode and atomic DB transaction - CLAUDE.md added for AI-assisted development context - README updated: correct reverse proxy port, accurate backup/restore commands, key rotation instructions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 14:59:11 +00:00 · 2026-04-22 14:59:11 +00:00 · fe4e69b9ad
commit fe4e69b9ad
parent 74e57a35c0
40 changed files with 2079 additions and 127 deletions
--- a/backend/scripts/backup.sh
+++ b/backend/scripts/backup.sh
@ -0,0 +1,51 @@
+#!/bin/bash
+# backup.sh — pg_dump | gzip | gpg encrypt → /backups/
+# Run inside the backend container:
+#   docker compose exec backend bash /app/scripts/backup.sh
+# Or from host:
+#   docker compose exec -e BACKUP_PASSPHRASE="$BACKUP_PASSPHRASE" backend bash scripts/backup.sh
+set -euo pipefail
+
+TIMESTAMP=$(date +%Y%m%d_%H%M%S)
+BACKUP_DIR="${BACKUP_DIR:-/app/backups}"
+BACKUP_FILE="${BACKUP_DIR}/${TIMESTAMP}.sql.gz.gpg"
+RETENTION_DAYS="${BACKUP_RETENTION_DAYS:-30}"
+
+# Require passphrase
+if [ -z "${BACKUP_PASSPHRASE:-}" ]; then
+  echo "[backup] ERROR: BACKUP_PASSPHRASE is not set" >&2
+  exit 1
+fi
+
+mkdir -p "${BACKUP_DIR}"
+echo "[backup] Starting at ${TIMESTAMP}"
+
+# GPG needs a writable home dir; appuser has no real home
+export GNUPGHOME=/tmp/.gnupg
+mkdir -p "${GNUPGHOME}"
+chmod 700 "${GNUPGHOME}"
+
+# pg_dump using the DATABASE_URL but swap asyncpg driver for psycopg2-compatible URL
+PG_URL="${DATABASE_URL/postgresql+asyncpg/postgresql}"
+
+pg_dump --clean --if-exists "${PG_URL}" \
+  | gzip \
+  | gpg --batch --yes --no-symkey-cache --pinentry-mode loopback \
+        --symmetric --cipher-algo AES256 \
+        --passphrase "${BACKUP_PASSPHRASE}" \
+        --output "${BACKUP_FILE}"
+
+SIZE=$(du -sh "${BACKUP_FILE}" | cut -f1)
+echo "[backup] Written ${SIZE} → ${BACKUP_FILE}"
+
+# List current backups
+COUNT=$(find "${BACKUP_DIR}" -name "*.sql.gz.gpg" | wc -l)
+echo "[backup] ${COUNT} backup(s) on disk"
+
+# Prune old backups
+PRUNED=$(find "${BACKUP_DIR}" -name "*.sql.gz.gpg" -mtime "+${RETENTION_DAYS}" -print -delete | wc -l)
+if [ "${PRUNED}" -gt 0 ]; then
+  echo "[backup] Pruned ${PRUNED} backup(s) older than ${RETENTION_DAYS} days"
+fi
+
+echo "[backup] Done"
--- a/backend/scripts/entrypoint.sh
+++ b/backend/scripts/entrypoint.sh
@ -0,0 +1,9 @@
+#!/bin/sh
+set -e
+
+# Fix ownership of bind-mounted directories so appuser can write to them.
+# This runs briefly as root before dropping privileges, which is the only
+# way to handle host directories that Docker creates as root.
+chown -R appuser:appuser /app/backups /app/uploads 2>/dev/null || true
+
+exec gosu appuser "$@"
--- a/backend/scripts/restore.sh
+++ b/backend/scripts/restore.sh
@ -0,0 +1,56 @@
+#!/bin/bash
+# restore.sh — decrypt and restore a backup
+# Usage:
+#   docker compose exec backend bash scripts/restore.sh <backup_file>
+#   docker compose exec backend bash scripts/restore.sh --list
+set -euo pipefail
+
+BACKUP_DIR="${BACKUP_DIR:-/app/backups}"
+
+if [ "${1:-}" = "--list" ]; then
+  echo "[restore] Available backups in ${BACKUP_DIR}:"
+  find "${BACKUP_DIR}" -name "*.sql.gz.gpg" -printf "  %f  (%s bytes)\n" | sort -r
+  exit 0
+fi
+
+if [ -z "${1:-}" ]; then
+  echo "Usage: restore.sh <backup_file.sql.gz.gpg>"
+  echo "       restore.sh --list"
+  exit 1
+fi
+
+BACKUP_FILE="$1"
+
+# Accept bare filename or full path
+if [ ! -f "${BACKUP_FILE}" ] && [ -f "${BACKUP_DIR}/${BACKUP_FILE}" ]; then
+  BACKUP_FILE="${BACKUP_DIR}/${BACKUP_FILE}"
+fi
+
+if [ ! -f "${BACKUP_FILE}" ]; then
+  echo "[restore] ERROR: File not found: ${BACKUP_FILE}" >&2
+  exit 1
+fi
+
+if [ -z "${BACKUP_PASSPHRASE:-}" ]; then
+  echo "[restore] ERROR: BACKUP_PASSPHRASE is not set" >&2
+  exit 1
+fi
+
+PG_URL="${DATABASE_URL/postgresql+asyncpg/postgresql}"
+
+echo "[restore] WARNING: This will overwrite the current database."
+echo "[restore] File: ${BACKUP_FILE}"
+read -r -p "[restore] Type 'yes' to continue: " CONFIRM
+if [ "${CONFIRM}" != "yes" ]; then
+  echo "[restore] Aborted"
+  exit 1
+fi
+
+echo "[restore] Decrypting and restoring…"
+gpg --batch --yes --decrypt \
+    --passphrase "${BACKUP_PASSPHRASE}" \
+    "${BACKUP_FILE}" \
+  | gunzip \
+  | psql "${PG_URL}"
+
+echo "[restore] Done"
--- a/backend/scripts/rotate_keys.py
+++ b/backend/scripts/rotate_keys.py
@ -0,0 +1,195 @@
+#!/usr/bin/env python3
+"""
+AES-256-GCM encryption key rotation.
+
+Re-encrypts all PII fields with a new key in a single atomic transaction.
+If anything fails the database is left unchanged.
+
+Usage (from inside the backend container):
+  python /app/scripts/rotate_keys.py --old-key <64-hex> --new-key <64-hex>
+
+  --dry-run   Decrypt and re-encrypt in memory only, do not write to DB.
+
+After the script reports success:
+  1. Update ENCRYPTION_KEY in your .env file to the new key.
+  2. Restart the backend: docker compose up -d backend
+"""
+import argparse
+import asyncio
+import os
+import sys
+from os import urandom
+from cryptography.hazmat.primitives.ciphers.aead import AESGCM
+
+
+# ---------------------------------------------------------------------------
+# Crypto helpers (standalone, no dependency on app code)
+# ---------------------------------------------------------------------------
+
+def _make_decrypt(key_hex: str):
+    key = bytes.fromhex(key_hex)
+    if len(key) != 32:
+        sys.exit("ERROR: key must be 32 bytes (64 hex characters)")
+    aesgcm = AESGCM(key)
+
+    def decrypt(data: bytes) -> str:
+        if not data:
+            return ""
+        iv, ct = data[:12], data[12:]
+        return aesgcm.decrypt(iv, ct, None).decode()
+
+    return decrypt
+
+
+def _make_encrypt(key_hex: str):
+    key = bytes.fromhex(key_hex)
+    if len(key) != 32:
+        sys.exit("ERROR: key must be 32 bytes (64 hex characters)")
+    aesgcm = AESGCM(key)
+
+    def encrypt(plaintext: str) -> bytes:
+        if not plaintext:
+            return b""
+        iv = urandom(12)
+        return iv + aesgcm.encrypt(iv, plaintext.encode(), None)
+
+    return encrypt
+
+
+def _rotate_bytes(data: bytes | None, decrypt, encrypt) -> bytes | None:
+    """Decrypt with old key, re-encrypt with new key. None/empty passes through."""
+    if not data:
+        return data
+    return encrypt(decrypt(data))
+
+
+def _rotate_hex(hex_str: str | None, decrypt, encrypt) -> str | None:
+    """Same as _rotate_bytes but field is stored as hex text (TOTP secret)."""
+    if not hex_str:
+        return hex_str
+    return encrypt(decrypt(bytes.fromhex(hex_str))).hex()
+
+
+# ---------------------------------------------------------------------------
+# Main rotation logic
+# ---------------------------------------------------------------------------
+
+async def rotate(old_key: str, new_key: str, dry_run: bool) -> None:
+    import asyncpg
+
+    decrypt = _make_decrypt(old_key)
+    encrypt = _make_encrypt(new_key)
+
+    db_url = os.environ.get("DATABASE_URL", "")
+    if not db_url:
+        sys.exit("ERROR: DATABASE_URL environment variable not set")
+    pg_url = db_url.replace("postgresql+asyncpg://", "postgresql://")
+
+    print(f"Connecting to database...")
+    conn = await asyncpg.connect(pg_url)
+
+    try:
+        async with conn.transaction():
+            # ── accounts ──────────────────────────────────────────────────
+            print("Rotating accounts (name, institution, notes)...")
+            rows = await conn.fetch(
+                "SELECT id, name, institution, notes FROM accounts"
+            )
+            updated = 0
+            for row in rows:
+                new_name  = _rotate_bytes(row["name"],        decrypt, encrypt)
+                new_inst  = _rotate_bytes(row["institution"], decrypt, encrypt)
+                new_notes = _rotate_bytes(row["notes"],       decrypt, encrypt)
+                if not dry_run:
+                    await conn.execute(
+                        "UPDATE accounts SET name=$1, institution=$2, notes=$3 WHERE id=$4",
+                        new_name, new_inst, new_notes, row["id"],
+                    )
+                updated += 1
+            print(f"  {'(dry-run) would rotate' if dry_run else 'rotated'} {updated} accounts")
+
+            # ── transactions ──────────────────────────────────────────────
+            print("Rotating transactions (description, merchant, notes)...")
+            rows = await conn.fetch(
+                "SELECT id, description, merchant, notes FROM transactions"
+            )
+            updated = 0
+            for row in rows:
+                new_desc  = _rotate_bytes(row["description"], decrypt, encrypt)
+                new_merch = _rotate_bytes(row["merchant"],    decrypt, encrypt)
+                new_notes = _rotate_bytes(row["notes"],       decrypt, encrypt)
+                if not dry_run:
+                    await conn.execute(
+                        "UPDATE transactions SET description=$1, merchant=$2, notes=$3 WHERE id=$4",
+                        new_desc, new_merch, new_notes, row["id"],
+                    )
+                updated += 1
+            print(f"  {'(dry-run) would rotate' if dry_run else 'rotated'} {updated} transactions")
+
+            # ── investment transactions ────────────────────────────────────
+            print("Rotating investment transaction notes...")
+            rows = await conn.fetch(
+                "SELECT id, notes FROM investment_transactions WHERE notes IS NOT NULL"
+            )
+            updated = 0
+            for row in rows:
+                new_notes = _rotate_bytes(row["notes"], decrypt, encrypt)
+                if not dry_run:
+                    await conn.execute(
+                        "UPDATE investment_transactions SET notes=$1 WHERE id=$2",
+                        new_notes, row["id"],
+                    )
+                updated += 1
+            print(f"  {'(dry-run) would rotate' if dry_run else 'rotated'} {updated} investment transaction notes")
+
+            # ── users — TOTP secret (hex-encoded) ─────────────────────────
+            print("Rotating user TOTP secrets...")
+            rows = await conn.fetch(
+                "SELECT id, totp_secret FROM users WHERE totp_secret IS NOT NULL"
+            )
+            updated = 0
+            for row in rows:
+                new_secret = _rotate_hex(row["totp_secret"], decrypt, encrypt)
+                if not dry_run:
+                    await conn.execute(
+                        "UPDATE users SET totp_secret=$1 WHERE id=$2",
+                        new_secret, row["id"],
+                    )
+                updated += 1
+            print(f"  {'(dry-run) would rotate' if dry_run else 'rotated'} {updated} TOTP secrets")
+
+            if dry_run:
+                raise _DryRunAbort()
+
+    except _DryRunAbort:
+        print("\nDry-run complete — no changes written.")
+        return
+    finally:
+        await conn.close()
+
+    print("\n✓ Key rotation complete.")
+    print("\nNext steps:")
+    print("  1. Update ENCRYPTION_KEY in your .env to the new key.")
+    print("  2. Restart the backend: docker compose up -d backend")
+
+
+class _DryRunAbort(Exception):
+    """Raised inside the transaction to trigger an asyncpg rollback in dry-run mode."""
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Rotate AES-256-GCM encryption key")
+    parser.add_argument("--old-key", required=True, help="Current key as 64-char hex")
+    parser.add_argument("--new-key", required=True, help="New key as 64-char hex")
+    parser.add_argument("--dry-run", action="store_true", help="Validate only, do not write")
+    args = parser.parse_args()
+
+    if args.old_key == args.new_key:
+        sys.exit("ERROR: old key and new key are identical — nothing to do")
+
+    print(f"{'DRY RUN — ' if args.dry_run else ''}AES key rotation starting...")
+    asyncio.run(rotate(args.old_key, args.new_key, args.dry_run))
+
+
+if __name__ == "__main__":
+    main()
--- a/backend/scripts/rotate_keys.sh
+++ b/backend/scripts/rotate_keys.sh
@ -0,0 +1,54 @@
+#!/bin/bash
+# rotate_keys.sh — re-encrypt all AES-256-GCM fields with a new key.
+# The application must be STOPPED before running.
+#
+# Usage:
+#   NEW_ENCRYPTION_KEY=$(python3 -c "import secrets; print(secrets.token_hex(32))")
+#   docker compose stop backend
+#   NEW_ENCRYPTION_KEY="$NEW_ENCRYPTION_KEY" ./scripts/rotate_keys.sh
+#   # On success, update ENCRYPTION_KEY in .env, then:
+#   docker compose up -d backend
+set -euo pipefail
+
+if [ -z "${NEW_ENCRYPTION_KEY:-}" ]; then
+  echo "ERROR: NEW_ENCRYPTION_KEY is not set"
+  echo "Generate with: python3 -c \"import secrets; print(secrets.token_hex(32))\""
+  exit 1
+fi
+
+if [ -z "${ENCRYPTION_KEY:-}" ]; then
+  # Try to load from .env in the project root
+  ENV_FILE="$(dirname "$0")/../.env"
+  if [ -f "${ENV_FILE}" ]; then
+    ENCRYPTION_KEY=$(grep -E '^ENCRYPTION_KEY=' "${ENV_FILE}" | cut -d= -f2- | tr -d '"' | tr -d "'")
+  fi
+fi
+
+if [ -z "${ENCRYPTION_KEY:-}" ]; then
+  echo "ERROR: ENCRYPTION_KEY (current key) could not be found in environment or .env"
+  exit 1
+fi
+
+if [ "${ENCRYPTION_KEY}" = "${NEW_ENCRYPTION_KEY}" ]; then
+  echo "ERROR: NEW_ENCRYPTION_KEY is the same as the current key — nothing to do"
+  exit 1
+fi
+
+echo "[rotate] This will re-encrypt ALL sensitive fields in the database."
+echo "[rotate] Ensure the application containers are stopped before continuing."
+read -r -p "[rotate] Type 'yes' to continue: " CONFIRM
+if [ "${CONFIRM}" != "yes" ]; then
+  echo "[rotate] Aborted"
+  exit 1
+fi
+
+echo "[rotate] Running key rotation inside the backend container…"
+docker compose exec \
+  -e ENCRYPTION_KEY="${ENCRYPTION_KEY}" \
+  -e NEW_ENCRYPTION_KEY="${NEW_ENCRYPTION_KEY}" \
+  backend python -m app.core.key_rotation
+
+echo ""
+echo "[rotate] SUCCESS. Next steps:"
+echo "  1. Update ENCRYPTION_KEY in .env to: ${NEW_ENCRYPTION_KEY}"
+echo "  2. Restart the backend: docker compose up -d backend"