Quick Start Guide¶

Estimated time to read: 3 minutes

This guide gets the LLMTrace proxy running and verifies that traces and findings are captured. All steps below map to the current codebase.

Prerequisites¶

OpenAI API key (or another OpenAI-compatible upstream)
Either Rust toolchain or Docker

Environment Setup¶

Set your LLM provider API key before running the proxy:

export OPENAI_API_KEY="sk-..."

The proxy forwards this key to the upstream provider. Without it, proxied requests will fail with 401 Unauthorized.

Option A: One-line Install (Fastest)¶

curl -sS https://raw.githubusercontent.com/epappas/llmtrace/main/scripts/install.sh | bash

This downloads the latest binary for your platform and a starter config.yaml. Then run:

llmtrace-proxy --config config.yaml

Option B: Cargo Install¶

cargo install llmtrace
cp config.example.yaml config.yaml
llmtrace-proxy --config config.yaml

Option C: Docker (GHCR)¶

docker pull ghcr.io/epappas/llmtrace-proxy:latest
docker run -p 8080:8080 ghcr.io/epappas/llmtrace-proxy:latest

Option D: From Source¶

git clone https://github.com/epappas/llmtrace
cd llmtrace
cargo build --release --bin llmtrace-proxy --features ml
cp config.example.yaml config.yaml
./target/release/llmtrace-proxy --config config.yaml

The --features ml flag enables the DeBERTa ML-based security detectors. Without it, only regex-based detection is available.

Optional: Start Infra + Dashboard (Docker Compose)¶

compose.yaml starts ClickHouse/Postgres/Redis and the dashboard. Run the proxy separately.

cp .env.example .env
docker compose up -d

# Then run the proxy (from source or Docker)

Test the Proxy¶

1) Send a test request¶

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello, LLMTrace!"}]}'

You should receive a normal OpenAI-compatible response.

2) Fetch traces¶

curl http://localhost:8080/api/v1/traces | jq '.[0]'

3) Fetch security findings¶

curl http://localhost:8080/api/v1/security/findings | jq

Quick Integration Example (OpenAI SDK)¶

import openai

client = openai.OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="your-key"
)

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello"}]
)

Troubleshooting¶

Symptom	Cause	Fix
`401 Unauthorized` from upstream	Missing or invalid API key	`export OPENAI_API_KEY="sk-..."`
`connection refused` on `:8080`	Proxy not running	Check proxy logs, ensure `--config config.yaml` is passed
No security findings	ML features not compiled	Rebuild with `--features ml` (source builds only)
`config.yaml not found`	Missing config file	`cp config.example.yaml config.yaml`