alexakhbar/llmsnarf repository overview

Github repository⁠

LLM Snarf is a lightweight, self-hosted proxy for OpenAI-compatible LLM APIs. It sits between your client application and any OpenAI- or Anthropic compatible provider (local or remote), and gives you full visibility into every request and response. It also collects token usage and calculate the projected costs.

⁠Intended use:

• Access all your models from a single base url by configuring multiple providers and route by model name
• Integration for clients requiring specific model names by configuring aliases
• Seamless transformation between OpenAI compatible- and Anthropic compatible APIs
• Turn your locally hosted models into agents by configuring web_search and web_fetch tools
• Monitor prompts from your agents BOS (BrowserOS)⁠, Hermes⁠, or tools like Open WebUI⁠, Continue.dev⁠
• Monitor token usage and costs by configuring token prices for your proxied models to debug prompts, monitor token usage and costs.

What it does:

• Routes requests to one or more configured backend endpoints, with model-based routing and aliasing
• Traces every request/response pair to disk as JSON (toggle per endpoint)
• Validates responses after the connection closes and records failures in a Validation Ledger for later inspection
• Optionally injects web search and web fetch capabilities via OpenSerp⁠
• Provides a built-in web UI for live tail, trace inspection, configuration, token usage, and the validation ledger

⁠Example Docker Compose stack

services:
  llm-snarf:
    build: .
    container_name: llmsnarf
    hostname: llmsnarf
    image: alexakhbar/llmsnarf:latest
    env_file:
      - .env
    ports:
      - "${PORT}:${PORT}"
    networks:
      - llmsnarf
    environment:
      - LLMSNARF_CONFIG=/app/config
    volumes:
      - ./config:/app/config
      - ./traces:/app/traces
      - ./logs:/app/logs
    depends_on:
      - openserp

  openserp:
    image: karust/openserp:latest
    container_name: openserp
    hostname: openserp
    command: serve -l
    ports:
      - "7000:7000"
    networks:
      - llmsnarf
    environment:
      - PORT=7000

networks:
  llmsnarf:
    driver: bridge