testing-proxy

testing-proxy

A modern SvelteKit application with Svelte 5 runes, Tailwind CSS, and Flowbite components. Also serves as a multi-provider OpenAI-compatible LLM proxy with OAuth 2.1 authentication.

Setup

Install dependencies:

pnpm install

Configuration

Copy the example configuration to create your local config.json:

cp config.json.example config.json

This file contains settings for database and server configuration. See Configuration below for available options.

Development

Start the dev server on http://localhost:3000:

npm run dev

# Auto-open in browser
npm run dev -- --open

Hot module reload enabled by default.

Build & Deploy

Create a production build:

npm run build

Preview the production build locally:

npm run preview

Production output goes to /build/ (configured for Node.js via @sveltejs/adapter-node).

Docker

The application includes a multi-stage Dockerfile for production deployments using Node.js 24 Alpine.

Docker Run

Build the Docker image:

docker build -t testing-proxy .

Run the container with a mounted configuration file:

docker run -d \
  --name testing-proxy \
  -p 3000:3000 \
  --volume ./config.json:/opt/app/server/config.json:ro \
  --volume ./data:/opt/app/data \
  testing-proxy

The flags:

• --volume ./config.json:/opt/app/server/config.json:ro — Mounts your local config.json as read-only inside the container
• --volume ./data:/opt/app/data — Persists the SQLite database across container restarts

Ensure config.json exists before running:

cp config.json.example config.json
mkdir -p data

Docker Compose

Use Docker Compose for orchestrated deployments with secrets management:

version: '3.8'

services:
  app:
    image: testing-proxy:latest
    container_name: testing-proxy
    ports:
      - "3000:3000"
    secrets:
      - source: config_secret
        target: /opt/app/server/config.json
        uid: "1000"
        gid: "1000"
        mode: 0400
    volumes:
      - ./data:/opt/app/data  # Persist SQLite database
    restart: unless-stopped

secrets:
  config_secret:
    file: ./config.json

Save as docker-compose.yml and run:

docker-compose up -d

The secrets section:

• source — Named secret reference (config_secret)
• target — Container path where the config is mounted (/opt/app/server/config.json)
• uid / gid — User/group IDs inside the container (1000 for app user)
• mode — File permissions (0400 = read-only for owner)

Configuration

Before running Docker containers, create your configuration file:

cp config.json.example config.json

Both docker run and docker-compose methods expect config.json to exist on your host machine and mount it into the container.

Data Persistence

For Docker Compose deployments, the ./data volume persists the SQLite database across container restarts. Ensure the data/ directory is writable:

mkdir -p data
chmod 755 data

LLM Proxy

The application exposes an OpenAI-compatible REST API under /v1 that proxies chat completion requests to configured LLM providers (AWS Bedrock, Anthropic, or any OpenAI-compatible endpoint). All /v1 endpoints require OAuth 2.1 Bearer token authentication.

Provider Setup

Providers are configured in config.json under the providers array. Only providers with enabled: true are exposed.

Supported provider types:

Type	Required Config	Credentials
bedrock	region, credentials	AWS access key ID + secret
anthropic	apiKey	Anthropic API key
openai-compatible	baseUrl	None (local endpoint)

Example config.json snippet:

{
  "providers": [
    {
      "id": "bedrock",
      "type": "bedrock",
      "enabled": true,
      "region": "us-east-1",
      "credentials": {
        "accessKeyId": "YOUR_ACCESS_KEY",
        "secretAccessKey": "YOUR_SECRET_KEY"
      },
      "models": {
        "claude-3-sonnet": "anthropic.claude-3-sonnet-20240229-v1:0"
      }
    },
    {
      "id": "anthropic",
      "type": "anthropic",
      "enabled": true,
      "apiKey": "YOUR_ANTHROPIC_API_KEY",
      "models": {
        "claude-3-5-sonnet": "claude-3-5-sonnet-20241022"
      }
    },
    {
      "id": "local",
      "type": "openai-compatible",
      "enabled": false,
      "baseUrl": "http://localhost:11434/v1",
      "models": {
        "llama3.1": "llama3.1:latest"
      }
    }
  ]
}

Models are addressed as provider/localModelId (e.g., bedrock/claude-3-sonnet). The models object maps the local key to the provider's native identifier.

Authentication

Every /v1 request must include an Authorization header with a valid Bearer access token:

Authorization: Bearer <access_token>

Tokens are issued by the local OAuth 2.1 authorization server. The flow is:

1. Register a client via POST /oauth/register
2. Log in via POST /oauth/login (sets a session cookie)
3. Authorize via GET /oauth/authorize with PKCE S256
4. Exchange the code via POST /oauth/token

Quick Start

Use the provided script to automate token acquisition:

# Make the script executable and run it
chmod +x scripts/get-token.sh
./scripts/get-token.sh

This script runs the full OAuth 2.1 flow and prints an access token. Then make a request:

# List available models
curl http://localhost:3000/v1/models \
  -H "Authorization: Bearer <token>"

# Chat completion (non-streaming)
curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bedrock/claude-3-sonnet",
    "messages": [{"role": "user", "content": "Hello!"}],
    "temperature": 0.7,
    "max_tokens": 256
  }'

# Chat completion (streaming)
curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-3-5-sonnet",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

API Endpoints

Endpoint	Method	Description
/v1/chat/completions	POST	Chat with streaming or non-streaming response
/v1/models	GET	List all available models across enabled providers
/v1/models/{model}	GET	Get details for a single model (URL-encode the slash)

All errors are returned in OpenAI-compatible JSON format. See docs/V1-API.md for full API documentation.

Project Structure

• src/routes/ — SvelteKit filesystem routes
• src/lib/ — Reusable components and utilities (access via $lib alias)
• src/app.html — HTML shell (dark mode enabled via class="dark")

Tech Stack

• Framework: SvelteKit 2.63.0 with Svelte 5 (runes mode)
• Styling: Tailwind CSS 4.3.0 via @tailwindcss/vite + Forms/Typography plugins
• Components: Flowbite Svelte 1.33.1 + Flowbite Icons
• Build Tool: Vite 8.0.16
• Adapter: Node.js (@sveltejs/adapter-node)
• LLM Providers: @aws-sdk/client-bedrock-runtime, Anthropic SDK, generic OpenAI-compatible endpoints
• Authentication: OAuth 2.1 with PKCE S256, Redis-backed token storage

Important Notes

Svelte 5 Runes (Mandatory)

All .svelte files use runes mode by default. Use:

• $state() for reactive variables
• $derived for computed values
  
• $effect() for side effects
• {#snippet} for component snippets (e.g., icon slots)

See Svelte 5 runes documentation.

Styling

• Dark mode hardcoded in app.html (remove class="dark" from <html> to disable)
• Use Tailwind utilities directly; no CSS Modules
• Scoped styles available via <style> blocks in .svelte files

Internationalization (i18n)

The project uses Paraglide JS for i18n with:

• Languages: English (en-US) and Spanish (es-MX)
• Auto-localized Routes: URLs automatically prefixed with language code (e.g., /en/ or /es/)
• Runtime API: Use getLocale(), setLocale(), and deLocalizeUrl() from $lib/paraglide/runtime.js
• Messages: Locale-aware functions in $lib/paraglide/messages.js

All routes are automatically localized. No manual language handling needed.

Configuration Reference

The config.json file (copied from config.json.example) controls server and database settings:

Database Configuration

{
  "database": {
    "type": "sqlite3",
    "sqlite3": {
      "fileMustExist": false,
      "filename": "app.db",
      "readonly": false,
      "timeout": 5000,
      "performance": {
        "foreignKeys": true,
        "journalMode": "wal",
        "synchronous": 1,
        "tempStore": "memory"
      }
    }
  },
  "password": {
    "algorithm": "argon2id",
    "options": {
      "memoryCost": 61440,
      "timeCost": 3,
      "parallelism": 1
    }
  }
}

• database.type — Database backend (currently sqlite3)
• database.sqlite3.filename — Path to the SQLite database file
• database.sqlite3.timeout — Query timeout in milliseconds
• database.sqlite3.performance — Performance tuning options (WAL mode, foreign keys, etc.)
• password.algorithm — Password hashing algorithm (currently argon2id)
• password.options — Argon2id configuration (memory cost in KiB, time cost iterations, parallelism level)
• providers — Array of LLM provider configurations (see Provider Setup)

Database Initialization

The database is automatically initialized on first creation with:

• users table — Stores user accounts with email and password (hashed) fields
• Admin user — Created automatically with email admin@delete.me and password deleteme (hashed with argon2id)

Database initialization is idempotent and runs only once when the database file is first created. The admin user is seeded only if no users exist.

Password Security

Passwords are hashed using Argon2id (configured in config.json). Crypto utilities are available in src/lib/crypto.js for hashing and verification.

Links

Framework & Tools

• SvelteKit Documentation
• Flowbite Svelte
• Tailwind CSS
• Paraglide JS
• better-sqlite3
• node-argon2

API & Authentication

• OpenAI API Reference — Chat completions and models endpoints
• AWS Bedrock Converse API — Bedrock provider backend
• Anthropic Messages API — Anthropic provider backend
• OAuth 2.1 Specification — Authorization server implementation
• PKCE RFC 7636 — S256 code challenge method

Local LLM Servers

• Ollama OpenAI Compatibility — Local OpenAI-compatible endpoint
• vLLM OpenAI-Compatible Server — Alternative local server

Project

• Project Instructions
• API Documentation
• OAuth 2.1 Testing Guide