testing-proxy/docs/QUICK-REFERENCE.md

AWS Bedrock LLM Proxy - Quick Reference

🎯 WHAT YOU'RE BUILDING

An OpenAI v1 API-compatible LLM proxy that routes requests to AWS Bedrock, enforces per-user monthly spending limits, and uses OAuth 2.1 + optional legacy Bearer tokens for authentication.

🏗️ ARCHITECTURE AT A GLANCE

Client (OpenAI SDK)
  ↓
[POST /v1/chat/completions with Bearer token]
  ↓
[Auth Middleware] → Detect token type (JWT vs opaque)
  ↓
[OAuth Validator OR Introspection]
  ↓
[Spending Check] → Estimate cost, verify budget
  ↓
[AWS Bedrock InvokeModel]
  ↓
[Atomic Deduction] → PostgreSQL transaction
  ↓
[OpenAI-formatted Response]

⚙️ CONFIGURATION

Config File: /run/secrets/bedrock-proxy-config.json (mounted as Docker secret)

{
  "oauth": {
    "discovery_url": "https://data.titleproject.space/.well-known/oauth-authorization-server"
  },
  "bedrock": {
    "aws_region": "us-east-1",
    "aws_access_key_id": "...",
    "aws_secret_access_key": "..."
  },
  "database": {
    "url": "postgresql://user:pass@db:5432/bedrock_proxy"
  },
  "spending": {
    "default_limit_usd": null
  }
}

That's it! OAuth metadata auto-fetches from discovery URL. Zero manual configuration.

🔐 DUAL AUTH (Both supported on same endpoint)

Option 1: OAuth 2.1 JWT (Recommended)

curl -X POST http://localhost:3000/api/v1/chat/completions \
  -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIs..." \
  -d '{"model":"claude-3-haiku-20240307","messages":[...]}'

Option 2: Legacy Bearer (Opaque Token)

curl -X POST http://localhost:3000/api/v1/chat/completions \
  -H "Authorization: Bearer abc123def456ghi..." \
  -d '{"model":"claude-3-haiku-20240307","messages":[...]}'

Detection: Automatic. If token has 3 dots → JWT. Otherwise → opaque Bearer.

📡 OPENAI API ENDPOINTS

POST /v1/chat/completions

{
  "model": "claude-3-haiku-20240307",
  "messages": [
    {"role": "user", "content": "Hello"}
  ],
  "stream": false
}

GET /v1/models

Returns list of supported Bedrock models with native names.

💰 SPENDING LIMITS

• Default: Unlimited (no cap)
• Per-user: Admin can set custom cap (e.g., $50/month)
• Monthly: Resets 1st of month, midnight UTC
• Enforcement:
  • Pre-flight check: Estimate cost, allow if budget remains
  • Post-flight deduction: Actual tokens deducted after Bedrock call
  • 1-request overage: User can exceed limit by ~1 request (~$1-5 max)

📊 DATABASE SCHEMA

-- 3 core tables
users (user_id, email, monthly_limit_usd, current_spend_usd, spend_reset_at)
spending_records (user_id, model_id, input_tokens, output_tokens, cost_usd)
bedrock_models (model_id, provider, input_price_usd, output_price_usd)

🚀 PHASE 1 DELIVERABLE (3 days)

Working curl endpoint:

curl -X POST http://localhost:3000/api/v1/chat/completions \
  -H "Authorization: Bearer <JWT_or_Bearer_token>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-3-haiku-20240307",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

# Response
{
  "id": "bedrock-1704067200000",
  "object": "chat.completion",
  "model": "claude-3-haiku-20240307",
  "choices": [{
    "message": {"content": "Hello! How can I help?"},
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 8,
    "total_tokens": 18
  }
}

📋 KEY FILES

File	Purpose
src/lib/config/loader.js	Load config, fetch OAuth discovery
src/lib/auth/middleware.js	Token detection + validation
src/hooks.server.js	SvelteKit auth integration
src/lib/bedrock/client.js	AWS SDK wrapper
src/lib/spending/enforcer.js	Cost estimation + atomic deduction
src/routes/api/v1/chat/completions/+server.js	Main endpoint
src/routes/api/v1/models/+server.js	Model list endpoint

✅ PRE-FLIGHT CHECKLIST

Before coding:

• AWS IAM user created (bedrock:InvokeModel permission)
• PostgreSQL database accessible
• OAuth discovery URL working: curl https://data.titleproject.space/.well-known/oauth-authorization-server
• title-graphql introspection endpoint: GET /oauth/introspect

🎯 PHASE 1 SUCCESS CRITERIA

• JWT and opaque Bearer tokens both work
• Spending deducted correctly from database
• OpenAI response format exact match
• Curl tests pass (no UI needed in Phase 1)

📝 ERROR RESPONSES (All in OpenAI format)

{
  "error": {
    "code": "authentication_error",
    "message": "Invalid or expired token",
    "type": "authentication_error"
  }
}

Common codes:

• authentication_error (401): Bad/missing token
• invalid_request_error (400): Bad parameters
• insufficient_credits (429): Budget exceeded
• model_not_found_error (404): Unsupported model
• server_error (500): Bedrock error

🔗 RELATED REPOS

• title-graphql: OAuth service (introspection endpoint)
• testing-proxy: SvelteKit host project (this repo)

📚 DOCUMENTATION & NAVIGATION

• 📋 Index → README.md
• 📝 Full Plan → bedrock-proxy-implementation-plan.md
• ✅ Checklist → implementation-checklist.md

QUICK LINKS

• "What should I build?" → implementation-checklist.md
• "How does auth work?" → bedrock-proxy-implementation-plan.md
• "Show me the code" → bedrock-proxy-implementation-plan.md