|
|
@@ -1,711 +0,0 @@
|
|
|
-# AWS Bedrock LLM Proxy - Complete Implementation Plan
|
|
|
-**Date**: June 27, 2026 | **Status**: Ready for Development
|
|
|
-
|
|
|
-## 📚 DOCUMENTATION & NAVIGATION
|
|
|
-
|
|
|
-- **📋 Index** → [README.md](README.md)
|
|
|
-- **⚡ Quick Reference** → [QUICK-REFERENCE.md](QUICK-REFERENCE.md)
|
|
|
-- **✅ Checklist** → [implementation-checklist.md](implementation-checklist.md)
|
|
|
-
|
|
|
-## QUICK LINKS
|
|
|
-
|
|
|
-- **"What should I build?"** → [implementation-checklist.md](implementation-checklist.md)
|
|
|
-- **"How does spending work?"** → [Part 5: Spending Enforcement](#part-5-spending-enforcement)
|
|
|
-- **"Database schema?"** → [Part 6: Database Schema](#part-6-database-schema)
|
|
|
-- **"Testing guide?"** → [Part 8: Testing & QA](#part-8-testing--qa)
|
|
|
-
|
|
|
----
|
|
|
-
|
|
|
-## PART 1: ARCHITECTURE OVERVIEW
|
|
|
-
|
|
|
-### System Flow
|
|
|
-```
|
|
|
-Client Request (OpenAI v1 compatible)
|
|
|
- ↓
|
|
|
-[Auth Middleware] → Detect Bearer token type (JWT vs opaque)
|
|
|
- ↓
|
|
|
-[OAuth Validator] OR [Legacy Bearer Validator]
|
|
|
- ↓
|
|
|
-[Spending Check] → Estimate cost, verify budget
|
|
|
- ↓
|
|
|
-[Bedrock InvokeModel] → AWS API call, get tokens
|
|
|
- ↓
|
|
|
-[Atomic Deduction] → PostgreSQL transaction
|
|
|
- ↓
|
|
|
-[OpenAI-formatted Response] → Back to client
|
|
|
-```
|
|
|
-
|
|
|
-### Key Architecture Decisions
|
|
|
-- **Config**: JSON file mounted as Docker secret at `/run/secrets/bedrock-proxy-config.json`
|
|
|
-- **OAuth Discovery**: Auto-fetch from `https://data.titleproject.space/.well-known/oauth-authorization-server` (cached 24h)
|
|
|
-- **Dual Auth**: Support both Bearer tokens (opaque + JWT) on same `/v1/*` endpoints
|
|
|
-- **API**: OpenAI v1 compatible (drop-in replacement for OpenAI clients)
|
|
|
-- **Models**: Native Bedrock model names (e.g., `claude-3-haiku-20240307`)
|
|
|
-
|
|
|
----
|
|
|
-
|
|
|
-## PART 2: CONFIGURATION SYSTEM
|
|
|
-
|
|
|
-### Docker Config File Structure
|
|
|
-**File**: `/run/secrets/bedrock-proxy-config.json`
|
|
|
-
|
|
|
-```json
|
|
|
-{
|
|
|
- "service": {
|
|
|
- "name": "bedrock-llm-proxy",
|
|
|
- "port": 3000,
|
|
|
- "log_level": "info"
|
|
|
- },
|
|
|
- "oauth": {
|
|
|
- "discovery_url": "https://data.titleproject.space/.well-known/oauth-authorization-server",
|
|
|
- "cache_ttl_hours": 24,
|
|
|
- "scopes_required": ["llm", "user"]
|
|
|
- },
|
|
|
- "bedrock": {
|
|
|
- "aws_region": "us-east-1",
|
|
|
- "aws_access_key_id": "AKIA...",
|
|
|
- "aws_secret_access_key": "..."
|
|
|
- },
|
|
|
- "database": {
|
|
|
- "url": "postgresql://user:pass@db:5432/bedrock_proxy",
|
|
|
- "pool_size": 20,
|
|
|
- "statement_timeout_ms": 5000
|
|
|
- },
|
|
|
- "spending": {
|
|
|
- "default_limit_usd": null,
|
|
|
- "monthly_reset_day": 1,
|
|
|
- "reset_hour_utc": 0
|
|
|
- }
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
-### Config Loading at Startup
|
|
|
-```javascript
|
|
|
-// src/lib/config/loader.js
|
|
|
-import fs from 'fs';
|
|
|
-
|
|
|
-export async function loadConfig() {
|
|
|
- const configPath = process.env.CONFIG_PATH || '/run/secrets/bedrock-proxy-config.json';
|
|
|
-
|
|
|
- if (!fs.existsSync(configPath)) {
|
|
|
- throw new Error(`Config file not found: ${configPath}`);
|
|
|
- }
|
|
|
-
|
|
|
- const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
|
|
|
-
|
|
|
- // Fetch OAuth discovery metadata (cached 24h)
|
|
|
- config.oauth.metadata = await fetchOAuthMetadata(config.oauth.discovery_url);
|
|
|
-
|
|
|
- return config;
|
|
|
-}
|
|
|
-
|
|
|
-async function fetchOAuthMetadata(discoveryUrl) {
|
|
|
- const response = await fetch(discoveryUrl);
|
|
|
- const metadata = await response.json();
|
|
|
-
|
|
|
- // Returns: { issuer, token_endpoint, introspection_endpoint, jwks_uri, ... }
|
|
|
- return metadata;
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
----
|
|
|
-
|
|
|
-## PART 3: DUAL AUTH MIDDLEWARE
|
|
|
-
|
|
|
-### Token Type Detection
|
|
|
-```javascript
|
|
|
-// src/lib/auth/middleware.js
|
|
|
-
|
|
|
-function isJwtToken(token) {
|
|
|
- // JWTs have exactly 3 base64url parts separated by dots
|
|
|
- const parts = token.split('.');
|
|
|
- return parts.length === 3 && parts[0] && parts[1] && parts[2];
|
|
|
-}
|
|
|
-
|
|
|
-export async function authMiddleware(request, config) {
|
|
|
- const authHeader = request.headers.get('authorization');
|
|
|
-
|
|
|
- if (!authHeader) {
|
|
|
- throw new Error('Missing Authorization header');
|
|
|
- }
|
|
|
-
|
|
|
- const [scheme, token] = authHeader.split(' ');
|
|
|
- if (scheme.toLowerCase() !== 'bearer') {
|
|
|
- throw new Error('Only Bearer authentication supported');
|
|
|
- }
|
|
|
-
|
|
|
- // Route to appropriate validator
|
|
|
- if (isJwtToken(token)) {
|
|
|
- return await validateOAuthToken(token, config);
|
|
|
- } else {
|
|
|
- return await validateLegacyBearerToken(token, config);
|
|
|
- }
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
-### OAuth Token Validation (JWT)
|
|
|
-```javascript
|
|
|
-import jwt from 'jsonwebtoken';
|
|
|
-import jwksClient from 'jwks-rsa';
|
|
|
-
|
|
|
-async function validateOAuthToken(token, config) {
|
|
|
- try {
|
|
|
- const decoded = jwt.decode(token, { complete: true });
|
|
|
- if (!decoded) throw new Error('Invalid JWT format');
|
|
|
-
|
|
|
- // Fetch signing key from JWKS
|
|
|
- const client = jwksClient({ jwksUri: config.oauth.metadata.jwks_uri });
|
|
|
- const key = await client.getSigningKey(decoded.header.kid);
|
|
|
-
|
|
|
- // Verify signature
|
|
|
- const claims = jwt.verify(token, key.getPublicKey(), {
|
|
|
- algorithms: ['RS256'],
|
|
|
- issuer: config.oauth.metadata.issuer
|
|
|
- });
|
|
|
-
|
|
|
- // Validate scope
|
|
|
- const tokenScopes = (claims.scope || '').split(' ');
|
|
|
- const requiredScopes = config.oauth.scopes_required; // ["llm", "user"]
|
|
|
-
|
|
|
- if (!requiredScopes.every(s => tokenScopes.includes(s))) {
|
|
|
- throw new Error(`Insufficient scope. Required: ${requiredScopes.join(', ')}`);
|
|
|
- }
|
|
|
-
|
|
|
- return {
|
|
|
- auth_type: 'oauth',
|
|
|
- user_id: claims.sub,
|
|
|
- email: claims.email,
|
|
|
- scope: claims.scope,
|
|
|
- expires_at: new Date(claims.exp * 1000)
|
|
|
- };
|
|
|
- } catch (error) {
|
|
|
- throw new Error(`OAuth validation failed: ${error.message}`);
|
|
|
- }
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
-### Legacy Bearer Token Validation
|
|
|
-```javascript
|
|
|
-async function validateLegacyBearerToken(token, config) {
|
|
|
- try {
|
|
|
- // Use OAuth introspection endpoint to validate opaque token
|
|
|
- const introspectionUrl = config.oauth.metadata.introspection_endpoint;
|
|
|
-
|
|
|
- const response = await fetch(introspectionUrl, {
|
|
|
- method: 'POST',
|
|
|
- headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
|
|
|
- body: `token=${encodeURIComponent(token)}`
|
|
|
- });
|
|
|
-
|
|
|
- if (!response.ok) throw new Error('Token introspection failed');
|
|
|
-
|
|
|
- const data = await response.json();
|
|
|
-
|
|
|
- if (!data.active) {
|
|
|
- throw new Error('Token inactive or expired');
|
|
|
- }
|
|
|
-
|
|
|
- return {
|
|
|
- auth_type: 'bearer',
|
|
|
- user_id: data.sub,
|
|
|
- email: data.username,
|
|
|
- scope: data.scope,
|
|
|
- expires_at: new Date(data.exp * 1000)
|
|
|
- };
|
|
|
- } catch (error) {
|
|
|
- throw new Error(`Bearer token validation failed: ${error.message}`);
|
|
|
- }
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
-### SvelteKit Hook Integration
|
|
|
-```javascript
|
|
|
-// src/hooks.server.js
|
|
|
-import { authMiddleware } from '$lib/auth/middleware.js';
|
|
|
-import { loadConfig } from '$lib/config/loader.js';
|
|
|
-
|
|
|
-let config;
|
|
|
-
|
|
|
-export async function init() {
|
|
|
- config = await loadConfig();
|
|
|
-}
|
|
|
-
|
|
|
-export async function handle({ event, resolve }) {
|
|
|
- // Skip auth for public endpoints
|
|
|
- if (!event.url.pathname.startsWith('/api/v1/')) {
|
|
|
- return resolve(event);
|
|
|
- }
|
|
|
-
|
|
|
- try {
|
|
|
- event.locals.user = await authMiddleware(event.request, config);
|
|
|
- } catch (error) {
|
|
|
- return new Response(
|
|
|
- JSON.stringify({
|
|
|
- error: {
|
|
|
- code: 'authentication_error',
|
|
|
- message: error.message,
|
|
|
- type: 'authentication_error'
|
|
|
- }
|
|
|
- }),
|
|
|
- { status: 401, headers: { 'Content-Type': 'application/json' } }
|
|
|
- );
|
|
|
- }
|
|
|
-
|
|
|
- return resolve(event);
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
----
|
|
|
-
|
|
|
-## PART 4: OPENAI v1 API IMPLEMENTATION
|
|
|
-
|
|
|
-### Endpoint: POST /v1/chat/completions
|
|
|
-
|
|
|
-**Request**:
|
|
|
-```json
|
|
|
-{
|
|
|
- "model": "claude-3-haiku-20240307",
|
|
|
- "messages": [
|
|
|
- {"role": "system", "content": "You are helpful."},
|
|
|
- {"role": "user", "content": "Hello"}
|
|
|
- ],
|
|
|
- "temperature": 0.7,
|
|
|
- "max_tokens": 1024,
|
|
|
- "stream": false
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
-**Response**:
|
|
|
-```json
|
|
|
-{
|
|
|
- "id": "chatcmpl-123abc",
|
|
|
- "object": "chat.completion",
|
|
|
- "created": 1704067200,
|
|
|
- "model": "claude-3-haiku-20240307",
|
|
|
- "choices": [
|
|
|
- {
|
|
|
- "index": 0,
|
|
|
- "message": {
|
|
|
- "role": "assistant",
|
|
|
- "content": "Hello! How can I help?"
|
|
|
- },
|
|
|
- "finish_reason": "stop"
|
|
|
- }
|
|
|
- ],
|
|
|
- "usage": {
|
|
|
- "prompt_tokens": 10,
|
|
|
- "completion_tokens": 8,
|
|
|
- "total_tokens": 18
|
|
|
- }
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
-**Implementation**: `src/routes/api/v1/chat/completions/+server.js`
|
|
|
-
|
|
|
-```javascript
|
|
|
-import { json } from '@sveltejs/kit';
|
|
|
-import { invokeBedrockModel } from '$lib/bedrock/client.js';
|
|
|
-import { checkSpendingLimit, deductSpending } from '$lib/spending/enforcer.js';
|
|
|
-import { estimateCost } from '$lib/bedrock/models.js';
|
|
|
-
|
|
|
-export async function POST({ request, locals }) {
|
|
|
- const payload = await request.json();
|
|
|
- const { model, messages, temperature, max_tokens, stream } = payload;
|
|
|
-
|
|
|
- // Validation
|
|
|
- if (!model || !messages) {
|
|
|
- return json({
|
|
|
- error: {
|
|
|
- code: 'invalid_request_error',
|
|
|
- message: 'model and messages are required'
|
|
|
- }
|
|
|
- }, { status: 400 });
|
|
|
- }
|
|
|
-
|
|
|
- const userId = locals.user.user_id;
|
|
|
-
|
|
|
- // Pre-flight spending check
|
|
|
- const costEstimate = estimateCost(model, messages, max_tokens);
|
|
|
- const spendingCheck = await checkSpendingLimit(userId, model, messages);
|
|
|
-
|
|
|
- if (!spendingCheck.allowed) {
|
|
|
- return json({
|
|
|
- error: {
|
|
|
- code: 'insufficient_credits',
|
|
|
- message: `Budget exceeded. Remaining: $${spendingCheck.remaining}`
|
|
|
- }
|
|
|
- }, { status: 429 });
|
|
|
- }
|
|
|
-
|
|
|
- try {
|
|
|
- // Call Bedrock
|
|
|
- const response = await invokeBedrockModel(model, messages, {
|
|
|
- temperature: temperature || 0.7,
|
|
|
- max_tokens: max_tokens || 1024
|
|
|
- });
|
|
|
-
|
|
|
- // Deduct actual cost
|
|
|
- await deductSpending(userId, model, response.input_tokens, response.output_tokens);
|
|
|
-
|
|
|
- // Format OpenAI response
|
|
|
- return json({
|
|
|
- id: `bedrock-${Date.now()}`,
|
|
|
- object: 'chat.completion',
|
|
|
- created: Math.floor(Date.now() / 1000),
|
|
|
- model,
|
|
|
- choices: [{
|
|
|
- index: 0,
|
|
|
- message: {
|
|
|
- role: 'assistant',
|
|
|
- content: response.completion
|
|
|
- },
|
|
|
- finish_reason: 'stop'
|
|
|
- }],
|
|
|
- usage: {
|
|
|
- prompt_tokens: response.input_tokens,
|
|
|
- completion_tokens: response.output_tokens,
|
|
|
- total_tokens: response.input_tokens + response.output_tokens
|
|
|
- }
|
|
|
- });
|
|
|
- } catch (error) {
|
|
|
- if (error.message.includes('Model not found')) {
|
|
|
- return json({
|
|
|
- error: {
|
|
|
- code: 'model_not_found_error',
|
|
|
- message: `Model not supported: ${model}`
|
|
|
- }
|
|
|
- }, { status: 404 });
|
|
|
- }
|
|
|
-
|
|
|
- return json({
|
|
|
- error: {
|
|
|
- code: 'server_error',
|
|
|
- message: error.message,
|
|
|
- type: 'server_error'
|
|
|
- }
|
|
|
- }, { status: 500 });
|
|
|
- }
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
-### Endpoint: GET /v1/models
|
|
|
-
|
|
|
-**Response**:
|
|
|
-```json
|
|
|
-{
|
|
|
- "object": "list",
|
|
|
- "data": [
|
|
|
- {
|
|
|
- "id": "claude-3-haiku-20240307",
|
|
|
- "object": "model",
|
|
|
- "created": 1704067200,
|
|
|
- "owned_by": "anthropic",
|
|
|
- "permission": [],
|
|
|
- "root": "claude-3-haiku-20240307",
|
|
|
- "parent": null
|
|
|
- }
|
|
|
- ]
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
-**Implementation**: `src/routes/api/v1/models/+server.js`
|
|
|
-
|
|
|
-```javascript
|
|
|
-import { json } from '@sveltejs/kit';
|
|
|
-import postgres from 'postgres';
|
|
|
-
|
|
|
-const sql = postgres(process.env.DATABASE_URL);
|
|
|
-
|
|
|
-export async function GET({ locals }) {
|
|
|
- const models = await sql`
|
|
|
- SELECT model_id, provider, created_at FROM bedrock_models WHERE supported = true
|
|
|
- `;
|
|
|
-
|
|
|
- return json({
|
|
|
- object: 'list',
|
|
|
- data: models.map(m => ({
|
|
|
- id: m.model_id,
|
|
|
- object: 'model',
|
|
|
- created: Math.floor(new Date(m.created_at).getTime() / 1000),
|
|
|
- owned_by: m.provider,
|
|
|
- permission: [],
|
|
|
- root: m.model_id,
|
|
|
- parent: null
|
|
|
- }))
|
|
|
- });
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
-### Streaming Endpoint: POST /v1/chat/completions (stream: true)
|
|
|
-
|
|
|
-**Response** (Server-Sent Events):
|
|
|
-```
|
|
|
-event: delta
|
|
|
-data: {"choices":[{"index":0,"delta":{"content":"Hello"}}]}
|
|
|
-
|
|
|
-event: delta
|
|
|
-data: {"choices":[{"index":0,"delta":{"content":" there"}}]}
|
|
|
-
|
|
|
-event: delta
|
|
|
-data: {"choices":[{"index":0,"finish_reason":"stop"}]}
|
|
|
-```
|
|
|
-
|
|
|
-**Implementation**:
|
|
|
-
|
|
|
-```javascript
|
|
|
-// Add to POST handler above:
|
|
|
-
|
|
|
-if (stream) {
|
|
|
- return new Response(streamBedrockResponse(model, messages, userId), {
|
|
|
- headers: {
|
|
|
- 'Content-Type': 'text/event-stream',
|
|
|
- 'Cache-Control': 'no-cache',
|
|
|
- 'Connection': 'keep-alive'
|
|
|
- }
|
|
|
- });
|
|
|
-}
|
|
|
-
|
|
|
-async function* streamBedrockResponse(model, messages, userId) {
|
|
|
- try {
|
|
|
- const response = await invokeBedrockModelStreaming(model, messages);
|
|
|
-
|
|
|
- for await (const delta of response.deltas) {
|
|
|
- yield `event: delta\ndata: ${JSON.stringify({
|
|
|
- choices: [{ index: 0, delta: { content: delta.token } }]
|
|
|
- })}\n\n`;
|
|
|
- }
|
|
|
-
|
|
|
- // Final message with finish_reason
|
|
|
- yield `event: delta\ndata: ${JSON.stringify({
|
|
|
- choices: [{ index: 0, finish_reason: 'stop' }]
|
|
|
- })}\n\n`;
|
|
|
-
|
|
|
- // Deduct actual cost after streaming completes
|
|
|
- await deductSpending(userId, model, response.input_tokens, response.output_tokens);
|
|
|
- } catch (error) {
|
|
|
- yield `event: error\ndata: ${JSON.stringify({
|
|
|
- error: {
|
|
|
- code: 'server_error',
|
|
|
- message: error.message
|
|
|
- }
|
|
|
- })}\n\n`;
|
|
|
- }
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
----
|
|
|
-
|
|
|
-## PART 5: SPENDING ENFORCEMENT
|
|
|
-
|
|
|
-### Pre-Flight Check
|
|
|
-```javascript
|
|
|
-// src/lib/spending/enforcer.js
|
|
|
-import postgres from 'postgres';
|
|
|
-
|
|
|
-export async function checkSpendingLimit(userId, modelId, messages) {
|
|
|
- const sql = postgres(process.env.DATABASE_URL);
|
|
|
-
|
|
|
- // Get user's current balance
|
|
|
- const [user] = await sql`
|
|
|
- SELECT user_id, current_spend_usd, monthly_limit_usd, spend_reset_at
|
|
|
- FROM users
|
|
|
- WHERE user_id = ${userId}
|
|
|
- `;
|
|
|
-
|
|
|
- if (!user) {
|
|
|
- throw new Error('User not found');
|
|
|
- }
|
|
|
-
|
|
|
- // Check if spending period has reset
|
|
|
- if (new Date() > user.spend_reset_at) {
|
|
|
- await sql`
|
|
|
- UPDATE users
|
|
|
- SET current_spend_usd = 0, spend_reset_at = NOW() + INTERVAL '1 month'
|
|
|
- WHERE user_id = ${userId}
|
|
|
- `;
|
|
|
- user.current_spend_usd = 0;
|
|
|
- }
|
|
|
-
|
|
|
- // If no limit set, allow unlimited
|
|
|
- if (user.monthly_limit_usd === null) {
|
|
|
- return { allowed: true, reason: 'unlimited' };
|
|
|
- }
|
|
|
-
|
|
|
- // Estimate tokens for this request
|
|
|
- const messageText = messages.map(m => m.content).join(' ');
|
|
|
- const estimatedTokens = Math.ceil(messageText.length / 4) * 1.2; // 20% buffer
|
|
|
- const estimatedCost = (estimatedTokens * 0.001); // Rough estimate
|
|
|
-
|
|
|
- const remainingBudget = user.monthly_limit_usd - user.current_spend_usd;
|
|
|
-
|
|
|
- if (estimatedCost > remainingBudget) {
|
|
|
- return {
|
|
|
- allowed: false,
|
|
|
- remaining: remainingBudget,
|
|
|
- estimated: estimatedCost
|
|
|
- };
|
|
|
- }
|
|
|
-
|
|
|
- return { allowed: true, estimated_cost: estimatedCost };
|
|
|
-}
|
|
|
-
|
|
|
-export async function deductSpending(userId, modelId, inputTokens, outputTokens) {
|
|
|
- const sql = postgres(process.env.DATABASE_URL);
|
|
|
-
|
|
|
- // Get model pricing
|
|
|
- const [model] = await sql`
|
|
|
- SELECT input_price_usd, output_price_usd FROM bedrock_models WHERE model_id = ${modelId}
|
|
|
- `;
|
|
|
-
|
|
|
- if (!model) throw new Error(`Model ${modelId} not found`);
|
|
|
-
|
|
|
- const actualCost = (
|
|
|
- (inputTokens * model.input_price_usd + outputTokens * model.output_price_usd) / 1_000_000
|
|
|
- );
|
|
|
-
|
|
|
- // Atomic deduction
|
|
|
- await sql`
|
|
|
- BEGIN;
|
|
|
-
|
|
|
- -- Lock row to prevent race conditions
|
|
|
- SELECT user_id FROM users WHERE user_id = ${userId} FOR UPDATE;
|
|
|
-
|
|
|
- -- Deduct cost
|
|
|
- UPDATE users
|
|
|
- SET current_spend_usd = current_spend_usd + ${actualCost}
|
|
|
- WHERE user_id = ${userId};
|
|
|
-
|
|
|
- -- Log transaction
|
|
|
- INSERT INTO spending_records (user_id, model_id, input_tokens, output_tokens, cost_usd)
|
|
|
- VALUES (${userId}, ${modelId}, ${inputTokens}, ${outputTokens}, ${actualCost});
|
|
|
-
|
|
|
- COMMIT;
|
|
|
- `;
|
|
|
-
|
|
|
- return { cost_deducted: actualCost };
|
|
|
-}
|
|
|
-```
|
|
|
-
|
|
|
----
|
|
|
-
|
|
|
-## PART 6: DATABASE SCHEMA
|
|
|
-
|
|
|
-```sql
|
|
|
--- Users table
|
|
|
-CREATE TABLE users (
|
|
|
- user_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
|
- oauth_sub VARCHAR(255) UNIQUE NOT NULL,
|
|
|
- email VARCHAR(255) UNIQUE NOT NULL,
|
|
|
- monthly_limit_usd DECIMAL(10, 2) DEFAULT NULL,
|
|
|
- current_spend_usd DECIMAL(10, 2) DEFAULT 0,
|
|
|
- spend_reset_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP + INTERVAL '1 month',
|
|
|
- created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
- updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
|
-);
|
|
|
-
|
|
|
--- Spending records
|
|
|
-CREATE TABLE spending_records (
|
|
|
- record_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
|
- user_id UUID NOT NULL REFERENCES users(user_id),
|
|
|
- timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
- model_id VARCHAR(64) NOT NULL,
|
|
|
- input_tokens INT NOT NULL,
|
|
|
- output_tokens INT NOT NULL,
|
|
|
- cost_usd DECIMAL(10, 4) NOT NULL
|
|
|
-);
|
|
|
-
|
|
|
--- Bedrock models
|
|
|
-CREATE TABLE bedrock_models (
|
|
|
- model_id VARCHAR(64) PRIMARY KEY,
|
|
|
- provider VARCHAR(32) NOT NULL,
|
|
|
- input_price_usd DECIMAL(10, 6) NOT NULL,
|
|
|
- output_price_usd DECIMAL(10, 6) NOT NULL,
|
|
|
- supported BOOLEAN DEFAULT TRUE,
|
|
|
- created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
|
- updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
|
-);
|
|
|
-
|
|
|
--- Indexes
|
|
|
-CREATE INDEX idx_users_oauth_sub ON users(oauth_sub);
|
|
|
-CREATE INDEX idx_spending_user_timestamp ON spending_records(user_id, timestamp DESC);
|
|
|
-
|
|
|
--- Monthly reset (cron: runs daily at 00:00 UTC)
|
|
|
-CREATE OR REPLACE FUNCTION reset_monthly_spend()
|
|
|
-RETURNS void AS $$
|
|
|
-BEGIN
|
|
|
- UPDATE users
|
|
|
- SET current_spend_usd = 0, spend_reset_at = NOW() + INTERVAL '1 month'
|
|
|
- WHERE spend_reset_at <= NOW();
|
|
|
-END;
|
|
|
-$$ LANGUAGE plpgsql;
|
|
|
-```
|
|
|
-
|
|
|
----
|
|
|
-
|
|
|
-## PART 7: IMPLEMENTATION PHASES
|
|
|
-
|
|
|
-### Phase 1: MVP (Days 1-3)
|
|
|
-- [x] Config loading from Docker secret
|
|
|
-- [x] OAuth discovery endpoint integration
|
|
|
-- [x] Dual auth middleware (JWT + Bearer)
|
|
|
-- [x] Single model (Claude Haiku)
|
|
|
-- [x] Basic spending tracking
|
|
|
-- [x] OpenAI `/v1/chat/completions` endpoint (non-streaming)
|
|
|
-
|
|
|
-**Deliverable**:
|
|
|
-```bash
|
|
|
-curl -X POST http://localhost:3000/api/v1/chat/completions \
|
|
|
- -H "Authorization: Bearer $JWT_TOKEN" \
|
|
|
- -H "Content-Type: application/json" \
|
|
|
- -d '{
|
|
|
- "model": "claude-3-haiku-20240307",
|
|
|
- "messages": [{"role": "user", "content": "Hello"}]
|
|
|
- }' | jq .
|
|
|
-```
|
|
|
-
|
|
|
-### Phase 2: Multi-Model + Dashboard (Days 4-6)
|
|
|
-- [ ] All Bedrock models support
|
|
|
-- [ ] `/v1/models` endpoint
|
|
|
-- [ ] Streaming support (`stream: true`)
|
|
|
-- [ ] User spend dashboard
|
|
|
-- [ ] Admin analytics
|
|
|
-
|
|
|
-### Phase 3: Admin Controls (Days 7-10, Optional)
|
|
|
-- [ ] Admin UI to adjust user limits
|
|
|
-- [ ] Email notifications at 50%/80%/100%
|
|
|
-- [ ] Spending reports (CSV export)
|
|
|
-
|
|
|
----
|
|
|
-
|
|
|
-## PART 8: TESTING & QA
|
|
|
-
|
|
|
-### Phase 1 QA (Curl-based)
|
|
|
-```bash
|
|
|
-# Test 1: Valid OAuth token
|
|
|
-TOKEN=$(curl -s https://title-graphql.example.com/oauth/token | jq -r .access_token)
|
|
|
-curl -X POST http://localhost:3000/api/v1/chat/completions \
|
|
|
- -H "Authorization: Bearer $TOKEN" \
|
|
|
- -d '{"model":"claude-3-haiku-20240307","messages":[{"role":"user","content":"Hi"}]}'
|
|
|
-
|
|
|
-# Test 2: Invalid token
|
|
|
-curl -X POST http://localhost:3000/api/v1/chat/completions \
|
|
|
- -H "Authorization: Bearer invalid_token" \
|
|
|
- -d '{"model":"claude-3-haiku-20240307","messages":[{"role":"user","content":"Hi"}]}'
|
|
|
-# Expected: 401 authentication_error
|
|
|
-
|
|
|
-# Test 3: Spending limit
|
|
|
-curl -X POST http://localhost:3000/api/v1/chat/completions \
|
|
|
- -H "Authorization: Bearer $TOKEN" \
|
|
|
- -d '{"model":"claude-3-haiku-20240307","messages":[{"role":"user","content":"'$(printf 'a%.0s' {1..10000})'"}]}'
|
|
|
-# Expected: 429 insufficient_credits (if limit exceeded)
|
|
|
-```
|
|
|
-
|
|
|
----
|
|
|
-
|
|
|
-## NEXT STEPS
|
|
|
-
|
|
|
-1. **Verify OAuth discovery works**: `curl https://data.titleproject.space/.well-known/oauth-authorization-server`
|
|
|
-2. **Setup PostgreSQL**: Run schema DDL
|
|
|
-3. **Create AWS IAM user**: With `bedrock:InvokeModel` permission
|
|
|
-4. **Mount config file**: `/run/secrets/bedrock-proxy-config.json`
|
|
|
-5. **Start Phase 1 implementation**: Begin with auth middleware
|
