Strona główna Odkrywaj Pomoc
Zarejestruj się Zaloguj się
efren
/
testing-proxy
1
0
Forkuj 0
Pliki Problemy 0 Oczekujące zmiany 0 Wiki
Drzewo: e83d6bd411
Gałęzie Tagi
testing-proxy
/
docs
/
implementation-checklist.md

implementation-checklist.md 3.9 KB
Historia Czysty

AWS Bedrock LLM Proxy - Implementation Checklist

📚 DOCUMENTATION & NAVIGATION

QUICK LINKS

PHASE 1: MVP (Days 1-3)

Day 1: Setup & Config

  • Create /run/secrets/bedrock-proxy-config.json template
  • Implement src/lib/config/loader.js (load config + fetch OAuth discovery)
  • Setup PostgreSQL schema (users, spending_records, bedrock_models tables)
  • Create .env for DATABASE_URL only

Day 1-2: Auth Middleware

  • Implement token type detection (JWT vs opaque Bearer)
  • Implement src/lib/auth/middleware.js:
    • JWT validation: JWKS from OAuth discovery
    • Bearer validation: Token introspection endpoint
  • Integrate into src/hooks.server.js
  • Test with curl: Bearer token → 200 OK, Invalid token → 401

Day 2-3: Bedrock Integration

  • AWS SDK setup (@aws-sdk/client-bedrock-runtime)
  • src/lib/bedrock/client.js: InvokeModel wrapper
  • src/lib/bedrock/models.js: Model registry + pricing
  • Implement POST /api/v1/chat/completions:
    • Pre-flight spending check
    • Bedrock call
    • Atomic cost deduction
  • OpenAI response formatting

Day 3: Testing

  • Manual curl tests:
    • Valid OAuth token → LLM response
    • Invalid token → 401
    • Spending limit → 429
  • Verify Bedrock token counts extracted correctly
  • Verify cost calculation matches pricing

PHASE 2: Multi-Model + Dashboard (Days 4-6)

Day 4: Model Registry

  • Add all Bedrock models to database
  • Implement GET /api/v1/models
  • Per-model pricing configuration

Day 5: Streaming

  • Implement streaming endpoint (SSE)
  • Handle mid-request token counting
  • Cost deduction after streaming completes

Day 6: Dashboard

  • User dashboard: /dashboard
    • Current spend / monthly limit
    • Spend history (last 30 days)
  • Admin analytics: /admin/analytics
    • All users' spend totals
    • Cost trends

PHASE 3: Admin Controls [OPTIONAL]

  • Admin UI to adjust per-user limits
  • Email notifications at spending thresholds
  • Spending reports (CSV export)

KEY TECHNICAL TASKS

Authentication

  • Token type detection (JWT vs opaque)
  • OAuth discovery integration
  • JWKS validation
  • Token introspection
  • Error handling (OpenAI format)

Spending Enforcement

  • Pre-flight cost estimation
  • Atomic deduction with row-level locks
  • Monthly reset trigger
  • Rate limiting per auth type

API Compatibility

  • POST /v1/chat/completions (non-streaming)
  • POST /v1/chat/completions (streaming)
  • GET /v1/models
  • Error responses (OpenAI format)
  • Request/response field mapping

Database

  • Schema design (users, spending_records, bedrock_models)
  • Indexing for performance
  • Monthly reset trigger/cron

QA MATRIX

Test Command Expected Status
OAuth JWT curl + JWT token 200 OK [ ]
Invalid JWT curl + expired JWT 401 auth_error [ ]
Opaque Bearer curl + title-graphql token 200 OK [ ]
Spending Limit 11 requests with $10 limit 429 on 11th [ ]
Cost Deduction Check database $X.XX deducted [ ]
Model List curl /v1/models JSON array [ ]
Streaming stream: true SSE format [ ]

CONFIG REQUIREMENTS

Before starting:

  1. AWS IAM user with bedrock:InvokeModel permission
  2. PostgreSQL database accessible
  3. OAuth discovery URL verified: https://data.titleproject.space/.well-known/oauth-authorization-server
  4. title-graphql introspection endpoint working