|
@@ -0,0 +1,208 @@
|
|
|
|
|
+# v1-api-multi-provider - Work Plan
|
|
|
|
|
+
|
|
|
|
|
+## TL;DR (For humans)
|
|
|
|
|
+
|
|
|
|
|
+**What you'll get:** An OpenAI-compatible `/v1` API that proxies chat completion requests to **three provider backends** — Amazon Bedrock, Anthropic (direct API), and any generic OpenAI-compatible endpoint (vLLM, Ollama, LiteLLM, etc.). All endpoints are protected by your existing OAuth 2.1 Bearer tokens with `llm` scope enforcement. The architecture uses a clean **provider abstraction**: model IDs are prefixed (`bedrock/claude-3-sonnet`, `anthropic/claude-3-5-sonnet`, `local/llama3.1`), and each provider handles its own request/response translation. The minimum viable implementation covers `POST /v1/chat/completions` (streaming + non-streaming) and `GET /v1/models`.
|
|
|
|
|
+
|
|
|
|
|
+**Why this approach:** Instead of hardcoding Bedrock-specific translation throughout the API layer, we abstract providers behind a unified interface (`chat(messages, model, stream) → { response | stream }`). This makes adding a 4th provider literally a matter of editing `config.json` — zero code changes. The Anthropic and generic OpenAI-compatible providers use native `fetch()`, keeping dependencies minimal. AWS Bedrock uses the official SDK for robustness.
|
|
|
|
|
+
|
|
|
|
|
+**What it will NOT do:** It will not support legacy `/v1/completions`, embeddings, image generation, audio, files, or assistants. It will not auto-discover models from providers — all model mappings are explicit in `config.json`. It will not add CORS, JWT tokens, or client_credentials grants. It will not log message content.
|
|
|
|
|
+
|
|
|
|
|
+**Effort:** Medium-High (~13 todos, 4 waves)
|
|
|
|
|
+**Risk:** Medium — Three different API shapes to translate; SSE streaming must exactly match OpenAI clients' expectations across all providers
|
|
|
|
|
+**Decisions to sanity-check:** Provider abstraction design, model ID prefix convention, config.json structure.
|
|
|
|
|
+
|
|
|
|
|
+Your next move: Approve this plan, then start work with `/start-work`.
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+> TL;DR (machine): Medium-High effort, Medium risk — OpenAI-compatible /v1 API proxy to Amazon Bedrock + Anthropic + generic OpenAI-compatible providers, OAuth 2.1 Bearer token validation against Redis, llm scope enforcement, SSE streaming support, provider abstraction architecture, SvelteKit +server.js handlers.
|
|
|
|
|
+
|
|
|
|
|
+## Scope
|
|
|
|
|
+### Must have
|
|
|
|
|
+- Bearer token validation helper (`src/lib/oauth/bearer.js`) — parses `Authorization: Bearer <token>`, looks up in Redis, checks expiry, returns token metadata
|
|
|
|
|
+- `llm` scope enforcement on all `/v1/*` endpoints — returns 403 if scope missing
|
|
|
|
|
+- OpenAI-compatible error format on all `/v1/*` endpoints — `{ error: { message, type, param, code } }`
|
|
|
|
|
+- Provider abstraction router (`src/lib/providers/index.js`) — picks provider by model ID prefix, unified interface for all backends
|
|
|
|
|
+- **Bedrock provider** (`src/lib/providers/bedrock.js`) — AWS SDK, Converse/ConverseStream API, request/response translation
|
|
|
|
|
+- **Anthropic provider** (`src/lib/providers/anthropic.js`) — native `fetch()`, Messages API, request/response translation
|
|
|
|
|
+- **Generic OpenAI-compatible provider** (`src/lib/providers/openai-compatible.js`) — native `fetch()`, almost pure proxy, minimal translation
|
|
|
|
|
+- `POST /v1/chat/completions` endpoint — accepts OpenAI request format, routes to provider by model prefix, returns OpenAI response format
|
|
|
|
|
+- Streaming support for `/v1/chat/completions` — SSE `data: {...}` chunks with `[DONE]` terminator, matching OpenAI format across all providers
|
|
|
|
|
+- `GET /v1/models` endpoint — aggregates model lists from all configured providers
|
|
|
|
|
+- `GET /v1/models/{model}` endpoint — returns details for a specific model
|
|
|
|
|
+- Config.json updates: `providers` array with `id`, `type`, `credentials`, `models` mapping per provider
|
|
|
|
|
+- `@aws-sdk/client-bedrock-runtime` dependency
|
|
|
|
|
+- Model request/response translation layers per provider:
|
|
|
|
|
+ - OpenAI `messages[]` ↔ provider-native message format
|
|
|
|
|
+ - OpenAI `usage` ↔ provider-native usage/tokens
|
|
|
|
|
+ - OpenAI `choices[]` ↔ provider-native response content
|
|
|
|
|
+- Provider health check / error propagation — Bedrock/AWS errors → OpenAI error format, Anthropic errors → OpenAI error format
|
|
|
|
|
+
|
|
|
|
|
+### Must NOT have (guardrails, anti-slop, scope boundaries)
|
|
|
|
|
+- Legacy `/v1/completions` endpoint (non-chat) — all providers are chat-native
|
|
|
|
|
+- `/v1/embeddings` — out of scope, no provider embedding config
|
|
|
|
|
+- `/v1/images/*`, `/v1/audio/*`, `/v1/files/*`, `/v1/assistants/*` — no provider equivalent
|
|
|
|
|
+- JWT access token support — existing opaque tokens only
|
|
|
|
|
+- CORS headers on `/v1/*` — not needed for server-to-server client calls
|
|
|
|
|
+- Logging of message content or prompts — PII guardrail
|
|
|
|
|
+- AWS credentials or API keys hardcoded in source files — only in config.json
|
|
|
|
|
+- Provider auto-discovery or model listing from provider APIs — static config only
|
|
|
|
|
+- Request/response transformation caching — stateless per-request translation only
|
|
|
|
|
+- Custom inference parameters beyond `temperature`, `max_tokens`, `top_p`, `stop` — keep translation minimal
|
|
|
|
|
+- Provider fallback / retry logic — single provider per request, no failover
|
|
|
|
|
+
|
|
|
|
|
+## Verification strategy
|
|
|
|
|
+> Zero human intervention - all verification is agent-executed.
|
|
|
|
|
+- Test decision: tests-after + agent-executed QA via curl/httpie against running dev server
|
|
|
|
|
+- Evidence: `.omo/evidence/task-<N>-v1-api-multi-provider.<ext>`
|
|
|
|
|
+
|
|
|
|
|
+## Execution strategy
|
|
|
|
|
+### Parallel execution waves
|
|
|
|
|
+> Target 5-8 todos per wave.
|
|
|
|
|
+
|
|
|
|
|
+**Wave 1: Foundation** — Config, dependency, bearer helper, provider abstraction skeleton
|
|
|
|
|
+**Wave 2: Provider Implementations** — Bedrock, Anthropic, generic OpenAI-compatible providers
|
|
|
|
|
+**Wave 3: API Endpoints** — Chat completions (non-streaming), models list/retrieve
|
|
|
|
|
+**Wave 4: Streaming & Final QA** — SSE streaming, full integration testing, docs
|
|
|
|
|
+
|
|
|
|
|
+### Dependency matrix
|
|
|
|
|
+| Todo | Depends on | Blocks | Can parallelize with |
|
|
|
|
|
+| --- | --- | --- | --- |
|
|
|
|
|
+| 1 | — | 2,3,4,5 | — |
|
|
|
|
|
+| 2 | 1 | 6,7,8,9 | — |
|
|
|
|
|
+| 3 | 1 | 6,7,8,9 | 2 |
|
|
|
|
|
+| 4 | 1 | 10,11 | 2,3 |
|
|
|
|
|
+| 5 | 1 | 10,11 | 2,3,4 |
|
|
|
|
|
+| 6 | 2,3,4 | 10,11 | 7,8,9 |
|
|
|
|
|
+| 7 | 2,3,4 | 10,11 | 6,8,9 |
|
|
|
|
|
+| 8 | 2,3,4 | 10,11 | 6,7,9 |
|
|
|
|
|
+| 9 | 2,3,4 | 10,11 | 6,7,8 |
|
|
|
|
|
+| 10 | 6,7,8,9 | 12 | — |
|
|
|
|
|
+| 11 | 6,7,8,9 | 12 | 10 |
|
|
|
|
|
+| 12 | 10,11 | 13 | — |
|
|
|
|
|
+| 13 | 12 | — | — |
|
|
|
|
|
+
|
|
|
|
|
+## Todos
|
|
|
|
|
+
|
|
|
|
|
+- [x] 1. Update config.json.example with multi-provider settings
|
|
|
|
|
+ What to do / Must NOT do: Replace existing `aws` section (if any) with `providers` array. Each provider object has: `id` (string, e.g. "bedrock", "anthropic", "local"), `type` ("bedrock" | "anthropic" | "openai-compatible"), `enabled` (boolean), and provider-specific config. For Bedrock: `region`, `credentials` { `accessKeyId`, `secretAccessKey` }. For Anthropic: `apiKey`, `baseUrl` (optional, defaults to https://api.anthropic.com/v1). For generic: `baseUrl`. All providers have `models` object mapping OpenAI model IDs to provider model IDs. Must NOT include real credentials in example — use placeholders. Must NOT remove existing baseUrl/database/password sections.
|
|
|
|
|
+ Parallelization: Wave 1 | Blocked by: — | Blocks: 2,3,4,5
|
|
|
|
|
+ References: `config.json.example:1-32`, `src/lib/config.js:1-19`
|
|
|
|
|
+ Acceptance criteria: `cat config.json.example | grep -q providers && cat config.json.example | grep -q bedrock && cat config.json.example | grep -q anthropic`
|
|
|
|
|
+ QA scenarios: Happy: valid JSON parses. Failure: invalid JSON breaks parse.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 2. Install AWS Bedrock Runtime SDK dependency
|
|
|
|
|
+ What to do / Must NOT do: `pnpm add @aws-sdk/client-bedrock-runtime`. Must NOT install full `@aws-sdk/client-bedrock` or unrelated AWS packages. Must ensure package is recorded in package.json.
|
|
|
|
|
+ Parallelization: Wave 1 | Blocked by: 1 | Blocks: 6,7,8,9
|
|
|
|
|
+ References: `package.json`, pnpm-workspace.yaml (native builds)
|
|
|
|
|
+ Acceptance criteria: `node -e "import('@aws-sdk/client-bedrock-runtime').then(m => console.log(typeof m.BedrockRuntimeClient))"` prints "function"
|
|
|
|
|
+ QA scenarios: Happy: SDK imports successfully. Failure: missing dependency throws MODULE_NOT_FOUND.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 3. Create Bearer token validation helper
|
|
|
|
|
+ What to do / Must NOT do: Create `src/lib/oauth/bearer.js` exporting `validateBearerToken(request, requiredScope)` which: (a) extracts `Authorization: Bearer <token>` header, (b) looks up `access_token:{token}` in Redis, (c) parses JSON metadata, (d) checks if expired, (e) checks if `scope` contains `requiredScope`, (f) returns `{ valid: true, clientId, userId, scope }` or `{ valid: false, error, status }`. Must NOT accept tokens from query strings. Must NOT accept JWT tokens differently from opaque tokens. Must use constant-time comparison for token prefix if any.
|
|
|
|
|
+ Parallelization: Wave 1 | Blocked by: 1 | Blocks: 10,11
|
|
|
|
|
+ References: `src/lib/redis.js:1-28`, `src/routes/oauth/token/+server.js:137-158` (token storage format), RFC 6750
|
|
|
|
|
+ Acceptance criteria: `node -e "import('./src/lib/oauth/bearer.js').then(m => console.log(typeof m.validateBearerToken))"` prints "function"
|
|
|
|
|
+ QA scenarios: Happy: Valid active token with correct scope returns valid=true. Failure: Missing header returns 401. Invalid token returns 401. Expired token returns 401. Wrong scope returns 403.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 4. Create provider abstraction router
|
|
|
|
|
+ What to do / Must NOT do: Create `src/lib/providers/index.js` exporting: `getProvider(modelId)` which parses `providerSlug/localModelId` prefix (e.g. "bedrock/claude-3-sonnet" → bedrock provider, "anthropic/claude-3-5-sonnet" → anthropic provider), `listProviders()` returning enabled providers, `listModels()` aggregating all provider model mappings. Must validate that model ID contains exactly one `/` separator. Must return clear error for unknown provider or unknown model. Must NOT instantiate providers on every request — cache enabled providers.
|
|
|
|
|
+ Parallelization: Wave 1 | Blocked by: 1 | Blocks: 10,11
|
|
|
|
|
+ References: `src/lib/config.js:1-19`, `config.json` provider structure
|
|
|
|
|
+ Acceptance criteria: `node -e "import('./src/lib/providers/index.js').then(m => m.getProvider('bedrock/test').then(p => console.log(p.id)))"` prints "bedrock"
|
|
|
|
|
+ QA scenarios: Happy: Valid model ID resolves to correct provider. Failure: Missing prefix returns clear error. Unknown provider returns clear error. Disabled provider returns clear error.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 5. Create generic OpenAI-compatible provider
|
|
|
|
|
+ What to do / Must NOT do: Create `src/lib/providers/openai-compatible.js` implementing the provider interface: `chat(messages, modelId, options)` and `chatStream(messages, modelId, options)` using native `fetch()` to provider's `baseUrl`. Request body is nearly pass-through OpenAI format. Response is nearly pass-through OpenAI format — minimal translation. Must handle `baseUrl` with or without trailing `/v1`. Must pass `Authorization: Bearer <apiKey>` if configured. Must set `Content-Type: application/json`. Must handle SSE streaming response. Must NOT modify message content. Must NOT log request body.
|
|
|
|
|
+ Parallelization: Wave 1 | Blocked by: 1 | Blocks: 10,11
|
|
|
|
|
+ References: `src/lib/providers/index.js`, OpenAI chat completions spec, native fetch() API
|
|
|
|
|
+ Acceptance criteria: `node -e "import('./src/lib/providers/openai-compatible.js').then(m => console.log(typeof m.chat))"` prints "function"
|
|
|
|
|
+ QA scenarios: Happy: Provider initializes with valid config. Failure: Invalid baseUrl throws clear error. Missing API key (if required) throws clear error.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 6. Create Bedrock provider module
|
|
|
|
|
+ What to do / Must NOT do: Create `src/lib/providers/bedrock.js` implementing provider interface: `chat(messages, modelId, options)` and `chatStream(messages, modelId, options)`. Use `@aws-sdk/client-bedrock-runtime` with `ConverseCommand` / `ConverseStreamCommand`. Translate OpenAI `messages[]` to Bedrock `messages[]` with `content` blocks (text only). Translate Bedrock response to OpenAI format: `id` (generated), `object: "chat.completion"`, `created` (unix timestamp), `model` (OpenAI model ID), `choices[].message.role/content`, `usage.prompt_tokens/completion_tokens/total_tokens`. For streaming: translate `ConverseStream` events to OpenAI `chat.completion.chunk` SSE format. Must NOT create AWS client on every request — use singleton per provider config. Must handle AWS region and credentials from config. Must NOT expose AWS credentials in error messages.
|
|
|
|
|
+ Parallelization: Wave 2 | Blocked by: 2,3,4 | Blocks: 10,11
|
|
|
|
|
+ References: `src/lib/providers/index.js`, `@aws-sdk/client-bedrock-runtime` docs, Bedrock Converse API docs, `config.json`
|
|
|
|
|
+ Acceptance criteria: `node -e "import('./src/lib/providers/bedrock.js').then(m => console.log(typeof m.chat))"` prints "function"
|
|
|
|
|
+ QA scenarios: Happy: Client initializes with valid config. Failure: Invalid AWS credentials throws clear error. Missing model mapping throws clear error. Invalid region throws clear error.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 7. Create Anthropic provider module
|
|
|
|
|
+ What to do / Must NOT do: Create `src/lib/providers/anthropic.js` implementing provider interface: `chat(messages, modelId, options)` and `chatStream(messages, modelId, options)` using native `fetch()` to `https://api.anthropic.com/v1/messages` (or configured `baseUrl`). Translate OpenAI request to Anthropic Messages API format: `model` → provider model ID, `messages[]` → `messages[]` (map `role: system` to top-level `system` param), `max_tokens` required (default to 4096 if missing), `temperature` and `top_p` passed through, `stream` boolean. Translate Anthropic response to OpenAI format: `content[].text` → `choices[].message.content`, `usage.input_tokens/output_tokens` → `usage.prompt_tokens/completion_tokens`. For streaming: read SSE from Anthropic, translate `content_block_delta` events to OpenAI `chat.completion.chunk` format. Must pass `x-api-key` and `anthropic-version: 2023-06-01` headers. Must NOT log request body. Must handle Anthropic-specific errors (e.g. `overloaded_error`).
|
|
|
|
|
+ Parallelization: Wave 2 | Blocked by: 2,3,4 | Blocks: 10,11
|
|
|
|
|
+ References: `src/lib/providers/index.js`, Anthropic Messages API docs, native fetch() API
|
|
|
|
|
+ Acceptance criteria: `node -e "import('./src/lib/providers/anthropic.js').then(m => console.log(typeof m.chat))"` prints "function"
|
|
|
|
|
+ QA scenarios: Happy: Provider initializes with valid config. Failure: Invalid API key throws clear error. Missing model mapping throws clear error. Malformed response handled gracefully.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 8. Implement POST /v1/chat/completions (non-streaming)
|
|
|
|
|
+ What to do / Must NOT do: Create `src/routes/v1/chat/completions/+server.js` with POST handler. Steps: (1) validate Bearer token + llm scope via bearer helper, (2) parse JSON body, (3) resolve provider from model ID prefix via `getProvider(model)`, (4) call provider.chat() with translated params, (5) return OpenAI-format JSON response. Must NOT support `stream: true` in this todo — handled in todo 12. Must NOT log message content. Must return 400 for unsupported parameters with OpenAI error format. Must handle provider errors and translate to OpenAI error format (502 for provider failure).
|
|
|
|
|
+ Parallelization: Wave 3 | Blocked by: 3,4,5,6,7 | Blocks: 12
|
|
|
|
|
+ References: `src/lib/oauth/bearer.js`, `src/lib/providers/index.js`, all provider modules, OpenAI chat completions spec
|
|
|
|
|
+ Acceptance criteria: `curl -s -X POST http://localhost:3000/v1/chat/completions -H "Authorization: Bearer VALID_TOKEN" -H "Content-Type: application/json" -d '{"model":"bedrock/claude-3-sonnet","messages":[{"role":"user","content":"Hello"}]}' | jq '.object'` returns "chat.completion"
|
|
|
|
|
+ QA scenarios: Happy: Valid token + valid request returns OpenAI-format response with choices and usage. Failure: Missing auth returns 401. Wrong scope returns 403. Invalid model returns 400 (unknown provider prefix or unknown model). Bad request body returns 400. Provider error returns 502 with OpenAI error format.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 9. Implement GET /v1/models and GET /v1/models/{model}
|
|
|
|
|
+ What to do / Must NOT do: Create `src/routes/v1/models/+server.js` with GET handler returning `{ object: "list", data: [...] }` where each model has `id` (full prefixed ID), `object: "model"`, `created` (static or current timestamp), `owned_by: "<provider-id>"`. Create `src/routes/v1/models/[model]/+server.js` with GET handler returning single model object. Both must validate Bearer token + llm scope. Data sourced from all enabled providers' `models` config. Must NOT query provider APIs for model list — static config only. Must return 404 for unknown model IDs.
|
|
|
|
|
+ Parallelization: Wave 3 | Blocked by: 3,4,5,6,7 | Blocks: 12
|
|
|
|
|
+ References: `src/lib/oauth/bearer.js`, `src/lib/providers/index.js`, OpenAI models spec
|
|
|
|
|
+ Acceptance criteria: `curl -s http://localhost:3000/v1/models -H "Authorization: Bearer VALID_TOKEN" | jq '.object'` returns "list"
|
|
|
|
|
+ QA scenarios: Happy: Valid token returns aggregated model list from all providers. Valid model ID returns model details. Failure: Missing auth returns 401. Unknown model returns 404.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 10. Add SSE streaming support to /v1/chat/completions
|
|
|
|
|
+ What to do / Must NOT do: Extend existing `+server.js` from todo 8 to handle `stream: true` in request body. Resolve provider, call `provider.chatStream()`, read async iterator, translate each chunk to OpenAI `chat.completion.chunk` format, send as SSE with `data: {...}` prefix, end with `data: [DONE]`. Must set `Content-Type: text/event-stream` and `Cache-Control: no-cache`. Must handle stream errors gracefully — send final error chunk or close connection. Must NOT buffer entire stream before sending. All three providers must produce identical SSE output format.
|
|
|
|
|
+ Parallelization: Wave 4 | Blocked by: 8,9 | Blocks: 13
|
|
|
|
|
+ References: `src/routes/v1/chat/completions/+server.js`, all provider modules, OpenAI streaming spec
|
|
|
|
|
+ Acceptance criteria: `curl -s -N -X POST http://localhost:3000/v1/chat/completions -H "Authorization: Bearer VALID_TOKEN" -H "Content-Type: application/json" -d '{"model":"bedrock/claude-3-sonnet","messages":[{"role":"user","content":"Hello"}],"stream":true}' | grep -q 'data: '`
|
|
|
|
|
+ QA scenarios: Happy: Streaming request returns SSE chunks with valid JSON deltas and [DONE]. Failure: Stream error closes connection gracefully. Invalid stream param returns 400.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 11. Add provider error translation to OpenAI error format
|
|
|
|
|
+ What to do / Must NOT do: Create `src/lib/providers/errors.js` exporting `translateProviderError(error, providerType)` which normalizes provider-specific errors into OpenAI error format: `{ error: { message, type, param, code } }`. Handle AWS SDK errors (e.g. `ThrottlingException`, `ValidationException`), Anthropic errors (e.g. `overloaded_error`, `invalid_request_error`), and HTTP errors from generic providers. Must NOT expose provider credentials or internal URLs in error messages. Must map provider error codes to reasonable OpenAI equivalents.
|
|
|
|
|
+ Parallelization: Wave 3 | Blocked by: 6,7 | Blocks: 8,9,10
|
|
|
|
|
+ References: All provider modules, OpenAI error format spec
|
|
|
|
|
+ Acceptance criteria: `node -e "import('./src/lib/providers/errors.js').then(m => console.log(typeof m.translateProviderError))"` prints "function"
|
|
|
|
|
+ QA scenarios: Happy: AWS throttling returns 429 with OpenAI error format. Anthropic overload returns 503 with OpenAI error format. Generic 401 returns 502 with OpenAI error format.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+- [x] 12. Final integration QA and documentation
|
|
|
|
|
+ What to do / Must NOT do: Run dev server. Execute full flow for each provider: obtain OAuth token → call `/v1/models` → call `/v1/chat/completions` (non-streaming) → call `/v1/chat/completions` (streaming) → verify error formats. Update `docs/` with `/v1` API documentation including provider setup and model ID examples. Must NOT skip negative testing (invalid token, wrong scope, missing auth, unknown provider, unknown model). Must verify OpenAI Python SDK can connect.
|
|
|
|
|
+ Parallelization: Wave 4 | Blocked by: 10,11 | Blocks: —
|
|
|
|
|
+ References: All `/v1` endpoint files, existing OAuth endpoints, all provider modules, OpenAI Python SDK
|
|
|
|
|
+ Acceptance criteria: Full end-to-end flow executes successfully for at least one configured provider (token issuance to streaming chat completion).
|
|
|
|
|
+ QA scenarios: Happy: Complete flow with valid token for each provider type. Failure: Invalid token returns 401 on all /v1 endpoints. Missing llm scope returns 403. Unknown provider prefix returns 400. Unknown model returns 404. Provider error returns 502 with OpenAI error format.
|
|
|
|
|
+ No commit — user handles all commits manually
|
|
|
|
|
+
|
|
|
|
|
+## Final verification wave
|
|
|
|
|
+> Runs in parallel after ALL todos. ALL must APPROVE.
|
|
|
|
|
+- [x] F1. Plan compliance audit — Verify every must-have is implemented and every must-not-have is absent
|
|
|
|
|
+- [x] F2. Code quality review — Verify consistent patterns across all providers, no hardcoded credentials, no PII logging
|
|
|
|
|
+- [x] F3. Real manual QA — Execute full OAuth → /v1 flow end-to-end with curl for at least one provider
|
|
|
|
|
+- [x] F4. Scope fidelity — Confirm no /v1/embeddings, no /v1/completions, no CORS, no JWT support, no PII logging, no provider auto-discovery
|
|
|
|
|
+
|
|
|
|
|
+## Commit strategy
|
|
|
|
|
+- **NO COMMITS** — user handles all commits manually when ready
|
|
|
|
|
+- Worker makes file changes only; git commit decisions belong to user
|
|
|
|
|
+- User may commit everything at once, or per-todo, or not at all — their call
|
|
|
|
|
+
|
|
|
|
|
+## Success criteria
|
|
|
|
|
+1. `POST /v1/chat/completions` accepts OpenAI-format requests with prefixed model IDs (e.g. `bedrock/claude-3-sonnet`) and returns OpenAI-format responses (non-streaming)
|
|
|
|
|
+2. `POST /v1/chat/completions` with `stream: true` returns SSE with `chat.completion.chunk` format and `[DONE]` terminator
|
|
|
|
|
+3. `GET /v1/models` returns aggregated OpenAI-format model list from all configured providers
|
|
|
|
|
+4. `GET /v1/models/{model}` returns OpenAI-format model object or 404
|
|
|
|
|
+5. All `/v1/*` endpoints require valid Bearer token from Redis OAuth flow
|
|
|
|
|
+6. All `/v1/*` endpoints require `llm` scope — tokens without it return 403
|
|
|
|
|
+7. Invalid/missing Bearer tokens return 401 with OpenAI error format
|
|
|
|
|
+8. Provider errors (AWS, Anthropic, generic) return 502 with OpenAI error format
|
|
|
|
|
+9. Credentials and model mappings live only in config.json (not source code)
|
|
|
|
|
+10. No message content or prompts logged anywhere
|
|
|
|
|
+11. Full end-to-end flow works: OAuth login → authorize → token → /v1/chat/completions
|
|
|
|
|
+12. Adding a new provider requires only config.json changes (no code changes)
|