Skip Bedrock credentials for OpenAI-format requests in legacy proxy#865
Conversation
Now that canonical OpenAI models list their Bedrock-served counterparts as fallback_models (e.g. gpt-5.5 -> openai.gpt-5.5, added in #857), getModelEndpointTypes() returns `bedrock` for those models. In the legacy proxy, getApiSecrets() therefore returns an org's Bedrock credential as a candidate for an OpenAI-format request, and fetchModelLoop picks a starting secret at random. When it lands on the Bedrock credential, fetchOpenAI throws `Bedrock does not support OpenAI format`, surfacing as an intermittent hard failure (the error is a ProxyBadRequestError and is rethrown without trying the next secret). Skip a Bedrock credential when the resolved model format is `openai`, mirroring fetchOpenAI's throw condition, so a compatible credential is used instead. This preserves Bedrock eligibility for anthropic/converse-format models (which Bedrock can serve) and leaves the fallback_models data untouched for the gateway. A Bedrock-only org requesting an OpenAI model now gets the clear "No API keys found" error instead of the confusing Bedrock one. Adds a proxyV1-level regression test covering: random-pick no longer routes an OpenAI request to Bedrock, the clean no-keys error, and no over-skipping of anthropic-format models. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ab35178ea2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| secret.type === "bedrock" && | ||
| (modelSpec?.format ?? "openai") === "openai" | ||
| ) { | ||
| continue; |
There was a problem hiding this comment.
When the random order visits a compatible OpenAI secret first and it returns a retryable 429/503, with the only remaining candidate being the Bedrock secret skipped here, the loop never reaches the existing i === secrets.length - 1 rate-limit branch because the last iteration just continues. That makes an OpenAI request return the raw retryable provider failure instead of sleeping/retrying for up to RATE_LIMIT_MAX_WAIT_MS once Bedrock appears in the candidate list; consider filtering incompatible secrets before the loop or basing the exhausted check on actually attempted endpoints.
Useful? React with 👍 / 👎.
2 participants
Now that canonical OpenAI models list their Bedrock-served counterparts as fallback_models (e.g. gpt-5.5 -> openai.gpt-5.5, added in #857), getModelEndpointTypes() returns
bedrockfor those models. In the legacy proxy, getApiSecrets() therefore returns an org's Bedrock credential as a candidate for an OpenAI-format request, and fetchModelLoop picks a starting secret at random. When it lands on the Bedrock credential, fetchOpenAI throwsBedrock does not support OpenAI format, surfacing as an intermittent hard failure (the error is a ProxyBadRequestError and is rethrown without trying the next secret).Skip a Bedrock credential when the resolved model format is
openai, mirroring fetchOpenAI's throw condition, so a compatible credential is used instead. This preserves Bedrock eligibility for anthropic/converse-format models (which Bedrock can serve) and leaves the fallback_models data untouched for the gateway. A Bedrock-only org requesting an OpenAI model now gets the clear "No API keys found" error instead of the confusing Bedrock one.Adds a proxyV1-level regression test covering: random-pick no longer routes an OpenAI request to Bedrock, the clean no-keys error, and no over-skipping of anthropic-format models.