Skip to content

fix: propagate anthropic-beta header and preserve cache_control.scope#1592

Open
kamalkalwa wants to merge 2 commits intoPortkey-AI:mainfrom
kamalkalwa:fix/anthropic-cache-control-scope
Open

fix: propagate anthropic-beta header and preserve cache_control.scope#1592
kamalkalwa wants to merge 2 commits intoPortkey-AI:mainfrom
kamalkalwa:fix/anthropic-cache-control-scope

Conversation

@kamalkalwa
Copy link
Copy Markdown

Problem

When using targets-based configs (e.g., fallback strategies), requests to Anthropic's /v1/messages with cache_control: { type: 'ephemeral', scope: 'global' } fail with:

anthropic error: system.1.cache_control.ephemeral.scope: Extra inputs are not permitted

The same request works fine when sent directly to Anthropic. This breaks Claude Code and Agent SDK traffic through Portkey when using the prompt-caching-scope-2026-01-05 beta.

Root Cause

1. anthropic-beta header not forwarded with targets-based configs

In handlerUtils.ts, anthropicConfig (which reads anthropic-beta from request headers) is only merged into provider options for inline configs. When using targets, the header never reaches api.ts, so it falls back to the default messages-2023-12-15 beta — and Anthropic rejects the scope field as unknown.

2. cache_control.scope stripped in OpenAI-to-Anthropic transforms

Six places in chatComplete.ts hardcode cache_control: { type: 'ephemeral' }, discarding any additional fields like scope. Same issue in Bedrock's utils.ts.

Fix

  • api.ts: Fall back to reading anthropic-beta and anthropic-version from the original request headers via Hono context (c.req.header(...)) when providerOptions and gatewayRequestBody don't include them
  • chatComplete.ts: Preserve the original cache_control object instead of replacing it with { type: 'ephemeral' } (6 locations)
  • bedrock/utils.ts: Same fix for the Bedrock Anthropic path
  • Type definitions: Add optional scope field to CacheControlEphemeral and PromptCache

Files Changed

  • src/providers/anthropic/api.ts — header fallback via Hono context
  • src/providers/anthropic/chatComplete.ts — preserve cache_control in transforms
  • src/providers/bedrock/utils.ts — preserve cache_control in Bedrock path
  • src/types/MessagesRequest.ts — add scope? to CacheControlEphemeral
  • src/types/requestBody.ts — add scope? to PromptCache

Fixes #1589

When using targets-based configs, the anthropic-beta request header
was not being forwarded to Anthropic's API. This caused requests with
cache_control.scope (used by the prompt-caching-scope-2026-01-05 beta)
to fail with "Extra inputs are not permitted".

The fix adds a fallback to read anthropic-beta and anthropic-version
directly from the original request headers via Hono context when
providerOptions and gatewayRequestBody don't include them.

Additionally, the OpenAI-to-Anthropic transform was hardcoding
cache_control to { type: 'ephemeral' }, stripping any additional
fields like scope. Now preserves the original cache_control object.

Fixes Portkey-AI#1589
- Add sanitizeCacheControl() that validates shape before forwarding,
  only allowing known fields (type, scope) instead of blind passthrough
- Add 6 tests for api.ts header fallback behavior (providerOptions >
  request body > Hono context > defaults)
- Add 8 tests for chatComplete.ts cache_control transform (scope
  preservation, backward compat, unknown field stripping)
- Apply same sanitization in bedrock/utils.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic /v1/messages rejects cache_control.scope that Anthropic direct accepts

1 participant