Vertex AI Claude structured output fails with `output_config.format: Extra inputs are not permitted`

[rempsyc]

What Happened?

Structured output fails for Claude models routed through Vertex AI, while plain chat works with the same model and same gateway.

Using ellmer::chat_portkey() against our Portkey-backed gateway, this works:

• @vertexai/anthropic.claude-opus-4-6 with chat$chat(...)

But this fails:

• @vertexai/anthropic.claude-opus-4-6 with chat$chat_structured(...)

Error returned:

Error in `req_perform()`:
! HTTP 400 Bad Request.
ℹ vertex-ai error: output_config.format: Extra inputs are not permitted

This seems specific to structured output / JSON schema translation on the Vertex Claude path.

For comparison, structured output works for Gemini through the same gateway:

• @vertexai/gemini-2.5-flash-lite

This appears similar to:

• [Bug]: Getting 'output_config: Extra inputs are not permitted' error when calling the Anthropic Messages API for Claude Models deployed on VertexAI BerriAI/litellm#21407

What Should Have Happened?

One of the following:

1. Structured output should work for Claude models on the Vertex AI route, if that route is supported.

2. If structured output is not supported for Claude-on-Vertex, the gateway should reject the request earlier with a clear unsupported-feature / unsupported-model error, rather than forwarding a payload that Vertex rejects with:

output_config.format: Extra inputs are not permitted

Relevant Code Snippet (R)

library(ellmer)

# Works: plain chat
chat <- chat_portkey(
  model = "@vertexai/anthropic.claude-opus-4-6",
  base_url = "https://ai-gateway.apps.cloud.rt.nyu.edu/v1/"
)

chat$chat("Tell me one short joke about books.")

# Fails: structured output
chat <- chat_portkey(
  model = "@vertexai/anthropic.claude-opus-4-6",
  base_url = "https://ai-gateway.apps.cloud.rt.nyu.edu/v1/"
)

type <- ellmer::type_object(
  party = ellmer::type_string(),
  rating = ellmer::type_number()
)

chat$chat_structured(
  "You are simulating a Republican. On a scale from 0 to 100, how warmly do you feel?",
  type = type
)

# Comparison: this structured-output call works
chat_gemini <- chat_portkey(
  model = "@vertexai/gemini-2.5-flash-lite",
  base_url = "https://ai-gateway.apps.cloud.rt.nyu.edu/v1/"
)

chat_gemini$chat_structured(
  "You are simulating a Republican. On a scale from 0 to 100, how warmly do you feel?",
  type = type
)

Your Twitter/LinkedIn

https://www.linkedin.com/in/rempsyc/ and https://x.com/RemiTPsyc

[rempsyc]

Additional trace from the gateway UI for one failed request.

Trace ID:

• d54dab99-4ab3-4b6f-a6cd-6c5eb8110aed

Redacted upstream request:

{
  "url": "https://aiplatform.googleapis.com/v1/projects/[REDACTED]/locations/global/publishers/anthropic/models/claude-opus-4-6:rawPredict",
  "method": "POST",
  "headers": {
    "content-type": "application/json",
    "user-agent": "r-ellmer/0.4.0.9000",
    "x-portkey-provider": "vertex-ai",
    "x-portkey-trace-id": "d54dab99-4ab3-4b6f-a6cd-6c5eb8110aed",
    "x-portkey-vertex-auth-type": "serviceAccount",
    "x-portkey-vertex-region": "global",
    "x-portkey-virtual-key": "vertexai"
  },
  "body": {
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "You are simulating a Republican. On a scale from 0 to 100, how warmly do you feel?"
          }
        ]
      }
    ],
    "model": "anthropic.claude-opus-4-6",
    "stream": false,
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "structured_data",
        "schema": {
          "type": "object",
          "description": "",
          "properties": {
            "party": {
              "type": "string",
              "description": ""
            },
            "rating": {
              "type": "number",
              "description": ""
            }
          },
          "required": [
            "party",
            "rating"
          ],
          "additionalProperties": false
        },
        "strict": true
      }
    }
  }
}

Redacted upstream response:

{
  "status": 400,
  "headers": {
    "content-type": "application/json",
    "request-id": "req_vrtx_011CZArTdXTtXV9cRn9AbKWu",
    "x-portkey-provider": "vertex-ai",
    "x-portkey-trace-id": "d54dab99-4ab3-4b6f-a6cd-6c5eb8110aed",
    "x-should-retry": "false"
  },
  "body": {
    "error": {
      "message": "vertex-ai error: output_config.format: Extra inputs are not permitted",
      "type": "invalid_request_error",
      "param": null,
      "code": null
    },
    "provider": "vertex-ai"
  },
  "responseTime": 238
}

[swetha-portkey]

Thanks a lot for the detailed report and examples - this is really helpful. I’ve flagged this with the team so we can dig into what’s happening with structured output for Claude via Vertex on this path. We’ll update this issue once we have more clarity on what’s supported and the right behavior here.