Skip to content

Claude Sonnet 4.5 Returns Incorrectly Wrapped Structured Response #4

Description

@MHindermann

Claude Sonnet 4.5 Returns Incorrectly Wrapped Structured Response

See RISE-UNIBAS/humanities_data_benchmark#84

Claude Sonnet 4.5 (claude-sonnet-4-5-20250929) is wrapping structured output responses in a parameter object instead of returning the Pydantic model directly, causing validation failures.

Expected Behavior

{
    "metadata": {...},
    "entries": [...]
}

Actual Behavior

{
    "parameter": {
        "metadata": {...},
        "entries": [...]
    }
}

See request_T0225_page_10.json.

Error

2026-01-24 13:52:32,459 INFO:root:Running T0225
2026-01-24 13:52:32,460 INFO:root:Loaded bibliographic_data from ../benchmarks\bibliographic_data\benchmark.py
2026-01-24 13:52:32,666 INFO:root:Running bibliographic_data (anthropic/claude-sonnet-4-5-20250929)...
2026-01-24 13:52:32,667 INFO:root:Found 5 objects to process.
2026-01-24 13:52:32,667 INFO:root:Processing T0225, page_10...
2026-01-24 13:52:32,705 INFO:ai_client.utils:Resized image page_10.jpeg from (1825, 2896) to (1291, 2048)
2026-01-24 13:53:02,806 INFO:httpx:HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"
2026-01-24 13:53:02,807 WARNING:ai_client.claude_client:Pydantic validation failed: 2 validation errors for Document
metadata
 Field required [type=missing, input_value={'parameter': {'metadata'... 'incomplete': False}]}}, input_type=dict]
   For further information visit https://errors.pydantic.dev/2.10/v/missing
entries
 Field required [type=missing, input_value={'parameter': {'metadata'... 'incomplete': False}]}}, input_type=dict]
   For further information visit https://errors.pydantic.dev/2.10/v/missing
2026-01-24 13:53:02,827 INFO:root:Saved answer to ..\results\2026-01-24\T0225\request_T0225_page_10.json
2026-01-24 13:53:02,827 INFO:root:Finished page_10 with score: {'fuzzy': 0.0}

Model

  • Provider: anthropic
  • Model: claude-sonnet-4-5-20250929
  • Test: T0225 (bibliographic_data benchmark)

Context

  • Using Anthropic's native tool calling/structured output
  • Pydantic model: Document with fields metadata and entries
  • Other Claude models may not have this issue (needs verification)

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions