Preserve thinking/redacted_thinking blocks through forward + reverse transforms#28
Open
danaimone wants to merge 1 commit intozacdcook:masterfrom
Open
Preserve thinking/redacted_thinking blocks through forward + reverse transforms#28danaimone wants to merge 1 commit intozacdcook:masterfrom
danaimone wants to merge 1 commit intozacdcook:masterfrom
Conversation
…erse transforms
Anthropic requires thinking/redacted_thinking content blocks to be echoed back
byte-identical to what the model originally produced. Any mutation triggers
"thinking or redacted_thinking blocks in the latest assistant message cannot
be modified. These blocks must remain as they were in the original response."
The proxy's string transformation pipeline was mutating these blocks in two
places:
1. Forward pass (processBody): Layer 2/3/6 split/join runs across the full
request body and rewrites any occurrence inside prior assistant thinking
blocks in history.
2. Reverse pass (reverseMap, SSE path): the tail-buffered flush rewrites
thinking_delta bytes on the way out. The client stores the mutated bytes
and echoes them on the next turn, and Anthropic rejects the comparison.
Fix:
- Add maskThinkingBlocks / unmaskThinkingBlocks helpers that scan for
{"type":"thinking"...} and {"type":"redacted_thinking"...} content blocks
with string-aware bracket matching, replace each with a unique placeholder
before transforms run, restore after.
- Wrap processBody with mask/unmask so Layer 2/3/6 never see assistant
thinking history.
- Replace the SSE tail-buffer flush with event-complete buffering (split on
\n\n) and track the current content block type across events. Pass
thinking/redacted_thinking events through unchanged; reverse-map
everything else. This also subsumes the cross-chunk pattern fix from zacdcook#11
since SSE events are self-contained.
- Wrap the non-streaming JSON response in the same mask/unmask around
reverseMap.
The reverse-mapping of thinking content was arguably always incorrect —
thinking blocks are the model's internal reasoning and shouldn't be rewritten
in either direction. The byte-equality requirement just turns a latent bug
into a hard failure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a request has extended thinking enabled and the conversation contains any prior assistant turns with
thinking/redacted_thinkingcontent blocks, the next turn fails with:Anthropic enforces byte-identical echo of thinking blocks on the latest assistant message. The proxy's string transformation pipeline mutates them in two places.
Root cause
Forward pass (
processBody) — Layer 2 replacements, Layer 3 tool renames, and Layer 6 property renames all run assplit/joinacross the full request body. If a prior assistant thinking block contains any rewritten substring (openclaw,HEARTBEAT,prometheus, or any"quoted"tool/property name), it gets rewritten alongside everything else. Anthropic then rejects.SSE reverse pass (
reverseMapin the streaming handler) — the more common failure mode. When Anthropic streams a response containingthinking_deltaevents, the tail-bufferedreverseMapmutates them on the way out. The client stores the mutated bytes and echoes them back on the next turn. Anthropic compares against what it originally sent and rejects. So even if the forward pass were clean, the reverse pass alone corrupts history and breaks the next turn.Non-streaming JSON response — same issue, same fix.
Repro
reverseMaprewrites e.g.ocplatform→openclawinside athinking_delta.Fix
Forward pass — add
maskThinkingBlocks/unmaskThinkingBlockshelpers that scan for{"type":"thinking"...}and{"type":"redacted_thinking"...}content blocks with string-aware bracket matching, replace each with a unique placeholder (__OBP_THINK_MASK_<n>__) before transforms run, and restore after. The placeholder sigil is chosen so no existing replacement / tool rename / property rename can match it.SSE reverse pass — switch from tail-buffered chunk flushing to SSE-event-aware buffering (split on
\n\n), track the current content block type across events via a state machine (content_block_start→ set,content_block_stop→ clear), and passcontent_block_*events unchanged while the current block type isthinkingorredacted_thinking. Reverse-map everything else as before. Bonus: event-complete buffering also subsumes the cross-chunk pattern fix from #11 since SSE events are self-contained, so patterns can't span event boundaries.Non-streaming JSON response — wrap
reverseMapin the same mask/unmask as the forward pass.Notes
Test plan
representative-claimheader is stillfive_hour(billing classification unchanged — this PR only affects body/stream handling, not the billing layer)