Skip to content

fix: call OnDiscard exactly once per item on encode failure#13

Merged
acoshift merged 1 commit into
mainfrom
fix/double-discard-on-retry
May 25, 2026
Merged

fix: call OnDiscard exactly once per item on encode failure#13
acoshift merged 1 commit into
mainfrom
fix/double-discard-on-retry

Conversation

@acoshift
Copy link
Copy Markdown
Member

Summary

  • Finding 1 — double-discard on retry: flush previously called invokeOnDiscard during JSON encoding, but left the item in the buffer slice. On every HTTP retry the same item was re-encoded (failing again) and the callback fired a second time. Fixed by collecting encode failures in a local slice and only calling invokeOnDiscard after a successful HTTP response.
  • Finding 2 — empty-body infinite retry: When every item in a batch was unencodable the encoded buffer was empty. flush still POSTed a zero-byte body; if the server rejected it retryFlush looped indefinitely, re-discarding items each iteration. Fixed by returning true early (and discarding all failures once) when buf.Len() == 0, skipping the HTTP request entirely.

Test plan

  • TestIngest_EncodeError_DiscardedExactlyOnce_OnHTTPFailure — server fails twice then succeeds; verifies OnDiscard fires exactly once for the unencodable item despite multiple retries
  • TestIngest_AllEncodeErrors_NoHTTPRequest — all items unencodable; verifies zero HTTP requests are made and OnDiscard fires once per item
  • Existing TestIngest_JSONEncodeError_CallsOnDiscard still passes (happy-path encode error with successful HTTP)
  • go test ./... passes

🤖 Generated with Claude Code

Previously flush() called invokeOnDiscard immediately when json.Encode
failed, but left the item in the buffer. On every HTTP retry the same
item was re-encoded (failing again) and the callback fired a second
(third, …) time.

Two related issues fixed together:
- Collect encode failures in a local slice; call invokeOnDiscard only
  after a successful HTTP response so retries cannot double-fire it.
- If every item in a batch is unencodable the encoded buffer is empty.
  Guard against sending a zero-byte body (which the server may reject,
  causing an infinite retry loop): return true immediately and discard
  all failures once, without making an HTTP request.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@acoshift acoshift merged commit 933021a into main May 25, 2026
1 check passed
@acoshift acoshift deleted the fix/double-discard-on-retry branch May 25, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant