Skip to content

feat: immediate-delivery capture (capture_immediate / capture_batch_immediate)#175

Open
eli-r-ph wants to merge 3 commits into
mainfrom
feat/capture-confirmed
Open

feat: immediate-delivery capture (capture_immediate / capture_batch_immediate)#175
eli-r-ph wants to merge 3 commits into
mainfrom
feat/capture-confirmed

Conversation

@eli-r-ph

@eli-r-ph eli-r-ph commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

capture / capture_batch are fire-and-forget: since v0.14.0 they enqueue onto the background transport worker and never report the outcome. That's the right default for analytics, but it leaves no path for a caller that must know a batch actually persisted before advancing its own durable state — the motivating case being a server-side importer that commits an upstream offset only once the events are safely ingested. Today such consumers are pinned to 0.13.3 (the last release where capture_batch returned a Result).

This restores an inline confirmed-delivery path as an additive, opt-in API without disturbing the fire-and-forget default:

  • Client::capture_immediate(event) and Client::capture_batch_immediate(events, historical_migration) on both the async and blocking clients.
  • They send inline (bypassing the worker) and return Result<CaptureSummary, Error> once the request reaches a terminal outcome, retrying transient failures per the client's existing retry configuration.
  • They reuse the existing sans-IO core (v1_capture / v0_capture / retry helpers), so both the v0 and capture-v1 pipelines are covered with no duplicated protocol logic.

New public CaptureSummary (#[non_exhaustive], accessor-only): submitted(), not_persisted(), all_persisted(), and event_results() (under capture-v1). On v0 a 2xx persists the whole batch (all_persisted() always true); on capture-v1 the per-event verdicts are surfaced, so a 2xx with drop/retry verdicts reports as not fully persisted.

Design notes:

  • Naming: *_immediate mirrors the captureImmediate method in posthog-node (send inline, resolve on completion). Unlike Node's Promise<void>, the Rust API keeps the richer Result<CaptureSummary> so callers can inspect per-event durability.
  • Fire-and-forget remains the primary, recommended API and is unchanged.
  • The immediate variants do not fire on_error hooks — the returned Result is the delivery signal, avoiding double-reporting. on_error stays the observability channel for the fire-and-forget path.
  • De-duplication: the I/O-free setup (event prep, request building, payload encoding, URL resolution) lives in shared prepare_immediate / build_attempt_parts helpers in v0_capture / v1_capture. Each of the four client methods keeps only its transport-specific send loop — async awaits tokio::time::sleep between retries, blocking uses std::thread::sleep — so there is no new runtime coupling and no protocol logic duplicated across clients.
  • Disabled clients and empty / fully before_send-filtered batches return a default CaptureSummary without sending a request.

💚 How did you test it?

tests/test_capture_immediate.rs with httpmock coverage across all four permutations (async x blocking, v0 x capture-v1): single/batch success, v1 partial-persist reporting, retryable-status-then-success and inline partial-retry pruning within one call, retry-budget exhaustion returning Err, terminal statuses without retry, historical-migration flag on the wire, gzip (query param + Content-Encoding header) on v0, disabled/empty no-ops, and an explicit assertion that immediate calls never fire on_error.

The retryable-exhaustion and terminal-no-retry cases are parameterized over the full shared status sets (408/500/502/503/504 retryable; 400/401/402/403, plus a bare 429 on v0, terminal) in every client/pipeline module, and the retry-then-empty-success case now asserts not_persisted() / all_persisted() directly.

Verified cargo test, cargo clippy --all-targets -- -D warnings, cargo fmt --check, and RUSTDOCFLAGS=-D warnings cargo doc clean across the feature matrix (default async v0, capture-v1, --no-default-features blocking v0/v1). Existing suites still pass (224 unit + integration). Public API snapshot regenerated via scripts/check-public-api.sh --update, and the downstream capture-v1-rs-smoke e2e harness still builds against this branch.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran sampo add to generate a changeset file

Add `capture_confirmed` / `capture_batch_confirmed` on both the async and
blocking clients, returning `Result<CaptureSummary, Error>` after sending
inline (bypassing the background worker) and awaiting a terminal outcome.
Reuses the existing sans-IO request-build/retry/response-classification core,
so both the v0 and capture-v1 pipelines are supported.

These are for callers that must know a batch persisted before advancing their
own durable state (e.g. a server-side importer committing an upstream offset).
Fire-and-forget capture/capture_batch remain the primary API. The confirmed
variants do NOT fire on_error hooks — the returned Result is the signal.
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

posthog-rs-v0 Compliance Report

Date: 2026-07-07 23:42:24 UTC
Duration: 15434ms

✅ All Tests Passed!

46/46 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 97ms
Format Validation.Event Has Uuid 110ms
Format Validation.Event Has Lib Properties 109ms
Format Validation.Distinct Id Is String 109ms
Format Validation.Token Is Present 109ms
Format Validation.Custom Properties Preserved 102ms
Format Validation.Event Has Timestamp 108ms
Retry Behavior.Retries On 503 5112ms
Retry Behavior.Does Not Retry On 400 2104ms
Retry Behavior.Does Not Retry On 401 2106ms
Retry Behavior.Respects Retry After Header 5050ms
Retry Behavior.Implements Backoff 15065ms
Retry Behavior.Retries On 500 5065ms
Retry Behavior.Retries On 502 5062ms
Retry Behavior.Retries On 504 5064ms
Retry Behavior.Max Retries Respected 15063ms
Deduplication.Generates Unique Uuids 68ms
Deduplication.Preserves Uuid On Retry 5015ms
Deduplication.Preserves Uuid And Timestamp On Retry 10025ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 5026ms
Deduplication.No Duplicate Events In Batch 19ms
Deduplication.Different Events Have Different Uuids 17ms
Compression.Sends Gzip When Enabled 22ms
Batch Format.Uses Proper Batch Structure 23ms
Batch Format.Flush With No Events Sends Nothing 15ms
Batch Format.Multiple Events Batched Together 82ms
Error Handling.Does Not Retry On 403 2045ms
Error Handling.Does Not Retry On 413 2067ms
Error Handling.Retries On 408 5046ms

Feature_Flags Tests

17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 56ms
Request Payload.Flags Request Uses V2 Query Param 40ms
Request Payload.Flags Request Hits Flags Path Not Decide 26ms
Request Payload.Flags Request Omits Authorization Header 23ms
Request Payload.Token In Flags Body Matches Init 17ms
Request Payload.Groups Round Trip 38ms
Request Payload.Groups Default To Empty Object 31ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 30ms
Request Payload.Disable Geoip Omitted Defaults To False 32ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 30ms
Request Lifecycle.No Flags Request On Init Alone 22ms
Request Lifecycle.No Flags Request On Normal Capture 25ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 49ms
Request Lifecycle.Mock Response Value Is Returned To Caller 27ms
Retry Behavior.Retries Flags On 502 233ms
Retry Behavior.Retries Flags On 504 227ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 16ms

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

posthog-rs-v1 Compliance Report

Date: 2026-07-07 23:42:56 UTC
Duration: 23024ms

✅ All Tests Passed!

111/111 tests passed


Capture_V1 Tests

94/94 tests passed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint 161ms
Endpoint And Method.Does Not Use Legacy Endpoints 161ms
Required Headers.Has Authorization Bearer Header 159ms
Required Headers.Has Content Type Json 159ms
Required Headers.Has Posthog Sdk Info Format 157ms
Required Headers.Has Posthog Attempt Header 159ms
Required Headers.Has Posthog Request Id 159ms
Required Headers.Has Posthog Request Timestamp 159ms
Required Headers.Has User Agent 163ms
Body Format.Body Has Created At And Batch 158ms
Body Format.No Api Key In Body 119ms
Body Format.No Sent At In Body 118ms
Event Format.Event Has Required Root Fields 119ms
Event Format.Event Uuid Is Valid 130ms
Event Format.Event Timestamp Is Rfc3339 132ms
Event Format.Distinct Id Is String 132ms
Event Format.Distinct Id At Root Not Properties 117ms
Event Format.Custom Properties Preserved 135ms
Event Format.Set Properties Preserved 132ms
Event Format.Set Once Properties Preserved 133ms
Event Format.Groups Properties Preserved 117ms
Event Format.Sdk Generates Uuid If Not Provided 132ms
Event Format.Event Has Required Root Fields Batch 153ms
Event Format.Event Uuid Is Valid Batch 153ms
Event Format.Event Timestamp Is Rfc3339 Batch 157ms
Event Format.Distinct Id Is String Batch 166ms
Event Format.Distinct Id At Root Not Properties Batch 165ms
Event Format.Custom Properties Preserved Batch 156ms
Event Format.Set Properties Preserved Batch 153ms
Event Format.Set Once Properties Preserved Batch 180ms
Event Format.Groups Properties Preserved Batch 145ms
Event Format.Sdk Generates Uuid If Not Provided Batch 130ms
Batch Behavior.Multiple Events In Single Batch 183ms
Batch Behavior.Batch Envelope Smoke 177ms
Batch Behavior.Flush With No Events Sends Nothing 80ms
Batch Behavior.Flush At Triggers Batch 1114ms
Batch Behavior.Created At Reflects Batch Creation Time 90ms
Deduplication.Generates Unique Uuids 172ms
Deduplication.Different Events Same Content Different Uuids 133ms
Deduplication.Preserves Uuid On Retry 5090ms
Deduplication.Preserves Timestamp On Retry 5085ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 5132ms
Deduplication.No Duplicate Events In Batch 147ms
Header Behavior On Retry.Attempt Header Starts At One 75ms
Header Behavior On Retry.Attempt Header Increments On Retry 10080ms
Header Behavior On Retry.Request Id Preserved On Retry 5081ms
Header Behavior On Retry.Different Requests Have Different Request Ids 2074ms
Header Behavior On Retry.Request Timestamp Changes On Retry 5072ms
Response Format Validation.Success Response Has Uuid Keyed Results 65ms
Response Format Validation.Success Response Has Ok For Each Event 32ms
Response Format Validation.Success No Retry After When All Ok 30ms
Response Format Validation.Success Retry After Present When Retry Events 32ms
Response Format Validation.Success No Retry After When Drop Only 34ms
Response Format Validation.Response Echoes Request Id 31ms
Retry Behavior.Retries On 408 5034ms
Retry Behavior.Retries On 500 5024ms
Retry Behavior.Retries On 503 5025ms
Retry Behavior.Retries On 504 5023ms
Retry Behavior.Retryable Errors Have Retry After 2024ms
Retry Behavior.Respects Retry After On Retryable Error 8024ms
Retry Behavior.Does Not Retry On 400 2027ms
Retry Behavior.Does Not Retry On 401 2026ms
Retry Behavior.Does Not Retry On 402 2034ms
Retry Behavior.Does Not Retry On 413 2027ms
Retry Behavior.Does Not Retry On 415 2027ms
Retry Behavior.Non Retryable Errors Have No Retry After 2026ms
Retry Behavior.Implements Backoff 15028ms
Retry Behavior.Max Retries Respected 15027ms
Partial Batch Handling.Handles 200 Full Success 2041ms
Partial Batch Handling.Handles 200 With All Ok 3050ms
Partial Batch Handling.Does Not Retry Dropped Events 3035ms
Partial Batch Handling.Does Not Retry Limited Events 3031ms
Partial Batch Handling.Prunes Ok Events On Partial Retry 5027ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry 5024ms
Partial Batch Handling.Retries Only Retry Events From Partial 5026ms
Partial Batch Handling.Partial Retry Preserves Uuids 5024ms
Partial Batch Handling.Partial Retry Attempt Header Increments 5029ms
Partial Batch Handling.Partial Retry Request Id Preserved 5030ms
Partial Batch Handling.Respects Retry After On Partial 5023ms
Partial Batch Handling.Unknown Result Treated As Terminal 3024ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry 3050ms
Compression.Sends Gzip Content Encoding 30ms
Compression.No Content Encoding When Disabled 24ms
Compression.Compressed Body Is Decompressible 22ms
Error Handling.Does Not Retry On Unknown 4Xx 2022ms
Event Options.Cookieless Mode Override 23ms
Event Options.Disable Skew Correction Override 21ms
Event Options.Process Person Profile Override 22ms
Event Options.Product Tour Id Override 22ms
Event Options.Unset Options Omitted 21ms
Event Options.Options Override In Batch 22ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties 22ms
Geoip And Historical Migration.Historical Migration Set In Body 22ms
Geoip And Historical Migration.Historical Migration Absent By Default 22ms

Feature_Flags Tests

17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 13ms
Request Payload.Flags Request Uses V2 Query Param 15ms
Request Payload.Flags Request Hits Flags Path Not Decide 17ms
Request Payload.Flags Request Omits Authorization Header 13ms
Request Payload.Token In Flags Body Matches Init 14ms
Request Payload.Groups Round Trip 14ms
Request Payload.Groups Default To Empty Object 16ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 15ms
Request Payload.Disable Geoip Omitted Defaults To False 17ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 13ms
Request Lifecycle.No Flags Request On Init Alone 11ms
Request Lifecycle.No Flags Request On Normal Capture 22ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 16ms
Request Lifecycle.Mock Response Value Is Returned To Caller 14ms
Retry Behavior.Retries Flags On 502 217ms
Retry Behavior.Retries Flags On 504 218ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 22ms

@greptile-apps

greptile-apps Bot commented Jul 2, 2026

Copy link
Copy Markdown

Comments Outside Diff (3)

  1. tests/test_capture_confirmed.rs, line 859-878 (link)

    P2 Single-status-code retry tests depart from the codebase's parameterised pattern

    The established test pattern in this repo (see test_v0_retry.rs, which loops for status in [408u16, 500, 502, 503, 504] and for batch in [false, true]) exercises the full retryable-status surface in one test. The new exhausting_retryable_status_returns_err and terminal-status tests each pick only one code (500/503 for retryable; 401/402 for terminal), leaving 408, 502, 504, and 400/403/429 untested and causing asymmetry between the async_v1 and async_v0 modules (async_v1 tests 402, async_v0 tests 401). The same gap applies to blocking_v1 and blocking_v0. Looping over the full expected set of retryable and terminal codes within the same test body would bring these in line with the rule that we always prefer parameterised tests.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

  2. tests/test_capture_confirmed.rs, line 799-826 (link)

    P2 retryable_status_then_success_within_one_call omits the interesting assertions

    The inline comment says "the one submitted event has no ok/warning verdict either", which means not_persisted() == 1 and all_persisted() == false — but neither of those is asserted. The test currently only checks summary.submitted() == 1, which would also pass if the implementation silently returned the wrong summary altogether. Adding assert_eq!(summary.not_persisted(), 1) and assert!(!summary.all_persisted()) would pin the observable contract described in the comment.

  3. src/client/async_client.rs, line 127-218 (link)

    P2 send_confirmed duplicated across async and blocking clients

    The entire retry loop — request building, header construction, body encoding, response text extraction, Step dispatch — is repeated in async_client.rs and blocking.rs for both the v0 and v1 paths (~200 lines each). The only difference is tokio::time::sleep(delay).await versus std::thread::sleep(delay) and response.text().await versus response.text(). A thin trait or a macro parameterised over the sleep call would satisfy the "OnceAndOnlyOnce" simplicity rule without introducing a new runtime dependency. As-is, any future change to retry logic, header stamping, or body encoding must be applied in four places.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Reviews (1): Last reviewed commit: "feat: add confirmed-delivery capture met..." | Re-trigger Greptile

@eli-r-ph eli-r-ph self-assigned this Jul 2, 2026
@eli-r-ph eli-r-ph marked this pull request as ready for review July 2, 2026 20:07
@eli-r-ph eli-r-ph requested a review from a team as a code owner July 2, 2026 20:07
A disabled client or a fully before_send-filtered batch returns a default
CaptureSummary (submitted == 0, all_persisted == true). Callers that advance
durable state on a confirmed delivery must also check submitted(), so document
that all_persisted() alone is not a durability gate.
@marandaneto

marandaneto commented Jul 3, 2026

Copy link
Copy Markdown
Member

@eli-r-ph python has this https://github.com/PostHog/posthog-python/blob/6f75afe77ff059e4f3b0b6b7b30912612a7b5ff1/posthog/client.py#L284-L285 which bypasses queuing the events, i think something like this could be used in the rust async client so you can await the request

Node has captureImmediate methods that bypasses queuing

the other option is an on_success hook very similar to on_error hook (same shape, but rather for success use cases)
i think this is also a nice to have since its been requested before in other SDKs

the other option is to change the hook to have 2 methods, on_success and on_error, since its behind the v1 feature, we could break this API and make it nicer, and ofc rename it to something that makes more sense like idk http_completion_handler or something similar

i think all of those approaches are maybe a bit nicer than the one suggested in this PR

WDYT? @PostHog/team-client-libraries opinions?

@eli-r-ph

eli-r-ph commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the reply, I'll take a look and update this accordingly - @PostHog/team-client-libraries please do chime in if you have additional thoughts on the matter, want to make sure I'm adding something that is in line with wider needs, doesn't just unblock internal SDK use cases 👍

@dustinbyrne

Copy link
Copy Markdown
Contributor

I like the idea of standardizing on something like captureImmediate/capture_immediate. I think the global client setting for Python doesn't offer enough control (i.e., cannot easily mix immediate and deferred capture without instantiating an additional client).

capture_confirmed is effectively the same as Node's captureImmediate though, isn't it? If it's just a matter of standardizing on a single naming scheme, let's go with what's already public.

For languages that have idiomatic return types that indicate a success/failure result, I think it's more expected that we prefer those over communicating via on_error/on_success hooks.

Rename capture_confirmed/capture_batch_confirmed to
capture_immediate/capture_batch_immediate for cross-SDK naming
consistency (mirrors the posthog-node captureImmediate method), keeping
the richer Result<CaptureSummary> return.

Extract the I/O-free setup (event prep, request building, payload
encoding, URL resolution) into shared prepare_immediate (and v1
build_attempt_parts) helpers in v0_capture/v1_capture, collapsing the
four client send methods to just the transport-specific send loop. No
change to execution logic or timing.

Parameterize the retryable-exhaustion and terminal-no-retry tests over
the full shared status sets across all four client/pipeline modules, add
a blocking-v1 terminal case, and assert not_persisted/all_persisted on
the retry-then-empty-success path. Regenerate the public API snapshot.
@eli-r-ph eli-r-ph changed the title feat: confirmed-delivery capture (capture_confirmed / capture_batch_confirmed) feat: immediate-delivery capture (capture_immediate / capture_batch_immediate) Jul 7, 2026
@eli-r-ph

eli-r-ph commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

👋 hey @dustinbyrne made a round of updates based on you an @marandaneto 's callouts, let me know if this feels like the right direction or needs larger rethink 🙇

@dustinbyrne

Copy link
Copy Markdown
Contributor

looks clean, nice addition

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants