Skip to content

test(benchmark): use stable curl JSON fixture - #3340

Open
lntutor wants to merge 1 commit into
rtk-ai:developfrom
lntutor:fix/benchmark-stable-curl-json
Open

test(benchmark): use stable curl JSON fixture#3340
lntutor wants to merge 1 commit into
rtk-ai:developfrom
lntutor:fix/benchmark-stable-curl-json

Conversation

@lntutor

@lntutor lntutor commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace the rotating https://mockhttp.org/json/1 benchmark input with the repository oc_pods.json fixture
  • keep raw and RTK curl commands pointed at the same deterministic payload
  • avoid false token-growth failures caused by two requests receiving unrelated JSON documents

Why

mockhttp.org/json/1 rotates among different response bodies. The benchmark performs separate raw and RTK requests, so it can compare different documents and report apparent token growth even when RTK behavior is unchanged. This is reproducible on develop.

Using a local file URL makes the comparison deterministic and requires no runtime code changes.

Validation

  • bash -n scripts/benchmark.sh
  • git diff --check
  • release build
  • curl benchmark row: 2523 -> 2523 (+0%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant