Skip to content

Add apm-sdks-benchmarks to CI #10206

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
sarahchen6 wants to merge 23 commits into
base: master
Choose a base branch
from
Draft

Add apm-sdks-benchmarks to CI #10206

sarahchen6 wants to merge 23 commits into from

Conversation

@sarahchen6
Copy link
Contributor

@sarahchen6 sarahchen6 commented Dec 12, 2025 •
edited
Loading

What Does This Do

Test https://github.com/DataDog/apm-sdks-benchmarks/pull/44

and https://github.com/DataDog/apm-sdks-benchmarks/pull/45

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@sarahchen6 sarahchen6 added the tag: do not merge Do not merge changes label Dec 12, 2025
@sarahchen6 sarahchen6 force-pushed the sarahchen6/test-merge-base-benchmark branch from c1cba5f to 732aefd Compare December 12, 2025 23:03
@pr-commenter
Copy link

pr-commenter bot commented Dec 13, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/test-merge-base-benchmark
git_commit_date 1768337169 1768342937
git_commit_sha 671cc8c 3d8f8c0
release_version 1.59.0-SNAPSHOT~671cc8cdbf 1.57.0-SNAPSHOT~3d8f8c0fd6
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1768344947 1768344947
ci_job_id 1348388676 1348388676
ci_pipeline_id 90588023 90588023
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-wjthq0x7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-wjthq0x7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 8 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~3d8f8c0fd6, baseline=1.59.0-SNAPSHOT~671cc8cdbf

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.098 s) : 0, 1097959
Total [baseline] (8.801 s) : 0, 8800534
Agent [candidate] (1.082 s) : 0, 1081680
Total [candidate] (8.742 s) : 0, 8741687
section iast
Agent [baseline] (1.221 s) : 0, 1220858
Total [baseline] (9.286 s) : 0, 9285805
Agent [candidate] (1.229 s) : 0, 1228711
Total [candidate] (9.334 s) : 0, 9333656
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.098 s -
Agent iast 1.221 s 122.899 ms (11.2%)
Total tracing 8.801 s -
Total iast 9.286 s 485.271 ms (5.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.082 s -
Agent iast 1.229 s 147.031 ms (13.6%)
Total tracing 8.742 s -
Total iast 9.334 s 591.968 ms (6.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~3d8f8c0fd6, baseline=1.59.0-SNAPSHOT~671cc8cdbf

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.214 ms) : 0, 1214
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (660.876 ms) : 0, 660876
BytebuddyAgent [candidate] (650.05 ms) : 0, 650050
GlobalTracer [baseline] (285.426 ms) : 0, 285426
GlobalTracer [candidate] (282.191 ms) : 0, 282191
AppSec [baseline] (33.085 ms) : 0, 33085
AppSec [candidate] (32.502 ms) : 0, 32502
Debugger [baseline] (68.004 ms) : 0, 68004
Debugger [candidate] (66.993 ms) : 0, 66993
Remote Config [baseline] (636.338 µs) : 0, 636
Remote Config [candidate] (642.716 µs) : 0, 643
Telemetry [baseline] (9.019 ms) : 0, 9019
Telemetry [candidate] (8.934 ms) : 0, 8934
Flare Poller [baseline] (3.844 ms) : 0, 3844
Flare Poller [candidate] (3.748 ms) : 0, 3748
section iast
crashtracking [baseline] (1.183 ms) : 0, 1183
crashtracking [candidate] (1.204 ms) : 0, 1204
BytebuddyAgent [baseline] (789.854 ms) : 0, 789854
BytebuddyAgent [candidate] (795.326 ms) : 0, 795326
GlobalTracer [baseline] (256.15 ms) : 0, 256150
GlobalTracer [candidate] (257.658 ms) : 0, 257658
IAST [baseline] (26.879 ms) : 0, 26879
IAST [candidate] (26.991 ms) : 0, 26991
AppSec [baseline] (34.241 ms) : 0, 34241
AppSec [candidate] (33.564 ms) : 0, 33564
Debugger [baseline] (64.562 ms) : 0, 64562
Debugger [candidate] (65.753 ms) : 0, 65753
Remote Config [baseline] (572.265 µs) : 0, 572
Remote Config [candidate] (570.281 µs) : 0, 570
Telemetry [baseline] (8.436 ms) : 0, 8436
Telemetry [candidate] (8.524 ms) : 0, 8524
Flare Poller [baseline] (3.578 ms) : 0, 3578
Flare Poller [candidate] (3.619 ms) : 0, 3619
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~3d8f8c0fd6, baseline=1.59.0-SNAPSHOT~671cc8cdbf

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.085 s) : 0, 1084814
Total [baseline] (10.741 s) : 0, 10741479
Agent [candidate] (1.086 s) : 0, 1085761
Total [candidate] (10.77 s) : 0, 10770133
section appsec
Agent [baseline] (1.265 s) : 0, 1265446
Total [baseline] (11.006 s) : 0, 11005748
Agent [candidate] (1.266 s) : 0, 1266021
Total [candidate] (10.982 s) : 0, 10982037
section iast
Agent [baseline] (1.231 s) : 0, 1230830
Total [baseline] (11.163 s) : 0, 11162815
Agent [candidate] (1.232 s) : 0, 1232269
Total [candidate] (11.238 s) : 0, 11238115
section profiling
Agent [baseline] (1.213 s) : 0, 1213085
Total [baseline] (10.93 s) : 0, 10930228
Agent [candidate] (1.205 s) : 0, 1205320
Total [candidate] (10.857 s) : 0, 10856590
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent appsec 1.265 s 180.632 ms (16.7%)
Agent iast 1.231 s 146.017 ms (13.5%)
Agent profiling 1.213 s 128.272 ms (11.8%)
Total tracing 10.741 s -
Total appsec 11.006 s 264.269 ms (2.5%)
Total iast 11.163 s 421.336 ms (3.9%)
Total profiling 10.93 s 188.748 ms (1.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.086 s -
Agent appsec 1.266 s 180.26 ms (16.6%)
Agent iast 1.232 s 146.507 ms (13.5%)
Agent profiling 1.205 s 119.558 ms (11.0%)
Total tracing 10.77 s -
Total appsec 10.982 s 211.904 ms (2.0%)
Total iast 11.238 s 467.982 ms (4.3%)
Total profiling 10.857 s 86.457 ms (0.8%) 
gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~3d8f8c0fd6, baseline=1.59.0-SNAPSHOT~671cc8cdbf

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.174 ms) : 0, 1174
crashtracking [candidate] (1.199 ms) : 0, 1199
BytebuddyAgent [baseline] (650.913 ms) : 0, 650913
BytebuddyAgent [candidate] (651.753 ms) : 0, 651753
GlobalTracer [baseline] (282.775 ms) : 0, 282775
GlobalTracer [candidate] (282.886 ms) : 0, 282886
AppSec [baseline] (32.685 ms) : 0, 32685
AppSec [candidate] (32.75 ms) : 0, 32750
Debugger [baseline] (68.187 ms) : 0, 68187
Debugger [candidate] (68.249 ms) : 0, 68249
Remote Config [baseline] (658.689 µs) : 0, 659
Remote Config [candidate] (627.433 µs) : 0, 627
Telemetry [baseline] (9.022 ms) : 0, 9022
Telemetry [candidate] (8.902 ms) : 0, 8902
Flare Poller [baseline] (3.867 ms) : 0, 3867
Flare Poller [candidate] (3.818 ms) : 0, 3818
section appsec
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (691.288 ms) : 0, 691288
BytebuddyAgent [candidate] (692.198 ms) : 0, 692198
GlobalTracer [baseline] (258.239 ms) : 0, 258239
GlobalTracer [candidate] (258.701 ms) : 0, 258701
IAST [baseline] (24.469 ms) : 0, 24469
IAST [candidate] (24.697 ms) : 0, 24697
AppSec [baseline] (172.736 ms) : 0, 172736
AppSec [candidate] (174.108 ms) : 0, 174108
Debugger [baseline] (68.267 ms) : 0, 68267
Debugger [candidate] (65.881 ms) : 0, 65881
Remote Config [baseline] (752.425 µs) : 0, 752
Remote Config [candidate] (783.255 µs) : 0, 783
Telemetry [baseline] (9.323 ms) : 0, 9323
Telemetry [candidate] (9.284 ms) : 0, 9284
Flare Poller [baseline] (3.73 ms) : 0, 3730
Flare Poller [candidate] (3.765 ms) : 0, 3765
section iast
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.193 ms) : 0, 1193
BytebuddyAgent [baseline] (796.49 ms) : 0, 796490
BytebuddyAgent [candidate] (796.251 ms) : 0, 796251
GlobalTracer [baseline] (257.808 ms) : 0, 257808
GlobalTracer [candidate] (257.809 ms) : 0, 257809
IAST [baseline] (27.136 ms) : 0, 27136
IAST [candidate] (27.144 ms) : 0, 27144
AppSec [baseline] (34.397 ms) : 0, 34397
AppSec [candidate] (34.976 ms) : 0, 34976
Debugger [baseline] (65.571 ms) : 0, 65571
Debugger [candidate] (66.409 ms) : 0, 66409
Remote Config [baseline] (620.69 µs) : 0, 621
Remote Config [candidate] (571.126 µs) : 0, 571
Telemetry [baseline] (8.498 ms) : 0, 8498
Telemetry [candidate] (8.623 ms) : 0, 8623
Flare Poller [baseline] (3.536 ms) : 0, 3536
Flare Poller [candidate] (3.607 ms) : 0, 3607
section profiling
crashtracking [baseline] (1.234 ms) : 0, 1234
crashtracking [candidate] (1.22 ms) : 0, 1220
BytebuddyAgent [baseline] (707.53 ms) : 0, 707530
BytebuddyAgent [candidate] (701.411 ms) : 0, 701411
GlobalTracer [baseline] (221.564 ms) : 0, 221564
GlobalTracer [candidate] (220.784 ms) : 0, 220784
AppSec [baseline] (32.289 ms) : 0, 32289
AppSec [candidate] (31.977 ms) : 0, 31977
Debugger [baseline] (68.634 ms) : 0, 68634
Debugger [candidate] (68.902 ms) : 0, 68902
Remote Config [baseline] (639.727 µs) : 0, 640
Remote Config [candidate] (651.839 µs) : 0, 652
Telemetry [baseline] (8.919 ms) : 0, 8919
Telemetry [candidate] (8.952 ms) : 0, 8952
Flare Poller [baseline] (3.732 ms) : 0, 3732
Flare Poller [candidate] (3.757 ms) : 0, 3757
ProfilingAgent [baseline] (97.881 ms) : 0, 97881
ProfilingAgent [candidate] (97.762 ms) : 0, 97762
Profiling [baseline] (98.462 ms) : 0, 98462
Profiling [candidate] (98.335 ms) : 0, 98335
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/test-merge-base-benchmark
git_commit_date 1768337169 1768342937
git_commit_sha 671cc8c 3d8f8c0
release_version 1.59.0-SNAPSHOT~671cc8cdbf 1.57.0-SNAPSHOT~3d8f8c0fd6
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1768345456 1768345456
ci_job_id 1348388677 1348388677
ci_pipeline_id 90588023 90588023
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-zh64hyin 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-zh64hyin 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 3 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:tracing:high_load worse
[+36.102µs; +98.574µs] or [+2.463%; +6.724%]		 unstable
[+222.174µs; +798.857µs] or [+5.893%; +21.189%]		 unstable
[-397.077op/s; +86.577op/s] or [-15.456%; +3.370%]		 1.533ms 4.281ms 2413.875op/s 1.466ms 3.770ms 2569.125op/s
scenario:load:insecure-bank:iast_FULL:high_load unsure
[-361.326µs; -104.858µs] or [-6.761%; -1.962%]		 better
[-1032.550µs; -279.167µs] or [-8.026%; -2.170%]		 unstable
[-39.196op/s; +105.758op/s] or [-5.090%; +13.734%]		 5.111ms 12.209ms 803.344op/s 5.344ms 12.865ms 770.062op/s
scenario:load:petclinic:profiling:high_load worse
[+0.973ms; +2.102ms] or [+5.359%; +11.582%]		 worse
[+0.811ms; +2.916ms] or [+2.747%; +9.875%]		 unstable
[-41.450op/s; +5.763op/s] or [-16.451%; +2.287%]		 19.687ms 31.392ms 234.125op/s 18.149ms 29.529ms 251.969op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~3d8f8c0fd6, baseline=1.59.0-SNAPSHOT~671cc8cdbf
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.251 ms) : 1239, 1264
.   : milestone, 1251,
iast (3.236 ms) : 3198, 3274
.   : milestone, 3236,
iast_FULL (6.006 ms) : 5945, 6066
.   : milestone, 6006,
iast_GLOBAL (3.639 ms) : 3578, 3700
.   : milestone, 3639,
profiling (2.069 ms) : 2049, 2088
.   : milestone, 2069,
tracing (1.746 ms) : 1732, 1761
.   : milestone, 1746,
section candidate
no_agent (1.18 ms) : 1169, 1191
.   : milestone, 1180,
iast (3.297 ms) : 3252, 3343
.   : milestone, 3297,
iast_FULL (5.753 ms) : 5696, 5810
.   : milestone, 5753,
iast_GLOBAL (3.728 ms) : 3669, 3786
.   : milestone, 3728,
profiling (2.038 ms) : 2019, 2056
.   : milestone, 2038,
tracing (1.864 ms) : 1848, 1880
.   : milestone, 1864,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.251 ms [1.239 ms, 1.264 ms] -
iast 3.236 ms [3.198 ms, 3.274 ms] 1.985 ms (158.7%)
iast_FULL 6.006 ms [5.945 ms, 6.066 ms] 4.755 ms (380.0%)
iast_GLOBAL 3.639 ms [3.578 ms, 3.7 ms] 2.388 ms (190.9%)
profiling 2.069 ms [2.049 ms, 2.088 ms] 817.453 µs (65.3%)
tracing 1.746 ms [1.732 ms, 1.761 ms] 495.293 µs (39.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.18 ms [1.169 ms, 1.191 ms] -
iast 3.297 ms [3.252 ms, 3.343 ms] 2.118 ms (179.5%)
iast_FULL 5.753 ms [5.696 ms, 5.81 ms] 4.574 ms (387.7%)
iast_GLOBAL 3.728 ms [3.669 ms, 3.786 ms] 2.548 ms (216.0%)
profiling 2.038 ms [2.019 ms, 2.056 ms] 858.118 µs (72.7%)
tracing 1.864 ms [1.848 ms, 1.88 ms] 684.049 µs (58.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~3d8f8c0fd6, baseline=1.59.0-SNAPSHOT~671cc8cdbf
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.166 ms) : 16993, 17339
.   : milestone, 17166,
appsec (18.592 ms) : 18406, 18778
.   : milestone, 18592,
code_origins (18.096 ms) : 17915, 18277
.   : milestone, 18096,
iast (17.471 ms) : 17295, 17647
.   : milestone, 17471,
profiling (18.522 ms) : 18337, 18707
.   : milestone, 18522,
tracing (17.808 ms) : 17633, 17982
.   : milestone, 17808,
section candidate
no_agent (17.747 ms) : 17571, 17923
.   : milestone, 17747,
appsec (18.268 ms) : 18084, 18451
.   : milestone, 18268,
code_origins (18.593 ms) : 18395, 18790
.   : milestone, 18593,
iast (17.541 ms) : 17364, 17718
.   : milestone, 17541,
profiling (19.941 ms) : 19735, 20148
.   : milestone, 19941,
tracing (17.701 ms) : 17525, 17877
.   : milestone, 17701,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.166 ms [16.993 ms, 17.339 ms] -
appsec 18.592 ms [18.406 ms, 18.778 ms] 1.426 ms (8.3%)
code_origins 18.096 ms [17.915 ms, 18.277 ms] 929.924 µs (5.4%)
iast 17.471 ms [17.295 ms, 17.647 ms] 304.97 µs (1.8%)
profiling 18.522 ms [18.337 ms, 18.707 ms] 1.356 ms (7.9%)
tracing 17.808 ms [17.633 ms, 17.982 ms] 641.765 µs (3.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.747 ms [17.571 ms, 17.923 ms] -
appsec 18.268 ms [18.084 ms, 18.451 ms] 520.55 µs (2.9%)
code_origins 18.593 ms [18.395 ms, 18.79 ms] 845.257 µs (4.8%)
iast 17.541 ms [17.364 ms, 17.718 ms] -206.648 µs (-1.2%)
profiling 19.941 ms [19.735 ms, 20.148 ms] 2.194 ms (12.4%)
tracing 17.701 ms [17.525 ms, 17.877 ms] -46.286 µs (-0.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master sarahchen6/test-merge-base-benchmark
git_commit_date 1768337169 1768342937
git_commit_sha 671cc8c 3d8f8c0
release_version 1.59.0-SNAPSHOT~671cc8cdbf 1.57.0-SNAPSHOT~3d8f8c0fd6
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1768345155 1768345155
ci_job_id 1348388678 1348388678
ci_pipeline_id 90588023 90588023
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-qu4buoyo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-qu4buoyo 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 1 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.365ms; -1.026ms] or [-37.164%; -27.936%]		 2.477ms 3.672ms
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~3d8f8c0fd6, baseline=1.59.0-SNAPSHOT~671cc8cdbf
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (3.672 ms) : 3455, 3888
.   : milestone, 3672,
iast (2.22 ms) : 2155, 2285
.   : milestone, 2220,
iast_GLOBAL (2.255 ms) : 2190, 2320
.   : milestone, 2255,
profiling (2.483 ms) : 2319, 2646
.   : milestone, 2483,
tracing (2.035 ms) : 1983, 2086
.   : milestone, 2035,
section candidate
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (2.477 ms) : 2424, 2529
.   : milestone, 2477,
iast (2.214 ms) : 2149, 2279
.   : milestone, 2214,
iast_GLOBAL (2.258 ms) : 2192, 2323
.   : milestone, 2258,
profiling (2.054 ms) : 2001, 2106
.   : milestone, 2054,
tracing (2.054 ms) : 2002, 2105
.   : milestone, 2054,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 3.672 ms [3.455 ms, 3.888 ms] 2.199 ms (149.2%)
iast 2.22 ms [2.155 ms, 2.285 ms] 746.428 µs (50.7%)
iast_GLOBAL 2.255 ms [2.19 ms, 2.32 ms] 781.875 µs (53.1%)
profiling 2.483 ms [2.319 ms, 2.646 ms] 1.01 ms (68.5%)
tracing 2.035 ms [1.983 ms, 2.086 ms] 561.606 µs (38.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.477 ms [1.465 ms, 1.488 ms] -
appsec 2.477 ms [2.424 ms, 2.529 ms] 999.753 µs (67.7%)
iast 2.214 ms [2.149 ms, 2.279 ms] 737.023 µs (49.9%)
iast_GLOBAL 2.258 ms [2.192 ms, 2.323 ms] 780.778 µs (52.9%)
profiling 2.054 ms [2.001 ms, 2.106 ms] 576.845 µs (39.1%)
tracing 2.054 ms [2.002 ms, 2.105 ms] 576.931 µs (39.1%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~3d8f8c0fd6, baseline=1.59.0-SNAPSHOT~671cc8cdbf
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.979 s) : 14979000, 14979000
.   : milestone, 14979000,
appsec (14.83 s) : 14830000, 14830000
.   : milestone, 14830000,
iast (18.54 s) : 18540000, 18540000
.   : milestone, 18540000,
iast_GLOBAL (17.896 s) : 17896000, 17896000
.   : milestone, 17896000,
profiling (14.959 s) : 14959000, 14959000
.   : milestone, 14959000,
tracing (14.636 s) : 14636000, 14636000
.   : milestone, 14636000,
section candidate
no_agent (15.549 s) : 15549000, 15549000
.   : milestone, 15549000,
appsec (14.737 s) : 14737000, 14737000
.   : milestone, 14737000,
iast (18.39 s) : 18390000, 18390000
.   : milestone, 18390000,
iast_GLOBAL (17.792 s) : 17792000, 17792000
.   : milestone, 17792000,
profiling (14.883 s) : 14883000, 14883000
.   : milestone, 14883000,
tracing (14.673 s) : 14673000, 14673000
.   : milestone, 14673000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.979 s [14.979 s, 14.979 s] -
appsec 14.83 s [14.83 s, 14.83 s] -149.0 ms (-1.0%)
iast 18.54 s [18.54 s, 18.54 s] 3.561 s (23.8%)
iast_GLOBAL 17.896 s [17.896 s, 17.896 s] 2.917 s (19.5%)
profiling 14.959 s [14.959 s, 14.959 s] -20.0 ms (-0.1%)
tracing 14.636 s [14.636 s, 14.636 s] -343.0 ms (-2.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.549 s [15.549 s, 15.549 s] -
appsec 14.737 s [14.737 s, 14.737 s] -812.0 ms (-5.2%)
iast 18.39 s [18.39 s, 18.39 s] 2.841 s (18.3%)
iast_GLOBAL 17.792 s [17.792 s, 17.792 s] 2.243 s (14.4%)
profiling 14.883 s [14.883 s, 14.883 s] -666.0 ms (-4.3%)
tracing 14.673 s [14.673 s, 14.673 s] -876.0 ms (-5.6%)

@sarahchen6
Copy link
Contributor Author

sarahchen6 commented Dec 19, 2025

Now testing https://github.com/DataDog/apm-sdks-benchmarks/pull/45

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

tag: do not merge Do not merge changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sarahchen6