Revert PR 2516 to run benchmarks.

[Nikhil-Manglore]

This is a test PR where I have reverted PR #2516 in order to run benchmarks and see how much performance improves.

[codecov]

Codecov Report

❌ Patch coverage is 85.34161% with 236 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.43%. Comparing base (ac5e44c) to head (6d48bb6).
⚠️ Report is 8 commits behind head on unstable.

Files with missing lines	Patch %	Lines
src/sentinel.c	0.00%	74 Missing ⚠️
src/module.c	21.73%	72 Missing ⚠️
src/debug.c	82.88%	19 Missing ⚠️
src/cluster_legacy.c	83.82%	11 Missing ⚠️
src/object.c	92.48%	10 Missing ⚠️
src/defrag.c	69.56%	7 Missing ⚠️
src/rdb.c	94.38%	5 Missing ⚠️
src/scripting_engine.c	58.33%	5 Missing ⚠️
src/t_list.c	90.90%	5 Missing ⚠️
src/replication.c	90.69%	4 Missing ⚠️
... and 15 more

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #3373      +/-   ##
============================================
- Coverage     74.53%   74.43%   -0.11%     
============================================
  Files           130      130              
  Lines         72731    72682      -49     
============================================
- Hits          54208    54098     -110     
- Misses        18523    18584      +61     

Files with missing lines	Coverage Δ
src/bitops.c	95.93% <100.00%> (-0.30%)	⬇️
src/cluster_migrateslots.c	92.26% <100.00%> (-0.01%)	⬇️
src/cluster_slot_stats.c	93.75% <100.00%> (ø)
src/commandlog.c	96.58% <100.00%> (ø)
src/eval.c	91.40% <100.00%> (-0.03%)	⬇️
src/expire.c	98.12% <100.00%> (+0.80%)	⬆️
src/functions.c	96.57% <100.00%> (ø)
src/latency.c	83.29% <100.00%> (ø)
src/lazyfree.c	88.38% <100.00%> (ø)
src/logreqres.c	72.72% <ø> (ø)
... and 36 more

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[hpatro]

Previous benchmark run failed while running a git command something to debug later @roshkhatri.

https://github.com/valkey-io/valkey/actions/runs/23224976768

[Nikhil-Manglore]

Benchmark failed again for the same reason

[sarthakaggarwal97]

@Nikhil-Manglore run it on the weekend :D

[roshkhatri]

Found the issue, fixing it

[github-actions]

Benchmark ran on this commit: 6d48bb6

Benchmark Comparison: unstable vs 55adef0 (averaged) - rps metrics

Run Summary:

• unstable: 80 total runs, 16 configurations (avg 5.00 runs per config)
• 55adef0: 80 total runs, 16 configurations (avg 5.00 runs per config)

Statistical Notes:

• CI99%: 99% Confidence Interval - range where the true population mean is likely to fall
• PI99%: 99% Prediction Interval - range where a single future observation is likely to fall
• CV: Coefficient of Variation - relative variability (σ/μ × 100%)

Note: Values with (n=X, σ=Y, CV=Z%, CI99%=±W%, PI99%=±V%) indicate averages from X runs with standard deviation Y, coefficient of variation Z%, 99% confidence interval margin of error ±W% of the mean, and 99% prediction interval margin of error ±V% of the mean. CI bounds [A, B] and PI bounds [C, D] show the actual interval ranges.

Configuration:

• architecture: aarch64
• benchmark_mode: duration
• clients: 1600
• cluster_mode: False
• data_size: 16
• duration: 180
• tls: False
• valkey_benchmark_threads: 90
• warmup: 30

Command	Metric	Pipeline	io_threads	unstable	55adef0	Diff	% Change
GET	rps	1	1	230836.604 (n=5, σ=770.455, CV=0.33%, CI99%=±0.687%, PI99%=±1.683%, CI[229250.227, 232422.981], PI[226950.790, 234722.418])	229988.410 (n=5, σ=1528.593, CV=0.66%, CI99%=±1.369%, PI99%=±3.352%, CI[226841.015, 233135.805], PI[222278.899, 237697.921])	-848.194	-0.367%
GET	rps	1	9	1509499.276 (n=5, σ=14562.443, CV=0.96%, CI99%=±1.986%, PI99%=±4.866%, CI[1479515.004, 1539483.548], PI[1436053.109, 1582945.443])	1486512.976 (n=5, σ=14250.579, CV=0.96%, CI99%=±1.974%, PI99%=±4.835%, CI[1457170.836, 1515855.116], PI[1414639.705, 1558386.247])	-22986.300	-1.523%
GET	rps	10	1	1248766.850 (n=5, σ=4148.950, CV=0.33%, CI99%=±0.684%, PI99%=±1.676%, CI[1240224.105, 1257309.595], PI[1227841.485, 1269692.215])	1277882.774 (n=5, σ=5774.497, CV=0.45%, CI99%=±0.930%, PI99%=±2.279%, CI[1265993.005, 1289772.543], PI[1248758.907, 1307006.641])	29115.924	+2.332%
GET	rps	10	9	2877336.250 (n=5, σ=22344.002, CV=0.78%, CI99%=±1.599%, PI99%=±3.917%, CI[2831329.641, 2923342.859], PI[2764643.534, 2990028.966])	3045224.750 (n=5, σ=30198.875, CV=0.99%, CI99%=±2.042%, PI99%=±5.002%, CI[2983044.850, 3107404.650], PI[2892915.722, 3197533.778])	167888.500	+5.835%
SET	rps	1	1	220427.540 (n=5, σ=1898.386, CV=0.86%, CI99%=±1.773%, PI99%=±4.344%, CI[216518.737, 224336.343], PI[210852.966, 230002.114])	221718.450 (n=5, σ=1111.859, CV=0.50%, CI99%=±1.033%, PI99%=±2.529%, CI[219429.116, 224007.784], PI[216110.750, 227326.150])	1290.910	+0.586%
SET	rps	1	9	1477702.528 (n=5, σ=8060.226, CV=0.55%, CI99%=±1.123%, PI99%=±2.751%, CI[1461106.411, 1494298.645], PI[1437050.511, 1518354.545])	1482299.424 (n=5, σ=12180.359, CV=0.82%, CI99%=±1.692%, PI99%=±4.144%, CI[1457219.897, 1507378.951], PI[1420867.379, 1543731.469])	4596.896	+0.311%
SET	rps	10	1	1044501.376 (n=5, σ=3228.102, CV=0.31%, CI99%=±0.636%, PI99%=±1.559%, CI[1037854.668, 1051148.084], PI[1028220.334, 1060782.418])	1075969.450 (n=5, σ=7425.333, CV=0.69%, CI99%=±1.421%, PI99%=±3.481%, CI[1060680.586, 1091258.314], PI[1038519.535, 1113419.365])	31468.074	+3.013%
SET	rps	10	9	1950431.948 (n=5, σ=19372.737, CV=0.99%, CI99%=±2.045%, PI99%=±5.010%, CI[1910543.214, 1990320.682], PI[1852724.904, 2048138.992])	2028836.476 (n=5, σ=15429.384, CV=0.76%, CI99%=±1.566%, PI99%=±3.836%, CI[1997067.161, 2060605.791], PI[1951017.865, 2106655.087])	78404.528	+4.020%

Configuration:

• architecture: aarch64
• benchmark_mode: duration
• clients: 1600
• cluster_mode: False
• data_size: 96
• duration: 180
• tls: False
• valkey_benchmark_threads: 90
• warmup: 30

Command	Metric	Pipeline	io_threads	unstable	55adef0	Diff	% Change
GET	rps	1	1	218481.598 (n=5, σ=486.767, CV=0.22%, CI99%=±0.459%, PI99%=±1.124%, CI[217479.338, 219483.858], PI[216026.573, 220936.623])	222279.564 (n=5, σ=1400.796, CV=0.63%, CI99%=±1.298%, PI99%=±3.178%, CI[219395.305, 225163.823], PI[215214.601, 229344.527])	3797.966	+1.738%
GET	rps	1	9	1458614.774 (n=5, σ=16459.738, CV=1.13%, CI99%=±2.323%, PI99%=±5.691%, CI[1424723.946, 1492505.602], PI[1375599.539, 1541630.009])	1465447.876 (n=5, σ=13922.774, CV=0.95%, CI99%=±1.956%, PI99%=±4.792%, CI[1436780.693, 1494115.059], PI[1395227.904, 1535667.848])	6833.102	+0.468%
GET	rps	10	1	1176300.798 (n=5, σ=8832.983, CV=0.75%, CI99%=±1.546%, PI99%=±3.787%, CI[1158113.564, 1194488.032], PI[1131751.354, 1220850.242])	1209124.124 (n=5, σ=1863.570, CV=0.15%, CI99%=±0.317%, PI99%=±0.777%, CI[1205287.008, 1212961.240], PI[1199725.149, 1218523.099])	32823.326	+2.790%
GET	rps	10	9	2225084.950 (n=5, σ=14413.237, CV=0.65%, CI99%=±1.334%, PI99%=±3.267%, CI[2195407.897, 2254762.003], PI[2152391.312, 2297778.588])	2276866.600 (n=5, σ=9640.282, CV=0.42%, CI99%=±0.872%, PI99%=±2.135%, CI[2257017.125, 2296716.075], PI[2228245.515, 2325487.685])	51781.650	+2.327%
SET	rps	1	1	211645.532 (n=5, σ=1185.584, CV=0.56%, CI99%=±1.153%, PI99%=±2.825%, CI[209204.399, 214086.665], PI[205666.001, 217625.063])	214956.870 (n=5, σ=635.266, CV=0.30%, CI99%=±0.609%, PI99%=±1.491%, CI[213648.848, 216264.892], PI[211752.884, 218160.856])	3311.338	+1.565%
SET	rps	1	9	1437004.100 (n=5, σ=4021.268, CV=0.28%, CI99%=±0.576%, PI99%=±1.411%, CI[1428724.254, 1445283.946], PI[1416722.703, 1457285.497])	1439990.354 (n=5, σ=11333.947, CV=0.79%, CI99%=±1.621%, PI99%=±3.970%, CI[1416653.601, 1463327.107], PI[1382827.217, 1497153.491])	2986.254	+0.208%
SET	rps	10	1	1031890.012 (n=5, σ=3631.320, CV=0.35%, CI99%=±0.725%, PI99%=±1.775%, CI[1024413.074, 1039366.950], PI[1013575.330, 1050204.694])	1067720.350 (n=5, σ=1923.787, CV=0.18%, CI99%=±0.371%, PI99%=±0.909%, CI[1063759.245, 1071681.455], PI[1058017.664, 1077423.036])	35830.338	+3.472%
SET	rps	10	9	1829924.124 (n=5, σ=22142.073, CV=1.21%, CI99%=±2.491%, PI99%=±6.103%, CI[1784333.289, 1875514.959], PI[1718249.842, 1941598.406])	1939694.026 (n=5, σ=20584.622, CV=1.06%, CI99%=±2.185%, PI99%=±5.352%, CI[1897310.004, 1982078.048], PI[1835874.799, 2043513.253])	109769.902	+5.999%

[sarthakaggarwal97]

Wow! The gains are back with the revert!