Add a HistFactory-style likelihood benchmark generator - #2002
Open
guitargeek wants to merge 1 commit into
Open
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
guitargeek
force-pushed
the
histfactory-benchmark
branch
from
August 21, 2026 12:43
d9b56f9 to
ba83b9f
Compare
benchmark/histfactory.py generates a scaled-down proxy of the code RooFit emits for a real 1470-parameter ATLAS likelihood, compiles it with the clad plugin, and runs it, printing the compilation time in the same table format as the benchmark executable's primal/gradient/hessian timings. The .cpp is a transient artifact in a temp directory (kept with --cpp); the compiler is read from the clad build tree's CMakeCache, so `histfactory.py --clad-build <dir>` is all that is needed to compare clad revisions. The generated model reproduces the statement patterns of the full likelihood at 1/16 scale: 91 parameters, 4 channels with straight-line per-sample yields (flexibleInterp over nuisance subsets, product factors, a lumi parameter), bin loops with uniformBinNumber shape lookups feeding binned nll terms, and a constraint block (gaussian/gaussianIntegral and poisson/poissonIntegral ratios into constraintSum). The driver validates the gradient against central differences of the primal and sampled hessian rows against central differences of the gradient. Measured against the full benchmark, the mini preserves the relative cost profile (gradient ~7x primal vs 12x; hessian-vector product ~2.2x gradient vs 2.9x) while compiling in seconds instead of minutes. benchmark/HistFactoryMathFuncs.h carries the RooFit::Detail::MathFuncs subset the model calls, with TMath/ROOT::Math replaced by std:: equivalents, the Cephes incomplete gamma behind poissonIntegral, and its custom clad derivatives copied verbatim from ROOT's Math/CladDerivator.h. The copied code is kept verbatim on purpose -- including the Cephes do-while loops, the ROOT naming and formatting, and the pullbacks' snapshots of not-yet-initialized values (their restores are dead by construction, but they trip the uninitialized-use compiler diagnostics that clang-tidy surfaces under -Wall) -- so the file stays diffable against the originals. clang-format off/on markers and a blanket NOLINTBEGIN/END block shield exactly that region; an explicit check list proved futile because the surfaced compiler diagnostics vary with the compilation flags. The benchmark's own glue (includes, header guard) conforms. constraintSum, the one function carrying `#pragma clad checkpoint loop`, is emitted into the generated main file instead: clad currently mis-attributes a checkpoint pragma living in an included header, because the planner selects pragmas by raw SourceLocation order, which does not match translation-unit order across files. The benchmark registers as the CTest test clad-HistFactory (labels "benchmark;short", RUN_SERIAL) under CLAD_ENABLE_BENCHMARKS. Because one of its metrics is the compilation time of the generated code, it compiles at test time via the script instead of going through CB_ADD_GBENCHMARK; CMake passes the tested clang explicitly through the new --compiler option, since the CMakeCache fallback would pick the host compiler on builds configured with gcc.
guitargeek
force-pushed
the
histfactory-benchmark
branch
from
August 21, 2026 12:57
ba83b9f to
2027578
Compare
Contributor
|
clang-tidy review says "All clean, LGTM! 👍" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
benchmark/histfactory.py generates a scaled-down proxy of the code RooFit emits for a real 1470-parameter ATLAS likelihood, compiles it with the clad plugin, and runs it, printing the compilation time in the same table format as the benchmark executable's primal/gradient/hessian timings. The .cpp is a transient artifact in a temp directory (kept with --cpp); the compiler is read from the clad build tree's CMakeCache, so
histfactory.py --clad-build <dir>is all that is needed to compare clad revisions.The generated model reproduces the statement patterns of the full likelihood at 1/16 scale: 91 parameters, 4 channels with straight-line per-sample yields (flexibleInterp over nuisance subsets, product factors, a lumi parameter), bin loops with uniformBinNumber shape lookups feeding binned nll terms, and a constraint block (gaussian/gaussianIntegral and poisson/poissonIntegral ratios into constraintSum). The driver validates the gradient against central differences of the primal and sampled hessian rows against central differences of the gradient. Measured against the full benchmark, the mini preserves the relative cost profile (gradient ~7x primal vs 12x; hessian-vector product ~2.2x gradient vs 2.9x) while compiling in seconds instead of minutes.
benchmark/HistFactoryMathFuncs.h carries the RooFit::Detail::MathFuncs subset the model calls, with TMath/ROOT::Math replaced by std:: equivalents, the Cephes incomplete gamma behind poissonIntegral, and its custom clad derivatives copied verbatim from ROOT's Math/CladDerivator.h (kept verbatim on purpose, including the Cephes do-while loops and the ROOT formatting, which clang-format off/on markers protect so the file stays diffable against the originals).
constraintSum, the one function carrying
#pragma clad checkpoint loop, is emitted into the generated main file instead: clad currently mis-attributes a checkpoint pragma living in an included header, because the planner selects pragmas by raw SourceLocation order, which does not match translation-unit order across files.The benchmark registers as the CTest test clad-HistFactory (labels "benchmark;short", RUN_SERIAL) under CLAD_ENABLE_BENCHMARKS. Because one of its metrics is the compilation time of the generated code, it compiles at test time via the script instead of going through CB_ADD_GBENCHMARK; CMake passes the tested clang explicitly through the new --compiler option, since the CMakeCache fallback would pick the host compiler on builds configured with gcc.
Output in the CI looks as follows:
[7](https://github.com/vgvassilev/clad/actions/runs/32480829289/job/96766612084?pr=2002#step:30:628) Start 6: clad-HistFactory 6: Test command: /usr/bin/python3.12 "/home/runner/work/clad/clad/benchmark/histfactory.py" "--clad-build" "/home/runner/work/clad/clad/obj" "--compiler" "/usr/lib/llvm-20/bin/clang++" 6: Working Directory: /home/runner/work/clad/clad/obj/benchmark 6: Test timeout computed to be: 1200 6: clad build : /home/runner/work/clad/clad/obj 6: compilation : 22.867 s 6: n params : 91 6: function value: 4413.41174684329 6: primal : 0.023 ms/call 6: gradient : 0.184 ms/call (8.1x primal) 6: hessian : 37.619 ms/call (204.9x gradient) 6: gradient vs numerical: worst rel. deviation 1.15e-06 (at index 10), 0 outside tolerance 6: hessian vs numerical (8 sampled rows): worst rel. deviation 5.62e-10 (at [18,39]), 0 outside tolerance 6: hessian asymmetry: worst rel. deviation 9.37e-16 6: validation passed 6/11 Test #6: clad-HistFactory ................. Passed 23.84 sec