Add BuildVariableGradientDag + mean-value-form range tightening - #166
Open
foolnotion wants to merge 12 commits into
Open
Add BuildVariableGradientDag + mean-value-form range tightening#166foolnotion wants to merge 12 commits into
foolnotion wants to merge 12 commits into
Conversation
… values Complements BuildJacobianDag (which differentiates w.r.t. optimizable coefficient weights) with a gradient w.r.t. each distinct input variable's underlying value, needed for mean-value-form interval range tightening. Deriv() is refactored to share its structural chain-rule logic between both differentiation modes via a DerivTarget discriminator, rather than duplicating the ~200-line recursive engine.
Combines a midpoint point-evaluation with BuildVariableGradientDag's interval-valued gradient (mean-value/first-order Taylor extension), intersected with the naive IntervalEvaluator enclosure. Mitigates interval arithmetic's dependency problem for trees where an input variable recurs (e.g. x - x*x), without needing pappus's bisection.
An unregistered symbolic-diff op (e.g. Abs) and a genuinely zero partial both surface as the same "zero" sentinel, so treating that as a real zero contribution let the mean-value term silently collapse the true range - now falls back to the naive enclosure whenever any variable's gradient can't be determined. Also, BuildVariableGradientDag now takes an optional coeff span so an optimizable variable's own weight is read live rather than baked from a potentially stale Node::Value.
The per-column NoGrad check alone missed a variable with only SOME occurrences behind an undifferentiated op (e.g. X + abs(X)): the root comes back nonzero but understates the true partial, which the earlier fix didn't catch. Replaced with a structural pre-check over the whole tree. Also trimmed comments across this branch's changes to bare facts.
Runs a GP evolution on Poly-10 and compares naive vs. TightenRange enclosure width across the final population, checking soundness against dense point sampling. Confirms mean-value form's benefit scales with domain width (1% at full range, ~13% at 1% width around the midpoint) - a real, multi-variable GP model's full input domain sees only modest improvement, since mean-value form is a local (first-order) method.
Recursively bisects on the variable whose gradient interval straddles zero the most, unioning both sub-box results and intersecting with the whole-box TightenRange result - never less sound, never worse. On Poly-10's real evolved population this raises average width reduction from 1.17% to 7.75% at full domain width, at ~960x naive's cost (still ~1ms/tree in absolute terms).
19 real physics formulas from Kronberger, de Franca, Burlacu, Haider & Kommenda (arXiv:2103.15624) - 16 from the AI Feynman database with its published domains, 3 fluid-dynamics-engineering ones reconstructed approximately (exact domains are in that paper's supplementary material, not available here). Several have genuine repeated-variable structure (I.9.18, I.15.3x/t, Flow psi). Confirms the Poly-10 finding: plain TightenRange gives ~0% improvement at these domain widths on every real formula, while TightenRangeBisected recovers real gains (up to 78% on I.41.16). 3 formulas hit a pre-existing IntervalEvaluator Pow limitation (interval^interval always domain-restricts to a non-negative base, even for a constant integer exponent) - documented, not a TightenRange bug.
IsSymbolicallyDifferentiable was checking IsDivision()/IsPow() without regard to arity, but Deriv() only handles Div at arity 1-2 (Zero for higher) and Pow only at exactly arity 2 (indexes children[1] unconditionally otherwise). Both diverge from the structural pre-check in the same unsound direction the earlier fixes were meant to close. Also changed the final arity>=3 fallback from true to false, matching Deriv()'s own fallthrough to Zero. Found independently by both glm-5.2 (via pi) and gpt-5.5 (via opencode) in a second review round; glm also flagged an untested (but sound) interaction between TightenRangeBisected's variable selection and an undifferentiated op, now covered by a regression test.
Keyed by structural hash (via a caller-supplied Zobrist instance) mixed with the actual coeff values and domain bounds - unlike the fitness cache's deliberately coefficient-independent key, an interval result genuinely depends on exact coefficient values, so this must be exact-match to stay sound. TightenRange/TightenRangeBisected take an optional cache parameter; TightenRangeBisected's own recursive TightenRange calls benefit automatically. On a real Poly-10 population, re-evaluating the same 400 individuals through a warm cache is ~1777x faster than the cold pass (983us -> 0.58us per tree).
…xponent IntervalEvaluator dispatched Pow through the general interval^interval overload unconditionally, which domain-restricts the base to non-negative even for a literal even-integer exponent, returning an empty interval for e.g. (negative-subtraction)^2. Detect a degenerate exponent interval and dispatch through pow(interval, Scalar) instead, which already handles this correctly. Unblocks I.9.18, I.32.17, and Jackson 2.11 in the Feynman benchmark suite.
foolnotion
changed the title
There was a problem hiding this comment.
Pull request overview
Adds first-order symbolic differentiation w.r.t. input variable values (distinct variable hashes) and uses it to implement mean-value-form interval range tightening (with optional bisection + caching) to improve range enclosures over naive interval evaluation.
Changes:
- Extend the existing symbolic
Deriv()engine with aDerivTargetdiscriminator to support both coefficient- and variable-value differentiation, and exposeBuildVariableGradientDag. - Add
TightenRangeandTightenRangeBisected(plusRangeCache) to compute tighter interval enclosures using gradient-based mean-value form. - Add extensive validation/coverage: new unit tests, Feynman benchmark suite, an example validator, and a fix for
Powinterval evaluation with negative bases when the exponent is a constant singleton interval.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/source/implementation/tree_diff.cpp | Adds structural + finite-difference tests for BuildVariableGradientDag. |
| test/source/implementation/range_tightening.cpp | Adds unit tests for TightenRange, TightenRangeBisected, and RangeCache. |
| test/source/implementation/pappus_backend.cpp | Adds a regression test for Pow interval evaluation with negative base and constant even exponent. |
| test/source/implementation/feynman_benchmarks.cpp | Adds a benchmark-style test suite validating tightening soundness/tightness on physics formulas. |
| test/CMakeLists.txt | Wires new test sources into the test target. |
| source/interpreter/range_tightening.cpp | Implements tightening + bisection + cache keyed by structural hash, coeffs, and domains. |
| source/interpreter/interval_evaluator.cpp | Fixes Pow interval dispatch for singleton exponent intervals. |
| source/core/tree_diff.cpp | Introduces DerivTarget and BuildVariableGradientDag (variable-value differentiation). |
| include/operon/interpreter/range_tightening.hpp | Declares TightenRange, TightenRangeBisected, and RangeCache. |
| include/operon/core/tree_diff.hpp | Declares VariableGradientDag and BuildVariableGradientDag. |
| example/range_tightening_validation.cpp | Adds an end-to-end validation example against evolved trees + cache warm/cold comparison. |
| example/CMakeLists.txt | Registers the new example target. |
| CMakeLists.txt | Adds the new tightening implementation to the library build. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+4
to
+10
| #include "operon/interpreter/range_tightening.hpp" | ||
|
|
||
| #include <bit> | ||
| #include <limits> | ||
|
|
||
| #include "operon/core/tree_diff.hpp" | ||
|
|
Comment on lines
+7
to
+13
| #include <gsl/pointers> | ||
|
|
||
| #include "operon/core/tree.hpp" | ||
| #include "operon/core/types.hpp" | ||
| #include "operon/hash/zobrist.hpp" | ||
| #include "operon/interpreter/interval_evaluator.hpp" | ||
| #include "operon/operon_export.hpp" |
Comment on lines
+4
to
+8
| #include <catch2/catch_approx.hpp> | ||
| #include <catch2/catch_test_macros.hpp> | ||
|
|
||
| #include <random> | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds symbolic differentiation of a tree with respect to an input variable's value (
BuildVariableGradientDag), as opposed to the existingBuildJacobianDag, which differentiates with respect to optimizable coefficients. Both share the sameDeriv()engine via aDerivTargetdiscriminator, so this doesn't duplicate the chain-rule logic.On top of that, adds mean-value-form interval range tightening built from the gradient DAG plus
IntervalEvaluator:TightenRange: given a domain box, computes a tighter output-range enclosure than naive interval evaluation by combining a midpoint value with an interval-bounded gradient term.TightenRangeBisected: extends this with sign-crossing-driven domain bisection for cases where a single mean-value bound is too loose.RangeCache: Zobrist-hash-keyed caching that makes repeated tightening calls over the same tree/domain cheap, a large warm-cache speedup over the naive case.Includes a Feynman-benchmark suite (extracted from the shape-constraints paper's problem set) validating the tightened bounds against ground truth across a range of realistic expressions, plus a fix for
Pow's interval evaluation with a negative base and constant integer exponent found during that validation.This is groundwork for the shape-constraint enforcement work in the follow-up PR:
BuildVariableGradientDagis the derivative engine that work's affine-bound checking is built on.