Age specialization: per-age regime functions/constraints and age-dependent continuous-state grids#398
Open
hmgaudecker wants to merge 44 commits into
Open
Age specialization: per-age regime functions/constraints and age-dependent continuous-state grids#398hmgaudecker wants to merge 44 commits into
hmgaudecker wants to merge 44 commits into
Conversation
…river) Two tests on the 4-CPU distributed seam: - test_distributed_solve_matches_single_device_per_type: the F6 correctness contract — a sharded solve must equal the single-device solve per type slice. Green now; the co-map fix must preserve it (catches a wrong device-local index). - test_distributed_solve_kernel_does_not_all_gather_continuation_v: the red driver — the backward-induction kernel currently all-gathers the continuation V across the type shard; it must read only its device-local slice. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
vmap_1d gains co_mapped_in_axes: an optional per-argument in_axes override so a pytree argument's leading axis can be mapped in lockstep with the mapped variables. The backward-induction co-map uses it to slice each next_regime_to_V_arr leaf to the device-local type, so the continuation-V interpolation reads only its own shard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixed, distributed states (e.g. a permanent type sharded one block per device) never transition, so a regime's continuation value depends only on its own slice of the next-period V-array. The grid-search solve kernel now co-maps each such state with the matching axis of every next_regime_to_V_arr leaf that carries it: an outer vmap peels the leading axis off both the state grid and the continuation V, so the interpolation reads only the device-local slice and XLA inserts no all-gather of the full V-array onto every device. - max_Q_over_a splits the co-mapped (leading) states from the inner productmap and wraps it in per-state co-map vmaps; the V-interpolator drops those coordinates and Q_and_F omits the sliced next-states. - processing detects the co-mappable states (distributed and identity-transition) and builds per-state, per-leaf in_axes so a target regime that prunes the state keeps its full leaf. - Simulation is unchanged: it keeps the full continuation V (subjects are not type-aligned across devices), so the co-map applies to the solve path only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…seam (#385) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ectations Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Add 'age' entry to initial_conditions in epstein_zin.md Run section - Document certainty_equivalent parameter in get_model docstring - Add docstrings to _power_transform and _power_inverse helpers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…model Rename `TransformedExpectation` to `QuasiArithmeticMean` and `PowerCertaintyEquivalent` to `PowerMean`; the `certainty_equivalent` field, params pseudo-function, and `risk_aversion` parameter are unchanged. Move the engine-side implementation (`CE_VALUE_ARG`, the power transform pair, and `resolve_certainty_equivalent`) into `_lcm/certainty_equivalent.py` so the public module is a thin, deep-module namespace and the solver seam in `Q_and_F.py` only imports the resolver. Collapse the toy `tests/test_models/epstein_zin_health.py` and the example into a single parametrized `lcm_examples.epstein_zin` (`EZRegimeId`, `get_model` with a required `certainty_equivalent` and grid-size knobs whose defaults reproduce the toy numerically). The numpy pinning references pass unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace `docs/examples/epstein_zin.md` with `docs/examples/epstein_zin.ipynb`: the same recursion, mapping, and pitfalls as markdown cells, plus code cells that solve and simulate a 20-period model for two risk-aversion values and render a plotly figure of mean wealth by age (grey vs accent, direct labels). Update the toctree in `myst.yml` and the link in `examples/index.md`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…he example - lcm.aggregators exposes the two standard Koopmans aggregators; the default H is now the public H_linear, and H_epstein_zin is parametrized by the intertemporal elasticity of substitution (curvature rho = 1 - 1/psi computed inside, psi = 1 as the Cobb-Douglas limit). - PowerMean handles risk_aversion = 1 as the geometric-mean (log) limit exp(E[log V']) instead of rejecting it; the numpy reference and pinning tests cover it. - Example notebook: merged the redundant positivity pitfalls, added an expected-utility baseline to the wealth figure, log_level='off'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… trace Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The example model gains three parameters, all defaulting to the previous behavior: income (above the consumption floor, saving becomes possible), health_cost (an out-of-pocket expense while in bad health - uninsurable expense risk in the spirit of the Atal et al. medical spending), and bequest_scale (prices the bequest in consumption-equivalent units; H_epstein_zin is a weighted power mean, so the alive value sits at the scale of per-period consumption and an unscaled sqrt(wealth) bequest makes death the good branch of the certainty equivalent). next_wealth clips to the wealth grid so none of the knobs can push states off-grid. The docs page is rewritten around the fixed model: - Atal, Fang, Karlsson & Ziebarth (2025) is now characterized correctly: their baseline is time-separable CARA expected utility; their robustness specification has exactly the CES-aggregator-around-a- certainty-equivalent structure used here, with a CARA certainty equivalent (expressible as a QuasiArithmeticMean). - The pylcm-mapping section reflects the shipped H_linear/H_epstein_zin and explains that per-period utility must live in consumption units because H is a power mean. - A new pitfall documents the bequest-scaling trap. - The figure simulates 1,000 subjects with saving, health costs, and a gentler mortality hazard, so survivor counts stay meaningful at every plotted age, and the closing text explains the economics: both types dissave early out of impatience, the risk-averse type holds and rebuilds a larger precautionary buffer, and the ranking flips when the bad-health spell becomes too expensive to self-insure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…IES/RA sweep The intro now describes the original Atal, Fang, Karlsson & Ziebarth (2025) model accurately: an annual Yaari life-cycle savings problem over ages 25-94, a seven-category health Markov chain driving expenditure and mortality, and the guaranteed-renewable GLTHI vs short-term insurance contracts. Their baseline is time-separable expected utility (CARA gamma=4e-4, CRRA sigma=4 robustness, delta=0.966); the headline is that GLTHI reaches ~96% of first-best welfare, robust to disentangling risk aversion and the IES (sec. VI.D.1, within 0.7%). A 'what this keeps and drops' paragraph is explicit that the equilibrium contract/premium layer is out of scope, so the paper's headline welfare gap is not something this consumer-block example reproduces. A new closing section makes the disentangling concrete: a 2D sweep of the welfare cost of the uninsurable health-expense risk over a log-2 grid of risk aversion and the IES, with expected utility marked as the exact anti-diagonal (IES = 1/gamma). The cost is a risk premium — it roughly triples down the risk-aversion axis and barely moves along the IES axis — so an EU model, confined to the anti-diagonal, confounds the two. This is exactly the degree of freedom the certainty_equivalent seam adds and the lever the paper's robustness section pulls. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The example model enters at age 25 rather than 60, matching the annual horizon of Atal et al. instead of a retirement-only slice. Only the entry age changes and periods stay annual, so the discount factor needs no recompounding. The docs page runs the full 25-to-85 lifecycle: 2,000 subjects under a Gompertz-like mortality hazard (low when young, rising with age), which keeps a well-populated surviving cohort through midlife so both figures read cleanly. The risk-averse agent holds a persistently larger precautionary buffer, and the welfare cost of the health-expense risk roughly doubles down the risk-aversion axis while barely moving along the IES axis. Prose and quoted numbers track the new lifecycle. The notebook executes in about ten seconds on CPU, within the Read-the-Docs build budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The intro markdown cell had lost its newlines and collapsed into a single
line, so its headings and paragraphs ran together on the rendered page.
Rebuild it with one array element per line. Also switch the recursion's
display equation from a ```{math}``` directive — which this project's MyST
does not process inside notebook cells, leaking raw LaTeX — to the $$ form
used by every other notebook, so it typesets.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A subject that enters a terminal regime is carried with frozen state through every later period the regime is active. to_dataframe now emits only the entry row per subject (terminal_rows="first", the default); terminal_rows="all" keeps the absorbing representation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The stored regression frames capture the absorbing representation, so the comparison must request it explicitly rather than inherit the collapsed default. Only the mortality model exposed this on CI — its subjects die before the horizon under float32 draws — but all four shape-pinning tests get the explicit knob. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JAX writes an executable to the persistent compilation cache only when its compile time exceeds jax_persistent_cache_min_compile_time_secs (default: 1 second). A pylcm model compiles as many small per-regime/per-period programs, most of which fall under that threshold, so the cache stayed empty and every fresh process recompiled the whole model — including the AOT-compiled solve and simulate programs, which do consult the cache once it is populated. Importing lcm zeroes the threshold via jax.config, so it takes effect regardless of whether jax was imported first; a user-set JAX_PERSISTENT_CACHE_MIN_COMPILE_TIME_SECS environment variable wins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Reject AgeSpecialized as a deterministic state-transition value and anywhere in terminal regimes (policy-dependent laws of motion use a plain transition reading an AgeSpecialized `functions` entry). - Reject additional_targets that read AgeSpecialized functions; record the specialized names on SimulationPhase. - Resolve AgeSpecialized inside Phased(solve=..., simulate=...) in the params template. - Resolve markers at a representative age for the used-variable check and the published function sets. - Document the two contracts and the supported scope in docs/user_guide/age_specialized.md (added to the toc). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… functions Regime-transition probabilities are built once, not per period, so a policy-specialized value flowing into next_regime — directly or through plain helper functions — would silently reuse one age's policy closure across all periods. Regime construction now walks the transition's parameter names transitively through the functions mapping (unwrapping Phased sides, per-target dicts, and MarkovTransition) and raises with a routing hint. Docstring and user guide state the full guarded surface. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The regime-transition guard covered bare AgeSpecialized nodes and dependency-graph reads through plain helper functions, but a MarkovTransition whose wrapped function is itself AgeSpecialized slipped through to a later failure. The scope check now rejects it at Regime construction, mirroring the state-transition check, across coarse, Phased, and per-target forms; docstring and user guide state the full guarded surface. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
…se + AgeSpecializedGrid marker + resolver/validator - _AgeSpecialized base class shared by AgeSpecializedFunction (functions) and the new AgeSpecializedGrid (age-varying continuous-state grid, shape-invariant). - age_specialization.py: grid resolution helpers (resolve_state_grids, state_grids_signature, has_age_specialized_grid) and the shape-invariance validator (same class + n_points at every age). - Word-boundary rename across engine, processing, validation, tests, docs. - Foundational only: engine wiring (per-period axis + continuation interp) still TODO.
…polation) - Accept AgeSpecializedGrid in Regime.states (typing + construction validators + phase splitter treat it as a both-phase continuous grid). - process_regimes: representative-resolve markers for all age-invariant machinery; build per-period VInterpolationInfo (continuation on period t+1's grid) and per-period state axes (V_t tabulated on period t's grid). - _build_Q_and_F_per_period: group signature folds in the continuation grid identity at t+1; each group's Q_and_F uses period t+1's interpolation info. - SolutionPhase.period_state_axes + backward_induction override the base axis with period-t nodes (same shape => shared kernel, no retrace). - Age-invariant AgeSpecializedGrid reproduces the plain fixed-grid solve bit-for-bit (verified). Simulate path still TODO.
…tion)
Thread period_to_regime_v_interp into _build_simulation_phase so the forward
simulation's Q_and_F interpolates V_{t+1} on period t+1's grid, matching solve.
Simulation states are per-subject particles (no grid axis), so only the
continuation needs per-period wiring. 165 simulate + regime-building tests green;
age-varying model simulates with finite, positive consumption.
tests/solution/test_age_specialized_grid_solve.py: - age-invariant grid reproduces the plain fixed-grid solve bit-for-bit; - age-tracking floor solves with no NaN poisoning (the feature's purpose); - V nondecreasing in wealth (economic sanity); - forward simulation gives finite, positive consumption; - non-shape-invariant grid (varying n_points) is rejected at build.
_state_grids_with_carried_domains passes AgeSpecializedGrid through unchanged; it is a continuous state, so every consumer (all filter for DiscreteGrid) skips it. Completes the marker's acceptance across the simulate->to_dataframe path.
- Wrap E501 long lines (docstrings/comments lengthened by the AgeSpecialized -> AgeSpecializedFunction rename); ruff-format the touched files. - Fix ty diagnostics introduced by the feature: type _states_for_period's state_action_space as StateActionSpace; getattr-based _grid_identity; runtime import of AgeGrid for validate_age_specialized_grids (a TYPE_CHECKING-only import broke beartype's runtime forward-ref resolution); drop unused ty:ignore comments. - Extract _states_for_period helper (fixes PLC0206 + PLR0915); PERF401 comprehension. ruff check clean; full tests-cpu suite 1229 passed / 0 failed; remaining 7 ty diagnostics are pre-existing on feat/age-specialized (canonicalize.py, mock_regime.py).
- canonicalize: _desugar_identity accepts AgeSpecializedGrid (branches only on DiscreteGrid, so a marker correctly yields ContinuousState). - processing: narrow carried_grids with isinstance(grid, Grid) — a runtime no-op (carried states are Phased(simulate=Grid) by the phase grammar) that fixes the union flowing into build_regime_transition_probs_functions. - tests: MockRegime.functions as Mapping[str, object] (documented loose typing, covariant); policy_bonus helpers typed UserFunction; drop a stale ty:ignore. ty: All checks passed (was 20 diagnostics before this series). ruff clean. Full tests-cpu suite: 1229 passed / 0 failed.
for more information, see https://pre-commit.ci
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds age specialization to pylcm: one regime definition whose functions,
constraints, or continuous-state grids vary with age. Two user-facing markers
share a common base (
_AgeSpecialized), each wrapping abuild(age)factoryplus a
signature(age)function — cells with equal signatures share one builtobject, so compiled programs are deduplicated across ages.
AgeSpecializedFunction— per-age functions and constraintsmotivating use case is calendar-policy tracking in cohort models: a GETTSIM
tax function whose policy environment moves with the agent's age, rebuilt per
(cohort, age)cell without duplicating regimes.Q_and_Fresolves the specialized function per age during backwardinduction; representative-age resolution in
_process_regime_corekeepstemplates and the params template consistent (including
Phased(AgeSpecializedFunction)).next_stateis period-indexed so per-age transitions simulatecorrectly;
to_dataframe(additional_targets=...)rejects age-specializedfunctions it cannot represent at a single age.
deterministic state transitions that are age-specialized (bare or via
MarkovTransition(...), including per-target andPhasedforms); regimetransitions that are age-specialized, wrapped, or whose dependency graph
reads an age-specialized helper; terminal-regime markers.
docs/user_guide/age_specialized.md.AgeSpecializedGrid— age-dependent continuous-state gridsn_points(andthe grid class) stay fixed — shape-invariant, so V-array shapes, the rolling
continuation template, and kernel-dedup topology are untouched. The
motivating use case is an asset state with an age-dependent borrowing floor
a̲(age)(Guvenen & Smith 2014, and any life-cycle model with an age-varyingconstraint): on a single fixed grid, cells below a tighter age's floor are
infeasible, and their
-infcontinuation poisons the value function throughinterpolation (
0 · (−inf) = NaN, propagated backward).values are substituted into the shared compiled kernel without retracing;
the continuation
V_{t+1}is interpolated on the target regime'speriod-(t+1) grid, with the
Q_and_Fgroup signature folding in thecontinuation-grid identity so periods never false-share a program.
per-period grids as the solve.
build(age)must return the same grid class andn_pointsatevery age (
RegimeInitializationErrorotherwise); markers are accepted onlyas state grids.
DESIGN-age-dependent-grids.md.Hardening
The function-specialization surface went through several rounds of external
adversarial review (ChatGPT-Pro code audit) driven by the downstream rmsm
project; every hole found (transitive regime-transition reads,
MarkovTransition-wrapped markers, params-template blind spots,representative-age leaks into
additional_targets) is closed with aregression test. Downstream, an annual lifecycle model with per-(cohort, age)
GETTSIM policy environments solves and simulates on this branch, and its
estimation runs on GPU. The grid surface is exercised end-to-end by the
Guvenen–Smith (2014) replication in lcm-replications, where the age-tracking
asset floor eliminates the
-inf/NaNpoisoning of the fixed-grid solve.Test plan
pixi run -e tests-cpu tests);ruff checkandtyboth fully clean.tests/regime_building/test_age_specialization.py(validation surface, sharing by signature),
tests/solution/test_age_specialized_solve.py(solve/simulateequivalence against a hand-built per-age model).
tests/solution/test_age_specialized_grid_solve.py— anage-invariant
AgeSpecializedGridreproduces the plain fixed-grid solvebit-for-bit; an age-tracking floor solves with no
NaNpoisoning;V monotone in wealth; simulation yields finite, positive consumption;
non-shape-invariant grids are rejected.
AgeSpecializedFunctionsignaturereproduces the unspecialized solution exactly; models without any
age-varying grid build byte-identically to the pre-feature path.
🤖 Generated with Claude Code
https://claude.ai/code/session_016BLLuvQwD2bdXgaTcJWWUm