pint units (GEP 10) · infra 2/3: currency & boundary#139
Closed
MImmesberger wants to merge 22 commits into
Closed
Conversation
#133) ## Summary ty now runs only as pre-commit hooks, under both backends, replacing the dedicated pixi `type-checking`/`type-checking-jax` features, environments, and tasks (`pixi run ty` / `pixi run ty-jax` no longer exist). - **numpy run**: the official [astral-sh/ty-pre-commit](https://github.com/astral-sh/ty-pre-commit) hook with `args: [--no-project]`. uv creates no `.venv`/`uv.lock` in this pixi-managed repo and resolves no dependencies itself (its default mode fails on the pygraphviz source build anyway); ty resolves third-party imports from the pixi `py314` environment via `tool.ty.environment.python` in `pyproject.toml`. - **jax run**: a minimal local hook running the same pinned ty with `--python .pixi/envs/py314-jax`. It inherits the entire `tool.ty` configuration — same rules, only the environment differs. Verified that jax genuinely resolves in this run (passes even with jax dropped from `analysis.allowed-unresolved-imports`). The jax variant cannot use the official hook today: `uv check` forwards nothing to ty (no `--python`, no `-c`, no trailing args), and ty rejects conda-style envs handed over via `VIRTUAL_ENV` (no `pyvenv.cfg`). The local hook's `ty==0.0.49` pin carries a comment to keep it in sync with the hook rev, since `prek auto-update` bumps only the rev. ## Other changes - The `run-ty` CI job installs `py314`/`py314-jax` and runs the hooks via prek, so the pre-commit config is the single definition of how ty runs everywhere. - Both ty hooks are skipped on pre-commit.ci (no network at hook runtime, no pixi environments). - The existing test environments provide everything ty needs (including type stubs), so the dedicated type-checking environments are gone — including their ~1800 lines in `pixi.lock`. pixi 0.70.2 does not prune removed environments from the lock; the stale blocks and orphaned `ty` package entries were removed manually and the result validated with `pixi lock`. - AGENTS.md: verification is now two steps; `prek run --all-files` covers type checking under both backends. - `.ai-instructions` is re-pinned to c2701d4 (the merged boilerplate: real tiering, distilled modules, ty-pre-commit guidance) — matching the rest of the gettsim family. - **ty bumped 0.0.48 → 0.0.49** (hook rev + local jax pin), matching the `.ai-instructions` boilerplate and the rest of the family. 0.0.49 flagged a non-exhaustive `shape_id` if/elif chain in `tests/test_end_to_end.py` as a possibly-unresolved reference; fixed with an `else` guard that raises on an unknown `shape_id`. ## Test plan - [x] `pixi run -e py314-jax prek run --all-files` — all hooks pass, including both ty variants (numpy + jax) on ty 0.0.49 - [x] `pixi run --locked -e py314 prek run ty --all-files` and `... ty-jax --all-files` (exact CI commands) - [x] `pixi run --locked -e py314-jax tests -n 7` — 1119 passed, 42 skipped - [x] `pixi lock` — lock file validates after pruning 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`.ai-instructions` + prek-hook bump, plus the code changes ty `0.0.52` requires. - `.ai-instructions` submodule → `68dc3f4` - prek hooks refreshed (ruff `v0.15.19`, pyproject-fmt `v2.25.0`, ty `v0.0.52`) - ty 0.0.52 fixes: `tt/vectorization.py` and `interface_dag_elements/policy_environment.py` - `pixi.lock` regenerated ty stays at `v0.0.52` — the latest **published** ty binary (the `ty-pre-commit` `v0.0.53` tag has no corresponding `astral-sh/ty` release). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
register_currency tolerated re-registration as long as the existing unit was currency-dimensioned, checking only the dimension and silently keeping the original factor. Re-registering `castar / 4` as `castar / 5`, or a derived currency as `base=True`, was swallowed and invalidated every later conversion. Compare the existing factor against CURRENCY with the requested one and reject any mismatch (GEP 10). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A unit-algebra primitive for the Layer-2 input check (GEP 10): divide out the currency component (converted at the boundary) and the flow period (screened against the name suffix), leaving the numerator scale that must match the declared unit exactly. Composes the existing currency/flow-period component helpers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the level-aware primitives to the dimension model. Each grouping level (the individual 'person', and one per *_id group discovered per build) becomes its own non-convertible pint base dimension; a leveled quantity carries its level as a denominator, read off the aggregation suffix. 'person' doubles as the [person] count dimension, so head counts ([person]/[group]) and per-person amounts cancel. New: register_grouping_levels (idempotent, always registers person), divide_by_grouping_level, grouping_level_count_unit, unit_token_carries_level (extensive/intensive default: currency/area/count carry a level; durations, calendar points, dimensionless, hours, per-area rates do not), reference_level on resolve_param_unit/resolve_scalar_param_unit, and resolved_unit_for_aggregation (SUM swaps source->target level, COUNT mints [person]/[target], MIN/MAX/MEAN preserve the source verbatim, ANY/ALL dimensionless). Orchestration (discovering levels from *_id, threading reference_level and the aggregation target/source levels, the suffix-vs-unit-level check) lands in the mandatory+edge-check PR. Validated by standalone tests preserved on wip-level-dims, run at the top of the cascade. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ount Standalone core-mechanics tests for the level primitives: levels as non-convertible base dimensions, level-as-denominator resolution, the [person] count bridge (division and multiplication cancel), cross-level rejection, reference_level on parameters, and the level-aware aggregation (SUM swaps source->target, COUNT mints [person]/[target], MIN/MAX/MEAN preserve the source verbatim, ANY/ALL dimensionless). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `Unit.HEADCOUNT`, a declarable head count `[person] / [level]`: base is the [person] leaf level (the count dimension), level-carrying, so a reference level enters as the denominator exactly as for currency/area. This lets a parameter, hand-written function, or input column *declare* a head count, which only aggregations could produce before. The reference level is mandatory but PERSON_LEVEL is allowed: a head count per individual is `[person]/[person]` = dimensionless, which is exactly right. A HEADCOUNT parameter must set `reference_level` (person or group); a column takes it from the name position. `unit_for_aggregation` now mints HEADCOUNT for COUNT (group and PID alike): for a group node the resolved unit is recomputed level-aware regardless, and for a PID node the token resolves through the column path to `[person]/[person]` = dimensionless — a head count per individual. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MImmesberger
force-pushed
the
gep10-infra-2-currency-boundary
branch
from
d89903e to
f5144c9
Compare
MImmesberger
force-pushed
the
gep10-infra-1-dimensional-core
branch
2 times, most recently
from
61f78a1 to
ff4c441
Compare
MImmesberger
force-pushed
the
gep10-infra-2-currency-boundary
branch
from
f5144c9 to
8ea1784
Compare
MImmesberger
force-pushed
the
gep10-infra-1-dimensional-core
branch
from
ff4c441 to
b829673
Compare
MImmesberger
force-pushed
the
gep10-infra-2-currency-boundary
branch
from
8ea1784 to
1dfc743
Compare
…aggregate, check) Wire the level-aware primitives from `tt/units.py` into the build: - `resolve_environment_units` registers the discovered grouping levels, then divides each level-carrying column/function unit by its aggregation-suffix level (unsuffixed = `person`); level-less tokens stay level-less regardless of an index-only suffix (`MIN`-of-age at `_fg`). - auto-aggregation nodes resolve through `resolved_unit_for_aggregation` (mint/swap/preserve), threading the source level (from the source token + suffix) and the target level (the agg's own suffix). - `reference_level` is threaded from the param spec into `resolve_param_unit` / `resolve_scalar_param_unit` for scalar, dict, raw, per-leaf-mapping, and schedule params. - the body dry-run now applies the index-vs-unit-level rule: an inferred level-denominator must equal the name's suffix level; a level-less inferred unit is exempt. The physical residual (currency, period, area) is compared with grouping levels stripped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Currency/area nodes now carry a grouping-level denominator read off the name suffix (an unsuffixed name is the individual [person] level), so the resolution assertions expect the leveled units.
A head count is COUNT *or* a SUM over a boolean (counting the persons the indicator is true for) — both must mint [person]/[target], not DIMENSIONLESS. The orchestration previously hardcoded SUM and reconstructed the source by stripping the name suffix, so COUNT never minted and hand-written counts (number_of_adults_fam sums 'adult', not 'number_of_adults') resolved wrong; the [person] count bridge was inert end-to-end. Store agg_type on AggByGroupFunction and branch on it in _resolve_agg_by_group_unit: COUNT and SUM-over-boolean mint [person]/[target]; SUM over an extensive value swaps the level; MEAN/MIN/MAX preserve; ANY/ALL dimensionless. The value source is read off the function signature, so both auto- and hand-written aggregations resolve. Per-capita divisions (rent_m_hh / number_of_individuals_hh -> CURRENCY/[person]) now type-check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add HEADCOUNT to the params-schema unitToken enum, and update the COUNT aggregation decorator test: a COUNT now auto-assigns the HEADCOUNT token (its level-aware resolved unit is unchanged, [person]/[target]). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Body-level regression tests for the grouping-level dry-run catch that previously existed only at the units_are_equivalent primitive level: group-vs-group and person-vs-group subtraction and an across-level ordering comparison all raise UnitConsistencyError, plus a same-group-level positive control. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An aggregation derives its unit from its source + agg_type, but the declared token was never checked against that derivation — so a SUM over a boolean (a head count, [person]/[target]) could declare DIMENSIONLESS undetected, and a SUM of currency could declare anything. Two coupled fixes in unit_checks.py: - Source detection in `_resolve_agg_by_group_unit` now excludes person pointers (`p_id_*`), not just group ids. An `agg_by_p_id` aggregation becomes an `AggByGroupFunction` once tree logic is removed, carrying its foreign-key pointer as a plain argument; previously that gave two "sources" and the node silently failed to resolve. Now pid aggregations derive from their source too. - `fail_if_environment_units_are_inconsistent` now validates every aggregation: the declared token's physical kind (period-free, level-free — the period and grouping level are the framework's to derive, and a group suffix may be a mere index level for MAX/MIN/MEAN) must equal what the aggregation derives, else UnitConsistencyError. This is the declared-vs-produced contract that hand-written bodies already had, extended to the derived node types. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A require_converter declaring input_unit:/output_unit: is left raw at build time; its converter's typed (piecewise / lookup) output is converted per axis in with_processed_params_and_scalars, so a polynomial coefficient of order j scales by f_out / f_in**j instead of by a single uniform factor (GEP-10 audit finding S3). A homogeneous require_converter keeps its single unit: and uniform scaling; one that carries a currency unit: yet produces a schedule is rejected, pointing the author at the per-axis declaration. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fail_if_input_units_are_inconsistent compared only the dimensionality of a tagged input column against its declared unit, so a HECTARES column tagged in m² (a 10,000-fold level error) or a YEARS age tagged in months passed the check and was then mis-stripped at the boundary — only the currency is converted there, area and intrinsic time are left untouched. Compare the residual after factoring out the currency (converted at the boundary) and the flow period (screened against the name suffix by the dedicated period guard); the remaining numerator scale must match exactly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MImmesberger
force-pushed
the
gep10-infra-1-dimensional-core
branch
from
b2f5a01 to
7c344d0
Compare
MImmesberger
force-pushed
the
gep10-infra-2-currency-boundary
branch
from
1063a04 to
e8b3a61
Compare
The schema let a scalar parameter declare `reference_period` (which the implementation forbids — a scalar takes its period from a time suffix on its name) and accepted a nested `unit:` mapping on a scalar (only a dict parameter may declare per-leaf units; construction later mishandles it). Condition both on `type: scalar`. Only the ttsim copy changes; the GEP copy stays on the pre-pint form and migrates wholesale later. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Plumb the reference_level facet end to end: the ParamObject field, the spec->object parsing, and the JSON schema. A per-person/per-group parameter names the grouping level it is denominated per by its *_id stem (reference_level: person / hh / ...), orthogonal to reference_period and - unlike reference_period - admissible on a scalar parameter. The resolution (resolve_param_unit) and the orchestration already consume it; this lets it flow from YAML. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MImmesberger
force-pushed
the
gep10-infra-2-currency-boundary
branch
from
e8b3a61 to
f949ac2
Compare
MImmesberger
force-pushed
the
gep10-infra-1-dimensional-core
branch
from
7c344d0 to
3c0f7f2
Compare
Member
Author
|
Collapsed into a single infra PR #138. The three-way infra split introduced the token-based unit model in #138 and replaced it with compositional units here/in #140, so the same lines were re-touched across PRs (units.py ~1379 lines, unit_checks.py ~671, …). #138 now contains the whole GEP-10 infra as one final-form diff (no token model anywhere); #141 (worked example) is retargeted onto it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Concrete-currency handling and the Layer-2 boundary: the currency knob, unit-annotated input/output boundary checks, per-axis
require_convertercurrency conversion, and scalar-param/reference_levelconstraints.🤖 Generated with Claude Code