Commit 38057ba
Phase 1A: modality-balanced encoder init + decoder variance-target scaling
Two new init toggles on RegularizedMultimodalVI, both off by default so
in-flight runs are byte-identical:
- use_modality_balanced_encoder_init (F1): rebalance per-modality
mean_encoder/var_encoder weights so per-element variance equals
1/(3·total_n_hidden) — equivalent to a single virtual Linear of
fan_in = sum_m n_hidden_m. Counters per-modality posterior collapse
driven by Kaiming's fan-in-only normalisation in multimodal encoders.
- decoder_init_variance_target_fraction (F2, plus burst_size_head
variant): rescale per-feature decoder weight rows so the pre-softplus
output variance lands at fraction × clip_per_feature_variance_to_theta_bounds(
excess_biological_g, mean_g, theta_min, theta_max). Symmetric to
z_loc_init_scale on the encoder side; lifts decoder output variance
off the Kaiming default 1/3 toward the per-gene biological excess.
Supporting changes:
- _compute_theta exposes excess_biological in diagnostics with
np.maximum(..., eps**2) floor (single source of truth).
- compute_bursting_init reads excess_biological from diagnostics and
uses the new clip_per_feature_variance_to_theta_bounds helper for
the stochastic_v_scale clip — byte-identical to the prior inline form.
- ValueError if decoder fraction is set with dispersion_init not in
{data, variance_burst_size}; LayerNorm warning if decoder LayerNorm
is disabled (Var(h_dim) ≈ 1 assumption broken).
Tests: 11 new (5 F1 + 6 F2); full suite 406/406.
Phase 1B sweep TSV (8 conditions × 3 datasets = 24 rows) with F1 always
on, F2 main fraction in {0.2, 0.02} and burst_size r = main / 10 for
bursting variants.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent f8edb08 commit 38057ba
7 files changed
Lines changed: 655 additions & 5 deletions
File tree
- docs/notebooks
- immune_integration
- model_comparisons
- src/regularizedvi
- tests
Lines changed: 17 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
116 | 116 | | |
117 | 117 | | |
118 | 118 | | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
119 | 124 | | |
120 | 125 | | |
121 | 126 | | |
| |||
427 | 432 | | |
428 | 433 | | |
429 | 434 | | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
430 | 441 | | |
431 | 442 | | |
432 | 443 | | |
| |||
484 | 495 | | |
485 | 496 | | |
486 | 497 | | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
487 | 501 | | |
488 | 502 | | |
489 | 503 | | |
| |||
612 | 626 | | |
613 | 627 | | |
614 | 628 | | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
615 | 632 | | |
616 | 633 | | |
617 | 634 | | |
| |||
Lines changed: 25 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
0 commit comments