Skip to content

fix: normalize output-axis DoRA with adapted weights - #15458

Draft
orangemagic123 wants to merge 1 commit into
Comfy-Org:masterfrom
orangemagic123:fix/dora-adapted-weight-norm
Draft

fix: normalize output-axis DoRA with adapted weights#15458
orangemagic123 wants to merge 1 commit into
Comfy-Org:masterfrom
orangemagic123:fix/dora-adapted-weight-norm

Conversation

@orangemagic123

Copy link
Copy Markdown

Summary

  • normalize output-axis DoRA weights using the adapted weight W + ΔW
  • keep the existing input-axis behavior unchanged
  • add regression coverage for linear and Conv2d-shaped weights

Problem

weight_decompose() currently builds the adapted weight first:

weight_calc = weight + lora_diff

but the output-axis branch calculates its norm from the original base weight:

weight_norm = norm(weight)

This applies:

m * (W + ΔW) / ||W||

instead of the DoRA decomposition:

m * (W + ΔW) / ||W + ΔW||

The input-axis branch already normalizes weight_calc, so this change also makes both branches consistent. The behavior now matches the current LyCORIS LoKr/DoRA implementation.

Tests

A focused CPU regression harness matching the added assertions passed both cases:

linear: passed
conv2d: passed

The added repository test can be run with:

pytest tests-unit/comfy_test/test_weight_adapter.py

The full ComfyUI test suite was not run in the connector environment; GitHub CI is expected to provide repository-level validation.

Related work

This is a separate semantic-correctness fix from the dora_scale broadcasting issue tracked in #12938 and PR #13011. The broadcasting fix may touch the same function, but the adapted-weight normalization remains required after that change.

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

✅ All contributors have signed the CLA. Thank you! This PR is ready to be merged.
Posted by the CLA Assistant Lite bot.

@orangemagic123

orangemagic123 commented Aug 10, 2026

Copy link
Copy Markdown
Author

I have read and agree to the Contributor License Agreement

@orangemagic123

Copy link
Copy Markdown
Author

recheck

comfy-legal added a commit to Comfy-Org/comfy-cla that referenced this pull request Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant