Skip to content

Support lazy-initialized Linear modules in SVDQW4A4Linear.from_linear - #929

Open
kj863257 wants to merge 3 commits into
nunchaku-ai:mainfrom
kj863257:main
Open

Support lazy-initialized Linear modules in SVDQW4A4Linear.from_linear#929
kj863257 wants to merge 3 commits into
nunchaku-ai:mainfrom
kj863257:main

Conversation

@kj863257

@kj863257 kj863257 commented Apr 7, 2026

Copy link
Copy Markdown

Motivation

Some integration paths may construct Linear modules lazily, with weight=None until state dict loading completes.

SVDQW4A4Linear.from_linear() currently assumes linear.weight is always available and directly accesses
linear.weight.dtype and linear.weight.device. In lazy-initialization scenarios, this can raise:

AttributeError: 'NoneType' object has no attribute 'dtype'

This PR adds safe fallback behavior for lazy-initialized Linear modules while preserving the existing behavior for
regular nn.Linear modules that already have materialized weights.

This change is part of a paired fix with the corresponding ComfyUI-nunchaku PR:

  • ComfyUI-nunchaku: Support lazy-init Linear layers in Z-Image patching
  • Link: <COMFYUI_NUNCHAKU_PR_URL>

Modifications

  • Update SVDQW4A4Linear.from_linear() in nunchaku/models/linear.py.
  • Keep the existing behavior unchanged when linear.weight is present.
  • When linear.weight is None, resolve torch_dtype from the explicit argument if provided, otherwise fall back to
    linear.weight_comfy_model_dtype, and finally torch.bfloat16.
  • Resolve device from the explicit argument if provided, otherwise fall back to weight.device, bias.device, and
    finally CPU.
  • Determine bias presence from linear.bias instead of assuming a populated weight tensor.
  • Add a lightweight unit test in tests/test_linear_lazy_init.py covering both lazy-initialized and regular Linear
    cases.
  • Run pre-commit on the touched files.

Checklist

nunchaku-ai/ComfyUI-nunchaku#817

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant