Skip to content

feat: add Qwen encoder runtimes (Qwen2.5-VL + Qwen3) for Nunchaku - #927

Open
tonera wants to merge 1 commit into
nunchaku-ai:mainfrom
tonera:feat/qwen_encoder_nunchaku
Open

feat: add Qwen encoder runtimes (Qwen2.5-VL + Qwen3) for Nunchaku#927
tonera wants to merge 1 commit into
nunchaku-ai:mainfrom
tonera:feat/qwen_encoder_nunchaku

Conversation

@tonera

@tonera tonera commented Mar 31, 2026

Copy link
Copy Markdown

Summary

This PR adds Nunchaku runtimes for Qwen-family text encoders and exposes them via the public nunchaku API. It includes:

  • A stable entry point (NunchakuQwenEncoderModel) that auto-dispatches to the correct runtime based on exported .safetensors metadata.
  • Dedicated runtimes for:
    • Qwen2.5-VL text-only encoder (NunchakuQwen2VLTextEncoderModel)
    • Qwen2.5-VL multimodal edit encoder (NunchakuQwen2VLEditEncoderModel)
    • Qwen3 text-only encoder (NunchakuQwen3TextEncoderModel, optional depending on transformers version)

Key changes

  • Public exports
    • Export Qwen encoder models from nunchaku/__init__.py and nunchaku/models/__init__.py
    • Add nunchaku/models/text_encoders/__init__.py exports
  • New Qwen encoder runtimes
    • nunchaku/models/text_encoders/qwen_encoder.py: dispatcher from_pretrained() selecting the correct runtime via checkpoint metadata (text_encoder_usage, model_type, runtime_class)
    • nunchaku/models/text_encoders/qwen2_vl_text_encoder.py: Qwen2.5-VL text-only runtime (rejects multimodal args)
    • nunchaku/models/text_encoders/qwen2_vl_edit_encoder.py: Qwen2.5-VL edit/multimodal runtime
    • nunchaku/models/text_encoders/qwen3_text_encoder.py: Qwen3 text-only runtime
    • nunchaku/models/text_encoders/qwen_common.py: unified base class, checkpoint metadata parsing, config building, quantized linear patching, and rotary buffer materializers
  • Transfer utilities
    • nunchaku/torch_transfer_utils.py: device normalization + pin-memory policy + optional “pretouch” helpers for CPU→CUDA staging

Files changed

  • Updated (overwritten)
    • nunchaku/__init__.py
    • nunchaku/models/__init__.py
    • nunchaku/models/text_encoders/__init__.py
  • Added
    • nunchaku/models/text_encoders/qwen_encoder.py
    • nunchaku/models/text_encoders/qwen_common.py
    • nunchaku/models/text_encoders/qwen2_vl_text_encoder.py
    • nunchaku/models/text_encoders/qwen2_vl_edit_encoder.py
    • nunchaku/models/text_encoders/qwen3_text_encoder.py
    • nunchaku/torch_transfer_utils.py

Test plan

  • Import smoke test:
    • python -c "from nunchaku import NunchakuQwenEncoderModel, NunchakuQwen2VLTextEncoderModel, NunchakuQwen2VLEditEncoderModel; print('ok')"
  • Dispatcher path:
    • python -c "from nunchaku import NunchakuQwenEncoderModel; m=NunchakuQwenEncoderModel.from_pretrained('path/to/qwen_encoder.safetensors', device='cpu'); print(type(m))"
  • (Optional) Qwen3 availability:
    • If transformers provides Qwen3Model, load a Qwen3 exported checkpoint; otherwise confirm it raises a clear ImportError message.

Model download link:

https://huggingface.co/tonera/Qwen2.5vl-Nunchaku
https://huggingface.co/tonera/Qwen3-text-Nunchaku

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant