Fix dict iteration bug in get_peft_state_maybe_zero_3#153
Open
Mr-Neutr0n wants to merge 1 commit intoBAAI-DCAI:mainfrom
Open
Fix dict iteration bug in get_peft_state_maybe_zero_3#153Mr-Neutr0n wants to merge 1 commit intoBAAI-DCAI:mainfrom
Mr-Neutr0n wants to merge 1 commit intoBAAI-DCAI:mainfrom
Conversation
Fix two bugs in the lora_only bias handling branch: 1. `for k, t in maybe_lora_bias` iterates over dict keys (strings), causing unpacking to split a key string into individual characters instead of yielding key-value pairs. Fixed by using `.items()`. 2. `bias_name` referenced the last value assigned in the previous loop rather than the current key `k`, so the wrong bias name was being checked and stored. Fixed by replacing `bias_name` with `k`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two bugs in the
lora_onlybranch ofget_peft_state_maybe_zero_3inbunny/train/train.py:Missing
.items()on dict iteration:for k, t in maybe_lora_biasiterates over dict keys (strings), causing Python to unpack each key string into individual characters rather than yielding key-value pairs. This would raise aValueErrorat runtime if any key is longer than 2 characters. Fixed by usingmaybe_lora_bias.items().Stale variable reference:
bias_namein the second loop referenced the last value assigned during the first loop's iteration, rather than the current keyk. This meant only the last bias name computed in the first loop was ever checked/stored, regardless of which bias parameter was actually being processed. Fixed by replacingbias_namewithk.Before
After
Test plan
train.pywhere this function was originally sourced from