Description
src/resnet.py lines 139-145
def _init_params(self):
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
elif isinstance(m, nn.BatchNorm2d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.weight, 0) # ← Bug: should be m.bias
Two issues:
- Line 145 sets
m.weight to 0, overriding the line above which just set it to 1. Should be nn.init.constant_(m.bias, 0)
_init_params is never called anywhere — the initialization is effectively dead code
Why It Matters
If anyone calls _init_params(), all BatchNorm weights become 0, which effectively disables the normalization layers and would cause severe training issues.
Suggested Fix
elif isinstance(m, nn.BatchNorm2d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0) # Fix: m.bias instead of m.weight
And either call self._init_params() in __init__ or remove the method.
Description
src/resnet.pylines 139-145Two issues:
m.weightto 0, overriding the line above which just set it to 1. Should benn.init.constant_(m.bias, 0)_init_paramsis never called anywhere — the initialization is effectively dead codeWhy It Matters
If anyone calls
_init_params(), all BatchNorm weights become 0, which effectively disables the normalization layers and would cause severe training issues.Suggested Fix
And either call
self._init_params()in__init__or remove the method.