ValueError: num_samples=0 when creating DataLoader during training

[yichuan-huang]

Issue Description

I'm encountering a ValueError when trying to train AttentionGAN. The error occurs during dataset creation, specifically when initializing the DataLoader.

Error Message

Traceback (most recent call last):
  File "/opt/data/private/AttentionGAN/train.py", line 29, in <module>
    dataset = create_dataset(opt)  # create a dataset given opt.dataset_mode and other options
  File "/opt/data/private/AttentionGAN/data/__init__.py", line 57, in create_dataset
    data_loader = CustomDatasetDataLoader(opt)
  File "/opt/data/private/AttentionGAN/data/__init__.py", line 75, in __init__
    self.dataloader = torch.utils.data.DataLoader(
  File "/opt/data/private/anaconda3/envs/py310/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 351, in __init__
    sampler = RandomSampler(dataset, generator=generator)  # type: ignore[arg-type]
  File "/opt/data/private/anaconda3/envs/py310/lib/python3.10/site-packages/torch/utils/data/sampler.py", line 144, in __init__
    raise ValueError(f"num_samples should be a positive integer value, but got num_samples={self.num_samples}")
ValueError: num_samples should be a positive integer value, but got num_samples=0

Environment

• Python version: 3.10
• PyTorch version: 2.4.1
• Operating System: Ubuntu 20.04

Dataset Setup

• Training dataset structure follows the standard format:

  dataset/
  ├── train_Input_Var/
  ├── train_Output_Var_actin/
  

• All folders contain images (tif)
• File permissions have been verified (755)

Training Command

python3 train.py --dataroot ./datasets/cell_datasets/3T3_10X/train/ --name 3T3_10X_actin_attentiongan --model attention_gan --dataset_mode unaligned --pool_size 50 --no_dropout --norm instance --lambda_A 10 --lambda_B 10 --lambda_identity 0.5 --load_size 286 --crop_size 256 --batch_size 1 --niter 300 --niter_decay 300 --gpu_ids 0 --display_id 0 --display_freq 100 --print_freq 100

What I've Tried

1. Verified dataset structure and file existence
2. Checked and corrected file permissions using chmod -R 755
3. Confirmed that all dataset folders contain image files
4. Verified that the dataset path is accessible and readable

Additional Information

The dataset appears to be properly structured and accessible, but the DataLoader is reporting 0 samples. This suggests that the dataset loading mechanism is not recognizing the files, despite them being present and accessible.

Request

Could you please provide guidance on:

1. How to debug why the dataset length is 0?
2. Are there specific file format requirements that might cause files to be ignored?
3. Any additional dataset validation steps I should perform?

Thank you for your help!