Any one get any video result by training this project?

[double-fire-0]

[chpk]

yes, got reasonable results using this code, I had a training dataset of around 3,000 videos in .gif format. Ran till 100000 iterations. During the training process the model parameters will be saved to './results' folder. Using the following sampling code you can load the trained parameters and generate videos:
import torch
from video_diffusion_pytorch import Unet3D, GaussianDiffusion, Trainer, video_tensor_to_gif

batch_size = 1

model = Unet3D(
dim = 64,
cond_dim = 768,
dim_mults = (1, 2, 4, 8),
)

diffusion = GaussianDiffusion(
model,
image_size = 64,
num_frames = 10,
channels = 3,
timesteps = 1000, # number of steps
loss_type = 'l1' # L1 or L2
)

trainer = Trainer(
diffusion,
'./data')

trainer.load(-1)

input_condition = torch.randn(batch_size, 768)

output_Gif_Tensor = diffusion.sample(cond = input_condition, batch_size = batch_size)

video_tensor_to_gif(output_Gif_Tensor[0], './output.gif')

[BiswajeetRaut]

Which dataset you used ?

[chpk]

custom dataset, around 10K training samples

[zzzbbt]

custom dataset, around 10K training samples

Can I use this model to generate growing GIFs using short GIFs？

[chpk]

The input & output number of frames are fixed and this parameter can be changed during training/inference, also the current code does not support taking .gifs as condition to generate outputs based on the given condition.

[zzzbbt]

The input & output number of frames are fixed and this parameter can be changed during training/inference, also the current code does not support taking .gifs as condition to generate outputs based on the given condition.

When I use your sampling script, I find that the generated gif is different from the gif I sampled. Is there any solution?

[Ke-Hao]

Hi, I was wondering what GPU setup is needed to run the code effectively. Could you share the recommended GPU specifications?