Progress bar

[florian6973]

Hi!

During featurization, which can be very long (several hours), it could be useful to have a global progress bar. It is easy to compute progress based on the number of generated files over the total expected (#windows x #agg x #shards). I am just wondering how to incorporate it with the current existing output.

Thanks!

[mmcdermott]

@florian6973, great suggestion -- this can be hard to integrate with other logging libraries in CLI tools, but it would be a major help. Thanks!

[florian6973]

Yes maybe the easiest way would be to have another command line tool that could be run manually in parallel if needed to show the progress.

[mmcdermott]

I think it is probably possible to do natively; just with smarter integration of logging and tqdm or something. I'm sure there are libraries for it.