timezone binary data compression

[jannikmi]

This package aims to be lightweight, but with the current timezone data the disk footprint is > 60 MB (wheel size around 51 MB).
Look into techniques for compressing the timezone data (binary files) without loosing precision or speed.

Idea: research the effect of using 16 bit integegers instead of 32bit. what is the worst case precision at the equator?
Note: there are other packages like tzfpy which are faster, but offer less accuracy. reducing the accuracy in timezonefinder probably diminishes the USP of this library (cf. ringsaturn/tzfpy#94)

[aryannikhil]

Hello,
I would like to work on this issue and begin exploring techniques to compress the timezone binary data, starting with evaluating the use of 16-bit integers as suggested. If no one is currently assigned, I would appreciate the opportunity to contribute. Please let me know if that would be acceptable.
Thank you.

[jannikmi]

@nikkhilaaryan thanks for your interest. Any contribution is highly welcome!

The results from ringsaturn/tzfpy#94 revealed that precision is one of the main unique selling points of timezonefinder and hence actually making the used timezone polygon data less precise (e.g. with 16bit int) does not make sense.

One potential improvement however, could be to compress the binary files and uncompress them once during execution.
There are a few side constraints with this:

• the distribution wheels have only around 50MB and hence seem to be compressed already: https://pypi.org/project/timezonefinder/#files I am not sure if things can be compressed beyond that
• the uncompression probably introduces an additional dependency, which needs to be justified
• this current PR will introduce a change in binary data format: Binary polygon data structure refactoring #287

From my perspective there are more pressing issues with higher impact, right now: https://github.com/jannikmi/timezonefinder/issues
My top pick would be the data update automation (high impact, low invest): #273 and later #167

I am closing this issue for now.

[jannikmi]

This seems to be relevant after all. Cf. #317

Try compressing the .fbs flat buffer binary files with https://pyzstd.readthedocs.io/en/latest/