Benchmarks vs numpy, scipy, sklearn, Pytorch

[nikolaydubina]

Did you guys compare this library for equivalent implementations in numpy, scipy, sklear, Pytorch, Tensorflow?

If GPU is not supported, you can try reporting CPU versions. AFAIK both Pytorch and Tensorflow have cpu modes for their tensor operations.

[matteo-grella]

Hi @nikolaydubina,

Yes, that's on our TODO list.

By the way, if you had to choose, would you prefer benchmarks on single operations (e.g., matrix-vector multiplication) or on higher level tasks, like NER and Question-Answering, etc?

[nikolaydubina]

Absolutely on low level tasks. I would expect people implementing their own high-level algorithms, and they would have their own benchmarks. That being said, golden standard for high-level tasks also useful, so that people will see if they are way off with their code. Thank you!

[matteo-grella]

It's decided then. To start, we will make a direct comparison of the basic operators of the spaGO auto-grad package with the PyTorch ones.

Do you feel like helping on this task? Every contribution is precious :)

[nikolaydubina]

Sure, I will see if I have time :)