PyTorchAlpaka: Batched inference with TensorCollections, eval and frozen model, FP16 convsersion support

[EmanueleCoradin]

This PR results from a collaboration with @valsdav. Thanks also to @Electricks94, who helped implement batched inference support.

PR description:

This PR introduces some new features to the PyTorchAlpaka interface:

• Batched inference support: a single SoA can now be splitted in multiple TensorCollection batches.
• Eval and Frozen model: Model wrapper now enforces that eval and optionally frozen mode. TorchScript modules cannot be moved to another device once they have been fronzen.
• Added optional half-precision conversion for input tensors during inference (to_half flag). If the model was exported in FP16 format, the whole inference will be performed in half-precision format, reducing peak GPU memory.

Validation

To validate the PR, I prepared two producers to launch inference in mini-batches: SimpleNetMiniBatch and TinyResNetMiniBatch.
I then modified InspectionSink.cc to check that the relative difference between the output of the default and batch implementation is lower than a compatibility threshold (default 1e-5).
You can launch the tests by:

cd src/PhysicsTools/PyTorchAlpakaTest/test/
cmsRun runPyTorchAlpakaTest.py  --compareBatch -e 3

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48664

[mmusich]

type ngt

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48680

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48680/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48680/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48687

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48687/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48687/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48691

[cmsbuild]

Pull request #50498 was updated.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48693

[cmsbuild]

Pull request #50498 was updated.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50498/48715

[cmsbuild]

Pull request #50498 was updated.