run inference on GPU

[oliverheard]

In the Kubernetes version of Scenescape could you add a menu item in the camera config to select the inference device. Currently only selecting decode device:

I would also suggest checking how many GPU devices are in the system and offereing to Select GPU.0... GPU.1 etc.

Does it currently support cross stream batching? ie with multiple cameras it will batch frames together for inference? this should gain higher performance on use cases with many cameras.

[saratpoluri]

For camera chain offload, we kept the same syntax as for Percebro i.e you can specify the chain like this:
retail=GPU+reid=CPU
We did it this way to enable building complex pipeline while allowing the user the choice on what should execute where.
In the multi-GPU case, you follow the same syntax as before i.e retail=GPU.0+reid=GPU.1.

HW-aware configurability is part of the roadmap and you filing the issue pulls it forward in the timeline.
DLS currently offload the multi-stream batching to OpenVino. Last I remember, when the inference results are obtained, the demuxing of the results is not done. This is problematic for downstream consumer like SceneScape that has camera parameters stored and uses per camera results to project them into shared coordinate space. We will request DLS to support this muxing and demuxing.

[tdorauintc]

@oliverheard thank you for the feedback. The functionalities of GPU enumeration and cross-stream batching are known limitations we mention in our documentation for the latest 2025.2 release.

In the Kubernetes version of Scenescape could you add a menu item in the camera config to select the inference device.

As @saratpoluri already explained, you can specify it directly in the Camera Chain UI field. Refer to the model chaining documentation for more details.

I would also suggest checking how many GPU devices are in the system and offereing to Select GPU.0... GPU.1 etc.

HW enumeration is planned but not yet supported by our dependency visual analytics service DLSPS (DLStreamer-Pipeline-Server) API. Let me follow up with the respective team on this.

Does it currently support cross stream batching? ie with multiple cameras it will batch frames together for inference? this should gain higher performance on use cases with many cameras.

Enabling cross-stream batching is definitely one of our priorities. It is already supported by DLSPS with some drawbacks and requires further effort to enable it in SceneScape.

Let me create separate sub-issues for both discussed limitations.

[tdorauintc]

We know from DLStreamer documentation that it is required to use the same GPU device for decoding that is used for inference. Therefore, from configurability perspective it is sufficient when the specific selected GPU device is specified once per camera pipeline and the system would automatically choose the same device for decoding and inference.

We are going to enable the requested feature in two phases:

1. Enabling support for non-default GPU device selection in camera pipeline configuration.
2. HW enumeration and population of available devices in UI (ideally including NPU) - [Feature Request] Enumerate GPU devices for selection in SceneScape UI #707

We will update on the timeline as soon as it is known.

[tdorauintc]

@oliverheard in the latest 2026.0 release we documented the selection of GPU device for inference in case of multiple GPU devices present in the system.

We are keeping the decoding device generic ({CPU, GPU, AUTO}) because DLStreamer pipeline automatically selects the same GPU for decoding and this is the most performant approach (zero-copy).

If the user decides to set a different GPU for decoding than for inference, it is still possible via editing the pipeline manually in the camera configuration page. This requires some GStreamer knowledge from the user but we consider it a good trade-off since currently we don't have the capability of HW enumeration from the analytics pipeline (DLStreamer Pipeline Server).