Bounding box drawing in appsink

[hteeyeoh]

Description:

I'm working with a dlstreamer pipeline that processes an input stream using gvadetect for object detection and outputs frames to appsink for rendering. I need to configure the pipeline to detect and display bounding boxes only for specific labels of interest.

Current Behavior

The pipeline detects multiple object types correctly
Bounding boxes are drawn for all detected objects, including unwanted labels
Attempted to use model-proc JSON configuration with gvadetect but it still renders bounding boxes for undesired labels

Expected Behavior

Only display bounding boxes for specified labels of interest
Filter out unwanted detections from the rendered output

Example

Interested label: "person"
Issue: Bounding boxes for "car" and other labels are still being drawn on the output frames

Question

Is there a proper method or configuration to filter detection results so that only bounding boxes for specific labels are rendered in the appsink output?

[brmarkus]

What does the pipeline exactly look like? The plugin gvadetect doesn't do the rendering of BB.
What have you tried in the model-proc file so far?
Which object-detection model do you use?

The latest version of gvawatermark just recently got extended to support include/exclude filters for rendering of BBs - via displ-cfg, have a closer look into https://github.com/open-edge-platform/dlstreamer/blob/main/docs/source/elements/gvawatermark.md

You could do the rendering in your application - as you mentioned appsink; you could process the metadata and prevent from rendering the BB depending on the classid.

Could you imagine to finetune/retrain a model to only detect your wanted type of objects?

[nszczygl9]

@hteeyeoh please provide a pipeline and model which you use

[brmarkus]

Another - but rather expensive - idea is to add a Python file filter into the pipeline:

gst-launch-1.0 filesrc location=video.mp4 ! decodebin3 ! \
  gvadetect model=yolov8.xml model-proc=yolov8.json ! queue ! \
  gvapython class=LabelFilter module=label_filter.py ! queue ! \
  gvawatermark ! autovideosink

And with the following Python file:

# label_filter.py
from gstgva import VideoFrame

ALLOWED_LABELS = {"car", "bicycle"}

class LabelFilter:
    def process_frame(self, frame: VideoFrame) -> bool:
        for roi in list(frame.regions()):
            label = roi.label()
            if label not in ALLOWED_LABELS:
                frame.remove_region(roi)
        return True

In case of a C++ application, have a look into https://github.com/open-edge-platform/dlstreamer/blob/main/samples/gstreamer/cpp/draw_face_attributes/README.md to implement a per-frame callback, analyze the attached metadata and e.g. draw your BB.
The same example, but in Pyton: https://github.com/open-edge-platform/dlstreamer/blob/main/samples/gstreamer/python/draw_face_attributes/README.md

[tbujewsk]

Here is example how to configure displ-cfg of gvawatermark to limit the annotations number
displ-cfg=show-labels=true,show-roi=person:car:truck

[hteeyeoh]

Hi @nszczygl9 ,
I'm just experimenting using a simple detection pipeline. I'm using the dlstreamer docker container image from OEP: https://github.com/open-edge-platform/edge-ai-libraries/tree/main/microservices/dlstreamer-pipeline-server

@brmarkus , I remembered i tried something like this but it just bump into error . Anyways will try again with your suggestion and share the error.

@tbujewsk , this is the documentation i referred: https://docs.openedgeplatform.intel.com/2025.2/edge-ai-libraries/dl-streamer/elements/gvawatermark.html it doesnt shows those element-properties. I maybe outdated here.

[hteeyeoh]

I have tried all the approach mentioned above. Below is the current docker images and pipeline configuration I have tested:

scenario:
rtsp input stream -> decode -> detection -> user configured specific label of interest -> show only bounding box for interested label object on UI application

docker image: intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc3
detection model: yolov8s (even tried for others like yolo11s etc.)

Experiments:

1. using gvawatermark 'displ-cfg=show-labels=true,show-roi=person' (not working)
   "pipeline": "rtspsrc location={source[uri]} protocols=tcp latency=100 name=source ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=BGRA ! gvadetect name=detection device=CPU model="/home/pipeline-server/detection_models/{parameters[detection_model_name]}/public/{parameters[detection_model_name]}/FP32/{parameters[detection_model_name]}.xml" model-proc="/home/pipeline-server/gvapython/publisher/yolo-v8.json" ! queue leaky=downstream max-size-buffers=1 max-size-time=0 max-size-bytes=0 ! gvawatermark displ-cfg=show-labels=true,show-roi=person ! appsink name=appsink",

2. using gvapython label_filter.py (not working)
   "pipeline": "rtspsrc location={source[uri]} protocols=tcp latency=100 name=source ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=BGRA ! gvadetect name=detection device=CPU model="/home/pipeline-server/detection_models/{parameters[detection_model_name]}/public/{parameters[detection_model_name]}/FP32/{parameters[detection_model_name]}.xml" model-proc="/home/pipeline-server/gvapython/publisher/yolo-v8.json" ! queue leaky=downstream max-size-buffers=1 max-size-time=0 max-size-bytes=0 ! gvapython class=LabelFilter module=/home/pipeline-server/gvapython/publisher/label_filter.py name=label_filter ! appsink name=appsink"

Output:
Expectation: not to show bounding box on car
Actual: bounding box show on car label detected

Not sure is it something additional that have been processed in dlstreamer docker images in OEP?

[brmarkus]

@tbujewsk Can you confirm whether the DLPS Docker container of ´intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc3´ already contains the additional properties/attributes of gvawatermark?

[brmarkus]

@brmarkus , I remembered i tried something like this but it just bump into error . Anyways will try again with your suggestion and share the error.

Yes, please share errors/warnings; feel free to add additonal log messages.
Please also share the used model-proc.
Can you try using an "official" model (and an official, unchanged, default model-proc) for reproduction, please? Maybe your used model uses different labels, class-ids, different spellings?

[hteeyeoh]

@brmarkus , I remembered i tried something like this but it just bump into error . Anyways will try again with your suggestion and share the error.

Yes, please share errors/warnings; feel free to add additonal log messages. Please also share the used model-proc. Can you try using an "official" model (and an official, unchanged, default model-proc) for reproduction, please? Maybe your used model uses different labels, class-ids, different spellings?

maybe previously i have used gvametaconvert somehow before gvapython or after im not sure hence i saw that error. Without gvametaconvert it dont have error. Just that the labels still showing...
Im using yolov8s and the default mode-proc json from here: https://github.com/open-edge-platform/dlstreamer/blob/main/samples/gstreamer/model_proc/public/yolo-v8.json without any modification

[nszczygl9]

@tbujewsk Can you confirm whether the DLPS Docker container of ´intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc3´ already contains the additional properties/attributes of gvawatermark?

yes, this image contains these additional properties

[nszczygl9]

@hteeyeoh please refer to the official docs from dev: https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/elements/gvawatermark.html
the link which you sent refers to the previous official release https://docs.openedgeplatform.intel.com/2025.2/edge-ai-libraries/dl-streamer/elements/gvawatermark.html

[brmarkus]

Inspired by #700 (comment) and using the DL-Streamer Docker container docker pull intel/dlstreamer:2026.0.0-ubuntu24-rc3 I can confirm gvawatermark being able to selectively display specific bounding boxes only (in this exotic case it's dog and person):

docker run -it \
    -v /dev:/dev --net=host --device=/dev/dri --group-add=110 \
    -v ~/.Xauthority:/home/dlstreamer/.Xauthority -v /tmp/.X11-unix -e DISPLAY=$DISPLAY -e XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR \
    -v /dev/bus/usb \
    -v /localdisk/videos:/localdisk/videos:rw \
    --device-cgroup-rule "c 81:* rmw" \
    --device-cgroup-rule "c 189:* rmw" \
    intel/dlstreamer:2026.0.0-ubuntu24-rc3 \
    /bin/bash

Using your referenced Model-Proc file from https://github.com/open-edge-platform/dlstreamer/blob/main/samples/gstreamer/model_proc/public/yolo-v8.json

Using model yolov8s

Pipeline:

gst-launch-1.0 filesrc location='./Serenity - HD DVD Trailer.mp4' ! decodebin3 ! gvadetect model=public/yolov8s/FP16/yolov8s.xml model-proc=yolo-v8.json device=GPU ! queue ! gvawatermark displ-cfg=show-labels=true,show-roi=dog:person ! videoconvertscale ! ximagesink

[hteeyeoh]

Inspired by #700 (comment) and using the DL-Streamer Docker container docker pull intel/dlstreamer:2026.0.0-ubuntu24-rc3 I can confirm gvawatermark being able to selectively display specific bounding boxes only (in this exotic case it's dog and person):

docker run -it \
    -v /dev:/dev --net=host --device=/dev/dri --group-add=110 \
    -v ~/.Xauthority:/home/dlstreamer/.Xauthority -v /tmp/.X11-unix -e DISPLAY=$DISPLAY -e XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR \
    -v /dev/bus/usb \
    -v /localdisk/videos:/localdisk/videos:rw \
    --device-cgroup-rule "c 81:* rmw" \
    --device-cgroup-rule "c 189:* rmw" \
    intel/dlstreamer:2026.0.0-ubuntu24-rc3 \
    /bin/bash

Using your referenced Model-Proc file from https://github.com/open-edge-platform/dlstreamer/blob/main/samples/gstreamer/model_proc/public/yolo-v8.json

Using model yolov8s

Pipeline:

gst-launch-1.0 filesrc location='./Serenity - HD DVD Trailer.mp4' ! decodebin3 ! gvadetect model=public/yolov8s/FP16/yolov8s.xml model-proc=yolo-v8.json device=GPU ! queue ! gvawatermark displ-cfg=show-labels=true,show-roi=dog:person ! videoconvertscale ! ximagesink

the pipeline flow mostly same with what I did. Not sure why it doesnt work on my case.. Let me try to replicate your pipeline on my end and check whether it works or not.

Thanks.

[hteeyeoh]

Inspired by #700 (comment) and using the DL-Streamer Docker container docker pull intel/dlstreamer:2026.0.0-ubuntu24-rc3 I can confirm gvawatermark being able to selectively display specific bounding boxes only (in this exotic case it's dog and person):

docker run -it \
    -v /dev:/dev --net=host --device=/dev/dri --group-add=110 \
    -v ~/.Xauthority:/home/dlstreamer/.Xauthority -v /tmp/.X11-unix -e DISPLAY=$DISPLAY -e XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR \
    -v /dev/bus/usb \
    -v /localdisk/videos:/localdisk/videos:rw \
    --device-cgroup-rule "c 81:* rmw" \
    --device-cgroup-rule "c 189:* rmw" \
    intel/dlstreamer:2026.0.0-ubuntu24-rc3 \
    /bin/bash

Using your referenced Model-Proc file from https://github.com/open-edge-platform/dlstreamer/blob/main/samples/gstreamer/model_proc/public/yolo-v8.json

Using model yolov8s

Pipeline:

gst-launch-1.0 filesrc location='./Serenity - HD DVD Trailer.mp4' ! decodebin3 ! gvadetect model=public/yolov8s/FP16/yolov8s.xml model-proc=yolo-v8.json device=GPU ! queue ! gvawatermark displ-cfg=show-labels=true,show-roi=dog:person ! videoconvertscale ! ximagesink

This approach doesn't function with the intel/dlstreamer-pipeline-server:2026.0.0-ubuntu24-rc3 image. It appears you're working with the base DLStreamer image rather than the DLSPS (DLStreamer Pipeline Server) image. I'm wondering if there are additional processes or modifications implemented in the DLSPS image. Interestingly, even when the gvawatermark element is removed from the pipeline, bounding boxes are still rendered automatically. This leads me to suspect that there might be some internal configuration in DLSPS that overrides user-defined settings. This is just my hypothesis based on the observed behavior."

[brmarkus]

Could you try a simplest possible pipeline including gvawatermark (and filtering)?
Can you share DLPS logs (console output? or with docker logs -f <CONTAINER_NAME>), please? To see what pipeline finally is used, if DLPS has pre-processed and restructured the pipeline.