[OV][ITT][GPU Plugin] Enable default ITT markers for inference and op submission by tovinkere · Pull Request #33313 · openvinotoolkit/openvino

tovinkere · 2025-12-18T20:07:57Z

Enables default ITT markers for higher level operations such as inference pass, op preparation and submission
Follows the same guidelines to standardize the conventions for namespaces: ov::phases::gpu::inference ov::op::gpu
Supports both synchronous and asynchronous operations

Enabling default GPU ITT markers using standard convention - Part 3

This PR is the third of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation.

The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play PR#33639.
The second PR will standardize ITT markers in the CPU and enhance support to include asynchronous execution PR#33312.
This third PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. Follow standardized conventions as described in 1 and 2
The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity.

Summary of the current PR (PR#3)

Use the same convention standardized in PR#33639
Ensures the namespace for GPU Plugin activity falls under:
ov::phases::gpu::inference
ov::op::gpu

Details:
GPU support is enabled with default ITT markers that support synchronous an asynchronous execution. This PR ensures a standardized convention is followed in namespaces used.

Tickets:
CVS-179230

@isanghao Please review this as you are generally aware of what was discussed

- Enables default ITT markers for higher level operations such as inference pass, op preparation and submission - Follows the same guidelines to standardize the conventions for namespaces: ov::phases::gpu::inference ov::op::gpu - Supports both synchronous and asynchronous operations Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

+ Adding base markers for node execution instead of a prepare() and execute() Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp

isanghao · 2026-01-06T08:35:07Z

src/plugins/intel_gpu/src/plugin/sync_infer_request.cpp

 void SyncInferRequest::infer() {
-    OV_ITT_SCOPED_TASK(itt::domains::intel_gpu_plugin, "SyncInferRequest::infer");
+    // String can be constructed once in the constructor
+    OV_ITT_SCOPED_TASK_BASE(itt::domains::intel_gpu_inference, "SyncInferRequestGPU::infer" +  m_graph->get_runtime_model()->get_name());


out of curiosity, is it OK to change the name here? you commented not to change the name in other place.. [Warning] The strings in ITT_SCOPED_TASK_BASE should NOT be deleted or edited!

@isanghao Please suggest what changes you need. I tried to make the strings in TASK_BASE to be similar for all plugins.

I mean, you changed the name from SyncInferRequest to SyncInferRequestGPU. If it is OK to change the name, why do you leave comment that we should not change the name?

@isanghao Previous task definition was incomplete. Also, it is not enabled by default. I enable the task by default using OV_ITT_SCOPED_TASK_BASE macro and renamed the string to be indicative of being executed by the GPU plugin. This new string cannot be changed as it acts as a SW contract and all tools that use this data will rely on the strings being invariant.

p-durandin · 2026-01-12T09:24:02Z

build_jenkins

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp

- This update fixes the GPU runtime performance regressions due to increased submission times. By caching the required string for each inference, the overheads are eliminated. Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

aobolensk · 2026-01-21T09:11:06Z

build_jenkins

src/plugins/intel_gpu/include/intel_gpu/plugin/sync_infer_request.hpp

isanghao · 2026-01-22T06:17:37Z

src/plugins/intel_gpu/src/plugin/sync_infer_request.cpp

-#include "intel_gpu/runtime/debug_configuration.hpp"

 #include <algorithm>
+#include <functional>


Please do not include style fix in the PR. It makes review difficult.

@isanghao Previous PRs had clang-format failures on changed files, so not I run clang-format as recommended by @praasz. Can defer this till the end in the future.

You don't need to reformat the whole file. You can just change the problematic line to pass the test..

It should consistent with current style in GPU, @isanghao has greater experience in this component I would follow his suggestions

src/plugins/intel_gpu/src/plugin/graph.cpp

src/plugins/intel_gpu/src/graph/network.cpp

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

src/plugins/intel_gpu/include/intel_gpu/plugin/compiled_model.hpp

- Moving string constructor to where it is used Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

tovinkere · 2026-01-27T05:30:38Z

@praasz @aobolensk @isanghao Any progress or next steps?

src/plugins/intel_gpu/include/intel_gpu/plugin/compiled_model.hpp

aobolensk · 2026-01-27T08:22:58Z

build_jenkins

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

- Also update the strings to be consistent across all plugins Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

tovinkere · 2026-01-30T06:40:55Z

@isanghao The style changes have been removed as requested. Should be easily reviewable now.

aobolensk

Overall LGTM

src/plugins/intel_gpu/src/plugin/sync_infer_request.cpp

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

aobolensk · 2026-01-30T08:50:52Z

build_jenkins

isanghao

LGTM for GPU

eparshut · 2026-02-02T18:24:25Z

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp

+    OV_ITT_DOMAIN(intel_gpu_inference, "ov::phases::gpu::inference");
+    // Domain namespace for all of the operators
+    OV_ITT_DOMAIN(intel_gpu_op, "ov::op::gpu");
    OV_ITT_DOMAIN(intel_gpu_plugin);


It would be beneficial to define this domain name as ov::intel_gpu_plugin to maintain consistency with other domains. This ensures the ov:: prefix is present, making it easier for profiling tools to filter and capture all ov events

Suggested change

OV_ITT_DOMAIN(intel_gpu_plugin);

OV_ITT_DOMAIN(intel_gpu_plugin, "ov::intel_gpu_plugin");

tovinkere · 2026-02-19T01:10:27Z

@isanghao @praasz Is this going to be merged?

@isanghao

… submission (#33313) - Enables default ITT markers for higher level operations such as inference pass, op preparation and submission - Follows the same guidelines to standardize the conventions for namespaces: ov::phases::gpu::inference ov::op::gpu - Supports both synchronous and asynchronous operations Enabling default GPU ITT markers using standard convention - Part 3 This PR is the **third** of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation. 1. The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play [PR#33639](#33639). 2. The second PR will standardize ITT markers in the CPU and enhance support to include asynchronous execution [PR#33312](#33312). 3. This **third** PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. Follow standardized conventions as described in 1 and 2 4. The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity. Summary of the current PR (PR#3) Use the same convention standardized in [PR#33639](#33639) Ensures the namespace for GPU Plugin activity falls under: ov::phases::gpu::inference ov::op::gpu Details: GPU support is enabled with default ITT markers that support synchronous an asynchronous execution. This PR ensures a standardized convention is followed in namespaces used. Tickets: [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) @isanghao Please review this as you are generally aware of what was discussed --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

tovinkere requested review from a team as code owners December 18, 2025 20:07

github-actions bot added the category: GPU OpenVINO GPU plugin label Dec 18, 2025

sys-openvino-ci added the ExternalIntelPR External contributor from Intel label Dec 18, 2025

[OV][ITT][GPU Plugin] Reduce ITT events in half

4e989a7

+ Adding base markers for node execution instead of a prepare() and execute() Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

mg-intel assigned isanghao and aobolensk Dec 23, 2025

aobolensk reviewed Dec 23, 2025

View reviewed changes

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp Outdated Show resolved Hide resolved

isanghao reviewed Jan 6, 2026

View reviewed changes

isanghao reviewed Jan 13, 2026

View reviewed changes

src/plugins/intel_gpu/include/intel_gpu/runtime/itt.hpp Show resolved Hide resolved

tovinkere and others added 3 commits January 20, 2026 13:54

Merge branch 'openvinotoolkit:master' into itt_markers_gpu

cf51b56

[OV][ITT] ITT markers for GPU - regression fix

3c51087

- This update fixes the GPU runtime performance regressions due to increased submission times. By caching the required string for each inference, the overheads are eliminated. Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

[OV][ITT] Making ITT string for GPU consistent

2e8591d

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

isanghao reviewed Jan 22, 2026

View reviewed changes

src/plugins/intel_gpu/include/intel_gpu/plugin/sync_infer_request.hpp Show resolved Hide resolved

isanghao reviewed Jan 22, 2026

View reviewed changes

aobolensk reviewed Jan 22, 2026

View reviewed changes

src/plugins/intel_gpu/src/plugin/graph.cpp Show resolved Hide resolved

aobolensk reviewed Jan 22, 2026

View reviewed changes

src/plugins/intel_gpu/src/graph/network.cpp Outdated Show resolved Hide resolved

[OV][ITT] Address reviewer feedback

0e7d052

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

praasz reviewed Jan 22, 2026

View reviewed changes

src/plugins/intel_gpu/include/intel_gpu/plugin/compiled_model.hpp Outdated Show resolved Hide resolved

[OV][ITT] Addressing reviewer comments

79a0773

- Moving string constructor to where it is used Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

isanghao reviewed Jan 27, 2026

View reviewed changes

src/plugins/intel_gpu/include/intel_gpu/plugin/compiled_model.hpp Show resolved Hide resolved

tovinkere added 2 commits January 29, 2026 22:13

[OV][ITT] Undo style changes

34f6696

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

[OV][ITT] Add the pref regression fix

480cb06

- Also update the strings to be consistent across all plugins Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

aobolensk approved these changes Jan 30, 2026

View reviewed changes

src/plugins/intel_gpu/src/plugin/sync_infer_request.cpp Outdated Show resolved Hide resolved

[OV][ITT] Fixed string separator in naming convention

62272d3

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

isanghao approved these changes Feb 2, 2026

View reviewed changes

eparshut reviewed Feb 2, 2026

View reviewed changes

praasz approved these changes Feb 19, 2026

View reviewed changes

praasz added this to the 2026.1 milestone Feb 19, 2026

praasz added this pull request to the merge queue Feb 19, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 19, 2026

praasz added this pull request to the merge queue Feb 20, 2026

Merged via the queue into openvinotoolkit:master with commit 86c0fd9 Feb 20, 2026
190 checks passed

	OV_ITT_DOMAIN(intel_gpu_plugin);
	OV_ITT_DOMAIN(intel_gpu_plugin, "ov::intel_gpu_plugin");

Conversation

tovinkere commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

p-durandin commented Jan 12, 2026

Uh oh!

Uh oh!

aobolensk commented Jan 21, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tovinkere commented Jan 27, 2026

Uh oh!

Uh oh!

aobolensk commented Jan 27, 2026

Uh oh!

tovinkere commented Jan 30, 2026

Uh oh!

aobolensk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aobolensk commented Jan 30, 2026

Uh oh!

isanghao left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tovinkere commented Feb 19, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

tovinkere commented Dec 18, 2025 •

edited

Loading