[NPUW] Fixed Gemma2 4K sliding window work with short prompts on NPU#32891
Merged
dmatveev merged 4 commits intoopenvinotoolkit:masterfrom Jan 21, 2026
Merged
[NPUW] Fixed Gemma2 4K sliding window work with short prompts on NPU#32891dmatveev merged 4 commits intoopenvinotoolkit:masterfrom
dmatveev merged 4 commits intoopenvinotoolkit:masterfrom
Conversation
github-actions
bot
added
category: NPU
AsyaPronina
force-pushed
the
gemma2_swa
branch
from
77fe1c0 to
86896a6
Compare
AlexanderKalistratov
approved these changes
Nov 19, 2025
dmatveev
requested changes
Nov 20, 2025
Contributor
dmatveev
left a comment
dmatveev
left a comment
There was a problem hiding this comment.
Probably it is a good chance to ask the transformation team on how to make things right
| OPENVINO_MATCHER_PASS_RTTI("npuw::LLMCompiledModel::Phi3SlidingMask2"); | ||
|
|
||
| Phi3SlidingMask2() { | ||
| Phi3SlidingMask2(const std::shared_ptr<ov::Model>& model) { |
Contributor
There was a problem hiding this comment.
Normally we have explicit here, but what's the reason to pass model to a pass that can, in fact, be applied to any other model?
Comment on lines
+671
to
+682
| std::shared_ptr<ov::Node> matched_position_ids = nullptr; | ||
| std::shared_ptr<ov::Node> matched_attention_mask = nullptr; | ||
| for (const auto& i : model->inputs()) { | ||
| if (i.get_any_name() == "position_ids") { | ||
| matched_position_ids = i.get_node_shared_ptr(); | ||
| } | ||
| if (i.get_any_name() == "attention_mask") { | ||
| matched_attention_mask = i.get_node_shared_ptr(); | ||
| } | ||
| } | ||
| OPENVINO_ASSERT(matched_position_ids, "position_ids input is not found!"); | ||
| OPENVINO_ASSERT(matched_attention_mask, "attention_mask input is not found!"); |
Contributor
There was a problem hiding this comment.
Why can't you make it through pass? Or, maybe, outside the pass?
Contributor
Author
There was a problem hiding this comment.
Because it is long chain of operations till the position_ids and attention_mask starting from point of the pattern and this chain can differ between models.
AsyaPronina
force-pushed
the
gemma2_swa
branch
2 times, most recently
from
44b0932 to
569aaad
Compare
| rewr.run_on_model(model); | ||
| ov::pass::Manager manager; | ||
| manager.register_pass<Phi3SlidingMask2>(); | ||
| if (!manager.run_passes(model)) { |
Contributor
There was a problem hiding this comment.
I think it would be better to move this logic into Phi3SlidingMask2 pass to make it handle all the cases.
And probably rename Phi3SlidingMask2 into Phi3SlidingMask and Phi3SlidingMask rename into Phi3SlidingMask_pattern_transformers_451 or something like that.
AsyaPronina
force-pushed
the
gemma2_swa
branch
from
569aaad to
91a477e
Compare
Naseer-010
pushed a commit
to Naseer-010/openvino
that referenced
this pull request
Feb 18, 2026
…penvinotoolkit#32891) ### Details: - *Relaxed Phi3SlidingMask2 pattern in order to allow `attention_mask` and `position_ids` to be located more flexibly in the model without breaking the sliding window mask calculation* ### Tickets: - *EISW-190610*
Naseer-010
pushed a commit
to Naseer-010/openvino
that referenced
this pull request
Feb 18, 2026
…penvinotoolkit#33819) ### Details: - *Hot-fix after openvinotoolkit#32891 - *Phi3 Sliding window patching is applied before attention mask is added to Whisper models. But as Whisper model doesn't need SWA patching, then added an early return if neither `attention_mask` nor `position_ids` were found* ### Tickets: - *N/A*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details:
attention_maskandposition_idsto be located more flexibly in the model without breaking the sliding window mask calculationTickets: