Skip to content
This repository was archived by the owner on Jan 28, 2026. It is now read-only.

Commit caf15cc

Browse files
authored
[NPU] Add IPEX_LLM_NPU_MTL to enable support on mtl (#12543)
1 parent c090d16 commit caf15cc

File tree

3 files changed

+11
-0
lines changed

3 files changed

+11
-0
lines changed

docs/mddocs/Quickstart/npu_quickstart.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,9 @@ For `ipex-llm` NPU support, set the following environment variable with active `
9090

9191
```cmd
9292
set BIGDL_USE_NPU=1
93+
94+
:: [optional] for MTL support
95+
set IPEX_LLM_NPU_MTL=1
9396
```
9497

9598
## Python API

python/llm/example/NPU/HF-Transformers-AutoModels/LLM/README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,9 @@ For optimal performance, it is recommended to set several environment variables.
6060
6161
```cmd
6262
set BIGDL_USE_NPU=1
63+
64+
:: [optional] for running models on MTL
65+
set IPEX_LLM_NPU_MTL=1
6366
```
6467
6568
## 3. Run Models

python/llm/src/ipex_llm/transformers/npu_models/convert_mp.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,11 @@ def convert_forward(m, target_m, new_forward):
3232

3333
def optimize_llm_pre(model: torch.nn.Module, qtype, mixed_precision,
3434
quantization_group_size=0, load=False, max_prompt_len=512):
35+
if os.environ.get("IPEX_LLM_NPU_MTL", "0") == "1":
36+
# For MTL support
37+
os.environ["IPEX_LLM_NPU_USE_LEVEL0"] = "0"
38+
os.environ["IPEX_LLM_NPU_DISABLE_COMPILE_OPT"] = "1"
39+
3540
if model.config.model_type == "baichuan":
3641
# process NormHead module in Baichuan2 7B
3742
if hasattr(model, 'lm_head') and model.lm_head is not None:

0 commit comments

Comments
 (0)