Commit 3ee4d48
authored
Fix ERNIE 4.5 model builder: rope_attrs and config architecture name (#2007)
## Description
This PR fixes the ERNIE 4.5 model builder to align with upstream renames
and the current codebase.
### 1. Use `rope_attrs` instead of `rotemb_attrs`
In
[b92970c](b92970c)
(Add OpenAI's gpt-oss, #1678), `rotemb_attrs` was replaced by
`rope_attrs` across the codebase. In [PR
#1862](#1862) (Model
builder refactoring), the shared `Model` class and its `rope_attrs` were
moved to `builders/base.py`. The ERNIE builder in `builders/ernie.py`
was still referring to the old attribute name. This PR updates
`ErnieModel` to use `rope_attrs` (inherited from the base via
`MistralModel`) for:
- **Interleaved RoPE**: `self.rope_attrs["interleaved"] = 1`
- **Compression ratio (RoPE scaling)**:
`self.rope_attrs["rescale_factors"] = 1.0 / config.compression_ratio`
when `compression_ratio` is set
### 2. Match updated Hugging Face config `architectures`
The official ERNIE 4.5 config on Hugging Face was updated:
`architectures` was changed from `Ernie4_5_ForCausalLM` to
`Ernie4_5ForCausalLM` (underscore removed before `ForCausalLM`):
-
[baidu/ERNIE-4.5-0.3B-Base-PT](https://huggingface.co/baidu/ERNIE-4.5-0.3B-Base-PT/commit/bf9499229d96e16442fd63992195b7369c7b2657)
-
[baidu/ERNIE-4.5-0.3B-PT](https://huggingface.co/baidu/ERNIE-4.5-0.3B-PT/commit/018ae39b66b9d73e17e9092434de6acd4dd4856a)
This PR updates the architecture check in `builder.py` from
`Ernie4_5_ForCausalLM` to `Ernie4_5ForCausalLM` so that the builder
correctly recognizes current ERNIE 4.5 models.1 parent f02bebe commit 3ee4d48
2 files changed
+3
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
213 | 213 | | |
214 | 214 | | |
215 | 215 | | |
216 | | - | |
| 216 | + | |
217 | 217 | | |
218 | 218 | | |
219 | 219 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
0 commit comments