run peft_infer.py and get TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]]

[flyingcornjuice]

System Info / 系統信息

After I finetuning the model (cogvlm2-llama3-chat-19B) and run the peft_infer.py，the inference cannot be done.

the error information:
root@autodl-container-119b47bef5-dd95cf60:~/autodl-tmp/CogVLM2-main/finetune_demo# python peft_infer.py
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████| 8/8 [00:09<00:00, 1.15s/it]
image path >>>>> /root/autodl-tmp/CogVLM2-main/175196420936689.png
Human:这个图片描述了什么
Traceback (most recent call last):
File "/root/autodl-tmp/CogVLM2-main/finetune_demo/peft_infer.py", line 56, in
input_by_model = model.build_conversation_input_ids(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/autodl-tmp/CogVLM2-main/models/modules/transformers_modules/THUDM/cogvlm2-llama3-chat-19B/f592f291cf528389b2e4776b1e84ecdf6d71fbe3/modeling_cogvlm.py", line 825, in build_conversation_input_ids
text_ids = tokenizer.encode(text, add_special_tokens=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2704, in encode
encoded_inputs = self.encode_plus(
^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 3095, in encode_plus
return self._encode_plus(
^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/transformers/tokenization_utils_fast.py", line 627, in _encode_plus
batched_output = self._batch_encode_plus(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/transformers/tokenization_utils_fast.py", line 553, in _batch_encode_plus
encodings = self._tokenizer.encode_batch(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]]

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

• The official example scripts / 官方的示例脚本
• My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

peft_infer:
import torch
from PIL import Image
from transformers import AutoModelForCausalLM, AutoTokenizer

Loading PEFT model

MODEL_PATH = "THUDM/cogvlm2-llama3-chat-19B" # The path to the base model (read tokenizer only)
PEFT_MODEL_PATH = "/root/autodl-tmp/CogVLM2-main/finetune_demo/output/checkpoint_epoch_10_step_100" # The path to the PEFT model

DEVICE = 'cuda' if torch.cuda.is_available() else 'cpu'
TORCH_TYPE = torch.bfloat16 if torch.cuda.is_available() and torch.cuda.get_device_capability()[0] >= 8 else torch.float16
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
PEFT_MODEL_PATH,
torch_dtype=TORCH_TYPE,
trust_remote_code=True,
device_map="auto",
).to(DEVICE).eval()

The following code is the same as the one in basic_demo/cli_demo.py

text_only_template = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: {} ASSISTANT:"

while True:
image_path = input("image path >>>>> ")
if image_path == '':
print('You did not enter image path, the following will be a plain text conversation.')
image = None
text_only_first_query = True
else:
image = Image.open(image_path).convert('RGB')

history = []

while True:
    query = input("Human:")
    if query == "clear":
        break

    if image is None:
        if text_only_first_query:
            query = text_only_template.format(query)
            text_only_first_query = False
        else:
            old_prompt = ''
            for _, (old_query, response) in enumerate(history):
                old_prompt += old_query + " " + response + "\n"
            query = old_prompt + "USER: {} ASSISTANT:".format(query)
    if image is None:
        input_by_model = model.build_conversation_input_ids(
            tokenizer,
            query=query,
            history=history,
            template_version='chat'
        )
    else:
        input_by_model = model.build_conversation_input_ids(
            tokenizer,
            query=query,
            history=history,
            images=[image],
            template_version='chat'
        )
    inputs = {
        'input_ids': input_by_model['input_ids'].unsqueeze(0).to(DEVICE),
        'token_type_ids': input_by_model['token_type_ids'].unsqueeze(0).to(DEVICE),
        'attention_mask': input_by_model['attention_mask'].unsqueeze(0).to(DEVICE),
        'images': [[input_by_model['images'][0].to(DEVICE).to(TORCH_TYPE)]] if image is not None else None,
    }
    # add any transformers params here.
    gen_kwargs = {
        "max_new_tokens": 2048,
        "pad_token_id": 128002,  # avoid warning of llama3
    }
    with torch.no_grad():
        outputs = model.generate(**inputs, **gen_kwargs)
        outputs = outputs[:, inputs['input_ids'].shape[1]:]
        response = tokenizer.decode(outputs[0], skip_special_tokens=True)
        print("\nCogVLM2:", response)
    history.append((query, response))

Expected behavior / 期待表现

the model should return me with an answer instead of an error