Skip to content

Commit 52edf5e

Browse files
authored
fix mtp acceptance rate decline (#6470)
1 parent 51f812a commit 52edf5e

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

fastdeploy/worker/input_batch.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -890,7 +890,8 @@ def reset_model_inputs(self) -> None:
890890
self.block_tables = paddle.clone(self.target_model_input_batch["block_tables"])
891891
self.input_ids = paddle.clone(self.target_model_input_batch["input_ids"])
892892
fill_paddle_tensor(self, "input_ids_cpu", -1)
893-
self.seq_lens_this_time_buffer = paddle.clone(self.target_model_input_batch["seq_lens_this_time"])
893+
# acceptance rate decline when reset seq_lens_this_time
894+
# self.seq_lens_this_time_buffer = paddle.clone(self.target_model_input_batch["seq_lens_this_time"])
894895

895896
self.seq_lens_encoder = paddle.clone(self.target_model_input_batch["seq_lens_encoder"])
896897
self.seq_lens_decoder = paddle.clone(self.target_model_input_batch["seq_lens_decoder"])

0 commit comments

Comments
 (0)