[功能需求] 添加idle unload 机制

[MapleEve]

🥰 需求描述

避免一直占着显存

🧐 解决方案

而且再启动，需要先查看哪个GPU显存有空闲再加载进去否则12G显存很占用

📝 补充信息

No response

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Title: [Functional requirements] Add idle unload mechanism

🥰 Description of requirements

Avoid occupying video memory all the time

🧐 Solution

And before starting, you need to check which GPU memory is free before loading it, otherwise the 12G memory will be very occupied.

📝 Supplementary information

No response

[github-actions]

👀 @MapleEve

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[MapleEve]

Fixed in VoScript v0.7.5 and current main.

Public references:

• feat: unload idle GPU models safely #15 added safe idle GPU model unload and reload-time CUDA device selection.
• fix: default idle model unload to 180 seconds #19 made idle unload enabled by default with MODEL_IDLE_TIMEOUT_SEC=180.
• fix: choose GPU per lazy-loaded model #20 updated multi-GPU lazy-load placement so models can choose from visible CUDA devices at load time.
• The v0.7.5 release notes and README/config docs now document the default behavior.

Please upgrade to v0.7.5, pull the latest Docker image, or use current main.

[github-actions]

✅ @MapleEve

This issue is closed, If you have any questions, you can comment and reply.
此问题已经关闭。如果您有任何问题，可以留言并回复。