Dynamic Vram: the massive memory optimization is now enabled by default.

[comfyanonymous]

Update: Dynamic Vram is now in stable ComfyUI.

Like everyone else we have also been very annoyed by the recent hike in RAM prices. Hopefully this will offer some relief.

Dynamic vram is the new ComfyUI memory optimization that should massively reduce ram usage and generally speed things up on Nvidia hardware on Windows and Linux. WSL may never be supported.

At the moment I am writing this it is only enabled by default in the latest git version so you will need to update to this and update your requirements if you want to try it out. If you are on portable you can do: update/update_comfyui.bat to update to the latest git version. Using latest pytorch 2.10 cu130 is recommended for best performance. If you are on desktop just wait a few days and it might be in the next stable version.

What you should notice:

• Lower RAM usage.
• Faster workflow execution on memory constrained hardware.
• No more vram OOMs because weights were not offloaded enough.
• Faster initial model Load.
• Faster lora applying.
• Run models larger than your RAM without paging.
• Higher vram usage is normal due to using vram more effectively.
• Windows task manager may not actually show a lower ram usage, weights will still be cached in ram if you have plenty of memory in order to speed things up. Unlike previous behaviour these cached weights will never end up in your page file and will be unloaded to make room when other applications need the memory.

Next steps:

• Fixing any reported performance and other regressions.
• AMD support.

We will very likely be posting a more detailed blog post in the very near future. For now if you want more technical explanation see: https://github.com/Comfy-Org/comfy-aimdo and #11845

If you notice any issues related to this please open an issue with a detailed report (full logs, workflow, your hardware and operating system) and we will fix it. For performance issues make sure to compare the total workflow execution time and not the it/s.

[monnky]

https://www.youtube.com/post/Ugkx-WQaQ1Z0P7KFpOyEX3IUiAzQ42AA9i8x

[comfyanonymous]

That's not related to dynamic vram. Dynamic vram is to lower ram usage not vram.

[darqsatyr1c0n]

That's not related to dynamic vram. Dynamic vram is to lower ram usage not vram.

do you have any preliminary numbers about lower usage? For how much? I don't see any difference with aimdo or not.

[GlamoramaAttack]

@monnky:

https://wccftech.com/nvidia-new-gpu-driver-is-a-disaster-it-has-now-been-pulled-back/

NVIDIA’s New GPU Driver is a Disaster & It Has Now Been Pulled Back

We have discovered a bug in the Game Ready and Studio 595.59 WHQL drivers and have removed the downloads temporarily while our team investigates. For users that have already installed this driver and are experiencing issues with fan control, please roll back to 591.86 WHQL.

• NVIDIA

The news is two days old.

[monnky]

That's not related to dynamic vram. Dynamic vram is to lower ram usage not vram.

sorry my bad

[monnky]

@monnky:

https://wccftech.com/nvidia-new-gpu-driver-is-a-disaster-it-has-now-been-pulled-back/

NVIDIA’s New GPU Driver is a Disaster & It Has Now Been Pulled Back

We have discovered a bug in the Game Ready and Studio 595.59 WHQL drivers and have removed the downloads temporarily while our team investigates. For users that have already installed this driver and are experiencing issues with fan control, please roll back to 591.86 WHQL.

• NVIDIA

The news is two days old.

👍

[Vmars666]

is it possible to to disable it ? its make thing so slow on my rtx5070ti 12 vram with flux2. klein9b but ltx2 is going really better.
i notice a lot less of pagefile accumulation too, so its good for multi models workflows.

[rattus128]

weve fixed a few things last 3 days. If its still slow, feel free to post before and after logs with your flux2 workflow.

https://github.com/Comfy-Org/ComfyUI/issues

[nonetrix]

Will this allow me to finally train SDXL/Anima LoRAs on 16GB AMD maybe and not have to use --cpu-vae? Training is super inefficient on AMD, ComfyUI training just didn't work when I tried it last and sd-scripts just OOMs even through I should have enough. Might post pone my GPU upgrade if so when supported on AMD and Comfy training werks well

[mmoalem]

what is the price paid for the optimization? speed? more writes to disk/pagefile?

[anr2me]

what is the price paid for the optimization? speed? more writes to disk/pagefile?

It reduce system memory usage, which also mean less swap/page file usage.

[anr2me]

Btw, only Nvidia and AMD GPU were mentioned 🤔 No Intel XPU support?

[Goldlionren]

希望能够尽快支持XPU

[Rogala]

The comfy-aimdo page states that only Windows 11+ is supported. What about Windows 10? Will it not be supported at all, or will it be fully functional?

[pbz134]

It's working fine on my Win10 laptop, terminal also says Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.

[3qyemo]

cu128不可以吗？

[anr2me]

cu130 is the recommended cuda version probably because of this https://blog.comfy.org/p/new-comfyui-optimizations-for-nvidia

An important caveat is that currently, ComfyUI only supports NVFP4 acceleration if you are running PyTorch built with CUDA 13.0 (cu130). Otherwise, while the model will still function, your sampling may actually be up to 2x slower than fp8. If you experience issues trying to get the full speed of NVFP4 models, checking your PyTorch version is the first thing you should try!

[ryukbk]

This Dynamic Vram feature breaks TensorRT nodes such as ComfyUI_TensorRT and ComfyUI-Rife-Tensorrt.

[E] [defaultAllocator.cpp::nvinfer1::internal::DefaultAllocator::allocate::53] Error Code 1: Cuda Runtime (In nvinfer1::internal::DefaultAllocator::allocate at C:\_src\common\dispatch\defaultAllocator.cpp:53)

Launching ComfyUI with --disable-dynamic-vram is the workaround.

[anr2me]

May be you should be opening an issue to be tracked 🤔 otherwise it could sink into the ocean of discussion and forgotten😅

[spike4379]

This is causing problems for me on a 4090 using klein9b. I keep getting out of memory errors now... before it was working perfectly. I have installed everything cleanly.

EDIT: the moment after posting this comfy was stuck doing a vae encode for klein 9b, I decided to simply close terminal like everyone does... Bluescreen.
Amazing update thankyou, I'll be rolling back because this feature is buggy as hell and untested

[yamfun]

I have 4070. After updated comfy, NV driver, python 3.13, cuda 130, pytorch +cu130. the speed of klein9b-fp8 is changed from 21s/22s to 16s/18s.

(But since I updated so many things I am not sure which one bring me the improvements)

[bhusann]

Dynamamic vram gives oom if reserve vram is in launch option, if I remove the reserve vram, now everything works great without oom, atleast for me.

[kgpabhi]

Great work. Writing the model to pagefile makes sense if it's modified, otherwise it's better to just dump the model and read it again in case of low RAM+VRAM. Good for lifespan of SSDs.

[lebakerino]

Getting this error intermittently since activating this update when using LTX-2:

00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000

Had blue screens twice when it's happened, and its happened maybe 4-5 times now since yesterday, always at the end at the vaedecode/save video. happens often after a batch of videos often I think though only sometimes, not sure what but got this from eventlog for one of the BSOD:

Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: python.exe (26416) consumed 44263952384 bytes, Cubase.exe (14952) consumed 2299682816 bytes, and waterfox.exe (31192) consumed 1212239872 bytes.

Never had this issue with the previous RAM management tbh but this is appealing due to being able to spare the SSD/Paging file abuse?

[rattus128]

how much RAM do you have and which LTX2 variant? If you want to submit a full bug report for with full log I can see if I can fix it.

[lebakerino]

00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000
aimdo: src/pyt-cu-plug-alloc-async.c:97:ERROR:aimdo_cuda_free_async: could not account free at 00000002B5B7E000

how much RAM do you have and which LTX2 variant? If you want to submit a full bug report for with full log I can see if I can fix it.

Hi I made a post: #12784

Thanks

[AnnieTheEagle]

This broke my workflow entirely with error: Expected all tensors to be on the same device, but got index is on cuda:0 (had plenty of RAM and VRAM available -- RTX 5090 and 128 GB of System RAM) and did try to disable custom nodes like Inspire's WildcardEncode as well, but got the same error on Load Checkpoint:

Enabling --disable-dynamic-vram fixed the issue completely however.

Not sure if I need to update my GPU driver for this to work but given NVIDIA rolled back 595.59.

[AnnieTheEagle]

Updating to 595.71 does not resolve this.

[monnky]

Updating to 595.71 does not resolve this.

it might be the error from ComfyUI node not GPU driver.

[Axseeee]

After this update, processing has become much slower. Is there a way to disable dynamic vram on the desktop version?

[mrreclusive3-beep]

had to look this up, this broke my entire comfy workflow with multigpu...
found this, going to try it. --disable-dynamic-vram

but still pretty sure sage attention and multigpu is broke.
not everyone is doing this on a 2060, i do not want my text encoders and model's on the same gpu...
takes 5 minutes for it to even start when before it was instant...

[anr2me]

Btw, 2060 is a Turing GPU isn't 🤔 does Sageattention2 support Turing GPU?

[mrreclusive3-beep]

Btw, 2060 is a Turing GPU isn't 🤔 does Sageattention2 support Turing GPU?

lol no idea. the problem is these changes are made for people using lower resources, I have 2 4090's and a 4080, and this update broke all of it, even with --disable-dynamic-vram it still is forcing everything onto a single gpu.

right now i have to go rewrite everyone's nodes to fix this...
TypeError: patched_load_state_dict_guess_config() got an unexpected keyword argument 'disable_dynamic'

[mrreclusive3-beep]

ok, for anyone else on multigpu..
line 33 - 45 of checkpoint_multigpu.py

def patched_load_state_dict_guess_config(sd, output_vae=True, output_clip=True, output_clipvision=False, embedding_directory=None, output_model=True, model_options={}, te_model_options={}, metadata=None, disable_dynamic=False): """Patched checkpoint loader with MultiGPU and DisTorch2 device placement support.""" from . import set_current_device, set_current_text_encoder_device, get_current_device, get_current_text_encoder_device sd_size = sum(p.numel() for p in sd.values() if hasattr(p, 'numel')) config_hash = str(sd_size) device_config = checkpoint_device_config.get(config_hash) distorch_config = checkpoint_distorch_config.get(config_hash) if not device_config and not distorch_config: return original_load_state_dict_guess_config(sd, output_vae, output_clip, output_clipvision, embedding_directory, output_model, model_options, te_model_options, metadata, disable_dynamic)

[anr2me]

Well if the custom nodes need to be patched in order to work properly, you should report this issue on the custom nodes repository, so they can officially be updated to support the new ComfyUI.

[mrreclusive3-beep]

Well if the custom nodes need to be patched in order to work properly, you should report this issue on the custom nodes repository, so they can officially be updated to support the new ComfyUI.

im sure they will, it is just annoying that pretty much all custom loaders just broke. and there is a lot of custom loaders in the comfy eco system, when they should of put a catch so disable_dynamic defaults to false if one is not supplied by the loader.

[bollerdominik]

Had to disable this on my 5090 and 4090 ComfyUI Servers. It causes a slowdown of more than 2x in our examples. From 2 second per image to average of 4.4 seconds

We are using https://github.com/ltdrdata/ComfyUI-Impact-Pack and https://github.com/jags111/efficiency-nodes-comfyui

RAM is 64GB and above

It seems like it is constantly loading (parts of) the model again. Any caching that was working before is not effective anymore

[mrreclusive3-beep]

yeah, this was the same issue I faced, even though i could hold full models on gpu, it was forcing load/unload between between each step because it wanted to.

[CHKE9]

causing this for some reason

[girzai]

please make it possible to disable this thing who seems to broke a lot of comfYUI setup ( especially when loading large models )

[anr2me]

You can use --disable-dynamic-vram

[mrreclusive3-beep]

You can use --disable-dynamic-vram

this doesn't seem to be working correctly either, it still kicks in, i can no longer use over 20gb of my 24gb gpu because dynamic vram kicks in and over compensates and oom's, even with --disable-dynamic-vram. (using fp4's, i don't think comfy is seeing it correctly, its seeing a 14gb fp4 as 60+gb for some reason)
and even worse, i can't even use it without --disable-dynamic-vram because it just crashes on startup.

[Axseeee]

Please add a disable option on Desktop Version.
I don't know why, but it really slows down all processes.
5090 128Gb

[jorismak]

Been very happy with this on my RTX 5060ti under Windows.
Under Linux on Lunar Lake (so Arc iGPU) it has ruined ComfyUI a bit for me. Workflows that worked fine now suddenly OOM and get comfyUI and it's terminal killed (as in, it's the OS that decides to klil processes to save ram, not ComfyUI who errors out with out-of-memory).

On iGPU it was always reported to ComfyUI that my entire system RAM is also available as video RAM. So, I had to use 'reserve-ram' before to let it reserve 12gb of 'video ram'. Basically giving me a good 16+ gb of video RAM while not starving my system RAM.

I now start ComfyUI with --reserve-vram and with --disable-dynamic-vram but it doesn't help. Workflows eat up all my ram/videoram till my system has nothing and Linux kicks in. Before, with just --reserve-vram (and no dynamic vram in ComfyUI) workflows would just work as if I had 16gb of VideoRAM. Which means that after loading the text encoders and doing prompting, it would unload those models and then start loading the diffusion models. Now it doesn't.

So, not only do I need this to keep ComfyUI some what usable on my system, there seems to be a problem where it doesn't respect --reserve-vram and even with --disable-dynamic-vram the loading behaviour in ComfyUI is now changes so much that it doesn't unload anymore and causes crashes.

[qbyss]

This is terrible with my 4090 using weights stored on HDD. I have 500-1000s delay for each generation. If I use the --disable-dynamic-vram for the same workflow I get down to 400s first generation, then 120s for second generation which is expected for that workflow. So the ability to disable this "feature" is essential to my workflow. Otherwise in this specific configuration it's unusable for video generation.

[anr2me]

Have you tried the latest nightly? They recently fixed the issue with dynamic vram and smart memory that causing models to keep being loaded from storage even when you have enough RAM/VRAM to store them.