0.0.34 claims to support PyTorch>=2.10 but it doesn't?

[levicki]

Release notes say:

Pre-built binary wheels are available for PyTorch 2.10.0 (and later).

However:

(venv) d:\AI\ComfyUI>pip install xformers==0.0.34
Collecting xformers==0.0.34
  Using cached xformers-0.0.34-cp39-abi3-win_amd64.whl.metadata (1.0 kB)
Requirement already satisfied: numpy in d:\ai\comfyui\venv\lib\site-packages (from xformers==0.0.34) (2.4.0)
INFO: pip is looking at multiple versions of xformers to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install xformers==0.0.34 because these package versions have conflicting dependencies.

The conflict is caused by:
    xformers 0.0.34 depends on torch==2.10.0
    The user requested (constraint) torch==2.11.0.dev20260124+cu130

Additionally, some packages in these conflicts have no matching distributions available for your environment:
    torch

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip to attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

My environment is Python 3.14.2, Windows 10 x64.

Note that the same error happens if I explicitly use pip install xformers==0.0.34 --index-url https://download.pytorch.org/whl/cu130.

Finally, if I install xformers and torch 2.10+cu130 I am still missing triton:

pytorch version: 2.10.0+cu130
A matching Triton is not available, some optimizations will not be enabled
Traceback (most recent call last):
  File "d:\AI\ComfyUI\venv\Lib\site-packages\xformers\__init__.py", line 57, in _is_triton_available
    import triton  # noqa
    ^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'triton'
xformers version: 0.0.34

I have tried building xformers with CUDA 13.0/13.1 myself but as I reported earlier the build is broken in the sense that with CUDA 13.x it doesn't seem to produce any triton related code in the wheel. Maybe that's on purpose and maybe it was announced somewhere but if so I am not aware of it. Would be nice to get a clarification.

Please advise.

[silvertuanzi]

In __init__.py it's just a warning, so it can just be ignored.

Or you can run pip install triton-windows to get triton.

[levicki]

In init.py it's just a warning, so it can just be ignored.

It can't be ignored because when you run the pipeline (in my case in ComfyUI) it fails at runtime because kernels are not available.

Or you can run pip install triton-windows to get triton.

Until CUDA 13.x this worked "automagically" when you install or build xformers and if I am not mistaken triton was built in the xformers tree. What changed? Is this now official advice to install 3rd party fork?

[levicki]

Ok, I installed triton-windows and I installed both xformers prebuilt wheel for CUDA 13 and torch 2.10+cu130 which is supposed to be supported.

What I am getting is this during runtime:

File "d:\AI\ComfyUI\venv\Lib\site-packages\xformers\ops\fmha\dispatch.py", line 83, in _run_priority_list
    raise NotImplementedError(msg)
NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:
     query       : shape=(2, 2752, 10, 64) (torch.float16)
     key         : shape=(2, 2752, 10, 64) (torch.float16)
     value       : shape=(2, 2752, 10, 64) (torch.float16)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`fa3F@0.0.0` is not supported because:
    requires device with capability <= (9, 0) but your GPU has capability (12, 0) (too new)
    operator wasn't built - see `python -m xformers.info` for more info
`fa2F@0.0.0` is not supported because:
    operator wasn't built - see `python -m xformers.info` for more info
`cutlassF-pt` is not supported because:
    requires device with capability <= (9, 0) but your GPU has capability (12, 0) (too new)

Running python -m xformers.info in venv produces this:

(venv) D:\AI\ComfyUI>python -m xformers.info
xFormers 0.0.34
memory_efficient_attention.ckF:                    unavailable
memory_efficient_attention.ckB:                    unavailable
memory_efficient_attention.ck_splitKF:             unavailable
memory_efficient_attention.cutlassF-pt:            available
memory_efficient_attention.cutlassB-pt:            available
memory_efficient_attention.cutlassF-blackwell:     unavailable
memory_efficient_attention.cutlassB-blackwell:     unavailable
memory_efficient_attention.fa2F@0.0.0:             unavailable
memory_efficient_attention.fa2B@0.0.0:             unavailable
memory_efficient_attention.fa3F@0.0.0:             unavailable
memory_efficient_attention.fa3B@0.0.0:             unavailable
memory_efficient_attention.fa3F_splitKV@0.0.0:     unavailable
memory_efficient_attention.triton_splitKF:         available
indexing.scaled_index_addF:                        available
indexing.scaled_index_addB:                        available
indexing.index_select:                             available
sp24.sparse24_sparsify_both_ways:                  available
sp24.sparse24_apply:                               available
sp24.sparse24_apply_dense_output:                  available
sp24._sparse24_gemm:                               available
sp24._cslt_sparse_mm_search@0.0.0:                 available
sp24._cslt_sparse_mm@0.0.0:                        available
is_triton_available:                               True
pytorch.version:                                   2.10.0+cu130
pytorch.cuda:                                      available
gpu.compute_capability:                            12.0
gpu.name:                                          NVIDIA GeForce RTX 5090
dcgm_profiler:                                     unavailable
build.info:                                        available
build.cuda_version:                                1300
build.hip_version:                                 None
build.python_version:                              3.10.11
build.torch_version:                               2.10.0+cu130
build.env.TORCH_CUDA_ARCH_LIST:                    7.5 8.0+PTX 8.0 9.0a 10.0a 10.3a 11.0a 12.0a 12.1a
build.env.PYTORCH_ROCM_ARCH:                       None
build.env.XFORMERS_BUILD_TYPE:                     Release
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS:        None
build.env.NVCC_FLAGS:                              -allow-unsupported-compiler
build.env.XFORMERS_PACKAGE_FROM:                   wheel-v0.0.34
build.nvcc_version:                                13.0.88
source.privacy:                                    open source

So pretty much the only conclusion is that xformers has become useless as it doesn't ship with any code that existed before in the official wheels and even the code that's there such as cutlassF-pt is not working on Blackwell because it's "too new". Exactly how is something launched in Q4 2024 "too new" at the end of January 2026? Rhetorical question of course.

Which brings us back to the other issue I reported back in September 2025 where I said that compiling for 12.0 arch doesn't produce working wheels and that issue has been ignored for 4 months and crippled unusable wheels have been built officially and released.

@lw Will anyone please give us any sort of offical response on how is this planned to be addressed?

[silvertuanzi]

In fact, in this commit xformers nominally enabled cutlass Blackwell, but the cutlass Blackwell component has never actually been successfully built; meanwhile, cutlass-pt is prohibited from being used on Blackwell GPUs.

The temporary workaround is to modify xformers/ops/fmha/cutlass.py at line 170, to remove the Blackwell restriction for cutlass-pt, and then wait for the xformers maintainers to fix the build process.

    OPERATOR = (
        get_operator("aten", "_efficient_attention_forward")
        if is_pt_cutlass_compatible()
        else None
    )
-   CUDA_MAXIMUM_COMPUTE_CAPABILITY = (9, 0)
+   CUDA_MAXIMUM_COMPUTE_CAPABILITY = (12, 1)
    SUPPORTED_DEVICES: Set[str] = {"cuda"}
    SUPPORTED_DTYPES: Set[torch.dtype] = {torch.float, torch.half, torch.bfloat16}

However, by the way, this issue didn’t first appear in this version. It should have already existed in torch 2.9.0 + xformers 0.0.33. Since you’re on an RTX 5090, shouldn’t it have started causing problems even earlier?

[levicki]

However, by the way, this issue didn’t first appear in this version. It should have already existed in torch 2.9.0 + xformers 0.0.33. Since you’re on an RTX 5090, shouldn’t it have started causing problems even earlier?

I reported it back in September 2025 so yes it started earlier — it started with CUDA 13.0 if I recall correctly.

Up to CUDA 12.8 it was possible to compile xformers with ARCH set to 8.9 and it would work on Blackwell (probably via PTX fallback?). But since CUDA 13.0 Cutlass stuff doesn't seem to exist in output.

What I can't understand how this wasn't caught eariler by the maintainers? I mean when you get a wheel which is just 3-4 MB in size instead of at least an order of magnitude larger then something is very wrong with your build.

Finally, even specifying just ARCH 8.9 (i.e. without 12.0) when building also doesn't provide same functionality we had with CUDA 12.8 builds on Blackwell. So basically CUDA 13.x builds could also possibly be broken on 40x0 series.

[jovan2009]

Looking at the size of the wheels (https://github.com/facebookresearch/xformers/actions/runs/21357344724) the size of cuda 13 windows wheel is... something to behold. I wonder why this discrepancy comparing with cuda 13 linux wheel.

My own xformers windows wheel built with cuda 13.1 against pytorch 2.11.0 nightly from latest xformers git is about 1.5 mb, so even smaller, I wonder why. It seems is working in ComfyUI on my 5060 Ti - meaning it's not crashing, but I don't know exactly what it really does or if it does something at that wheel size. I have built separately Flash Attention 2.8.3 from git and xformers aknowledges it when I run `xformers.info`, I hope it uses it. FA 2 wheel has about 130 MB so if "big" xformers wheels include some form of FA 2, that would explain size difference.

[levicki]

If you check the above xformers.info I posted from the official wheel:

memory_efficient_attention.cutlassF-blackwell:     unavailable
memory_efficient_attention.cutlassB-blackwell:     unavailable
memory_efficient_attention.fa2F@0.0.0:             unavailable
memory_efficient_attention.fa2B@0.0.0:             unavailable
memory_efficient_attention.fa3F@0.0.0:             unavailable
memory_efficient_attention.fa3B@0.0.0:             unavailable
memory_efficient_attention.fa3F_splitKV@0.0.0:     unavailable

It is clear the official wheel is lacking memory efficient attention code for blackwell.

If you built it separately somehow that explains why yours is "working".

[jovan2009]

If you check the above xformers.info I posted from the official wheel:

memory_efficient_attention.cutlassF-blackwell:     unavailable
memory_efficient_attention.cutlassB-blackwell:     unavailable
memory_efficient_attention.fa2F@0.0.0:             unavailable
memory_efficient_attention.fa2B@0.0.0:             unavailable
memory_efficient_attention.fa3F@0.0.0:             unavailable
memory_efficient_attention.fa3B@0.0.0:             unavailable
memory_efficient_attention.fa3F_splitKV@0.0.0:     unavailable

It is clear the official wheel is lacking memory efficient attention code for blackwell.

If you built it separately somehow that explains why yours is "working".

I got used to not have Flash Attention built inside xformers, it used to crash if I try to build xformers with it enabled so I resorted to disable it myself at build time via environment variable - and now it doesn't even try to build FA anyway, environment variable or not. I turned to building FA 2 myself and installing it separately, so my xformers.info shows it as enabled. FA 3 I think it's not compatible with rtx 5xxx GPUs (as far as I understand) so I ignore that. Cutlass-blackwell, yeah, I don't have that and I wonder if anybody had ever seen that enabled. It's total chaos in my mind about what "blackwell" is or should be, because there is "consumer blackwell" and "data center blackwell" and what works on one is not necessarily working on the other, so I am pretty accustomed with "things not working on my card", not to mention that because I am on Windows chances to "things not working" are increased.

[levicki]

It is all the same chip.

This is all NVIDIA's fault anyway, they are sitting on their posteriors and 1+ year after product has been launched (Q4 2024) it's still "too new" and not fully supported and utilized in software no matter if it's their own (CUDA, CUTLASS, etc) or 3rd party.

[jovan2009]

It is all the same chip.

Well, look at the current cuda compiler
https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/#gpu-name-gpuname-arch

Before Blackwell - 8 arches
Blackwell - 15 arches

... I'm not even mad

[levicki]

Yes but sm_120 is a base subset of all the others so it's not like those are really that much different.

[jovan2009]

Yes but sm_120 is a base subset of all the others so it's not like those are really that much different.

Yeah, but there are some consequences, sometimes. I remember one instance, a few weeks ago, when you could not compile llama.cpp as native arch without specifying 120f instead of 120 (before the llama.cpp developers fixed the issue).

[levicki]

I don't think that's the arch problem. By the way, setup.py is blocking use of any other sm except sm80 and sm90 for flash attention and it outright doesn't even try to compile it under win32 with CUDA version > 13.0. So people who wrote this code know why they disabled it (reason is if I am not mistaken that new CUDA requires 128-byte alignment on stack for sm120 and MSVC can't provide it).

[lw]

Thanks for flagging the incorrect constraint of the PyTorch dependency. We did intend to make the release compatible with later versions too but we didn't execute that correctly. We hope to have fixed that with c42443a, but you'll need to wait for the next release to benefit from it.

[lw]

As for Blackwell support, it's unlikely to be coming to xFormers anytime soon. Our recommandation is to either migrate to MSLK (https://github.com/meta-pytorch/MSLK) which has some Blackwell support, or directly use Flash-Attention 4 (if your system / driver allows it), as it provides support for all GPU generations since Ampere.