Wheels for rocm7.1

[Calandracas606]

❓ Questions and Help

I've noticed the lack of pre-built wheels for rocm7.1.

Wheels are available for rocm6.4 from the pytorch index: https://download.pytorch.org/whl/rocm6.4

These work fine:

xFormers 0.0.32.post2
memory_efficient_attention.ckF:                    available
memory_efficient_attention.ckB:                    available
memory_efficient_attention.ck_splitKF:             available
memory_efficient_attention.cutlassF-pt:            unavailable
memory_efficient_attention.cutlassB-pt:            unavailable
memory_efficient_attention.fa2F@0.0.0:             unavailable
memory_efficient_attention.fa2B@0.0.0:             unavailable
memory_efficient_attention.fa3F@0.0.0:             unavailable
memory_efficient_attention.fa3B@0.0.0:             unavailable
memory_efficient_attention.fa3F_splitKV@0.0.0:     unavailable
memory_efficient_attention.triton_splitKF:         available
indexing.scaled_index_addF:                        available
indexing.scaled_index_addB:                        available
indexing.index_select:                             available
sp24.sparse24_sparsify_both_ways:                  available
sp24.sparse24_apply:                               available
sp24.sparse24_apply_dense_output:                  available
sp24._sparse24_gemm:                               available
sp24._cslt_sparse_mm_search@0.0.0:                 available
sp24._cslt_sparse_mm@0.0.0:                        available
swiglu.dual_gemm_silu:                             available
swiglu.gemm_fused_operand_sum:                     available
swiglu.fused.p.cpp:                                available
is_triton_available:                               True
pytorch.version:                                   2.8.0+rocm6.4
pytorch.cuda:                                      available
gpu.compute_capability:                            11.0
gpu.name:                                          Radeon RX 7900 XTX
dcgm_profiler:                                     unavailable
build.info:                                        available
build.cuda_version:                                None
build.hip_version:                                 6.4.43484-123eb5128
build.python_version:                              3.9.23
build.torch_version:                               2.8.0+rocm6.4
build.env.TORCH_CUDA_ARCH_LIST:                    
build.env.PYTORCH_ROCM_ARCH:                       None
build.env.XFORMERS_BUILD_TYPE:                     Release
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS:        None
build.env.NVCC_FLAGS:                              -allow-unsupported-compiler
build.env.XFORMERS_PACKAGE_FROM:                   wheel-v0.0.32.post2
source.privacy:                                    open source

Unfortunately none are available for rocm7.1 at https://download.pytorch.org/whl/nightly/rocm7.1

This requires manual building, which can be a bit confusing, especially for architectures not supported by composable_kernel (see ROCm/composable_kernel#1958)

side note: xformers can still build on these systems by exporting XFORMERS_CK_FLASH_ATTN=0

xformers does build successfully for rocm7.1, using packages from the pytorch nightly index:

XFORMERS_CK_FLASH_ATTN=0 pip install git+https://github.com/facebookresearch/xformers.git@v0.0.33.post2#egg=xformers --no-build-isolation

It would be beneficial to have xformers available in the pytorch rocm7.1 index

Please let me know if this is the wrong place to make such a request.

[ComradePenguin-1917]

Where are you running the linux system? WSL or physical machine?

[Calandracas606]

Where are you running the linux system? WSL or physical machine?

Docker on a Linux host

[lw]

xFormers tries to match the PyTorch build matrix, and the latest stable PyTorch version (2.9.1) targeted ROCm 6.4. We have updated xFormers to build against ROCm 7.1 but you'll need to wait for the next PyTorch release to get binary wheels.