Installation with CUDA

[aerpearson]

I am attempting to build relax using CUDA, but it fails when running ./waf. The configure step runs correctly using the following code: FCFLAGS="-I/usr/lib64/gfortran/modules" ./waf configure --use-fftw --use-cuda --cuda-dir=/usr/local/cuda

However, it fails on building it with ./waf, with the following error:
[ 5/22] Compiling src/cugreen.cu
[ 9/22] Compiling src/cuelastic.cu
nvcc fatal : Unknown option '-osrc/cugreen.cu.1.o'

nvcc fatal : Unknown option '-osrc/cuelastic.cu.1.o'

It seems to be an issue only with compiling those two codes, but I can't find anything in particular about that kind of error. Thanks in advance for any help, and sorry if this isn't really the kind of thing to put in an issue!

[smasuti]

Dear Anna, Could you please look for the config.log file? Also this code was used and tested long back ~2015/16. So there might be some cudaflags that need to be edited depending on the GPU and its architecture. Best, Sagar

…

On Fri, Mar 14, 2025 at 3:46 AM Anna Pearson ***@***.***> wrote: I am attempting to build relax using CUDA, but it fails when running ./waf. The configure step runs correctly using the following code: FCFLAGS="-I/usr/lib64/gfortran/modules" ./waf configure --use-fftw --use-cuda --cuda-dir=/usr/local/cuda image.png (view on web) <https://github.com/user-attachments/assets/ebc84098-c790-4373-b445-e6a64fe63795> However, it fails on building it with ./waf, with the following error: [ 5/22] Compiling src/cugreen.cu [ 9/22] Compiling src/cuelastic.cu nvcc fatal : Unknown option '-osrc/cugreen.cu.1.o' nvcc fatal : Unknown option '-osrc/cuelastic.cu.1.o' It seems to be an issue only with compiling those two codes, but I can't find anything in particular about that kind of error. Thanks in advance for any help, and sorry if this isn't really the kind of thing to put in an issue! — Reply to this email directly, view it on GitHub <#12>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI6JRNDHXDHJEQRS7T5ZRL2UH7TZAVCNFSM6AAAAABY7LPP4WVHI2DSMVQWIX3LMV43ASLTON2WKOZSHEYTQNBZGQZDIMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> [image: aerpearson]*aerpearson* created an issue (geodynamics/relax#12) <#12> I am attempting to build relax using CUDA, but it fails when running ./waf. The configure step runs correctly using the following code: FCFLAGS="-I/usr/lib64/gfortran/modules" ./waf configure --use-fftw --use-cuda --cuda-dir=/usr/local/cuda image.png (view on web) <https://github.com/user-attachments/assets/ebc84098-c790-4373-b445-e6a64fe63795> However, it fails on building it with ./waf, with the following error: [ 5/22] Compiling src/cugreen.cu [ 9/22] Compiling src/cuelastic.cu nvcc fatal : Unknown option '-osrc/cugreen.cu.1.o' nvcc fatal : Unknown option '-osrc/cuelastic.cu.1.o' It seems to be an issue only with compiling those two codes, but I can't find anything in particular about that kind of error. Thanks in advance for any help, and sorry if this isn't really the kind of thing to put in an issue! — Reply to this email directly, view it on GitHub <#12>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI6JRNDHXDHJEQRS7T5ZRL2UH7TZAVCNFSM6AAAAABY7LPP4WVHI2DSMVQWIX3LMV43ASLTON2WKOZSHEYTQNBZGQZDIMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[aerpearson]

Dear Sagar,

The GPU I'm using is Tesla K80 and the cuda version is release 11.4.

Here's the cuda parts of the config.log file, let me know if you want me to attach the whole thing:
Checking for program 'nvcc'
/usr/local/cuda/bin/nvcc
find program=['nvcc'] paths=['/usr/local/bin', '/usr/local/cuda/bin', '/usr/local/bin', '/usr/local/bin', '/usr/bin', '/usr/local/sbin', '/usr/sbin', '/home/aerp/bin'] var='NVCC' -> ['/usr/local/cuda/bin/nvcc']

Checking for library cudart
==>

int main(int argc, char **argv) {
(void)argc; (void)argv;
return 0;
}

<==
[1/2] Compiling ^[[32mbuild/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/test.cpp^[[0m

['/usr/bin/g++', '-I/usr/local/cuda/include', '../test.cpp', '-c', '-o/home/aerp/relax/build/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/testbuild/test.cpp.1.o']
[2/2] Linking ^[[33mbuild/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/testbuild/testprog^[[0m

['/usr/bin/g++', 'test.cpp.1.o', '-o', '/home/aerp/relax/build/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/testbuild/testprog', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/usr/local/cuda/lib64', '-lcudart']
yes

Checking for library cufft
==>

int main(int argc, char **argv) {
(void)argc; (void)argv;
return 0;
}

<==
[1/2] Compiling ^[[32mbuild/.conf_check_58a21f147535a961b92a60d6812316b8/test.cpp^[[0m

['/usr/bin/g++', '-I/usr/local/cuda/include', '../test.cpp', '-c', '-o/home/aerp/relax/build/.conf_check_58a21f147535a961b92a60d6812316b8/testbuild/test.cpp.1.o']
[2/2] Linking ^[[33mbuild/.conf_check_58a21f147535a961b92a60d6812316b8/testbuild/testprog^[[0m

['/usr/bin/g++', 'test.cpp.1.o', '-o', '/home/aerp/relax/build/.conf_check_58a21f147535a961b92a60d6812316b8/testbuild/testprog', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/usr/local/cuda/lib64', '-lcufft']
yes

Checking for cuda.h
==>
#include <cuda.h>

int main(int argc, char **argv) {
(void)argc; (void)argv;
return 0;
}

<==
[1/2] Compiling ^[[32mbuild/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/test.c^[[0m

['/usr/bin/gcc', '-O3', '-fPIC', '-I/usr/local/cuda/include', '../test.c', '-c', '-o/home/aerp/relax/build/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/testbuild/test.c.1.o']
[2/2] Linking ^[[33mbuild/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/testbuild/testprog^[[0m

['/usr/bin/gcc', 'test.c.1.o', '-o', '/home/aerp/relax/build/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/testbuild/testprog', '-Wl,-rpath,/usr/local/cuda/lib64', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/usr/local/cuda/lib64', '-lcudart', '-lcufft', '-lstdc++']
yes

Thanks so much!
Anna

[smasuti]

Dear Anna, Sorry I am not able to tell you what the issue is from the logs. Could you set *arch*=compute_37 and *code*=sm_37 in the *wscript* file and check? If the above change does not solve the problem then, probably we can have a zoom call where you can share your screen and sort out. But before we do this please check if the Tesla K80 has sufficient memory to handle the research problem you want to investigate with the Relax GPU version. As I see, K80 has 2 cuda cores of 12 GB, that's not much. Best, Sagar

…

On Fri, Mar 14, 2025 at 10:43 PM Anna Pearson ***@***.***> wrote: Dear Sagar, The GPU I'm using is Tesla K80 and the cuda version is release 11.4. Here's the cuda parts of the config.log file, let me know if you want me to attach the whole thing: Checking for program 'nvcc' /usr/local/cuda/bin/nvcc find program=['nvcc'] paths=['/usr/local/bin', '/usr/local/cuda/bin', '/usr/local/bin', '/usr/local/bin', '/usr/bin', '/usr/local/sbin', '/usr/sbin', '/home/aerp/bin'] var='NVCC' -> ['/usr/local/cuda/bin/nvcc'] Checking for library cudart ==> int main(int argc, char **argv) { (void)argc; (void)argv; return 0; } <== [1/2] Compiling ^[[32mbuild/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/test.cpp^[[0m ['/usr/bin/g++', '-I/usr/local/cuda/include', '../test.cpp', '-c', '-o/home/aerp/relax/build/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/testbuild/test.cpp.1.o'] [2/2] Linking ^[[33mbuild/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/testbuild/testprog^[[0m ['/usr/bin/g++', 'test.cpp.1.o', '-o', '/home/aerp/relax/build/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/testbuild/testprog', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/usr/local/cuda/lib64', '-lcudart'] yes Checking for library cufft ==> int main(int argc, char **argv) { (void)argc; (void)argv; return 0; } <== [1/2] Compiling ^[[32mbuild/.conf_check_58a21f147535a961b92a60d6812316b8/test.cpp^[[0m ['/usr/bin/g++', '-I/usr/local/cuda/include', '../test.cpp', '-c', '-o/home/aerp/relax/build/.conf_check_58a21f147535a961b92a60d6812316b8/testbuild/test.cpp.1.o'] [2/2] Linking ^[[33mbuild/.conf_check_58a21f147535a961b92a60d6812316b8/testbuild/testprog^[[0m ['/usr/bin/g++', 'test.cpp.1.o', '-o', '/home/aerp/relax/build/.conf_check_58a21f147535a961b92a60d6812316b8/testbuild/testprog', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/usr/local/cuda/lib64', '-lcufft'] yes Checking for cuda.h ==> #include <cuda.h> int main(int argc, char **argv) { (void)argc; (void)argv; return 0; } <== [1/2] Compiling ^[[32mbuild/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/test.c^[[0m ['/usr/bin/gcc', '-O3', '-fPIC', '-I/usr/local/cuda/include', '../test.c', '-c', '-o/home/aerp/relax/build/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/testbuild/test.c.1.o'] [2/2] Linking ^[[33mbuild/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/testbuild/testprog^[[0m ['/usr/bin/gcc', 'test.c.1.o', '-o', '/home/aerp/relax/build/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/testbuild/testprog', '-Wl,-rpath,/usr/local/cuda/lib64', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/usr/local/cuda/lib64', '-lcudart', '-lcufft', '-lstdc++'] yes Thanks so much! Anna — Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI6JROMYTYQABO2I3JOTRT2UME2PAVCNFSM6AAAAABY7LPP4WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRVGI4DGMJYGY> . You are receiving this because you commented.Message ID: ***@***.***> [image: aerpearson]*aerpearson* left a comment (geodynamics/relax#12) <#12 (comment)> Dear Sagar, The GPU I'm using is Tesla K80 and the cuda version is release 11.4. Here's the cuda parts of the config.log file, let me know if you want me to attach the whole thing: Checking for program 'nvcc' /usr/local/cuda/bin/nvcc find program=['nvcc'] paths=['/usr/local/bin', '/usr/local/cuda/bin', '/usr/local/bin', '/usr/local/bin', '/usr/bin', '/usr/local/sbin', '/usr/sbin', '/home/aerp/bin'] var='NVCC' -> ['/usr/local/cuda/bin/nvcc'] Checking for library cudart ==> int main(int argc, char **argv) { (void)argc; (void)argv; return 0; } <== [1/2] Compiling ^[[32mbuild/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/test.cpp^[[0m ['/usr/bin/g++', '-I/usr/local/cuda/include', '../test.cpp', '-c', '-o/home/aerp/relax/build/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/testbuild/test.cpp.1.o'] [2/2] Linking ^[[33mbuild/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/testbuild/testprog^[[0m ['/usr/bin/g++', 'test.cpp.1.o', '-o', '/home/aerp/relax/build/.conf_check_3756d58eae39d54fc9dff1b984d3e69d/testbuild/testprog', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/usr/local/cuda/lib64', '-lcudart'] yes Checking for library cufft ==> int main(int argc, char **argv) { (void)argc; (void)argv; return 0; } <== [1/2] Compiling ^[[32mbuild/.conf_check_58a21f147535a961b92a60d6812316b8/test.cpp^[[0m ['/usr/bin/g++', '-I/usr/local/cuda/include', '../test.cpp', '-c', '-o/home/aerp/relax/build/.conf_check_58a21f147535a961b92a60d6812316b8/testbuild/test.cpp.1.o'] [2/2] Linking ^[[33mbuild/.conf_check_58a21f147535a961b92a60d6812316b8/testbuild/testprog^[[0m ['/usr/bin/g++', 'test.cpp.1.o', '-o', '/home/aerp/relax/build/.conf_check_58a21f147535a961b92a60d6812316b8/testbuild/testprog', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/usr/local/cuda/lib64', '-lcufft'] yes Checking for cuda.h ==> #include <cuda.h> int main(int argc, char **argv) { (void)argc; (void)argv; return 0; } <== [1/2] Compiling ^[[32mbuild/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/test.c^[[0m ['/usr/bin/gcc', '-O3', '-fPIC', '-I/usr/local/cuda/include', '../test.c', '-c', '-o/home/aerp/relax/build/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/testbuild/test.c.1.o'] [2/2] Linking ^[[33mbuild/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/testbuild/testprog^[[0m ['/usr/bin/gcc', 'test.c.1.o', '-o', '/home/aerp/relax/build/.conf_check_a5cfc6d0e2acbb0b78a0aedef8f84687/testbuild/testprog', '-Wl,-rpath,/usr/local/cuda/lib64', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/usr/local/cuda/lib64', '-lcudart', '-lcufft', '-lstdc++'] yes Thanks so much! Anna — Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI6JROMYTYQABO2I3JOTRT2UME2PAVCNFSM6AAAAABY7LPP4WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRVGI4DGMJYGY> . You are receiving this because you commented.Message ID: ***@***.***>

[aerpearson]

Hi Sagar,
I changed those flags in the wscript and there wasn't any difference in the errors or the build/config.log.

I also just did a test with the non-Cuda version and got a maximum resident set size of 8 GB, so I think it is doable with the K80 memory?

Thanks,
Anna