Project

General

Profile

Bug #2208

Feature #2054: PME on GPU

cuFFT linking

Added by Szilárd Páll over 2 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Affected version - extra info:
Affected version:
Difficulty:
uncategorized
Close

Description

The current mdrun linking against libcufft seems to be incorrect in two ways:
- static linking is not respected (observed symptom: when libcudart is linked statically, libcufft is still linked dynamically);
- RPATH is not set (presumably only shows up with the former)


Related issues

Related to GROMACS - Task #2500: detect and allow linking external clFFT, or no clFFTClosed

History

#1 Updated by Aleksei Iupinov over 2 years ago

  • Parent task set to #2054
  • Affected version changed from git master to 2018

This isn't really about master, but PME GPU changes, starting from https://gerrit.gromacs.org/#/c/6212.
All the effort to link libcufft so far is contained in a single line:

target_link_libraries(libgromacs PRIVATE ${CUDA_CUFFT_LIBRARIES})

(https://gerrit.gromacs.org/#/c/6212/49/src/gromacs/CMakeLists.txt).

The reason for using PRIVATE there is to avoid the target_link_libraries keyword conflict with TNG library.
I see now that it links to the dynamic libcufft, but static linking might require some cmake proficiency and effort,
as described at http://docs.nvidia.com/cuda/cufft/index.html#static-library.
Any improvements are welcome.

#2 Updated by Szilárd Páll over 2 years ago

Aleksei Iupinov wrote:

This isn't really about master, but PME GPU changes, starting from https://gerrit.gromacs.org/#/c/6212.

Well r2017 is as correct or wrong as master -- the code is neither, plus r2017 doesn't even exist (and it'll be merged in into master first anyway).

I see now that it links to the dynamic libcufft, but static linking might require some cmake proficiency and effort,
as described at http://docs.nvidia.com/cuda/cufft/index.html#static-library.
Any improvements are welcome.

That's the low/med prio part of the issue IMO. The high pro part is I think the lack of RPATH when libcudart in linked statically but libcufft ends up linked dynamically.

#3 Updated by Aleksei Iupinov over 2 years ago

  • Priority changed from High to Normal

So I've just built both the master and the first PME patch that requires cufft (gerrit 6212), only with cuda/8.0 module loaded, with no additional cmake options.
The binary of the PME version works with cuda module unloaded;
the RPATH of the master libgromacs.so is $ORIGIN/../lib:/opt/tcbsys/cuda/8.0/lib64/stubs,
the RPATH of the patch libgromacs.so is $ORIGIN/../lib:/opt/tcbsys/cuda/8.0/lib64:/opt/tcbsys/cuda/8.0/lib64/stubs
I assume that means that the target_link_libraries() command actually works correctly for dynamic linking of cufft.
So, indeed, the issue 2 must have been caused from 1, as you have observed those on my build where I've definitely touched the static build variables.

As for issue 1, if the static libcufft linking were to be implemented, would it be the reasonable default behaviour? We have GMX_PREFER_STATIC_LIBS which defaults to OFF; cmake provides CUDA_USE_STATIC_CUDA_RUNTIME which defaults to ON, but for some reason does not mention cufft_static (which has existed since CUDA 6.5, which is our minimum).

#4 Updated by Aleksei Iupinov over 1 year ago

  • Related to Task #2500: detect and allow linking external clFFT, or no clFFT added

Also available in: Atom PDF