Convert GROMACS 1 OpenMP thread per rank fatal error into a warning
Dear GROMACS team,
GROMACS defines a Fatal error when one selects 1 OpenMP thread per rank.
The specific GROMACS message that I am referring to is:
Your choice of number of MPI ranks and amount of resources results in using 1
OpenMP threads per rank, which is most likely inefficient. The optimum is
usually between 2 and 6 threads per rank. If you want to run with this setup,
specify the -ntomp option. But we suggest to change the number of MPI ranks.
We (at CSCS) have crossed several input files that were bound by non bonded computations on the GPU and the best achievable performance was obtained one was using only one OpenMP thread per rank with max number of MPI ranks per node, where all ranks share the GPU.
An example can be seen on CSCS's user portal (https://user.cscs.ch/computing/applications/gromacs/)
I believe that setting this as a Fatal error is not very helpful for, at least, these cases.
Can you please convert this "error" into a warning?
- Status changed from New to Feedback wanted
The test system has rather strange parameter settings. It combines a 1.4 nm cut-off with a 0.12 nm FFT grid spacing. A much coarser spacing could be used, which changes the computational load quite a lot. It also uses Berendsen T-coupling, which is bad, but should not affect performance.
Do you have other systems where 1 OpenMP thread per rank is best?
#2 Updated by Erik Lindahl 3 months ago
Victor: First, you should be able to circumvent it by explicitly having the flag "-ntomp 1".
But, I still agree that if there are use cases where this actually helps performance (even if it's not the settings we would use), we shouldn't turn it into a fatal error - in particular not when it works just fine in the code, and it's rather just a performance optimisation check.