Project

General

Profile

Bug #2472

Updated by Szilárd Páll over 1 year ago

The r2018 code does not allow setting only the OpenMP thread count in a GPU run (in tMPI builds), but as the OpenMP thread count handling was changed and part of the reporting was short-circuited (i.e. the env var-related reporting from the @omp_nthreads@ module), this leads to potentially confusing error messages that lack context.

$ OMP_NUM_THREADS=2 gmx mdrun -nsteps 0
[...]

<pre>
GROMACS: gmx mdrun, version 2018
Executable: /opt/tcbsys/gromacs/2018/AVX2_256/bin/gmx
Data prefix: /opt/tcbsys/gromacs/2018/AVX2_256
Working dir: /home/pszilard/projects/gromacs/testing/water-048k
Command line:
gmx mdrun -nsteps 0

Back Off! I just backed up md.log to ./#md.log.95#
Reading file topol.tpr, VERSION 4.6-beta3-dev-20121222-492378e (single precision)
Note: file tpx version 82, software tpx version 112
The number of OpenMP threads was set by environment variable OMP_NUM_THREADS to 2

-------------------------------------------------------
Program: gmx mdrun, version 2018
Source file: src/gromacs/taskassignment/resourcedivision.cpp (line 224)

Fatal error:
When using GPUs, setting the number of OpenMP threads without specifying the
number of ranks can lead to conflicting demands. Please specify the number of
thread-MPI ranks as well (option -ntmpi).

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors
-------------------------------------------------------
</pre>

In contrast, in r2016, besides there being no error, it is pretty clear that the environment variable's value is used (that may have not been set by the user / at the time of mdrun invocation):

<pre>
GROMACS: gmx mdrun, version 2016
Executable: /opt/tcbsys/gromacs/2016/AVX2_256/bin/gmx
Data prefix: /opt/tcbsys/gromacs/2016/AVX2_256
Working dir: /home/pszilard/projects/gromacs/testing/water-048k
Command line:
gmx mdrun -nsteps 0

Back Off! I just backed up x_2016.log to ./#x_2016.log.1#


Running on 1 node with total 4 cores, 8 logical cores, 2 compatible GPUs
Hardware detected:
CPU info:
Vendor: Intel
Brand: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
SIMD instructions most likely to fit this hardware: AVX2_256
SIMD instructions selected at GROMACS compile time: AVX2_256

Hardware topology: Basic
GPU info:
Number of GPUs detected: 2
#0: NVIDIA GeForce GTX 1080, compute cap.: 6.1, ECC: no, stat: compatible
#1: NVIDIA GeForce GTX 960, compute cap.: 5.2, ECC: no, stat: compatible

Reading file topol.tpr, VERSION 4.6-beta3-dev-20121222-492378e (single precision)
Note: file tpx version 82, software tpx version 110
Changing nstlist from 10 to 40, rlist from 1 to 1.101

The number of OpenMP threads was set by environment variable OMP_NUM_THREADS to 2

Overriding nsteps with value passed on the command line: 0 steps, 0 ps

Using 2 MPI threads
Using 2 OpenMP threads per tMPI thread

2 compatible GPUs are present, with IDs 0,1
2 GPUs auto-selected for this run.
Mapping of GPU IDs to the 2 PP ranks in this node: 0,1

</pre>

Back