Project

General

Profile

Bug #1568

inconsistent/incorrect threading checks and reporting in mdrun

Added by Szilárd Páll about 5 years ago. Updated about 3 years ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
mdrun
Target version:
-
Affected version - extra info:
Affected version:
Difficulty:
uncategorized
Close

Description

mdrun does some multithreading consistency checks and reporting. There are two uses-cases in which the behavior is incorrect:
  • although passing the number of threads is possible either with env. vars. or command line options, due to the scattered and incomplete checks this is not fully possible, e.g. setting -ntomp_pme requires setting -ntomp, OMP_NUM_THREADS is not enough (and vice versa);
  • different number of threads used in PME reported correctly only when -ntomp_pme is set, but not with GMX_PME_NUM_THREADS;
  • additionally the multi-threading checks in runner.c:check_and_update_hw_opt_1() are somewhat out of place and should be moved into the gmx_omp_nthreads module (which will help make the consistency checks more consistent too :).

Related issues

Related to GROMACS - Bug #1563: Wallcycle output is incorrect if GMX_*_NUM_THREADS is setRejected

History

#1 Updated by Mark Abraham about 5 years ago

  • Related to Bug #1563: Wallcycle output is incorrect if GMX_*_NUM_THREADS is set added

#2 Updated by Mark Abraham over 3 years ago

We've done a bunch of cleanup here, but I don't know if the issue itself is resolved. For our own sanity, we should remove a bunch of these alternatives, anyway.

#3 Updated by Erik Lindahl over 3 years ago

I guess it's the same old story where we provide umpteen ways to set something, and then fail to check all combinations :-)

Is there any specific critical reason why it has to be possible to adjust the number of threads with environment variables instead of the command line options? Removing the environment variables would both make the code simpler and avoid the challenges of handling cases where both are provided.

#4 Updated by Mark Abraham over 3 years ago

Erik Lindahl wrote:

I guess it's the same old story where we provide umpteen ways to set something, and then fail to check all combinations :-)

Is there any specific critical reason why it has to be possible to adjust the number of threads with environment variables instead of the command line options? Removing the environment variables would both make the code simpler and avoid the challenges of handling cases where both are provided.

Any such cases are probably to do with "catering" for heterogeneous hardware setups. The most likely relevant case is different GPU setups such that you want to set GMX_GPU_ID differently on different nodes at mpirun time. But if people don't want to put time into building machinery that shows such things work in the relevant cases (e.g. with test mocks like Teemu has just written for the affinity-setting behaviours), then we should not offer the features, because they come with bugs+uncertainty for everyone.

We should be honouring external things like OMP_NUM_THREADS, but if someone's MPI library sets that, then requiring them to set -ntomp consistently, in order to get access to -ntomp_pme is perfectly reasonable.

#5 Updated by Szilárd Páll about 3 years ago

I'll test if these conditions are still true for 5.1/2016 and report back. I'll comment on the rest of the points pokes later.

Also available in: Atom PDF