race condition with lincs + openmp + free-energy
When I converted regressiontests/freeenergy/expanded and transformAtoB to the Verlet scheme and switched them to use LINCS, I exposed a race condition in the way LINCS reduces dvdlambda at http://redmine.gromacs.org/projects/gromacs/repository/revisions/release-5-0/entry/src/gromacs/mdlib/clincs.c#L470, which was caught by the TSAN build on Jenkins - see http://jenkins.gromacs.org/job/Gromacs_Gerrit_master/7403/OPTIONS=Compiler=gcc%20CompilerVersion=4.9%20CMAKE_BUILD_TYPE=TSAN%20host=bs_nix1310,label=bs_nix1310/testReport/junit/(root)/freeenergy/expanded/.
I have reproduced this locally with a no-futex build of gcc-4.9 with
mdrun -ntmpi 1 -ntomp 2, but of course
mdrun -ntmpi 1 -ntomp 1 and
mdrun -ntmpi 2 -ntomp 1 are both fine.
I am not sure if release-4-6 is affected, because those test cases don't run with the Verlet scheme of that era. complex/nbnxn-free-energy does not use constraints, so we didn't catch it there, either. We should expedite converting regressiontests so we can run both cutoff schemes, even if we don't do so in Jenkins for every commit.
#pragma omp barrier with
#pragma omp critical fixes the issue. I will upload a fix.
Avoid race on dvdl with Verlet+OpenMP+LINCS+FE+VV
Also restructured the dH/dlambda reduction in do_lincs (used for
coordinates and not affected by the race issue) to work similar
do the do_lincsp code and properly use thread parallelization.