Project

General

Profile

Bug #2541

BUILD_OWN_FFTW fails regression tests

Added by Erik Lindahl 9 months ago. Updated 9 months ago.

Status:
Closed
Priority:
High
Assignee:
-
Category:
build system
Target version:
Affected version - extra info:
Affected version:
Difficulty:
uncategorized
Close

Description

Ubuntu-18.04, default compilers, and default builds.

If fftw-3.3.5 is downloaded manually and compiled, all regression tests pass.

With -DBUILD_OWN_FFTW=ON, the fft/correlationfunctions tests fail.

Associated revisions

Revision c17474b3 (diff)
Added by Erik Lindahl 9 months ago

Bump GMX_BUILD_OWN_FFTW to 3.3.8 to avoid bugs

FFTW-3.3.8 seems to fix build errors with AVX-512, and
as reported on the FFTW web page it also removes the -ffast-math
flag which results in errors with gcc-8.

Fixes #2541.

Change-Id: Ibe4ef0040986e4b83f92f0bb404c72fa1b5e11ea

History

#1 Updated by Szilárd Páll 9 months ago

The FFT unit tests fail too. The funny thing is that it does not fail on a Threadripper machine, but it does fail on a Xeon Silver CPU. Even more fun is that running with relative path (i.e. bin/fft-test) passes, but not with absolute path as invoked by ctest:

#0  0x00007ffff7787c82 in t1fv_3 () from /tmp/gromacs-18/build/bin/../lib/libgromacs.so.3
#1  0x00007ffff77ba29b in apply () from /tmp/gromacs-18/build/bin/../lib/libgromacs.so.3
#2  0x00007ffff74bbdb0 in apply () from /tmp/gromacs-18/build/bin/../lib/libgromacs.so.3
#3  0x00007ffff77b5b07 in apply_dit () from /tmp/gromacs-18/build/bin/../lib/libgromacs.so.3
#4  0x00007ffff74bbdb0 in apply () from /tmp/gromacs-18/build/bin/../lib/libgromacs.so.3
#5  0x00007ffff73eb954 in gmx_fft_1d (fft=0x55555587bac0, dir=GMX_FFT_FORWARD, in_data=0x555555861ae0, out_data=0x555555862270)
    at /tmp/gromacs-18/src/gromacs/fft/fft_fftw3.cpp:457
#6  0x00007ffff73eb989 in gmx_fft_many_1d (fft=0x55555587bac0, dir=GMX_FFT_FORWARD, in_data=0x555555861ae0, out_data=0x555555862270)
    at /tmp/gromacs-18/src/gromacs/fft/fft_fftw3.cpp:470
#7  0x0000555555577e86 in (anonymous namespace)::ManyFFTTest_Complex1DLength48Multi5Test_Test::TestBody (this=0x555555841020)
    at /tmp/gromacs-18/src/gromacs/fft/tests/fft.cpp:231
#8  0x00005555555d35ca in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (object=0x555555841020, 
    method=&virtual testing::Test::TestBody(), location=0x5555555e3df3 "the test body") at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/src/gtest.cc:2078
#9  0x00005555555cec27 in testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=0x555555841020, 
    method=&virtual testing::Test::TestBody(), location=0x5555555e3df3 "the test body") at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/src/gtest.cc:2114
#10 0x00005555555b74c8 in testing::Test::Run (this=0x555555841020) at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/src/gtest.cc:2150
#11 0x00005555555b7d18 in testing::TestInfo::Run (this=0x55555583d910) at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/src/gtest.cc:2326
#12 0x00005555555b838e in testing::TestCase::Run (this=0x55555583da90) at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/src/gtest.cc:2444
#13 0x00005555555bebd8 in testing::internal::UnitTestImpl::RunAllTests (this=0x55555583d460) at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/src/gtest.cc:4315
#14 0x00005555555d464b in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x55555583d460, 
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x5555555be918 <testing::internal::UnitTestImpl::RunAllTests()>, 
    location=0x5555555e4538 "auxiliary test code (environments or event listeners)") at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/src/gtest.cc:2078
#15 0x00005555555cfa69 in testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x55555583d460, 
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x5555555be918 <testing::internal::UnitTestImpl::RunAllTests()>, 
    location=0x5555555e4538 "auxiliary test code (environments or event listeners)") at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/src/gtest.cc:2114
#16 0x00005555555bd95c in testing::UnitTest::Run (this=0x5555558198c0 <testing::UnitTest::GetInstance()::instance>)
    at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/src/gtest.cc:3926
#17 0x000055555557fe98 in RUN_ALL_TESTS () at /tmp/gromacs-18/src/external/gmock-1.7.0/gtest/include/gtest/gtest.h:2290
#18 0x000055555557fe1c in main (argc=1, argv=0x7fffffffd428) at /tmp/gromacs-18/src/testutils/unittest_main.cpp:85

Both Complex1DLength48Multi5Test fw/bw seems to segv. Can't add much more.

#2 Updated by Szilárd Páll 9 months ago

Removing the --enable-avx512 flag also eliminates the issue: http://termbin.com/vbu5

I've had a quick look at the perf scan I did on FFTW 3.3.7 and while the data was awfully noisy, I did saw a few cases where the builds with sse+avx+avx2+avx512 did seems to be faster, but in most cases the sse+avx+avx2+avx512 runs faster.

As a side-note, the current code seems heavy-handed with adding the AVX512 flag: it uses it for compiling FFTW when any flavor of AVX is used. In the current setup, as we already use conditionality, and given that a single FFTW AVX512 kernel might throttle the CPU by another 10-15% (and result in performance loss in other parts of the code build explicitly with AVX2), we should change the condition on line 70 to:

elseif(${GMX_SIMD_ACTIVE} MATCHES "^(AVX512)")

#3 Updated by Erik Lindahl 9 months ago

I just had a look at the FFTW page, and apparently FFTW-3.3.7 in combination with GCC-8 results in broken builds due to -ffast-math, so it seems prudent that we also upgrade to use FFTW-3.3.8 before GCC-8 becomes widespread.

#4 Updated by Erik Lindahl 9 months ago

FFTW-3.3.8 seems to fix this problem too, so I'll push a change where we simply bump the version for now.

#5 Updated by Gerrit Code Review Bot 9 months ago

Gerrit received a related patchset '1' for Issue #2541.
Uploader: Erik Lindahl ()
Change-Id: gromacs~release-2018~Ibe4ef0040986e4b83f92f0bb404c72fa1b5e11ea
Gerrit URL: https://gerrit.gromacs.org/7975

#6 Updated by Erik Lindahl 9 months ago

  • Status changed from Accepted to Resolved

#7 Updated by Mark Abraham 9 months ago

  • Category set to build system
  • Status changed from Resolved to Closed
  • Target version set to 2018.2

Also available in: Atom PDF