Project

General

Profile

Bug #2276

warnings from CUDA system headers with CUDA 9.0

Added by Szilárd Páll about 2 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
mdrun
Target version:
Affected version - extra info:
Affected version:
Difficulty:
uncategorized
Close

Description

Tons of the following warnings are emitted with the final release of CUDA 9.0 and gcc 6.4 as host compiler:

/opt/tcbsys/cuda/9.0/include/crt/math_functions.hpp:160:5: warning: "_GLIBCXX_HAVE_OBSOLETE_ISNAN" is not defined [-Wundef]
 #if _GLIBCXX_HAVE_OBSOLETE_ISNAN && !_GLIBCXX_NO_OBSOLETE_ISINF_ISNAN_DYNAMIC
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /opt/tcbsys/cuda/9.0/include/crt/common_functions.h:258:0,
                 from /opt/tcbsys/cuda/9.0/include/common_functions.h:50,
                 from /opt/tcbsys/cuda/9.0/include/cuda_runtime.h:115,
                 from <command-line>:0:
/opt/tcbsys/cuda/9.0/include/crt/math_functions.h:8901:5: warning: "_GLIBCXX_HAVE_OBSOLETE_ISNAN" is not defined [-Wundef]
 #if _GLIBCXX_HAVE_OBSOLETE_ISNAN && !_GLIBCXX_NO_OBSOLETE_ISINF_ISNAN_DYNAMIC
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/tcbsys/cuda/9.0/include/crt/math_functions.h:8909:5: warning: "_GLIBCXX_HAVE_OBSOLETE_ISINF" is not defined [-Wundef]
 #if _GLIBCXX_HAVE_OBSOLETE_ISINF && !_GLIBCXX_NO_OBSOLETE_ISINF_ISNAN_DYNAMIC
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /opt/tcbsys/cuda/9.0/include/crt/math_functions.h:10198:0,
                 from /opt/tcbsys/cuda/9.0/include/crt/common_functions.h:258,
                 from /opt/tcbsys/cuda/9.0/include/common_functions.h:50,
                 from /opt/tcbsys/cuda/9.0/include/cuda_runtime.h:115,
                 from <command-line>:0:


Related issues

Related to GROMACS - Bug #2285: gcc6 + old glibc isinf/isnan build failureClosed

Associated revisions

Revision 4e5f675c (diff)
Added by Mark Abraham almost 2 years ago

Import cmake Modules/FindCUDA.cmake

CUDA 9.0 issues large numbers of -Wundef warnings from its internal
headers. FindCUDA.cmake should be including such headers as "system"
headers, so to prepare for a patch where it is modified to do that,
this commit imports that file from v3.4.3 of the CMake repository,
because that is a choice likely to work with all future versions of
CMake.

It needs some supporting cmake files that are included unmodified,
so GROMACS does not assert copyright on those. The main FindCUDA.cmake
file is modified only to be able to find those files

Refs #2276

Change-Id: I69ad39dc805648a6cc5e27bb7fcd229f5f2a538a

Revision 97e9dd85 (diff)
Added by Mark Abraham almost 2 years ago

Fix FindCUDA.cmake for -Wundef

CUDA 9.0 issues large numbers of -Wundef warnings from its internal
headers. FindCUDA.cmake should be including such headers as
"system" headers, so it is modified to do that.

Fixes #2276

Change-Id: I36ca1208a0597215bf2f1e38d849786e9ac2fed4

Revision 382b492a (diff)
Added by Mark Abraham almost 2 years ago

Remove FindCUDA from source tarball

When we decided to bundle FindCUDA.cmake and supporting files to keep
our developer and Jenkins builds free of warnings, we did not take
care to ensure that the user gets the behaviour from their cmake,
since the combination of their cmake and its FindCUDA.cmake is much
more likely to work correctly than their cmake and FindCUDA.cmake from
cmake 3.4.3.

The testing of the release tarball as part of the release workflow
will ensure that we don't silently ship a broken CUDA build.

Refs #2276

Change-Id: I9ed6fa97921eb5c3ce3b4a6ecc401b4dd26e1251

History

#1 Updated by Mark Abraham almost 2 years ago

nvcc ought to be including its own headers with -isystem so that warnings enabled by the user are not issued from those headers. If it should be doing that already, then I presume FindCUDA must be mangling that, somehow.

We want -Wundef generally, because we want to use #if rather than #ifdef, because that is more maintainable.

#2 Updated by Aleksei Iupinov almost 2 years ago

I looked at the FindCUDA in cmake 3.4.3 and cmake 3.9.4.
Both just call standard include_directories(), and document it uses "-I".
Here is a 2 years old bug-report: https://cmake.org/Bug/view.php?id=15458

#3 Updated by Aleksei Iupinov almost 2 years ago

What a pain, I hoped for a quick fix.

Everything is wrapped in cuda_add_library call - it adds its list of includes (with CUDA_TOOLKIT_DIR) using "-I" to the directory include flags, and then propagates same to NVCC flags as well.

So with just amending INCLUDE_DIRECTORIES of libgromacs afterwards I don't get the desired effect - generated NVCC flags have to be hacked to have -isystem as well.

Maybe we should just copy and improve cuda_add_library().

#4 Updated by Mark Abraham almost 2 years ago

  • Subject changed from warnings with CUDA 9.0 + gcc 6.4 to warnings from CUDA system headers with CUDA 9.0

The issue arises also with gcc 4.9 and 5.

I found that it is straightforward to hack the cmake FindCUDA.cmake so that in effect it uses -isystem. I'll upload a patch for that. If we find it stable, then we should probably send it upstream to cmake.

I think our main alternative is to disable warnings when compiling CUDA source files, and that is obviously unpalatable.

#5 Updated by Gerrit Code Review Bot almost 2 years ago

Gerrit received a related patchset '2' for Issue #2276.
Uploader: Mark Abraham ()
Change-Id: gromacs~master~I36ca1208a0597215bf2f1e38d849786e9ac2fed4
Gerrit URL: https://gerrit.gromacs.org/7075

#6 Updated by Szilárd Páll almost 2 years ago

Mark Abraham wrote:

The issue arises also with gcc 4.9 and 5.

Indeed.

I think our main alternative is to disable warnings when compiling CUDA source files, and that is obviously unpalatable.

We could consider the native CUDA support that a CMake dev proposed a while ago, it may allow avoiding the issue too. Additionally, if we're importing FindCUDA into the source tree again, we could also consider completely switching to the new native CUDA support a lot sooner.

#7 Updated by Gerrit Code Review Bot almost 2 years ago

Gerrit received a related patchset '1' for Issue #2276.
Uploader: Mark Abraham ()
Change-Id: gromacs~master~I69ad39dc805648a6cc5e27bb7fcd229f5f2a538a
Gerrit URL: https://gerrit.gromacs.org/7124

#8 Updated by Aleksei Iupinov almost 2 years ago

  • Related to Bug #2285: gcc6 + old glibc isinf/isnan build failure added

#9 Updated by Mark Abraham almost 2 years ago

  • Target version set to 2018

Szilárd Páll wrote:

Mark Abraham wrote:

The issue arises also with gcc 4.9 and 5.

Indeed.

I think our main alternative is to disable warnings when compiling CUDA source files, and that is obviously unpalatable.

We could consider the native CUDA support that a CMake dev proposed a while ago, it may allow avoiding the issue too. Additionally, if we're importing FindCUDA into the source tree again, we could also consider completely switching to the new native CUDA support a lot sooner.

Indeed, as noted in my patches that's available from CMake 3.8 (or 3.9 for MSVC). That's a bit unfriendly for users, so perhaps we should consider that only for the 2018 release.

Currently targetting the 2017 release, but we could also do something for the 2016 release branch - perhaps there -Wno-undef is appropriate, since the risk of us changing something in the remaining maintenance lifetime of the 2016 branch is low, and the warnings are only relevant for being a safety net for developers.

Reminder: users doing normal release builds don't see the warnings we are discussing. So we could also leverage that by only using the imported FindCUDA.cmake for a Jenkins and/or GMX_COMPILER_WARNINGS=on?

#10 Updated by Szilárd Páll almost 2 years ago

Mark Abraham wrote:

Szilárd Páll wrote:

Mark Abraham wrote:

The issue arises also with gcc 4.9 and 5.

Indeed.

I think our main alternative is to disable warnings when compiling CUDA source files, and that is obviously unpalatable.

We could consider the native CUDA support that a CMake dev proposed a while ago, it may allow avoiding the issue too. Additionally, if we're importing FindCUDA into the source tree again, we could also consider completely switching to the new native CUDA support a lot sooner.

Indeed, as noted in my patches that's available from CMake 3.8 (or 3.9 for MSVC). That's a bit unfriendly for users, so perhaps we should consider that only for the 2018 release.

I do realize now that it required changes in cmake not only in the FindCUDA modules. That makes it less useful as we'd need both the legacy fallback (current) and native mode implemented (as demonstrated by the cmake dev back in Jan), but otherwise I'm not sure why would it be unfriendly?

Currently targetting the 2017 release, but we could also do something for the 2016 release branch - perhaps there -Wno-undef is appropriate, since the risk of us changing something in the remaining maintenance lifetime of the 2016 branch is low, and the warnings are only relevant for being a safety net for developers.

Good point. Depends on how much we want to strive to keep builds warning-free. Passing '-Wno-undef' to all .cu file compilations with CUDA 9.0 seems like a reasonable compromise.

Reminder: users doing normal release builds don't see the warnings we are discussing. So we could also leverage that by only using the imported FindCUDA.cmake for a Jenkins and/or GMX_COMPILER_WARNINGS=on?

Do they not? Can you remind me why; admittedly I don't know what gets changes in the release tarball?

#11 Updated by Szilárd Páll almost 2 years ago

PS: as noted on gerrit, I think picking a FindCUDA that's as new as possible/reasonable might the better option. We get a few features/fixes (which, admittedly, we don't need now and may or may not care about later), but more importantly, I think we can be more certain that e.g. FindCUDA v3.8 will remain compatible and trouble-free with future CMake releases than forward compatibility of FindCUDA v3.4.3.

A rough summary of changes from having diffed the FindCUDA source files:
3.4.3 ~> 3.6.3
- FP16
- aarch64 cross compilation
- some low-lever verbosity control stuff

3.6 ~> 3.7
- CUDA_SELECT_NVCC_ARCH_FLAGS
-
- more ARM cross-compile support
- separable compilation changes

3.8 ~> 3.9
- CUDA_LINK_LIBRARIES_KEYWORD: support for PRIVATE|PUBLIC|INTERFACE

This is what the git log of the cmake tree shows:
$ git log v3.4.3..v3.9.4 --oneline Modules/FindCUDA.cmake Modules/FindCUDA/*

9e24af0 Use string(APPEND) in Modules
9f41bfd FindCUDA: Add option to use modern form of target_link_libraries
934eb49 FindCUDA: Fix PTX selection with multiple architectures
645dd3c FindCUDA: Fix typo in CUDA_DETECT_INSTALLED_GPUS doc string
c1f3086 FindCUDA: Enable CUDA_SELECT_NVCC_ARCH_FLAGS Auto for MSVC
6a9b30a Merge topic 'cuda_use_cudadevrt_with_separable_compilation'
891e0eb FindCUDA: find cudadevrt and link it if CUDA_SEPARABLE_COMPILATION is ON
900ee0b FindCUDA: Allow cuda_compile* macros to be called more than once per directory
6442709 FindCUDA: Fix for broken cuda_compile* commands.
2dc9a75 FindCUDA: Support `2.1(2.0)` architecture notation
7ded655 FindCUDA: Take NVCC include directories from target properties
1aa5c1b FindCUDA: Fix missing librt on Linux when using static cuda runtime.
aab1f6e FindCUDA: Restore default behavior of CUDA_USE_STATIC_CUDA_RUNTIME
8ba870b Merge topic 'FindCUDA-no-windows-librt'
15a6ae5 FindCUDA: Add search path for nvcc on Arch
f4e979b FindCUDA: Do not look for librt on Windows
5d0d980 Use string(APPEND) in Modules
5660861 Fix typos.
90d114e FindCUDA: Use the correct runtime in REQUIRED_VARS check
7229ae7 FindCUDA: Refactor Android(Tegra) support
e9bfe8d FindCUDA: Adjust vertical whitespace
81e73b7 FindCUDA: Add support for finding the cublas_device library
6b190b5 FindCUDA: Add support for generator expressions in CUDA_NVCC_FLAGS
cc95001 FindCUDA: Detect and report FP16 support
49e82c1 Fix spelling typos in comments and documentation (#16037)
f29fd5b Merge topic 'FindCUDA-verbatim'
72a97b7 FindCUDA: Fix regression in separate compilation (#16027)
1b369aa Merge topic 'FindCUDA-verbatim'
1911cda FindCUDA: Fix regression under Visual Studio generators
e739ef7 FindCUDA: Only warn about non-existent dependency files in verbose mode
81ecc72 FindCUDA: Added some additional comments about non-existent dependency files.
4b561b4 FindCUDA: Fix for when a non-existent dependency file is found.
6ccc307 FindCUDA: Support special characters in path (#15919)
ae1003c FindCUDA: Add paths for crosscompiling to aarch64
b1efb02 FindCUDA: drop CUDA_dl_LIBRARY

#12 Updated by Mark Abraham almost 2 years ago

Szilárd Páll wrote:

Mark Abraham wrote:

Indeed, as noted in my patches that's available from CMake 3.8 (or 3.9 for MSVC). That's a bit unfriendly for users, so perhaps we should consider that only for the 2018 release.

I do realize now that it required changes in cmake not only in the FindCUDA modules. That makes it less useful as we'd need both the legacy fallback (current) and native mode implemented (as demonstrated by the cmake dev back in Jan), but otherwise I'm not sure why would it be unfriendly?

Because users would not be able to rely on standard cmake in reasonably recent distros. That's why we chose 3.4.3 for the minimum cmake required 2017 release.

Currently targetting the 2017 release, but we could also do something for the 2016 release branch - perhaps there -Wno-undef is appropriate, since the risk of us changing something in the remaining maintenance lifetime of the 2016 branch is low, and the warnings are only relevant for being a safety net for developers.

Good point. Depends on how much we want to strive to keep builds warning-free. Passing '-Wno-undef' to all .cu file compilations with CUDA 9.0 seems like a reasonable compromise.

OK let's do that.

Reminder: users doing normal release builds don't see the warnings we are discussing. So we could also leverage that by only using the imported FindCUDA.cmake for a Jenkins and/or GMX_COMPILER_WARNINGS=on?

Do they not? Can you remind me why; admittedly I don't know what gets changes in the release tarball?

See GMX_COMPILER_WARNINGS in the main CMakeLists.txt, which reacts to the checks in cmake/gmxBuildTreeInfo.cmake. There's some simple checks for whether we have a git repo or a tarball we built in the intended manner for releases.

So we could just avoid packaging the FindCUDA.cmake stuff in the tarballs, so that the user gets the behaviour they get from their cmake, and developers+Jenkins get whatever behaviour we choose. Users building from git might possibly have a confusing time, but not likely and not likely to be a serious problem.

#13 Updated by Mark Abraham almost 2 years ago

Szilárd Páll wrote:

PS: as noted on gerrit, I think picking a FindCUDA that's as new as possible/reasonable might the better option. We get a few features/fixes (which, admittedly, we don't need now and may or may not care about later), but more importantly, I think we can be more certain that e.g. FindCUDA v3.8 will remain compatible and trouble-free with future CMake releases than forward compatibility of FindCUDA v3.4.3.

I don't see why there's any question of compatibility of any version of FindCUDA.cmake moving forward. FindCUDA.cmake is only useful in the case where CUDA is not a first-class language - new development will happen there, and does not affect our decision. That is, I presume FindCUDA.cmake is effectively legacy support in CMake 3.8. It's necessarily built on standard CMake stuff (because it's been around for a while) and that has clearly been pretty stable over the 3.4+ period. So both FindCUDA.cmake from 3.4.3 and 3.8 are likely to keep working forever, or to the same point of a breaking change in CMake.

Side point - my comment about FindCUDA in the pre-submit-matrix.txt is probably based on mis-remembering what changed in cmake 3.8 - the new language support doesn't affect our builds, except if they'd broken things, which it seems they have not.

The one from 3.8 is reasonably likely to work with all early versions too (because built on standard stuff), but there's some risk that its constructs rely on fixes that happened since 3.4.3 and this could give us strange behaviour.

I didn't see anything in the changelist you compiled (thanks) that woudl make our lives better to standardise for developers, given that we anyway still have to support working with the FindCUDA.cmake in 3.4.3.

Note that the release matrix builds from the tarball, and we should definitely require some 3.4.3 builds there.

#14 Updated by Szilárd Páll almost 2 years ago

Mark Abraham wrote:

Szilárd Páll wrote:

Mark Abraham wrote:

Indeed, as noted in my patches that's available from CMake 3.8 (or 3.9 for MSVC). That's a bit unfriendly for users, so perhaps we should consider that only for the 2018 release.

I do realize now that it required changes in cmake not only in the FindCUDA modules. That makes it less useful as we'd need both the legacy fallback (current) and native mode implemented (as demonstrated by the cmake dev back in Jan), but otherwise I'm not sure why would it be unfriendly?

Because users would not be able to rely on standard cmake in reasonably recent distros. That's why we chose 3.4.3 for the minimum cmake required 2017 release.

They would, what I suggested would not break backward-compatibility as with the FindCUDAwrapper as implemented by Rober Maynard, the legacy support would by definition remain untouched.

Again, this all is less relevant in the current context as I originally forgot that internal CMake support is needed for native CUDA support, but I did want to clarify things as there seems to be a misunderstanding in that there is no benefit to using the native CUDA support until we require CMake 3.8 (and/or we reimplement our CUDA support).

Currently targetting the 2017 release, but we could also do something for the 2016 release branch - perhaps there -Wno-undef is appropriate, since the risk of us changing something in the remaining maintenance lifetime of the 2016 branch is low, and the warnings are only relevant for being a safety net for developers.

Good point. Depends on how much we want to strive to keep builds warning-free. Passing '-Wno-undef' to all .cu file compilations with CUDA 9.0 seems like a reasonable compromise.

OK let's do that.

The alternatives are either the above or using the modified FindCUDA in all dev builds. If we can't have a new FindCUDA in the source tree, I prefer disabling -Wundef wothj CUDa 9.0 because locking all devs to an old FindCUDA in counter-productive.

Reminder: users doing normal release builds don't see the warnings we are discussing. So we could also leverage that by only using the imported FindCUDA.cmake for a Jenkins and/or GMX_COMPILER_WARNINGS=on?

Do they not? Can you remind me why; admittedly I don't know what gets changes in the release tarball?

See GMX_COMPILER_WARNINGS in the main CMakeLists.txt, which reacts to the checks in cmake/gmxBuildTreeInfo.cmake. There's some simple checks for whether we have a git repo or a tarball we built in the intended manner for releases.

Thanks, I did not know about that.

So we could just avoid packaging the FindCUDA.cmake stuff in the tarballs, so that the user gets the behaviour they get from their cmake, and developers+Jenkins get whatever behaviour we choose. Users building from git might possibly have a confusing time, but not likely and not likely to be a serious problem.

Sure, sounds good, but I have a moderate objection as explain above.

#15 Updated by Mark Abraham almost 2 years ago

Szilárd Páll wrote:

Mark Abraham wrote:

Szilárd Páll wrote:

I do realize now that it required changes in cmake not only in the FindCUDA modules. That makes it less useful as we'd need both the legacy fallback (current) and native mode implemented (as demonstrated by the cmake dev back in Jan), but otherwise I'm not sure why would it be unfriendly?

Because users would not be able to rely on standard cmake in reasonably recent distros. That's why we chose 3.4.3 for the minimum cmake required 2017 release.

They would, what I suggested would not break backward-compatibility as with the FindCUDAwrapper as implemented by Rober Maynard, the legacy support would by definition remain untouched.

OK I understand you now. Still we'd have to show that our CUDA support got better in some material way in order to want to add an additional kind of thing to support.

Again, this all is less relevant in the current context as I originally forgot that internal CMake support is needed for native CUDA support, but I did want to clarify things as there seems to be a misunderstanding in that there is no benefit to using the native CUDA support until we require CMake 3.8 (and/or we reimplement our CUDA support).

CUDA is a native language from CMake 3.8, but we're currently committed to supporting 3.4.3+, so any attempt to use the new stuff has to continue to support FindCUDA.cmake until we decided to bump the requirement.

The alternatives are either the above or using the modified FindCUDA in all dev builds. If we can't have a new FindCUDA in the source tree, I prefer disabling -Wundef wothj CUDa 9.0 because locking all devs to an old FindCUDA in counter-productive.

OK release-2016 will use -Wundef with CUDA 9.0

So we could just avoid packaging the FindCUDA.cmake stuff in the tarballs, so that the user gets the behaviour they get from their cmake, and developers+Jenkins get whatever behaviour we choose. Users building from git might possibly have a confusing time, but not likely and not likely to be a serious problem.

Sure, sounds good, but I have a moderate objection as explain above.

Even if our developer FindCUDA is more recent, we still have to support a tarball built from FindCUDA.cmake in 3.4.3. That's a risk we could accept, but we should know what concrete benefit we'd get for it.

#16 Updated by Szilárd Páll almost 2 years ago

Mark Abraham wrote:

Szilárd Páll wrote:

Mark Abraham wrote:

Szilárd Páll wrote:

I do realize now that it required changes in cmake not only in the FindCUDA modules. That makes it less useful as we'd need both the legacy fallback (current) and native mode implemented (as demonstrated by the cmake dev back in Jan), but otherwise I'm not sure why would it be unfriendly?

Because users would not be able to rely on standard cmake in reasonably recent distros. That's why we chose 3.4.3 for the minimum cmake required 2017 release.

They would, what I suggested would not break backward-compatibility as with the FindCUDAwrapper as implemented by Rober Maynard, the legacy support would by definition remain untouched.

OK I understand you now. Still we'd have to show that our CUDA support got better in some material way in order to want to add an additional kind of thing to support.

Well, if we were not at the "11th hour" my original argument why this is beneficial would still be relevant: given that this is new CUDA functionality, we would actually benefit from rolling it out for i) a part of the user-base by default (i.e. only those user would get the FindCUDAwrapper-based native CUDA support which have >=v3.8 cmake) ii) we could give feedback while having a workaround available for any issues (that is use <=v3.7 cmake if native CUDA support happens to not work smoothly).

Again, this all is less relevant in the current context as I originally forgot that internal CMake support is needed for native CUDA support, but I did want to clarify things as there seems to be a misunderstanding in that there is no benefit to using the native CUDA support until we require CMake 3.8 (and/or we reimplement our CUDA support).

CUDA is a native language from CMake 3.8, but we're currently committed to supporting 3.4.3+, so any attempt to use the new stuff has to continue to support FindCUDA.cmake until we decided to bump the requirement.

Indeed, but as you said FindCUDA is legacy so it will likely remain largely unchanged. I'm not even sure CUDA 9.0 feature will be implemented in it.

The alternatives are either the above or using the modified FindCUDA in all dev builds. If we can't have a new FindCUDA in the source tree, I prefer disabling -Wundef wothj CUDa 9.0 because locking all devs to an old FindCUDA in counter-productive.

OK release-2016 will use -Wundef with CUDA 9.0

Why only release-2016?
WIP or should I have a go at it?

So we could just avoid packaging the FindCUDA.cmake stuff in the tarballs, so that the user gets the behaviour they get from their cmake, and developers+Jenkins get whatever behaviour we choose. Users building from git might possibly have a confusing time, but not likely and not likely to be a serious problem.

Sure, sounds good, but I have a moderate objection as explain above.

Even if our developer FindCUDA is more recent, we still have to support a tarball built from FindCUDA.cmake in 3.4.3. That's a risk we could accept, but we should know what concrete benefit we'd get for it.

What more does "support" mean than having one or two configs in jenkins? FindCUDA is the legacy CUDA support in CMAke, right? If so, it will inherently not need more support than CI tests ensuring we don't brake it in r2017.

#17 Updated by Mark Abraham almost 2 years ago

  • Status changed from New to Resolved

#18 Updated by Mark Abraham almost 2 years ago

  • Status changed from Resolved to Accepted

There are still some warnings based on use of deprecated intrinsics, which I hope to look at shortly

#19 Updated by Aleksei Iupinov almost 2 years ago

Any news, Mark? I don't get any intrinsics warnings with gcc 6.4/CUDA 9.0 - I thought we fixed these long ago? (The sm_20 deprecation warning and the unused values in the intrinsics wrappers).

#20 Updated by Gerrit Code Review Bot almost 2 years ago

Gerrit received a related patchset '1' for Issue #2276.
Uploader: Mark Abraham ()
Change-Id: gromacs~release-2018~I9ed6fa97921eb5c3ce3b4a6ecc401b4dd26e1251
Gerrit URL: https://gerrit.gromacs.org/7368

#21 Updated by Erik Lindahl almost 2 years ago

  • Status changed from Accepted to Fix uploaded

#22 Updated by Erik Lindahl almost 2 years ago

  • Status changed from Fix uploaded to Resolved

#23 Updated by Erik Lindahl almost 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF