Project

General

Profile

Bug #1939

Ubuntu Xenial packages of 5.1.2 fail to build

Added by Szilárd Páll about 3 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
documentation
Target version:
Affected version - extra info:
Affected version:
Difficulty:
uncategorized
Close

Description

Looks like the manual has some doxygen issues:

UILDDIR»/build/documentation/docs/sphinx-input/user-guide/mdrun-performance.rst
make[4]: *** [docs/doxygen/doxygen-user-timestamp.txt] Error 1
docs/doxygen/CMakeFiles/doxygen-user.dir/build.make:62: recipe for target 'docs/doxygen/doxygen-user-timestamp.txt' failed
docs/doxygen/CMakeFiles/doxygen-user.dir/build.make:62: recipe for target 'docs/doxygen/doxygen-user-timestamp.txt' failed
make[4]: Leaving directory '/«PKGBUILDDIR»/build/documentation'
CMakeFiles/Makefile2:1735: recipe for target 'docs/doxygen/CMakeFiles/doxygen-user.dir/all' failed
make[3]: *** [docs/doxygen/CMakeFiles/doxygen-user.dir/all] Error 2
make[3]: *** Waiting for unfinished jobs....
cd /«PKGBUILDDIR»/build/documentation/docs && /usr/bin/cmake -E copy /«PKGBUILDDIR»/docs/user-guide/mdp-options.rst /«PKGBUILDDI
R»/build/documentation/docs/sphinx-input/user-guide/mdp-options.rst
cd /«PKGBUILDDIR»/build/documentation/docs && /usr/bin/cmake -E copy /«PKGBUILDDIR»/docs/user-guide/file-formats.rst /«PKGBUILDD
IR»/build/documentation/docs/sphinx-input/user-guide/file-formats.rst
cd /«PKGBUILDDIR»/build/documentation/docs && /usr/bin/cmake -E copy /«PKGBUILDDIR»/docs/user-guide/cmdline.rst /«PKGBUILDDIR»/b
uild/documentation/docs/sphinx-input/user-guide/cmdline.rst
cd /«PKGBUILDDIR»/build/documentation/docs && /usr/bin/cmake -E copy /«PKGBUILDDIR»/docs/user-guide/environment-variables.rst /«
PKGBUILDDIR»/build/documentation/docs/sphinx-input/user-guide/environment-variables.rst
cd /«PKGBUILDDIR»/build/documentation/docs && /usr/bin/cmake -E copy /«PKGBUILDDIR»/docs/user-guide/plotje.gif /«PKGBUILDDIR»/build/documentation/docs/sphinx-input/user-guide/plotje.gif
cd /«PKGBUILDDIR»/build/documentation/docs && /usr/bin/cmake -E copy /«PKGBUILDDIR»/docs/user-guide/xvgr.gif /«PKGBUILDDIR»/build/documentation/docs/sphinx-input/user-guide/xvgr.gif
cd /«PKGBUILDDIR»/build/documentation/docs && /usr/bin/cmake -E copy /«PKGBUILDDIR»/docs/conf.py /«PKGBUILDDIR»/build/documentation/docs/sphinx-input/conf.py
make[4]: Entering directory '/«PKGBUILDDIR»/build/documentation'
make[4]: Nothing to be done for 'src/gromacs/CMakeFiles/libgromacs.dir/build'.
make[4]: Leaving directory '/«PKGBUILDDIR»/build/documentation'
[ 92%] Built target libgromacs
cd /«PKGBUILDDIR»/build/documentation/docs && /usr/bin/cmake -E touch /«PKGBUILDDIR»/build/documentation/docs/sphinx-input-timestamp.txt
make[4]: Leaving directory '/«PKGBUILDDIR»/build/documentation'
[ 94%] Built target sphinx-input
The following warnings were produced by Doxygen:
/«PKGBUILDDIR»/src/gromacs/selection/tests/selectioncollection.cpp:100: warning: Conditional section does not have a corresponding \endcond command within this file.
/«PKGBUILDDIR»/src/gromacs/simd/simd.h:881: warning: @copydetails or @copydoc target 'gmx_simd_exponent_f' not found
/«PKGBUILDDIR»/src/gromacs/simd/simd.h:890: warning: @copydetails or @copydoc target 'gmx_simd_mantissa_f' not found
/«PKGBUILDDIR»/src/gromacs/simd/simd.h:949: warning: @copydetails or @copydoc target 'gmx_simd_or_fn' not found
/«PKGBUILDDIR»/src/gromacs/simd/simd.h:881: warning: @copydetails or @copydoc target 'gmx_simd_exponent_f' not found
/«PKGBUILDDIR»/src/gromacs/simd/simd.h:890: warning: @copydetails or @copydoc target 'gmx_simd_mantissa_f' not found
/«PKGBUILDDIR»/src/gromacs/simd/simd.h:949: warning: @copydetails or @copydoc target 'gmx_simd_or_fn' not found
cd /«PKGBUILDDIR»/build/documentation/docs/doxygen && /usr/bin/cmake -E touch /«PKGBUILDDIR»/build/documentation/docs/doxygen/doxygen-xml-timestamp.txt
make[4]: Leaving directory '/«PKGBUILDDIR»/build/documentation'
[ 94%] Built target doxygen-xml
make[3]: Leaving directory '/«PKGBUILDDIR»/build/documentation'
CMakeFiles/Makefile2:1313: recipe for target 'docs/CMakeFiles/webpage.dir/rule' failed
make[2]: *** [docs/CMakeFiles/webpage.dir/rule] Error 2
make[2]: Leaving directory '/«PKGBUILDDIR»/build/documentation'
Makefile:679: recipe for target 'webpage' failed

History

#1 Updated by Szilárd Páll about 3 years ago

Given that Xenial is soon out, I proposed to the maintainers to upgrade from 5.1.1 to 5.1.2, so it would be great if we could help fix this issue, so Ubuntu comes out with the latest 5.1. Possibly, if 5.1.3 is planned to be released soon, we could propose that too.

#2 Updated by Mark Abraham about 3 years ago

As noted in part of that output

"NOTE: You are using Doxygen version 1.8.11. The documentation is designed for 1.8.5. Other versions may or may not work, but very likely produce extra warnings."

If the Ubuntu build system can't provide Doxygen 1.8.5 then all bets are off from our end. Frankly, I'd recommend they not attempt to build our documentation at all - most of it probably works, but requires Sphinx and pdflatex. Regardless, cmake -DDOXYGEN_EXECUTABLE=off is surely a step in the right direction. The simplest path is to take our tarball, install it, and scrape the share subdirectory.

#3 Updated by Szilárd Páll about 3 years ago

Debian usually fixes our doxygen because they won't cater for our doxygen version wishes (e.g. see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=815678). It's not in the interest of any distro to pull in a specific version of a dependency that upstream likes.

I don't know how maintainers package GROMACS, but I believe there is value in having developer manual installed with the "-dev" packages. I could suggest the "workaround", but I think it's not really an appropriate one.

#4 Updated by Szilárd Páll about 3 years ago

BTW, those are warnings so they should not cause a build to fail and off-redmine communication with maitainers hints that too.

#5 Updated by Teemu Murtola about 3 years ago

The log has this (just before the section where you pasted the text into the description):

CMake Error at RunDoxygen.cmake:54 (message):
  Doxygen failed.  Please run '/usr/bin/doxygen Doxyfile-user' in the
  docs/doxygen/ subdirectory in the build tree to see the details.

#6 Updated by Szilárd Páll about 3 years ago

Teemu Murtola wrote:

The log has this (just before the section where you pasted the text into the description):

[...]

You're right. Current theory is that the hosts may not have enough memory/core for parallel build jobs which caused the failed build. I wonder, though how reasonable is that assumption given that this was the doxygen phase. Does that even parallelize?

#7 Updated by Mark Abraham about 3 years ago

Szilárd Páll wrote:

I don't know how maintainers package GROMACS, but I believe there is value in having developer manual installed with the "-dev" packages. I could suggest the "workaround", but I think it's not really an appropriate one.

We support a build with Doxygen 1.8.5. We provide a way for them to get a useful chunk of pre-built documentation. We can even provide them with the Jenkins-built tarball (containing the developer manual) that we unpack to go on our website, if they really want it. IMO "-dev" packages should have headers and libraries for third-party use. They don't have to be perfect. I get that distro maintainers have constraints, but so do Doxygen and GROMACS developers, and so they should be willing to consider compromise.

#8 Updated by Szilárd Páll about 3 years ago

Mark Abraham wrote:

Szilárd Páll wrote:

I don't know how maintainers package GROMACS, but I believe there is value in having developer manual installed with the "-dev" packages. I could suggest the "workaround", but I think it's not really an appropriate one.

We support a build with Doxygen 1.8.5. We provide a way for them to get a useful chunk of pre-built documentation. We can even provide them with the Jenkins-built tarball (containing the developer manual) that we unpack to go on our website, if they really want it. IMO "-dev" packages should have headers and libraries for third-party use. They don't have to be perfect. I get that distro maintainers have constraints, but so do Doxygen and GROMACS developers, and so they should be willing to consider compromise.

What compromise have they not considered? Doxygen warnings are not fatal, and if Doxygen would crash that's a real issue - or it can become one sooner or later for us too. In any case, doxygen warning do not make doxygen fail, as mentioned above the doxygen error could have been a transient error because Debian buldd can build 5.1.2 with the exact same build rules:
https://buildd.debian.org/status/package.php?p=gromacs

Off-topic: notice the number of arch they do build successfully? Including Motorola 68000 and System Z? :)

#9 Updated by Teemu Murtola about 3 years ago

Szilárd Páll wrote:

Teemu Murtola wrote:

The log has this (just before the section where you pasted the text into the description):

[...]

You're right. Current theory is that the hosts may not have enough memory/core for parallel build jobs which caused the failed build. I wonder, though how reasonable is that assumption given that this was the doxygen phase. Does that even parallelize?

I don't know whether doxygen does some internal parallelization. But make can run some other build rules in parallel with the doxygen rule (in some cases, even other doxygen runs).

#10 Updated by Mark Abraham about 3 years ago

Szilárd Páll wrote:

Mark Abraham wrote:

Szilárd Páll wrote:

I don't know how maintainers package GROMACS, but I believe there is value in having developer manual installed with the "-dev" packages. I could suggest the "workaround", but I think it's not really an appropriate one.

We support a build with Doxygen 1.8.5. We provide a way for them to get a useful chunk of pre-built documentation. We can even provide them with the Jenkins-built tarball (containing the developer manual) that we unpack to go on our website, if they really want it. IMO "-dev" packages should have headers and libraries for third-party use. They don't have to be perfect. I get that distro maintainers have constraints, but so do Doxygen and GROMACS developers, and so they should be willing to consider compromise.

What compromise have they not considered?

Perhaps the one you think is "inappropriate" to suggest? :-)

Doxygen warnings are not fatal, and if Doxygen would crash that's a real issue - or it can become one sooner or later for us too. In any case, doxygen warning do not make doxygen fail, as mentioned above the doxygen error could have been a transient error because Debian buldd can build 5.1.2 with the exact same build rules:
https://buildd.debian.org/status/package.php?p=gromacs

A crash in a version of Doxygen that we do not support isn't a real issue for us. Particularly if they have an alternative to actually running Doxygen on our code.

Off-topic: notice the number of arch they do build successfully? Including Motorola 68000 and System Z? :)

#11 Updated by Szilárd Páll about 3 years ago

Mark Abraham wrote:

Szilárd Páll wrote:

Mark Abraham wrote:

Szilárd Páll wrote:

I don't know how maintainers package GROMACS, but I believe there is value in having developer manual installed with the "-dev" packages. I could suggest the "workaround", but I think it's not really an appropriate one.

We support a build with Doxygen 1.8.5. We provide a way for them to get a useful chunk of pre-built documentation. We can even provide them with the Jenkins-built tarball (containing the developer manual) that we unpack to go on our website, if they really want it. IMO "-dev" packages should have headers and libraries for third-party use. They don't have to be perfect. I get that distro maintainers have constraints, but so do Doxygen and GROMACS developers, and so they should be willing to consider compromise.

What compromise have they not considered?

Perhaps the one you think is "inappropriate" to suggest? :-)

There's a workflow that package building/testing has to follow, they can't just use doxygen 1.8.5 (at least not with so little time until the release in 2-3 weeks) and they can't just take a tarball from our jenkins when they take the source tarballs from upstream (Debian) and apply patches as needed before rolling it out on launchpad.

So solutions that do not fit in their scheme are reasonably called "inappropriate" - and that's the meaning of the word I indented. Or I could have called it "lacking consideration of the needs of the Ubuntu project".

Whether dropping the dev docs from the deb is appropriate, I don't know, but it sounds like a lazy solution to a problem that should be possible to address - at least that's what I assumed when filing this issue. And dropping the manual just seems counter-productive, it's clearly beneficial to have the reference manual together with the program. It's portable, self-contained, and available offline.

Doxygen warnings are not fatal, and if Doxygen would crash that's a real issue - or it can become one sooner or later for us too. In any case, doxygen warning do not make doxygen fail, as mentioned above the doxygen error could have been a transient error because Debian buldd can build 5.1.2 with the exact same build rules:
https://buildd.debian.org/status/package.php?p=gromacs

A crash in a version of Doxygen that we do not support isn't a real issue for us. Particularly if they have an alternative to actually running Doxygen on our code.

Note my wording: "that's a real issue - or it can become one".

In addition, the resistance to just helping roll out the latest GROMACS release in the most popular Linux distro seems strange and counter-productive. Even if it may require considering some tweaks with a more recent doxygen (which has not even been determined thus far), i) a significant fraction of our user base apt-get installs from Ubuntu (/Debian/Mint), so having up-to-date GROMACS in these distros is IMO important ii) we may want to adopt the respective Doxygen version sometime in the future anyway.

#12 Updated by Szilárd Páll about 3 years ago

  • Status changed from New to Rejected

In any case, considering the response it's likely best if I to take the issue off-redmine and will work with them and will try to help track down and iron out issues if I can.

#13 Updated by Szilárd Páll about 3 years ago

  • Status changed from Rejected to Closed

#14 Updated by Mark Abraham about 3 years ago

Doxygen is neither forward nor backward compatible. My experience is that it isn't feasible to support multiple versions, and the present evidence confirms that its non-zero work to update to a newer version.

I already said I know that distro packagers have constraints, but so do we and there's no common ground at this time for implementing the best possible solution for the tiny fraction of GROMACS+distro users who might want the -dev package to develop off it, and then want to use the installed offline docs. They don't have to make a -dev package available, and it doesn't have to be perfect.

Helping support a distro package for GROMACS 5.1.2 is fine - the resistance is about not trying to solve a problem whose solution has low impact even if one does actually exist.

#15 Updated by Szilárd Páll about 3 years ago

Mark Abraham wrote:

I already said I know that distro packagers have constraints, but so do we and there's no common ground at this time for implementing the best possible solution for the tiny fraction of GROMACS+distro users who might want the -dev package to develop off it, and then want to use the installed offline docs. They don't have to make a -dev package available, and it doesn't have to be perfect.

That's not up to me (and frankly IMO not up to us) to judge.

Helping support a distro package for GROMACS 5.1.2 is fine - the resistance is about not trying to solve a problem whose solution has low impact even if one does actually exist.

The resistance that surprised me was present even it was determined what the nature of the issue is and whether it requires any effort from the GROMACS side. I opened this issue because I know little about doxygen an I thought I could get quick feedback. I got that and more.

#16 Updated by Mark Abraham almost 2 years ago

Szilárd Páll wrote:

Mark Abraham wrote:

I already said I know that distro packagers have constraints, but so do we and there's no common ground at this time for implementing the best possible solution for the tiny fraction of GROMACS+distro users who might want the -dev package to develop off it, and then want to use the installed offline docs. They don't have to make a -dev package available, and it doesn't have to be perfect.

That's not up to me (and frankly IMO not up to us) to judge.

We judge that we're supporting a particular Doxygen version, and that it's not feasible to do anything else unless someone wants to do the work. If they then judge that because their build system can't provide a matching Doxygen version (which is a question distinct from the Doxygen version supported in the distro itself), this means they can't provide a -dev package that includes the developer manual, then that's life.

Helping support a distro package for GROMACS 5.1.2 is fine - the resistance is about not trying to solve a problem whose solution has low impact even if one does actually exist.

The resistance that surprised me was present even it was determined what the nature of the issue is and whether it requires any effort from the GROMACS side. I opened this issue because I know little about doxygen an I thought I could get quick feedback. I got that and more.

You opened with "looks like it has issues" after apparently not having read the warning messages in the log. That's just as frustrating to the people who worked on this as it is to you when someone on gmx-users is (eg) doing node sharing and has ignored log file warnings about thread affinity not being set. Then, when you learned what the constraints were, you were not prepared to compromise on the vision of a smoothly supported complete -dev package, and inferred that there was resistance to supporting a smooth life building the normal package also. That's frustrating for your teammates - be prepared to compromise, particularly if you can't/won't do it yourself!

Also available in: Atom PDF