multiple issues with physical validation suite
The most useful documentation for the physical validation suite is only in the commit message. I suggest that information is copied somewhere under docs.
The commit message is incorrect in mentioning "check-prepare-phys" when it is called "check-phys-prepare".
If I understand the commit message correctly "make check-phys" doesn't do anything useful by itself without first manual running the simulation (no where mentioned how to do that). If the user does try to run without running the simulation first they should get at least a warning message. Also CMakeLists.txt only mentions the check-phys target which thus might be most likely be called by the user.
Why is there both the targets check-phys-prepare and prepare-phys-tests as wells as check-phys-run and run-phys-tests. What's the difference between each pair? And why are both needed? Please document.
Also the the dependencies don't seem to work correctly. Trying to run check-phys-run after running cmake gives:
ERROR: No gmx executable defined. Set before attempting to run!
It seems this target should depend on gmx.
Also check-phys-run fails with:
Traceback (most recent call last): File "/home/schulzro/gromacs/tests/physicalvalidation/gmx_physicalvalidation.py", line 11, in <module> from physical_validation import integrator, ensemble, kinetic_energy File "/nfs/home/schulzro/gromacs/tests/physicalvalidation/physical_validation/__init__.py", line 43, in <module> from . import kinetic_energy File "/nfs/home/schulzro/gromacs/tests/physicalvalidation/physical_validation/kinetic_energy.py", line 36, in <module> from .util import kinetic_energy as util_kin File "/nfs/home/schulzro/gromacs/tests/physicalvalidation/physical_validation/util/kinetic_energy.py", line 40, in <module> from . import plot File "/nfs/home/schulzro/gromacs/tests/physicalvalidation/physical_validation/util/plot.py", line 31, in <module> import matplotlib.pyplot as plt File "/usr/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 97, in <module> _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup() File "/usr/lib64/python2.7/site-packages/matplotlib/backends/__init__.py", line 25, in pylab_setup globals(),locals(),[backend_name]) File "/usr/lib64/python2.7/site-packages/matplotlib/backends/backend_gtkagg.py", line 10, in <module> from matplotlib.backends.backend_gtk import gtk, FigureManagerGTK, FigureCanvasGTK,\ File "/usr/lib64/python2.7/site-packages/matplotlib/backends/backend_gtk.py", line 13, in <module> import gtk; gdk = gtk.gdk File "/usr/lib64/python2.7/site-packages/gtk-2.0/gtk/__init__.py", line 64, in <module> _init() File "/usr/lib64/python2.7/site-packages/gtk-2.0/gtk/__init__.py", line 52, in _init _gtk.init_check()
without X. It doesn't actually seem to need X if so that should be fixed too.
Also check-prepare-phys fails with
Gtk-Message: Failed to load module "pk-gtk-module" Gtk-Message: Failed to load module "canberra-gtk-module"
on CentOS 7 without PackageKit-gtk3-module and libcanberra-gtk2. Again it seems it doesn't actually need Gtk.
After running check-prepare-phys it should print how to actually run the simulation now that it is prepared.
While running the simulations there should be some output for progress (e.g. a message after each simulation is finished).
Documentation and fixes for physical validation
Adds documentation for the physical validation suite in
As this was misunderstandable, changed the default behavior of
`make check-phys` and `make check-all` to actually run the simulations.
This might take very long, but since the physical validation tests need to
be turned on explicitly via cmake option, the chances of somebody using the
tests by mistake are low. The `check` targets are:
- `make check`: Run unit and regression tests (unchanged)
- `make check-phys`: Run simulations needed for physical validation, then
run physical validation tests
- `make check-phys-analyze`: Only run physical validation tests, assuming
that simulations were run previously and are available.
- `make check-all`: Combination of `make check` and `make check-phys`
Additionally, `make check-phys-prepare` can be used to prepare GROMACS
input files and a script to run the simulations needed for the physical
Bugfixed dependecy issue that attempted to run validation test before
Fix in physical_validation package to avoid including matplotlib by default
(and hence requiring X and Gtk).
Improved output of physical validation script (update while running
simulation, info about script after preparing input files, etc).
Additional minor changes not mentioned in #2349:
- Allowing name of test to be different of directory of input files, which
allows to define several tests on the same input files.
- Adapted some tests - more thorough checking of ensembles (check
temperature- and pressure-dependence separately and collectively), and
updated tolerance levels.
- Bumped physical_validation package to newest version to include bugfixes
and stability improvements.
#2 Updated by Pascal Merz over 2 years ago
Thank you for the report. I'm on it, should be able to address most points by tomorrow.
A question about the documentation: What would be a good place to put the documentation? An subpage of http://www.gromacs.org/Developer_Zone/Programming_Guide, as for the regression tests (http://www.gromacs.org/Developer_Zone/Programming_Guide/Regression_Tests)? A section in the developer guide as for the unit tests (http://manual.gromacs.org/documentation/2018-beta2/dev-manual/testutils.html)? Or another place?
#3 Updated by Roland Schulz over 2 years ago
Great. Thanks. I think all new docs should go into the source repo at /docs/. Where depends I think on who you want to run it. If you think it should only be run by developers it should go into the dev-manual folder. Either in its own file or in testutils.rst. If you think that some users (e.g. beta testers) should also run it, you might want to put a summary into install-guide/index.rst.