Project

General

Profile

Bug #2824

treatment of -ndec for .gro and pdb .gro files inconsistent with documentation and stdout

Added by Michael Shirts 8 months ago. Updated 7 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
analysis tools
Target version:
Affected version - extra info:
Affected version:
Difficulty:
simple
Close

Description

the flag -ndec is silently ignored by trjconv in the creation of .gro files. This is in contrast to behavior in at least 5.1.3, where -ndec worked, and the manual (http://manual.gromacs.org/documentation/2019/onlinehelp/gmx-trjconv.html), which states "The precision is always taken from -ndec, when this option is set." It also says "The precision of the .xtc output is taken from the input file for .xtc, .gro and .pdb, and from the -ndec option for other input formats" which is somewhat in conflict, but it's not true either.

Note that the stdout still SAYS the output precision is what is set by -ndec, i.e. it says: "Setting output precision to 1e-10 (nm)." And no warnings are issued that -ndec is ignored.

Looking at the gmxio.c logfile message written by Berk on Aug 23 2016, it says
"The gro precision is now fixed to 3, 4 and 5 decimal places for
x, v and box respectively to ensure compatibility with other software.
Variable precision reading is still supported."

More discussion was in issue #2037 - basically, it seems to be because VMD was coded to use a fixed format, so more precision broke it, and some of the original documentation said the format was fixed (though gromacs itself didn't), it was changed to fixed format.

Editorial: This broke a number of our scripts (silently, until I tried to figure out why crystal packings now had a bunch of noise in them). I do think it's problematic that g96 is the only ascii file version left with definable precision. This means g96 is the only human readable way what the coordinates are to machine precision (and nobody uses that). We run a number of analyses scripts where we just parsed high-precision ASCII files, which worked great. ASCII is limited precision, but it's always portable. 3 digits of precision is not sufficient to actually work with things (minimization doesn't actually give minimized output if the output is three digits of precision).

We'll have to see how easy it is to use 3rd party tools to extract trr into Python for other types of analysis that need high precision for the future (I guess we could use .g96, but that could change as well, presumably, and there aren't any tools that use it already), though that adds complexity. .tng is also an option, but doesn't seem to be many .tng reading libraries out there (mdtraj doesn't do it).

For now, I think we'll just using an old version of gromacs to do it, since the .trr is back-compatible. That's a trivial workflow change.

Would be interesting to know what other people now do to get analysis of coordinates into Python (or other non-gromacs code).

I guess I can't complain too much since I missed the 2016 conversation on this issue (probably was getting ready to teach general chem for the first time that semester).

But at least the documentation and the program output should be updated to state what it is going on (i.e. -ndec is ignored for .gro and .pdb).


Related issues

Related to GROMACS - Bug #2037: trjconv produces gro files unreadable by VMDClosed

Associated revisions

Revision 83192f1d (diff)
Added by Mark Abraham 8 months ago

Fix trjconv -ndec

This only works with XTC writing, but the documentation
and implementation was wrong. That mean that the terminal
feedback to the user was also wrong when writing a .gro file.

Fixes #2824
Refs #2037

Change-Id: I9f047d0b1042fa37e366ee8c99dabbb1f3458b0a

Revision cfb3ca1f (diff)
Added by Mark Abraham 7 months ago

Fix trjconv -ndec

This only works with XTC writing, but the documentation
and implementation was wrong. That mean that the terminal
feedback to the user was also wrong when writing a .gro file.

Fixes #2824
Refs #2037

Change-Id: I9f047d0b1042fa37e366ee8c99dabbb1f3458b0a

History

#1 Updated by Michael Shirts 8 months ago

  • Description updated (diff)

#2 Updated by Mark Abraham 8 months ago

  • Related to Bug #2037: trjconv produces gro files unreadable by VMD added

#3 Updated by Mark Abraham 8 months ago

  • Status changed from New to Accepted
  • Assignee set to Mark Abraham

Understood. I'll patch trjconv to fix docs and code to make clear that -ndec applies only to xtc output and clarify the terminal output accordingly. (It was still thinking that it could write variable precision to gro.)

The idea behind -ndec sounds nice, but there is no way the implementation of file handling in GROMACS could in principle cope with it in any kind of general way. Obviously one does not want to foist upon the whole ecosystem the need to read variable width columns in what was originally a fixed-width format. So, since one could get ASCII with g96, and full binary with .trr, we simplified life.

There are a few python implementations of .trr reading, but I haven't used any of them.

#4 Updated by Gerrit Code Review Bot 8 months ago

Gerrit received a related patchset '1' for Issue #2824.
Uploader: Mark Abraham ()
Change-Id: gromacs~release-2019~I9f047d0b1042fa37e366ee8c99dabbb1f3458b0a
Gerrit URL: https://gerrit.gromacs.org/8923

#5 Updated by Mark Abraham 8 months ago

  • Status changed from Accepted to Fix uploaded

#6 Updated by Michael Shirts 8 months ago

Mark Abraham wrote:

There are a few python implementations of .trr reading, but I haven't used any of them.

This is something that really needs to be addressed when looking at analysis APIs. It should be trivial to take Gromacs outputs and put them them into other analysis pipelines, which includes full precision coordinates (or, as full as are in the .trrs/.xtcs/.tngs).

We've used mdtraj a bit, but not clear if the development is continuing for now. And adds some overhead.

#7 Updated by Mark Abraham 8 months ago

Michael Shirts wrote:

Mark Abraham wrote:

There are a few python implementations of .trr reading, but I haven't used any of them.

This is something that really needs to be addressed when looking at analysis APIs. It should be trivial to take Gromacs outputs and put them them into other analysis pipelines, which includes full precision coordinates (or, as full as are in the .trrs/.xtcs/.tngs).

Sure (and that's orthogonal to the desire to write more precision to .gro or some ASCII format). But since e.g. mdanalysis, mdtraj and cpptraj all read trr (and there is even a pure-python trr reader at https://github.com/andersle/pytrr) the "put them into other analysis pipelines" bit is solved in pratice. One might argue that the problem here is that your pipeline did something like rolling your own file conversion and reading stages, instead of using what was out there ;-)

Our API design obviously won't involve writing files to disk so we can read them back in Python. We'll have the ability to read trr and all kinds of gro files nearly for free, so there's nothing I can see on the GROMACS end to handle.

We've used mdtraj a bit, but not clear if the development is continuing for now. And adds some overhead.

At some point I imagine we'll have the ability to wrap the canonical GROMACS trr reading in something one can call from python. Then the feature requests to support every different python numerical library memory layout will start!

And yeah, mdtraj was a bit like "oh the other similar projects aren't perfect and it's nicer to build one's own and not collaborate, so we'll build a new one and hope its sustainable." And now we have n+1 projects trying to be sustainable...

#8 Updated by Mark Abraham 8 months ago

  • Status changed from Fix uploaded to Resolved

#9 Updated by Paul Bauer 8 months ago

  • Status changed from Resolved to Closed

#10 Updated by Gerrit Code Review Bot 7 months ago

Gerrit received a related patchset '1' for Issue #2824.
Uploader: Paul Bauer ()
Change-Id: gromacs~release-2018~I9f047d0b1042fa37e366ee8c99dabbb1f3458b0a
Gerrit URL: https://gerrit.gromacs.org/9012

Also available in: Atom PDF