Feature #2585: Infrastructure supporting external API
gmxapi integration testing
The new public API is intended to provide a stable interface for client code built against installed copies of GROMACS (without access to the build tree). Revisions to the API specification are asynchronous with GROMACS releases (patch and minor releases coming more frequently, especially in near-term development, and hopefully much slower larger updates at maturity). This issue discusses integration testing requirements and solutions.
- Gmxapi interfaces should continue functioning with unchanged semantics for other GROMACS changes, or API level needs to be incremented according to semantic versioning.
- External projects need to be tested outside of the gromacs build tree to validate external interfaces of installation. Suggested external projects: Python package, sample_restraint, yet-to-be-written integration test suite.
- Tests should be clear about the API version they are testing, and we should test all versions that aren’t unsupported (though we need a policy in this regard) and we can note whether new API updates are backwards compatible.
- Forward-compatibility testing: we should at least know whether we are breaking old client code and include release notes, regardless of policy
- ABI compatibility testing? (should we test mismatched compilers and such?)
- Example code in documentation should be tested, if possible.
#1 Updated by Eric Irrgang about 1 year ago
The current API revision is 0.0.6 at https://github.com/kassonlab/gromacs-gmxapi . I think we can wrap a 0.0.7 release that is compatible in full or partial implementation between the GitHub fork and the master branch in Gerrit. It will likely be incompatible with 0.0.6, but I can probably back port some updates to 0.0.6.1 or 0.0.6.2. I plan to make adjustments to 0.0.7 to allow for robust handling of incomplete feature sets so that we can cross-test things during development. It also sounds like a way to do integration testing on Jenkins would be to have a copy of a 0.0.7 release of the Python package and extension code available on the Jenkins build hosts so that we can test GROMACS master as 0.0.7 is implemented there
In the even that gmxapi features need to have incompatible interface changes made for inclusion to GROMACS master, those patch sets can bump their version number to 0.0.8.
There isn't currently an aspect of the gmxapi::Version interface to check whether a release number represents a final specification or a proposed specification. Maybe we need to add one.