Improve FEP testing
FEP test coverage should be improved. This redmine should serve two purposes:
- Define a set of tests which we believe would give us good coverage
- Discuss the infrastructure needed to implement these tests.
Michael has a number of ideas for point 1, but everyone is welcome to chime in. Once we converge on a set of tests to be implemented, point 2 can be discussed.
#1 Updated by Michael Shirts 5 months ago
We want to test:
- intra and intermolecular interactions. Suggest a ethanol-like molecule, with LJ on the hydrogen, with 4-5 water molecules nearby. Somewhat larger charges on all atoms. Cutoffs can be very short. In a periodic box. This can be used for most of the examples. Can be run at low T so that the molecules stay in a cluster.
- May also need to test changing total charge, will need a separate topology.
Parameters we want to test:
- turning off Coulomb and van der Waals successively
- turning off Coulomb and van der Waals concurrently.
- Explicit topology changes
- MDP changes with intramol keyword
- MDP changes with intermol keyword
- Different types of cutoffs (shift, switch, etc.)
- PME with intermediate lambdas, endpoint lambdas
- PME with both VDW and Coul, including both changing at the same time.
- md, md-vv, sd
- With constraints (Rattle, settle, lincs)
- Changing torsion
- Changing angle
- Changing bond
- Changing restraints
- Changing nonbonded pairs (may need different topology)
- In all cases, are endpoint lambda energies equal to non-free energy topologies?
- Will want to verify that pressures are correct.
- Should be OK to run through 1-2 neigborsearches - don't need that many steps.
- Expanded ensemble
For all of these, will need to look at total energy, derivatives of energy, delta energy (in dhdl file). It will be hard to test Hamitonian replica exchange.
I will keep brainstorming.
Many of these options will need to be tested combinatorially, but many combinations are mostly redundant.
#2 Updated by Michael Shirts 4 months ago
- Put in folder FreeEnergy/
- each test has own folder.
- put .top/.gro/.itp/README.txt in each subfolder.
- README gives 1-2 line description of each test.
- Use standard naming for the tests.
- Still try to number the tests.
- README.md on the outside describing naming convention.
#3 Updated by Erik Lindahl 4 months ago
I would strongly urge you not to do this only as end-to-end tests running a complete simulation, but do it on the code level as unit tests:
1. Create a test harness that sets up the data structures you need.
2. Create a bunch of tests that call e.g. just the force evaluation and check that specific free energy terms are correct
3. Call integration routines the same way, and test individual algorithms.
By doing this on the code level, each test should finish in ~0.01-0.1 seconds, which enables you to have hundreds of tests.
#4 Updated by Michael Shirts 4 months ago
I don't have the time to pull everything out now that failed in the past, but by feeling has been that the majority of the free energy bugs have been because of properly designed individual components of GROMACS being misinterpreted by either developers or users, resulting in unintended results.
I can think about unit tests as well, but this is what I'm planning to do for now. If you want to allocate funds at Stockholm for having someone write unit tests, then of course that could only be good as well.