For the first GROMACS 2020 beta, the Context implementations in gmxapi.operation are very minimal. Ensemble awareness is minimal and only serial execution of operations for each ensemble members is supported. (Note that gmxapi.simulation.mdrun uses the legacy gmxapi.simulation.context module for parallel execution of ensemble simulations.)
In conjunction with supporting parallel execution, we should expand the interface between Context and operations to describe coscheduling requirements and data locality issues.
Additionally, we need to improve the interaction between Contexts, such as with subscribability of Futures.
This is also related to data shaping issues (#2994) and management of working files.
Some operations (e.g. scatter(), gather() and reduce()) will need to be incorporated into the Context specification, and existing implementations in gmxapi.operation will no longer be standalone functions.
#3 Updated by Eric Irrgang 6 months ago
gather() does not yet support input resulting directly from
scatter(), which produces an EnsembleDataSource instead of a Future with ensemble width. As a workaround, one could call EnsembleDataSource.node() for each member to get a Future, and call Future.result() to get the elements of an array.