Project

General

Profile

Task #1070

GPU documentation

Added by Justin Lemkul almost 7 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
documentation
Target version:
Difficulty:
uncategorized
Close

Description

For the 4.6 release, manual section 6.12 needs to be overhauled, as much of the information there is no longer correct. Not sure who is best to handle this, so again I'll list Mark so that it can be delegated.


Related issues

Related to GROMACS - Task #1217: GPU website documentationClosed04/08/2013

History

#1 Updated by Mark Abraham almost 7 years ago

  • Assignee changed from Mark Abraham to Szilárd Páll

Szilard, this seems like your sandbox - probably overlaps a bit with papers. Some of this material is also on the wiki pages, but only of some of that belongs here.

As a first effort at describing the scope for this manual section, we need to
  • make clear the feature sets both required for and supported by GPU-enabled code,
  • discuss the use of mdrun command line options (since mdrun -h is already overburdened with stuff; probably we should say nothing much there and instead tell users to read the manual GPU section. This should apply to other features too, like multi-simulations),
  • do a decent job of disclosing conditions that we know are bad and will stay bad (but leave compilers out of it, that should be the job of the installation guide),
  • prefer to describe features by name, rather than a CMake variable name that might change,
  • document that the old OpenMM code is available in contrib, but unsupported,
  • document the hybrid mode,
  • document the emulation mode,
  • discuss thread pinning,
  • discuss interaction with PME and DD load balancing,
  • others...?

Feel free to add/remove/discuss such points.

#2 Updated by Szilárd Páll almost 7 years ago

Ack, I need to start working on this. However, I would very much prefer to not have everything completely separated. The GPU acceleration is IMO is in many aspects not a whole lot different from theVerlet scheme CPU SIMD acceleration: both are accelerated non-bonded algorithms part of the Verlet/NxN scheme with a certain (nearly identical feature support), methodological and algorithmic properties, and very similar run-time behavior. Other things like mdrun options or PP-PME load balancing are not worth discussing separately for GPUs either.

Therefore, all the common aspects should be discussed together both in the documentation and the wiki and only the try GPU-specific peculiarities need a separate GPU acceleration section in the manual. I'll need to have a look at the current level of documentation of the common aspects as well as how/where to add specifics of GPU acceleration. Suggestions are welcome.

Finally, the current GPU section and http://www.gromacs.org/gpu page should be renamed to Gromacs-OpenMM. Update to the OpenMM-related stuff is also required, but we might not have a whole lot of time to do more than moving/renaming the content and perhaps changing the software/hardware requirements and compatibility specs.

#3 Updated by Justin Lemkul almost 7 years ago

Szilárd Páll wrote:

Ack, I need to start working on this. However, I would very much prefer to not have everything completely separated. The GPU acceleration is IMO is in many aspects not a whole lot different from theVerlet scheme CPU SIMD acceleration: both are accelerated non-bonded algorithms part of the Verlet/NxN scheme with a certain (nearly identical feature support), methodological and algorithmic properties, and very similar run-time behavior. Other things like mdrun options or PP-PME load balancing are not worth discussing separately for GPUs either.

Therefore, all the common aspects should be discussed together both in the documentation and the wiki and only the try GPU-specific peculiarities need a separate GPU acceleration section in the manual. I'll need to have a look at the current level of documentation of the common aspects as well as how/where to add specifics of GPU acceleration. Suggestions are welcome.

Maybe the best way to approach this is to make a section called "Accelerated Nonbonded Algorithms" and address all of the above points there. They seem inextricably linked, so it certainly makes sense, as you say, to address them together. Anything specific to GPU usage could be put in a subsection, including anything specifically related to OpenMM, and it should be clear that it is a contributed feature and will not have long-term support.

Finally, the current GPU section and http://www.gromacs.org/gpu page should be renamed to Gromacs-OpenMM. Update to the OpenMM-related stuff is also required, but we might not have a whole lot of time to do more than moving/renaming the content and perhaps changing the software/hardware requirements and compatibility specs.

I renamed the page and did some edits. I left specs and compatibility stuff alone since I can't speak authoritatively on that stuff.

#4 Updated by Szilárd Páll almost 7 years ago

Justin Lemkul wrote:

Szilárd Páll wrote:

Ack, I need to start working on this. However, I would very much prefer to not have everything completely separated. The GPU acceleration is IMO is in many aspects not a whole lot different from theVerlet scheme CPU SIMD acceleration: both are accelerated non-bonded algorithms part of the Verlet/NxN scheme with a certain (nearly identical feature support), methodological and algorithmic properties, and very similar run-time behavior. Other things like mdrun options or PP-PME load balancing are not worth discussing separately for GPUs either.

Therefore, all the common aspects should be discussed together both in the documentation and the wiki and only the try GPU-specific peculiarities need a separate GPU acceleration section in the manual. I'll need to have a look at the current level of documentation of the common aspects as well as how/where to add specifics of GPU acceleration. Suggestions are welcome.

Maybe the best way to approach this is to make a section called "Accelerated Nonbonded Algorithms" and address all of the above points there. They seem inextricably linked, so it certainly makes sense, as you say, to address them together. Anything specific to GPU usage could be put in a subsection, including anything specifically related to OpenMM, and it should be clear that it is a contributed feature and will not have long-term support.

The thing is that there are several aspects that need to be discussed in several places. Both manual and wiki, as well as install guide need some mentioning of all aspects which are:
  • CPU SIMD (acceleration) in general (there is quite a bit of SIMD-accelerated, code now outside of the NB kenrels);
  • GPU acceleration in general: OpenMM vs native (which is largely equivalent to CPU SIMD except some restrictions + some complications in practical use)
  • new heterogeneous + multi-level parallelization scheme;
  • practical things including compiling, running, optimal launch configuration, etc.

I'm still not sure what goes where and how and I would rather like to discuss the structure and ideal place/context for all these before starting to add information to arbitrary resources in a possibly badly organized manner.

Finally, the current GPU section and http://www.gromacs.org/gpu page should be renamed to Gromacs-OpenMM. Update to the OpenMM-related stuff is also required, but we might not have a whole lot of time to do more than moving/renaming the content and perhaps changing the software/hardware requirements and compatibility specs.

I renamed the page and did some edits. I left specs and compatibility stuff alone since I can't speak authoritatively on that stuff.

Thanks, I'll deal with the rest later.

#5 Updated by Szilárd Páll almost 7 years ago

Szilárd Páll wrote:

I renamed the page and did some edits. I left specs and compatibility stuff alone since I can't speak authoritatively on that stuff.

Thanks, I'll deal with the rest later.

Actually, the change is not necessarily the most advantageous. The url which google and others know about is still www.gromacs.org/gpu, bit the title and content is now Gromacs-OpenMM specific.

I suggest that we:
- move the current page to either /GROMACS-OpenMM or perhaps better /mdrun-openmm;
- use the /GPU page to describe briefly the difference between OpenMM and native GPU support;
- IMO the information on native GPU acceleration should go elsewhere, the best option I see is to have all relevant information where it belongs: some stuff on the parallelization/acceleration page, other stuff on the Cut-off schemes page, install info at the installation instructions; not sure where would hardware/software compatibility information and prerequisites and related topics should probably get a separate page that contains information relavant to GROMACS in general (compilers, libraries, build tools, etc). The latter will anyway need more emphasis as it changes significantly with 4.6.

#6 Updated by Erik Lindahl almost 7 years ago

Just as everybody else wants to advertise the strong sides of their GPU code, I think we should obviously use www.gromacs.org/gpu to feature and describe the performance of the new GPU code we have - in particular since we want to get some PR for it, and also highlight algorithms & papers we have developed around it.

Then we should of course have a link somewhere there to describe e.g. implicit solvent capabilities available on the OpenMM page.

The stuff at www.gromacs.org/gpu should be focused on explaining to people with some prior Gromacs experience how they get their simulations running efficiently on GPUs, or what they should think of when buying new hardware. Then we'll refer to other pages for details about settings or installation.

#7 Updated by Szilárd Páll almost 7 years ago

Erik, that sounds like a good suggestion, but I would still like to keep the main body of GPU-related information on the pages where it belongs, as suggested before, e.g. the details on the parallelization scheme on the "Acceleration and parallelization" page, etc.

Of course, we can keep this page as one that highlights the GPU acceleration as a new feature, but I'd prefer to keep it short and sweet instead of long and documentation/guide-like.

As always, I am picky, so here's the nitpick: if we devote a page to GPU acceleration, why not devote one to the awesome strong scaling that enables us to scale to at least 100 atoms/core (and probably) below with decent parallel efficiency (>50%)? :)

#8 Updated by Justin Lemkul almost 7 years ago

There's definitely more at work here than when I made the initial edits ;) For now, I have reverted what I did so that www.gromacs.org/gpu points to a page titled "GPUs," as it did before. It seems to me that this page should definitely be the go-to page for everything related to GPUs, with links to various supporting pages. For instance, on the GPU homepage, we might consider having:

1. Short explanation of features available on GPU via native acceleration.
2. Mention OpenMM and link to a page titled "OpenMM Support" (or something similar, but now as I look at it, pages like "mdrun-openmm" or "GROMACS-OpenMM" might even be too specific)
3. Expound upon the awesome performance with whatever statistics we'd like to provide, and link to the various pages Szilard mentioned, regarding the nuts and bolts of how the performance is achieved
4. Hardware/software compatibility and requirements, since I think that this is very key information for people who want to use the GPU features, so this should be central
5. Link to installation instructions with a focus on CMake-related stuff
6. Link to a page with run parameters and optimizations for various platforms

I think the above framework would largely avoid an imposing wall of text, while still highlighting the most critical information (capabilities, performance, hardware) right up front.

#9 Updated by Szilárd Páll almost 7 years ago

OK, let's sort out first what goes where, I/we can move & rename things later. What's certain is that everything the current GPU page contains (as well as the similarly named parts of the manual) refer to what we call now Gromacs-OpenMM/mdrun-openmm and therefore should be moved to a page/section in the manual which reflects this.

The new GPU page needs to be built up from scratch and your list of suggestions sounds like a great starting point to build that page.

#10 Updated by Erik Lindahl over 6 years ago

  • Target version changed from 4.6 to 4.6.1

#11 Updated by Berk Hess over 6 years ago

  • Status changed from New to In Progress
  • Assignee changed from Szilárd Páll to Berk Hess

I uploaded a fix on Redmine.

#12 Updated by Berk Hess over 6 years ago

  • Status changed from In Progress to Closed

Also available in: Atom PDF