Project

General

Profile

Bug #559

g_bar disk I/O very slow with the edr support

Added by Szilárd Páll about 9 years ago. Updated about 9 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
analysis tools
Target version:
Affected version - extra info:
Affected version:
Difficulty:
uncategorized
Close

Description

With the free energy values stored in the edr output (separate_dhdl_file=no mdp option) the disk g_bar disk I/O levels out around 500 Kb/s (in contrast, with the xvg output the disk I/O is typically around 15 Mb/s). As result, processing a set of 15 edr files from 20ns free energy simulations (free energies written every 200 step) that results in ~70 Mb edr files takes up to 1 hour.

By looking briefly at the code, the problem we saw is that several memory allocations are made when reading every frame. This code probably need reordering to increase the disk I/O and make this new g_bar feature usable.

History

#1 Updated by Sander Pronk about 9 years ago

What are the values for nstenergy and nstdhdl in this case?

#2 Updated by Szilárd Páll about 9 years ago

(In reply to comment #1)

What are the values for nstenergy and nstdhdl in this case?

nstenergy = 200
nstdhdl = 10

I've put a dataset that you can try on to the usual location, in my public directory on AFS.

#3 Updated by Sander Pronk about 9 years ago

That is an extremely low value for nstenergy: you're writing an energy frame every 200 steps(!). When outputting to edr, nstdhdl is still the value that determines the amount of free energies that get written out; they're just collected into an array that then gets written out during the energy frame writing. If you set it to something more sensible (like 10000), I/O will improve tremendously.

#4 Updated by Szilárd Páll about 9 years ago

(In reply to comment #3)

That is an extremely low value for nstenergy: you're writing an energy frame
every 200 steps(!). When outputting to edr, nstdhdl is still the value that
determines the amount of free energies that get written out; they're just
collected into an array that then gets written out during the energy frame
writing. If you set it to something more sensible (like 10000), I/O will
improve tremendously.

True, I just realized this.

#5 Updated by Sander Pronk about 9 years ago

Thanks for raising this issue; I'll update the manual. I could try to implement some optimizations in g_bar, when I have time.

Also available in: Atom PDF