Project

General

Profile

Feature #1242

Documentation reorganization

Added by Mark Abraham about 4 years ago. Updated about 1 year ago.

Status:
Accepted
Priority:
Normal
Assignee:
Category:
documentation
Target version:
-
Difficulty:
uncategorized
Close

Description

Specifically for the online manual:
Some of this content should die because it is out of date and/or difficult to maintain. Some should go to a wiki page and fester. Some should go into the PDF manual. The parts that are automatically generated can remain, but mkhtml needs to update the date and version timestamps on them (or use a CMake configuration script to do that). Some of it should move into the code and be generated by Doxygen.

documentation.pdf - work-in-progress illustration of monolithic PDF from sphinx (1.25 MB) Mark Abraham, 05/27/2016 11:10 AM


Related issues

Related to GROMACS - Task #2146: salvage documentation from old webpage New

Associated revisions

Revision 93174074 (diff)
Added by Teemu Murtola almost 4 years ago

Remove old generated HTML files.

Related to #1242.

Change-Id: Ic641c8ee09325d5abf124a94ed2a1c1b525cda31

Revision abc93fe9 (diff)
Added by Teemu Murtola almost 4 years ago

HTML export from the wrapper binary.

Implement HTML output format for 'gmx help -export'.
Fix an issue that caused 'mdrun -man html' to segfault.
Move the HTML header and footer generation out of wman.cpp and clean it
up (close incomplete tags, remove the date, etc.).

Clean up the layout of the online manual (on the file system) by putting
the generated files into a separate directory. Also generate an
alphabetical list of programs into its own file instead of directly on
the front page. By-topic list is currently missing.

Left out installation rules for now; can be reinstantiated once #1242
and the general approach to the whole online manual is clearer.

Will fix hyperlinks etc. in the generated output in separate change(s).

Related to #685, #969, and #1242.

Change-Id: Iaf8fc28d563f05a8e00c7c52d58b91cd1dabf369

Revision 88d213b1 (diff)
Added by Teemu Murtola over 3 years ago

Add listing of programs by topic to HTML output.

Now the HTML-exported help contains also a list of programs by topic,
similar to what used to be generated from programs.txt. Removed the
mkhtml script, since it is now fully replaced by the mechanism in the
wrapper binary.

The same mechanism could also be used to replace the gromacs.7 man page,
but in its current form, it contains so much boilerplate code that I
didn't want to copy-paste that all into the source file. And it could
also be used to make the output of 'gmx help' more structured.

Related to #685, #969, and #1242.

Change-Id: I6c2efe10c53f10f7fde90b3386ddea7fbea34b89

Revision 4d916bcd (diff)
Added by Teemu Murtola over 3 years ago

Remove more obsolete content from share/html.

- yourown.html was not referenced.
- Will replace ADD_INCLUDE and include_*.html with something more useful
in the future.
- FAQ link in online.html was referencing a file deleted in release-4-6.
- Other changes remove content that was referencing share/tutor/, which
was removed earlier.

Related to #1242.

Change-Id: I0e04438694cf251eae789c08e2ae1087d6fd99fc

Revision 4c15f726 (diff)
Added by Teemu Murtola over 3 years ago

Add back mechanism for cross-tool hyperlinks.

The exported HTML pages again can contain hyperlinks to other tools.
Instead of replicating the old functionality, made the links of the form
[gmx-distance] instead of recognizing plain text words. This requires
all the output formats to be aware of them (not a big deal in the
current code layout), and makes it possible to define the appearance of
the links centrally instead of relying on all links being wrapped in,
e.g., [TT]...[tt]. The new link syntax is similar to Markdown implicit
links (at least Doxygen accepts them without the second []).

Also added support for [THISMODULE] tag in the help text, which expands
to the name of the current module, again allowing the appearance to be
defined centrally.

Only changed the help text for gmx-mindist to take advantage of the new
mechanism as a proof-of-concept. Bulk work for other tools is better
done as a separate commit once the general approach is accepted.

Part of #685 and #1242.

Change-Id: Ibd263cc2c131dc18d2a5d9046fecc6b1c08734f9

Revision 3676eb71 (diff)
Added by Teemu Murtola over 3 years ago

Uniform headers in HTML pages

The version number is now also automatically updated in the page headers
for 'make html'. Also, fix most links in the file format pages.
If we in the future move into some proper HTML publishing system (e.g.,
based on Markdown), it should probably take care of this and also
cross-links between these pages and the generated pages, so not much
effort has been put to make the approach or the headers/footers
particularly elegant.

Related to #685 and #1242.

Change-Id: I083d1f9714ddf68dfc2977799378299a43a05b73

Revision 2f14f668 (diff)
Added by Teemu Murtola over 3 years ago

Uniform behavior for 'make man' and 'make html'

Both man and HTML pages are now generated and installed using similar
installation rules. GMX_BUILD_HELP controls automatic generation of
both. They are also put into the source distribution using the same
mechanism for both.

Part of #685 and #1242.

Change-Id: Id61e75623861b46f765da49c4b34047a6b327083

Revision ec2c9197 (diff)
Added by Mark Abraham about 3 years ago

Convert install guide to markdown

Adds `make install-guide` to make both a plain-text INSTALL for the
tarball, and a monolithic HTML install guide for uploading to the
website. Intended for processing with pandoc, but I don't think
there's any pandoc-specific extensions used. Naturally, if the build
system doesn't have pandoc, then this make target is not available,
just like the other documentation features.

Added more description about various SIMD instruction sets,
portability, and how to use compilers and the GMX_SIMD setting.

Added summary of important CMake options for typical GROMACS
installations.

This now does some variable interpolation to keep accuracy up without
active maintenance, so setting some such variables gets changed a bit.

Mentioned more optional build components

Minor textual changes to the install guide.

Fixes #1440. Refs #1242

Change-Id: Iff8e165759fbf8cc5199c8447e912403b8a79a17

Revision 9f500c77 (diff)
Added by Mark Abraham almost 3 years ago

Unify documentation for webpage build

The new webpage build can be run either from a code repo (for
development), or from the unpacked source tarball (for actual
deployment). The latter is useful to ensure the documentation reflects
the tarball, and because the tarballs and their md5sum values must be
available for building the webpage. A Jenkins flow job can build the
tarballs and then construct the webpage for automated delivery. The
machinery is active only if GMX_BUILD_WEBPAGE is set (and other pieces
in place), so will be transparent to normal users.

Created configurable top-level index.md inside the new docs top-level
directory. Various components have moved into this directory, but each
can still be built independently.

Imported some basic (but pandoc-specific) CSS from
https://gist.github.com/ryangray/1882525

Made a new top-level index.md, and made it build HTML with Pandoc and
use the CSS. Later, we can consider making the whole static site build
work with some generator, e.g. to make links automatically.

Established CMake variables for configuring the webpage build with
the names of tarballs and their md5sum values. Builds from the repo
don't include this part of the webpage.

Moved the install guide into the new docs directory. Minor fixes to
header levels. Generated PDF install guide also. Used
PANDOC_EXECUTABLE instead of hard-coded "pandoc." Made it use the
CSS.

Moved reference manual build into the new docs directory, and linked
its default setting to GMX_BUILD_WEBPAGE.

Moved old html guide content into the new docs directory. This still
installs in the old way. Fixed broken links for xpm2ps and grompp.
Added rudimentary hdb.html to fix missing link target. Removed text
that linked to ancient material that no longer semes to exist.

Moved doxygen generation into the new docs directory. The useful
output now goes directly to build/docs/html, and logs to
build/docs/doxygen.

Bumped copyright years to keep uncrustify script happy.

Bumped top-level CMakeLists.txt and CPackInit.cmake to check for the
new correct paths when building tarballs.

Added config file for linkchecker to help automate checking
the links all work.

Refs #1242

Change-Id: I8e5cf98b2997b76f56b24f45262c9c9eebf9900e

Revision 4850184f (diff)
Added by Mark Abraham over 2 years ago

Implemented documentation with sphinx

Added sections for install guide, main file of old online reference,
and stubs, tools and getting-started sections of user guide.

Removed corresponding parts of old documentation.

Remaining markdown files in the user guide will be converted
to rst in other patches.

Refs #1242

Change-Id: I798f57e39eea47301fb51750099d7988d81fc72d

History

#1 Updated by Justin Lemkul about 4 years ago

Good idea. This definitely needs to be done. I see very little within the online manual that needs to be kept beyond man pages and file formats. Both of these can be easily merged into the wiki site to augment the existing pages, most of which are very sparse. Does version information really matter at that point? Once on the wiki, the pages can be freely edited and the page history will log the dates of changes, so they will more or less correspond with releases (and if they don't, it will be obvious). I will look into mkhtml a bit to see if it can be improved unless somehow we can make CMake do work for us. The Doxygen stuff is beyond what I've looked into and I need to familiarize myself with it. I will have a fair bit of time on my hands over the next few weeks that I can devote to working on this. If there are specific features and/or changes to make, feel free to list them here.

#2 Updated by Szilárd Páll about 4 years ago

While in its current form it is not very useful, an online, easily google-able, and linkable manual is rather useful, I think. Some information can certainly be transferred to the wiki, but there are plenty of things which need to be static and versioned simply because documentation is not universal, it relates to code with a certain version.

Secondly, as the online manual is a user manual, I don't see how would doxygen-based documentation replace part of it. Doxygen generates developer manual from source-code comments and while it can probably be abused to generate something that resembles user documentation, it is not designed for that.

#3 Updated by Mark Abraham about 4 years ago

Justin Lemkul wrote:

Good idea. This definitely needs to be done. I see very little within the online manual that needs to be kept beyond man pages and file formats.

The latter should really go into the I/O code and be generated by Doxygen, else they cannot stay in sync.

Both of these can be easily merged into the wiki site to augment the existing pages, most of which are very sparse. Does version information really matter at that point?

Not really. Dates definitely seem superfluous. I did update the files on the server for 4.6.1, but many of the pages didn't change, so claim to be the version/date of the last change "by hand". If there is a need to have and update a version, then git grep PROJECT_VERSION will show how we do that in the other use cases. Thus build-cmake/admin/mkhtml might become the actual functional script (and would need a comment to that effect).

Once on the wiki, the pages can be freely edited and the page history will log the dates of changes, so they will more or less correspond with releases (and if they don't, it will be obvious). I will look into mkhtml a bit to see if it can be improved unless somehow we can make CMake do work for us.

If a version or date adds value, then it should be generated rather than replicated. However, I imagine most of the versions and dates do not add value.

The Doxygen stuff is beyond what I've looked into and I need to familiarize myself with it.

Have a look at how the master branch uses Doxygen to generate module documentation for option parsing (for example). For the moment, stuff like file format descriptions can just go in a new module at top level. I suggest making changes in master branch, and cherry-picking fixes/deletions for the more egregious things into release-4-6 (e.g. half the content of the FAQ).

I have plans to implement the new trajectory-reading API in master shortly, and we can get file format descriptions more organized then.

I will have a fair bit of time on my hands over the next few weeks that I can devote to working on this. If there are specific features and/or changes to make, feel free to list them here.

Great, thanks!

If you're looking for a project, re-working the install guide source into reStructuredText (http://docutils.sourceforge.net/rst.html) will be good for the future, so we can easily update source and publish HTML and PDF versions. I forget which Redmine issue discusses the various alternatives, but I think the short version is that rST is a superset of Markdown (which is good if simple and HTML is all one is after) and Doxygen has support for Markdown format. There is a small amount of thought of deprecating LaTeX as the format of the PDF manual if something else can do the job well. A proof of concept that rST can do the backbone of the job well would be a good exploratory exercise. Preferring to use only a Markdown-like subset of the available syntax will help developers to not have to learn so many syntaxes to write documentation. (Currently, some knowledge of Doxygen, LaTeX, HTML, some GROMACS-specific markup is needed, along with a bunch of custom scripts and parsers. Reducing that to Doxygen plus a minimal superset of Markdown seems like a positive step.)

#4 Updated by Mark Abraham about 4 years ago

Szilárd Páll wrote:

While in its current form it is not very useful, an online, easily google-able, and linkable manual is rather useful, I think. Some information can certainly be transferred to the wiki, but there are plenty of things which need to be static and versioned simply because documentation is not universal, it relates to code with a certain version.

Sure, but I would not do much work on that principle for things whose content is nearly static (like the index file format). Aside from mdp_opt.html, the content of share/html/online has about one substantive update a year. Moreover, much of the content is applicable to a wide range of versions, so stamping it with the latest version is not a good solution either. In theory, we should have an archive of the documentation for each version, which I think we should start doing once we've done this kind of clean-up.

Secondly, as the online manual is a user manual, I don't see how would doxygen-based documentation replace part of it. Doxygen generates developer manual from source-code comments and while it can probably be abused to generate something that resembles user documentation, it is not designed for that.

The human-readable description of a file format must be with the code or it goes out of date. We can't write a custom parser, so Doxygen is the vehicle for getting stuff in the code out into user-visible space. I think a wiki page with an example and a link into the appropriate module of the developer manual is quite appropriate for a file format description. One aspect of that is starting to publish the Doxygen content so that it too is linkable and Google-able.

#5 Updated by Mark Abraham about 4 years ago

Don't know if there are any "back-parsers" from HTML to Markdown or rST, but if they exist, they might save some work.

#6 Updated by Teemu Murtola about 4 years ago

Mark Abraham wrote:

If you're looking for a project, re-working the install guide source into reStructuredText (http://docutils.sourceforge.net/rst.html) will be good for the future, so we can easily update source and publish HTML and PDF versions. I forget which Redmine issue discusses the various alternatives, but I think the short version is that rST is a superset of Markdown (which is good if simple and HTML is all one is after) and Doxygen has support for Markdown format. There is a small amount of thought of deprecating LaTeX as the format of the PDF manual if something else can do the job well. A proof of concept that rST can do the backbone of the job well would be a good exploratory exercise. Preferring to use only a Markdown-like subset of the available syntax will help developers to not have to learn so many syntaxes to write documentation. (Currently, some knowledge of Doxygen, LaTeX, HTML, some GROMACS-specific markup is needed, along with a bunch of custom scripts and parsers. Reducing that to Doxygen plus a minimal superset of Markdown seems like a positive step.)

The issue is #969. Just to correct a misconception here, Markdown and rST are completely different sets of syntaxes; they don't have much in common. rST is more comprehensive (the original parser is Python docutils, which has a lot of features), while Markdown is a very simple set of formatting constructs (I think the original parser is a relatively simple perl script), which more complicated parsers extend with mutually incompatible extensions. Not sure how easy it is to integrate external files with just Markdown text into a doxygen documentation; at least it must use just those extensions that Doxygen supports, and may also require some scripting (e.g., to make that file appear as it were a source file with all the text in a comment).

#7 Updated by Teemu Murtola about 4 years ago

Mark Abraham wrote:

Secondly, as the online manual is a user manual, I don't see how would doxygen-based documentation replace part of it. Doxygen generates developer manual from source-code comments and while it can probably be abused to generate something that resembles user documentation, it is not designed for that.

The human-readable description of a file format must be with the code or it goes out of date. We can't write a custom parser, so Doxygen is the vehicle for getting stuff in the code out into user-visible space. I think a wiki page with an example and a link into the appropriate module of the developer manual is quite appropriate for a file format description. One aspect of that is starting to publish the Doxygen content so that it too is linkable and Google-able.

Depending on what stuff from the code needs to get into the documentation, there are two approaches:
  1. Doxygen, if it really pertrains to the code (as, e.g., function documentation), or is more or less independent of it (as a form of Doxygen "related pages" documentation).
  2. Generating the documentation using code; this is the approach currently used for the man pages etc., and I can't see how Doxygen could replace this without writing some custom scripting somewhere. I would rather go the other way, and also generate the mdp option listing this way.

It is possible to generate all kinds of stuff with Doxygen (the whole Doxygen home page, with user manual and all, is generated with Doxygen), but I'm not sure whether this is the best approach. If there is a lot of text that doesn't apply directly to the code, it still makes sense to maintain those in separate files. With this, stuff that has no direct relation to the code still gets outdated unless someone remembers to update it, and whether the format it is in is Doxygen or some other markup doesn't really matter.

Reducing the places that one needs to search for outdated documentation makes sense, though, so I agree that manually maintained portions of the online manual should be merged somewhere.

#8 Updated by Szilárd Páll about 4 years ago

Mark Abraham wrote:

The human-readable description of a file format must be with the code or it goes out of date. We can't write a custom parser, so Doxygen is the vehicle for getting stuff in the code out into user-visible space. I think a wiki page with an example and a link into the appropriate module of the developer manual is quite appropriate for a file format description. One aspect of that is starting to publish the Doxygen content so that it too is linkable and Google-able.

I see your point and for certain things that map well to entities in the code - which doxygen was designed for -, like mdp options, it does make sense to use in-code documentation. However, putting user-documentation of file formats into source code still sounds bizarre to me. And by user documentation I mean content like the current .gro file format page which has very little to do with code. However, it may be the case that for many if not most of the file formats there is a well suited source file to hold user docs, case in which a few exceptions, like the .gro format is acceptable.

#9 Updated by Mark Abraham about 4 years ago

How would you solve the problem, Szilard? It's very frustrating for those of trying to solve long-standing problems to be told that our proposed solutions sound bizarre, without an alternative being proposed. :-)

I don't mind how we organize all our information, so long as there's a decent chance that code and documentation stays in step. Clearly the online manual has been a near-wasteland. When I finish porting the PDF manual into the main repo, we could re-organize into something like
  • an install guide (content roughly as it is now)
  • a user guide (lots of practical how-to material like using acceleration, GPU, etc., file formats, simulation workflows, probably parts of manual chapter 5, 6 and 8 and appendix D, probably a fair chunk of wiki content can move in here)
  • a reference manual (describe physics, algorithms, their implementation and implementation details, containing all the rest of the content of the current PDF manual, but usage information should be in the user guide)
    I'm not sure whether .mdp options belong in the user guide or reference manual.

These all get built with common machinery that can publish HTML for us (so we can cross-link between them easily, and only if necessary into Doxygen developer docs), and preferably also publish PDF for those who prefer that. Version stamping happens automatically and we retain old versions of documentation in their own directories on the www server.

#10 Updated by Roland Schulz about 4 years ago

#969 discusses some options of how to generate the PDF and html pages from the same documentation sources (and how to integrate it with man pages and developer documentation). http://sphinx-doc.org/ and http://johnmacfarlane.net/pandoc/ as tools sound promising. spinx might be better to organize everything. pandoc lets one convert formats (e.g. Tex->Rst/Markdown).

#11 Updated by Roland Schulz over 3 years ago

cmake has converted their documentation to rst for 3.0, and use sphinx for it. Looks nice on a first quick glance.

#12 Updated by Teemu Murtola over 3 years ago

  • Tracker changed from Bug to Feature
  • Subject changed from online manual should die to Documentation reorganization
  • Description updated (diff)
  • Status changed from New to Accepted
  • Assignee changed from Justin Lemkul to Mark Abraham
  • Target version set to 5.0

Mark has indicated that he will work on this. Changed the title to more widely reflect the scope, as has been indicated in the comments.

There are at least the following classes of documentation that should be considered in the organization:
  1. Developer documentation for actual code constructs
    • Extracted by Doxygen; this we probably do not want to change
  2. Developer documentation for code on a higher level
    • Currently partly in Doxygen, but some random/outdated stuff is also in the wiki.
    • Needs to heavily reference 1.
  3. Developer documentation about tools (uncrustify, Gerrit, Jenkins)
    • Currently partly in scripts (e.g., comments in uncrustify.sh), partly in wiki.
    • Parts that directly relate to the source code (e.g., uncrustify.sh) would be best if part of the repo.
    • Parts that do not relate to the actual source code (like Jenkins configuration or using git/Gerrit), may benefit from being outside the repo. But getting review for these parts on Gerrit wouldn't hurt, either.
  4. Developer documentation about policies (code formatting, allowed features, recommended design, …)
    • Currently partly in Doxygen, partly in wiki.
    • Some parts need to reference 1-3.
  5. Algorithm descriptions
    • Currently in the PDF manual
    • Would benefit from ability to reference the code (or perhaps the other way around).
  6. Reference for usage (man pages, online manual, mdp option listing, list of available tools …)
    • Currently partly in man pages, partly in online manual, partly in PDF manual.
    • Currently partially maintained manually, partially generated from the code.
    • Parts that directly relate to a list in code (like command line options, mdp options, environment variables, …) would benefit from automatic generation from the source code to ensure they stay in sync.
    • General information that ties this all together does not benefit from being generated from the source, but needs to heavily reference the generated information.
  7. Installation instructions
    • Currently a separate file in the repo that gets copied into the wiki. PDF manual may also have some information about this.

#13 Updated by Erik Lindahl about 3 years ago

  • Target version changed from 5.0 to 5.x

Some of this has already been accomplished, and there are a bunch of 5.0-related changes that have already been committed or are in gerrit.

Doesn't mean we're done, but the next steps will have to wait for later releases.

#14 Updated by Mark Abraham about 3 years ago

Moving some email discussion between Teemu and I here...

Mark said
Also, I haven't forgotten that doxygen is not officially deployed yet - I literally ran out of time. Hopefully there's not anything that >> I really forgot! :-) I'll be back in the loop in a few days.

Teemu said
For deploying Doxygen and other similar documentation, I think the ideal situation would be that there would be a website similar to >manual.gromacs.org, but which would contain all the documentation, versioned. So that, e.g.,
manual.gromacs.org/5.0/online/ would contain the current online manual,
manual.gromacs.org/5.0/doxygen/ would contain the Doxygen documentation,
manual.gromacs.org/5.0/install-guide/ would contain the installation instructions,
etc. The discussion for this could continue in Redmine #1242. In the long run, even the current PDF manual should go there.

I also have a half-written user guide that would join the party.

Further, ideally this would be deployed more or less directly from Jenkins so that the same structure would be visible already when Jenkins builds it, and at most, the official deployment would mean copying the stuff to the other server. Getting the release version numbers there may require bundling this action with creating the source tar ball, which may require some extra Jenkins work, but could pay off in the future.

Indeed, I wholly agree. I had made a start on this, but didn't get time to harden it enough to use for 5.0. Some WIP at http://jenkins.gromacs.org/job/Build_website_5_0/configure that would eventually be hooked into http://jenkins.gromacs.org/job/Deploy_5_0/configure. Most of the details are still up for improvement, and at some point we'd probably roll the script parts of the mechanisms into releng.git so we can have a bit of version and mutual quality control.

Or, if Jenkins is a reliable enough platform, the actual contents could reside on the Jenkins server, as builds that are marked as permanently kept, and appropriate links would be posted elsewhere, removing the need to deploy anything else than references to the Jenkins builds.

Sure, using web-server redirects (or such) to make things work would resolve the question of how to automate the transfer of files safely. We could also have http://manual.gromacs.org/release-5-0/online etc. up all the time, e.g. built nightly. One ongoing caveat is to make sure that source and tests tarballs go somewhere that's got enough infrastructure behind it. Since Google Code no longer accepts uploads, I've been putting the latter on a KTH DropBox-style thing and redirecting from gerrit.gromacs.org to there. That's mildly irritating, but it's probably better than loading our own servers.

#15 Updated by Mark Abraham over 2 years ago

https://gerrit.gromacs.org/#/c/4388/1 re-implements the documentation intended for placement on a webpage. (I have more work basically ready to upload, also.)

It uses http://sphinx-doc.org/, which is primarily intended for Python, but is also used by the CMake and LLVM projects. That should be enough weight that we'll find useful things in the ecosystem. Erik intends to suggest the Copernicus docs move to Sphinx also. And there's already http://breathe.readthedocs.org/en/latest/ if we need a bridge between Sphinx and Doxygen. Sphinx can handle PDF output via LaTeX, and inline maths via that or MathJax, so I think our ongoing needs should be able to be managed.

There's some messiness while we deal with documentation remaining in other formats (the old HTML online reference and PDF reference manual), but in time I think we can have a nicely cross-referenced infrastructure that is easy to version and publish.

#16 Updated by Gerrit Code Review Bot over 2 years ago

Gerrit received a related patchset '13' for Issue #1242.
Uploader: Teemu Murtola ()
Change-Id: I798f57e39eea47301fb51750099d7988d81fc72d
Gerrit URL: https://gerrit.gromacs.org/4388

#17 Updated by Mark Abraham over 2 years ago

Following https://gerrit.gromacs.org/#/c/4429/7 moving generated docs to rst, we need to go through and remove use of [BR] from tools and selection docs.

#18 Updated by Mark Abraham over 2 years ago

Mark Abraham wrote:

Following https://gerrit.gromacs.org/#/c/4429/7 moving generated docs to rst, we need to go through and remove use of [BR] from tools and selection docs.

Teemu's way ahead of me, as usual :)

#19 Updated by Mark Abraham about 1 year ago

Update - particularly Berk and Szilard find the friction of not having the .mdp options in a monolithic PDF is high. People would like the ability to link into the .mdp section, which Berk thinks we could do in olden days.

We certainly had cross-links within the .mdp settings, which will still do. But we had none from anything into the mdp options (e.g. there's none in the 5.0 PDF reference manual from either the pull code or the Verlet scheme), which is still true (modulo the inconvenience of not being in the same PDF).

We now have cross links from the user guide into the Verlet settings mdp section, e.g. http://manual.gromacs.org/documentation/5.1.2/user-guide/cutoff-schemes.html#how-to-use-the-verlet-scheme, which illustrates the kind of place that we all want to get to. We're not going back to everything in LaTeX and then work out how to have reasonable HTML. Sphinx can do the full job once we put some more effort in - primarily how to handle the tables, images, citations and maths in the current reference manual, in both HTML (via CSS) and PDF. (No small task!)

For example, attached is the PDF from Sphinx for everything not currently in the PDF reference manual, plus an initial port of the introduction chapter of the reference manual. There's some obvious improvements to make, but this seems to be the way forward so long as we can find some technical solutions.

#20 Updated by Gerrit Code Review Bot about 1 year ago

Gerrit received a related patchset '1' for Issue #1242.
Uploader: Mark Abraham ()
Change-Id: I2afed23b48b9ef23c1957b3932089a5f82d4a227
Gerrit URL: https://gerrit.gromacs.org/5911

#21 Updated by Szilárd Páll about 1 year ago

Mark Abraham wrote:

For example, attached is the PDF from Sphinx for everything not currently in the PDF reference manual, plus an initial port of the introduction chapter of the reference manual. There's some obvious improvements to make, but this seems to be the way forward so long as we can find some technical solutions.

Looks good, but I'm not sure it's beneficial to dump everything into this PDF, e.g. the developer documentation seems to be targeting a totally different audience.

I'm not certain that the conversion of the current manual into Sphinx is a beneficial shot-/mid-term plan. It takes a lot of effort and the typesetting will likely look nothing like the current one. It feels like improving the manual as a stand-alone method/algorithm background and hacking in some cross-references could be more beneficial -- at least while the rest of code and documentation is also changing relatively fast.

#22 Updated by Mark Abraham about 1 year ago

Szilárd Páll wrote:

Mark Abraham wrote:

For example, attached is the PDF from Sphinx for everything not currently in the PDF reference manual, plus an initial port of the introduction chapter of the reference manual. There's some obvious improvements to make, but this seems to be the way forward so long as we can find some technical solutions.

Looks good, but I'm not sure it's beneficial to dump everything into this PDF, e.g. the developer documentation seems to be targeting a totally different audience.

Good point, we could/should indeed not do that.

I'm not certain that the conversion of the current manual into Sphinx is a beneficial shot-/mid-term plan. It takes a lot of effort and the typesetting will likely look nothing like the current one.

Why wouldn't it look comparable? RST->LaTeX->PDF should be able to do about as well as LaTeX->PDF. See e.g. the maths in 4.1 of the above PDF.

It feels like improving the manual as a stand-alone method/algorithm background and hacking in some cross-references could be more beneficial -- at least while the rest of code and documentation is also changing relatively fast.

Would be useful ~now, but would need someone to work out how to get it done by e.g. hacking cross links into a PDF concatenated from two different sources, or working out how to combine the two LaTeX sources. My preference will be for the long term solution, when we can e.g. use :mdp:`nstlist` everywhere. So far I've only had a go at that single introduction chapter, but I've had good success with maths, citations and simple tables (not yet available on Gerrit). I haven't tried equation numbering or section+equation cross references, yet.

Even longer term, I think that much of the user-level documentation that is now separate from the code could move closer to the code. For example, the pull code could have the kind of stuff currently in the reference manual, plus some practical examples, all together in the module, and the developer documentation could incorporate the user subset, which would help reduce text duplication. Example file snippets could also be tested to work. This is easier to get done if all the pieces of text are in a consistent format

Similarly, to understand how R-B dihedrals work at the moment, you have to look up the right bits of chapters 4 and 5. The cross links I added to the tables in chapter 5 are very useful, but there's none back the other way. We've also seen several cases over the years where stuff was out of sync because it was in three places. This also side-steps the complexity of typesetting those big tables in chapter 5.

#23 Updated by Mark Abraham about 1 year ago

  • Target version deleted (5.x)

#24 Updated by Mark Abraham 4 months ago

  • Related to Task #2146: salvage documentation from old webpage added

Also available in: Atom PDF