Project

General

Profile

Task #3275

Task #3272: Port complete CI testing to Gitlab

Document Gitlab CI instead of Jenkins and releng

Added by Paul Bauer 3 months ago. Updated 26 days ago.

Status:
New
Priority:
Normal
Assignee:
Category:
build system
Difficulty:
uncategorized
Close

Description

To complete the switch to Gitlab, the documentation currently found in the releng repository needs to be ported and adapted to match the Gitlab build and test system.

Associated revisions

Revision f005f5d5 (diff)
Added by Eric Irrgang about 2 months ago

Establish stubs for documenting GitLab configuration.

Insert an `infrastructure` document under the `tools` node to own the
`jenkins` doc and a new `gitlab` doc.

Refs #3275

Change-Id: I4aa03949b4216290933716beb88b439a8a0940c4

Revision d2e82fbd (diff)
Added by Eric Irrgang 13 days ago

Reorganize release templates.

Rename "release" config file to "archive" to collect jobs related to
preparing archives for export. "release" jobs are just special cases of
jobs that already fit in other categories.

Move several jobs between the "lint," "documentation," "testing-matrix,"
and "archive" groupings. Sort job definitions within files in terms of
execution dependency, definition dependency, and toolchain version.

In the future, we can probably reduce the distinction between jobs that
are 'release' or not (jobs that always run may choose a code path
according to to $GROMACS_RELEASE) so I'll leave the 'release' jobs
clustered with the non-'release' jobs.

Refs #3275

Change-Id: If675be3c64dfc2b42cdc4eba6b940226661ddbf3

Revision 0f080f59 (diff)
Added by Eric Irrgang 13 days ago

Adopt CI job naming guidelines.

Refs #3275

Change-Id: Ib1fdf4eb468fe2a4620e523f265226c09051bba2

History

#1 Updated by Paul Bauer about 1 month ago

  • Target version changed from 2020.1 to 2021-infrastructure-stable

still needs work

#2 Updated by Erik Lindahl about 1 month ago

My plan is that we should stick entirely to Gitlab's default docs for the overall infrastructure (which is far better than ours), and only document our deviations from their typical flow.

We might also need to document a few things about how we create images, but here too our first task is to learn to keep it simple. Once we've had things running smoothly for 6 months we can start to think about customisation :-)

#3 Updated by Eric Irrgang 26 days ago

The biggest challenge for me right now is in understanding the naming schemes and many sources of job composition. It is not clear where standard parameters are supposed to come from.

Note that the depth and breadth of extends usage makes it harder to understand the GROMACS YAML configuration in terms of standard GitLab documentation, and might be considered deviant from typical flow.

Other deviations include the large number of build stages and extensive use of "needs" instead of "dependencies".

Local details to document include the various stages defined, internally relevant variables, reusable artifacts, and conditional job logic (both suggested heuristics and which configuration mechanics should be used, particularly since we aren't using the newer "rules" syntax).

#4 Updated by Paul Bauer 26 days ago

Standard parameters are defined in the variable template and the build/test script templates and are also printed to the output for the jobs that do the actual CMake configuration.
I tried to make the individual included parts individually small so that we are able to compose the actual final job according to a combination of different extends template, something I also saw recommended somewhere in the Gitlab documentation.

Needs is recommended to create non-blocking dependecies, so that different test jobs only need to wait for their build job to be done and not all of them. I think waiting on a complete stage to finish is counterproductive for our kinds of testing, as we have many different build and test jobs that don't depend on others and can fire their dependent jobs before all the jobs in the same stage are done.

#5 Updated by Eric Irrgang 26 days ago

I added some provisional notes about variables and added a question about needs to https://gerrit.gromacs.org/c/gromacs/+/15880

I removed the "private" marker from https://gerrit.gromacs.org/c/gromacs/+/15881 to help identify conventions that should also be documented (and to try to reduce inheritance complexity).

Erik Lindahl wrote:

we should stick entirely to Gitlab's default docs for the overall infrastructure

It would be great if we could just list the job parameters and such that we use, link to GitLab docs for what they mean, and document how/why we choose the values we use. I don't think that the way the current jobs are composed fits such a documentation approach, though. At a high level, the main thing missing is a set of guidelines for composing the jobs.

Also available in: Atom PDF