Project

General

Profile

Task #2178

Task #1587: improve the configurability of regression tests

Move checks for specific warnings to source repo

Added by Mark Abraham over 2 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
testing
Target version:
-
Difficulty:
uncategorized
Close

Description

Considering https://redmine.gromacs.org/issues/2141 suggested https://gerrit.gromacs.org/#/c/6613 and https://gerrit.gromacs.org/#/c/6609 which are fine on their own, but that procedure then forces every commit in master branch to rebase in order to pass Jenkins. This has happened before, and I have refrained from changing warning text because we have to do all this extra work. That's not good.

There are only a few kinds of distinct warnings (try ls -l $(find * -name \*.warn) | sort -k 5 | less in regressiontests) so it will be fairly easy to port them to the GoogleTest framework:
  • stop regressiontests generating warnings (e.g. for center of mass removal) that aren't part of the intent of the test
  • refactor warnings in grompp to do something like the new logging framework at Warning level
  • use callGrompp on suitable inputs (e.g. including that a default mdp file does not trigger a warning)
  • assert on the number of warnings, via a test-specific implementation of a logger
  • perhaps assert on the text of the warning (if we think we need to)

To arrange to send warnings to the memory buffer, I suggest we

History

#1 Updated by Mark Abraham over 2 years ago

  • Description updated (diff)

#2 Updated by Mark Abraham over 2 years ago

  • Parent task set to #1587

#3 Updated by Szilárd Páll over 2 years ago

Mark Abraham wrote:

To arrange to send warnings to the memory buffer, I suggest we

Some thought missing?

Could we export warnings/errors (using some tags?) to automate generating a list of all warning/fatal msg?

Also available in: Atom PDF