get result of test.verifyThat

Dear Support-team,

I am working on a custom criteria script for the test manager.

In the script, I use the command "test.verifyThat" like this:

test.verifyThat(dataPair(i).ts_data.Data, IsEqualTo(dataPair(i).ts_ref.Data, 'Within', AbsoluteTolerance(absTol) | RelativeTolerance(relTol)));

I would like to get the result as a variable (true or false) if the test passed or not, i.e. something like:

test_passed = test.verifyThat(...

Could you tell me if this is possible and how it would work?

Thank you in advance!

Regards, Hans

[Steven Lord]

How are you hoping or planning to use that information if you were able to get access to that information? There may be an alternative way to achieve your goal (like onFailure.)

[Hans Barrios]

We run several test in our pipeline. In some of them, multiple criteria for one single simulation are checked. For example, in one test we call the verifyThat-command 24 times in our custom criteria script to check several signals for the same simulation run. Since we also generate several plots, the report generated by the test manager is quite large. And it is sometimes quite laborious to find which one of the 24 verifyThat-test did not pass. Thus, we want to create an overview, e.g. a table or a matrix, where we can see directly which test passed or not.

Answer 1

[Bhargavi Maganuru]

Hi,

There is no output argument for verifyThat so you cannot assign output something like this test_passed = test.verifyThat(..)

You can run the test using runtests which returns TestResult array which has properties passed, failed, incomplete and others, these indicates whether test is passed or failed.

Hope this helps!

[Hans Barrios]

Hi,

ok, thank you for your help! I will then figure out a different way of getting the result I need. Maybe I will replicate the verifyThat command I am using in order to get the test result

BR, Hans

Answer 2

[Steven Lord]

I recommend specifying a diagnostic in your verifyThat call. See the section of the example on that page starting from the last block of code in the example:

testCase.verifyThat(1, IsEqualTo(2), ...
    'actual was supposed to be equal to expected' )

When this fails, you can see that it will display the diagnostic string. There are other types of diagnostics you can specify as well.

How are you running your tests? Do you receive a TestResults object from the runner? If so the table method of that class will show which of your test methods failed.