Main Content

Test Case Verification Status Distribution

Since R2021b

Metric ID

TestCaseVerificationStatusDistribution

Description

Use this metric to count the number of tests that do not have pass/fail criteria and the number of tests that do have pass/fail criteria.

A test has pass/fail criteria if it has at least one of the following:

  • at least one executed verify statement

  • at least one executed temporal or logical assessment

  • custom criteria that has a pass/fail status in Test Manager

  • baseline criteria which determine the pass/fail criteria of the test

This metric returns the result as a distribution of the results of the Test Case Verification Status metric.

Supported Artifacts

You can collect this metric for the Units in your project. To control what the dashboard classifies as a unit, see Categorize Models in Hierarchy as Components or Units.

Computation Details

The metric:

  • Includes only tests in the project that test the model or subsystems in the unit for which you collect metric data.

  • Does not count the pass/fail criteria of tests that run in software-in-the-loop (SIL) or processor-in-the-loop (PIL) mode. The metric shows these tests as Missing Pass/Fail Criteria.

Collection

To collect data for this metric:

  • In the Model Testing Dashboard, in the Model Test Status section, place your cursor over the Inconclusive widget.

  • Use getMetrics with the metric ID TestCaseVerificationStatusDistribution.

Collecting data for this metric loads the model file and test files and requires a Simulink® Test™ license.

Results

For this metric, instances of metric.Result return Value as a distribution structure that contains these fields:

  • BinCounts — The number of tests in each bin, returned as a vector.

  • BinEdges — The outputs of the TestCaseVerificationStatus metric, returned as a vector. The outputs represent the three test verification statuses:

    • 0 — The test is missing pass/fail criteria.

    • 1 — The test has pass/fail criteria.

    • 2 — The test was not run.

Compliance Thresholds

The default compliance thresholds for this metric are:

  • Compliant0 unit tests are missing pass/fail criteria

  • Non-Compliant1 or more unit tests do not have pass/fail criteria

  • Warning — None

See Also

Related Topics