Creating and Using Coverage Filters
This example shows how to use Simulink® Coverage™ model coverage filters to exclude model elements from coverage analysis and justify missing coverage in reports.
During the verification process, a model can contain several constructs that prevent full model coverage, such as a subsystem that contains a driver for a controller that is not tested and is not relevant to the validation process. You can exclude this subsystem from the coverage results.
Alternatively, you may have testing criteria that requires exercising certain aspects of a block, such as hitting particular decision points. If it is not feasible to satisfy all coverage outcomes for this block, and you did not intend for your tests to exercise these unsatisfied outcomes, then you can justify the missing coverage.
Filtering these constructs in coverage results by excluding or justifying them allows you to focus on other aspects of missing coverage that can and should be tested.
Coverage filters are stored in CVF files. Each filter consists of rules that exclude or justify certain model objects or individual coverage objective outcomes. You can apply multiple filter files to coverage results for a model. Multiple models can also use the same filter file.
You can create and apply coverage filters either before or after simulating a model.
This example uses the
slvnvdemo_covfilt model, which includes examples of common patterns filtered from coverage results.
Specify Items to Exclude from Coverage Results Before Simulation
The library block
slvnvdemo_covfilt_lib/protected division protects against division by zero. If you determine that your testing is not expected or intended to fully cover every instance of this block in this context, you can exclude this block from the coverage results.
In the Simulink Editor, right-click the
protected division library block and click Coverage > Exclude referenced library: slvnvdemo_covfilt_lib/protected division to filter all references to the library.
This opens the Filter Editor pane of the Coverage Results Explorer. Note that the Coverage Results Explorer created a new filter file, initially named
Untitled, and added a filter rule that excludes all references to the library block.
Specify a name and description for the new filter file by using the Name and Description fields. In the table, double-click the Rationale field for the new rule and enter text describing why this block is excluded, such as
division by zero protection. Click Apply to save the filter file. A file dialog prompts you to specify where to save this file.
Reuse Existing Filter File
If you have models that contains similar constructs, you can use a filter file in multiple models.
For example, the filter file
Filter_Tick.cvf excludes the Stateflow temporal event
tick from coverage results. This event can never be false and, therefore, could prevent full condition and MCDC coverage in any model using
tick in event-based temporal logic in Stateflow.
slvnvdemo_covfilt/Mode Logic contains this construct, you can apply the filter file
Filter_Tick.cvf to the model.
To apply this existing filter file, right-click the Applied filters node in the Coverage Results Explorer and select Load filter. In the file dialog, select
Filter_Tick.cvf and click Open.
Note that Applied filters now lists both
Simulate and Review Filtered Coverage Results
Click the Run (Coverage) button to simulate the model and record coverage. When the simulation completes, Simulink Coverage highlights the model with the coverage results and the Coverage Details pane opens.
Both references to the
protected division library block are dimmed in the Simulink canvas, which indicates that Simulink Coverage does not analyze them due to the exclusion rule.
In the Coverage Details pane, the Objects Filtered from Coverage Analysis section lists each of the excluded elements and the corresponding rationales for each. Both
Filter_Tick appear here.
Create a New Filter File
Create another filter file to capture filter rules exclusively relevant to this model.
In the Coverage Results Explorer, right-click the Applied filters node and select New filter.
Enter a name and description for this filter file. Click Apply and specify where to save the file.
Exclude Items from Coverage Results After Simulation
You can also create and apply filter rules to coverage results after simulation. This allows you to review coverage results, create or adjust filters, and generate a new coverage report without having to rerun the simulation.
For example, consider the
Switchable config subsystem, which models a common design pattern that uses constant values to drive subsystem enable ports to change model configurations. However, the enable logic and subsystem contents might lead to missing coverage. Because this model does not use this configuration, you can exclude it from coverage analysis.
In the Simulink Editor, click the
Switchable config subsystem. The Coverage Details pane displays the coverage details for this subsystem. Click the Justify or Exclude link.
The link creates a new filter rule and adds it to the currently selected filter file in the Filter Editor pane. In the Filter Rules section, the default setting for the Mode column is
Excluded. Enter the rationale for this rule, such as
Click Apply to save the changes to the filter file and update the coverage results. The
Switchable config subsystem is now dimmed, which indicates that the filter excluded it from the coverage results.
Justify Individual Objective Outcomes from the Coverage Results
In the Simulink Editor, click the
Saturation block and review the coverage results in the Coverage Details pane. Two decision outcomes are unsatisfied because the
Saturation block has a lower limit of
0 and an upper limit of
200. However, the input to this block is the
rate signal, which can never be less than or equal to
0. As such, the lower limit of the
Saturation block is not fully exercised, so you can justify the corresponding decision outcome.
Click the Add justification rule icon next to the false outcome for the decision
input > lower limit.
The Coverage Results Explorer adds a new filter rule to the currently selected filter file in the Filter Editor pane. Specify a justification rationale, such as
rate > 0.
Click Apply to save the filter file and update the coverage results.
Note that, in the Coverage Details pane, the justified outcome of the
Saturation block is light blue and links to the rationale. Because you did not filter the true outcome of the decision
input >= upper limit and the analysis was insufficient to exercise this outcome, the
Saturation block has missing coverage and stays red.
Automatically Generate Filter Rules for Dead Logic
In some cases, missing coverage is due to dead logic and the associated coverage objectives are unsatisfiable. If this logic is meant for elements that you do not wish to remove from your model, then you can justify these missing coverage outcomes.
If you have a Simulink Design Verifier™ license, you can automatically create justification filter rules for dead logic.
In the Coverage Results Explorer, select the Applied filters node. In the Filter Editor pane, select Make justification filter rules for dead logic (using Simulink Design Verifier).
This option uses Simulink Design Verifier™ to analyze the model for dead logic. Simulink Design Verifier creates a new filter and adds justification rules for each of the corresponding coverage outcomes.
Simulink Design Verifier detects and justifies dead logic for two condition outcomes, seven decision outcomes, and one MCDC outcome.
Specify a name and description for this filter file. Click Apply when finished. In the file dialog, specify where to save this filter file.
Close the Simulink Design Verifier Results windows.
Review Filtered Coverage Results
In the Simulink Editor, in the Coverage tab, in the Review Results section, select Coverage Highlighting.
After applying the four filters in this example, the simulation now achieves 100% condition, 63% decision, and 75% execution coverage for this model. The coverage results no longer contain model objects that receive MCDC, so the coverage report does not list this metric. The coverage report does not display the MCDC outcome that the dead logic analysis filters out because the coverage filter
Filter_Tick contains an exclusion rule that eliminates that outcome from the report.
The remaining missing coverage in the
Mode Logic chart,
time capture subsystem, and
Saturation block indicates inadequate testing. You can address unresolved coverage of this type by extending your testing to more thoroughly exercise these model elements.