Main Content

Creating and Executing Test Cases

This example uses:

Open Script

This example shows how to use Simulink® Design Verifier™ functions to log input signals, create a harness model, generate test cases for missing coverage, merge harness models, and execute test cases.

The example starts by logging input signals to the component that implements the controller in its parent model and creating harness model for the controller from that logged data. You use Simulink Design Verifier to find a new test case that achieves the missing coverage. Then you merge the first harness model with the harness model generated after the Simulink Design Verifier analysis. Finally, you capture all test cases and execute the controller with those test cases in simulation mode and Software-In-the-Loop (SIL) mode, and compare the results using CGV API.

Check Product Availability

This example requires a valid Stateflow® license. To demonstrate test execution in Software-In-the-Loop (SIL) mode it also requires valid Simulink® Coder™ and Embedded Coder™ licenses.

if ~license('test','Stateflow')
    return;
end

canUseSIL = license('test','Real-Time_Workshop') && ...
    license('test','RTW_Embedded_Coder');

Logging Input Signals to the Component and Creating the Harness Model

The slvnvdemo_powerwindow model contains a power window controller and a low-order plant model. The component control is a Model block that references the model slvnvdemo_powerwindow_controller, which implements the controller with a Stateflow® chart.

To create a harness model for the controller with the signals that simulate the controller in the plant model, first log the input signals and then invoke harness generation with that logged data.

open_system('slvnvdemo_powerwindow');
load_system('slvnvdemo_powerwindow_controller');

loggedSignalsPlant = ...
    sldvlogsignals('slvnvdemo_powerwindow/power_window_control_system/control');

harnessModelFilePath = ...
    sldvmakeharness('slvnvdemo_powerwindow_controller',loggedSignalsPlant);
[~,harnessModel] = fileparts(harnessModelFilePath);

Measuring the Coverage with Logged Signals

Use the cvtest (Simulink Coverage) and cvsim (Simulink Coverage) functions to measure the model coverage achieved for the controller model slvnvdemo_powerwindow_controller with the logged signals that are captured in the harness model.

The cvhtml (Simulink Coverage) function produces a report that indicates that 40% Decision, 35% Condition, and 10% MCDC coverage is achieved by simulating the test cases captured from the closed-loop model.

test = cvtest(harnessModel);
test.modelRefSettings.enable = 'On';
test.modelRefSettings.excludeTopModel = 1;

covDataFromLoggedSignals = cvsim(test);
cvhtml('Coverage with Logged Test Cases',covDataFromLoggedSignals);

Finding Test Cases for Missing Coverage

Before you can use existing coverage data during test generation, the data must be saved to a coverage data file(.cvt). You can use the existing coverage data by specifying the coverage data path in the Coverage data file parameter and setting the Ignore objectives satisfied in existing coverage data parameter to on in the Test Generation pane of Simulink Design Verifier configuration parameters.

As you can see in the report, Simulink Design Verifier restricts test generation to the coverage objectives that are not covered in the existing coverage file. Notice that 8 coverage objectives in the Stateflow chart slvnvdemo_powerwindow_controller;|control| are proven to be unsatisfiable. This indicates unnecessary redundant logic that cannot be tested.

cvsave('existingCovFromLoggedSignals',covDataFromLoggedSignals);

opts = sldvoptions;
opts.IgnoreCovSatisfied = 'on';
opts.CoverageDataFile = 'existingCovFromLoggedSignals.cvt';
opts.ModelCoverageObjectives = 'MCDC';
opts.TestSuiteOptimization = 'LongTestcases';
opts.SaveHarnessModel = 'on';
opts.ModelReferenceHarness = 'on';
opts.MaxProcessTime = 500;

[status, fileNames] = sldvrun('slvnvdemo_powerwindow_controller',opts,true);
[~, newHarnessModel] = fileparts(fileNames.HarnessModel);
open_system(newHarnessModel);

Merging Test Cases from Harness Models

Now use sldvmergeharness to combine generated test cases with logged test case. The command takes a list of harness models as arguments.

sldvmergeharness(harnessModel, newHarnessModel);

Logging Test Cases of the Harness Model

In order to programmatically execute the model slvnvdemo_powerwindow_controller with the test cases captured in the merged harness model, first use the sldvlogsignals function to obtain the input values of all test cases in the necessary data format.

loggedSignalsMergedHarness = sldvlogsignals(harnessModel);
disp(loggedSignalsMergedHarness);

    LoggedTestUnitInfo: [1x1 struct]
             TestCases: [1x2 struct]

Execute the Model in Simulation Mode with CGV API

Use the sldvruncgvtest function to execute the model slvnvdemo_powerwindow_controller in simulation mode, with test cases captured from the harness model.

runopts = sldvruntestopts('cgv');
disp(runopts);

runopts.cgvConn = 'sim';
cgvSim = sldvruncgvtest('slvnvdemo_powerwindow_controller',...
    loggedSignalsMergedHarness,runopts);

     DefectChecker: 'off'
           testIdx: []
    allowCopyModel: 0
       cgvCompType: 'topmodel'
           cgvConn: 'sim'

Starting execution:
  ComponentType: topmodel
  Connectivity: sim
  InputData:
  /tmp/Bdoc24b_2725827_9201/tp2af2bd46/sldv-ex48508823/cgv_runtest/slvnvdemo_powerwindow_controller/slvnvdemo_powerwindow_controller_cgv_input_tc_1.mat
End CGV execution: status completed.
Starting execution:
  ComponentType: topmodel
  Connectivity: sim
  InputData:
  /tmp/Bdoc24b_2725827_9201/tp2af2bd46/sldv-ex48508823/cgv_runtest/slvnvdemo_powerwindow_controller/slvnvdemo_powerwindow_controller_cgv_input_tc_2.mat
End CGV execution: status completed.

Execute the Model in Software-In-the-Loop (SIL) Mode with CGV API

Now use the sldvruncgvtest function to execute the model slvnvdemo_powerwindow_controller in SIL mode, with the same test cases.

if canUseSIL
    runopts.cgvConn = 'sil';
else
    % When SIL is not possible, the example runs another simulation.
    runopts.cgvConn = 'sim';
end
cgvSil = sldvruncgvtest('slvnvdemo_powerwindow_controller',...
    loggedSignalsMergedHarness,runopts);

Starting execution:
  ComponentType: topmodel
  Connectivity: sil
  InputData:
  /tmp/Bdoc24b_2725827_9201/tp2af2bd46/sldv-ex48508823/cgv_runtest/slvnvdemo_powerwindow_controller/slvnvdemo_powerwindow_controller_cgv_input_tc_1_1.mat
### Starting build procedure for: slvnvdemo_powerwindow_controller
### Successful completion of build procedure for: slvnvdemo_powerwindow_controller

Build Summary

Top model targets:

Model                             Build Reason                                         Status                        Build Duration
===================================================================================================================================
slvnvdemo_powerwindow_controller  Information cache folder or artifacts were missing.  Code generated and compiled.  0h 0m 11.146s 

1 of 1 models built (0 models already up to date)
Build duration: 0h 0m 11.738s
### Preparing to start SIL simulation ...
Building with 'gcc'.
MEX completed successfully.
### Starting SIL simulation for component: slvnvdemo_powerwindow_controller
### Application stopped
### Stopping SIL simulation for component: slvnvdemo_powerwindow_controller
End CGV execution: status completed.
Starting execution:
  ComponentType: topmodel
  Connectivity: sil
  InputData:
  /tmp/Bdoc24b_2725827_9201/tp2af2bd46/sldv-ex48508823/cgv_runtest/slvnvdemo_powerwindow_controller/slvnvdemo_powerwindow_controller_cgv_input_tc_2_1.mat
### Starting build procedure for: slvnvdemo_powerwindow_controller
### Successful completion of build procedure for: slvnvdemo_powerwindow_controller

Build Summary

Top model targets:

Model                             Build Reason                     Status                        Build Duration
===============================================================================================================
slvnvdemo_powerwindow_controller  Generated code was out of date.  Code generated and compiled.  0h 0m 9.4938s 

1 of 1 models built (0 models already up to date)
Build duration: 0h 0m 9.9969s
### Preparing to start SIL simulation ...
Building with 'gcc'.
MEX completed successfully.
### Starting SIL simulation for component: slvnvdemo_powerwindow_controller
### Application stopped
### Stopping SIL simulation for component: slvnvdemo_powerwindow_controller
End CGV execution: status completed.

Compare Results of Simulation and SIL Modes

The sldvruncgvtest returns a cgv.CGV object after running tests. Use the CGV API to compare the results of executions in simulation and SIL modes for each test case designed in the harness model and show that they are equal.

for i=1:length(loggedSignalsMergedHarness.TestCases)
    simout = cgvSim.getOutputData(i);
    silout = cgvSil.getOutputData(i);

    [matchNames, ~, mismatchNames, ~ ] = ...
        cgv.CGV.compare(simout, silout);

    fprintf('\nTest Case(%d):  %d Signals match, %d Signals mismatch', ...
        i, length(matchNames), length(mismatchNames));
end

Test Case(1):  4 Signals match, 0 Signals mismatch
Test Case(2):  4 Signals match, 0 Signals mismatch

Clean Up

To complete the example, close all models.

close_system(harnessModel,0);
close_system(newHarnessModel,0);
close_system('slvnvdemo_powerwindow',0);
close_system('slvnvdemo_powerwindow_controller',0);