Run Tests for Variant Models Using Variant Configurations

This example shows how to author and run tests for your variant model by specifying the variant configurations to use for test cases or test case iterations. You can do this from the Simulink Test Manager user interface or by using the Simulink Test™ programmatic interface.

Variant elements in Simulink®, including variant blocks and variant parameters, enable you to represent multiple design alternatives within a single model. For such a model, you can define variant configurations to represent combinations of variant choices across the model hierarchy. These configurations comprise a set of variant control variables and their corresponding values, allowing you to activate specific variants within the model hierarchy.

You can use Simulink Test Manager to create test cases for Simulink® models and code in different execution environments, run tests, view results, and create reports. With the Simulink Test programmatic interface, you can perform these operations from the command line. In both these approaches, you can specify the variant configuration to use when running a test case. You can also run the same test case for different variant configurations in the model by creating test case iterations. This workflow offers these benefits:

• Using variant configurations scales well for models with large number of variant control variables. If you do not create variant configurations for your model, you must write additional scripts to set the value of variant control variables and use those variables as input to create test iterations.

• Using variant configurations provides traceability from the test results to the variant configuration used for each test case or iteration.

Note: You must install the Variant Manager for Simulink® support package using Add-On Explorer to create variant configurations for a model.

If your model does not have variant configurations defined, refer to these examples for the testing workflow:

• Verify and Validate Variant Models with Code Compile Activation Time

• Verify and Validate Variant Models with Startup Activation Time

Explore Example Model

Open the model slexVariantManagement.

The model has multiple named variant configurations created using Variant Manager. The configurations are stored in a variant configuration data object, vcd, defined in the data dictionary associated with the model.

modelName = "slexVariantManagement";
open_system(modelName);

Specify Variant Configuration in Simulink Test Manager

Since R2024b

This example creates a baseline test from Simulink Test Manager to test the slexVariantManagement model for different variant configurations.

1. Open Simulink Test Manager for the slexVariantManagement model.

In the Simulink model window, on the Apps tab, click Simulink Test from the Model Verification, Validation, and Test section. Then click Test Manager in the Tests tab. For more information on Test Manager, see Simulink Test Manager (Simulink Test).

2. Create a new baseline test for the model.

a. In the Simulink Test Manager, create a new test file. The test file structure opens in the Test Browser pane.The default structure for the test has one test suite that has one test case.

b. In the Test Browser pane, select the test case.

c. In the right pane, in the System Under Test section, check that the Model field contains slexVariantManagement as the model to test.

d. In the Simulation Outputs section of the test case, click Add and select the signals to log from the model.

e. Set up iterations in the Iterations section of a test case if you want to run the same test case with different data.

3. Set variant configurations for the test case or test iterations.

a. In the Parameter Overrides section of a test case, the Variant Configuration list shows the available configurations from the variant configuration object associated with the model. Select the configuration to apply to the test case from this list.

b. Click the Launch Variant Manager button to view the configurations or to modify and save them to the variant configuration object from Variant Manager.

c. If you modified the variant configuration object from Variant Manager, click the Refresh button to update the list of configurations in the Simulink Test Manager.

d. If your test case uses iterations, you can select a configuration in the Variant Configuration column in the Table Iterations subsection of the Iterations section.

4. Capture the baseline data. In the Baseline Criteria section, capture logged signal data from the model to use as the baseline criteria.

5. Run the test and view the results.

a. Click Run in the Simulink Test Manager toolstrip to execute the test.

b. After a test case finishes running in the Simulink Test Manager, the test case result appears in the Results and Artifacts pane. In the Results and Artifacts pane, when you click a test case or test iteration, the variant configuration appears in the Simulation Metadata and Test Overrides sections, if applicable to the test case.

For an example that shows how to create baselines tests for a model, see Creating Baseline Tests (Simulink Test).

Specify Variant Configuration Programmatically

Since R2024a

This example creates a simulation test case programmatically for the slexVariantManagement model and assigns a variant configuration to the test case and test iteration.

To view the variant configurations in the variant configuration object vcd of the slexVariantManagement model, enter these commands in the MATLAB® command line:

vcd = Simulink.VariantManager.getConfigurationData(modelName);
cellfun(@(name)(fprintf("%s\n", name)), {vcd.Configurations(:).Name});

LinInterExpNoNoise
LinInterExpWithNoise
LinInterStd
NonLinExterLowFid
NonLinExterHighFid
SmartAIExterHighFid
LinExterHighFid

To set a variant configuration programmatically, use the VariantConfiguration property in the sltest.testmanager.TestCase (Simulink Test) and sltestiteration (Simulink Test) objects.

1. Create a new test file with default test suite and test case.

% Create a test file with default test suite
testFileName = "slexVariantManagement_TestFile";
testFile = sltest.testmanager.TestFile(testFileName);

% Get the default test suite object from the test file
testSuites = getTestSuites(testFile);

% Get the test case object
testCase = getTestCases(testSuites);

2. Modify the test case settings using the setProperty method.

% Assign the system under test to the test case
setProperty(testCase,"Model",modelName);

% Set a default variant configuration for the test case.
setProperty(testCase,"VariantConfiguration","NonLinExterLowFid");

3. Add test iterations and use the setTestParam method to set the iteration parameters.

% Set VariantConfiguration property for first test iteration
testIteration1 = sltestiteration;
testIteration1.setTestParam...
    ("VariantConfiguration","LinInterExpWithNoise");

% Add the test iteration object to the test case
addIteration(testCase,testIteration1);

% Use the default variant configuration from the 
% test case for the second test iteration
testIteration2 = sltestiteration;

% Add the iteration object to the test case
addIteration(testCase,testIteration2);

% Run the test case with iterations
testResults = run(testCase);

4. Access the test case and test iteration result data:

% Get test case results
testCaseResults = testResults.getTestCaseResults;

% Get test iteration results
testIterationResults = testCaseResults.getIterationResults;

5. Generate reports from the results:

% Generate report of test specifications and test results
sltest.testmanager.TestSpecReport(testCase, "testSpec.pdf",...
    "LaunchReport", true);
sltest.testmanager.report(testResults, "testResult.pdf",...
    "IncludeTestResults", 1);

Limitations

Specifying variant configurations in Simulink Test is not supported for multiple-release testing and real-time testing.