Testing a Lane-Following Controller with Simulink Test
Perform requirements-based testing for an automotive lane-following system.
In this example, you will:
- Author high level test requirements for a closed loop model implementing a lane-following algorithm
- Author tests in Simulink Test™ to verify safe operation for each requirement
- Execute tests and review verification status
This example shows how to verify an automated lane-keep assist algorithm using the test manager and model verification blocks. This example is based closely on a shipping Model Predictive Control Toolbox™ example titled Lane Following Control with Sensor Fusion and Lane Detection. For details on the control algorithm and closed-loop system model, see docid:mpc_ug#mw_a8d9048b-3649-4d29-9da1-58a53299410e.
Set Up the Example Files and Open the Simulink Project
Run the following commands to create and open a working copy of the project files. The project organizes files into several folders. The controller and system model files are in the Models folder. The high level requirements for the controller are captured in LaneFollowingTestRequirements.slreqx within the Requirements folder. The test manager test file is in the Tests folder.xs
[projectFolder,~]=Simulink.ModelManagement.Project.projectDemoSetUp(fullfile(matlabroot,'toolbox','simulinktest','simulinktestdemos','sltestLaneFollowing.zip'),,true,'gendemo'); proj = simulinkproject(projectFolder);
Warning: Directory already exists. Initializing: Project Path Setting up working folders Identifying shadowed project files
Open the System Model
Open the LaneFollowingTestBenchExample model. The lane following controller is implemented by the reference block Lane Following Controller. The vehicle dynamics and driving environment is modeled within the Vehicle and Environment subsystem. The road, lane and traffic scenarios are modeled using synthetic data generated by the Automated Driving System Toolbox™ and saved to scenario files within the Data folder. The scenario files are read into the simulation by the Scenario Reader block within the Vehicle and Environment subsystem.
mdl = 'LaneFollowingTestBenchExample'; open_system(mdl);
Defining Driving Scenario Requirements
A total of nine driving scenarios are available for use. The Requirements Editor app in Simuink Requirements™ is used to author high level requirements to represent these scenarios. Open the Requirements Editor using Analysis > Requirements > Requirements Editor and chose the LaneFollowingTestRequirements.slreqx Requirements Set file. Alternately, run the command:
There are nine requirements, each representing a driving scenario. The first four requirements test tracking ability of the control algorithm and the next five focus on lane following ability under various road conditions:
- Scenario 1: ACC_ISO_TargetDiscriminationTest. This is a basic test to ensure the controller can track a lead car in the travel lane.
- Scenario 2: ACC_ISO_AutoRetargetTest. Test if the controller can re-target to a new car in the travel lane when the current target switches lanes.
- Scenario 3: ACC_ISO_CurveTest. Test if the controller can track a slowing car in the travel lane while navigating the curvature of the road.
- Scenario 4: ACC_StopnGo. This test simulates stop and go movement in the travel lane due to heavy traffic
- Scenario 5: LFACC_DoubleCurveDecelTarget. Track the decelerating lead car through two S curves.
- Scenario 6: LFACC_DoubleCurve_AutoRetarget. Test ability to retarget to a new lead car on a curve.
- Scenario 7: LFACC_DoubleCurveStopnGo. This test simulates stop and go movement on a curved highway.
- Scenario 8: LFACC_Curve_CutInOut. This test ensures the controller can identify a car cutting into and out of the travel lane
- Scenario 9: LFACC_Curve_CutInOut_TooClose. This test repeats the previous test with shorter seperation distance between the ego and lead car.
Defining Assessment Criteria
There are three main assessment criteria used to verify satisfactory operation of the controller:
- Collision avoidance: Ensure that the ego car does not collide with the lead car at any point during the driving scenario
- Safe distance: Ensure that the time gap between the ego car and the lead car is above 0.8s. The time gap between the two cars is defined as the ratio of the calculated headway and the ego car velocity.
- Lane following: Ensure that the lateral deviation from the centerline of the lane is within 0.2m
The first two criteria are verified by calculated by the LaneFollowingTestBenchExample/Collision Detection/Test Assessments Test Sequence block. A simple verify statement checks if a collision is ever detected at any point during the simulation.
A second verify statement, is used to ensure that the calculated time gap between the two cars does not fall below 0.8s. Notice the use of the duration operator to allow for transient failures due to sudden changes in road conditions, sensor inputs etc. The duration operator allows failures with this assessment for upto 5s at a time.
The lane following assessment is modeled using a Test Sequence block named LFRefMdl/Test Assessments within the controller model.
A verify statement, in combination with the duration operator, is used to ensure that the absolute value of the lateral deviation from the centerline of the travel lane does not exceed 0.2m for more than 5s at a time.
Running an Interactive Simulation
To configure an interactive simulation, follow these steps:
- Select a driving scenario by setting the scenarioId variable to a value between 1-9 in the MATLAB base workspace.
- Run the helperLFSetUp script to load required types and data.
- Open the Bird's-Eye Scope using the visualization drop-down in the Simulink® model toolbar and set it up to observe the simulation by clicking Find signals. See docid:driving_ref#mw_59742eb7-dce8-4938-9c2e-44d34c7b8891 for details on using the scope.
- Simulate the model to visualize the chosen driving scenario
- Run the script plotLFResults to assess controller performance script
Alternately, execute these commands to run the simulation and plot results for the first scenario:
scenarioId = 1; helperLFSetUp; sim(mdl); plotLFResults(logsout); Simulink.sdi.view;
The results of the verify statements within the Test Sequence blocks are automatically streamed to the Simulink Data Inspector. Open the data inspector to review the results of the assessments.
The controller performance can be assessed using the MATLAB plot figures created by the plotLFResults command above. There are two plots created - one for assessing spacing performance and another for assessing lateral performance. For details on how to evaluate these plots, see docid:mpc_ug#mw_a8d9048b-3649-4d29-9da1-58a53299410e.
Systematic Testing with the Test Manager
Open the LaneFollowingTestScenarios.mldatx test file in the test manager. The test file has a set of nine test cases, one for each of the nine test scenarios described above.
Warning: File 'LaneFollowingTestScenarios.mldatx' was created with a newer version (R2019a) of Simulink Test. Some sections might be incomplete.
Each test case uses the Post-Load callback to set the appropriate scenario ID and run the setup utility. In addition, each test case also links to the corresponding requirement in the Requirements Editor for traceability.
Run the tests by clicking the Play button or use the following command: sltest.testmanager.run
The results section of the test manager shows aggregated results for all nine test cases. For each test case, the verify statements are aggregated to compute overall Pass/Fail results.
The controller assessment figures are automatically captured as part of test case execution and are available for review in the test result summary pane.
Open the Requirements Editor and choose Display > Verification Status to see a roll-up sumary of verification status for each of the requirments. The results of test execution are shown the editor as green and red bars and roll-up automatically to a summary result for the Requirements Set file.
This example showed how to use Simulink Requirements and Simulink Test to automate verification of a automotive lane following controller.
Close all open windows and models
close_system(mdl, 0); clear mdl; sltest.testmanager.clear; sltest.testmanager.clearResults; close(proj); sltest.testmanager.close; Simulink.sdi.close;
Closing Project Models Clearing: Project Path Closing working folders