Calibration and Sensor Fusion
Most modern autonomous systems in applications such as manufacturing, transportation, and construction employ multiple sensors. Sensor fusion is the process of bringing together data from multiple sensors, such as lidar sensors and cameras. The fused data enables greater accuracy because it leverages the strengths of each sensor to overcome the limitations of the others. Lidar Toolbox™ functions support sensor fusion processes such as projecting lidar points on images, fusing color information in lidar point clouds, and projecting bounding boxes from images to point clouds and from point clouds to images.
To understand and correlate the data from individual sensors, you must develop a geometric correspondence between them. Calibration is the process of developing this correspondence. Use Lidar Toolbox functions to perform lidar-camera calibration. To get started, see What Is Lidar-Camera Calibration?
You can also interactively calibrate the sensors by using the Lidar Camera Calibrator app. For more information, see Get Started with Lidar Camera Calibrator.
Apps
| Lidar Camera Calibrator | Find rotation and translation between lidar and camera |
Functions
Topics
- What Is Lidar-Camera Calibration?
Fuse lidar and camera data.
- Get Started with Lidar Camera Calibrator
Interactively calibrate lidar and camera sensors.
- Calibration Guidelines
Guidelines to help you achieve accurate results for lidar-camera calibration.
- Coordinate Systems in Lidar Toolbox
Overview of coordinate systems in Lidar Toolbox.
- Read Lidar and Camera Data from Rosbag File
This example shows how to read and save images and point cloud data from a rosbag file.
Featured Examples
Lidar Camera Calibration and Fusion
Estimate the rigid transformation between a lidar sensor and a camera, and how to use the calibration results to fuse the point cloud data from the lidar sensor and the image data from the camera.
Multi-Lidar Calibration
Calibrate multiple 3-D lidar sensors mounted on a vehicle to estimate a relative transformation between them. Traditional methods, such as marker-based registration, are difficult when the lidar sensors have a negligible overlap between their fields of view (FOVs). The calibration also becomes more difficult as the number of lidar sensors increases. This example demonstrates the use of the trajectories of individual lidar sensors to estimate the transformation between them. This method of calibration is also known as hand-eye calibration.
Detect Vehicles in Lidar Using Image Labels
Detect vehicles in lidar using label data from a co-located camera with known lidar-to-camera calibration parameters. Use this workflow in MATLAB® to estimate 3-D oriented bounding boxes in lidar based on 2-D bounding boxes in the corresponding image. You will also see how to automatically generate ground truth as a distance for 2-D bounding boxes in a camera image using lidar data. This figure provides an overview of the process.
Track-Level Fusion of Radar and Lidar Data
Generate an object-level track list from measurements of a radar and a lidar sensor and further fuse them using a track-level fusion scheme. You process the radar measurements using an extended object tracker and the lidar measurements using a joint probabilistic data association (JPDA) tracker. You further fuse these tracks using a track-level fusion scheme. The schematic of the workflow is shown below.
