fuseCameraToLidar

Fuse image information to lidar point cloud

Syntax

ptCloudOut = fuseCameraToLidar(I,ptCloudIn,intrinsics)

ptCloudOut = fuseCameraToLidar(I,ptCloudIn,intrinsics,tform)

ptCloudOut = fuseCameraToLidar(___,nonoverlapcolor)

[ptCloudOut,colormap] = fuseCameraToLidar(___)

[ptCloudOut,colormap,indices] = fuseCameraToLidar(___)

Description

ptCloudOut = fuseCameraToLidar(I,ptCloudIn,intrinsics) fuses information from an image, I, to a specified point cloud, ptCloudIn, using the camera intrinsic parameters, intrinsics.

The function crops the fused point cloud, ptCloudOut, so that it contains only the points present in the field of view of the camera.

ptCloudOut = fuseCameraToLidar(I,ptCloudIn,intrinsics,tform) uses the camera to lidar rigid transformation tform to bring the point cloud into image frame before fusing it to the image information. Use this syntax when the point cloud data is not in the camera coordinate frame.

ptCloudOut = fuseCameraToLidar(___,nonoverlapcolor) returns a fused point cloud of the same size as the input point cloud. The function uses the specified color nonoverlapcolor for points that are outside the field of view of the camera in addition to any combination of input arguments from previous syntaxes.

[ptCloudOut,colormap] = fuseCameraToLidar(___) returns the colors of the points colormap of the fused point cloud.

[ptCloudOut,colormap,indices] = fuseCameraToLidar(___) returns linear indices of the points in the fused point cloud that are in the field of view of the camera in addition to output arguments from previous syntaxes.

Examples

Fuse Color Information from Camera to Lidar

Load a MAT file containing ground truth data into the workspace. Extract the image and point cloud from data.

dataPath = fullfile(toolboxdir('lidar'),'lidardata','lcc','sampleColoredPtCloud.mat');
gt = load(dataPath);
im = gt.im;
ptCloud = gt.ptCloud;

Plot the extracted point cloud.

pcshow(ptCloud)
title('Original Point Cloud')

Extract the lidar to camera transformation matrix and camera intrinsic parameters from the ground truth data.

intrinsics = gt.camParams;
camToLidar = gt.tform;

Fuse the image to the point cloud.

ptCloudOut = fuseCameraToLidar(im,ptCloud,intrinsics,camToLidar);

Visualize the fused point cloud.

pcshow(ptCloudOut)
title('Colored Point Cloud')

Input Arguments

I — Color or grayscale image

H-by-W-by-C array

Color or grayscale image, specified as an H-by-W-by-C array.

• H — This specifies the height of the image.

• W — This specifies the width of the image.

• C — This specifies the number of color channels in the image. The function supports up to three color channels in an image.

Data Types: single | double | int16 | uint8 | uint16

ptCloudIn — Point cloud

pointCloud object

Point cloud, specified as a pointCloud object.

intrinsics — Camera intrinsic parameters

cameraIntrinsics object

Camera intrinsic parameters, specified as a cameraIntrinsics object.

tform — Camera to lidar rigid transformation

rigidtform3d object

Camera to lidar rigid transformation, specified as a rigidtform3d object.

nonoverlapcolor — Color specification for points outside camera field of view

color name | short color name | RGB Triplet

Color specification for points outside the camera field of view, specified as a color name, short color name, or RGB triplet.

For a custom color, specify an RGB triplet. An RGB triplet is a three-element row vector whose elements specify the intensities of the red, green, and blue components of the color. The intensities must be in the range [0,1]; for example, [0.4 0.6 0.7]. Alternatively, you can specify some common colors by name. This table lists the named color options and the equivalent RGB triplet values.

Color Name	Color Short Name	RGB Triplet	Appearance
'red'	'r'	[1 0 0]
'green'	'g'	[0 1 0]
'blue'	'b'	[0 0 1]
'cyan'	'c'	[0 1 1]
'magenta'	'm'	[1 0 1]
'yellow'	'y'	[1 1 0]
'black'	'k'	[0 0 0]
'white'	'w'	[1 1 1]

Data Types: single | double | char

Output Arguments

ptCloudOut — Fused point cloud

pointCloud object

Fused point cloud, returned as a pointCloud object.

colormap — Point cloud color map

M-by-3 matrix of real values in the range [0, 1] | M-by-N-by-3 array of real values in the range [0, 1]

Point cloud color map, returned as one of these options:

• M-by-3 matrix — For unorganized point clouds

• M-by-N-by-3 array — For organized point clouds

Each row of the matrix or channel of the array contains the RGB triplet for the corresponding point in the point cloud. The function returns them as real values in the range [0, 1]. If you do not specify a nonoverlapcolor argument, then the color value for points outside the field of view of the camera is [0 0 0] (black).

Data Types: uint8

indices — Linear indices of fused point cloud points in camera field of view

vector of positive integers

Linear indices of the fused point cloud points in the camera field of view, returned as a vector of positive integers.

Data Types: single | double

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

Version History

Introduced in R2020b

R2022b: Supports rigidtform3d objects

You can now specify tform as a rigidtform3d object, which uses the premultiply convention. Although you can still specify tform as a rigid3d object, this object is not recommended because it uses the postmultiply convention. For more information, see Migrate Geometric Transformations to Premultiply Convention.