Code Generation for Lidar Object Detection Using PointPillars Deep Learning

This example uses:

This example shows how to generate CUDA® MEX for a PointPillars object detector. For more information, see Lidar 3-D Object Detection Using PointPillars Deep Learning example from the Lidar Toolbox™.

Third-Party Prerequisites

Required

CUDA enabled NVIDIA® GPU and compatible driver.

Optional

For non-MEX builds such as static and dynamic libraries or executables, this example has the following additional requirements.

NVIDIA CUDA toolkit.
NVIDIA cuDNN library.
Environment variables for the compilers and libraries. For more information, see Third-Party Hardware (GPU Coder) and Setting Up the Prerequisite Products (GPU Coder).

Verify GPU Environment

To verify that the compilers and libraries for running this example are set up correctly, use the coder.checkGpuInstall (GPU Coder) function.

envCfg = coder.gpuEnvConfig('host');
envCfg.DeepLibTarget = 'cudnn';
envCfg.DeepCodegen = 1;
envCfg.Quiet = 1;
coder.checkGpuInstall(envCfg);

Pretrained PointPillars Network

Load the pretrained pointPillarsObjectDetector trained in the Lidar 3-D Object Detection Using PointPillars Deep Learning example. To train the detector yourself, see Lidar 3-D Object Detection Using PointPillars Deep Learning.

matFile = 'pretrainedPointPillarsDetector.mat';
pretrainedDetector = load('pretrainedPointPillarsDetector.mat','detector');
detector = pretrainedDetector.detector;

`pointpillarsDetect` Entry-Point Function

The pointpillarsDetect entry-point function takes in the point cloud and confidence threshold as input and passes them to a trained pointPillarsObjectDetector for prediction through the pointpillarDetect function. The pointpillarsDetect function loads the detector object from the MAT file into a persistent variable and reuses the persistent object for subsequent prediction calls.

type('pointpillarsDetect.m')

function [bboxes,scores,labels] = pointpillarsDetect(matFile,dataLoc,dataInt,threshold)
% Predict the output of network and extract the confidence, x, y,
% width, height, and class.

% load the deep learning network for prediction
persistent pointPillarObj;

if isempty(pointPillarObj)
    pointPillarObj = coder.loadDeepLearningNetwork(matFile);
end

ptCloud = pointCloud(dataLoc,'Intensity',dataInt);

[bboxes,scores,labels] = pointPillarObj.detect(ptCloud,'Threshold',threshold);
end

Evaluate the Detector for Object Detection

Read the point cloud.

pc = pcread('pandasetDrivingData.pcd');

Use the detect method on the pretrained detector.

confidenceThreshold = 0.7;
[bboxes,~,labels] = detect(detector,pc,'Threshold',confidenceThreshold);
bboxesCar = bboxes(labels == 'Car',:);
bboxesTruck = bboxes(labels == 'Truck',:);

Display the detections on the point cloud.

helperDisplay3DBoxesOverlaidPointCloud(pc.Location,bboxesCar,'green',...
                      bboxesTruck,'magenta','Predicted bounding boxes');

Figure contains an axes object. The axes object with title Predicted bounding boxes contains an object of type scatter.

Generate CUDA MEX

To generate CUDA® code for the pointpillarsDetect entry-point function, create a GPU code configuration object for a MEX target and set the target language to C++. Use the coder.DeepLearningConfig (GPU Coder) function to create a cuDNN deep learning configuration object and assign it to the DeepLearningConfig property of the GPU code configuration object.

cfg = coder.gpuConfig('mex');
cfg.TargetLang = 'C++';
cfg.DeepLearningConfig = coder.DeepLearningConfig(TargetLibrary='cudnn');

dataLoc = pc.Location;
dataInt = pc.Intensity;

args = {coder.Constant(matFile) coder.typeof(dataLoc,[Inf,3],[1 0]) coder.typeof(dataInt,[Inf,1],[1 0]) coder.typeof(confidenceThreshold)};

codegen -config cfg pointpillarsDetect -args args -report

Code generation successful: View report

Run the Generated MEX

Call the generated CUDA MEX with the point cloud. Display the results.

[bboxes,~,labels] = pointpillarsDetect_mex(matFile,dataLoc,dataInt,confidenceThreshold);
bboxesCar = bboxes(labels == 'Car',:);
bboxesTruck = bboxes(labels == 'Truck',:);

helperDisplay3DBoxesOverlaidPointCloud(pc.Location,bboxesCar,'green',...
                      bboxesTruck,'magenta','Predicted bounding boxes');