LiDAR - trainPoint​PillarsObj​ectDetecto​r() Shape error

Hello currently I am working on a LiDAR-Project.

I already set up the PCD-Files with the viewer app

• cropping
• denoising
• downsampling

And I already used the lidar labeler app.

I am working with datastores. And I follow the instructions of this tutorial series:

Deep Learning with Point Clouds | Deep Learning for 3D Object Detection, Part 3

https://www.youtube.com/watch?v=pwoUySMYUlM

But I use a different Dataset for my own project.

But currently I receive an error within the training step.

1. Train DeepNetwork (PointPillarNetwork)

anchorBoxes = calculateAnchorsPointPillars(gTruth.LabelData);
classNames = ["ParkingCar", "Fence", "ApprochingCar","Cyclist","Pedestrian","RowOfHouses"];

xMin = -14; %-13.45
xMax = 14; %13.09
yMin = 6; % 5.14
yMax = 44; %43.38
zMin = -2; %-1.75
zMax = 11; %10.75
pcRange = [xMin, xMax, yMin, yMax, zMin, zMax];

% Definieren Sie den Bereich der Punktwolke
PointCloudRange = [xMin, xMax; yMin, yMax; zMin, zMax];

% Definieren Sie die Voxelgröße
VoxelSize = [0.01, 0.01, 0.01]; % Beispielwerte in Metern

% Überprüfen Sie, ob die Voxelgröße gültig ist
if mod(round((PointCloudRange(1,2) - PointCloudRange(1,1)) / VoxelSize(1)), 8) ~= 0 || ...
   mod(round((PointCloudRange(2,2) - PointCloudRange(2,1)) / VoxelSize(2)), 8) ~= 0
    error('Die Voxelgröße muss so definiert werden, dass die Dimensionen durch 8 teilbar sind.');
end

% Erstellen Sie den Punkt-Pillars-Objekterkenner

pretrainedDetector = load("pretrainedPointPillarsDetector.mat","detector");
oldDetector = pretrainedDetector.detector;

detector = pointPillarsObjectDetector(oldDetector.Network,pcRange,classNames,anchorBoxes,VoxelSize=[0.01 0.01]);

2. Specificy training options

options = trainingOptions('adam',...
    'MaxEpochs',1,...
    'MiniBatchSize',3,...
    'GradientDecayFactor',0.9,...
    'SquaredGradientDecayFactor',0.999,...
    'LearnRateSchedule',"piecewise",...
    'InitialLearnRate',0.0002,...
    'LearnRateDropPeriod',15,... 
    'LearnRateDropFactor',0.8,...
    'BatchNormalizationStatistics','moving',...
    'ResetInputNormalization',false);

3. Train the Object detector

[detector,info] = trainPointPillarsObjectDetector(dsOversampled,detector,options);

Error-Messag

Error using dlnetwork/forward (line 667)

Execution failed during layer(s) "pillars|scatter_nd".

Error in pointPillarsObjectDetector/forward (line 145)

[varargout{1:nargout}] = forward(net,dlX{:},'Outputs',net.OutputNames);

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Error in images.dltrain.internal.MetricLogger/initializeMetrics (line 134)

[outputs{:}] = forward(network,inputs{:});

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Error in images.dltrain.internal.MetricLogger (line 99)

initializeMetrics(self,network,trainingQueue);

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Error in images.dltrain.internal.dltrain (line 55)

logger = MetricLogger(metrics,valQueue,lossMetricName,objectiveMetricName,net,queue);

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Error in trainPointPillarsObjectDetector (line 107)

[detector,info] = images.dltrain.internal.dltrain(mbq,detector,options,lossFcn,metrics,validationPatienceMetric,'ExperimentMonitor',params.ExperimentMonitor);

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Caused by:

Error using reshape

Number of elements must not change. Use [] as one of the size inputs to automatically calculate the appropriate size for that dimension.

Error in dlarray/reshape (line 40)

objdata = reshape(objdata, varargin{:});

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Error in lidar.internal.cnn.scatterLayerFcnPointPillars (line 20)

Z = reshape(a_r, [shape C N]);

^^^^^^^^^^^^^^^^^^^^^^^^^

Error in lidar.internal.pointPillarsNetworkCreation>@(X1,X2)lidar.internal.cnn.scatterLayerFcnPointPillars(X1,X2,shape) (line 33)

fcnHandle = @(X1,X2) lidar.internal.cnn.scatterLayerFcnPointPillars(X1,X2,shape);

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Error in nnet.cnn.layer.FunctionLayer/predict (line 51)

[varargout{1:layer.NumOutputs}] = layer.PredictFcn(varargin{:});

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Error in nnet.layer.Layer/forward (line 129)

[varargout{1:layer.NumOutputs+layer.PrivateNumStates}] = predict( layer, varargin{:} );

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

here is the information about my Datastore:

dsOversampled

dsOversampled =

TransformedDatastore with properties:

UnderlyingDatastores: {matlab.io.datastore.CombinedDatastore}

SupportedOutputFormats: ["txt" "csv" "dat" "asc" "xlsx" "xls" "parquet" "parq" "png" "jpg" "jpeg" "tif" "tiff" "wav" "flac" "ogg" "opus" "mp3" "mp4" "m4a"]

Transforms: {[@(x)pcBboxOversample(x,dsSampled,classNames,totalObjects)]}

IncludeInfo: 0

Thank you for your help!

Answer 1

[sneha]

Hello,

You’re encountering a "reshape" error during the training of your custom PointPillars network in MATLAB, specifically in the scatter_nd layer. The mismatch might come from:

1. Incorrect VoxelSize or pcRange not consistent with the pretrained network.
2. Your datastore preprocessing (oversampling, or coordinate frame difference) changed the size of the feature tensor.
3. Using a pretrained network (oldDetector.Network) that expects a specific pillar grid (e.g., 432×496 grid for KITTI dataset) but feeding point clouds cropped differently.

The point cloud range and voxel size must match the input dimensions expected by the PointPillars backbone network.

Refer to

https://www.mathworks.com/help/lidar/ref/trainpointpillarsobjectdetector.html - look at the Input Arguments section (how voxel size and range affect training).

https://www.mathworks.com/help/lidar/ug/train-pointpillars-object-detector.html -check Step 2: Specify the Point Cloud Range and Voxel Size.

Thanks