Use ROIAlign Layer in a FastRCNN network?

Hello,

I have defined a Fast RCNN network which I have trained with image data from ~9000 images of 1 class + background, split into train/validate/test. The network completed training with >95% validated accuracy, but I found that the bounding box/ROI was not very accurate.

I have discovered that MATLAB includes the ROIAlign layer which can supposedly produce bounding boxes with much higher positional accuracy and I have attempted to include this in my network with no success.

It seems that the output of the roi-input is not compatible with the input of the ROIAlign layer.

I have posted the code to generate my network:

params = load("params_2023_05_17__16_25_31.mat");

lgraph = layerGraph();

tempLayers = [

imageInputLayer([576 720 3],"Name","imageinput","Normalization","none")

convolution2dLayer([5 5],64,"Name","conv_6","Padding","same")

reluLayer("Name","relu_2")

batchNormalizationLayer("Name","batchnorm_3")

maxPooling2dLayer([3 3],"Name","maxpool_1","Padding","same","Stride",[2 2])

convolution2dLayer([5 5],128,"Name","conv_2","Padding","same")

reluLayer("Name","relu")

batchNormalizationLayer("Name","batchnorm_2")

maxPooling2dLayer([3 3],"Name","maxpool_3","Padding","same","Stride",[2 2])

convolution2dLayer([5 5],256,"Name","conv_1","Padding","same")

reluLayer("Name","relu_3")

batchNormalizationLayer("Name","batchnorm_1")

maxPooling2dLayer([3 3],"Name","maxpool_2","Padding","same","Stride",[2 2])

convolution2dLayer([5 5],512,"Name","conv_3","Padding","same")

reluLayer("Name","relu_1")];

lgraph = addLayers(lgraph,tempLayers);

tempLayers = roiInputLayer("Name","roi-input");

lgraph = addLayers(lgraph,tempLayers);

% ROIAlign would go here:

tempLayers = [

roiMaxPooling2dLayer([16 16],"Name","roi-max-pooling")

dropoutLayer(0.5,"Name","dropout")];

lgraph = addLayers(lgraph,tempLayers);

tempLayers = [

fullyConnectedLayer(4,"Name","fc_2")

rcnnBoxRegressionLayer("Name","rcnn-regression")];

lgraph = addLayers(lgraph,tempLayers);

tempLayers = [

fullyConnectedLayer(2,"Name","fc_1")

softmaxLayer("Name","softmax")

classificationLayer("Name","classoutput")];

lgraph = addLayers(lgraph,tempLayers);

% clean up helper variable

clear tempLayers;

lgraph = connectLayers(lgraph,"roi-input","roi-max-pooling/roi");

lgraph = connectLayers(lgraph,"relu_1","roi-max-pooling/in");

lgraph = connectLayers(lgraph,"dropout","fc_2");

lgraph = connectLayers(lgraph,"dropout","fc_1");

plot(lgraph);

The documentation for ROIAlignLayer simply states that it requires an input feature map HWCN, and mine is 72,90,512,1, and a list of ROIs to pool, and that logically seems to mean the ROI input layer in my fastRCNN network, or perhaps the output of a region proposal network? I have actually tried both of these with no success.

The error I receive when I analyse the network is:

The RPN output was a slightly different size of 1(S) x 5(C) x 1(B), but it also failed.

Any help would be appreciated

Thanks,

[Siraj]

Is it possible for you to share the param(params_2023_05_17__16_25_31.mat) file.

[Tian,HCong]

Hello, has your problem been resolved? I also encountered a similar problem when applying roialignLayer to FastRCNN network.