LSTM-CNN "The size of the convolution dimension of the padded input data must be larger than or equal to the filter size"

Hello everyone,

I am trying to implement LSTM-CNN for speech recognition, I have a matrix for train and test which already are converted it to cell, when I excuted the code I got the error below:

net = trainNetwork(AllCellTrain, YCA, layers, options);

Caused by:

Layer 3: The size of the convolution dimension of the padded input data must be larger than or equal to the filter

size. For networks with sequence input, this check depends on the MinLength property of the sequence input layer. To

ensure that this check is accurate, set MinLength to the shortest sequence length of your training data.

The code that I have used:

% Define LSTM-CNN model architecture
numHiddenUnits = 100; % Number of hidden units in the LSTM layer
numFilters = 100; % Number of filters in the CNN layer
%filterSize = [3, 3]; % Size of the filters in the CNN layer
filterSize=3;
num_features = 39;
layers = [
    sequenceInputLayer(num_features)
    lstmLayer(numHiddenUnits,'OutputMode','sequence')
    convolution1dLayer(filterSize, numFilters)
    maxPooling2dLayer(2, 'Stride', 2)
    fullyConnectedLayer(num_classes)
    softmaxLayer
    classificationLayer
];

% Specify the training options
max_epochs = 26;
mini_batch_size = 128;
initial_learning_rate = 0.001;

options = trainingOptions('adam', ...
    'MaxEpochs', max_epochs, ...
    'MiniBatchSize', mini_batch_size, ...
    'InitialLearnRate', initial_learning_rate, ...
    'GradientThreshold', 1, ...
    'Shuffle', 'every-epoch', ...
    'Verbose', 1, ...
    'ExecutionEnvironment','auto', ...
    'Plots', 'training-progress');

% Train the LSTM-CNN model

YCA = categorical(CA);

net = trainNetwork(AllCellTrain, YCA, layers, options);

Thanks in advance!

Answer 1

[Meet]

Hi Hamza,

The “convolution1dLayer” is intended for 1D sequence data, but it might alter the feature dimension or sequence length depending on the filter size and padding settings.

The pooling operation also reduces the sequence length, which can cause incompatibility with layers expecting a specific input size.

Instead, you could use this configuration as shown below:

layers = [
    sequenceInputLayer(numFeatures)
    lstmLayer(numHiddenUnits, 'OutputMode', 'last') % Changed to 'last' to output a single time step
    fullyConnectedLayer(numClasses)
    softmaxLayer
    classificationLayer
];

The above configuration works because it avoids sequence length alterations by directly using the final output of the LSTM layer.

You can refer to the resources below for more information:

“sequenceInputLayer”: https://www.mathworks.com/help/releases/R2023a/deeplearning/ref/nnet.cnn.layer.sequenceinputlayer.html?s_tid=doc_ta#mw_40b741f4-1283-494c-aad5-8e8242fc3a10

Output Mode lstmLayer: https://www.mathworks.com/help/releases/R2023a/deeplearning/ref/nnet.cnn.layer.lstmlayer.html#mw_9f7c5f93-4bf2-4ddb-b922-b1c122668b9a_sep_mw_4fd4123f-c49f-4a38-95d1-85f9e542d641