2D Convolution on sequential input

Hi all!

I have the following problem:

I have a dataset of 1000 matrices which have the following form: 8x10 (8 Sensor Values at 10 Time Steps)

My response dataset is a categorical vector of 1000 values (0,1 or 2) classifying each of the 1000 time series.

Using a CNN I want to make a 2D convolution so I get 50 Feature maps in the form of 1 by 10 (1 represents convoluted sensors, and 10 a value for each time step. Afterwards I want to add a LSTM layer to get information about the time domain of the signal.

However I can only use a 1D CNN when I use the sequence input layer.

And I dont want to transform the matrix into an image to apply a 2D conv.

Here is my sample code:

Input is the time series of length 10 with 8 features

layers = [ ...

sequenceInputLayer(8,"MinLength",10)

convolution2dLayer([8,1],50,"Padding","same")

reluLayer

layerNormalizationLayer

lstmLayer(50,"OutputMode","last")

flattenLayer

fullyConnectedLayer(numClasses)

softmaxLayer

classificationLayer("Classes",classes,"ClassWeights",classweights)

];

And the error I get:

Error using trainNetwork (line 184)

Invalid network.

Caused by:

Layer 2: Input size mismatch. Size of input to this layer is different from the expected input size.

Inputs to this layer:

from layer 1 (size 8(C) × 1(B) × 10(T))

Is there a way to solve this problem? In Python it is no problem to use 2D Conv on sequential input data.

[Yu]

Hello, Patrick.

How did you solve this problem?

[Sotudeh Hoseini Ghafari]

I have exactly the same problem. How did you deal with this issue @Patrick Wais?

Answer 1

[yanqi liu]

yes，sir，in 2021b，can use convolution1dLayer，such as

https://ww2.mathworks.cn/matlabcentral/answers/1651895-how-to-train-1d-cnn-on-custom-dataset-in-matrix-form-in-matlab

Answer 2

[Md Zahidul Islam]

I have faced similar problem recently.

For using 2D Conv you need input sequence in the form of (size (S) × (S) × (C)), But the size has given in the model is: sequenceInputLayer(8,"MinLength",10), which means size 8(C) × (B) × (T). Thus, you are getting an error.

Note that, sequenceInputLayer(8) means the input has (8 features * T time steps). Matlab read T time steps from the input data during training.

Therefore, If want to use 2D Conv with time series, one may try as below. Otherwise, try 1D Conv as @yanqi liu attached.

 layers = [ ...
        sequenceInputLayer([8 10 1],"MinLength",10)
        sequenceFoldingLayer('Name','fold')
        convolution2dLayer([3,3],50,"Padding","same")
        reluLayer
        layerNormalizationLayer
        sequenceUnfoldingLayer('Name','unfold')
        flattenLayer
        lstmLayer(50,"OutputMode","last")
        fullyConnectedLayer(numClasses)
        softmaxLayer
        classificationLayer("Classes",classes,"ClassWeights",classweights)
        ];
 layers = layerGraph(layers);
 layers= connectLayers(layers,'fold/miniBatchSize','unfold/miniBatchSize');

[Xie Shipley]

@Md Zahidul Islam have you tried sequenceInputLayer([8 10 1],"MinLength",10) on GPU ?

I got CUDNN_STATUS_EXECUTION_FAILED while running on GPU, you can click Here to see more detail