Error importing Keras Network

I have succesfully trained a model with TensorFlow and Keras. I have saved the trained model as a h5 file, I'm willing to make test on the model in MATLAB, by the way I'm having trouble importing the model.

First of all I tried to the function

net = importKerasLayers('optflow.h5')

It load the network without any problem, I've also checked the layout in the Deep Neural Designer app.

net = 

  LayerGraph with properties:

         Layers: [23×1 nnet.cnn.layer.Layer]
    Connections: [24×2 table]
     InputNames: {'imageinput'}
    OutputNames: {'RegressionLayer_regressionoutput'}
    

Now wen i try to import the network and the training weights with the following line of code

net = importKerasNetwork("optflow.h5")

I get the following error

Error using assembleNetwork (line 47)
Invalid network.

Error in nnet.internal.cnn.keras.importKerasNetwork (line 39)
Network         = assembleNetwork(LayersOrGraph);

Error in importKerasNetwork (line 76)
Network = nnet.internal.cnn.keras.importKerasNetwork(modelfile, varargin{:});

Caused by:
    Layer 'concat': Input size mismatch. Size of input to this layer is different from the expected input size.
    Inputs to this layer:
        from layer 'fc_1' (size 1(S) × 1(S) × 200000(C) × 1(B))
        from layer 'fc_2' (size 1(S) × 1(S) × 50000(C) × 1(B))
        from layer 'fc_3' (size 1(S) × 1(S) × 12500(C) × 1(B))
        

This is the model definition in python

## Model definition
input_layer = Input(shape=(500, 1000, 3), name='imageinput')

x = AveragePooling2D(pool_size=(20, 20), padding='same', name='avgpool2d_3')(input_layer)
x = Conv2D(filters=10, kernel_size=(3, 3), padding='same', name='conv_3')(x)
x = ReLU(name='relu_3')(x)
x = Conv2D(filters=10, kernel_size=(3, 3), padding='same', name='conv_6')(x)
x = ReLU(name='relu_6')(x)
x = Flatten(name='fc_3')(x)

y = AveragePooling2D(pool_size=(10, 10), padding='same', name='avgpool2d_2')(input_layer)
y = Conv2D(filters=10, kernel_size=(3, 3), padding='same', name='conv_2')(y)
y = ReLU(name='relu_2')(y)
y = Conv2D(filters=10, kernel_size=(3, 3), padding='same', name='conv_5')(y)
y = ReLU(name='relu_5')(y)
y = Flatten(name='fc_2')(y)

z = AveragePooling2D(pool_size=(5, 5), padding='same', name='avgpool2d_1')(input_layer)
z = Conv2D(filters=10, kernel_size=(5, 5), padding='same', name='conv_1')(z)
z = ReLU(name='relu_1')(z)
z = Conv2D(filters=10, kernel_size=(5, 5), padding='same', name='conv_4')(z)
z = ReLU(name='relu_4')(z)
z = Flatten(name='fc_1')(z)

concat = Concatenate(axis=1, name='concat')([z, y, x])
fc_4 = Dense(units=5, name='fc_4')(concat)
output_layer = Dense(units=3, activation='linear', name='regressionoutput')(fc_4)

model = Model(inputs=input_layer, outputs=output_layer)

Is there a solution?

Answer 1

[Vinayak Choyyan]

Hello Fellarrusto,

As per my understanding, you have trained a model in Python using Keras and you would like to import the trained model to MATLAB. You are trying to use the ‘importKerasNetwork()’ function while you faced this issue.

As per the documentation of ‘importKerasNetwork()’ function (Import pretrained Keras network and weights - MATLAB importKerasNetwork - MathWorks India), the ‘Concatenate’ function of TensorFlow-Keras is translated into ‘depthConcatenationLayer()’ in MATLAB. A depth concatenation layer takes inputs that have the same height and width and concatenates them along the third dimension (the channel dimension). Since you had flattened the outputs ‘x’, ‘y’, ‘z’ to get ‘fc_1’, fc_2’, ‘fc_3’ which will be of two dimensional.

Try running the following code to verify that ‘depthConcatenationLayer()’ does give a similar error as you had received and replacing it with ‘concatenationLayer()’ does resolve the concat error.

im=imageInputLayer([500 1000 3],Normalization="none");
layer1 = [im
    averagePooling2dLayer([20 20],Padding="same")
    convolution2dLayer([3 3],10,Padding="same")
    convolution2dLayer([3 3],10,Padding="same")
    flattenLayer
    ];
layer2 = [im
    averagePooling2dLayer([10 10],Padding="same")
    convolution2dLayer([3 3],10,Padding="same")
    convolution2dLayer([3 3],10,Padding="same")
    flattenLayer
    ];
layer3 = [im
    averagePooling2dLayer([5 5],Padding="same")
    convolution2dLayer([5 5],10,Padding="same")
    convolution2dLayer([5 5],10,Padding="same")
    flattenLayer
    ];
l=layerGraph;
l=addLayers(l,layer1);
l=addLayers(l,layer2);
l=addLayers(l,layer3);
%l=addLayers(l,concatenationLayer(1,3));
l=addLayers(l,depthConcatenationLayer(3));
l=connectLayers(l,'flatten','concat/in1');
l=connectLayers(l,'flatten_1','concat/in2');
l=connectLayers(l,'flatten_2','concat/in3');

plot(l);
analyzeNetwork(l);
dlnet=dlnetwork(l);
analyzeNetwork(dlnet);

As a workaround for the issue you are facing, I would suggest you could build the model in MATLAB, similar to the above code, and then load the weights into the model. You can get the trained weights of your model from Python and save it to a .mat file and then use this .mat file to load the weights into MATLAB.

Here are some resources I found online on how one can get the weights from a Keras model and how to save data to a .mat file.

• How to get weights of layers in TensorFlow - gcptutorials
• scipy.io.savemat — SciPy v1.10.0 Manual

For more details, please check out the following documentations too.

• Flatten layer - MATLAB - MathWorks India
• Concatenation layer - MATLAB - MathWorks India
• Depth concatenation layer - MATLAB - MathWorks India

I hope this workaround helps resolves the issue you are facing.