combine 2 neural network with different inputs

I have 2 types of input that I need to classify together. Image and numerical information.

I need to transfer the image through CNN for example Squeeze Net and then enter the numeric information (array) and make a classification on everything.

I can not train one network for the images and then connect it to the numeric information because I have no information about the classification of the images, I only have information about the images and the numerical information together.

I tried to use deep network designer, to do something like this:

But there is an error that says that these are 2 different types of data ...

How can I make a network that receives two different types of inputs?

Answer 1

[Srivardhan Gadila]

I think as of R2021a, it is not possible to have different input layers for multiple inputs. But from R2021b I think it is possible. But again the output of featureInputLayer would be a 'CB' data whereas the output of pooling layer would be 'SSCB'.

So you will have to insert a custom layer which either removes the 'SS' dimension from the ouptut of pooling layer or expand the output of featureInputLayer to 'SSCB' (Refer to this page: dimension labels). Alternatively you can use the imageInputLayer itself instead of using featureInputLayer and resize the numerical training data accordingly by adding extra dimensions (like 1x1xnumFeatures).

Other way to work on this is to use the dlnetwork and custom training loops based workflow where you don't need any input layers.

In addition to the hyperlinks, refer to the following pages for more information: Multiple-Input and Multiple-Output Networks, Train Network Using Custom Training Loop.