How to do transfer learning with multi-channel EEG data with Googlenet?

Hi Sir/Madam,

Thanks for your time. I am trying to do some deep learning classification on multi-channel EEG data with googlenet. The dimension of my data is 1000(time series points)*60(channels)*3038(samples). By doing the Continous wavelet transformation, each 1000 points can be transformed to be a time-frequency scalogram and the size I set is 227*227*1(greyscale images). Now I have the data dimension as 227(image height)*227(image width)*60(channels)*3038(samples) and I would like to fit it into google net for transfer learning. Two problems occurs now and I need some help. First, how to define the training set, if I create a matrix with dimension [227,227,60,3038], It will have an 'out of memory' error. Do we have another way to define the training set? Noted that for each sample, I have 60 images for input, since the 60 images are time correlated. Second, for the google net, the required input is [224,224,3], do I need to create some methods to make the 60 [224,224,1] images into one [224,224,3] image for transfer learning?

Great thanks for your help.

Best regards,

Manlin Wang

Answer 1

[Srivardhan Gadila]

You can copy the layerGraph of the pretrained network and change the imageInputLayer, the first convolutionLayer to match the input image channel dimension & convolution filter dimensions. Then you can freeze/unfreeze the existing pretrained weights during training the new network accordingly.

You can do something like below:(N=60)

net = googlenet();
analyzeNetwork(net)
lgraph = layerGraph(net);

N = 60;

%%
newlgraph = replaceLayer(lgraph,'data',imageInputLayer([224 224 N],'Name','input'));
% or you can use the following as well
% newlgraph = replaceLayer(lgraph,'data',imageInputLayer([227 227 N],'Name','input'));
newlgraph = replaceLayer(newlgraph,'conv1-7x7_s2',convolution2dLayer(7,64,'stride',[2 2],'padding',[3 3 3 3],'Name','conv1-7x7_s2'));
analyzeNetwork(newlgraph)

You can refer to Get Started with Transfer Learning & Deep Learning Toolbox.

You can refer to Datastores for Deep Learning for loading the training data required to train the network.

[Alok]

Thanks for providing this useful suggestion.

Yes, this works for googlenet and resnet. However, it gives error with EfficientNetB0. Could you provide some insights as to how to ironout EfficientNetB0 for multi-channel (N>3)