Custom filedatastore for deep learning

I started to implement a Deep Learning network to classify the modulation of RF signals.

I use a 2D image input layer with dimensions 1x1024x2. I converted the dataset into variables which are saved in mat files. The variables have a dimension of 1x1024x2xN, where N is the number of signals which is 24*4096. The labels are stored in separate mat files and the variable is an Nx1 categorical cell array.

I have multiple mat files for the training data but I am unable to create a filedatastore which will read all of the signals along the 4th dimension from one file and then do the same for the rest of the files.

Answer 1

[Bence Cserkuti]

Thank Mahesh,

I ended up writing a custom Datastore class. In case others face the same problem, I recommend reading these pages carefully:

https://uk.mathworks.com/help/matlab/import_export/develop-custom-datastore.html

https://uk.mathworks.com/help/matlab/import_export/testing-guidelines-for-custom-datastores.html

In my case, when I initialise my custom class, I pass two arguments: the file names of my signals and the file names of the corresponding labels. I then create a filedatastore for each dataset. What was a key observation is that your custom read function must return 1 observation at a time by default (this can be increased by implementing the ReadSize property). In this case that means then if I call ds.preview it will return a row that contains a 1x1024x2 signal and the corresponding categorical array.

Answer 2

[Mahesh Taparia]

Hi

In this case, you can create a custom ReadFcn while creating a file datastore. For more information, you can refer this documentation. The custom read function will load the file and read the data as per required.Hope it will help!

[Bence Cserkuti]

Thank Mahesh,

I ended up writing a custom Datastore class. In case others face the same problem, I recommend reading these pages carefully:

https://uk.mathworks.com/help/matlab/import_export/develop-custom-datastore.html

https://uk.mathworks.com/help/matlab/import_export/testing-guidelines-for-custom-datastores.html

In my case, when I initialise my custom class, I pass two arguments: the file names of my signals and the file names of the corresponding labels. I then create a filedatastore for each dataset. What was a key observation is that your custom read function must return 1 observation at a time by default (this can be increased by implementing the ReadSize property). In this case that means then if I call ds.preview it will return a row that contains a 1x1024x2 signal and the corresponding categorical array.