Feed data into Neural Networks file-by-file
2 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Currently for training my neural network, I need to load a huge dataset and it costs almost available RAM in the computer. That's why the training step (using GPU) is very slow. The pseudo-code is following:
Loading and pre-processing dataset (300 files)
Training Neural Network
So how can I don't have to load all at one time, and will be loaded in time the neural network is being trained, like this:
For file in Dataset
Loading and pre-processing file
Training Neural Network
or
Training Neural network
At some epoch:
Loading and Pre-processing a file in dataset
Continue to train.
Does MATLAB currently support this feature?
0 Kommentare
Antworten (1)
Joss Knight
am 1 Nov. 2019
Datastores are designed for precisely this purpose. It may be that you're after an imageDatastore processed by a transform.
0 Kommentare
Siehe auch
Kategorien
Mehr zu Image Data Workflows finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!