How to split matrix data into training, validation and testing data based on a categorical label?
5 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Kimberly Cardillo
am 20 Aug. 2020
Beantwortet: Srivardhan Gadila
am 23 Aug. 2020
I have a 184 x 1 cell array called aefm_4 and each cell is 1 x 1599. Each row (184) has a corresponding categorical label in a separate variable called f and its either labeled 'crack-related' or 'noise'. I want to split up my data so that I have 70% training data, 10% validation data and 20% testing data. I want about even amounts of noise and crack related signals in each training, validation and testing dataset. How would I do that?
Right now, I have been manually separating them as seen below but I will be running this for more data and it is too time consuming to do this.
dataTrain = aefm_4([1:9,14:23,28:47,55:75,87:90,93:101,108:152,169:177],:); %127 hits (70%)
fTrain = f([1:9,14:23,28:47,55:75,87:90,93:101,108:152,169:177],:);
dataVal = aefm_4([12:13,26:27,53:54,83:86,92,105:107,163:168],:); %20 hits (10%)
fVal = f([12:13,26:27,53:54,83:86,92,105:107,163:168],:);
dataTest = aefm_4([10:11,24:25,48:52,76:82,91,102:104,151:162,178:184],:); %39 hits (20%)
fTest = aefm_4([10:11,24:25,48:52,76:82,91,102:104,151:162,178:184],:);
0 Kommentare
Akzeptierte Antwort
Srivardhan Gadila
am 23 Aug. 2020
You can do something like below:
%% Create categorical label data
numLabels = 18
labels = randi([1 2],numLabels,1);
labelsCat = categorical(labels,[1 2],{'crack-related','noise'});
%% find lndices w.r.t each label
crackInd = find(labelsCat==categorical("crack-related"));
noiseInd = find(labelsCat==categorical("noise"));
%% divide into train, val, test for each label
[trainCrack,valCrack,testCrack] = dividerand(numel(crackInd),0.7,0.1,0.2);
[trainNoise,valNoise,testNoise] = dividerand(numel(noiseInd),0.7,0.1,0.2);
%% get final indices
trainInd = [crackInd(trainCrack); noiseInd(trainNoise)];
valInd = [crackInd(valCrack); noiseInd(valNoise)];
testInd = [crackInd(testCrack); noiseInd(testNoise)];
You can optimize the above code based on your requirement like, dividing the indices randomly or not. Refer to Divide Data for Optimal Neural Network Training, dividerand, divideblock, divideint, divideind for more information.
trainInd = trainInd(randperm(numel(trainInd)));
0 Kommentare
Weitere Antworten (0)
Siehe auch
Kategorien
Mehr zu Statistics and Machine Learning Toolbox finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!