not getting same accuracy as classifier app

12 views (last 30 days)
Ashwin Kannan
Ashwin Kannan on 7 May 2018
Commented: Bernhard Suhm on 12 Jun 2018
I train a classifier, say SVM, with data standardization & 50 fold cross validation in Classifier App GUI. I get an accuracy of 90%. Now I export the model, and run prediction on the same features that was used to train the model. The result is that I get very low accuracy (~40-50%). Shouldn't I be getting high accuracy since I'm testing my train set? Is it got something to do with standardizing the test data? or does 'm.predictFcn()' take care of standardizing test data?

Answers (1)

Bernhard Suhm
Bernhard Suhm on 12 Jun 2018
The model trained inside the Learner app doesn't exactly match the exported model because the latter will be trained on the complete data set, with no evaluation data held out. But that should rather increase than decrease accuracy. You do need to ensure you have applied the same "standardization" (I assume you mean something like subtracting the mean and dividing by the standard deviation) that was applied to the "training" data, the predict function doesn't "automagically" perform such normalization, and a mismatch in that would explain the low accuracy.
  1 Comment
Bernhard Suhm
Bernhard Suhm on 12 Jun 2018
I obtained some additional clarification: you must have checked "Standardize data" among the Advanced options in the Classification Learner. That does perform the standard normalization I referred to above. And the exported model does carry the mean and standardization (in m.ClassificationSVM.Mu and *.Sigma), and the predictFcn will apply that normalization to whatever table you pass into it. I just verified that yields an accuracy slightly higher than the one reported in the app on the training set, just like I expected. So you must have got something else messed up.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by