File Exchange

image thumbnail

Binary and Multi-class SVM

version 3.1.0.0 (21 KB) by Karl Ezra Pilario
SVMs for classifying 2D data to 2 or more classes, with 3D/2D visualizations as output.

39 Downloads

Updated 21 Jan 2019

View License

This code is intended to help visualize the learned classifier, when training nonlinear C-SVMs for classifying 2-dimensional data (2 features) to 2 or more classes. The case when C = Inf gives the hard margin classifier, while C < Inf gives the 1-norm soft margin classifier. (See Proposition 6.12 and Eq. (7.1) in [2])

MATLAB's quadprog is used to solve the dual variables, a. The solver is set to use the interior-point method. The Gaussian radial basis function (RBF) kernel is used to generate nonlinear boundaries.

In the Binary classification file (SVMtrial.m): There are 6 different training sets to play with. Outputs are the 3D mesh plot of the classifier and the number of support vectors.

Data sets for binary classification:
[1] TYPICAL
[2] SADDLE
[3] RANDOM
[4] RANDOM, IN ELLIPSE W/ 1 OUTLIER
[5] SPIRAL
[6] IMBALANCED + OVERLAP

In the Multi-class classification file (DAGsvm.m): There are 5 different training sets to play with. Outputs are the 3D mesh plots of the K*(K-1)/2 classifiers, a plot of the training set, and a list of misclassified training samples. You can also have the code estimate the RBF kernel width, according to [4]. I have used the DAG-SVM algorithm from [3] for multi-class classification. The output mesh plots are, thus, arranged in a directed acyclic graph (DAG).

Data sets for multi-class classification:
[1] (3 classes) FISHER IRIS - PETALS
[2] (4 classes) FAN W/ 4 ARMS
[3] (6 classes) RANDOM CIRCLES
[4] (5 classes) SOUTHEAST ASIAN MAP
[5] (7 classes) RAINBOW

You can start by simply running either SVMtrial.m or DAGsvm.m. With these, the user can change certain data points or hyperparameters (kernel width and box constraint) and directly see the effects to the 3D manifold visually. I hope this is useful for educational purposes.

References:

[1] Coursera - Machine Learning by Andrew Ng.
[2] Support Vector Machines, Cristianini & Shawe-Taylor, 2000
[3] Platt et al. Large Margin DAGs for Multiclass Classification, Advances in NIPS, 2000.
[4] Karatzoglou et al. Support Vector Machines in R, Journal of Statistical Software, 15(9), 2006.

Cite As

Karl Ezra Pilario (2019). Binary and Multi-class SVM (https://www.mathworks.com/matlabcentral/fileexchange/65232-binary-and-multi-class-svm), MATLAB Central File Exchange. Retrieved .

Comments and Ratings (2)

Chang hsiung

hua ye

hua ye (view profile)

Updates

3.1.0.0

Changed the title; Made minor code corrections

3.0.0.0

Added multi-class classification using DAG-SVM.

2.3.0.0

Added an imbalanced data set with an overlap between classes.

2.2.0.0

Wrote a red-white-blue colormap code where white must coincide with the decision boundary.

2.1.0.0

Changed the description.

2.0.0.0

Implemented 1-Norm Soft Margin SVM.

MATLAB Release Compatibility
Created with R2013b
Compatible with any release
Platform Compatibility
Windows macOS Linux
Acknowledgements

Inspired by: Red Blue Colormap, SVM

Discover Live Editor

Create scripts with code, output, and formatted text in a single executable document.


Learn About Live Editor