ClassificationKNN

k-nearest neighbor classification

Description

ClassificationKNN is a nearest neighbor classification model in which you can alter both the distance metric and the number of nearest neighbors. Because a ClassificationKNN classifier stores training data, you can use the model to compute resubstitution predictions. Alternatively, use the model to classify new observations using the predict method.

Creation

Create a ClassificationKNN model using fitcknn.

Properties

KNN Properties

BreakTies — Tie-breaking algorithm

'smallest' (default) | 'nearest' | 'random'

Tie-breaking algorithm used by predict when multiple classes have the same smallest cost, specified as one of the following:

• 'smallest' — Use the smallest index among tied groups.

• 'nearest' — Use the class with the nearest neighbor among tied groups.

• 'random' — Use a random tiebreaker among tied groups.

By default, ties occur when multiple classes have the same number of nearest points among the k nearest neighbors. BreakTies applies when IncludeTies is false.

Change BreakTies using dot notation: mdl.BreakTies = newBreakTies.

Distance — Distance metric

'cityblock' | 'chebychev' | 'correlation' | 'cosine' | 'euclidean' | 'hamming' | function handle | ...

Distance metric, specified as the comma-separated pair consisting of 'Distance' and a valid distance metric name or function handle. The allowable distance metric names depend on your choice of a neighbor-searcher method (see NSMethod).

NSMethod	Distance Metric Names
exhaustive	Any distance metric of ExhaustiveSearcher
kdtree	'cityblock', 'chebychev', 'euclidean', or 'minkowski'

This table includes valid distance metrics of ExhaustiveSearcher.

Distance Metric Names	Description
'cityblock'	City block distance.
'chebychev'	Chebychev distance (maximum coordinate difference).
'correlation'	One minus the sample linear correlation between observations (treated as sequences of values).
'cosine'	One minus the cosine of the included angle between observations (treated as vectors).
'euclidean'	Euclidean distance.
'hamming'	Hamming distance, percentage of coordinates that differ.
'jaccard'	One minus the Jaccard coefficient, the percentage of nonzero coordinates that differ.
'mahalanobis'	Mahalanobis distance, computed using a positive definite covariance matrix C. The default value of C is the sample covariance matrix of X, as computed by cov(X,'omitrows'). To specify a different value for C, use the 'Cov' name-value pair argument.
'minkowski'	Minkowski distance. The default exponent is 2. To specify a different exponent, use the 'Exponent' name-value pair argument.
'seuclidean'	Standardized Euclidean distance. Each coordinate difference between X and a query point is scaled, meaning divided by a scale value S. The default value of S is the standard deviation computed from X, S = std(X,'omitnan'). To specify another value for S, use the Scale name-value pair argument.
'spearman'	One minus the sample Spearman's rank correlation between observations (treated as sequences of values).
@distfun	Distance function handle. distfun has the form function D2 = distfun(ZI,ZJ) % calculation of distance ... where ZI is a 1-by-N vector containing one row of X or Y. ZJ is an M2-by-N matrix containing multiple rows of X or Y. D2 is an M2-by-1 vector of distances, and D2(k) is the distance between observations ZI and ZJ(k,:).

If you specify CategoricalPredictors as 'all', then the default distance metric is 'hamming'. Otherwise, the default distance metric is 'euclidean'.

Change Distance using dot notation: mdl.Distance = newDistance.

If NSMethod is 'kdtree', you can use dot notation to change Distance only for the metrics 'cityblock', 'chebychev', 'euclidean', and 'minkowski'.

For definitions, see Distance Metrics.

Example: 'Distance','minkowski'

Data Types: char | string | function_handle

DistanceWeight — Distance weighting function

'equal' | 'inverse' | 'squaredinverse' | function handle

Distance weighting function, specified as one of the values in this table.

Value	Description
'equal'	No weighting
'inverse'	Weight is 1/distance
'squaredinverse'	Weight is 1/distance2
@fcn	fcn is a function that accepts a matrix of nonnegative distances and returns a matrix of the same size containing nonnegative distance weights. For example, 'squaredinverse' is equivalent to @(d)d.^(–2).

Change DistanceWeight using dot notation: mdl.DistanceWeight = newDistanceWeight.

Data Types: char | function_handle

DistParameter — Parameter for distance metric

positive definite covariance matrix | positive scalar | vector of positive scale values

Parameter for the distance metric, specified as one of the values described in this table.

Distance Metric	Parameter
'mahalanobis'	Positive definite covariance matrix C
'minkowski'	Minkowski distance exponent, a positive scalar
'seuclidean'	Vector of positive scale values with length equal to the number of columns of X

For any other distance metric, the value of DistParameter must be [].

You can alter DistParameter using dot notation: mdl.DistParameter = newDistParameter. However, if Distance is 'mahalanobis' or 'seuclidean', then you cannot alter DistParameter.

Data Types: single | double

IncludeTies — Tie inclusion flag

false (default) | true

Tie inclusion flag indicating whether predict includes all the neighbors whose distance values are equal to the kth smallest distance, specified as false or true. If IncludeTies is true, predict includes all of these neighbors. Otherwise, predict uses exactly k neighbors (see the BreakTies property).

Change IncludeTies using dot notation: mdl.IncludeTies = newIncludeTies.

Data Types: logical

NSMethod — Nearest neighbor search method

'kdtree' | 'exhaustive'

This property is read-only.

Nearest neighbor search method, specified as either 'kdtree' or 'exhaustive'.

• 'kdtree' — Creates and uses a Kd-tree to find nearest neighbors.

• 'exhaustive' — Uses the exhaustive search algorithm. When predicting the class of a new point xnew, the software computes the distance values from all points in X to xnew to find nearest neighbors.

The default value is 'kdtree' when X has 10 or fewer columns, X is not sparse, and the distance metric is a 'kdtree' type. Otherwise, the default value is 'exhaustive'.

NumNeighbors — Number of nearest neighbors

positive integer value

Number of nearest neighbors in X used to classify each point during prediction, specified as a positive integer value.

Change NumNeighbors using dot notation: mdl.NumNeighbors = newNumNeighbors.

Data Types: single | double

Other Classification Properties

CategoricalPredictors — Categorical predictor indices

[] | vector of positive integers

This property is read-only.

Categorical predictor indices, specified as a vector of positive integers. CategoricalPredictors contains index values indicating that the corresponding predictors are categorical. The index values are between 1 and p, where p is the number of predictors used to train the model. If none of the predictors are categorical, then this property is empty ([]).

Data Types: double

ClassNames — Names of classes in training data Y

categorical array | character array | logical vector | numeric vector | cell array of character vectors

This property is read-only.

Names of the classes in the training data Y with duplicates removed, specified as a categorical or character array, logical or numeric vector, or cell array of character vectors. ClassNames has the same data type as Y. (The software treats string arrays as cell arrays of character vectors.)

Data Types: categorical | char | logical | single | double | cell

Cost — Cost of misclassification

square matrix

Cost of the misclassification of a point, specified as a square matrix. Cost(i,j) is the cost of classifying a point into class j if its true class is i (that is, the rows correspond to the true class and the columns correspond to the predicted class). The order of the rows and columns in Cost corresponds to the order of the classes in ClassNames. The number of rows and columns in Cost is the number of unique classes in the response.

By default, Cost(i,j) = 1 if i ~= j, and Cost(i,j) = 0 if i = j. In other words, the cost is 0 for correct classification and 1 for incorrect classification.

Change a Cost matrix using dot notation: mdl.Cost = costMatrix.

Data Types: single | double

ExpandedPredictorNames — Expanded predictor names

cell array of character vectors

This property is read-only.

Expanded predictor names, specified as a cell array of character vectors.

If the model uses encoding for categorical variables, then ExpandedPredictorNames includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames is the same as PredictorNames.

Data Types: cell

ModelParameters — Parameters used in training ClassificationKNN

object

This property is read-only.

Parameters used in training the ClassificationKNN model, specified as an object.

Mu — Predictor means

numeric vector

This property is read-only.

Predictor means, specified as a numeric vector of length numel(PredictorNames).

If you do not standardize mdl when training the model using fitcknn, then Mu is empty ([]).

Data Types: single | double

NumObservations — Number of observations

positive integer scalar

This property is read-only.

Number of observations used in training the ClassificationKNN model, specified as a positive integer scalar. This number can be less than the number of rows in the training data because rows containing NaN values are not part of the fit.

Data Types: double

PredictorNames — Predictor variable names

cell array of character vectors

This property is read-only.

Predictor variable names, specified as a cell array of character vectors. The variable names are in the same order in which they appear in the training data X.

Data Types: cell

Prior — Prior probabilities for each class

numeric vector

Prior probabilities for each class, specified as a numeric vector. The order of the elements in Prior corresponds to the order of the classes in ClassNames.

Add or change a Prior vector using dot notation: mdl.Prior = priorVector.

Data Types: single | double

ResponseName — Response variable name

character vector

This property is read-only.

Response variable name, specified as a character vector.

Data Types: char

RowsUsed — Rows used in fitting

[] | logical vector

This property is read-only.

Rows of the original training data used in fitting the ClassificationKNN model, specified as a logical vector. This property is empty if all rows are used.

Data Types: logical

ScoreTransform — Score transformation

'none' (default) | 'doublelogit' | 'invlogit' | 'ismax' | 'logit' | function handle | ...

Score transformation, specified as either a character vector or a function handle.

This table summarizes the available character vectors.

Value	Description
"doublelogit"	1/(1 + e–2x)
"invlogit"	log(x / (1 – x))
"ismax"	Sets the score for the class with the largest score to 1, and sets the scores for all other classes to 0
"logit"	1/(1 + e–x)
"none" or "identity"	x (no transformation)
"sign"	–1 for x < 0 0 for x = 0 1 for x > 0
"symmetric"	2x – 1
"symmetricismax"	Sets the score for the class with the largest score to 1, and sets the scores for all other classes to –1
"symmetriclogit"	2/(1 + e–x) – 1

For a MATLAB^® function or a function you define, use its function handle for score transform. The function handle must accept a matrix (the original scores) and return a matrix of the same size (the transformed scores).

Change ScoreTransform using dot notation: mdl.ScoreTransform = newScoreTransform.

Data Types: char | function_handle

Sigma — Predictor standard deviations

numeric vector

This property is read-only.

Predictor standard deviations, specified as a numeric vector of length numel(PredictorNames).

If you do not standardize the predictor variables during training, then Sigma is empty ([]).

Data Types: single | double

W — Observation weights

vector of nonnegative values

This property is read-only.

Observation weights, specified as a vector of nonnegative values with the same number of rows as Y. Each entry in W specifies the relative importance of the corresponding observation in Y.

Data Types: single | double

X — Unstandardized predictor data

numeric matrix

This property is read-only.

Unstandardized predictor data, specified as a numeric matrix. Each column of X represents one predictor (variable), and each row represents one observation.

Data Types: single | double

Y — Class labels

categorical array | character array | logical vector | numeric vector | cell array of character vectors

This property is read-only.

Class labels, specified as a categorical or character array, logical or numeric vector, or cell array of character vectors. Each value in Y is the observed class label for the corresponding row in X.

Y has the same data type as the data in Y used for training the model. (The software treats string arrays as cell arrays of character vectors.)

Data Types: single | double | logical | char | cell | categorical

Hyperparameter Optimization Properties

HyperparameterOptimizationResults — Cross-validation optimization of hyperparameters

BayesianOptimization object | table

This property is read-only.

Cross-validation optimization of hyperparameters, specified as a BayesianOptimization object or a table of hyperparameters and associated values. This property is nonempty when the 'OptimizeHyperparameters' name-value pair argument is nonempty when you create the model using fitcknn. The value depends on the setting of the 'HyperparameterOptimizationOptions' name-value pair argument when you create the model:

• 'bayesopt' (default) — Object of class BayesianOptimization

• 'gridsearch' or 'randomsearch' — Table of hyperparameters used, observed objective function values (cross-validation loss), and rank of observations from lowest (best) to highest (worst)

Object Functions

compareHoldout	Compare accuracies of two classification models using new data
crossval	Cross-validate machine learning model
edge	Edge of k-nearest neighbor classifier
gather	Gather properties of Statistics and Machine Learning Toolbox object from GPU
lime	Local interpretable model-agnostic explanations (LIME)
loss	Loss of k-nearest neighbor classifier
margin	Margin of k-nearest neighbor classifier
partialDependence	Compute partial dependence
plotPartialDependence	Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots
predict	Predict labels using k-nearest neighbor classification model
resubEdge	Resubstitution classification edge
resubLoss	Resubstitution classification loss
resubMargin	Resubstitution classification margin
resubPredict	Classify training data using trained classifier
shapley	Shapley values
testckfold	Compare accuracies of two classification models by repeated cross-validation

Examples

Train k-Nearest Neighbor Classifier

Train a k-nearest neighbor classifier for Fisher's iris data, where k, the number of nearest neighbors in the predictors, is 5.

Load Fisher's iris data.

load fisheriris
X = meas;
Y = species;

X is a numeric matrix that contains four petal measurements for 150 irises. Y is a cell array of character vectors that contains the corresponding iris species.

Train a 5-nearest neighbor classifier. Standardize the noncategorical predictor data.

Mdl = fitcknn(X,Y,'NumNeighbors',5,'Standardize',1)

Mdl = 
  ClassificationKNN
             ResponseName: 'Y'
    CategoricalPredictors: []
               ClassNames: {'setosa'  'versicolor'  'virginica'}
           ScoreTransform: 'none'
          NumObservations: 150
                 Distance: 'euclidean'
             NumNeighbors: 5

Mdl is a trained ClassificationKNN classifier, and some of its properties appear in the Command Window.

To access the properties of Mdl, use dot notation.

Mdl.ClassNames

ans = 3x1 cell
    {'setosa'    }
    {'versicolor'}
    {'virginica' }

Mdl.Prior

ans = 1×3

    0.3333    0.3333    0.3333

Mdl.Prior contains the class prior probabilities, which you can specify using the 'Prior' name-value pair argument in fitcknn. The order of the class prior probabilities corresponds to the order of the classes in Mdl.ClassNames. By default, the prior probabilities are the respective relative frequencies of the classes in the data.

You can also reset the prior probabilities after training. For example, set the prior probabilities to 0.5, 0.2, and 0.3, respectively.

Mdl.Prior = [0.5 0.2 0.3];

You can pass Mdl to predict to label new measurements or crossval to cross-validate the classifier.

Tips

• The compact function reduces the size of most classification models by removing the training data properties and any other properties that are not required to predict the labels of new observations. Because k-nearest neighbor classification models require all of the training data to predict labels, you cannot reduce the size of a ClassificationKNN model.

Alternative Functionality

knnsearch finds the k-nearest neighbors of points. rangesearch finds all the points within a fixed distance. You can use these functions for classification, as shown in Classify Query Data. If you want to perform classification, then using ClassificationKNN models can be more convenient because you can train a classifier in one step (using fitcknn) and classify in other steps (using predict). Alternatively, you can train a k-nearest neighbor classification model using one of the cross-validation options in the call to fitcknn. In this case, fitcknn returns a ClassificationPartitionedModel cross-validated model object.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

• The predict function supports code generation.

• When you train a k-nearest neighbor classification model by using fitcknn, the following restrictions apply.

  • The value of the 'Distance' name-value pair argument cannot be a custom distance function.

  • The value of the 'DistanceWeight' name-value pair argument can be a custom distance weight function, but it cannot be an anonymous function.

  • The value of the 'ScoreTransform' name-value pair argument cannot be an anonymous function.

For more information, see Introduction to Code Generation.

GPU Arrays

Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™.

Usage notes and limitations:

• The following object functions fully support GPU arrays:

  • crossval

  • gather

  • resubEdge

  • resubLoss

  • resubMargin

  • resubPredict

• The following object functions offer limited support for GPU arrays:

  • compareHoldout

  • edge

  • loss

  • margin

  • partialDependence

  • plotPartialDependence

  • predict

• The object functions execute on a GPU if either of the following apply:

  • The model was fitted with GPU arrays.

  • The predictor data that you pass to the object function is a GPU array.

For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).

Version History

Introduced in R2012a