kfoldfun

Cross-validate function for regression

collapse all in page

    Syntax

    vals = kfoldfun(CVMdl,fun)

    Description

    vals = kfoldfun(CVMdl,fun) cross-validates the function fun by applying fun to the data stored in the cross-validated model CVMdl. You must pass fun as a function handle.

    example

    Examples

    collapse all

    Open Live Script

    Train a regression tree model, and then cross-validate it using a custom k-fold loss function.

    Load the imports-85 data set. Train a regression tree using a subset of the data.

    load imports-85
Mdl = fitrtree(X(:,[4 5]),X(:,16),...
    'PredictorNames',{'Length','Width'},...
    'ResponseName','Price');

    Cross-validate the regression tree, and obtain the mean squared error.

    CVMdl = crossval(Mdl);
L = kfoldLoss(CVMdl)
    L = 
1.9167e+07

    Examine the error when you use a simple averaging of training responses instead of predictions in the calculation.

    f = @(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)...
    mean((Ytest-mean(Ytrain)).^2)
    f = function_handle with value:
    @(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)mean((Ytest-mean(Ytrain)).^2)


    mean(kfoldfun(CVMdl,f))
    ans = 
6.3586e+07

    Input Arguments

    collapse all

    Cross-validated function, specified as a function handle. fun has the syntax:

    testvals = fun(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)

    • CMP is a compact model stored in one element of the CVMdl.Trained property.

    • Xtrain is the training matrix of predictor values.

    • Ytrain is the training array of response values.

    • Wtrain contains the training weights for the observations.

    • Xtest and Ytest are the test data, with associated weights Wtest.

    • The returned value testvals must have the same size across all folds.

    Data Types: function_handle

    Output Arguments

    collapse all

    Cross-validation results, returned as a numeric matrix. vals contains the arrays of testvals output, concatenated vertically over all folds. For example, if testvals from every fold is a numeric vector of length N, kfoldfun returns a KFold-by-N numeric matrix with one row per fold.

    Data Types: double

    Extended Capabilities

    expand all

    Version History

    Introduced in R2011a

    expand all

    See Also

    | | | | | | |