Matérn kernel functions for support vector regression

Hello to all,

i have seen that for gauss process regression matern kernel functions are available. But I want to use the Martern kernel function for the support vector regression (SVR). Furthermore I want to adjust the kernel scale parameters automatically with 'OptimizeHyperparameters'.

Is there a possibility or does the martern kernel work in SVR even if it is not listed in the documentation for SVR? And is it than possible to fit the kernel scale parameter here?

Answer 1

[Aditya Patil]

The KernelFunction argument accepts custom function. You can create a function for matern kernel, and pass it to KernelFunction argument.

However, OptimizeHyperparameters only selects one of the inbuilt Kernel functions that are provided. So it's not possible to provide a custom function to OptimizeHyperparameters.

As a workaround you can use Bayesian optimization. Consider the following example,

load carsmall.mat
X = [Horsepower,Weight];
Y = MPG;
% This is how you pass custom KernelFunction
mdl = fitrsvm(X, Y, 'KernelFunction', 'myKernel'); 

% List out the functions you want to test
kernel = optimizableVariable('kernel', {'myKernel', 'myKernel2'}, 'Type', 'categorical');
% Define the function to optimize. Typically this will be
% the loss function on test dataset. For simplicity, I have passed the training data
fun = @(x)loss(fitrsvm(X, Y, 'KernelFunction', char(x.kernel)), X, Y);
% Run bayesian optimization
results = bayesopt(fun, [kernel]);

myKernel and myKernel2 are defined separately. You can define the required functions.

function G = myKernel(U,V)
    [m, ~] = size(U);
    [n, ~] = size(V);
    G = ones([m,n]);
end

And,

function G = myKernel2(U,V)
    [m, ~] = size(U);
    [n, ~] = size(V);
    G = zeros([m,n]);
end

Note that we define multiple kernel functions as kernel function itself does not take arguments except U and V.

Answer 2

[Steffen Funk]

This looks good,

I will give it a try.

I have already tried a bit to write my own kernel function and I will try to include your suggestion.

Thank You