AIC values for a OGPR model trained in the Regression Learner App

Hello!

I'd like to establish an objective criteria for selecting a model trained using the Regression Learner app, especially when several models exhibit similar R2 and RMSE values in the prediction. Could you help me, please, to write a code to estimate the AIC from an exported model?

I'm attaching two exported models for comparison.

I'd really appreciate your help with this!

best,

Laura

Answer 1

[Shantanu Dixit]

Hi Laura, AIC (Akaike Information Criterion) can be used to evaluate and rank different models based on their performance for a given dataset. Here's how you can calculate AIC for the trained models:

1. Load the models and data: Load the trained models (TrainedRegressionModel.mat) and some test data against which the models can be evaluted.
2. Compute the log-likelihood: Use the predictions from the model (in this case predictFcn) and the ground truth values to calculate the log-likelihood for each model.
3. Calculate AIC: Use the likelihood and number of parameters (k) to compute AIC as 2k – 2(log-likelihood)

Refer to the example code below for calculating AIC for a model

modelFile = 'TrainedRegressionModel.mat';
loadedData = load(modelFile);
trainedModel = loadedData.trainedModel;

T = yourTestData;

predictedY = trainedModel.predictFcn(T);
actualY = T.target;  %% Assuming target contains the actual values

n = length(actualY);
residuals = actualY - predictedY;
sigma2 = var(residuals);
logLikelihood = -n/2 * log(2*pi) - n/2 * log(sigma2) - sum(residuals.^2) / (2*sigma2);

numPredictors = width(T)-1;  %% excluding the response variable
k = numPredictors + 1;

AIC = 2*k - 2*logLikelihood;

Refer to following documentation for detailed instructions on making predictions using models trained in Regression Learner App:

www.mathworks.com/help/stats/export-regression-model-to-predict-new-data.html