Regression Learner App. Model Performance

Hello everyone, I am performing a regression analysis using Regression Learner App., and the results include both high and low values, as well as some negative values. According to the documentation page smaller values indicate better model performance. Because of this, I am confused about how to interpret my results. Am I doing something wrong? My data includes 4 categorical and 4 numerical values. Thanks for the help.

Answer 1

[Stephen23]

You are probably not doing anything wrong. The confusion comes from the fact that the Regression Learner App reports multiple performance metrics, and they do not all have the same interpretation. These are error metrics:

• MSE (Mean Squared Error) -> lower is better
• RMSE (Root Mean Squared Error) -> lower is better
• MAE (Mean Absolute Error) -> lower is better

These measure prediction error directly. A value of 0 would indicate perfect predictions.

R-squared (R²) is different.

• R² = 1 -> perfect fit
• R² = 0 -> model performs about as well as predicting the mean
• R² < 0 -> model performs worse than simply predicting the mean

Therefore, for R², larger values are better, not smaller.

Your best models based on the test set:

1- Neural Network

• RMSE = 7.124 (lowest)
• R² = 0.531 (highest)

2- Ensemble

• RMSE = 7.176
• R² = 0.524

3-Tree

• RMSE = 7.549
• R² = 0.474

These three models appear to be performing reasonably similarly.

The SVM model stands out as a poor model:

• RMSE = 10.547
• R² = -0.027

The negative R² means the SVM model predicts worse than a simple baseline that always predicts the average response value.

[MB]

Thank you very much for your detailed explanation.