How can I improve the consistency of training algorithms result?

I have 32 set of data given 4 input and 2 experimental output . I would like to compare and choose the best training algorithms that suitable for the set of data. However, the result seems not consistent in which the regression would be 0.3 and jump to 0.9 for the same training algorithms. I have try to increase eproch but it does not work. I try to change the division of training, validate and test set. It is not applicable too.

Is there any other ways that I can use to improve the consistency?

Parts of the code are like this.

net = feedforwardnet([i,2],'trainbfg');

net.trainParam.lr=0.05;

net.trainParam.epochs=1000;

net.trainParam.goal=0.001;

net=train(net,NInp,NOut);

Answer 1

[Avadhoot]

Hi Choong Yee,

There could be several reasons for the lack of consistency in the results. Here are a few things to implement:

1. Data:

• Ensure that each class has enough samples for training to avoid bias.
• Remove outliers from the data to prevent inconsistencies.

2. Regularization:

• Inconsistent results can occur due to overfitting. Implement regularization techniques to deal with overfitting.
• You can use Bayesian Regularization with the "trainbr" function by adding the following line to your code:

net.trainFcn = ‘trainbr’ 

3. Monitoring and Loss function:

• If the testing accuracy is significantly lower than the training accuracy, apply regularization techniques.
• Monitor the value of the loss function for any sharp spikes. Consider changing the loss function to one that is more suitable for your data.

For more information about the following topics, refer to the links attached herewith:

1. “trainbr” function: https://www.mathworks.com/help/deeplearning/ref/trainbr.html#responsive_offcanvas
2. Choosing a loss function: https://www.mathworks.com/help/ident/ug/model-quality-metrics.html#buzo41z:~:text=the%20noise%20model.-,Model,-Quality%20Metrics

I hope it helps,

Regards,

Avadhoot.