Image regression never converge Issue

Hello everyone, now I have a lot images within certain amont of catgories. I want to use these imgaes to train the CNN and try to predict the image catgories I neverseen before. Basically, all images are belongs to different number, so I hope to predict a number which is not trained before. I used the example https://www.mathworks.com/help/deeplearning/examples/convert-classification-network-into-regression-network.html

But the problem is that the training never converges (if I do classification via googlenet etc. the traning and testing will goes to 100% which is very accurate). So I hope to ask you guys what is the problem for my traning. I am appreciate any answer about it.