CNN Performance: CPU Consistency vs. GPU Variance - Why?
2 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Hello everyone,
I have executed CNN code multiple times using rng(0) with CPU and consistently obtained the same result. However, when I attempted to accelerate the training process using the GPU, the results differed. Has anyone else faced this issue?
Thank you in advance!
0 Kommentare
Akzeptierte Antwort
Ruth
am 23 Nov. 2023
Hi Hamza,
Even when using "gpurng" some small non-deterministic behavior is expected to happen in the GPU during training, particularly during the backward pass. This is out of our control.
However the behavior should be deterministic in the forward pass and subsequently at prediction time.
If one sets the learning rate to be almost zero (e.g. 1e-16, meaning nothing is updated in the backward pass), the output of training (using "rng" and "gpurng") should look deterministic.
Best wishes,
Ruth
0 Kommentare
Weitere Antworten (1)
Edric Ellis
am 23 Nov. 2023
I'm not certain if it will make everything consistent, but note that random state on the GPU is controlled by the gpurng function.
Siehe auch
Kategorien
Mehr zu Parallel and Cloud finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!