why Nvidia A100 GPUs slower than RTX 3090 GPUs?

Hello, we have RTX3090 GPU and A100 GPU.

Using the Matlab Deep Learning Toolbox Model for ResNet-50 Network, we found that the A100 was 20% slower than the RTX 3090 when learning from the ResNet50 model.

The questions are as follows.

1. I heard that the speed of A100 and 3090 is different because there is a difference between the number of CUDA cores and the number of Tensor cores, so can only use Cuda cores for Matlab?

If you can use it, I would appreciate it if you could send me a link if you have an example site using Tensor core.

2. You can specify single inference, double inference, and half inference methods when learning GPU. I heard that Matlab uses double inference automatically, so please check if it is the correct answer.

Thank you.

Answer 1

[David Willingham]

See this answer for an explanation:

https://www.mathworks.com/matlabcentral/answers/1711380-rtx-3090-vs-a100-in-deep-learning?s_tid=ta_ans_results

[재호 곽]

Thank you for your answer. However, I have already checked the URL you sent me. Currently, I have used GPU cord, and I am conducting training, not an inference. I wonder that A100 is slower than 3090 in the training process. I wonder if double precision is available in the training process in Matlab.

[Joss Knight]

It is possible to train models in double precision, using model functions, or using a dlnetwork and converting its weights to double precision before training.

However, I don't believe this is what you want. You won't get a speedup over the RTX 3090 training in single precision, it will still be considerably slower.