FLOPs of DAG neural network

Hi, everyone. Is there any way to measure the FLOPs or computational complexity of DAG neural network or functions? I tried to statistic the excution time then calculate the FLOPs by profile roughly, but I think DAG is based on C++ or accelerated, therefore, the result of FLOPs is not trustable? Any suggestion?

Answer 1

[David Willingham]

Hi Dianxin,

FLOPs is a performance measure that's not typically used for Deep Learning. Performance can be measured in many ways, here's a list of some:

Throughput - E.g. Predictions per sec

Training Time - E.g. Time to reach x% of validation accuracy

Memory - E.g. How many MB of the network based on the weights

Power - E.g. For embedded devices, how much energy is required to make a prediction

Regards,

David

[DL]

Hi David,

I am really grateful for the help!

I also have a question on DAG network, is that accelerated by C++ or any optimazation from the foundamental layer? I found that CNN cost not much time, is that unrealistic to compare the time cost of DAG CNN with the code of my own? I mean if that is accelerated by C++ or any optimazation from the foundamental layer, it is not comparable if I did a CNN version of my code.

Best wishes,

Dianxin

[DL]

Is there a way to access the memory usage like profile?

[David Willingham]

Hi, by default optmizations are not enabled for inference. You can download this support package to enable it for the "predict function".

If you want more optimizations, these are built into our Coder products, which automatically generate native code for the target environment.

MATLAB Coder (C & C++) - including Intel MKL-DNN for Intel processors and ARM Compute Library for ARM Cortex processors

GPU Coder (Cuda) - NVIDIA® CUDA libraries, including TensorRT™, cuDNN, cuFFT, cuSolver, and cuBLAS

Deep Learning HDL Coder (FPGA)

[David Willingham]

For memory, see this post.