Why my GradCAM is not localized based on the predictions of the network?

I use Grad-CAM to intepret the predictions from my neural network but it seems the class activation maps does not localized based on the two clasess (fracture and background) as shown in the image below.

This is my code for the GradCAM. The network that I used is u-net. I hope the selection for feature layer and reduction layer is correct.

img = readimage(imdstest,673);

labels = semanticseg(img,net);

featureLayer = "Decoder-Stage-4-ReLU-2";

reductionLayer = "Softmax-Layer";

segMap = labeloverlay(img,labels);

gradCAMMap = gradCAM(net,imgTest,classNames,"FeatureLayer",featureLayer, ...

"ReductionLayer",reductionLayer);

figure;

ax(1) = subplot(2,2,1);

imshow(img)

colormap(ax(1),"gray")

title("Test Image")

subplot(2,2,2)

imshow(segMap)

title("Semantic Segmentation")

ax(2) = subplot(2,2,3);

imshow(img)

hold on

imagesc(gradCAMMap(:,:,1),"AlphaData",0.5)

title("Grad-CAM: " + classNames(1))

colormap(ax(2),"jet")

ax(3) = subplot(2,2,4);

imshow(img)

hold on

imagesc(gradCAMMap(:,:,2),"AlphaData", 0.5)

title("Grad-CAM: " + classNames(2))

colormap(ax(3),"jet")

Answer 1

[Shivansh]

Hi Wan!

GradCam is typically used for interpreting the predictions of convolutional neural networks (CNNs) on image classification tasks. However, your use case involves a U-Net architecture, which is commonly used for semantic segmentation tasks. This might be the reason why you are not getting the expected localization for your classes.

The U-Net architecture is different from the typical CNNs used for image classification. It consists of a contracting path (encoder), a bottleneck, and an expansive path (decoder).

In U-Net, the last layers of the decoder are responsible for upscaling the feature maps and making dense predictions for every pixel. You should choose a feature layer from the decoder that is close to the output layer but before the final classification layer. It will ensure that the feature maps are detailed enough for localization but also contain enough high-level information to represent the classes.

Hope it helps!