Dose deepDreamImage function show filter(kernel) of specific convolution layer or represent learned features from specific convolution layer of an initial image?

If deepDreamImage function represents the filter(kernel) why we need coloured filter at intermediate stages since when we visualze feature map after applying first convolution layer all is gray?

c = 2; % select layer 2 of CNN,  Conv 32 5×5×3
layer_name = net.Layers(c).Name; % name of layer
channels = 1:32;
f = deepDreamImage(net,layer_name,channels,'PyramidLevels',1,'Verbose',0, 'NumIterations',15);
f = imtile(f,'ThumbnailSize',[32 32]);
figure, imshow(f);

c = 2; % select layer 2 of CNN,  Conv 32 5×5×3
layer_name = net.Layers(c).Name; % name of layer
channels = 1:32;
f = deepDreamImage(net,layer_name,channels,'PyramidLevels',1,'Verbose',0, 'NumIterations',15, "InitialImage",im); %Define intial image
f = imtile(f,'ThumbnailSize',[32 32]);
figure, imshow(f);
    

c = 2; % select layer 2 of CNN,  Conv 32 5×5×3
layer_name = net.Layers(c).Name; % name of layer
layer_activations = activations(net,im,layer_name);
sz = size(layer_activations)
layer_activations_reshape = reshape(layer_activations,[sz(1) sz(2) 1 sz(3)]);
New = imtile(mat2gray(layer_activations_reshape),'GridSize',[6 6]); %visulaze fueture map
figure, imshow(New);

c = 5; % select layer 5 of CNN,  Conv 32 5×5×32
layer_name = net.Layers(c).Name; % name of layer
channels = 1:32;
f = deepDreamImage(net,layer_name,channels,'PyramidLevels',1,'Verbose',0, 'NumIterations',15);
f = imtile(f,'ThumbnailSize',[32 32]);
figure, imshow(f);

c = 5; % select layer 5 of CNN,  Conv 32 5×5×32
layer_name = net.Layers(c).Name; % name of layer
channels = 1:32;
f = deepDreamImage(net,layer_name,channels,'PyramidLevels',1,'Verbose',0, 'NumIterations',15, "InitialImage",im); %Define intial image
f = imtile(f,'ThumbnailSize',[32 32]);
figure, imshow(f);

c = 5; % select layer 5 of CNN,  Conv 32 5×5×32
layer_name = net.Layers(c).Name; % name of layer
layer_activations = activations(net,im,layer_name);
sz = size(layer_activations)
layer_activations_reshape = reshape(layer_activations,[sz(1) sz(2) 1 sz(3)]);
New = imtile(mat2gray(layer_activations_reshape),'GridSize',[6 6]); %visulaze fueture map
figure, imshow(New);

Answer 1

[Milan Bansal]

Hi Amir,

As per my understanding you want to know if the "deepDreamImage" function show filter of specific convolution layer or learned features from specific convolution layer. You also want to know why do we need coloured feature maps at the intermediated stages of the CNN.

The "deepDreamImage" function in MATLAB is mainly used to visualize the learned features from a specific convolutional layer of a neural network, rather than showing the filters (kernels) directly. It works by amplifying the features that a given layer of the network has learned to recognize in an input image, effectively creating an image that maximizes the activation of specific neurons or channels within that layer. This process is often used to understand what kind of features a network has learned at different layers. It gives insight into what the network "sees" or "pays attention to" at different depths.

The colorful feature maps produced by the "deepDreamImage" function at intermediate stages of a neural network are not a requirement but rather a byproduct of how the optimization process manipulates the input image to maximize activations. These colorful, dream-like images provide insights into the complex, learned features of the network in a visually intuitive way. Below are some advantages of the colored featured maps generated by "deepDreamImage".

• Visual Interpretability: Colorful feature maps can make the abstract features learned by intermediate layers more visually interpretable to humans.
• Understanding Network Behavior: The colors in the images produced by deepDreamImage can help researchers and practitioners understand how different layers of the network respond to various features in the input image.
• Highlighting Feature Combinations: The colors in the generated images can represent combinations of features that the network has learned to recognize. In intermediate layers, the network no longer sees individual pixels but rather abstract representations of the input data.
• Enhancing Feature Distinction: In complex images, where multiple patterns and textures might be present, color can help separate and highlight these features more clearly than grayscale visualizations.
• Artistic Exploration: Beyond their utility in understanding neural networks, the colorful, surreal images generated by Deep Dream have artistic value. They have inspired artists and creators to explore the intersection of artificial intelligence and art.

Please refer to the following documentaiton link to learn more about "deepDreamImage" function.

https://www.mathworks.com/help/deeplearning/ref/deepdreamimage.html

Hope this helps!