Segmenting a huge batch of images

I have a huge set of images taken from an SEM (a couple of thousands) and i want to automate the whole image processing...process: segmentation and extracting features are the main things to focus on right now.

The goal is to calculate the width of each ROI (i specified the ROI on the image). I count the number of pixels in each column for each ROI and divide by the scale to get the width per column.

Attached is a shitty example (i have even worse images) and my problem is of course with the shitty ones. The good images have nice histograms where you can easily differentiate between the regions just by looking at the histograms and a Otsu thresholding usually does the job. Some others have multimodal, overlapping or just a cluster#@$%. I don't know how to segment the areas when there are cracks, holes, scratches or anything similar. Sometimes the image looks tilted or upside down or the ROI has an L shape etc.

I am very new to image processing so explain like i'm 5.

Answer 1

[Image Analyst]

Try multithresh():

% Demo by Image Analyst
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;

%--------------------------------------------------------------------------------------------------------
%    READ IN IMAGE
grayImage = imread('SampleDim.jpg');

% Get the dimensions of the image.
% numberOfColorChannels should be = 1 for a gray scale image, and 3 for an RGB color image.
[rows, columns, numberOfColorChannels] = size(grayImage)
if numberOfColorChannels > 1
	% It's not really gray scale like we expected - it's color.
	% Extract the red channel (so the magenta lines will be white).
	grayImage = grayImage(:, :, 1);
end

%--------------------------------------------------------------------------------------------------------
% Display the image.
subplot(3, 3, 1);
imshow(grayImage, []);
axis('on', 'image');
title('Original Image', 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;
hFig = gcf;
hFig.WindowState = 'maximized'; % May not work in earlier versions of MATLAB.
drawnow;

% Binarize
thresh = multithresh(grayImage,5);
thresh = [0, thresh, inf]
for k = 1 : length(thresh) - 1
	fprintf('Getting mask between %d and %d.\n', thresh(k), thresh(k+1) - 1);
	caption = sprintf('Mask between %d and %d', thresh(k), thresh(k+1) - 1);
	mask = grayImage > thresh(k) & grayImage < thresh(k+1) - 1;
	
	% Take only largest blobs.
	mask = bwareaopen(mask, 5000);
	% Display the image.
	subplot(3, 3, k+1);
	imshow(mask, []);
	axis('on', 'image');
	title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
	impixelinfo;
end

[Swag Mane]

Thanks for the quick reponse! I actually just need to seperate the 2 regions in the middle as specified inside the rectangle (check 'Sample.jpg'). The bright region on top and the darker region at the bottom are not needed for feature extraction but it is good to have them separated too. Have a look at 'export.jpg', blue and red are my 2 most important ROIs

• Another problem with the thresholding method is a huge chunk of 0 valued pixels which stem from the absolute dark area above the bright region. That is a void i absolutely don't need for anything and it just ruins thresholding. How can i ignore these huge chunks of 0 pixel intensities when thresholding? For example, i assigned NaNs to these zeros but i don't know how to proceed or wether my way of thinking is viable:

    % consider the image is in grayscale
    % also consider that the huge chunk of 0s comes only from the void, not cracks, scratches or holes
    image=im2double(image); % convert image to double and assign 0 values to NaNs
    black= (image==0); image(black)=nan;
    
    % is this even a viable method?
    image_no_nans=image(~isnan(image));
    thresh=graythresh(image_no_nans);
    % image=imbinarize(image,thresh); % or idk, what now?

• Do you think i need a much more advanced segmentation method to separate my ROIs? Is this enough batch processing of "more or less" similar images? (Check more shitty examples)

In the shitty examples you will notice the common problems that i have to deal with when seperating the regions and this is why i ask whether i need advanced methods.

Edit: Gotta wait 24h for next upload...

[Image Analyst]

I don't understand your question. The black at the top of the image did not ruin the thresholding. As you can see I got threshold limits for every gray level range corresponding to a layer in your image.

Don't use image for the name of your variable since it is the name of a built in function.

If you want, you could just extract all the non-zero pixels into a list and pass that into multithresh():

darkMask = grayImage < 10; % Or whatever.
nonZeroGLs = grayImage(darkMask);
thresh = multithresh(grayImage,4);

but I don't think that's necessary unless you think I'm not getting the thresholds accurately.

[Swag Mane]

Usually I use img for image. I just used it this one time in my comment.

What i meant is: I want to exclude that black part from the thresholding calculation. I dont want the algorithm to consider those 0 pixels in the calculation. I dont even need that area segmented. If it was possible to crop images according to a random shape i would have actually done that beforehand.

In other words, i dont have a good grasp on what the multithresh() algorithm does behind the scenes so that why i'm saying "it will ruin the thresholding calculation"

I also changed the absolute boundaries for the thresholding value because i dont want zeros... and anything at 255 too

thresh=[1, thresh, 254]; 

• Do you also think i need any other segmentation methods beyond thresholding to separate my ROIs?

I dont know much beyond the built in thresholding functions. I am guessing triangle thresholding would work with overlapping histograms. I looked into hysteresis thresholding too while checking out Canny edge detection. Of course, i dont understand much of it either.

And like i said, i have a huge data set and the code will most likely fail at certain "inevitably" bad images. (Check examples.zip)

[Image Analyst]

I'm pretty sure it doesn't matter if you include the dark pixels and have N+1 intensity ranges or exclude them and tell it you have N intensity ranges. Either way you get a set of thresholds and have to tell it which low and high threshold to use to segment the layer you want.

Once you have the layer, you can alter the mask if you want to, for example, fill in cracks with neighboring pixel intensities, include the cracks as zero in any computations. By default, only the intensity range is chosen by the thresholds and any dark holes or cracks/nooks/crannies/bays/incursions into the mask are excluded (not in the mask). Not sure what you want to do really. One way to get the mean thickness is to just scan the mask for that single particular layer and find the first and last row for each column. then subtract them to get the thickness, then take the mean of those.

[rows, columns] = size(mask);
mask = bwareafilt(mask, 1); % Take largest connected blob only, ignore small noise blobs.
firstRows = zeros(1, columns);
lastRows = zeros(1, columns);
for col = 1 : columns
    fr = find(mask(:, col), 1, 'first')
    if ~isempty(fr)
        firstRows(col) = fr;
        lastRows(col) = find(mask(:, col), 1, 'last')
    end
end
% Get the thicknesses (or depths, widths, heights 
% or whatever you want to call them) at every column.
thicknesses = lastRows - firstRows;
% Get the mean thickness.
goodIndexes  = firstRows ~= 0;
meanThickness = mean(thicknesses(goodIndexes))