How to properly detect the edges of the rail surface for cropping?

Hi, i have tried to use sobel edge detection and hough transform to detect the lines of the rail image and the cropping it after. However, it does not seem to be detecting the edges that i need. I want it to detect the rail edges of the image only, so i am able to crop it. Please see attached codes and imaged for reference. Please help. Thanks.

clc;
close all;

% Prompt User to get video file
[file,path] = uigetfile('*');
if isequal(file,0)
   disp('User selected Cancel');
else
   disp(['User selected ', fullfile(path,file)]);
end

%Reading of image
Img=imread(fullfile(path,file));

grayscale=rgb2gray(Img);
grayscale2=histeq(grayscale);

%Sobel Edge detection & filter weak edges.
    DetectedEdges = edge(grayscale2,'sobel','horizontal');
    FilteredImage = bwareaopen(DetectedEdges,60);
    
    %Hough Transformation with Peak & Lines identification.
    [H,theta,rho] = hough(DetectedEdges);
    Peaks = houghpeaks(H,5,'threshold',ceil(0.6*max(H(:))));
    Lines = houghlines(DetectedEdges,theta,rho,Peaks,'FillGap',10,'MinLength',100);
    
    %Display Filtered Weak Edges & Draw Detected Lines on Grayscaled Frame.
    figure('Name','Filtered Weak Edges & Lines Detection','NumberTitle','off');
    subplot(2,1,1), imshow(FilteredImage);
    subplot(2,1,2), imshow(grayscale2), hold on
    
    %Identify Frame dimensions & line coordinates in both upper & lower rail edge.
    [size_x,size_y] = size(grayscale2);
    min_y = size_y;
    max_y = 0;
    
    %Set condition to determine upper & lower edge.
    for k = 1:length(Lines)
        xy = [Lines(k).point1;Lines(k).point2];
        if xy(1,2) <= min_y
            min_y = xy(1,2);
            min_y_line = xy;
        end
        if xy(1,2) >= max_y
            max_y = xy(1,2);
            max_y_line = xy;
        end
    end
    
    %Use straight line formula to determine y-intercepts & gradients.
    min_y_gradient = (min_y_line(2,2) - min_y_line(1,2)) / (min_y_line(1,2) - min_y_line(1,1)); 
    min_y_intercept = min_y_line(1,2) - min_y_gradient*min_y_line(1,1);
    plot(min_y_line(:,1),min_y_line(:,2),'LineWidth',2,'Color','green');
    
    max_y_gradient = (max_y_line(2,2) - max_y_line(1,2)) / (max_y_line(1,2) - max_y_line(1,1)); 
    max_y_intercept = max_y_line(1,2) - max_y_gradient*max_y_line(1,1);
    plot(max_y_line(:,1),max_y_line(:,2),'LineWidth',2,'Color','red');
    hold off
    %Note that 'hold on' in Line 45 retains plots in the current axes.
    
        %Identify coordinates & crop out the area of interest.
  topRow = min(min_y_line(:,2));
  bottomRow = max(max_y_line(:,2));
  croppedImage = grayscale2(topRow:bottomRow,:);
  figure, imshow(croppedImage);
   

[DGM]

Since the appearance of the working surface of the rail changes and has little consistent contrast (either color or intensity) against the flange/background (at least from this one sample), I don't know how you're going to do this reliably by edge detection or thresholding. The compression/noise isn't going to help either.

The presence of any clamps, bolted joints, transitions, crossings, switches, etc are potentially going to confound a simple automated routine, but again, I've only seen one picture.

If the camera position is fixed with respect to the rail, simple cropping may be possible, but I doubt the position is that well fixed. If the apparent width of the rail crown is the same in all frames, but drifts vertically with the lateral truck position, then maybe there could be something done to follow it with a fixed window.

Maybe someone else has a better idea than I do.

[Jun Han Tan]

Hi, thanks for the reply, yes the contrast is inconsistence which is why the threshold for the edge detection has to be change everytime i use a different frame. I have attached a few more images for reference. Yes i feel that fixed window for now is the best option, but im looking for a solution that can be flexible cropped according to where the rail surface is.

Answer 1

[Prachi Kulkarni]

Hi,

As I mentioned before, there are indeed many different approaches that can be taken to find the edges. Using the color information in the HSV color space was one approach. However, you can also detect the edges using Sobel edge detection.

Using the edge function with the Sobel method gives a binary image containing 1s where the function finds edges and 0s elsewhere. You can observe that there is a marked increase in the number of 1s near the required edges. This can be used to detect the edges as follows.

[file,path] = uigetfile('*');
if isequal(file,0)
    disp('User selected Cancel');
else
    disp(['User selected ', fullfile(path,file)]);
end
I=imread(fullfile(path,file));
figure, imshow(I);
BW = edge(rgb2gray(I),'Sobel');
profile = sum(BW,2)/size(I,2);
figure, plot(profile);
rows = find(profile > 0.11);
rows = rows([1 end]);
Icrop = I(rows(1):rows(2),:,:);
figure, imshow(Icrop);

This seems to work for all the images you have attached.

[Jun Han Tan]

Hi, thanks for the help, however the sobel edge detection still can't seem to work on some of my rail images, is there any threshold i can adjust to increase or decrease sensitivity of the detection so i can work around it to make it work for all my images?

[Prachi Kulkarni]

Hi,

The vector profile contains the proportion of edge pixels in each row of the image. Currently, the threshold for profile is set to 0.11. This means that the first row where profile is above 0.11 is the top edge and the last row where profile is above 0.11 is the bottom edge. This is shown in the code as follows.

rows = find(profile > 0.11);

You can change the threshold value of 0.11 to make it suitable for all the images you have.

Answer 2

[Prachi Kulkarni]

Hi,

There can be many possible approaches to identifying the edges. One possible way would be to look at the image in HSV color space. The saturation values of the image show a significant increase near the required edges.

Here is a sample code to plot the sum of saturation values for each row in the image. The plot shows the required edges are near the peaks in the plot.

[file,path] = uigetfile('*');
if isequal(file,0)
    disp('User selected Cancel');
else
    disp(['User selected ', fullfile(path,file)]);
end
I=imread(fullfile(path,file));
figure, imshow(I);
Ihsv = rgb2hsv(I);
saturation = Ihsv(:,:,2);
figure, imshow(saturation);
saturationProfile = sum(saturation,2);
figure, plot(saturationProfile);

For more information about color spaces, please refer to the following documentation.

Understanding Color Spaces and Color Space Conversion

[DGM]

I don't know how general that approach is, given that this rail section is orange because it has been recently dressed by grinding. A more weathered section will likely have a heavier, darker patina on the crown. Similarly, a freshly ground section would be all white.

[Jun Han Tan]

Hi Prachi,

Thanks for the assist. But may i know whats the next step after changing to HSV colour space. After getting the plot and the saturated image, i tried to use sobel and hough to get the cropping but it still doesnt detect the edges. Is there an another way to do it?