How to detect orientation of an object

Hello!

I've been stuck on this particular task for a while, so I thought I would ask the Computer Vision gurus on here. So far I've tried regionprops (specifically - orientation property of regionprops), which has been fairly poor for in terms of results. I was wondering if anyone has any tips or method for which I could calculate the orientation of a letter in the attached image. I've also attached an example image of what I would like to achieve.

Thank you.

Edit: Attached an input example image

Edit2: Attached imgmat.mat file containing the original image and a binary image.

Edit3: Attached shapeLetters.mat binary and an RGB image containing both letters and shapes for possible advice on clean segmentation of shapes and letters.

[Matt J]

It would be quickest if you attach the input image you are passing to regionprops, so we can play with it. It's not obvious why the orientation measurement wouldn't work, if your BW image is good.

[Teg]

Hi Matt!

I have now attached an example input image. I manually extract the centroid of these blocks via some index and area filtering.

[Matt J]

Please attach as a .mat file. Is this the direct input to regionprops? I'm looking for either the logical matrix BW or the label matrix L that you put into regionprops and got a bad Orientation measurement.

[Teg]

Oh. My apologies, Matt.

I have now attached an imgmat.mat file which contains the original image and a binary image which I created using masks/segmentation.

If possible, would you also be able to advice me how to best extract the letters in the image so I can find their centroids.

[Matt J]

Centroids? What happened to orientation?

[Teg]

Hi Matt! I am working on finding both the centroids and orientation. I was having terrible luck with calculation of the orientation, whereas with centroids I was having slightly better luck (but not perfect), hence why I included centroids later on. I will try your methods when I get home and let you know of my results :).

Answer 1

[Image Analyst]

First find the letters. Pretty easy. Just threshold and call imclearborder() to get rid of the background, then do a little cleanup with bwareafilt() or bwareaopen().

Then detect which letter is which with Hu's moments http://www.youtube.com/watch?v=Nc06tlZAv_Q or Deep Learning. Or else just get the orientation from regionprops() and look at the angle delta from a correctly oriented letter.

Answer 2

[Matt J]

If possible, would you also be able to advice me how to best extract the letters in the image so I can find their centroids.

This seems to do a decent job,

tmp=medfilt2(~binaryImg,[1,7]);
tmp=medfilt2(tmp,[7,1]);

BWsymbols=logical(imfill(tmp,'holes')-tmp);

props=regionprops(BWsymbols,'Centroid','Orientation');

imagesc(BWsymbols);axis image; shg

[Matt J]

Note, however, that you cannot use only the "Orientation" extracted for each symbol in the image alone. As Image Analyst said, you need a reference orientation value for each individual symbol. For example, you need to measure the orientation of an upright 'A' and compare to what you get in the image.

[Image Analyst]

props has their centroids. You can see them by printing them out

for k = 1 : length(props)
    xCentroid(k) = props(k).Centroid(1);
    yCentroid(k) = props(k).Centroid(2);
    fprintf('Centroid of blob #%d is at (%.1f, %.1f), and has orientation %.1f degrees.\n', k, xCentroid(k), yCentroid(k), props(k).Orientation);
end

[Matt J]

Their centroids, yes. Their orientations, only partially.

[Image Analyst]

Not sure what you mean Matt. Every blob has an orientation. Why partially? Why would some blobs not have their orientations included in the props structure?

[Matt J]

I'm just echoing what you said in your answer. The regionprops output will give you a field called props.Orientation, but there is no way to interpret that value in isolation. You need the orientation delta that you get from running regionprops on the letter both in the given position and in a correctly oriented reference position.

[Image Analyst]

OK, right. Actually I don't even know why orientation is needed. You can figure out what character it is without reorienting/rotating the letter into the upright position, so why bother?

[Teg]

Hi guys!

I need the orientation to know by how much the blob is rotated by for computer vision. I've been able to detect which letter is which via Deep Learning, but I still don't understand how I can calculate the orientation of each letter (blobs don't really matter since they are symmetrical). How could I go about this?

I haven't been able to try the codes that you guys provided me with yet, but I will let you guys know about the centroids when I get home.

Thank you so much! You guys are awesome.

[Image Analyst]

Teg, you first have to take an image with all the characters upright. Then determine the reference orientation. Then again with a test image and determine the test orientations. The difference in angles between then is the angle it's rotated by. Now orientation doesn't tell you if it's upside down or right side up, since those will both have the same angle. So you'd have to do something else to determine that but I'm sure you can think of some feature, like area fraction in each quadrant, or normalized cross correlation with the letter that you know it to be from your deep learning.

[Teg]

Thank you, Image Analyst. I shall give that a go!

[Image Analyst]

You're welcome, but again, why do you NEED the orientation? Saying you need it "to know by how much the blob is rotated by for computer vision." is like saying you need it because you need it, basically a non-answer. Just curious.

[Teg]

My current project will be used in another project as base code where I will be controlling a robot. For some part of the other project, it is required that I be able to detect the orientation of the objects within view of the camera on the robot. The actual specs haven't been laid out yet, only that I should be able to detect orientation of objects in vision.