Image segmentation using two binary masks

The aim of my image segmentation task is to completely extract the object of interest. I have two image segmentation methods. With the first method I get a portion of the desired object (not all of it, somewhere around 10-50% of the total object) but there are no false positives in this image segmentation mask.

With the second method I get all of my desired object along with other false positives. Now I need to use these two images to completely extract my object of interest. I was thinking along the lines of using connected component analysis or using the first image as a seed to grow the region in the second image. Any thoughts or suggestions would be welcome (pertinent to the topic, of course).

Below I've attached the two images i am referring to. the bottom one is the minimal mask and the top one is the noisy mask.

[Omair]

Anybody, any useful ideas or tips on how I could go about tackling this problem. I tried correlation between the patch and the second image but not much luck.

Another idea i had was to create a mixture of gaussian using the patch from the second image and then calculating the probability of pixels from the first image belonging to this mixture. But without a false model I can't do much with this model.

Over here I've just shown the binary masks but i have the gray scale and the rgb images available as well.

Answer 1

[Image Analyst]

It would be useful to see the original image. But if you're sure you can get the large blob, knowing that the small blob is contained within it, then you can use imreconstruct(). It will take one of more "markers" in a binary image. Then, given a binary image which contains more blobs, some of which may contain the small marker blobs, and some blobs which don't contain any marker blob, then it will give you only the blobs that contain marker blobs and exclude the blobs that don't contain marker blobs.

[Omair]

That sounds like an interesting approach. I'll try that.

I've been reading up on self similarity based approaches, particularly this paper by Eli Shechtman ( http://www.wisdom.weizmann.ac.il/~vision/VideoAnalysis/Demos/SelfSimilarities/SelfSimilarities_ShechtmanIrani07.pdf) . Do you think such an approach could be useful or is it more oriented towards objects with a rigid structure.

[Image Analyst]

I just scanned the paper but it looks like it has impressive results. It's that kind of research that will help stock photography databases become more useful. It's just amazing the results they are getting now saying things like "show me pictures of babies" or beaches or whatever. Fast progress is being made in the CBIR field. You can look up CBIR in a search to find out more. You might also look up MSD or "Main Subject Detection" which attempts to figure out what the main subject in a scene is, for example the group of people in the foreground, not the building or mountains in the background.

[Omair]

imreconstruct is pretty useful. I'll have to adapt it to my method but it seems to be a promising path. Thanks for the help. These are the actual masks that i'm working with.

<<

[Omair]

I'm getting a problem with imreconstruct when i used it with two grayscale images. This is my code snippet:

im1 = mask1 .* gray_im;
  im2 = mask2 .* gray_im;
im = imreconstruct(im1,im2);

I'm getting this error MARKER pixels must be <= MASK pixels. im1 and im2 are both 349 by 629. The pixel values in im1 are the same as in im2 or they are 0. In the above images, im2 is the top one (noisy) and im1 is the bottom one.

Any idea what could be the problem here.

Edit: I only get this error when i'm multiplying the masks with the gray scale image. If i do this:

IM = imreconstruct(gray_im.*im1,gray_im);

then it works, the output is what i would expect from this operation. However i want to test it using the two masks.

[Image Analyst]

Why are you using grayscale images? You should use binary (logical) images.

[Omair]

I'm using binary images. I wanted to see the result on gray scale as well. The segmented portion in im1 (no noise/small segment) has a particular texture which is similar to the rest of the true positive region. With just a binary image all the area connected to the seed region is kept. Which includes a fair amount of false positives. Below is the image after using imreconstruct

[Image Analyst]

If your larger binary image is junk, then imreconstruct will pull out just one junky region of it. You need to improve your segmentation of it. You're probably doing just simple thresholding, which is often not a good method for color scenes.

[Omair]

The result shown above is from a GMM based segmentation. But when a colour based segmentation method is used there are bound to be objects with similar colour to the true class objects.

I tried working with a System identification approach (doretto et al). Using the detected patch as an identifier. The results of that were not as good as I expected.

Can you suggest any other patch based object recognition approaches.