How to study the effectiveness of feature extraction ?

I am working on the extraction of fine detailed features in fingerprint mainly pores(white blobs). Now I need to validate the proposed extraction process,most of the research work checks it through state of art matching algorithms.I cannot implement those matching algorithms in the limited time available.How else can I justify my extraction process?

Answer 1

[Jeff E]

Manually identify the pores in a (small) set of images, and compare the two results. Ideally by looking at the overlapping and non-overlapping events and computing the confusion matrix.

Confusin matrix

[researcher]

How do you define overlapping and non-overlapping events?

[Jeff E]

When you manually define your pores, use roipoly to create a binary mask. You can then compare the manually annotated mask to the one you need to validate, with the manual mask serving as the ground truth.

true_positive = bwarea(manual_mask & automated_mask) ;
true_negative = bwarea(~manual_mask & ~automated_mask) ;
false_positive = bwarea(~manual_mask & auotmated_mask) ;
false_negative = bwarea(manual_mask & ~automated_mask) ;

[researcher]

     I = imread('exp.jpg');
figure, imshow(I)
BW = roipoly;

As per your answer I tried using roipoly.But when I try to circle a small pore,the output is a black screen.Ho can I create the mask?Attaching the image.

Answer 2

[Image Analyst]

The usual method is to construct an ROC curve.

See http://en.wikipedia.org/wiki/Receiver_operating_characteristic.

"ROC analysis since then has been used in medicine, radiology, biometrics, and other areas for many decades and is increasingly used in machine learning and data mining research........"

This is a good opportunity for you to learn about it.

[researcher]

Image Analyst, I studied about ROC,I have taken two parameters.One is the actual count of the number of pores counted manually,other is the through the algorithm.I have checked this under 5 threshold values.Now if I draft a ROC curve how should I calculate false positives?true positives would be the number of manually counted pores minus algorithm counted pores divided by manually counted pores?Can you please guide?