Negative samples for training cascade object detector

Currently I am experimenting with training a cascade object detector (2017a version) and I am wondering about how the negative samples are generated from the negative images. I want to detect different types of push buttons on i.e. automotive steering wheels, so for the negative images i am using the positive images with the buttons cropped out (basically drawing a mono colored rectangle on top).

Both my original image and the negative image currently have a size of 4034x3024 pixel, and the size of positive instances is set to 30x32 automatically. What would a negative sample look like in this case? Does it randomly extract a 30x32 region as a negative sample from the negative images or would it also pick a larger area (possibly the entire image) and resize it to 30x32 pixel? If it was the latter case, I would prefer using blockproc to cut the image into smaller tiles and feed as negative data so I can easily omit the areas where buttons are located. Another benefit is that I can easily integrate negative samples I do not want the detector to confuse with buttons like automotive brand symbols.

I would appreciate any information on how it works.

Jimmy

[Rummy18]

Facing the same question...do we have a clear answer now? According to the documentation "training a cascade object detector", "the function runs the detector consisting of the stages already trained on the negative images. Any objects detected from these image are false positives...". Let's assume we have the large image mentioned in your question above (4034x3024 pixel). The detector is 30x32 pixel. The "trainCascadeObjectDetector" includes the input argument "ObjectTraningSize". It says, "before training, the function resizes the positive and negative samples to ObjectTrainingSize in pixels", i.e. to 30x32 pixels. The negative image and the detector now have the same size, they perfectly fit on top of each other, right? Hence, the detector checks only once whether it can find the desired object or not. This would mean that large images should be cut into smaller tiles to improve the negative samples, as the detector does not slide over the whole large image. Is this correct?

And a second question: If we have 100 negative images in our negative database and the training says "using at most 50 negative samples per stage". Does it mean, that 50 negative images are randomly selected from the database, giving the chance to use the images 51-100, or does it just use the images 1-50 for the whole training?

Thank you!

Answer 1

[Nachiket Katakkar]

Hi Jimmy,

I think that your understanding is accurate in that it seems to randomly select 30x32 sized regions from negative samples. The example shown here contains 42 positive images and only 5 negative images:

https://www.mathworks.com/help/vision/ref/traincascadeobjectdetector.html?s_tid=srchtitle#responsive_offcanvas

However, while training, since the "NegativeSamplesFactor" is set to 2 by default, it creates 84 negative samples from the 5 negative images.

You should be able to create further negative samples by omitting the buttons and place them in the negativeImages folder so that it does not obtain false positives. You have further control over the size of the trained object using the "ObjectTrainingSize" Name-Value pair.

[Jimmy]

Thank you for your response and sorry for my late reply.

That is what I did after after all. I now both included the full sized negative images and some smaller tiles in the negative image folder since I cannot seem to get more information on the exact sample generation. The source code documentation does not help me either, here is the relevant snipped extracted using 'type traincascadeobjectdetector':

function instances = createNegativeInstancesFromFolder(imagesFoldername)

    % createNegativeInstancesFromFolder Creates a struct array with information about 
    %  image names for all images that exist in the folder called foldername
    %  instances = createNegativeInstancesFromFolder(imagesFoldername)
    %  Returns a cell array that contains image filenames 
    %  All files in the folder that are readable with imread are included in the cell array.

    instances = [];
    numImages = 0;

Testing the impact of varying the negative sample generation (here full sized negatives vs. smaller tiles) on the detection performance is unfortunately not possible for me in the moment. Training time with my algorithm (around 6000 positive instances, and a total of above 360k negative images of different solution) exceeds 24 hours each (default training parameter with auto sized positive instances to 30 x 32 pixel). I yet have to increase the positive instance size (expected detection size is around 90 x 100), but that would also tremendously increase training time.