How to reshape bounding box when I rotate the face for live video ?

% Create the face detector object.

faceDetector = vision.CascadeObjectDetector();

% Create the point tracker object.

pointTracker = vision.PointTracker('MaxBidirectionalError', 2);

% Create the webcam object.

cam = webcam();

% Capture one frame to get its size.

videoFrame = snapshot(cam);

frameSize = size(videoFrame);

% Create the video player object.

videoPlayer = vision.VideoPlayer('Position', [100 100 [frameSize(2), frameSize(1)]+30]);

%% Detection and Tracking

% Capture and process video frames from the webcam in a loop to detect and

% track a face. The loop will run for 400 frames or until the video player

% window is closed.

runLoop = true;

numPts = 0;

frameCount = 0;

while runLoop && frameCount < 400

% Get the next frame.

videoFrame = snapshot(cam);

videoFrameGray = rgb2gray(videoFrame);

frameCount = frameCount + 1;

if numPts < 10

% Detection mode.

bbox = faceDetector.step(videoFrameGray);

if ~isempty(bbox)

% Find corner points inside the detected region.

points = detectMinEigenFeatures(videoFrameGray, 'ROI', bbox(1, :));

% Re-initialize the point tracker.

xyPoints = points.Location;

numPts = size(xyPoints,1);

release(pointTracker);

initialize(pointTracker, xyPoints, videoFrameGray);

% Save a copy of the points.

oldPoints = xyPoints;

% Convert the rectangle represented as [x, y, w, h] into an

% M-by-2 matrix of [x,y] coordinates of the four corners. This

% is needed to be able to transform the bounding box to display

% the orientation of the face.

bboxPoints = bbox2points(bbox(1, :));

% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]

% format required by insertShape.

bboxPolygon = reshape(bboxPoints', 1, []);

% Display a bounding box around the detected face.

videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon, 'LineWidth', 3);

% Display detected corners.

videoFrame = insertMarker(videoFrame, xyPoints, '+', 'Color', 'white');

end

else

% Tracking mode.

[xyPoints, isFound] = step(pointTracker, videoFrameGray);

visiblePoints = xyPoints(isFound, :);

oldInliers = oldPoints(isFound, :);

numPts = size(visiblePoints, 1);

if numPts >= 10

% Estimate the geometric transformation between the old points

% and the new points.

[xform, oldInliers, visiblePoints] = estimateGeometricTransform(...

oldInliers, visiblePoints, 'similarity', 'MaxDistance', 4);

% Apply the transformation to the bounding box.

bboxPoints = transformPointsForward(xform, bboxPoints);

% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]

% format required by insertShape.

bboxPolygon = reshape(bboxPoints', 1, []);

% Display a bounding box around the face being tracked.

videoFrame = insertShape(videoFrame, 'Polygon', bboxPolygon, 'LineWidth', 3);

% Display tracked points.

videoFrame = insertMarker(videoFrame, visiblePoints, '+', 'Color', 'white');

% Reset the points.

oldPoints = visiblePoints;

setPoints(pointTracker, oldPoints);

end

end

% Display the annotated video frame using the video player object.

step(videoPlayer, videoFrame);

% Check whether the video player window has been closed.

runLoop = isOpen(videoPlayer);

end

% Clean up.

clear cam;

release(videoPlayer);

release(pointTracker);

release(faceDetector);

I would like to reshape bounding box at my face. I mean when I rotate my face, I would like the width of bounding box reduce from rotation of my face. Is it possible ?

Thank you very much for your answer.

Answer 1

[Pujitha Narra]

Hi Sangchai,

I think the ‘CascadeObjectDetector’object returns only a square.

You can use the following code in place of the call to the ‘insertshape()’ function to achieve the change.

   [xlim,ylim] = boundingbox(polyshape(points.Location(:,1),points.Location(:,2)));
   tl=[xlim(1) ylim(1)];
   dr=[xlim(2) ylim(2)];
   tr=[dr(1) tl(2)];
   l=pdist([dr;tr]);
   w=pdist([tl;tr]);
   videoFrame = insertShape(videoFrame, 'Rectangle', [tl,w,l], 'LineWidth', 3);

[SP]

Thank you so much.