How can I deploy a face tracking algorithm to Raspberry Pi 4?

I've created an algorithm to perform face tracking and recognition using a deep neural network. I've adapted the code in order to perform the code generation and it works if i run it from matlab. If i try to deploy it on the raspberry pi i always get some compilers error (see the question https://it.mathworks.com/matlabcentral/answers/887539-can-t-deploy-the-executable-on-raspberry-pi-and-matlab-coder-doesn-t-generate-the-elf-file) . I tried to split the code in different sections and to deploy them singularly: the section on the use of the deep neural network had no issues, but the section about the face detection and tracking gave me a lot of problems.

I've also tried this example https://it.mathworks.com/help/vision/ug/track-face-raspberry-pi2.html but i always had some problems related to the compilation.

Searching on the MATLAB forum i found that the Cascade Object Detector can only be used in simulations and cannot be used for code generation and deploy. How can i work around this problem?

Answer 1

[David Fink]

The Cascade Object Detector supports code generation, with some limitations:

https://www.mathworks.com/help/vision/ref/vision.cascadeobjectdetector-system-object.html#bs_hhwg-1_codegen

In particular, for deployment, see the information on using OpenCV libraries.

Regarding the face tracking example, please reach out to MathWorks Technical Support, who can assist with setup and configuration issues.

[Pietro Ventrella]

I'm using a supported compiler for opencv libraries (Visual Studio C++ 2019) as i previously received the relative error when i forgot to set it up using "mex -setup c++" command

The strange thing is that even if I try to deploy a function that only detects the face, I get errors.

"Regarding the face tracking example, please reach out to MathWorks Technical Support, who can assist with setup and configuration issues."

I have set up everything using the MATLAB Support Package for Raspberry Pi Hardware and i've followed all the steps mentioned in the example.

In addition I've also tried to adapt the code at the base of the example cited, so it can be released on the raspberry with the simple command "deploy", but the deployment fails.

The code is as follows

% Adaptation of the code faceTrackingARMKernel.m of the 'Face Tracking on ARM Target using Code Generation' example

function face_tracking_test()

%#codegen

persistent faceDetector

persistent pointTracker

persistent numPts

persistent oldPoints

persistent bboxPoints

%% Initialize persistent variables

% Create the face detector object.

if isempty(faceDetector)

faceDetector = vision.CascadeObjectDetector();

end

if isempty(numPts)

numPts = 0;

end

if isempty(oldPoints)

oldPoints = single([0 0]);

end

if isempty(bboxPoints)

bboxPoints = [0 0];

end

% Inizializzo l'input del raspberry (ho due possibilità)

raspiObj = raspi();

%cam = webcam(raspiObj,1);

cam = cameraboard(raspiObj,'Resolution','640x480');

% Ottengo un frame

videoFrame = snapshot(cam);

% Get the next frame.

videoFrameGrayFULL = rgb2gray(videoFrame);

% Resize frame

decimFactor = 3;

videoFrameGray = videoFrameGrayFULL(1:decimFactor:end,1:decimFactor:end);

% Create the point tracker object.

if isempty(pointTracker)

pointTracker = vision.PointTracker('MaxBidirectionalError', 2);

% Initialize tracker with dummy points

initialize(pointTracker, single([10 10]), videoFrameGray);

end

%% Creo un loop per gestire i frame

while true

videoFrame = snapshot(cam);

videoFrameGrayFULL = rgb2gray(videoFrame);

% Resize frame

decimFactor = 3;

videoFrameGray = videoFrameGrayFULL(1:decimFactor:end,1:decimFactor:end);

%% Detection and Tracking

if numPts < 10

% Detection mode.

bbox = faceDetector.step(videoFrameGray);

assert(size(bbox, 1) < 10);

if ~isempty(bbox)

% Find corner points inside the detected region.

points = detectMinEigenFeatures(videoFrameGray, 'ROI', bbox(1, :));

% Re-initialize the point tracker.

xyPoints = points.Location;

numPts = size(xyPoints,1);

if ~pointTracker.isLocked

initialize(pointTracker, xyPoints, videoFrameGray);

end

step(pointTracker, videoFrameGray);

setPoints(pointTracker, xyPoints);

% Save a copy of the points.

oldPoints = xyPoints;

% Convert the rectangle represented as [x, y, w, h] into an

% M-by-2 matrix of [x,y] coordinates of the four corners. This

% is needed to be able to transform the bounding box to display

% the orientation of the face.

bboxPoints = bbox2points(bbox(1, :));

% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]

% format required by insertShape.

bboxPolygon = reshape(bboxPoints', 1, []);

% Display a bounding box around the detected face.

videoFrameOut = insertShape(videoFrame, 'Polygon', bboxPolygon.*decimFactor);

% Display detected corners.

videoFrameOut = insertMarker(videoFrameOut, xyPoints.*decimFactor, '+', 'Color', [255 255 255]);

else

videoFrameOut = videoFrame;

end

else

% Tracking mode.

[xyPoints, isFound] = step(pointTracker, videoFrameGray);

assert(size(xyPoints, 1) < 500);

visiblePoints = xyPoints(isFound, :);

oldInliers = oldPoints(isFound, :);

numPts = size(visiblePoints, 1);

if numPts >= 10

% Estimate the geometric transformation between the old points

% and the new points.

[xform, ~, visiblePoints] = estimateGeometricTransform(...

oldInliers, visiblePoints, 'similarity', 'MaxDistance', 4);

% Apply the transformation to the bounding box.

bboxPoints = double(transformPointsForward(xform, bboxPoints));

% Convert the box corners into the [x1 y1 x2 y2 x3 y3 x4 y4]

% format required by insertShape.

bboxPolygon = reshape(bboxPoints', 1, []);

% Display a bounding box around the face being tracked.

videoFrameOut = insertShape(videoFrame, 'Polygon', bboxPolygon.*decimFactor);

% Display tracked points.

assert(size(visiblePoints, 1) < 500);

videoFrameOut = insertMarker(videoFrameOut, visiblePoints.*decimFactor, '+', 'Color', [255 255 255]);

% Reset the points.

oldPoints = visiblePoints;

setPoints(pointTracker, oldPoints);

else

videoFrameOut = videoFrame;

end

end

%Mostro l'immagine in uscita

displayImage(raspiObj, videoFrameOut);

end

end

[Pietro Ventrella]

@David Fink

Today i've re-installed raspberry pi operative system via its hardware support package (in particular I've downloaded "mathworks_raspbian_DL_R21.1.0") and I found the files of the example mentioned yesterday pre-installed inside the raspberry in a folder named "mathworks". The executable of the example works fine, but trying again to deploy a simple script from MATLAB again gives me an error

function faceDetector()

%#codegen

raspiObj = raspi();

cam = cameraboard(raspiObj, 'Resolution', '640x480');

videoFrame = snapshot(cam);

videoFrame = imrotate(videoFrame, 180);

videoFrameGray = rgb2gray(videoFrame);

detector = vision.CascadeObjectDetector();

bbox = detector.step(videoFrameGray);

videoFrameOut = insertShape(videoFrame, 'Rectangle', bbox);

displayImage(raspiObj, videoFrameOut);

[David Fink]

Hi Pietro,

If requesting assistance with an error, be sure to include the steps you followed to reach the error, and the text of the error message (and error stack, if displayed).

MathWorks Technical Support can work with you to determine the cause of the error.

Best,

David

[Pietro Ventrella]

@David Fink

I've linked another question i've made some time ago and there are described all steps that brought me to those errors.

In the comments there is also the error stack.

If you check, you have already posted an answer on that question ( https://it.mathworks.com/matlabcentral/answers/887539-can-t-deploy-the-executable-on-raspberry-pi-and-matlab-coder-doesn-t-generate-the-elf-file )

Answer 2

[Rilwan Shekoni]

Hello @Pietro Ventrella, I am having a similar issue. Have you found a resolve?

Code.............

function rpiFace %#codegen

%Create raspi & webcam obj

raspiObj = raspi();

cam = webcam(raspiObj,2);

%Initialize DNN and the input size

net = coder.loadDeepLearningNetwork('FaceNet.mat');

inputSize = [227, 227, 1]; %net.Layers(1).InputSize(1:2);

%faceDetector = vision.CascadeObjectDetector;

%faceDetector.MergeThreshold = 8;

bboxes = [0,0,100,100];

text = 'No face detected';

start = tic;

fprintf('Initialing....\n');

while true

%Capture image from webcam

img = snapshot(cam);

imgSet = rgb2gray(img);

elapsedTime = toc(start);

%Process frames at 1 per second

if elapsedTime > 1

%Resize the image

imgSize = imresize(imgSet,inputSize(1:2));

%Classify the input image

[label,score] = classify(net,imgSize);

maxScore = max(score);

strlabel = cellstr(label);

text = sprintf('%s %f',strlabel{:},maxScore);

%bboxes = faceDetector(img);

start = tic;

end

%Display the predicted label

img = insertObjectAnnotation(img,'rectangle',bboxes,text,'TextBoxOpacity',0.9,'FontSize',8);

%img = insertShape(img,'rectangle',(bboxes));

%img = insertText(img,[0,0],text);

displayImage(raspiObj,img);

end

end

Error........

Error executing command "touch -c /home/pi/MATLAB_ws/R2020b/C/Users/RilSh/Documents/MATLAB/DL_Rpi/codegen/exe/piTestB/*.*;make -f piTestB_rtw.mk all MATLAB_WORKSPACE="/home/pi/MATLAB_ws/R2020b" -C /home/pi/MATLAB_ws/R2020b/C/Users/RilSh/Documents/MATLAB/DL_Rpi/codegen/exe/piTestB". Details:

........

??? Build error: C++ compiler produced errors. See the Build Log

for further details.

More information

Code generation failed: View Error Report

[Pietro Ventrella]

Unfortunately no...