YOLOv2 Detector In Simulink

Is it possible to use a YOLOv2 detector to detect images from a multimedia file in Simulink? I've looked at the examples listed here:

https://www.mathworks.com/help/vision/computer-vision-with-simulink.html

But none of them feature loading in a trained YOLOv2 detector and using that in a simulink block.

I have tried myself to implement my model in Simulink using both a MATLAB function block and Stateflow to no avail. I am trying to develop a Simulink block that can take in an image, run the YOLOv2 detector algorithm on it, then output the detected image:

function exampleCommandDetectPartsARbot(rgbImg)
%#codegen
    % Needed to bypass code generation
    coder.extrinsic('exampleHelperARbotPickPlace')
    % ARbot contains configuration parameters and holds the YOLOv2 Detector in ARbot.DetectorModel
    ARbot = exampleHelperARbotPickPlace(rgbImg);
    
    % Initialize images
    rgbImg = zeros(216,216,3)
    resizedImg = zeros(224,224,3) 
    bboxes = 1; 
    labels = 1;
    % variable size arrays for bounding boxes and labels
    coder.varsize('bboxes',[50,4])
    coder.varsize('labels',[50,1])
    
    resizedImg = imresize(rgbImg,[224 224]);
    [bboxes,labels] = detect(ARbot.DetectorModel,resizedImg);
    
    if ~isempty(labels)
        labeledImg = insertObjectAnnotation(resizedImg,'Rectangle',bboxes,cellstr(labels));
        % imshow(labeledImg);
    end
    
    ARbot.Bboxes = bboxes;
    ARbot.Labels = labels;
end

MATLAB Function Block Error:

Attempt to extract field 'DetectorModel' from 'mxArray'

Stateflow Error:

Full assignment to data 'detect' is missing. Specify values for all elements of Stateflow chart locals and outputs. 
For more information, see Supported Operations for Vectors and Matrices. For structure types, a full assignment 
is not necessary when the type is specified as "Bus: typename" and not inherited or parameterized.

Is it possible to implement load a YOLOv2 detector into Simulink? If so how would one load the detector in and use the 'detect' function?

Thanks in advance!

Answer 1

[Raynier Suresh]

Hi, To avoid this error you can detect and annotate the image inside the extrinsic function itself. The below code will give you an example,

function y = fcn(u) %MATLAB Function Block
   y = ones(size(u));
   %The Extrinsic function "myFun" take the input image and provides annotated image as output
   coder.extrinsic("myFun"); 
   y = myFun(u);
end

function y = myFun(x) %Extrinsic function
    persistent net;
    if isempty(net)
        net = vehicleDetectorYOLOv2; %Pretrained YoloV2 available in MATLAB
    end
    [box,~,~] = detect(net,x); %Detect Vehicles
    y = double(insertObjectAnnotation(x,'Rectangle',box,'vehicle')); %Draw Bounding Box
end

Using the "From Multimedia File" block you can feed the input and vizualize the results using a "Video Viewer" Block.

As of R2020b release of MATLAB, you can use the MATLAB Function block as well as the Deep Learning Toolbox block library to simulate and generate code from trained deep learning models in Simulink directly. For more information refer the below link

https://www.mathworks.com/help/deeplearning/ug/lane-vehicle-detection-simulink-using-predict-block.html

[Lukas Strauß]

This answer has helped a lot.

I've had a simmilar issue where I've tried to get a yolov4 object detector to detect and annotate in a Simulink (R2023a) function block and I got code generation errors.

One thing I had struggels with was the line

y = double(insertObjectAnnotation(...));  

For some reason it made the video at the end nearly all white with floating bounding boxes.

Dropping the double() typcast and instead using

y = ones(size(u), 'uint8');

as means of getting input- and output-image matrices to be of the same type did resolve this issue