For loop optimization and performance increase

Hi, I am still new to matlab environment and I have been developing this CAN message interpret app.

The app is to read CAN message and actually creates a new timetable that interprets the CAN message to string information like the unit and definition; for example, 1 = off and 0 = on or 1V if there is no definition.

The code below is the code to do the workflow that I presented but it takes about 70 - 90 seconds to complete the task...

I am not sure if it is due to the log file that I have been working with since the log is about 30 mins long log.

Or maybe this for loop can be optimized to make this loop go faster.

Can anyone help me with optimizing this loop? Thanks in advance!

I have attached an example CAN message file and other essential database as a zip file.

app.MsgName = B4_ST0
numMsg = height(app.(app.MsgName));

            numberofmessage = width(app.(app.MsgName));
            numberofstrings = repmat({'string'}, 1, numberofmessage);
            columnNames = app.(app.MsgName).Properties.VariableNames;

            app.Status = timetable('Size',[numMsg numberofmessage],'VariableTypes',numberofstrings,'RowTimes',app.(app.MsgName).Time,'VariableNames', columnNames);

            for j = 1:numel(columnNames)
                ValueTable_Extract = signalInfo(app.canDB,app.MsgName, columnNames{j});
                ValueTable_Extracted  = convertCharsToStrings(ValueTable_Extract.ValueTable);
                Unit_Extracted = convertCharsToStrings(ValueTable_Extract.Units);
                ValueTable_Emptiness = isempty(ValueTable_Extracted);
                if ValueTable_Emptiness == 0
                    for ii = 1:numMsg
                        app.Status.(columnNames{j})(ii) = valueTableText(app.canDB,(app.MsgName), columnNames{j},app.(app.MsgName).(columnNames{j})(ii));
                    end
                elseif ValueTable_Emptiness == 1
                    for ii = 1:numMsg
                        app.Status.(columnNames{j})(ii) = "Unit: " + "'" + Unit_Extracted + " '";
                    end
                else
                    for ii = 1:numMsg
                        app.Status.(columnNames{j})(ii) = "Check the DBC File";
                    end
                end
            end

Answer 1

[Abhas]

Hi Min,

Parallel processing in MATLAB can significantly speed up operations that are independent across iterations, like processing CAN messages. MATLAB's Parallel Computing Toolbox allows you to execute loops in parallel using "parfor" instead of "for", distributing iterations across available workers in a parallel pool. Some of the pre-requisites to use parfor are:

1. Ensure Parallel Computing Toolbox is Installed: Check if you have the Parallel Computing Toolbox installed and configured.
2. Data Transfer Consideration: Be aware that using parallel processing involves transferring data from the client MATLAB session to the workers. If the data to be transferred is large, this can offset the performance gains. Optimize data transfer by minimizing the data needed by each worker.

The optimized MATLAB code using parallel processing is as follows:

% Assuming app.MsgName, app.canDB, and other properties are pre-defined
numMsg = height(app.(app.MsgName));
numberofmessage = width(app.(app.MsgName));
columnNames = app.(app.MsgName).Properties.VariableNames;

% Pre-allocate timetable with default values
app.Status = timetable('Size',[numMsg, numberofmessage],'VariableTypes',repmat({'string'}, 1, numberofmessage),'RowTimes',app.(app.MsgName).Time,'VariableNames', columnNames);

% Start a parallel pool if not already started
if isempty(gcp('nocreate'))
    parpool; % Adjust based on your system's capabilities
end

% Parallel loop over each column (signal)
parfor j = 1:numel(columnNames)
    ValueTable_Extract = signalInfo(app.canDB, app.MsgName, columnNames{j});
    ValueTable_Extracted  = convertCharsToStrings(ValueTable_Extract.ValueTable);
    Unit_Extracted = convertCharsToStrings(ValueTable_Extract.Units);
    ValueTable_Emptiness = isempty(ValueTable_Extracted);
    
    % Initialize a temporary variable for storing results
    tempResults = strings(numMsg, 1); % Pre-allocate for speed
    
    if ValueTable_Emptiness == 0
        % Assuming valueTableText cannot be vectorized and needs to be called per message
        for ii = 1:numMsg
            tempResults(ii) = valueTableText(app.canDB, app.MsgName, columnNames{j}, app.(app.MsgName).(columnNames{j})(ii));
        end
    elseif ValueTable_Emptiness == 1
        tempResults(:) = "Unit: '" + Unit_Extracted + " '";
    else
        tempResults(:) = "Check the DBC File";
    end
    
    % Assign the results outside the parfor loop to avoid communication overhead
    app.Status.(columnNames{j}) = tempResults;
end

The optimized code runs 3 times faster than the original code by using the above methods.

You may refer to the following documentation links to have a better understanding on parallel processing:

1. https://www.mathworks.com/help/parallel-computing/parallel-for-loops-parfor.html
2. https://www.mathworks.com/help/parallel-computing/index.html

[Min]

I understand that Matlab has this toolbox to increase the processing speed but since I am trying to achieve without purchasing other toolbox I wanted to see if there are other ways to make this for loop to be more efficient.

Would you be able to provide some ideas what I might be able to do without the toolbox?

Thanks!