Creating a dataset: long calculation times

1 Ansicht (letzte 30 Tage)
Maya Pivert
Maya Pivert am 7 Jun. 2024
Kommentiert: Maya Pivert am 19 Jun. 2024
Hello,
I need to create a dataset to train a neural network. To do this, I vary three variables called sigma, alpha and beta. The problem is that today, the matlab code stops in the middle of compilation. So I tried replacing the for loop with a parfor loop, allowing me to do parrallel computing, but this time I got error messages like ‘Warning: Error in iteration 40: Index in position 1 exceeds array bounds. Index must not exceed 41888’ appear, whereas this was not the case before. You will find the associated codes below. If anyone has any clues as to how to get me out of this mess and save me some time, I'd love to hear from you. Thank you very much.
i = 0;
mat_1 = zeros(10000,11);
for sigma = 0:0.01:0.99
for alpha = 0.25:0.25:1
for beta = 0.1:0.4:1.3
[indices, observation] = LQ_LQe_Simu(sigma,alpha,beta);
m = length(observation);
store = indices(2) + 0.05;
if (indices(3) <= indices(1)) && (indices(4) <= store)
for j = 1:m
mat_1(j+i,1) = sigma;
mat_1(j+i,2) = alpha;
mat_1(j+i,3) = beta;
mat_1(j+i,4) = observation(j,1); % x
mat_1(j+i,5) = observation(j,2); % x_dot
mat_1(j+i,6) = observation(j,3); % y
mat_1(j+i,7) = observation(j,4); % y_dot
mat_1(j+i,8) = observation(j,5); % e_x
mat_1(j+i,9) = observation(j,6); % e_x_dot
mat_1(j+i,10) = observation(j,7); % e_y
mat_1(j+i,11) = observation(j,8); % e_y_dot
end
i = i+m;
end
end
end
end
mat_1 = mat_1(1:i, :);
function [indices, observation] = LQ_LQe_Simu(sigma,alpha,beta)
%% SYSTEM TO CONTROL %%
h=0.03;
sampleTime=h;
rho= 4.2;
v_0 = 0.0112; % [m/s]
theta_0 = pi/6;
x0 = [0 v_0*cos(theta_0) 0 v_0*sin(theta_0)]';
StopTime = 2*pi*20/beta;
Time=[0 StopTime];
R = 0.033;
L = 0.287;
vehicle = DifferentialDrive(R,L);
Qd=[rho^2 0 0 0; 0 0 0 0 ; 0 0 rho^2 0; 0 0 0 0];
Rd = diag([2 3]);
Ad = [1 h 0 0; 0 1 0 0; 0 0 1 h; 0 0 0 1];
Bd = [h^2/2 0; h 0; 0 h^2/2; 0 h];
Cd = [1 0 0 0; 0 0 1 0];
Dd = [0 0 0 0;0 0 0 0];
sysD= struct(...
'Ad' , Ad ,...
'Bd' , Bd,...
'Cd' , Cd,...
'x0' , x0 ,...
'alpha' , alpha ,...
'beta' , beta ...
);
[Kd,Pd,lambda]=dlqr(Ad,Bd,Qd,Rd) ;
Ed = Ad'*Pd*Ad-Pd;
Fd = Ad'*Pd*Bd*Kd;
Gd = Kd'*Bd'*Pd*Bd*Kd;
Qdbar = Qd+Kd'*Rd*Kd;
Fdbar = Ad'*Pd*Bd;
Gdbar = Bd'*Pd*Bd;
ctrlD = struct(...
'ui' , [] ,...
'Kd' , Kd ,...
'Pd' , Pd ,...
'Ed' , Ed,...
'Fd' , Fd ,...
'Fdbar' , Fdbar ,...
'Gd' , Gd ,...
'Gdbar' , Gdbar ,...
'Qdbar' , Qdbar ,...
'sigma' , sigma ...
);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Simu
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
simOut = sim("Commande_LQ_LQe_bloc_robot.slx", 'SrcWorkspace', 'current');
% Définition des structures de réponse
% Résultats commande LQ
LQ.td = simOut.x.time;
LQ.xd = simOut.x.signals.values(:,1:length(sysD.Ad));
LQ.ud = simOut.u.signals.values(:,1:size(sysD.Bd,2));
LQ.yd = simOut.y.signals.values(:,1:size(sysD.Cd,1));
LQ.nb = simOut.eventnb.signals.values(:,1);
%Résultats commande LQe
LQe.td = simOut.x.time;
LQe.xd = simOut.x.signals.values(:,1+length(sysD.Ad):2*length(sysD.Ad))
LQe.ud = simOut.u.signals.values(:,1+size(sysD.Bd,2):2*size(sysD.Bd,2));
LQe.yd = simOut.y.signals.values(:,1+size(sysD.Cd,1):2*size(sysD.Cd,1));
LQe.ed = simOut.event.signals.values(:,1);
LQe.nb = simOut.eventnb.signals.values(:,2);
% Extraction des instants des événements
instants = find([0; LQe.nb(2:end) - LQe.nb(1:end-1)]);
LQe.ti = LQe.td(instants);
LQe.xi = LQe.xd(instants);
%% INDICES OF PERFORMANCE %%
[IAE_TT, IAU_TT, IAE, IAU, IAEP, IAUP] = deal(0);
for i = 1:length(LQ.xd)
IAE_TT = IAE_TT + norm(simOut.xref([1,3], i)' - LQ.yd(i,:))*h; % Time-triggered
IAU_TT = IAU_TT + norm(LQ.ud(i,:))*h;
IAE = IAE + norm(simOut.xref([1,3], i)' - LQe.yd(i,:))*h; % Time-triggered
IAU = IAU + norm(LQe.ud(i,:))*h;
IAEP = IAEP + norm(LQ.yd(i,:) - LQe.yd(i,:))*h;
IAUP = IAUP + norm(LQ.ud(i,:) - LQe.ud(i,:))*h;
end
indices = [IAE_TT, IAU_TT, IAE, IAU, IAEP, IAUP, LQe.nb(end)];
observation = [LQe.xd simOut.e.data];
end

Antworten (1)

Simar
Simar am 17 Jun. 2024
Bearbeitet: Simar am 17 Jun. 2024
Hi Maya,
As per my understanding the error message you received indicates that there is an attempt to access an index of an array that exceeds its bounds, which is a common issue when dealing with parallel loops due to the non-deterministic order of execution. Here are a few workarounds and suggestions to resolve the issue:
1. Preallocate and Manage Array Indexing:
Ensure each iteration of loop writes to a unique portion of array. One way to manage this is by preallocating a large enough array and calculating indices in a way that they do not overlap between iterations. An alternative approach is to use cell arrays or structures where one writes to a separate cell or field, which are then concatenated or merged after the loop.
2. Use parfor with Independent Iterations:
Ensure iterations of parfor loop are independent. Error could be due to dependencies between iterations that are not apparent. Given the current indexing strategy is dependent on the cumulative sum of m, consider redesigning this part. A solution approach-
  • Independent Data Collection: Use parfor to collect data in a way that each iteration's output is independent and does not rely on previous iterations. This might involve using a cell array or struct array to store results.
  • Post-Processing: After collecting all data independently, perform any necessary sequential operations on the data outside of the parfor loop.
Here is a conceptual example to adjust the code:
% Assuming LQ_LQe_Simu returns vectors/arrays that can vary in size,
% Use a cell array to collect outputs
numIterations = ...; % Calculate the total number of iterations
results = cell(numIterations, 1);
parfor idx = 1:numIterations
[sigma, alpha, beta] = ...; % Calculate or retrieve sigma, alpha, beta for this iteration
[indices, observation] = LQ_LQe_Simu(sigma, alpha, beta);
% Store in a cell array or struct
results{idx} = struct('indices', indices, 'observation', observation, 'params', [sigma, alpha, beta]);
end
% Convert results from cell array to a structured format that you need
mat_1 = ...; % Initialize your matrix or other data structure
for idx = 1:numIterations
% Extract data from results{idx} and populate mat_1 or other variables
% This is where you would handle the logic currently inside your nested loops
End
Please refer to following documentation links-
3. Debugging and Validation:
Before running the full parfor loop, validate the approach with small number of iterations to ensure correctness. Ensure that any dynamic calculation of indices is correctly implemented, especially under the parallel execution model where iterations run in an unpredictable order.
Transitioning from serial to parallel computation, especially in complex data processing scenarios, requires rethinking the data flow and processing strategy to fit the parallel model. Ensuring data independence between iterations and correctly managing data aggregation post-parallel processing are key to leveraging “parfor effectively.
Hope it helps!
Best Regards,
Simar
  1 Kommentar
Maya Pivert
Maya Pivert am 19 Jun. 2024
Thank you for your reply. I'll see if I can manage with the advice you've given me. :)

Melden Sie sich an, um zu kommentieren.

Produkte


Version

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by