Fasten the speed of assigning values to a large tensor
4 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Hi! Now I have a for-loop assigning values to some entries of a large tensor at every iteration. But it is too slow since the tensor is large. Now I am thinking if there is any way to fasten this speed. One simple example is shown below. Here I used 'ndSparse' function to create a four dimensional tensor instead of 'zeros' function. The link of 'ndSparse' is https://www.mathworks.com/matlabcentral/fileexchange/29832-n-dimensional-sparse-arrays. ndSparse generalizes 'sparse' function from sparse matrix to high dimensional sparse tensor.
% Simple example
M = 500;
A = ndSparse.build([M,M,M,M]);
n = 10;
for i=1:n
disp(i)
idx1 = M/n*(i-1)+1:M/n*i;
idx2 = M/n*(i-1)+1:M/n*i;
idx3 = M/n*(i-1)+1:M/n*i;
idx4 = M/n*(i-1)+1:M/n*i;
tmp = randn([M/n,M/n,M/n,M/n]);
A(idx1,idx2,idx3,idx4) = tmp;
end
Is there any way to make it faster?
Thank you very much!
5 Kommentare
Antworten (2)
Jan
am 28 Apr. 2021
Bearbeitet: Jan
am 28 Apr. 2021
Avoid creating index vectors explicitly:
idx1 = a1:b1;
A(idx1, :) = tmp; % Slow
A(a1:b1, :) = tmp; % Faster
In the first case, each element of idx1 is tested, if it is a valid index:
% For each index something like this is performed:
if idx(1) == flor(idx(1)) && idx(1) > 0 && idx(1) < size(A, 1)
With a1:b1 Matlab checks the validity of the first and last element only.
Please post a minimal working example. Before the readers can test, if their suggestion is faster, they currently have to implement your code based on guesses, what
A = create a sparse all zero 6-D tensor of size (M,M,M,M,M,M);
or
idx1 = a1:b1; (b1-a1=m)
explicitly means.
2 Kommentare
Jan
am 3 Mai 2021
@Jason Yang: You are right, ndSparse is infact an exception, because this user defined class is not supoorted by Matlab's JIT acceleration.
Matt J
am 3 Mai 2021
Bearbeitet: Matt J
am 3 Mai 2021
You can save some time if you pre-allocate the array:
% Simple example
M = 400;
n = 10;
tic
A = ndSparse.spalloc([M,M,M,M],(M/n)^4*n); %Pre-allocate
%A=ndSparse.build([M,M,M,M]);
for i=1:n
idx1 = M/n*(i-1)+1:M/n*i;
idx2 = M/n*(i-1)+1:M/n*i;
idx3 = M/n*(i-1)+1:M/n*i;
idx4 = M/n*(i-1)+1:M/n*i;
tmp = randn([M/n,M/n,M/n,M/n]);
A(idx1,idx2,idx3,idx4) = tmp;
end
toc%Elapsed time is 1.282638 seconds.
I don't think there are great gains to be made, however. Most of the computation is being spent just generating the entries for the array:
clear B
tic;
[B{1:4}]=ndgrid(1:M/n);
B = reshape( cat(5,B{:}) ,[],4);
[C,vals]=deal(cell(n,1));
for i=1:n
C{i}=B+M/n*(i-1);
vals{i} = randn(prod([M/n,M/n,M/n,M/n]),1);
end
C=cell2mat(C); vals=cell2mat(vals);
I=sub2ind([M,M,M,M],C(:,1),C(:,2),C(:,3));
J=C(:,4);
toc; %Elapsed time is 1.499094 seconds.
tic
S=sparse(I,J,vals, M^3,M, (M/n)^4*n);
toc; %Elapsed time is 0.344042 seconds.
tic;
A=ndSparse(S,[M,M,M,M]);
toc;%Elapsed time is 0.016531 seconds.
0 Kommentare
Siehe auch
Kategorien
Mehr zu Data Type Conversion finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!