script running for 13 hours. should I be worried?
5 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
I have a function going through 9 excel files with about 1.7mil cells. It imports them, puts everything into one column then goes through and removes all the 0's and saves the output of about 300k cells. Its been running for about 13 hours of so now and I'm concerned that I may have messed up the code so I was hoping someone would help me out a fresh set of eyes.
it does look like its working, maybe? the output files look with theyve been edited, but I can't really be sure. this is the code i'm using at the moment.
function [ m ] = combine( input_args )
source = 'C:\Users\aoneil\Desktop\ODEN\trimmed'; %input the directory to choose the source folder
dest_dir = 'C:\Users\aoneil\Desktop\ODEN\graphs'; %choose where to output the resulting file
files = dir(fullfile(source, '*.CSV'));
for j = 1:length(files) %loops through all the file in the directory folder
a = xlsread(fullfile(source, files(j).name));
for x = 1 : size(a,1) %steps through the matrix
d((1+((x-1)*size(a,2))):(size(a,2)*x), 1) = a(x,1:end); %saves all the values of original to matrix "d"
end
n=1;
for y=1:size(d,1)
if d(y,1) > 0
m(n,1) = d(y,1)
n=n+1;
end
end
csvwrite(fullfile(dest_dir, files(j).name), m); %outputs matrix "d" to the file name/directory chosen
end
end
0 Kommentare
Akzeptierte Antwort
Jan
am 5 Feb. 2015
You urgently require a "pre-allocation". Letting arrays grow iteratively is extremely expensive. Example:
x = [];
for k = 1:1e5
x[k] = rand;
end
Now in each iteration a completely new vector x is created and the older values are copied. This means, that the OS has to reserve sum(1:1e5) * 8 bytes in total and copy almost the same amount of data in addition.
A vectorization solves the pre-allocation implicitly:
n=1;
for y=1:size(d,1)
if d(y,1) > 0
m(n,1) = d(y,1)
n=n+1;
end
end
Better without loop and IF:
m = d(d > 0);
0 Kommentare
Weitere Antworten (1)
Alessandro Masullo
am 5 Feb. 2015
You can use a task manager to check if matlab is still using the CPU. If not, it probably got stuck.
0 Kommentare
Siehe auch
Kategorien
Mehr zu MATLAB Parallel Server finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!