Read large number of .h5 datasets

3 Ansichten (letzte 30 Tage)
Ashraf El Gaaly
Ashraf El Gaaly am 15 Aug. 2018
Kommentiert: per isakson am 28 Okt. 2018
Hi all,
I'm working with these h5 files that have tens of thousands of datasets, that contains vectors of numerical values and all of the same size. My goal is to read the datasets and create one large matrix from these vectors. The datasets are named 0-xxxxx I was able to read them and get the matrix but it takes forever to do so. I was wondering if you can take a look at my code and suggest a way to make it run faster
t =[];
for i = 0:40400 % there are 40401 datasets in this particular file
j = int2str(i);
p = '/mesh/'; % The parent group
s = strcat(p,j); % to create the full path of a dataset e.g. '/mesh/0'
r = h5read('temp.h5',s); % the file name is temp and s has the dataset path
t = [t;r];
end
in this file there are 40401 datasets, each has 80802x1 vector of numerical values. Therefore eventually I want to create 80802x40401 matrix. This code has been running for over 6 hours now and still not done. I tried another smaller file yesterday with 10801 datasets and it took about 4 hours to get the answer.
  3 Kommentare
Ashraf El Gaaly
Ashraf El Gaaly am 24 Okt. 2018
Yes I have enough ram. The program takes over a day to run, but eventually runs
per isakson
per isakson am 28 Okt. 2018
Did you read and understand the text on Preallocation?

Melden Sie sich an, um zu kommentieren.

Antworten (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by