create multiple tables from a CSV in a for loop

hello everyone,
I have a large CSV file (too large to upload ~ 50 GB) that I stored in a table. the table columns are time,energy.
I'd like to save segments from this table into separate tables. for that I need the new table name to be changed in some way, lets say:
T=1000;
for i = 1:6
segment = find(old_table.time<T);
new_table(i) = old_table(segment,:); %note
T = T+1000;
end
the row with the note is what I'm trying to do but not sure how.

5 Kommentare

Stephen23
Stephen23 am 14 Sep. 2021
Bearbeitet: Stephen23 am 14 Sep. 2021
"for that I need the new table name to be changed in some way"
No, you don't need to change the table name.
Changing the table name would be a slow, more complex, and very inefficient approach.
The neat, simple, and very efficient solution would be to use basic MATLAB indexing and a cell array.
"the row with the note is what I'm trying to do but not sure how."
As an alternative you could just use one tall array: https://www.mathworks.com/help///matlab/large-files-and-big-data.html
sani
sani am 14 Sep. 2021
thanks for your answer!
the problam is that when I'm trying to write it into a predetermined call array if fails since not all segments contain the same number of rows. I predetermined the row in the array to be 1E7 (which is more than the max segment I'm expecting).
the error is "unable to perform the assingment because the size of the left side in 1e7-by-2 and the right side is 56468-by-2"
I don't think you need to predetermine the segment size to store the table in cell array. You can just edit your code as follows.
new_table{i} = old_table(segment,:);
left_side(1:size(right_side,1), 1:size(right_side,2)) = right_side;
sani
sani am 19 Sep. 2021
Thank you all, it solved my problam!

Melden Sie sich an, um zu kommentieren.

Antworten (0)

Kategorien

Mehr zu Matrices and Arrays finden Sie in Hilfe-Center und File Exchange

Gefragt:

am 14 Sep. 2021

Kommentiert:

am 19 Sep. 2021

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by