What is the fastest way of writing large Simulink output data to csv file

2 Ansichten (letzte 30 Tage)
swenia
swenia am 12 Jun. 2018
Kommentiert: Jan am 12 Jun. 2018
I've run 1000 simulations with very small time steps in Simulink and stored the data in Simulink.SimulationOutput. I'd like to extract some info from this output and write it to a CSV file for each case. I tried creating a table for each simulation in a for loop and used the following command to save 1000 CSV files:
filename = fullfile('Results', filesep, sprintf('out_%d.csv',j));
writetable(T, fname)
Apparently for some simulations I have uneven numbers of rows so Table didn't work for me. I don't know which cases have uneven numbers, and I ran my simulation on a cluster. So to inspect that I then I tried saving the output with the following command with an intention to carry it to my computer to find out which simulation result(s) causes the problem.
save('results', 'out', '-v7.3')
The problem is that this file is ending up to be much larger than I expected (it's still running and the last time I checked it was going over 15 GB). Even after saving this, I won't be able to open it on my PC due to the file size. And I'd like to process the data with another software, so I'd like to save each simulation as a separate CSV file. What is the fastest way to do this for a handful of variables with different sizes?
PS: I don't want to decrease my simulation sample time any further, so I have to work with what I have right now.
  3 Kommentare
swenia
swenia am 12 Jun. 2018
Jan, thank you for your comment. Per your suggestion, I tried saving each case as a separate MAT file and that was really fast! But the problem is, I still need it as a txt or a csv format because the software I need to process this data with does not read MAT files.
Jan
Jan am 12 Jun. 2018
Does it accept another type of binary file formats?

Melden Sie sich an, um zu kommentieren.

Antworten (0)

Kategorien

Mehr zu Large Files and Big Data finden Sie in Help Center und File Exchange

Produkte


Version

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by