How to import csv file of 200 rows by 4096 columns without truncation?

I am facing a problem trying to read in a csv file of 200 rows by 4096 columns. I have tried the following commands to read in the file: csvread, xlsread, load However, each time I noticed that I was only able to read in 200 rows by 256 columns to a data array. When I did a preview of the csv file in Matlab, I saw the message 'Preview truncated at twenty thousand characters'. If the number of rows is only 100 rows, then the number of columns that I am able to read in is 2048. It seems that there is a limit to the number of data values that Matlab will be able to read in from a file. May I know if there is a way to extend the memory space of Matlab so that I can read in data values from a csv file with more than 200 rows by 4096 columns?

Answer 1

[Niklas Nylén]

Please try importdata, http://www.mathworks.se/help/matlab/ref/importdata.html

[Derrick Tiew]

Try importdata just now, but still face the same problem. Data truncated at column 257. I believe there is a memory physical limit on how much data can be imported to Matlab from external file. When I tried to view the csv file in Matlab editor, I notice the message on column 257 stating 'Preview truncated at twenty thousand characters'. Any other workarounds on importing large data from external file to Matlab? Thanks.

[Niklas Nylén]

To me it sounds like it is only the preview that is truncated, not the data itself.

This works for me:

 % Write a csv file with random values, size 200*4096
 x = rand(200,4096);
 csvwrite('test.csv',x)

 a = csvread('test.csv');
 b = importdata('test.csv');

Except for small rounding errors, x, a and b are identical:

 >> max(max(abs(x-a)))

 ans =

   5.0000e-006

[Derrick Tiew]

Thanks for the info. Yup, it works all right now after a second try with the same csv file. All (cvsread, xlsread, load and importdata) can load in 200rows by 4096cols without issue. It was strange that we did fabricate 4096cols before in the csv file and was only able to load it half of the cols. Anyway, it works now.