MATLAB Answers

Yu Jia
0

Textscan to convert ASCII file to mat file

Asked by Yu Jia
on 17 May 2019
Latest activity Edited by per isakson
on 20 May 2019
I would like to be able to convert large ASCII file to .mat file for further data analysis and creating plots. My script below didn't generate error but also didn't give me the resutls that I want. C turned out to be an empty 1x15 cell array which basically means that it didn't do the job. I want to be able to convert the data file into .mat files that I can just plot by calling plot(C.TestTime,C.Amps).
My codes:
fileName='JL1_JWCEVCL12C1C204356W5P0K3.006';
fileID = fopen(fileName);
% formatSpec=['%d %d %d %{dd hh:mm:ss}D %{dd hh:mm:ss}D %f32 %f32 %f32 %f32 %s %d8 %u %f %f %s %f')];
formatSpec = ['%*d %d %d %s %s %d %f %f %f %f %s %d %s %f %f %f *s' repmat('%*d', [1,47]) '%*[^\t]'];
C = textscan(fileID,formatSpec,'Delimiter','\t');
fclose(fileID);
disp C
Results :
Part of the raw data file attached for test. The full file is large so I trimmed off a lot of data to make it <2MB.

  3 Comments

@Yu Jia: please upload a sample file by clicking the paperclip button.
@Stephen Cobeldick, sample file is uploaded. I accidengtally clicked the submit without being able to upload the sample file first then I realized I had to trim off quite a lot of data to make it smaller than 5MB to upload it. You are so fast! Impressive!
@Yu Jia: that tab-delimited file should be quite easy to import... except that it was saved with each line completely enclosed in double quotes:
"A B C ... X Y Z"
This is very unfortunate, as MATLAB's importing tools can trivially import CSV/TSV/SKV files... as long as they follow basic norms of such files (such as not making each line one text field in double quotes). Do you have any possibilty to save the file/s without the double quotes? (if this task is a one-off then the answer is likely "yes". If you have to process ten thousand such files, then this might not be an option for you...)

Sign in to comment.

1 Answer

Answer by Fangjun Jiang on 17 May 2019
Edited by Fangjun Jiang on 17 May 2019

You need to
  • skip two headerlines
  • not to specify 'Delimiter' as '\t'
  • carefully specify the format to match the file.
I got this result with the below changes.
formatSpec = ['"%d %d %d %s %s %s %s %f %f %f %f %s %d %s %f %f %f %s' repmat(' %f', [1,15]) repmat(' %d', [1,32]) '"'];
C = textscan(fileID,formatSpec,'Headerlines',2);
C =
1×65 cell array
Columns 1 through 3
{4997×1 int32} {4997×1 int32} {4997×1 int32}
Columns 4 through 6
{4997×1 cell} {4997×1 cell} {4997×1 cell}

  2 Comments

Answer by Yu Jia 12 minuter ago converted to this comment
Thanks so much for your response! I used your codes but still got the same results. C still shows as the empty array of 1×65 cell.
This works well on R2018b
%%
fid = fopen( 'JL5_JPC_P1_CL_12C2C0056W5CP.006.csv' );
formatSpec = [ '"%d%d%d%s%s%s%s%f%f%f%f%s%d%s%f%f%f%s' ...
, repmat('%f',[1,15]), repmat('%d',[1,32]),'"' ];
C = textscan( fid, formatSpec, 'Headerlines',2 );
fclose( fid );
%%
whos C
Name Size Bytes Class Attributes
C 1x65 5923728 cell
"C still shows as the empty array of 1×65 cell." Here C contains nearly 6 MB of data.
>> num = C{8};
>> whos num
Name Size Bytes Class Attributes
num 4997x1 39976 double
Proposal: Add 'ReturnOnError',false to the call of textscan(). That might give you a useful error message.

Sign in to comment.