Opening csv file using textscan returns [0x1 double]

I originally had a code that opened text files so that I could read the data: here is an example data file:

Frequency/Hz  ZRe/Ohm  ZIm/Ohm
25000          123.973  -55.621
22471.91   130.936  -58.299
20000         131.607 -61.111
17977.53   132.36  -66.828
16064.26   134.676  -70.063
14466.55   137.25  -75.438
12841.09   138.59  -82.377
11494.25   141.42  -83.387
10335.92   142.741  -96.216
9195.4   151.153  -100.342
8205.13   146.893  -111.375
7326.01   152.058  -121.697
6568.14   157.189  -133.012
5882.35   160.671  -145.992
5259.7   163.611  -160.421
4705.88   166.562  -176.182
4203.89   169.427  -193.571
3766.48   170.511  -212.774
3361.34   173.806  -235.36
3013.18   179.252  -258.61

and the beginning of the code:

function [zizr]= getdata3(filename)
currentfolder=pwd
PathStr=pwd;
textFiles=dir([PathStr '*.txt'])
MyData=zeros(100,22); %assume data has 100 rows
for k = 1:3;
  length(textFiles)
  textFilename = [filename num2str(k) '.txt']
  %Open text file, 'rt' is for reading a text file only
  fid=fopen(textFilename, 'rt')
  %Read file into 3 seperate variables
  Data = textscan(fid,'%f %f %f','Headerlines',1)

that code works fine, however now I would like to open csv files that have 5 headerlines:

Type:,FixedPotential 
Date and time:,2015-09-02 08:56:49
Title:,CH1-090215SynthUrine-3
series:,0.060,0.029
freq / Hz,Z' / Ohm,Z'' / Ohm,
25000.00,123.973,-55.621,
22471.91,130.936,-58.299,
20000.00,131.607,-61.111,
17977.53,132.360,-66.828,
16064.26,134.676,-70.063,
14466.55,137.250,-75.438,
12841.09,138.590,-82.377,
11494.25,141.420,-83.387,
10335.92,142.741,-96.216,
9195.40,151.153,-100.342,
8205.13,146.893,-111.375,
7326.01,152.058,-121.697,
6568.14,157.189,-133.012,
5882.35,160.671,-145.992,
5259.70,163.611,-160.421,
4705.88,166.562,-176.182,
4203.89,169.427,-193.571,
3766.48,170.511,-212.774,
3361.34,173.806,-235.360,
3013.18,179.252,-258.610,

I would like to do the exact same thing as in my previous code, just with csv files. Here is what i have but it doesnt seem to work.

    function [zizr]= getdata3(filename)
    currentfolder=pwd
    PathStr=pwd;
    textFiles=dir([PathStr '*.csv'])
    MyData=zeros(100,22); %assume data has 100 rows
    for k = 1:3;
      length(textFiles)
      textFilename = [filename num2str(k) '.csv']
      %Open text file, 'rt' is for reading a text file only
      %fid=fopen(textFilename, 'r')
      fid=fopen(textFilename,'rt')
      %Read file into 3 seperate variables
      Data = textscan(fid,'%f %f %f','Delimiter',',','headerlines',5)

I keep getting : Data =

    [0x1 double]    [0x1 double]    [0x1 double]

I am mainly concerned with the textscan inputs I have and if something is wrong there.

[dpb]

I pasted a small subset of your data above into a file and

>> fid=fopen('alano.csv','r');
>> d=textscan(fid,'%f %f %f','delimiter',',','headerlines',5)
d = 
  [7x1 double]    [7x1 double]    [7x1 double]
>> fid=fclose(fid);

worked just fine.

Several questions/comments, though...

1. What's the k loop 1:3 for? Will overwrite Data as written unless there's other code to somehow sort things out; move it to some other variable.
2. Why use dir then ignore the results and build a filename manually? Why not just iterate over the files found (with a suitable wildcard based on the passed in file name to be selective)?
3. Have you checked the file(s) are actually there? You didn't check the result of fopen that a valid file handle was returned; mayhaps that's the problem with the data being missing...

[Alano Ogata]

1.) the k loop is for 3 .csv files that I want opened one after another. in the loop it extracts some of the data and stores and once the loop has gone through 3 times I have an array made up of data from the 3 files. 2.)my fopen is giving me a valid output, so the file is there I believe.

I have no idea why this is not working. I copied my data again into a simple file and tried exactly what you did and still ended up with this:

>> fid=fopen('alanao.csv','r')

fid =

      35

>> d=textscan(fid,'%f %f %f','delimiter',',','headerlines',5)

d = 

      [0x1 double]    [0x1 double]    [0x1 double]

The file is saved as a .csv excel sheet. Could that have anything to do with it?

[dpb]

"fid = 35"

WOWSERS!!!! You've got another 32 files already open????? How about a

close all

and then retry this? I don't know what practical limits there are on open file handles, but that's an inordinate number to have open all at once. If that isn't just a remnant of not having closed a bunch, that's a bad habit; one can lose data that way. If it's deliberate, the outline of the problem needs serious rework.

If it's unrecognized, it's quite possible the file you're trying to access is already opened by another handle and you're just not accessing it or it has already been read and is at the EOF mark so there's no further data after the present file position, all sorts of possible issues.

"The file is saved as a .csv excel sheet...."

You mean the file is in Excel? If so, then yes, that'll have a lot to do with it. Use xlsread instead.

If it has, indeed, been save outside of Excel as a CSV file, unless there's some option that also saves other information than just the data then it should work just fine.

But, let's go back to the first issue and resolve that first; I have a strong hunch therein "there be dragons!".

[Alano Ogata]

It still didnt work :(

>> fid=fopen('090415_D2_PBS-1.csv','r')

fid =

       5

>> d=textscan(fid,'%f %f %f','delimiter',',','headerlines',5)

d = 

      [0x1 double]    [0x1 double]    [0x1 double]

My data files are saved as a .csv through another program. I run measurements and the software saves the files as .csv so I don't htink xlsread is what I need.

Here is one my files:

[per isakson]

The problem is caused by the encoding of the file. See help on fopen.

[Walter Roberson]

If the encoding of the file is a problem try

fid=fopen('090415_D2_PBS-1.csv','rt');

If that is not enough then you an try

    fid=fopen('090415_D2_PBS-1.csv','rt', 'ieee-le', 'UCS-2');

and if it complains about UCS-2 not being known, try 'UTF-16' instead (UTF-16 expands on UCS-2 to allow additional code points)

[Alano Ogata]

Thanks for all the help! it was an encoding problem.

    fid=fopen('090415_D2_PBS-1.csv','rt', 'ieee-le', 'UCS-2');

fixed the problem.

Thanks

Answer 1

[per isakson]

A bit of googling in combination with trial and error

>> cac = cssm('090315_D1_20BSA-3.csv')
Warning: The encoding 'UTF-16LE' is not supported.
See the documentation for FOPEN. 
> In cssm at 2 
cac = 
    [20x1 double]    [20x1 double]    [20x1 double]

where

function    zizr = cssm( filespec )
    fid = fopen( filespec, 'r', 'l', 'UTF16-LE' );
    str = fread( fid, '*char' )';
    fclose( fid );
    zizr = textscan( str(3:end), '%f%f%f', 'Delimiter', ',', 'headerlines', 5 );
end

• Reference: Working with Low Level File I/O and Encodings, Loren Shure, September 20, 2006
• str(3:end) skips the BOM-character

&nbsp

*This should work, but it doesn't :-(* &nbsp (Can anyone explain why?)

>> cac = cssm('090315_D1_20BSA-3.csv')
Warning: The encoding 'UTF-16' is not supported.
See the documentation for FOPEN. 
> In cssm at 2 
Warning: The encoding 'UTF-16' is not supported.
See the documentation for FOPEN. 
> In cssm at 4 
cac = 
    [0x1 double]    [0x1 double]    [0x1 double]

where

function    zizr = cssm( filespec )
    fid = fopen( filespec, 'rt', 'ieee-le', 'UCS-2');
    fseek( fid, 2, 'bof' ); % skip BOM-character
    zizr = textscan( fid, '%f%f%f', 'Delimiter', ',', 'headerlines', 5 );
    fclose( fid );
end

fread honors the encoding settings, but textscan doesn't ???

Answer 2

[shannon stoffel]

[filename1,filepath1]=uigetfile({'*.*','All Files'}, 'Select Data File 1');
cd(filepath1);
rawdata1=dlmread(filename1);
% Opens the file 
M = csvread(filename1,1,1);

[dpb]

dlmread and csvread won't handle the headers; they require numeric data only.

[Walter Roberson]

I noticed recently that dlmread() now has an example of a non-numeric header.

[dpb]

Hmmmm....so the doc now says. Besides the "Introduced" date, such major functionality changes should also be in the footnotes as to when it is introduced (or vice versa, when removed as sometimes also happens).