How to ignore some data?

Hi all!

I have files which contain 2055 columns. I want to divide the columns 8:2055 with the first column. Then, I want to ignore the 1st and 2nd columns and I save the results. I have the script:

format1='%*f %*f %f %f %f %f/%f/%f %f '; %gia na agnoei ta IT & filter
for i=1:2048
  if i>=1 && i<=44
    format1=[format1 '%*f '];
  else 
    format1=[format1 '%f '];
  end %gia na agnoei ta wl<300nm
end

files=dir('*spe');
for fl=1:length(files)
  fid=fopen(files(fl).name,'r');
  Phaethon3_data=textscan(fid,format1);
  DirectSun=cell2mat(Phaethon3_data);
  fclose(fid);
  format2='%2.6f %3.6f %2.6f %d/%d/%d %2.6f';
  for j=1:2004
    format2=[format2 ' %1.6e'];
    if j==2004
      format2=[format2 '\n'];
    end %gia na allazei grammi
  end
  qdoas_filename=files(fl).name;
  fid = fopen(fullfile( ...
      ['C:\Users\fanigk\Desktop\DirectSun' '\' qdoas_filename]),  ...
      'wt'); %sto qdoas_path dineis to path pou 8eleis na apo8ikeftoun ta arxeia
  fprintf(fid, format2, DirectSun'); 
  fclose(fid);
end

My script ignore the 1st and 2nd colums, but i can not divide the others columns with the 1st. How can i do this?

Thank you!

[Image Analyst]

Exactly what does "divide the columns 8:2055 with the first column" mean??? Please state what columns you want extracted and put into some output array.

Answer 1

[dpb]

First just some "tricks" on writing formatting strings --

format1='%*f %*f %f %f %f %f/%f/%f %f '; %gia na agnoei ta IT & filter
for i=1:2048
if i>=1 && i<=44
  format1=[format1 '%*f '];
else 
  format1=[format1 '%f '];
end %gia na agnoei ta wl<300nm
end

Can be written as

fmt1=[repmat('%*f',1,2)  repmat('%f',1,3) '%f/%f/%f %f' ...
      repmat('%*f',1,44) repmat('%f',1,2004)];

repmat is extremely useful for the repeat counts instead of having to try to count manually or use loops and building piecemeal...not your problem but a useful thing to see...

While on the subject of format strings, the format '%2.6f' and similar probably isn't doing what you want. The .6 will place 6 digits after the decimal but the width of '2' isn't doing anything since it's less than the number of precision digits. If you're trying to align numeric values, then you need a width that's at least 6+decimal+sign+space for the column.

And, you can use the output of functions as input to succeeding ones in Matlab so

Phaethon3_data=textscan(fid,format1);
DirectSun=cell2mat(Phaethon3_data);

could be written simply as

DirectSun=cell2mat(textscan(fid,format1));

eliminating the superfluous cell array. (I've wished for and requested enhancement that textscan would allow a direct return of data arrays instead of cells where not needed but no joy so far...)

To divide columns by the first you just use the "dot" operator for element-by-element as

DirectSun(:,8:end)=DirectSun(:,8:end)./DirectSun(:,1);

I've used 8:end as shorthand for the columns beginning in column 8. However, which columns you actually want of the 2055 that you say are in the files are dependent upon those which you didn't actually read by use of the '%*f' field string. I didn't try to count and keep that straight to figure out "who's who in the zoo" in memory compared to the file; you probably already know that knowing what you want and don't.

It could possibly be simpler to just read the whole file including columns you don't want instead of the complex format strings and then just delete the unwanted columns by an operation such as

data(:,1:2)=[];

etc., etc., ...

In that case your fmt statement could probably simply be

fmt1=[repmat('%f',1,5) '%f/%f/%f repmat('%f',1,2049)];

I'm assuming your count of 2055 includes the date column as one, not three. Depending on the file size this might be a noticeable speedup (or perhaps not :) )...