How can I read in a txt file, which is being created by a different programm, and plot the numeric data into a contourf plot continously?

 Hi there, 
So far I have written one function to read in any kind of txt file, which is created by another software (Example is attached, yet in the future these files will be a lot larger). Then, the data was added into the workspace in Matlab. The second script, which I wrote, uses the data, which was created, and plots it continously. Both of these scripts worked independently real well.
Now, I tried to merge these two scripts, so that the data shall be read from the txt file and be plotted at the same time (thats why I start the function with a while cycle). 
The txt file, which is created in another program, appends a new row of data every 3 seconds. Ideally, my function will read and update the plot at the same rate.

This is my code:

%first, make sure the data which is read in should be errorless
%Moli: read file and cut the temperature data from raw data(.txt)
% 2nd, write in command window : addpath ('D:\matlab-test'); to make sure 
%the directory is correct
%addpath eg ('E:\matlab_daten');
% 3rd, start function with defining dataset and writing name of function
%eg  "TD23111501 = ReadTempData ('23.11.20150_1.txt') into
% command window 
% then dataset TD23111501 was created and function was read 
function [data,Nrows] = ReadPlotTempData(FileName)  
count=0;% starts cycle with reading and plotting 
while (1)%keeplooping
 fid = fopen(FileName); % open data
Nrows = numel(textread(FileName,'%1c%*[^\n]'))   % find the number of rows
%4th, data  rows 1-15 is only read but not used
%Moli:explore data unitl 'data begin'
for i = 1:15
    tline = fgetl(fid);
end
% 5th, a new matrix  called eg "data" or "TD23111501" is created, leaving
%the first 15 rows out and using 20 columns 
data = zeros(Nrows-15,20); %create matrix (# of rows, # of columns eg 20)
for i = 1:Nrows-15  %ab row 16 daten werden verwendet 
    tline = fgetl(fid);    % read the line
    k = strfind(tline, ',');   %find positions of comma (,) in each line
    restline = tline((k(29)+1):(k(49)));  
%    %Moli: cut the data, from real data (first record of temperature) to end
    % the restline ... ab komma 29-49 mit jeweils 1 leerzeichen nach dem komma 
    data(i,:) = str2num(restline);
end
   fclose(FileName) 
%end of the ReadTempData script! 
%script: contourf_script_while_cycle starts now 
           figure (1)
    [hc hc] = contourf(data,100) % 100 gibt die genauheit der interpolation an 
    colorbar; % add aditional things eg 
    caxis auto %([4 12])%%bestimmt die Temperaturintervall
    % welches angezeigt werden soll, 'auto' legt das Intervall selbst fest
    legend ('Temperatur in C°')
    set (hc,'Linestyle','none');
    set (gca,'fontsize',14);
    xlabel ('Anzahl der Sensoren', 'fontsize',16);
    ylabel ('Zeitreihe', 'fontsize', 16);
    colormap jet; colorbar;   
     drawnow
   count=count+1
  end
end

My problem / question now is: In the first part of the function, I have to define the dataset, which will be created (TD23111501). Later, when creating the plot, I want to use this data TD23111501 and plot it. However, it doesn't work. Is there another way of defining the dataset and retrieving the data within the same function? Or is it even not possible at all? Thank you for your help

[adnan ali awan]

The way I would have done is simply by:

1) Calling and Reading the file;

2) Know what's the type of data! if the original data is in matrix form then use mat2dataset!

3) Use a timer function that calls the reading function

4) For Plotting use hold on with drawnow() ....

Answer 1

[Ingrid]

first of all, you call fopen without a corresponding fclose. This will get you into trouble so always close the file again after you are done reading it.

Secondly, you want to use TD23111501 but you do not create it anywhere. There is no example data attached so I an only guess what you are trying to do, but I guess you want the "data" to be used. If so you need to make sure that both variables have the same name. It is however not clear how many lines you read in per time, if only the last line is changed each time, you should not read in the whole file each time but only that time. If your program runs for a long time this will save you much time.

you will also need a timer of some sort (e.g. wait 3 seconds each loop) or check to see when the file is updated (<http://stackoverflow.com/questions/17522463/matlab-get-the-last-modification-time-of-a-file>)

[Catharina Keim]

Hi Ingrid, Thank you for your quick response. Now, I edited my code and changed the things you mentioned. Also, I attached the txt file, which I forgot earlier. The best option would be so that only the last row will be read and plotted (annexed). Also, the idea with the timer is really great. How can I add after each cycle only the last row? And how is the function updated every 3 seconds? In case, I need to add a timer, it should be one that has an endless loop since I never know in advance for how long I will acquire my data. Thanks again for your help

[Ingrid]

for the timer you can use the pause function http://nl.mathworks.com/help/matlab/ref/pause.html but you have to be careful as it might lead to some problems when you run your code for a longer time as there might be a mismatch in the time counting of your code and the one of the other program. I think the safest bet is to check if the file has been updated or not before proceeding, adding a pause of 1 second to avoid excessive checking.

is think this is more what you want?

function ReadPlotTempData(FileName)
count=0;% starts cycle with reading and plotting
figure
while (1)%keeplooping
    fid = fopen(FileName); % open data
    formatString = '%*f %*f %*f %*f %*f %*f %*f %*s %*f %*f %*s %*f %*s %*f %*f %*f %*f %*s %*f %*s %*s %*s'; % it seems you do not want these 21 first columns so use * to discard them
    for ii = 1:28
        formatString = [formatString ' %f'];
    end
    rawData = textscan(fid,formatString,'HeaderLines',15,'Delimiter',',');
    data = cell2mat(rawData);
    fclose(fid);

      [hc hc] = contourf(data,100); % 100 gibt die genauheit der interpolation an
      colorbar; % add aditional things eg
      caxis auto %([4 12])%%bestimmt die Temperaturintervall
      % welches angezeigt werden soll, 'auto' legt das Intervall selbst fest
      legend ('Temperatur in C°')
      set (hc,'Linestyle','none');
      set (gca,'fontsize',14);
      xlabel ('Anzahl der Sensoren', 'fontsize',16);
      ylabel ('Zeitreihe', 'fontsize', 16);
      colormap jet; colorbar;
      drawnow
      count=count+1
      pause(3);
  end

[Catharina Keim]

Hi, thanks a lot for your help! I first got an error while running your new code. Now, I changed it slightly. I am not familiar with the approach of the 'formatString'. I want to read and plot starting from row 15 ( that's already working) and only using columns 30-49, the last column shall not be used. How can I change the columns, which are read and used by Matlab? I already added more '%*f' so that only the last 21 columns are read. How can I discard the last column as well?

Code:

function [data] =  ReadPlotTempData1(FileName) %function [data,Nrows] = ReadPlotTempData(FileName)
count=0;% starts cycle with reading and plotting
figure
while (1)%keeplooping
    fid = fopen(FileName); % open data
    formatString = '%*f %*f %*f %*f %*f %*f %*f %*s %*f %*f %*s %*f %*s %*f %*f %*f %*f %*s %*f %*s %*s %*s %*f %*f %*f %*f %*f %*f %*f'; 
    % it seems you do not want these 21 first columns so use * to discard them
    %f is for numbers, s is for letters/other characters
    % fist relevant column is #30
    for ii = 1:21
        formatString = [formatString ' %f'];
    end
    rawData = textscan(fid,formatString,'HeaderLines',15,'Delimiter',',');
    data = cell2mat(rawData);
    fclose(fid);
    [hc hc] = contourf(data,100); % 100 gibt die genauheit der interpolation an
      colorbar; % add aditional things eg
      caxis auto %([4 12])%%bestimmt die Temperaturintervall
      % welches angezeigt werden soll, 'auto' legt das Intervall selbst fest
      legend ('Temperatur in C°')
      set (hc,'Linestyle','none');
      set (gca,'fontsize',14);
      xlabel ('Anzahl der Sensoren', 'fontsize',16);
      ylabel ('Zeitreihe', 'fontsize', 16);
      colormap jet; colorbar;
      drawnow
      count=count+1
      pause(3);
  end

[Ingrid]

simple, just use %*f instead of %f for the last column. The formatString is just a variable name that I have chosen. For more information on this you can look at the documentation of textscan. It just seemed to me that this is the way to go for you instead of doing all the parsing of your data manually after reading it in line by line

[Catharina Keim]

Thank you, you helped me a lot. And I learned new things :)