subtracting and saving in an array

My data looks like this, where column headers are No and Time,respectively.

    '1'      '0.000000000' 
    '2'      '0.100244000' 
    '3'      '0.199460000' 
    '4'      '0.299659000' 
    '5'      '0.399856000' 
    '6'      '0.499070000' 
     .
     .
    '50'     '4.893317000' 
    '51'     '4.993553000' 
    '53'     '5.095700000' 
    '55'     '5.194844000' 
    '57'     '5.295069000'

How to find:

1. I want to find the values for (0.000000000 - 0.100244000), (0.100244000 - 0.199460000) and write the answers in a separate array.(eg:No, Time, and Time difference).
2. In the 1st column,after value 51, next value is 53. I want to count how many values have been jumped like that instead of following a sequence as 51,52,53...

[Image Analyst]

Is your data in a cell array or a table? Or something else like a structure array?

[Chathu]

Image Analyst- these data are stored in a notepad. using text scan i have imported it to MatLab. But i want to find the time difference(which is the 2nd column) for each No.(which is the 1st column). Any suggestions?

[Chathu]

Does anyone have any idea how to find the time difference(in the 2nd column)?

And how to see how many values have been jumped(in the 1st column)?

Really appreciate your feedback.

Answer 1

[Stephen23]

Based on the text file that you uploaded, I altered my first answer to read all of the columns in your data:

fid  = fopen('temp.txt', 'r');
data = textscan(fid, '%d%f%s%s%s%d%s', 'MultipleDelimsAsOne',true, 'HeaderLines',1);
fclose(fid);

jumps = sum(abs(diff(data{1})>1));
timeD = diff(data{2});

After placing this in a script and running it, I can confirm that it correctly extracts the data from your sample data file:

>> jumps
jumps =

     1

>> data{1}
ans =

           1
           2
           3
           4
           5
           7
           8
           9
          10
          11
          12
          13
          14
          15

Note that I altered your text file by removing data row six to provide a jump in the first column, which it seems you want to detect, otherwise it is identical to what you uploaded earlier.

[Chathu]

@ Stephen- fantastic!!! it works beautifully. Thanks a million Stephen. i really appreciate your continuous support. Thank you once again.:-)

[Stephen23]

My pleasure. If you have any more questions, please post them here on MATLAB Answers.

[Stephen23]

PS: you can display skipped times with this code:

fprintf('time %d is skipped\n',setdiff(data{1}(1):data{1}(end),data{1}))

[Chathu]

@ Stephen- thank you so much for your response, as always. when i execute the for loop(which i wrote). I add your fprintf cmd as well.Here is what i obtain:

timeD value is: 00001

timeD value is: 00001

timeD value is: 00002

timeD value is: 00002

timeD value is: 00003

timeD value is: 00003

timeD value is: 00004

timeD value is: 00004

timeD value is: 00005

timeD value is: 00005 . . .

timeD value is: 00015

I wonder why these values are duplicating? (eg: timeD value is: 00001

timeD value is: 00001)

[Stephen23]

I suspect that you just need to change the precision of how you display the numbers.

You don't tell us anything about the code that you use to display these numbers. Here is one way to display them to a higher precision, without a for loop:

>> fprintf('timeD %d is %g\n',[(1:numel(timeD));timeD.'])
timeD 1 is 0.100244
timeD 2 is 0.099216
timeD 3 is 0.100199
timeD 4 is 0.100197
timeD 5 is 0.199409
timeD 6 is 0.100244
timeD 7 is 0.099205
timeD 8 is 0.100155
timeD 9 is 0.100261
timeD 10 is 0.099153
timeD 11 is 0.100289
timeD 12 is 0.100173
timeD 13 is 0.099208

You can change the displayed precision of fprintf, this is explained in the documentation.

If you wish to ignore the time-differences where there is a jump, then you can also use some indexing (there are multiple changes to this code!):

fid  = fopen('temp.txt', 'r');
data = textscan(fid, '%d%f%s%s%s%d%s', 'MultipleDelimsAsOne',true, 'HeaderLines',1);
fclose(fid);

idx = diff(data{1})==1;
jumps = sum(~idx);
timeD = diff(data{2});

fprintf('No. %d is missing\n',setdiff(data{1}(1):data{1}(end),data{1}))
fprintf('timeD %d is %g\n',[double(data{1}([false;idx])),timeD(idx)].')

When I run this script, I get the following displayed in my command window:

No. 6 is missing
timeD 2 is 0.100244
timeD 3 is 0.099216
timeD 4 is 0.100199
timeD 5 is 0.100197
timeD 8 is 0.100244
timeD 9 is 0.099205
timeD 10 is 0.100155
timeD 11 is 0.100261
timeD 12 is 0.099153
timeD 13 is 0.100289
timeD 14 is 0.100173
timeD 15 is 0.099208

We can see that data for No. 6 is missing (as per the datafile that I uploaded with my answer), and it also ignores the two timeD values that would depend on the time value of No. 6.

[Chathu]

Hi Stephen, Thank you soo much for your response.

Below is just an extract of the original file.

    No        Time
    '1'      '0.000000000' 
    '2'      '0.100244000' 
    '3'      '0.199460000' 
    '4'      '0.299659000' 
    '5'      '0.399856000' 
    '6'      '0.499070000'

what i want is:

(1) In the "No column"--> 1st value- 2nd value = 1 only, i want to find the value for--> 1st value- 2nd value in the "TIme column".

If this condition does not satisfies then skip(do not count the time difference). That's why i used a for loop (and If else statement).

(2) Later i want to find how many numbers have been jumped in the "No column". (you've already answered this-thanks)

I executed following For Loop:

n=15;

for i=1:n

for j=1:2

if (abs(diff(data{1})==1))& diff(data{2})

fprintf('timeD value is:%6.5d\n',i)

else

fprintf('time %d is skipped\n',setdiff(data{1}(1):data{1}(end),data{1}))

end

end

end

Thank you so much for your guidance, once again :)

[Stephen23]

Both of your requests are already addressed in my last comment. Did you try running it? Was there some problem interpreting the displayed text? Did you try it on your own data and get an unexpected result?

Note that in MATLAB there is no need to use a loop or if statements, indexing can be used to achieve your requested outputs, which is what I did. You do not need to put my code inside any for loops, as you have shown in your comment: it already works without them, and prints the information that you seem to want. Just copy the code exactly as it is shown in my last comment into a script, and run it.

Also note that when you jump one data sample you lose two timeD values. This is also shown in my last comment, where the absent No 6 results in two missing timeD values (6 and 7).

PS: do not use the names i or j for your loop variables, as these are both the names of the inbuilt imaginary unit .

[Chathu]

Hi Stephen, After reading your description i was able to clarify my doubt. One last question regarding your code.

Can you kindly explain what the below line means?

double(data{1}([false;idx])),timeD(idx)

Yes, i ran it and it works very well. Stephen, thank you so much for all you've done to help me out. Really appreciate your continuous support. Hats off to you!!

[Stephen23]

Of course. It concatenates together two vectors: one of timeD values, and one of row No. values, which are then displayed using the fprintf command. Lets have a look at its constituent parts.

Firstly have a look at the line idx = diff(data{1})==1;, this defines a logical vector which is true only where there is no jump, and false where there is a jump. You can look at its values in the variable viewer panel (double click on any variable in your workspace to open it in the variable viewer).

Then with timeD(idx) we use logical indexing to obtain only those timeD values where there is no jump (e.g. excluding No. 7 in my example data, as No. 6 is already absent). These data are of class double.

The code data{1} gets us our first column of data, the No. values, which are of integer data class. Using data{1}([false;idx]) we again select only the ones that correspond to where there is no jump. I shifted the index by one place using the false value to try to make it slightly more intuitive to read when it is displayed. Hint: try removing this and see what happens.

We want to join these two vectors together (for displaying) but they are of different data classes, so we convert the No. values to double using double(data{1}([false;idx])).

Now we can concatenate them together using square brackets simply [double(data{1}([false;idx])),timeD(idx)]. Because MATLAB is row-major the final step is to transpose this matrix with .', so that each column contains one No. value and one timeD value. This gives us the final code segment [double(data{1}([false;idx])),timeD(idx)].', which is then displayed ( without loops !) using fprintf.

Here is a neat trick in MATLAB that you should try: highlight any piece of code and press the key f9 (on the very top of your keyboard) to evaluate that highlighted section of code. You can use this trick to look at the individual parts of any line of code and see what they evaluate to. Try it by highlighting timeD(idx) in your script, press the f9 button, and see how it is different to plain-old timeD !

[Stephen23]

EDIT: The OP deleted their previous comment above.

This is really a new topic, so it deserves a new question.

however...

You code is almost right, just the function floating is not known to MATLAB. It doesn't exist, unless you have written it yourself, and gives you an error ("Undefined function..."). But that is alright, because the values that we have in data{2} are already floating point (that array is of class double), so we don't need to convert the second column values to another data type. Try this:

num = data{2} < 121;
sum(~num)

Answer 2

[Ilham Hardy]

A wild guess, you may need:

1. cell2mat (to convert your cell into workable matrix)
2. diff (to find the difference between adjacent elements)
3. find (to find how many values that you have been jumped)

You can search the documentation by typing eg. doc cell2mat into the command window.

[Chathu]

llham- thanks for your response.i know that you've told that u're guessing. Unfortunately cell2mat doesn't work. I am getting the following error:

Error using cat. CAT arguments dimensions are not consistent

Any suggestions ..

[Chathu]

In addition to the 1st and 2nd column as mentioned above, 3rd column is present.It is quite different because it is alphanumeric.

 How to find the difference between '01f600000239'-'01f60000023a which corresponds to 6th and 7th No.

 '1'       '0.000000000'      '01f600000234'
    '2'       '0.100244000'      '01f700000235'
    '3'       '0.199460000'      '01f600000236'
    '4'       '0.299659000'      '01f600000237'
    '5'       '0.399856000'      '01f600000238'
    '6'       '0.499070000'      '01f600000239'
    '7'       '0.599265000'      '01f60000023a'
    '8'       '0.699509000'      '01f60000023b'

[Ilham Hardy]

Seeing several answers below and you have not solve your problem, i would say that it might be better if you can upload your .txt file so that we could see what are the actual problems.

If it too big, you can copy several (10-20) lines of your .txt file. Use the attach (paperclip) button for attaching file.

[Chathu]

llham- you are right. I would have upload the attachment earlier-my mistake..:( Thanks alot for all who have shed me some light to solve this issue. Especially Stephen,thank you soo much for your continuous support. it is simply wonderful. Thanks alot.

Answer 3

[Stephen23]

I saved the data that you gave in a comment to Ilham Hardy's answer in a simple text file "temp.txt", removing row six to get a jump in the first column:

'1'       '0.000000000'      '01f600000234'
'2'       '0.100244000'      '01f700000235'
'3'       '0.199460000'      '01f600000236'
'4'       '0.299659000'      '01f600000237'
'5'       '0.399856000'      '01f600000238'
'7'       '0.599265000'      '01f60000023a'
'8'       '0.699509000'      '01f60000023b'

Then the following code can be used to extract this data and convert it into numeric & string data:

fid  = fopen('temp.txt', 'r');
data = textscan(fid, '%d%f%s', 'MultipleDelimsAsOne',true, 'Whitespace','''\b\t');
fclose(fid);

You can then find how many jumps there are in the first column using this code:

jumps = sum(abs(diff(data{1})>1));

And the time differences from the second column

timeD = diff(data{2});

[Chathu]

Hi Stephen,

it is very kind of you to do extra piece of work with the issue i had. Really appreciate your effort.

Here is the outcome which i obtain after running your 'textscan' function:

data{:}

ans =

   Empty matrix: 0-by-1

ans =

   Empty matrix: 0-by-1

ans =

   Empty cell array: 0-by-1

I simply don't understand why does all 3 cell arrays become zero. I really want to convert this structure to numerical values. what's really going on here? (between i am working with R2012a,is that the issue?)

[Stephen23]

Try this exactly:

• Copy the data from my answer into a text file, and save this as "temp.txt".
• Copy the code from my answer into an Mfile, and save this as "temp.m".
• Run the script temp.m.

This does not result in empty arrays. The fact that it does not work on your data means that your data is different somehow to what you are describing to us. It may be that your data is either incomplete (has some missing or values of a different type) or has different delimiters, or some other subtle difference. These differences are however very significant when importing into MATLAB, and you must describe that data file format exactly as it is.

• Does the data file really include those single quotation marks around every field (including numeric)?
• Are there really multiple delimiters between every field?
• Are there really leading delimiters (as your original question has)?
• How many columns does the data file contain?
• Does the data file have a header, a footer, any text lines or any blank lines?
• You wrote that "these data are stored in a notepad", however "Notepad" is not a file format, it is a Windows program for editing text files. Is the data stored in a text file?

The best solution would be for you to upload the data file, and I can adapt the code to work with your exact data.

[Chathu]

hi Stephen,

Thank you sooo much for your response. It is a a huge file.Hence can u be kind enough to let me mail it to you(if you don't mind only)

[Stephen23]

Sure, send me a massage via my profile page.

Did my example work when you tried it exactly as I described?

[Chathu]

well, it didn't work. I wasn't aware that MatLab is very precise in importing data. Originally my data file has 7 columns.

[Stephen23]

Please try this exactly:

• Copy the data from my answer into a text file, and save this as "temp.txt" in your current directory. Do not use your data.
• Copy all of the code from my answer into an Mfile (script), and save this as "temp.m" in your current directory.
• Run the script temp.m.

Do not make any changes to the code or the data. Does this really result in empty arrays?

In your question you wrote that the data has two columns. Then you said later that it has three. Now it turns out to have seven. While I might be able to cope with these random changes to your data, MATLAB cannot. Did you read my earlier comment: "These differences are however very significant when importing into MATLAB, and you must describe that data file format exactly as it is." Please help us by being clear and giving the details that we request. For example I asked you six specific questions about the formatting of your data, and you answered one, even though all of these answers could have a significant effect on how the data needs to be imported.

MATLAB is very precise with importing data, because it has so many options and can deal with many different formats, but you need to be able to define the file format yourself. Some tools semi-automate this:

http://www.mathworks.com/help/matlab/ref/uiimport.html

[Stephen23]

Exactly how huge is this "huge" file? Before sending me the whole data, you can upload a subsample of the data file here. Take the first 100 lines or so, and save these in a new file. Please upload this file in a new comment (use the paperclip button above the text field). I will have a look at it and see how we can get this thing working for you :)

Answer 4

[Chathu]

PS: i am adding this to here because the question is totally related to the same question.

Suppose i want to write a for loop for this question.

Eg: suppose that i want : (abs(diff(data{1})==1)) & diff(data{2})

so my for loop would be:

n=15;

for i=1:n

for j=1:2

if (abs(diff(data{1})==1))& find timeD=diff(data{2})

fprintf('timeD value is:%6.5d\n',i)

else disp('timeD is:skipped')

end

end

end

Any suggestions?