How to compare elements with above and below element in a column?

Hello

I have a matrix of 4 columns. I would like to compare each element in 4th column with previous and next element in the same column. If the difference is more than 50, the elemnt should turn to 0. (This is to eliminate unexpectedly high or low values inside 4th column. Other columns should not change at all)

My written code does not work, so I will be glad if some one can help in this. I expect the result look like this:

A = [15 2 7 36

20 3 14 41

25 4 21 0

30 5 28 22]

A = [15 2 7 36
     20 3 14 41
     25 4 21 567
     30 5 28 22]
M = length (A(:,4)),
i = A(1,4)                         %first element of the column
k = A(M,4)                         %last element of the column
  for  T = 2:M-1
    idx = A(T,4) 
        if idx> 'A(T+1,4)'+200
        A(idx,4) = 0;
        if idx < 'A(T-1,4)'-200
        A(idx,4) = 0;
        end
        end
        fname = sprintf('new.csv');
    csvwrite (fname,A);                        % Write to CSV file
   end

Answer 1

[James Tursa]

Is this what you want?

for  T = 2:M
    if( abs(A(T,4) - A(T-1,4)) > 50 )
        A(T,4) = 0;
    end
end

[Masoud Taleb]

It works for this matrix but when I apply to my main data file, it changes many elements to 0 by mistake. I attach my main file here and also past the code for importing it. I think something in loop dose not work correctly.

A = readmatrix ('l4.csv');              %A = all data
M = length (A(:,4)),
 for  T = 2:M
    if( abs(A(T,4) - A(T-1,4)) > 50 )
        A(T,4) = 0;
    end
end
fname = sprintf('new.csv');
csvwrite (fname,A);                        % Write to CSV file

[James Tursa]

You need to tell us what your conditions are for the outliers, if it is not a simple test for difference > 50.

[Masoud Taleb]

Oh James.... I found the problem. My values in example matrix were below 50 and in real data file are below 100. Then, many of data turned to 0.

Then, this works.... Thanks.

Answer 2

[Guillaume]

Rather than coming up with your own code, I'd recommend using the filloutliers method. It doesn't have a detection method for 'more than 50 away from the two nearest neighbours' but the many detections methods are probably a lot more robust than this anyway.

A(:, 4) = filloutliers(A(:, 4), 0);  %replace outliers by 0

It also has better fill methods than replacing outliers by 0.

[Masoud Taleb]

Dear Guillaume

This is so nice way. I did not know there is so short and effective command. I replace above solution with this :) Thanks