Detect and remove outliers in signals

Hi there,

I have a signal, which is attached, and I want to remove outliers, which are shown in the figure, in datasets, or just replaced them with 0.

I try to use the rmoutliers function but it reduces the length of datasets while I want to get the same length of original datasets as a result.

Any suggestion would be appreciated.

Answer 1

[Bjorn Gustavsson]

Be very cautious when changing bad-data-values to some default-value - that will eventually lead to (since we all know that the exact procedure one takes to do this will be forgotten far too soon) bad errors in some statistical measures of that data-set. If you really want the same-side of the out-lier-removed data I suggest you plug in nan instead of zeros. If you use rmoutliers you could do this:

[~,idx_outliers] = rmoutliers(Data);
Data(idx_outliers) = nan;

Then you can still do the statistics on the signal - this way you will have to correctly handle the nan-s in the data, which isn't too tricky.

HTH

[payam samadi]

Thanks, but it did not work for my other signals. For example in the attached dataset and figure (z direction), It didnt goes well.

[Bjorn Gustavsson]

Don't you get a rejection of the 3 spikes in the ECMestimatedPos? Or what is the problem? Outlier-rejection in these cases (the third spike isn't that far out relative to the other local minimas) starts to become a difficult problem.

[payam samadi]

Yes, I didnt get a rejection of the 3 spikes in the ECMestimatedPos.

I agree with you.

Answer 2

[Steven Lord]

You could try filling the outliers with NaN values (or some other method) using the filloutliers function.

x = [1 2 99 4 5]

x = 1×5

1 2 99 4 5

filloutliers(x, NaN)

ans = 1×5

1 2 NaN 4 5

filloutliers(x, 'linear')

ans = 1×5

1 2 3 4 5