MATLAB Answers

How can I effectively pre- process data?

2 views (last 30 days)
Olukayode Majekodunmi
Olukayode Majekodunmi on 1 Sep 2013
I have an input-output dataset to be used in ANFIS modelling. I need to pre-process the data! I've already removed the outliers outside +/-3 standard deviation of the group mean! I've also employed the Moving Average Smoothing technique. The issue here is that the RMSE is still undesirably large! I need a more effective way to pre-process the data!

  4 Comments

Show 1 older comment
Olukayode Majekodunmi
Olukayode Majekodunmi on 1 Sep 2013
Yes! There is a significant amount of "Noise". It is noticeable from the plot. The system I'm trying to describe exhibit a high degree of non-linearity coupled with some errors incurred during measuring and recording! I discovered smoothing the data reduces the RMSE!
Walter Roberson
Walter Roberson on 1 Sep 2013
Perhaps you could give us an example plot or two? I would be interested in original data, then outlier-removed, then smoothed; and, of course, a line showing approximately what your ideal output would be.
Olukayode Majekodunmi
Olukayode Majekodunmi on 5 Sep 2013
I'm sorry for the delayed response! The two inputs are to be modeled separately with each of the outputs (Sugeno Fuzzy). Here are the download links for the data plots! Figure 1: shows the original data http://www.sendspace.com/file/vsghku Figure 2: shows the data after the outliers have been removed http://www.sendspace.com/file/suiwet Figure 3: plot after applying the moving average smoothing technique http://www.sendspace.com/file/1svdkd Figure 4: after scaling the smoothed dataset http://www.sendspace.com/file/1gs4yi

Sign in to comment.

Answers (0)

Sign in to answer this question.