Interpolate non-monotonic test data

Hello all,

I want to interpolate at specific values the curve in the picture attached (say every 0.01 in the x axis). Any idea how? The test data consists of a huge number of data points, so I want to clean them a bit.

Thanks

Answer 1

[Konstantinos Belivanis]

What I did was to run through all the data for the amount of times I wanted and interpolated when the point of interest was between the data. The code I used was:

k=1;

lo=50;
ul=49;

i=k;
j=1;
i=1;
for j = 1:lo
    x1=0;
    x2(j,1)=j/100;
    while x2(j,1)>=x1
        x1=A(i,1);
        x3=A(i+1,1);
        y1=A(i,2);
        y3=A(i+1,2);
        if x2(j,1)<x3
            B(j,1)=j/100;
            B(j,2)=(x2(j,1)-x1)*(y3-y1)/(x3-x1)+y1;
            y2(j,1) = B(j,2);
        end 
    i=i+1;
    end
end

k=i;
l=length(x2);
j=1;
for j = 1:ul
    x1=lo/100;
    x2(j+l,1)=lo/100-j/100;
    while x2(j+l,1)<=x1
        x1=A(k,1);
        x3=A(k+1,1);
        y1=A(k,2);
        y3=A(k+1,2);
        if x2(j+l,1)>x3
            B(j+l,1)=lo/100-j/100;
            B(j+l,2)=(x2(j+l,1)-x1)*(y3-y1)/(x3-x1)+y1;
            y2(j+l,1) = B(j+l,2);
        end 
    k=k+1;
    end
end

etc... for as many cycles I wanted it. It worked well.

[Image Analyst]

Without comments, that's rather daunting to try to understand. You might like to look up interp1() if you want to fill in missing data. But again, interp1() or any interpolation, even if the sampled points are less frequent, will still be noisier than regression.

Answer 2

[Image Analyst]

Try interp1(). By the way, interpolation won't "clean" the data. Why do you think it will? It will just give you more points in between the noisy ones. Perhaps you meant "regress" instead of "interpolate"??? If you meant regress, see my attached demo on polyfit().

Please READ THIS and then attach your screenshot (which you forgot to attach), and original data file, and maybe some code to read in the data file.

[Walter Roberson]

Maybe they want to do a spline smoothing? Or some kind of low-pass filtering?

[Image Analyst]

I most certainly don't know everything about splines but I thought you give it training points, and it fits polynomials (like cubics) between them so that the slopes match at the training points. So if the original data is used as training points for the splines, then that noisy/erratic data is still in there. Sure you can interpolate points in between the noisy points, as my attached demo shows:

So, between the noisy data points it's smooth but the noisy data points are still there in the data.

Now maybe there's a different kind of spline that I need to learn about. That's quite possible as I know there are many flavors of splines and I'm just familiar with cubic splines.

For smoothing 1-D signals, I often use a Savitzky-Golay filter like I just did in this Answer: http://www.mathworks.com/matlabcentral/answers/234003-how-to-plot-a-histogram-with-distribution-fitting#answer_189490

[Walter Roberson]

smoothing spline