Relationship between 2 poly curve/ 2 data sets , how can I correlate it ?

I have 2 sets of data.

First set is black , 6% measuring target that gives me certain stat error at given distance . Same goes for white, 90% measuring target.

2 sets of data have different length. I use polyfit and polyval to obtain the equations .

Ex:

y1 = 0.0001x^4 - 0.0022x^3 + 0.0312x^2 -0.1143x + 0.4783 for 6%

y2 = 0.0004x^2 -0.0024x +0.1762 for 90%

How can I obtain an equation for 20% measuring target ? I need to estimate/approximate a set of data for 20% based on my available data (6% and 90%). Is there any method to do this ?

My codes as below:

DistaveB=Data(3:15 , 2);

DistaveB=cell2mat(DistaveB);

DistaveW=Data(16:32 , 2);

DistaveW=cell2mat(DistaveW);

StatErrorB=Data(3:15 , 3);

StatErrorB=cell2mat(StatErrorB);

StatErrorW=Data(16:32 , 3);

StatErrorW=cell2mat(StatErrorW);

SignalaveB=Data(3:15 , 4);

SignalaveB=cell2mat(SignalaveB);

SignalaveW=Data(16:32, 4);

SignalaveW=cell2mat(SignalaveW);

CDistaveB=DistaveB./1000;

p = polyfit(CDistaveB, StatErrorB, 4);

f = polyval(p,CDistaveB);

figure(2)

plot(CDistaveB, StatErrorB , 'o')

hold on

plot(CDistaveB, f)

CDistaveW=DistaveW./1000;

pW = polyfit(CDistaveW, StatErrorW, 2);

fW = polyval(pW,CDistaveW);

plot(CDistaveW, StatErrorW , 'o')

plot(CDistaveW, fW)

Any suggestions will be helpful, Thank you!

ps: I do not have any additional toolbox to solve this.

[John D'Errico]

You have two "equations" that represent something that only you know what they mean. One of them applies to 6%, one to 90%. What those percentages means, we also are not told. Sigh. Now, you want to somehow infer what equation would apply at 20%? You MIGHT want to use some sort of interpolation. Since you have only two pieces of information, that is, two curves, at best normally you could use a linear interpolation of some sort, but I could imagine other methods that would be appropriate. Of course, since we are not told what any of this means, any type of interpolation may be appropriate.

How any of this is related to the concept of correlation, I have no idea.

Is there any method to do this? Since I have no idea what "this" is, I'm at a loss. The answer is yes. Or maybe no. Probably. Maybe. Possibly.

https://www.mathworks.com/matlabcentral/answers/6200-tutorial-how-to-ask-a-question-on-answers-and-get-a-fast-answer

[William Rose]

@Eric Aw,

I thank @John D'Errico. for his excellent suggestions. Please do address his questions.

In the spreadsheet you posted, there are columns labelled "Average distance, mm", "Stat Error", and "Average Signal". Your code is missing one or more lines at the top, to read in the data from file data.xlsx. Please provide the code to read the data from the file.

Your code indicates that column "Average Distance" corresponds to "x" in your fit, and "Stat Error" correpsonds to "y". You read in the Average Signal data, but you never use it or plot it. Is that what you intend? Are you really trying to predict the Stat Error from the Avg Distance? Is it correct that you do not care about Average Signal?

I see you have 13 rows for "black, 0.06" and 17 rows for "white, 0.9". Is each row the result of analyzing mutltiple data points? The column headings suggest that may be the case. Are those original individual points not available? It is always preferable to use the raw data, if available.

[Eric Aw]

@John D'Errico @William Rose

Thank you very much for the response. Sorry for the confusion

Regarding the missing information:

1) 6% and 90% represent the diffuse reflectance of my target. I send a light signal (650 nm) out and it reflect back to my sensor. The data obtained represent my result , at certain distances, I obtain the stat error and signal .

2) Yes, I miss out copying xlsread for first line , sorry about that

3) I haven't use the data from "average signal" yet. As I need to solve for "stat error" before moving to "average signal"

4) This is consider the raw data, as my device has preprocess some raw data (mutliple points like 10 values) to give me this final average values.

As I do not have a target that represent 20% diffuse reflectance , I need to do an approximation based on my current data.

Answer 1

[William Rose]

@Eric Aw,

Here is an implentation of my previous suggesitons. The attached code produces the console output and the plot below.

>> EricAw3
20% fit: y20=a0+a1*x+a2*x^2+a3*x^3+a4*x^4
a0=0.428,a1=-0.0957,a2=0.0263,a3=-0.00179,a4=4.81e-05

[Eric Aw]

@William Rose

Thank you very much , I think it solve what I want .

Answer 2

[William Rose]

@Eric Aw,

See my quesitons in the ealrier comments.

I moved your data into a text file (attached) and made a few tweaks to your code (see attached code). It runs without error . It produces the plot below.

I understand from your original post that you want to find an equation to make predictions for the 20% case, based on the data you have for 6% and 90%. Here's how you can do it.

1. Only do this over the Distance range where you have data for both 6% and 90%. Don't extraolate the 6% model to larger distances.
2. Use the 6% and 90% polynomials to predict data at N(=30 should be enough) evenly spaced distances, within the range of overlapping distances.
3. Interpolate to 20%, to get N predictions for 20%.
4. Fit a 4th order polynomial to the 20% predictions.
5. Plot the 20% poly to make sure it looks OK.

Why use a 4th order poly for 13 points, and a second order for 17 points? Did you use standard statistical methods to choose the polynomial order for each case?

[Eric Aw]

@William Rose

Thank you for the input.

As for the order poly selections, I tried using other order poly to check how does the curve looks for both black and white. (6% and 90%), and it was this selected order (4th and 2nd) represent the curve I want the most.

Example, I tried using higer order poly for white (90%) , but the result shows mostly 0 for higher order , that's why I choose 2nd order .

In your opinion, do you have another suggestion to choose the poly order ?

[William Rose]

@Eric Aw, Choosing the order is a complicated topic and is beyond the scope of this discussion board. You have used the "pick the lowest order that looks reasonable" algorithm for order selection, which is a good start.

[Eric Aw]

@William Rose

Understand. Thank you