the average of multiple Probability density functions

Hallo,

I have a cell array of {1000,1} pds, they are all the same distribution, but they differ slightly in the parameter values

Does any one know how to get the average of these pds? I have got them in iterative process and would like now to know the mean or average of them

I though of taking the mean of the parameter values in the pds in for loop but I am truggling with the right way to do it because I keep getting errors.

it would be helpful if some one gives me a hint :)

I have attached the cell array (Temp) in mat file

Thanks in advanced!

[J. Alex Lee]

when i load the .mat file, the variable Temp is an emtpy cell array

[Rima Habib]

Hey, I tried it on my PC and it works so I dont know where the problem is, but I upload it again just in case, maybe this time will open correctly

[J. Alex Lee]

Oh maybe it's because i don't have statistics toolbox...when you say pd's, i guess you mean some kind of statistics object? sorry i can't help!

[Image Analyst]

J, it should work regardless of what toolboxes you have

 s=load('temp.mat')
 cellArray = s.Temp
s = 
  struct with fields:
    Temp: {1000×1 cell}
    
cellArray =
  1000×1 cell array
    {1×1 prob.GeneralizedParetoDistribution}
    {1×1 prob.GeneralizedParetoDistribution}
    {1×1 prob.GeneralizedParetoDistribution}
    {1×1 prob.GeneralizedParetoDistribution}
etc.

[Jeff Miller]

What do you mean by "the average of the probability distributions"?

For example, suppose one distribution is normal(0,1) and another distribution is normal(10,2). Is the average normal(5,1.5)? Or is the average bimodal with one mode around 0 and another around 10? Or is the average distribution something else?

Answer 1

[Ridwan Alam]

The idea of "average" is ill-defined for pdfs.

In your case, all the 1000 distridutions are generalized paretos. Your theta is fixed in all pdfs, so you can certainly take the average of the sigma's and the k's. But what meaning/significance those "averages" carry, you need to come up with that.

If you are looking for the average expected value, you can calculate the mean (expected value) for each pareto and then average those. mean = theta + (sigma/(1-k)). ref: https://en.wikipedia.org/wiki/Generalized_Pareto_distribution

Also, if you are looking for the overall distribution parameters, you can combine the input data [available in Temp{}.InputData.data] and use gppdf().

Hope this helps.

[Rima Habib]

@Jiff Miller @Ridwan Alam

I am trying to generate artificial Time series, so after an iterative methode, taking the average of the resulted pdfs is the best way to get the gppdf that suits the data.

so the only idea I could came up with is to take the average of the sigmas and ks but I am also interessted in your other approch which is combinig the inputdata then using gppdf.

how to do that exactly with a cell array full of pdfs. I couldnt figure it out?

Thanks you for your Help!

[Ridwan Alam]

Rima, as I mentioned one approach could be to stack all the inputdata from all the pdfs.

allInputData=[];
for i = 1:length(Temp)
    allInputData = [allInputData; Temp{i}.InputData.data];
end

% in your case, theta is fixed
newTheta = Temp{1}.theta;

paramhat = gpfit(allInputData-newTheta);
newK = paramhat(1);
newSigma = paramhat(2);

Hope this helps.

Please accept the response as an answer, and vote up if you liked the conversation.