Log likelihood

Hi!

I was wondering how to compute (which function to use) in Matlab the log likelihood but when the data is not normally distributed. Thanks!

Nuchto

Answer 1

[Tom Lane]

If you have the most recent release of the Statistics Toolbox:

>> x = poissrnd(4,20,1);
>> pd = fitdist(x,'poisson');
>> pd.NLogL
ans =
   39.0221

If you do not:

>> mu = poissfit(x);
>> -sum(log(poisspdf(x,mu)))
ans =
   39.0221

[Nuchto]

Thanks, Tom.

a) Could you explain me step by step what you are doing? I read the help files for those functions, but it takes too long for me to understand.

b) What if I don't know the distribution of my data?

Nuchto

[Tom Lane]

[a] The second version fits the data to the Poisson distribution to get parameter estimate mu. Then it evaluates the density of each data value for this parameter value. (The density is the likelihood when viewed as a function of the parameter.) The overall log likelihood is the sum of the individual log likelihoods.

[b] You can try fitting different distributions. But your question was about the likelihood, and that depends on the distribution.

[Nuchto]

Thanks. This assumes I know what distribution my data is, and I don't know the distribution. "Probably" :) it has no typical distribution. So how to proceed? Along with this question, how can I know which distribution my data is? Is there a function to test my data in matlab this way?

[Oleg Komarov]

Check out the link I posted.

[Tom Lane]

Oleg's link looks interesting (I haven't tried it myself). Another option is to find a nonparametric estimate using either fitdist with the 'kernel' option, or ksdensity.

[Nuchto]

Thanks, Oleg. I tried your function but I don't know how to interpret the results. @Tom: my distribution has been produced using ksdensity with the data, and the shape doesn't resemble to any known distribution. So I didn't understand why I should use ksdensity... ksdensity doesn't tell you anything.

[Tom Lane]

It's certainly possible for data to have a distribution that doesn't correspond to any theoretical distribution that has been given a name. The ksdensity function will estimate the density directly without estimating parameters of a theoretical distribution.

[Nuchto]

Ok. So how does the log likelihood here work?

[Tom Lane]

In my example I used poisspdf to get the Poisson density. You could instead use ksdensity to get a nonparametric density estimate. You're likely to find the ksdensity likelihood is higher, since it is based on the sample you provide to it.

[Nuchto]

Thanks, then how do I interpret the resulting coefficient? What does 39.02 say to you?

[Tom Lane]

Sadly, it doesn't say much on its own. You could compare it to the likelihood of other fits.

[Jessica Hopf]

Im curious where the documentation for pd.NLogL is? specifically, I can't find how you would know to do this without having found this answer

Answer 2

[the cyclist]

If you have the Statistics Toolbox, you can calculate the (negative) log likelihood for several functional forms.

For example, there is a betalike() function that will calculate the NLL for a beta function.

[Nuchto]

The problem is I can't find any of those functions for non-lognormal data.

[the cyclist]

I'm not sure I understand what you mean. When you say you can't "find" them, do you mean they are not in your version of MATLAB? Do you have the Statistics Toolbox?

Or do you mean that you see all those functions, but none of them are for the distribution you are trying to use?

Or do you mean something else?

[Nuchto]

I meant the last: none of the functions listed in Matlab R2011a are for my distribution. My distribution is non-log. Anyway, is there a way to know which distribution is my data? I am very much a newbie.

Answer 3

[Oleg Komarov]

You can try the following submission: http://www.mathworks.co.uk/matlabcentral/fileexchange/34943-fit-all-valid-parametric-probability-distributions-to-data

It will fit several distributions and should return the NLL (NegLogLik) for each.