A problem when using truncate function for exponential distribution

Hi everyone,

I want to sampling from a truncated exponential distribution with lambda, and the truncate values are 40 for lower bound, and 10000 for upper bound. It means that we only care samples from the interval [40, 10000]. When I try with the following codes:

    pd=makedist('Exponential','mu',0.005);
    t=truncate(pd,40,10000);

I got the message

Error using prob.TruncatableDistribution/checkTruncationArgs (line 495)
The truncation values define an interval of zero probability.

Error in prob.TruncatableDistribution/truncate (line 59)
            checkTruncationArgs(this,lower,upper);

Can anyone help me by explaining why this happened. If possible, what is a correct solution.

Best, regards,

[John D'Errico]

Cleaned up the code formatting to make it more readable.

Answer 1

[John D'Errico]

Whenever you see something like this, think about what it is telling you. Do some experimentation. GET YOUR HANDS DIRTY. (figuratively)

pd=makedist('Exponential','mu',0.005);

% use long g, to see if there is ANY probability there.
format long g

pd.cdf(40)
ans =
     1

pd.cdf(10000)
ans =
     1

Now, re-read that error message! Think about the distribution you have chosen, and the parameter you have specified. mu was 0.005.

Even for a pretty small value, the CDF is still almost maxed out.

pd.cdf(.1)
ans =
         0.999999997938846

Any guesses what the probability of getting a value that is 40 or LARGER from that distribution? I'd need to do it using HPF, in a few thousand digits. Hmm........

So what is the CDF of your exponential?

the PDF is

PDF(x,mu) = exp(-x/mu)/mu

(The mean is mu in this form.)

The CDF is

CDF(x,mu) = 1 - exp(x/mu)

So now, using HPF in a few hundred digits of precision ... (yes, I COULD have used the symbolic TB, but when you wrote your own, why not use it?)

CDF = @(x,mu) 1 - exp(-hpf(x)/hpf(mu));
DefaultNumberOfDigits 10000

Compute the PDF at x = 40. As it turns out, this number is 0.9999...999, with a LOT of 9's. How many?

P40 = CDF(40,0.005);
double(log10(1 - P40))
ans =
         -3474.35585522601

I think the answer is, you need to understand the distribution you are working with. Perhaps you simply don't realize that there are TWO ways to define an exponential distribution? One uses a parameterization in terms of the mean. The other uses lambda, the inverse of that mean. I wonder if this is what has you mixed up? Perhaps this would help, if I tried...

pd=makedist('Exponential','mu',1/0.005);

pd.cdf(40)
ans =
         0.181269246922018

pd.cdf(10000)
ans =
     1

Just a guess though.

[Cuong]

Thank you for your detailed answer.

However, I do not agree with you. What I am working here is a truncated distribution, but not the original exponential distribution.

For example, if I modify the above codes a little bit, such as.

pd=makedist('Exponential','mu',1);
t=truncate(pd,20,Inf);

There is no issue with the above lines of codes. The mean of truncated distribution t is 20+1=21. (Because the mean of the original distribution is mu=1).

When I type:

cdf(t,22)

The output should be: 0.8647. It means that there is a lot of chance that a large number of values between 20 and 22 will be selected.

I am not sure when I replace 20 by 40, the codes can not work. i.e : pd=makedist('Exponential','mu',1); t=truncate(pd,20,Inf);

Best

[John D'Errico]

Whether or not you agree with me does not matter. It simply says you don't understand floating point arithmetic. It is simple here, black and white. The CDF at both points is identically 1 in floating point arithmetic. READ THE ERROR MESSAGE.

MATLAB does not understand that SYMBOLICALLY, one can work with a truncated exponential as you want it to do. NUMERICALLY, in terms of the floating point arithmetic, the truncate function does NOT look at your distribution and recognize that oh, gosh, a truncated exponential is a tremendously simple thing! None of the other common distributions have that property, so to do so would be a surprise, and since you got the error message you did, the answer is obvious. Instead, truncate obviously looks at the CDF and sees (and I will quote MATLAB here)

"The truncation values define an interval of zero probability."

The fact is, you are NOT working with a truncated distribution. You are creating the complete exponential distribution, and THEN you want MATLAB to truncate it.

If you want to write your OWN truncated exponential distribution, it is easy enough to do. So do so instead of trying to argue with me. All you will do is waste your time and mine. I'm not the author of that code, and what it does is obvious. Again, the disconnect here is that you are thinking MATHEMATICS, NOT in term of floating point arithmetic.