Cumulant Generating Function applied on a Gamma Distribution

Posted on July 23, 2010 by csubakan

The Gamma distribution is given by;

\displaystyle G(x|k,\theta)=\frac{x^{k-1}e^{-x/\theta}}{\theta^k \Gamma(k)} \ \ \ \ \ (1)

We collect all the terms to the exponent, and have;

\displaystyle G(x|k,\theta)=\frac{e^{(-x/\theta)+(k-1)\ln x}}{e^{k \ln \theta + \ln \Gamma(k)}} \ \ \ \ \ (2)

Since we know this distribution to be a valid one, from the axioms of probability theory, we know that, if we integrate this expression with respect to x in {(0, \infty)}, we must have one. And we observe that the denominator is a constant, which can’t be of any use other than being a normalization constant. So we define a function,

\displaystyle A(\alpha_1, \alpha_2)=\ln \int e^{(-x/\theta)+(k-1)\ln x}dx \ \ \ \ \ (3)

where; {\alpha_1=-1/\theta}; {\alpha_2 = k-1}. Note that, the integral inside the logarithm gives the normalization constant. The latter can be seen from equation (2), since the integral of the numerator must be equal to the denominator, which is the normalization constant itself. So, for example if we consider, {\partial A / \partial \alpha_1}, we get;

\displaystyle \frac{\partial A(\alpha_1, \alpha_2)}{\partial d\alpha_1}=\frac { \int xe^{(-x/\theta)+(k-1)\ln x}dx} { \int e^{(-x/\theta)+(k-1)\ln x}dx} = E[x] \ \ \ \ \ (4)

We find the expected value of x.But the thing is, we know a super-easy to differentiate expression for {A}, since it is the natural logarithm of the normalization constant;

\displaystyle A(\alpha_1, \alpha_2)=\ln \Big\{ {e^{k \ln \theta + \ln \Gamma(k)}}\Big\}=k \ln \theta + \ln \Gamma(k) \ \ \ \ \ (5)

So;

\displaystyle E[x] = \frac{\partial A(\alpha_1, \alpha_2)}{\partial \alpha_1}= \frac{\partial (k \ln \theta + \ln \Gamma(k))}{\partial (-1/\theta)}=k\theta \ \ \ \ \ (6)

and;

\displaystyle E[\ln x] = \frac{\partial A(\alpha_1, \alpha_2)}{\partial d\alpha_2}= \frac{\partial (k \ln \theta + \ln \Gamma(k))}{\partial (k-1)}=\frac{\partial (k \ln \theta + \ln \Gamma(k))}{\partial k}= \ln \theta + \frac{\Gamma'(k)}{\Gamma(k)} \ \ \ \ \ (7)

There’s a special name given for {\frac{\Gamma'(k)}{\Gamma(k)}}, which is digamma function. We denote it by {\psi(k)}. So, according to the convention, {E[\ln x]= \ln \theta + \psi ( k)}.

This entry was posted in Probability, Uncategorized and tagged , , , , . Bookmark the permalink.

Leave a comment