Cumulant Generating Function applied on a Gamma Distribution

The Gamma distribution is given by;

\displaystyle  G(x|k,\theta)=\frac{x^{k-1}e^{-x/\theta}}{\theta^k \Gamma(k)} \ \ \ \ \ (1)

We collect all the terms to the exponent, and have;

\displaystyle G(x|k,\theta)=\frac{e^{(-x/\theta)+(k-1)\ln x}}{e^{k \ln \theta + \ln \Gamma(k)}} \ \ \ \ \ (2)

Since we know this distribution to be a valid one, from the axioms of probability theory, we know that, if we integrate this expression with respect to x in {(0, \infty)}, we must have one. And we observe that the denominator is a constant, which can’t be of any use other than being a normalization constant. So we define a function,

\displaystyle  A(\alpha_1, \alpha_2)=\ln \int e^{(-x/\theta)+(k-1)\ln x}dx \ \ \ \ \ (3)

where; {\alpha_1=-1/\theta}; {\alpha_2 = k-1}. Note that, the integral inside the logarithm gives the normalization constant. The latter can be seen from equation (2), since the integral of the numerator must be equal to the denominator, which is the normalization constant itself. So, for example if we consider, {\partial A / \partial \alpha_1}, we get;

\displaystyle \frac{\partial A(\alpha_1, \alpha_2)}{\partial d\alpha_1}=\frac { \int xe^{(-x/\theta)+(k-1)\ln x}dx} { \int e^{(-x/\theta)+(k-1)\ln x}dx} = E[x] \ \ \ \ \ (4)

We find the expected value of x.But the thing is, we know a super-easy to differentiate expression for {A}, since it is the natural logarithm of the normalization constant;

\displaystyle A(\alpha_1, \alpha_2)=\ln \Big\{ {e^{k \ln \theta + \ln \Gamma(k)}}\Big\}=k \ln \theta + \ln \Gamma(k) \ \ \ \ \ (5)


\displaystyle E[x] = \frac{\partial A(\alpha_1, \alpha_2)}{\partial \alpha_1}= \frac{\partial (k \ln \theta + \ln \Gamma(k))}{\partial (-1/\theta)}=k\theta \ \ \ \ \ (6)


\displaystyle E[\ln x] = \frac{\partial A(\alpha_1, \alpha_2)}{\partial d\alpha_2}= \frac{\partial (k \ln \theta + \ln \Gamma(k))}{\partial (k-1)}=\frac{\partial (k \ln \theta + \ln \Gamma(k))}{\partial k}= \ln \theta + \frac{\Gamma'(k)}{\Gamma(k)} \ \ \ \ \ (7)

There’s a special name given for {\frac{\Gamma'(k)}{\Gamma(k)}}, which is digamma function. We denote it by {\psi(k)}. So, according to the convention, {E[\ln x]= \ln \theta + \psi ( k)}.

This entry was posted in Probability, Uncategorized and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s