From the Poisson Distribution to the Normal Distribution

In this note, we will study how one obtains the normal distribution as a particular limit of the Poisson distribution. Recall that the Poisson distribution is a discrete probability distribution.

(1)

\begin{align} P(n) = \frac{e^{-\lambda}\ \lambda^n}{n!}\quad n \in \mathbb{Z}_{\geq0} \end{align}

Let $X$ be a Poissonian random variable. It is known that all cumulants $\kappa_r\ , r=1,2,\ldots$ are equal to $\lambda$, the only parameter appearing in the probability distribution. In particular, the mean ($\kappa_1$) and variance ($\kappa_2$) are both equal to $\lambda$. In order to obtain the Normal distribution, we need to do a couple of transformations.

1. We first construct a new random variable $Y=(X-\lambda)$. This has zero mean but all other cumulants remain unchanged and equal to $\lambda$
2. We now re-scale to obtain a new variable $Z=\delta\, Y$. We expect the variable $Z$ to become a continuous variable taking values in $\mathbb{R}$ when we take the $\delta\rightarrow0$ limit. Before taking the limit, let us consider the cumulants of $Z$.

(2)

\begin{align} \kappa_1=0\quad,\quad \kappa_r = \lambda\ \delta^r \quad \text{for } r=2,3,\ldots \end{align}

The naive continuum limit $\delta\rightarrow0$ will not work as ALL cumulants vanish. Suppose, instead we carry out the double scaling limit:

(3)

\begin{align} \delta\rightarrow0 \textrm{ and }\lambda\rightarrow\infty \textrm{ keeping }\kappa_2=\lambda\ \delta^2 =\sigma^2\ , \textrm{ a constant.} \end{align}

In this limit the variance doesn't vanish but higher ones vanish. We then see that limiting distribution is continuous and has zero mean, variance equal $\sigma^2$ with all higher cumulants vanishing. Thus we obtain a normal distribution with zero mean and variance $\sigma^2$ as it is the unique probability distribution with all higher cumulants vanishing.

Remark: (by Prof. V. Balakrishnan) The double scaling is precisely the diffusion limit of a discrete time random walk with time step $(\lambda)^{-1}$ and length step $\delta$.

The probability distribution before the double scaling limit is (treating $Z$ as a continuum variable)

(4)

\begin{align} p_Z(z) = \frac1\delta \frac{e^{-\lambda}\ \lambda^{\lambda + (z/\delta)}}{\Gamma(1+\lambda+(z/\delta))} \end{align}

Exercise: Show that we recover the normal distribution on taking the doubling scaling limit.

Below we plot the continuous probability density function after setting $\delta=1/ \sqrt{\lambda}$ keeping $\lambda$ as a free parameter. In the limit $\lambda\rightarrow 0$ this corresponds to $\sigma^2=1$ in the limit. We plot the probability density for $\lambda=1,5,10,50,100$ with the normal distribution with zero mean and unit variance plotted in orange as a reference. We see that at $\lambda=100$, the normal distribution is a very good approximation.

Note: This post was inspired by Prof. Sudhir Raniwala's question on another forum.