Let $X,Y,Z$ represent random variables and let $p_X(x)$ represent the probability density for the random variable $X$. Similarly, we write $p_{X,Y}(x,y)$ for the joint probability distribution of two random variables.
Let $f(x)$ be any function of $x$. Consider the random variable $Z$ defined by the equation
(1)What is the probability density of the random variable $Z$ given the probability density of $X$? Let us pick a value $z$ and let $x_1,x_2,\ldots$ be the set of points that solve the equation $z=f(x)$. Thus, the probability density $p_Z(z)$ will be determined by the probability density of the random variable $X$ at $x_1,x_2,\ldots$. We need to add the different contributions with some weight that has to be determined. How do we do that? We claim that the following holds:
(2)The above formula satisfies all the required conditions: (i) given a value $z$, it gets contributions only from the set of points $x_1,x_2,\ldots$ that solve the equation $z=f(x)$; (ii) it is suitably normalized. A simple way to prove the above is to first consider the case when the random variables take values in a discrete set. Then, one deals with probabilities rather than densities and the Dirac delta function is replaced by the Kronecker delta. Of course, the integral gets replaced by a sum.
Further simplification of the above formula can be carried out by using the following identity involving delta functions:
(3)Using this we can see that:
(4)Note that $z$ is implicitly present in the RHS of the above equation as the roots $x_i$ implicitly depend on it.
Another generalization of the above formula occurs when we consider multi-variable functions such as $z=f(x,y)$. The corresponding equation involving random variables $X,Y,Z$ is
(5)One writes a formula similar to Eq. (2) above:
(6)When $X$ and $Y$ are independent random variables, then the joint probability density is nothing but the product of the individual probability densities. Then, we can write
(7)The most common function that is considered is the sum i.e., $z=x+y$. In such situations, one of the integrals in the above equation can be carried out to obtain the probability density as the convolution of the two probability densities.
(8)