Econometric Theory/Probability Density Function (PDF)

Probability Mass Function of a Discrete Random Variable

A probability mass function f(x) (PMF) of X is a function that determines the probability in terms of the input variable x, which is a discrete random variable (rv).

A pmf has to satisfy the following properties:

• $f(x) =\begin{cases} P(X = x_i) & \mbox{for } i = 1, 2, \cdots, n \\ 0 & \mbox{for } x \ne x_i \end{cases}$

• The sum of PMF over all values of x is one:

$\sum_i f(x_i)= 1.$

Probability Density Function of a Continuous Random Variable

The continuous PDF requires that the input variable x is now a continuous rv. The following conditions must be satisfied:

• All values are greater than zero.

$f(x) \ge 0$

• The total area under the PDF is one

$\int_{- \infty}^{\infty} f(x) \, dx = 1$

• The area under the interval [a, b] is the total probability within this range

$\int_{a}^{b} f(x) \, dx = P(a \le x \le b)$

Joint Probability Density Functions

Joint pdfs are ones that are functions of two or more random variables. The function

$\begin{align} f(X \in A, Y \in B) & = \int_{A} \, \int_{B} f(x,y) \, dx \, dy \\ & = 0, \mbox{if } x \notin A \mbox{ and } y \notin B \\ \end{align}$

is the continuous joint probability density function. It gives the joint probability for x and y.

The function

$\begin{align} p(X \in A, Y \in B) & = \sum_{X \in A} \sum_{Y \in B} p(x, y) \\ & = 0, \mbox{if } x \notin A \mbox{ and } Y \notin y \\ \end{align}$

is similarly the discrete joint probability density function

Marginal Probability Density Function

The marginal PDFs are derived from the joint PDFs. If the joint pdf is integrated over the distribution of the X variable, then one obtains the marginal PDF of y, $f(y)$. The continuous marginal probability distribution functions are:

$f(x) = \int_{y}^B f(x,y) dy$

$f(y) = \int_{x}^A f(x,y) dx$

and the discrete marginal probability distribution functions are

$p(x) = \sum_{y \in B} p(x, y)$

$p(y) = \sum_{x \in A} p(x, y)$

Conditional Probability Density Function

$f(x \mid y) = P(X = x, Y = y) = \frac{f(x,y)}{f(y)}$

$f(y \mid x) = P(Y = y, X = x) = \frac{f(x,y)}{f(x)}$

Statistical Independence

• Gujarati, D.N. (2003). Basic Econometrics, International Edition - 4th ed.. McGraw-Hill Higher Education. pp. 870-877. ISBN 0-07-112342-3.