Skip to content

Probability

This page reviews concepts from probability theory that are useful for econometrics.

Related pages: Measure theory (an advanced treatment of probability) • Statistics

§1. Basic probability theory

Probability function

TBD

TBD

Conditional probability

TBD

Bayes' rule

TBD

§2. Random variables

Discrete random variables

Continuous random variables

Moments

Expectation \(\mu\)

Variance \(\sigma^2\)

Skewness

Kurtosis

Higher-order moments

Moment-generating function (MGF)

§3. Common univariate distributions

Discrete distributions

Bernoulli distribution

A random variable \(X\) follows a Bernoulli distribution with parameter \(p\) if

\[\begin{equation} X = \begin{cases} 1, & \text{with probability } p, \\ 0, & \text{with probability } 1 - p. \end{cases} \end{equation}\]

We have:

\[\begin{align} \mathrm{E}(X) &= p \\ \mathrm{Var}(X) &= p(1-p). \end{align}\]

Binomial distribution

Poisson distribution

Classical inference distributions

Normal distribution

A normal distribution \(\mathcal{N}(\mu,\sigma^{2})\) has a location parameter \(\mu\) (mean) and a scale parameter \(\sigma^2\) (variance). Its density function follows:

\[\begin{equation} f(x) = \frac{1}{\sqrt{2\pi\sigma^{2}}} e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}. \end{equation}\]

When \(\mu=0\) and \(\sigma^2=1\), the distribution is called a standard normal distribution, denoted by \(\mathcal{N}(0,1)\). Its density function simplifies to:

\[\begin{equation} f(x) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^{2}}{2}}, \end{equation}\]

whose graph is shown below:

-2-11201

For the standard normal distribution, the density function is usually denoted by \(\phi\), and the cumulative distribution function is usually denoted by \(\Phi\).

Links:

Student's t-distribution

TBD

Chi-squared distribution

TBD

F-distribution

TBD

Common continuous distributions

Uniform distribution

Exponential distribution

Gamma distribution

Logistic distribution

TBD

Cauchy distribution

Pareto distribution

Extreme value distributions

GEV distribution

TBD

Gumbel distribution

TBD

Weibull distribution

Fréchet distribution

§4. Multivariate parametric distributions

Independence

TBD

Covariance and correlation

TBD
TBD

Conditional expectation

Law of iterated expectations

TBD

Useful multivariate distributions

Multivariate normal distribution

Multinomial distribution

Dirichlet distribution

§5. Inequalities in probability theory

Chebyshev's inequality

Markov's inequality

Jensen's inequality

TBD