Probability
This page reviews concepts from probability theory that are useful for econometrics.
Related pages: Measure theory (an advanced treatment of probability) • Statistics
§1. Basic probability theory¶
Probability function¶
TBD
TBD
Conditional probability¶
TBD
Bayes' rule¶
TBD
§2. Random variables¶
Discrete random variables¶
Continuous random variables¶
Moments¶
Expectation \(\mu\)¶
Variance \(\sigma^2\)¶
Skewness and kurtosis¶
Higher-order moments¶
Moment-generating function (MGF)¶
§3. Common univariate distributions¶
Discrete distributions¶
Bernoulli distribution¶
A random variable \(X\) follows a Bernoulli distribution with parameter \(p\) if
\[\begin{equation}
X =
\begin{cases}
1, & \text{with probability } p, \\
0, & \text{with probability } 1 - p.
\end{cases}
\end{equation}\]
We have:
\[\begin{align}
\mathrm{E}(X) &= p \\
\mathrm{Var}(X) &= p(1-p).
\end{align}\]
Binomial distribution¶
Poisson distribution¶
Classical inference distributions¶
Normal distribution¶
A normal distribution \(\mathcal{N}(\mu,\sigma^{2})\) has a location parameter \(\mu\) (mean) and a scale parameter \(\sigma^2\) (variance). Its density function follows:
\[\begin{equation}
f(x) = \frac{1}{\sqrt{2\pi\sigma^{2}}}
e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}.
\end{equation}\]
When \(\mu=0\) and \(\sigma^2=1\), the distribution is called a standard normal distribution, denoted by \(\mathcal{N}(0,1)\). Its density function simplifies to:
\[\begin{equation}
f(x) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^{2}}{2}},
\end{equation}\]
whose graph is shown below:
Student's t-distribution¶
TBD
Chi-squared distribution¶
TBD
F-distribution¶
TBD
Common continuous distributions¶
Uniform distribution¶
Exponential distribution¶
Gamma distribution¶
Logistic distribution¶
TBD
Cauchy distribution¶
Pareto distribution¶
Extreme value distributions¶
GEV distribution¶
TBD
Gumbel distribution¶
TBD
Weibull distribution¶
Fréchet distribution¶
§4. Multivariate parametric distributions¶
Independence¶
TBD
Covariance and correlation¶
TBD
TBD
Conditional expectation¶
Law of iterated expectations¶
TBD
Useful multivariate distributions¶
Multivariate normal distribution¶
Multinomial distribution¶
Dirichlet distribution¶
§5. Inequalities in probability theory¶
Chebyshev's inequality¶
Markov's inequality¶
Jensen's inequality¶
TBD