⑦ Probability
§1. Basic probability theory¶
Probability function¶
TBD
TBD
Conditional probability¶
TBD
Bayes' rule¶
TBD
§2. Random variables¶
Discrete random variables¶
Continuous random variables¶
Moments¶
Expectation \(\mu\)¶
Variance \(\sigma^2\)¶
Skewness¶
Kurtosis¶
Higher-order moments¶
§3. Common univariate distributions¶
Discrete distributions¶
Bernoulli distribution¶
A random variable \(X\) follows a Bernoulli distribution with parameter \(p\) if
\[\begin{equation}
X =
\begin{cases}
1, & \text{with probability } p, \\
0, & \text{with probability } 1 - p.
\end{cases}
\end{equation}\]
We have:
\[\begin{align}
\mathrm{E}(X) &= p \\
\mathrm{Var}(X) &= p(1-p).
\end{align}\]
Binomial distribution¶
Poisson distribution¶
Classical inference distributions¶
Normal distribution¶
A normal distribution \(\mathcal{N}(\mu,\sigma^{2})\) has a location parameter \(\mu\) (mean) and a scale parameter \(\sigma^2\) (variance). Its density function follows:
\[\begin{equation}
f(x) = \frac{1}{\sqrt{2\pi\sigma^{2}}}
e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}.
\end{equation}\]
When \(\mu=0\) and \(\sigma^2=1\), the distribution is called a standard normal distribution, denoted by \(\mathcal{N}(0,1)\). Its density function simplifies to:
\[\begin{equation}
f(x) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^{2}}{2}},
\end{equation}\]
whose graph is shown below:
For the standard normal distribution, the density function is usually denoted by \(\phi\), and the cumulative distribution function is usually denoted by \(\Phi\).
Links:
Student's t-distribution¶
TBD
Chi-squared distribution¶
TBD
F-distribution¶
TBD
Common continuous distributions¶
Uniform distribution¶
Exponential distribution¶
Gamma distribution¶
Logistic distribution¶
TBD
Cauchy distribution¶
Pareto distribution¶
Extreme value distributions¶
GEV distribution¶
TBD
Gumbel distribution¶
TBD
Weibull distribution¶
Fréchet distribution¶
§4. Multivariate parametric distributions¶
Independence¶
TBD
Covariance and correlation¶
TBD
TBD
Conditional expectation¶
Law of iterated expectations¶
TBD
Useful multivariate distributions¶
Multivariate normal distribution¶
Multinomial distribution¶
Dirichlet distribution¶
References¶
TBD