Probability Distributions and their Stories

Given any module that deals with statistics, one basic skill you must have is to be able to program and create plots of probability distributions typically encountered in the field of data science. This tutorial should remind you of various distributions introduced in this section, but now they are phrased using the scipy.stats module.

Continuous Multivariate distributions

Multivariate Gaussian, a.k.a. Multivariate Normal, distribution

  • Story. This is a generalization of the univariate Gaussian.
  • Example. Finch beaks are measured for beak depth and beak length. The resulting distribution of depths and length is Gaussian distributed. In this case, the Gaussian is bivariate, with μ=(μ_d,μ_l) and \begin{align}\mathsf{\Sigma} = \begin{pmatrix}\sigma_\mathrm{d}^2 & \sigma_\mathrm{dl} \\\sigma_\mathrm{dl} & \sigma_\mathrm{l}^2\end{pmatrix}\end{align}.
  • Parameters. There is one vector-valued parameter, μ, and a matrix-valued parameter, Σ, referred to respectively as the mean and covariance matrix. The covariance matrix is symmetric and strictly positive definite.
  • Support. The K-variate Gaussian distribution is supported on \mathbb{R}^K.
  • Probability density function.

    \begin{align}f(\mathbf{y};\boldsymbol{\mu}, \mathsf{\Sigma}) = \frac{1}{\sqrt{(2\pi)^K \det \mathsf{\Sigma}}}\,\mathrm{exp}\left[-\frac{1}{2}(\mathbf{y} - \boldsymbol{\mu})^T \cdot \mathsf{\Sigma}^{-1}\cdot (\mathbf{y} - \boldsymbol{\mu})\right].\end{align}

  • Usage The usage below assumes that mu is a length K array, Sigma is a K×K symmetric positive definite matrix, and L is a K×K lower-triangular matrix with strictly positive values on teh diagonal that is a Cholesky factor.

    • Package Syntax
    NumPy np.random.multivariate_normal(mu, Sigma)
    SciPy scipy.stats.multivariate_normal(mu, Sigma)
    Stan multi_normal(mu, Sigma)
    NumPy Cholesky np.random.multivariate_normal(mu, np.dot(L, L.T))
    SciPy Cholesky scipy.stats.multivariate_normal(mu, np.dot(L, L.T))
    Stan Cholesky multi_normal_cholesky(mu, L)


  • Related distributions.
    • The Multivariate Gaussian is a generalization of the univariate Gaussian.
  • Notes.
  • The covariance matrix may also be written as Σ=S⋅C⋅S, where
    S=\sqrt{diag(Σ)},
    and entry i, j in the correlation matrix C is

    C_{ij}=σ_{ij}/σ_iσ_j.

    Furthermore, because Σ is symmetric and strictly positive definite, it can be uniquely defined in terms of its Cholesky decomposition, L, which satisfies Σ=L ⋅ LT. In practice, you will almost always use the Cholesky representation of the Multivariate Gaussian distribution in Stan.