Probability Distributions and their Stories

Given any module that deals with statistics, one basic skill you must have is to be able to program and create plots of probability distributions typically encountered in the field of data science. This tutorial should remind you of various distributions introduced in this section, but now they are phrased using the scipy.stats module.

Continuous distributions

Half-Normal distribution

  • Story. The Half-Normal distribution is a Gaussian distribution with zero mean truncated to only have nonzero probability for positive real numbers.
  • Parameters. Strictly speaking, the Half-Normal is parametrized by a single positive parameter σ. We could, however, translate it so that the truncated Gaussian has a maximum at μ and only values greater than or equal to μ have nonzero probability.
  • Probability density function.
    \begin{align}f(y;\mu, \sigma) = \left\{\begin{array}{ccc}\frac{2}{\sqrt{2\pi \sigma^2}}\,\mathrm{e}^{-(y-\mu)^2/2\sigma^2} & & y \ge \mu\\0 && \text{otherwise}\end{array}\right.\end{align}

  • Support. The Half-Normal distribution is supported on the set [μ,∞). Usually, we have μ=0, in which case the Half-Normal distribution is supported on the set of nonnegative real numbers.
  • Usage

    Package Syntax
    NumPy np.random.normal(mu, sigma)
    SciPy scipy.stats.halfnorm(mu, sigma)
    Stan real<lower=mu> y; y ~ normal(mu, sigma)

  • Notes.
    • In Stan, a Half-Normal is defined by putting bounds on the variable and then using a Normal distribution.
    • The Half-Normal distibution is a useful prior for nonnegative parameters that should not be too large and may be very close to zero.
params = [dict(name='µ', start=-0.5, end=0.5, value=0, step=0.01),
          dict(name='σ', start=0.1, end=1.0, value=0.2, step=0.01)]
app = distribution_plot_app(x_min=-0.5,
                            x_max=2,
                            scipy_dist=st.halfnorm,
                            params=params,
                            x_axis_label='y',
                            title='Half-Normal')
bokeh.io.show(app, notebook_url=notebook_url)