A review of the 25 most popular distributions in statistics using Python

One subject that I never studied in my undergraduate and graduate days is statistics. I therefore have had to catch up on the education that I have lost by studying statistics now. One area in statistics is distributions.

A distribution is a summary of all the possible of values or ranges of values in a dataset and how frequently they appear. Distributions are important because they provide a person with a snapshot of the data he is using. Distributions reveal patterns, showing things like average value (central tendency), how spread out the values are (variability), and whether more values are on one side of the average than the other (skewness). Distributions form a basis for predictions and are the backbone of statistical models.

Distributions form the backbone of statistical models. They help the scientist to make educated guesses or predictions about larger groups based on smaller samples of data. It is important to know what type of distribution data forms, whether it is normal, binomial, or another type to give the scientist an idea of what tests he should be carrying out on the data.

Distributions and probabilities are linked because they help the scientist to understand how likely different outcomes will be.

Distributions are a fundamental concept in statistics. They help in analysis of data and making correct predictions. Each type of distribution offers a unique lens by which the scientist can view and understand data.

All probability distributions can be classified as being either discrete probability distributions or continuous probability distributions, depending on whether they define probabilities associated with either discrete or continuous variables.

Discrete distributions

A discrete distribution is a range of values that can be counted. A distribution that gives the finite trials of a discrete random variable at a given point in time is called a discrete probability distribution.

Discrete probability distribution is defined as the probability at a specific value for a discrete random variable. The discrete probability distributions represent the probability distributions with finite outcomes.

The probability distribution of a discrete random variable lies between 0 and 1, and the sum of probabilities is always equal to 1.

The probability mass function (PMF) is given by:-

The cumulative distribution function (CDF) is given by:-

The eight discrete probability distributions discussed in this blog post are:-

1. Binomial distribution
2. Two point distribution
3. Bernoulli distribution
4. Poisson distribution
5. Geometric distribution
6. Shifted geometric distribution
7. Hypergeometric distribution
8. Discrete uniform distribution

Binomial distribution

The binomial distribution represents the number of successes in a set number of tries. The binomial shape can be symmetric or skewed, depending on the likelihood of success in each try.

The binomial distribution with parameters n and p is the discrete probability distribution of the number of successes in a sequence of n independent experiments, each asking a yes-no question and each with its own boolean valued outcome:- success (with probability p and) or failure (with probability q=1-p).

A single success/failure experiment is called a Bernoulli trial and a sequence of outcomes is called a Bernoulli process. For a single trial (n=1) the binomial distribution is Bernoulli distribution.

The binomial distribution is frequently used to model the number of successes in a sample size n drawn with replacement from a population size N.

The probability mass function (PMF) of the binomial distribution is:-

The Python code below will create a binomial distribution:-

Two point distribution

The two-point distribution is a discrete probability distribution that describes a random variable, which can take on exactly two distinct values. The distribution is a generalisation of the Bernoulli distribution (discussed below) where the two outcomes are not necessarily 0 and 1.

The formula for the probability mass function (PMF) of a two point distribution is:-

The Python code below will create a two point distribution:-

Bernoulli distribution

The Bernoulli distribution is named after the Swiss named after the Swiss mathematician, Jacob Bernoulli, is the discrete probability distribution of a random variable that takes the value of 1 with the probability p and the value 0 with the probability q = 1-p. Less formally, the Bernoulli distribution can be thought of as a model for a set of outcomes of any single experiment that asks a yes-no question, which lead to a boolean valued outcome.

The Bernoulli distribution is a special case of the binomial distribution, where a single trial is conducted so that n would be 1 in such a distribution. It is also a special case of the two-point distribution, where the possible outcome need not be 0 and 1.

The formula for the probability mass function (PMF) for this distribution is:-

The Python code below will create a Bernoulli distribution:-

Poisson distribution

The Poisson is used to count events in a fixed period, such as the number of emails that a person can receive in an hour. The Poisson distribution can be right-skewed for low average numbers of events, but becomes more symmetrical as the average numbers increase.

The Poisson distribution, named after the French mathematician Simeon Denis Poisson, is a discrete probability distribution that expresses the probability of a given number of events occurring in a fixed interval of time if these events occur with a known constant mean rate and independently of the time since the last event. It can also be used for the number of events in other types of intervals than time and in a dimension greater than 1. This distribution plays an important role in discrete-stable distributions.

The probability mass function (PMF) of the Poisson distribution is:-

The Python code below will create a Poisson distribution:-

Geometric distribution

The geometric distribution is a discrete probability distribution that includes the successive failure probability until the success probability is encountered.

The probability mass function of the geometric distribution is:-

The Python code below will create a geometric distribution:-

Shifted geometric distribution

The shifted geometric distribution is a variation of the geometric distribution.

The geometric distribution is the number of trials needed to get the first success in a series of independent Bernoulli trials, with each trial having a possible of two outcomes, being success or failure.

The shifted geometric distribution specifically refers to the total number of trials (including the first success) required to achieve the first success. This means that if X follows a geometric distribution then X+1 follows a shifted geometric distribution.

The formula for the probability mass function (PMF) of the shifted geometric distribution is:-

The Python code below will create a shifted geometric distribution:-

Hypergeometric distribution

The hypergeometric distribution is a discrete probability function that describes the probability of obtaining a specific number of successes in a sequence of draws from a finite population without replacement.

The hypergeometric distribution differs from the binomial distribution, where each draw is independent and with replacement.

The probability mass function (PMF) of the hypergeometric distribution is:-

The Python code below will create a hypergeometric distribution:-

Discrete uniform distribution

The discrete uniform distribution, or rectangle, is a symmetric probability distribution wherein a finite number of values are equally likely to be observed, with every one of n values has equal probability 1/n.

The formula for the discrete uniform distribution is:-

The Python code below will create a discrete uniform distribution:-

Continuous distributions

A continuous, or smooth, distribution is an infinite range of values, which means that the number of possible outcomes is uncountable.

The 17 continuous probability distributions discussed in this blog post are:-

1. Normal distribution
2. Student’s t distribution
3. Compound distribution
4. Uniform distribution
5. Exponential distribution
6. Log-normal distribution
7. Beta distribution
8. Cauchy distribution
9. Levy distribution
10. Gamma distribution
11. Inverse-gamma distribution
12. Erlang distribution
13. Chi-squared distribution
14. Wishart distribution
15. Logistic distribution
16. Tukey lambda distribution
17. Wiebull distribution

Normal distribution

The normal distribution, also known as a bell curve, is symmetrical, with most of the data clusters centring around the mean, or average. It follows the empirical rule, where 68% of the data falls within one standard deviation of the mean, 95% falls within two standard deviations of the mean, and 99.7% falls within three standard deviations of the mean.

The standard deviation tells the scientist how spread out the numbers are in a dataset, revealing how much the data varies from the mean.

The normal distribution is a type of continuous probability distribution for a real valued random variable. Normal distributions are important in statistics and are often used in the natural and social sciences to represent real valued random variables whose distributions are not known.

The normal distribution’s importance is partly due to the central limit theorem, which states that under some conditions the average of many samples of a random variable with finite mean and variance is itself a random variable — whose distribution converges to a normal distribution as the number of samples increases. Therefore, physical quantities that are expected to be the sum of many different processes often have distributions that are nearly normal.

The formula for the normal distribution is:-

The Python code below will create a normal distribution:-

Student’s t distribution

The student’s t distribution is a continuous distribution that generalises the standard normal distribution. Like the normal distribution, it is symmetrical around the 0 make and is bell shaped.

The student’s t distribution has heavier tails than the normal distribution and the amount of probability mass in the tails is controlled by the parameter v.

When v = 1, the distribution becomes the Cauchy distribution and has very fat tails. When v = infinity the distribution becomes the standard normal distribution and has very thin tails.

The formula for student’s t distribution is:-

The Python code below will create a student’s t distribution:-

Compound distribution

A compound, also known as a mixture or contagious distribution, is a probability distribution that results from assuming that a random variable is distributed according to some parameterized distribution. The compound distribution is the result of marginalising over the latent random variable(s) representing the parameters of the parametrized distribution.

The probability density function (PDF) of a compound distribution is:-

The Python code below will create a compound distribution:-

Continuous uniform distribution

The continuous uniform distribution is a type of probability distribution where all outcomes within a specified range are equally likely.

This distribution is defined by two parameters:-

1. A is the minimum value
2. B is the maximum value

The probability density function (PDF) for the continuous uniform distribution is fairly constant, thereby forming a rectangular shape when plotted on a graph.

The formula for the continuous uniform distribution is:-

The Python code below will create a continuous uniform distribution:-

Uniform distribution

In the previous two descriptions of uniform distribution, being discrete uniform distribution and continuous uniform distribution, Python’s scipy library was used to create these distributions.

There is another way to create a normal distribution in Python, however, and that is with Python’s numeric library, numpy.

The numpy code below produces a uniform distribution (continuous):-

The numpy code below produces a uniform distribution (discrete):-

Exponential distribution

The exponential distribution is skewed to the right and is used to measure the time between events. This type of distribution is useful to measure the time between everyday events, such as the time between an earthquake or the time it will take to wait in a queue.

The exponential distribution, or negative exponential distribution, is the probability distribution of the distance between events in a Poisson point process being a process that occurs continuously and independently at a constant average rate. This distribution is a particular case of the gamma distribution. It is the continuous analogue of the geometric distribution and has the key property of being memoryless.

The formula for the exponential function is:-

The Python code below produces an exponential distribution:-

Log-normal distribution

The log-normal distribution is a continuous probability distribution of a random variable whose logarithm is normally distributed. A random variable that is log-normally distributed takes only positive real values. It is a convenient and useful model for measurements in exact and engineering sciences, as well as medicine, economics, and other topics.

The log-normal distribution is occasionally referred to as Galton distribution, after Francis Galton, who was a polymath and originator of eugenics in the Victorian era.

The formula for the log-normal distribution is:-

The Python code below produces a log-normal distribution:-

Beta distribution

The beta distribution is a family of continuous probability distributions defined on the interval (0,1) in terms of two positive parameters, being alpha (a) and beta (b), which appear as exponents of the variable and its complement to 1. Respectively, and control the shape of the distribution.

The formula for the beta distribution is:-

Where k is the number of successes and n is the number of trials.

The Python code below produces a beta distribution:-

Cauchy distribution

The Cauchy distribution, named after the mathematician, engineer, and physicist, Augistin Cauchy, is a continuous probability distribution. It is the distribution of two independent normally distributed random variables with a mean of 0.

The Cauchy distribution is often used in statistics as the canonical example of a pathological distribution since both its expected value and variance are undefined. This distribution does not have finite moments of order greater than or equal to 1, with only factional absolute moments existing.

The Cauchy is one of the few stable distributions with a probability density function (PDF) that can be expressed analytically, with the others being the normal distribution and the Levy distribution.

For the standard Cauchy distribution, where X0=1 and \gamma=1, the probability density function (PDF) is:-

The Python code below produces a Cauchy distribution:-

Levy distribution

The Levy distribution, named after French mathematician Paul Levy, is a continuous probability distribution for a non-negative random variable. This distribution is a special case of the inverse-gamma distribution.

The probability density function (PDF) of the Levy distribution over the domain x >= u:-

The Python code below produces a Levy distribution:-

Gamma distribution

The gamma distribution is a versatile two parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the gamma distribution.

The formula for the gamma distribution is:-

The Python code below produces a gamma distribution:-

Inverse-gamma distribution

The inverse-gamma distribution is a two parameter family of continuous probability distributions on the positive real line, which is the distribution of the reciprocal of a variable distributed according to the gamma distribution. In simple terms, the inverse-gamma distribution is the reciprocal of the gamma distribution, having two positive parameters, being alpha (a) and beta (b).

The shape of the parameter a controls the height. The higher the alpha, the taller the probability density function (PDF) and the thinner the tails.

The scale parameter b controls the spread.

The generalised inverse-gamma distribution has two additional parameters:-

1. The mean, u, is always 0 in the two parameter version.
2. Y, which controls the concentration near the X axis, is always set to 1 in the two parameter version.

The distribution over the interval u to infinity. As the value for X increases, the function decreases exponentially, giving the inverse-gamma distribution very then tails.

The inverse-gamma distribution has the probability density function (PDF):-

The Python code below produces a inverse-gamma distribution:-

Erlang distribution

The Erlang distribution, developed by Danish mathematician, statistician and engineer, A K Erlang, is a two parameter distribution family of continuous probability distributions.

The two parameters of the Erlang distribution are:-

1. The positive integer k is the shape
2. The positive real number l is the rate. The scale, b, is the reciprocal of the rate and is sometimes used instead.

The Erlang and Poisson distributions are complementary, in that while. The Poisson distribution counts the number of events that occur in a fixed amount of time, and the Erlang distribution counts the amount of time until the occurrence of a fixed number of events.

When k=1 the distribution simplifies to an exponential distribution.

The Erlang distribution is a special case of the gamma distribution, where the shape of the distribution is discretized.

The formula for the Erlang distribution is:-

The Python code below produces a Erlang distribution:-

Chi-squared distribution

The chi-squared distribution with k degrees of freedom is a distribution of a sum of the squares of k independent standard normal random variables.

The chi-squared distribution X squared K is a special case of the gamma distribution and the univariate Wishart distribution.

The chi-squared distribution is one of the most widely used distributions in inferential statistics, notably in hypothesis testing and the construction of confidence intervals.

The chi-squared distribution is used in the common chi-squared tests for:-

1. Goodness of fit of an observed distribution to a theoretical one
2. The independence of two criteria of classification of qualitative data
3. For finding the confidence interval for estimating the population standard deviation of a normal distribution from a sample standard deviation.

The formula for chi-squared distribution is:-

The Python code below produces a chi-squared distribution:-

Wishart distribution

The Wishart distribution, named after Scottish mathematician and statistician John Wishart in 1928, is a generalisation of the gamma distribution to multiple dimensions. It is a family of distributions defined over symmetric, positive-definite random matrices.

The formula for the Wishart distribution is:-

The Python code below produces a Wishart distribution:-

Logistic distribution

The logistic distribution is a continuous probability distribution. Its cumulative distribution function (CDF) is the logistic function, which appears in logistic regression and feed forward neural networks. Thi logistic distribution resembles the normal distribution, but has heavier tails. The logistic distribution is a special case of the Tukey Lambda distribution.

The probability density function (PDF) of the logistic distribution is:-

The cumulative distribution function (CDF) of the logistic distribution is:-

The Python code below produces a logistic distribution:-

Tukey lambda distribution

The Tukey lambda distribution, formalised by American mathematician and statistician John Tukey, is a continuous symmetrically defined in terms of quantile function. It is typically used to identify an appropriate distribution.

The quantile function outputs the value of a random variable such that its probability is less than or equal to an input probability value. The quantile function associates with a range at and below a probability input the likelihood that a random variable is realised in that range for some probability distribution.

The quantile function is one way of prescribing a probability distribution and is an alternative to the probability density function (PDF) or probability mass function, the cumulative distribution function (CDF) and the comparative function.

The quantile function Q of a probability distribution is the inverse of its cumulative distribution function (CDF) F.

The Python code below produces a Tukey lambda distribution:-

Weibull distribution

The Weibull distribution is a continuous probability distribution named after Swedish mathematician Waloddi Weibull. This distribution models a broad range of random variables, largely in the nature of a time to failure or a time between events.

A couple of examples of where the Weibull distribution can be used is:-

1. Maximum one-day rainfall
2. The time a user spends on a web page

The Python code below produces a Tukey lambda distribution:-

Conclusion

When I began putting together this blog post on the most popular distributions in use today, I thought I would be presented with a handful of distributions at most. What I found, however, was a group of 25 separate distributions, being discrete and continuous.

Python’s scientific library, scipy, has about a dozen discrete distributions and around 100 continuous distributions. I could not put all 112 distributions in this blog post because that would make it absolutely enormous. I therefore chose the distributions that came up in my research to include in this post.

Because my life circumstances required me to go to work at a young age, I never had the opportunity to study higher mathematics in my education. I have, however included the formulas where possible for those individuals who are mathematically inclined.