A foray into the functions that apply to continuous distributions

I have been studying statistics for a few years now and one thing that I have discovered is that whenever I learn one concept, I realise there are other concepts that I have yet to learn.

As I have begun studying distributions, I have realised that there are several functions that can be used with those distributions to help to calculate values associated with the distribution.

With regard to continuous distributions, of which the normal distribution is one, there are five functions that can be applied:-

1. Probability density function (PDF)
2. Cumulative distribution function (CDF)
3. Percent point function (PPT)
4. Survival function
5. Hazard function

In this blog I intend to explore each of the five functions listed above.

Probability density function (PDF)

The PDF of a continuous random variable X describes the likelihood of the variable taking on a specific value. Unlike discrete random variables, where the probability is expected to be an exact value, with continuous random variables, the probability of x being a specific value is zero. Instead, the PDF is used to calculate the probability that a value will be within a specific range.

The PDF is denoted as f(x) and satisfies the integral:-

The above integral satisfies the probability that the random variable X falls within the interval [a,b].

The PDF is a non-negative number, being f(x) >= 0 for all x.

The total area under the PDF curve is equal to 1, ensuring that the sum of all probabilities over all possible values of the random variable is 1:-

The Python code below is a visualisation of PDF in a standard normal distribution:-

The PDF curve shows the density of the probabilities at different values of x. The peak of the curve at x=0 indicates that the most likely value of the standard normal distribution is 0.

The area under the curve of any two points gives the probability that the random variable falls within that interval.

A practice question utilising PDF can be seen below:-

Cumulative distribution function (CDF)

The CDF for a continuous distribution provides the probability that a continuous random variable X takes on a value less than or equal to a specific value x.

The CDF, denoted by F(x) for a continuous random variable X is defined as:-

The CDF is non-decreasing, meaning that F(x1) < F(x2) if x1 is less than x2.

As x approaches negative infinity, CDF approaches zero.

As x approaches positive infinity, CDF approaches one.

The value of CDF at a particular point x, F(x) represents the probability that the random variable X is less than or equal to x.

The Python code below is a visualisation of the CDF:-

The applications of CDF are:-

1. Probability calculations
2. Statistical analysis
3. Risk management

A second practice problem in Python, using 1-CDF, can be seen below:-

Percent point function (PPF)

PPF is also known as Quantile Function or Inverse cumulative Distribution Function (Inverse CDF) gives the value below which a given percentage of observations in a continuous distribution falls. It provides the value corresponding to a specific cumulative probability.

PPF is valued at F(x) = p.

The PPF is used to find quantiles.

The PPF is the inverse of the CDF. While the CDF maps values to cumulative probabilities, the PPF maps cumulative probabilities back to values.

An example of PPF can be found in the Python code below. The exercise is to find the 90th percentile of a standard normal distribution with a mean of 75 and a standard deviation of 10. The answer to this question is 87.82:-

The PPF can be used in:-

1. Statistics
2. Finance
3. Quality control

Survival function

The survival function is also known as the complementary cumulative distribution function (CCDF) and is defined as:-

S(x) = P(X >x) = 1-F(x)

While the CDF gives the probability that X is less than or equal to x, the survival function gives the probability that X is greater than x.

The survival function is non-increasing.

As x approaches negative infinity, the survival function approaches one.

As x approaches positive infinity, the survival function approaches zero.

The illustration below show how the survival function changes from one to zero.

Below is the standard normal distribution made depicting the survival function:-

The survival function can be used in:-

1. Reliability engineering
2. Survival analysis
3. Risk management

A sample question using the survival function can be seen below:-

A visualisation in Python of the above question can be seen below:-

Hazard function

The hazard function is a crucial concept in survival analysis and reliability engineering. It represents the instantaneous rate at which an event occurs, given that the individual or item has survived up to a certain time.

The hazard function h(t) is defined as the limit of the probability that an event occurs in a small interval of time, given that the individual or item has survived up to the start of the interval.

The mathematical representation for a continuous random variable T representing the time until an event occurs, is defined as:-

The hazard function can be expressed in terms of the PDF f(t) and the survival function S(t); with f(t) representing the time until the event occurs and S(t) representing the probability of surviving beyond time t.

The hazard function h(t) represents the instantaneous risk of the event occurring at time t, given that the individual or item has survived up to time t.

The cumulative hazard function H(t) is the integral of the hazard function over time, being:-

The cumulative hazard function can be used to express the survival function, as being:-

Applications of the hazard function are:-

1. Reliability engineering
2. Survival analysis
3. Risk management

In theory, the hazard function can be calculated for any probability distribution, but it is more commonly associated with distributions in survival analysis and reliability engineering, wuch as exponential, log-normal, and Weibull distributions.

The Python code below calculates the hazard function in a log-normal distribution:-

The below practice problem using a log-normal distribution reveals that the hazard increases with time, making the two variables directly proportional to each other:-

To summarise, I have covered five functions that are associated with continuous distributions. Statistics is a very in depth subject and there is much to learn before an individual can become proficient in all aspects of it, if it is at all possible.