Differentiating Sigmoid Function

Sigmoid function is an extremely popular function that we leverage in modern day machine learning. Specially while solving classification problems from logistic regression to neural networks. Also often we need to compute gradient of sigmoid function. In this post we will see how we can compute gradient of sigmoid function.

We will take a quick look at how the sigmoid function looks like

import math
import numpy as np
import matplotlib.pyplot as plt

def sigmoid(x):
    return(1/(1+math.exp(1)**(-x)))

domain_def = np.arange(-10,10,1)
plt.figure(figsize=(12,8))
plt.plot(domain_def,sigmoid(domain_def))

If you look the function values range between 0 to 1

Differentiating Sigmoid function

In this section we will derive the gradient of sigmoid function. And we will notice that the derivative takes a beautiful form.

Sigmoid function has the following functional form.

Differentiating

Note

This implies

Look at the beautiful form the function takes :)