Artificial Neural Networks (ANN) Explained from Scratch: The Easiest Guide for Beginners

Introduction

Imagine waking up in the morning and unlocking your phone just by looking at it. No password. No fingerprint. Just your face.

Later, while having breakfast, you open Netflix, and it instantly recommends a movie that matches your taste perfectly. During the day, Google Maps suggests a faster route before traffic even starts building up. In the evening, you ask ChatGPT a question, and within seconds it gives you an answer that feels surprisingly human.

Have you ever stopped to think about how these technologies work?

Years ago, computers could only perform tasks by following instructions written by programmers. If you wanted a computer to solve a problem, you had to explain every single step. But today’s intelligent systems can recognize faces, understand speech, translate languages, recommend products, detect diseases, and even generate images. These are tasks that are difficult to solve using fixed rules alone.

So what changed?

Instead of teaching computers every possible rule, scientists developed a way for computers to learn from data, just like humans learn from experience. This idea gave birth to Artificial Intelligence, Machine Learning, and eventually Deep Learning.

At the heart of Deep Learning lies one of the most powerful concepts in modern AI: the Artificial Neural Network (ANN).

Artificial Neural Networks are inspired by the way the human brain processes information. They allow computers to identify patterns, make predictions, and improve their performance by learning from examples rather than relying on hardcoded instructions.

Today, ANN powers many of the intelligent systems we use every day — from virtual assistants and recommendation systems to medical diagnosis and autonomous vehicles.

In this article, we will explore Artificial Neural Networks completely from scratch. We’ll understand why they are needed, how they work, what happens inside each neuron, how a neural network learns from its mistakes, and finally build our own ANN using Python and TensorFlow.

Don’t worry if you’re new to Deep Learning. Every concept will be explained using simple language, real-world examples, diagrams, and practical code.

Let’s begin our journey into one of the most exciting technologies in Artificial Intelligence.

Why Do We Need Artificial Neural Networks?

Before understanding Artificial Neural Networks, let’s first understand the problem they were created to solve.

Imagine you are asked to build a computer program that can identify whether an image contains a cat or a dog.

At first, it sounds simple.

You might think of writing rules like these:

• If the animal has pointed ears, then it is a cat.
• If the animal is larger, then it is a dog.
• If it has long whiskers, then it is a cat.

But after seeing hundreds of images, you quickly realize that these rules don’t always work.

Some dogs have pointed ears.

Some cats are larger than small dogs.

Animals appear in different lighting conditions, different backgrounds, different colors, and different positions.

Writing rules for every possible situation becomes nearly impossible.

Now think about how a child learns.

When a child sees hundreds of cats and dogs, nobody teaches them thousands of rules. Instead, the child naturally starts recognizing patterns. After enough examples, they can identify a new cat or dog even if they’ve never seen that exact one before.

Wouldn’t it be amazing if computers could learn the same way?

That is exactly what Artificial Neural Networks do.

Instead of following manually written rules, an ANN learns patterns directly from data. The more examples it sees, the better it becomes at making predictions.

This ability to learn from experience makes Artificial Neural Networks extremely powerful.

Today, they are used for tasks such as:

• Face Recognition
• Speech Recognition
• Handwriting Recognition
• Medical Image Analysis
• Fraud Detection
• Language Translation
• Chatbots
• Recommendation Systems
• Autonomous Vehicles

Traditional programming struggles with these problems because the rules are too complex. Neural Networks solve them by learning those rules automatically.

What is an Artificial Neural Network?

Now that we understand where ANN belongs, let’s define it properly.

An Artificial Neural Network (ANN) is a computational model inspired by the structure and functioning of the human brain.

Just as the human brain contains billions of interconnected neurons that communicate with one another, an ANN contains many artificial neurons connected together in layers.

Each neuron receives information, processes it, and passes the result to the next neuron.

When millions of these tiny computations happen together, the network becomes capable of solving extremely complex problems.

The primary goal of an ANN is to learn patterns from data and use those patterns to make predictions on unseen data.

Unlike traditional programs, an ANN does not need explicit rules.

Instead, it learns those rules automatically during training.

For example, if we provide thousands of handwritten digits, the ANN gradually learns what each number looks like.

After training, it can correctly recognize handwritten numbers it has never seen before.

This ability to generalize is what makes Artificial Neural Networks so powerful.

A Simple Real-Life Analogy

Imagine a group of students solving a difficult mathematics problem.

The first student reads the question and understands some information.

The second student receives that information, processes it further, and adds new ideas.

The third student continues improving the solution.

Finally, the last student arrives at the correct answer.

Each student contributes only a small part of the final solution.

Artificial Neural Networks work in a very similar way.

Each neuron performs a small calculation.

When thousands of neurons work together, they solve problems that would be impossible for a single neuron alone.

Key Characteristics of ANN

Artificial Neural Networks have several important characteristics:

• They learn directly from data.
• They improve with experience.
• They recognize hidden patterns.
• They make predictions on new data.
• They can solve highly complex problems.
• They perform exceptionally well with images, speech, and text.

Human Brain vs Artificial Neural Network

Before understanding how an Artificial Neural Network works, let’s first see where the idea came from.

Artificial Neural Networks are inspired by the human brain. Although they are much simpler than the biological brain, they follow a similar concept of learning from experience.

The human brain consists of nearly 86 billion neurons. These neurons communicate with each other through electrical signals, allowing us to think, recognize faces, remember information, and make decisions.

For example, imagine you see a dog for the first time. Initially, your brain doesn’t know what it is. But after seeing many dogs, your brain starts recognizing common features such as four legs, ears, eyes, fur, and a tail. Eventually, you can identify a dog instantly, even if it is a different breed.

Artificial Neural Networks learn in a similar way.

Instead of biological neurons, they contain artificial neurons. Instead of electrical signals, they use mathematical calculations. Instead of memories, they store learned knowledge in weights and biases.

The more examples an ANN sees during training, the better it becomes at making predictions.

Although an ANN is inspired by the human brain, it is not an exact copy. It is simply a mathematical model designed to imitate some aspects of how humans learn from experience.

Building Blocks of an Artificial Neural Network

Now that we understand the inspiration behind ANN, let’s explore the components that make up a neural network.

Every Artificial Neural Network, whether simple or complex, is built using the following components:

• Input Layer
• Hidden Layer(s)
• Output Layer
• Neurons
• Weights
• Bias
• Activation Function

Each component has a specific role, and together they enable the network to learn from data and make predictions.

Understanding a Neuron

A neuron is the smallest unit of an Artificial Neural Network.

Just as the human brain uses neurons to process information, an ANN uses artificial neurons to perform calculations.

Each neuron receives information from the previous layer, processes it, and sends the result to the next layer.

Think of a neuron as a small decision-maker. Individually, one neuron cannot solve a complex problem. However, when thousands or even millions of neurons work together, they can recognize faces, translate languages, and even generate images.

Each neuron performs three simple steps:

1. Receive input values.
2. Perform a mathematical calculation.
3. Pass the result to the next neuron.

This process happens repeatedly throughout the network until a final prediction is produced.

The Three Layers of an Artificial Neural Network

An Artificial Neural Network is not made up of just one neuron. Instead, it consists of many neurons organized into different layers. These layers work together to process information and produce accurate predictions.

A basic Artificial Neural Network consists of three main layers:

• Input Layer
• Hidden Layer
• Output Layer

Each layer has a specific responsibility, and data flows through them in sequence.

1. Input Layer

The Input Layer is the first layer of an Artificial Neural Network. It receives the raw data that we want the model to learn from.

Every feature (or input variable) in the dataset is represented by one neuron in the input layer.

For example, suppose we want to predict whether a student is addicted to smartphones. Our dataset contains the following features:

• Age
• Daily Screen Time
• Social Media Usage
• Gaming Hours
• Sleep Hours

Since there are five input features, the input layer will contain five neurons.

It is important to understand that the input layer does not perform any calculations. Its only job is to receive the input data and pass it to the next layer.

Think of the input layer as the entrance gate of the neural network. It simply accepts information from the outside world and forwards it for further processing.

Ex:

2. Hidden Layer

The Hidden Layer is the most important part of an Artificial Neural Network because this is where the actual learning takes place.

After receiving data from the input layer, each neuron in the hidden layer performs mathematical calculations. It analyzes the incoming information, identifies useful patterns, and passes the processed information to the next layer.

Unlike the input layer, hidden layers do much more than simply forward data. They help the network understand complex relationships between different features.

For example, in our smartphone addiction prediction model, the hidden layer may discover patterns such as:

• Higher screen time combined with lower sleep hours may indicate a higher risk of addiction.
• Frequent app usage and high notification counts often occur together.
• Gaming hours may have a stronger influence on addiction than age.

These relationships are not programmed manually. The neural network discovers them automatically during training.

A network can have one hidden layer or multiple hidden layers. As the number of hidden layers increases, the network becomes capable of learning more complex patterns. This is why Deep Learning models often contain many hidden layers.

3. Output Layer

The Output Layer is the final layer of the network. After all the processing is complete, this layer produces the final prediction.

The number of neurons in the output layer depends on the type of problem being solved.

For a binary classification problem, such as predicting whether a student is addicted or not addicted, only one output neuron is needed.

Example Output:

• 0 → Not Addicted
• 1 → Addicted

For a multi-class classification problem, such as recognizing handwritten digits from 0 to 9, the output layer would contain 10 neurons, one for each possible digit.

For a regression problem, such as predicting house prices, the output layer usually contains a single neuron that predicts a continuous value.

The output layer is where the neural network delivers its final decision based on everything it has learned during training.

How Data Flows Through an Artificial Neural Network

Now that we understand the three layers, let’s see how data flows through an Artificial Neural Network.

1. The Input Layer receives the raw data.
2. The Hidden Layer processes the information and learns patterns.
3. The Output Layer generates the final prediction.

Although this process appears simple, thousands or even millions of neurons work together behind the scenes to solve complex problems with remarkable accuracy.

Understanding Weights and Bias

So far, we have learned that an Artificial Neural Network consists of neurons organized into different layers. But one important question still remains:

How does a neuron decide what output to produce?

The answer lies in two key components of every neuron:

• Weights
• Bias

These components help the network learn patterns from data and make accurate predictions.

What are Weights?

A weight determines the importance of each input feature. In simple terms, it tells the neural network which inputs should have a greater influence on the final prediction.

Let’s understand this with a simple example.

Imagine you are predicting whether a student is addicted to smartphones based on the following features:

• Daily Screen Time
• Sleep Hours
• Age

Among these features, Daily Screen Time is likely to have a greater impact on smartphone addiction than Age. During training, the neural network learns this relationship automatically by assigning a higher weight to Daily Screen Time and a lower weight to Age.

In other words, features that are more important receive larger weights, while less important features receive smaller weights.

One of the most remarkable aspects of Artificial Neural Networks is that these weights are not assigned manually. Instead, they are learned automatically during the training process.

What is Bias?

While weights determine the importance of each input, they are not enough on their own. Every neuron also includes a bias, which acts as an additional value added before the neuron produces its output.

You can think of bias as a way of adjusting or shifting the neuron’s output. It gives the network extra flexibility, allowing it to learn more complex patterns.

A simple real-life analogy is a weighing scale. Even if there is nothing placed on the scale, it may not start exactly at zero. You adjust it so that it shows the correct reading. Similarly, the bias helps the neuron make more accurate predictions by adjusting its output.

Without bias, the network’s learning capability would be limited, and it would struggle to model many real-world problems effectively.

How Do Weights and Bias Work Together?

Every neuron combines the input values, their corresponding weights, and the bias to calculate an intermediate value.

Mathematically, this is represented as:

Z=(X1×W1)+(X2×W2)+⋯+ (Xn​×Wn​)+b

Where:

• X = Input values
• W = Weights
• b = Bias
• Z = Weighted sum (input to the activation function)

This equation is the foundation of every Artificial Neural Network.

It tells us that each input is first multiplied by its corresponding weight, all the results are added together, and then the bias is added to produce the neuron’s output before the activation function is applied.

Ex:

Let’s understand this with a numerical example.

Suppose a neuron receives two input values:

• Screen Time = 8 hours
• Sleep Hours = 6 hours

The network has learned the following weights:

• Weight for Screen Time = 0.8
• Weight for Sleep Hours = -0.4

The bias is 1.5.

Using the equation:

Z=(8×0.8)+(6×−0.4)+1.5Z = (8 \times 0.8) + (6 \times -0.4) + 1.5Z=(8×0.8)+(6×−0.4)+1.5 Z=6.4−2.4+1.5Z = 6.4–2.4 + 1.5Z=6.4−2.4+1.5 Z=5.5Z = 5.5Z=5.5

This value (Z = 5.5) is then passed to the activation function, which decides the neuron’s final output.

This simple calculation is performed by every neuron in the network, often millions of times during training.

Why Are Weights and Bias Important?

Weights and bias are the learnable parameters of an Artificial Neural Network. During training, the network continuously updates these values based on its prediction errors.

As training progresses:

• The weights become better at identifying important features.
• The bias helps fine-tune the neuron’s output.
• The overall prediction becomes more accurate.

This learning process is what enables an Artificial Neural Network to improve over time.

Understanding Activation Functions

After a neuron calculates the weighted sum (Z), the computation is still incomplete. The neuron must decide whether the information should be passed to the next layer and how much of it should be passed.

This decision is made by an Activation Function.

In simple terms, an activation function acts like a decision-maker for the neuron. It takes the calculated value (Z) as input and transforms it into an output that is sent to the next layer.

Without an activation function, every neuron would simply perform linear calculations. No matter how many hidden layers we add, the network would behave like a simple linear model and would not be able to solve complex real-world problems.

Activation functions introduce non-linearity, allowing Artificial Neural Networks to learn complex patterns such as recognizing faces, understanding speech, translating languages, and making intelligent predictions.

Why Do We Need Activation Functions?

Let’s understand this with a simple example.

Imagine you want to predict whether a student is addicted to smartphones.

If the neural network only performs multiplication and addition, it can learn only simple relationships between the input features and the output.

However, real-world data is rarely simple.

For example:

• Two students may have the same screen time but different sleep patterns.
• A person with high gaming hours may not necessarily be addicted.
• Notifications, social media usage, and sleep hours may interact in complex ways.

These relationships cannot be captured using only linear calculations.

Activation functions solve this problem by introducing non-linearity, enabling the neural network to model these complex relationships accurately.

How Does an Activation Function Work?

The process inside a neuron follows these steps:

1. The neuron receives the input values.
2. Each input is multiplied by its corresponding weight.
3. The weighted values are added together.
4. The bias is added to obtain the weighted sum (Z).
5. The activation function processes Z.
6. The final output is passed to the next layer.

This process is repeated for every neuron in the network until the final prediction is generated.

Common Activation Functions

Over the years, researchers have developed several activation functions. Each has its own strengths and is suitable for different types of problems.

The four most commonly used activation functions are:

• Sigmoid
• Tanh
• ReLU (Rectified Linear Unit)
• Softmax

Let’s understand each one.

1. Sigmoid Activation Function

The Sigmoid function converts any input value into a number between 0 and 1.

Because of this property, it is widely used for binary classification problems, where the output represents a probability.

For example:

• Will the customer buy the product?
• Is the email spam or not?
• Is the patient suffering from a disease?

The Sigmoid function is defined as:

σ(z)=1/1+e−z

As the input value increases, the output approaches 1. As the input decreases, the output approaches 0.

Advantages

• Easy to understand.
• Produces probability values.
• Suitable for binary classification.

Limitations

• Suffers from the Vanishing Gradient Problem.
• Slower training in deep neural networks.

2. Tanh (Hyperbolic Tangent)

The Tanh function is similar to the Sigmoid function but produces outputs between -1 and 1.

Unlike Sigmoid, the output is centered around zero, which often helps the model learn faster.

It is commonly used in certain deep learning architectures, especially older recurrent neural networks.

Advantages

• Output centered around zero.
• Learns faster than Sigmoid in many cases.

Limitations

• Still suffers from the Vanishing Gradient Problem.

3. ReLU (Rectified Linear Unit)

The ReLU activation function is the most widely used activation function in modern deep learning.

It follows a very simple rule:

• If the input is positive, return the same value.
• If the input is negative, return zero.

Mathematically,

ReLU(z)=max⁡(0,z)ReLU(z)=\max(0,z)ReLU(z)=max(0,z)

Because of its simplicity, ReLU is computationally efficient and helps deep neural networks train much faster.

Today, ReLU is the default choice for most hidden layers.

Advantages

• Very fast.
• Simple computation.
• Reduces the Vanishing Gradient Problem.
• Works well in deep neural networks.

Limitations

• Some neurons may stop learning if they always receive negative values (Dead ReLU problem).

4. Softmax Activation Function

The Softmax function is mainly used for multi-class classification problems.

Instead of producing a single probability, it converts the outputs into a probability distribution across multiple classes.

For example, when recognizing handwritten digits (0–9), Softmax assigns a probability to each digit, and the class with the highest probability is selected as the prediction.

Advantages

• Produces valid probability distributions.
• Ideal for multi-class classification.

Limitation

• Mostly used only in the output layer.

Understanding Forward Propagation

Now that we understand how a neuron performs calculations using weights, bias, and activation functions, let’s see how an Artificial Neural Network actually makes a prediction.

The process of sending information from the Input Layer to the Output Layer is called Forward Propagation.

In simple terms, forward propagation is the process in which data moves through the network, allowing each neuron to perform calculations until a final prediction is generated.

Whenever you use a trained neural network to make a prediction, forward propagation is performed.

Whether it is recognizing a face, predicting house prices, translating languages, or classifying images, every prediction begins with forward propagation.

How Does Forward Propagation Work?

Let’s understand the process step by step.

Step 1: Receive the Input Data

The process starts when the neural network receives input data.

For example, suppose we want to predict whether a student is addicted to smartphones.

The input features might be:

• Age = 22
• Daily Screen Time = 8 Hours
• Social Media Usage = 3 Hours
• Sleep Hours = 6 Hours

These values are sent to the neurons in the Input Layer.

Step 2: Multiply Inputs by Weights

Each input is connected to neurons in the next layer.

Every connection has its own weight.

The neuron multiplies every input value by its corresponding weight.

This allows the network to determine how important each feature is.

For example,

• Screen Time × Weight
• Sleep Hours × Weight
• Social Media Usage × Weight

More important features usually receive larger weights.

Step 3: Add the Bias

After multiplying all inputs by their respective weights, the neuron adds the bias.

The weighted sum is calculated as:

Z=(X1W1)+(X2W2)+…+(XnWn)+b

This value (Z) represents the total input received by the neuron before applying the activation function.

Step 4: Apply the Activation Function

The weighted sum (Z) is then passed through an activation function.

The activation function determines whether the neuron should pass the information to the next layer and how much information should be passed.

Depending on the problem, the network may use:

• ReLU
• Sigmoid
• Tanh
• Softmax

The output produced by the activation function becomes the input for the next layer.

Step 5: Repeat the Process

The same process is repeated for every neuron in every hidden layer.

Each neuron performs the following operations:

• Receive inputs
• Multiply by weights
• Add bias
• Apply activation function
• Pass output to the next layer

This continues until the information reaches the Output Layer.

Step 6: Generate the Final Prediction

Once the processed information reaches the output layer, the network generates its final prediction.

For example:

• 0.96 → Addicted
• 0.08 → Not Addicted

For classification problems, the output usually represents a probability.

The class with the highest probability becomes the network’s prediction.

Ex: Suppose a neuron receives two inputs.

Assume the learned weights are:

• Weight₁ = 0.8
• Weight₂ = -0.4

Bias = 1.5

The neuron calculates:

Z=(8×0.8)+(6×−0.4)+1.5Z=(8\times0.8)+(6\times-0.4)+1.5Z=(8×0.8)+(6×−0.4)+1.5 Z=6.4−2.4+1.5Z=6.4–2.4+1.5Z=6.4−2.4+1.5 Z=5.5Z=5.5Z=5.5

This value is passed through an activation function.

If we use the Sigmoid activation function, the output might be:

Output = 0.996

Since the probability is very close to 1, the network predicts that the student is Addicted.

This entire process is called Forward Propagation.

Why is Forward Propagation Important?

Forward propagation is responsible for generating predictions from a neural network.

Without it, the model cannot produce any output.

Every prediction made by an Artificial Neural Network follows the same sequence:

1. Receive input data.
2. Multiply inputs by weights.
3. Add the bias.
4. Apply the activation function.
5. Pass the output to the next layer.
6. Generate the final prediction.

This process happens in milliseconds, even for networks containing millions of neurons.

Understanding the Loss Function

After completing Forward Propagation, the neural network generates a prediction. But making a prediction is only the first step.

An important question still remains:

How does the neural network know whether its prediction is correct or wrong?

This is where the Loss Function comes into the picture.

A Loss Function is a mathematical method used to measure how far the model’s prediction is from the actual value. In simple terms, it tells the neural network how much error it has made.

The larger the error, the higher the loss. The smaller the error, the better the model is performing.

The ultimate goal of training a neural network is to minimize this loss as much as possible.

Why Do We Need a Loss Function?

Imagine you are learning to play darts.

Every time you throw a dart, you compare where it landed with the center of the target.

• If the dart lands very close to the center, you performed well.
• If it lands far away, you know you need to improve your next throw.

A neural network learns in a very similar way.

After making a prediction, it compares the predicted value with the actual value. The difference between them is called the loss.

The network then uses this loss to improve its future predictions.

Ex:

Suppose we are building a neural network to predict whether a student is addicted to smartphones.

The actual answer is:

Actual Value = 1 (Addicted)

After forward propagation, the neural network predicts:

Predicted Value = 0.82

Since the prediction is close to the actual value, the loss is relatively small.

Now consider another prediction:

Predicted Value = 0.15

This prediction is much farther from the correct answer.

As a result, the loss becomes much larger.

The larger the loss, the more the network needs to adjust its parameters.

How Does the Loss Function Work?

The process is simple:

1. The neural network makes a prediction.
2. The prediction is compared with the actual value.
3. The loss function calculates the error.
4. The error is sent back through the network.
5. The network updates its weights and bias to reduce the loss.

This cycle repeats many times until the model becomes highly accurate.

Common Loss Functions

Different types of problems require different loss functions.

The most commonly used loss functions are:

1. Mean Squared Error (MSE)

Mean Squared Error is mainly used for regression problems, where the output is a continuous numerical value.

For example:

• Predicting house prices
• Predicting stock prices
• Predicting temperature

MSE calculates the average squared difference between the predicted values and the actual values.

A smaller MSE indicates better model performance.

2. Binary Cross-Entropy Loss

Binary Cross-Entropy is used for binary classification problems, where there are only two possible classes.

Examples include:

• Spam or Not Spam
• Addicted or Not Addicted
• Disease or No Disease

This loss function measures how close the predicted probability is to the correct class.

It is one of the most commonly used loss functions in binary classification tasks.

3. Categorical Cross-Entropy Loss

Categorical Cross-Entropy is used for multi-class classification problems, where the output can belong to one of several classes.

Examples include:

• Recognizing handwritten digits (0–9)
• Identifying different animal species
• Classifying flower types

This loss function evaluates how well the predicted probabilities match the actual class.

Why Is the Loss Function Important?

The loss function acts like a teacher for the neural network.

It tells the model:

• How good its prediction is.
• How much error it has made.
• Whether it needs to improve.

Without a loss function, the network would have no way of knowing whether its predictions are correct or incorrect.

As training progresses, the network continuously adjusts its weights and bias to reduce the loss, leading to more accurate predictions over time.

Understanding Backpropagation

So far, we have learned that the neural network makes predictions using Forward Propagation and measures its error using a Loss Function.

But one important question still remains:

How does the neural network reduce its mistakes and improve over time?

The answer is Backpropagation.

Backpropagation is the learning process of an Artificial Neural Network. It uses the error calculated by the loss function to update the network’s weights and bias, helping the model make more accurate predictions in the future.

In simple terms, backpropagation teaches the neural network by showing it where it made mistakes and how to correct them.

Why Do We Need Backpropagation?

Imagine you are learning to ride a bicycle.

The first time you ride, you may lose balance and fall. Instead of giving up, you observe your mistakes, adjust your balance, and try again.

With each attempt, you become better until you can ride confidently.

Artificial Neural Networks learn in exactly the same way.

After making a prediction, the network compares it with the actual answer. If the prediction is incorrect, it adjusts its internal parameters so that the next prediction is more accurate.

This continuous process of learning from mistakes is called Backpropagation.

How Does Backpropagation Work?

Backpropagation follows a series of steps.

Step 1: Make a Prediction

The network first performs Forward Propagation and generates a prediction.

For example:

• Actual Value = 1
• Predicted Value = 0.65

Step 2: Calculate the Error

The predicted value is compared with the actual value using the Loss Function.

If the prediction is close to the actual value, the loss will be small.

If the prediction is far from the actual value, the loss will be large.

This error tells the network how much improvement is needed.

Step 3: Send the Error Backward

Instead of moving forward, the error now travels backward through the network.

The error starts from the Output Layer, moves through each Hidden Layer, and finally reaches the Input Layer.

As the error travels backward, the network determines which weights and biases contributed most to the incorrect prediction.

This backward movement of error gives the process its name: Backpropagation.

Step 4: Update the Weights and Bias

Once the network identifies which weights and biases caused the error, it updates them slightly.

The objective is simple:

• Increase weights that improve predictions.
• Decrease weights that increase errors.

These small adjustments help the network produce better predictions in the next iteration.

This update is repeated thousands of times during training.

Step 5: Repeat the Process

The updated weights are used in the next round of Forward Propagation.

The network again:

• Makes a prediction.
• Calculates the loss.
• Performs backpropagation.
• Updates the weights.

This cycle continues until the loss becomes very small or the model reaches the desired accuracy.

Ex:

Imagine a teacher checking a student’s exam.

The student scores 60 out of 100.

Instead of simply giving the marks, the teacher points out every mistake and explains how to improve.

The student studies those mistakes, practices again, and scores 75, then 90, and eventually 98.

Backpropagation works in exactly the same way.

The Loss Function acts like the teacher, identifying mistakes.

Backpropagation acts like the learning process, helping the neural network correct those mistakes.

Over time, the network becomes more accurate.

Why Is Backpropagation Important?

Without backpropagation, a neural network would never improve.

It would continue making the same mistakes repeatedly because its weights and bias would never change.

Backpropagation enables the network to:

• Learn from its mistakes.
• Reduce prediction errors.
• Improve accuracy over time.
• Build intelligent models capable of solving complex problems.

It is one of the most important algorithms in Deep Learning and is responsible for the success of modern Artificial Neural Networks.

Understanding Optimizers

In the previous section, we learned that Backpropagation calculates the error and determines how the weights and bias should change. However, another important question arises:

Who actually updates these weights?

The answer is an Optimizer.

An optimizer is an algorithm that adjusts the weights and bias of a neural network to reduce the loss and improve the model’s performance.

Think of an optimizer as a guide that helps the neural network move in the right direction during learning. Its main goal is to find the set of weights that produces the smallest possible error.

Without an optimizer, the network would know that it made a mistake, but it would not know how to correct it.

Why Do We Need an Optimizer?

Imagine you are climbing a mountain covered in thick fog.

Your goal is to reach the lowest point in the valley, but you cannot see the entire landscape. The only way to move is by taking one step at a time and checking whether you are moving downward.

If every step takes you closer to the valley, you continue in that direction.

An optimizer works in a similar way.

It examines the error calculated by the loss function and decides how the weights should be adjusted to move toward a smaller loss.

By taking many small steps, the optimizer gradually helps the neural network reach the best possible solution.

How Does an Optimizer Work?

The optimizer follows these steps:

1. The neural network makes a prediction using Forward Propagation.
2. The Loss Function measures the prediction error.
3. Backpropagation calculates how much each weight contributed to the error.
4. The Optimizer updates the weights and bias.
5. The process repeats until the loss becomes very small.

This continuous cycle allows the network to improve after every training iteration.

Common Optimizers

Several optimization algorithms have been developed for training neural networks. Some of the most commonly used optimizers are:

• Gradient Descent
• Stochastic Gradient Descent (SGD)
• RMSprop
• Adam

Let’s briefly understand each one.

1. Gradient Descent

Gradient Descent is one of the earliest and most fundamental optimization algorithms.

It updates the weights by calculating the direction in which the loss decreases the fastest. After each iteration, the weights are adjusted slightly to reduce the prediction error.

Although Gradient Descent is simple and easy to understand, it can be slow when working with very large datasets.

2. Stochastic Gradient Descent (SGD)

Stochastic Gradient Descent is an improved version of Gradient Descent.

Instead of processing the entire dataset at once, SGD updates the weights using one training example at a time.

This makes learning faster and more efficient, especially for large datasets.

However, because it updates weights more frequently, the learning process can be a little noisy.

3. RMSprop

RMSprop is designed to improve learning by adjusting the learning rate automatically during training.

Instead of using the same learning rate for every weight, it adapts the step size based on previous updates.

This often leads to faster and more stable convergence.

RMSprop is commonly used in recurrent neural networks and time-series applications.

4. Adam (Adaptive Moment Estimation)

Adam is one of the most popular optimizers in modern Deep Learning.

It combines the advantages of both Gradient Descent and RMSprop, allowing the network to learn efficiently while maintaining stable updates.

Adam is widely used because it:

• Converges quickly.
• Handles large datasets efficiently.
• Requires very little manual tuning.
• Works well for most deep learning problems.

For these reasons, Adam is the default optimizer in many TensorFlow and Keras projects.

Building Your First Artificial Neural Network Using TensorFlow and Keras

So far, we have explored the theoretical concepts behind Artificial Neural Networks, including neurons, layers, weights, bias, activation functions, forward propagation, loss functions, backpropagation, and optimizers.

Now it’s time to put these concepts into practice by building our first Artificial Neural Network using TensorFlow and Keras.

TensorFlow is one of the most popular open-source deep learning frameworks developed by Google. It provides powerful tools for building and training neural networks efficiently.

Keras is a high-level API built on top of TensorFlow that simplifies the process of creating deep learning models. With just a few lines of code, we can design, train, and evaluate an Artificial Neural Network.

In this example, we’ll build a simple ANN for a binary classification problem. The goal is to train the network to classify input data into one of two possible classes.

Step 1: Import the Required Libraries

Before building the neural network, we need to import the necessary libraries.

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

Understanding the Code

• tensorflow provides all the tools required to build and train deep learning models.
• Sequential is used to create a neural network where layers are arranged one after another.
• Dense is used to create fully connected layers, where every neuron is connected to every neuron in the next layer.

Step 2: Create the Neural Network

Now let’s build the architecture of our Artificial Neural Network.

model = Sequential([
    Dense(16, activation='relu', input_shape=(4,)),
    Dense(8, activation='relu'),
    Dense(1, activation='sigmoid')
])

Understanding the Code

Let’s understand each line in detail.

First Layer

Dense(16, activation='relu', input_shape=(4,))

This is the first hidden layer.

• 16 represents the number of neurons.
• ReLU is used as the activation function.
• input_shape=(4,) means the model expects four input features.

Second Layer

Dense(8, activation='relu')

This is the second hidden layer.

It contains 8 neurons, which continue learning complex patterns from the previous layer.

Output Layer

Dense(1, activation='sigmoid')

This is the output layer.

Since this is a binary classification problem, only one neuron is required.

The Sigmoid activation function converts the output into a probability between 0 and 1.

For example:

• 0.92 → Class 1
• 0.08 → Class 0

Step 3: Compile the Model

Once the architecture is ready, we need to compile the model.

model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy']
)

Understanding the Code

Before training begins, TensorFlow needs to know three things:

Optimizer

optimizer='adam'

Adam is responsible for updating the weights and bias after every iteration.

Loss Function

loss='binary_crossentropy'

Since this is a binary classification problem, Binary Cross-Entropy is the appropriate loss function.

Evaluation Metric

metrics=['accuracy']

Accuracy tells us how many predictions were correct during training.

Step 4: Train the Model

Now we train the Artificial Neural Network using the training dataset.

history = model.fit(
    X_train,
    y_train,
    epochs=20,
    batch_size=32,
    validation_split=0.2
)

Understanding the Code

• X_train contains the training features.
• y_train contains the training labels.
• epochs=20 means the dataset is passed through the network 20 times.
• batch_size=32 means the model processes 32 samples before updating its weights.
• validation_split=0.2 reserves 20% of the training data for validation.

During training, the network repeatedly performs:

• Forward Propagation
• Loss Calculation
• Backpropagation
• Weight Update

until the model learns the underlying patterns in the data.

Step 5: Evaluate the Model

After training, we evaluate how well the model performs on unseen data.

loss, accuracy = model.evaluate(X_test, y_test)

print("Loss:", loss)
print("Accuracy:", accuracy)

Understanding the Code

The evaluate() function calculates the model’s performance using the test dataset.

It returns:

• Loss
• Accuracy

A lower loss and higher accuracy generally indicate better model performance.

Step 6: Make Predictions

Finally, we use the trained model to make predictions on new data.

predictions = model.predict(X_test)
print(predictions[:5])

Understanding the Code

The predict() function generates predictions for the input data.

For binary classification, the output is a probability between 0 and 1.

For example:

0.95
0.12
0.87
0.03
0.76

If the probability is greater than 0.5, the model predicts Class 1.

If it is less than 0.5, the model predicts Class 0.

ANN Workflow

Now let’s summarize the complete workflow of building and training an Artificial Neural Network.

1. Import the required libraries.
2. Build the ANN architecture.
3. Compile the model.
4. Train the model using the training data.
5. Evaluate its performance.
6. Make predictions on new data.

Advantages of Artificial Neural Networks

Artificial Neural Networks offer several advantages that make them one of the most popular deep learning models.

• Learns patterns directly from data.
• Handles complex and non-linear problems.
• Provides high prediction accuracy with sufficient training data.
• Automatically learns important features from raw data.
• Widely used in image recognition, speech processing, and natural language processing.

Limitations of Artificial Neural Networks

Despite their strengths, ANNs also have some limitations.

• Requires large amounts of training data.
• Training can be computationally expensive.
• Models may take longer to train.
• Often difficult to interpret (Black Box Problem).
• Can overfit if not properly trained.

Conclusion

Artificial Neural Networks are one of the core technologies behind modern Artificial Intelligence. Inspired by the human brain, they enable machines to learn from data, recognize patterns, and make intelligent predictions.

In this article, we explored the complete ANN workflow — from understanding neurons and layers to activation functions, forward propagation, loss functions, backpropagation, optimizers, and building an ANN using TensorFlow and Keras.

Learning ANN is the first step toward mastering advanced deep learning models such as CNNs, RNNs, LSTMs, and Transformers. Keep practicing with real-world datasets, and you’ll build a strong foundation in Deep Learning.

Happy Learning!