The Perceptron: The Building Block of Neural Networks

Every neural network, no matter how complex, is ultimately built upon the same fundamental idea introduced by the perceptron. In this article, we’ll explore how a perceptron transforms inputs into decisions using weights, a bias, and an activation function. Along the way, we’ll build one from scratch using a simple example anyone can follow. This friendly link is for those without a subscription.

1. Introduction

My main interest is Large Language Models (LLMs). Like many developers, I’ve spent a lot of time using them, building applications around them, and reading about how they work. Eventually, I realized that if I wanted to truly understand modern AI, I couldn’t start with billions of parameters and complex Transformer architectures — I needed to start with the fundamentals.

That’s why this series begins with the perceptron.

Modern LLMs don’t use perceptrons directly. They rely on much more sophisticated neural network architectures and training algorithms. However, many of the core ideas behind today’s models — weighted inputs, bias, learning from data, and adjusting parameters based on mistakes — can be traced back to the perceptron. Understanding this simple model provides a solid foundation for understanding the more advanced systems that followed.

This series is the result of my own learning journey.

I’m exploring these concepts myself, implementing them from scratch in Java, and sharing what I discover along the way. My goal isn’t to present myself as an AI researcher or machine learning expert, but to build a solid understanding of the ideas behind modern AI and hopefully help others who are on the same path.

We’ll take things one step at a time, starting with one of the simplest learning algorithms ever created. If you notice something that could be explained better or corrected, I’d genuinely appreciate your feedback.

2. A Simple Example: To Eat or Not to Eat?

Before we dive into the theory, let’s work through a simple, everyday example that we’ll use throughout this post.

One of the easiest ways to understand a new concept is to see it solve a familiar problem. Rather than introducing formulas and definitions first, we’ll start with a decision that almost everyone has made.

Imagine you’re deciding whether to eat something right now. To make that decision, you consider two factors:

• How hungry am I? A value between 0 (not hungry at all) and 1 (extremely hungry).
• How much food is available? A value between 0 (nothing to eat) and 1 (plenty of food available).

Based on these two pieces of information, you make a simple decision:

• Eat
• Don’t eat

This may seem like a trivial example, but it captures the essence of what a perceptron does. It receives a set of inputs, evaluates their relative importance, and produces a single decision.

As we introduce each concept — inputs, weights, bias, and the activation function — we’ll return to this example to see how each piece contributes to the final decision. By the end of this article, you’ll understand what a perceptron is and how it transforms numerical inputs into a decision.

3. The Building Blocks: Inputs and Weights

Before we define what a perceptron is, let’s take a closer look at the components it uses to make decisions.

In our example, we have two pieces of information:

• How hungry am I?
• How much food is available?

These are called inputs. They represent the information the perceptron receives from the outside world.

However, not every input should influence the decision equally.

Imagine you’re extremely hungry, but there’s very little food available. Even if there’s only a small snack left, you’ll probably eat it because your hunger is so strong.

Now consider the opposite situation. Imagine your refrigerator is full of food, but you’ve just finished a large meal and aren’t hungry at all. Despite having plenty of food available, you’ll probably decide not to eat.

This suggests that, in our example, hunger has a greater influence on the decision than food availability.

To represent this idea, each input is assigned a weight. A weight determines how strongly an input influences the final decision. Larger weights give an input more influence, while smaller weights reduce its impact. Weights can even be negative, meaning an input pushes the decision in the opposite direction.

For our example, let’s assign the following weights:

• Hunger weight = 0.8
• Food availability weight = 0.6

These values tell the perceptron that hunger should have a stronger influence on the decision than food availability.

It’s important to note that these numbers are just examples. In a real machine learning model, the perceptron doesn’t know the correct weights in advance — it learns them from data during training. We’ll see how that happens later in this article.

Inputs provide the information. Weights determine how much each piece of information matters.

In the next section, we’ll see how a perceptron combines these inputs and weights into a single score it uses to make a decision.

💡 Why are the weights 0.8 and 0.6?

You might be wondering why we chose weights of 0.8 and 0.6. Why not 8 and 6, or even 80 and 60?

The answer is that the exact numbers aren’t important. What matters is their relative magnitude.

For example:

Hunger = 0.8, Food = 0.6

Hunger = 8, Food = 6

Hunger = 80, Food = 60

All three express exactly the same idea: hunger has more influence than food availability.

Multiplying both weights by the same constant simply scales the weighted sum. It doesn’t change which input has the greater influence.

For this article, we’ll use 0.8 and 0.6 because they’re small, easy to read, and keep the calculations simple.

In practice, a perceptron learns its own weights during training. Those weights might be 1.37, -0.42, 5.18, or any other real numbers. They are not chosen manually — they are learned from data.

4. Combining the Inputs: The Weighted Sum

Now that we know what inputs and weights are, the next question is:

How does a perceptron combine them into a single value?

The answer is surprisingly simple. Each input is multiplied by its corresponding weight, and the results are added together. This calculation produces a single value known as the weighted sum.

Using the weights we defined in the previous section:

• Hunger weight = 0.8
• Food availability weight = 0.6

Let’s consider the following situation:

• Hunger = 0.9
• Food available = 0.5

The weighted sum is calculated as:

Weighted Sum = (0.9 × 0.8) + (0.5 × 0.6)

Breaking it down:

• Hunger contributes 0.72 (0.9 × 0.8)
• Food availability contributes 0.30 (0.5 × 0.6)

Adding these contributions together gives:

Weighted Sum = 0.72 + 0.30 = 1.02

At this stage, 1.02 is just a score. It doesn’t mean “Eat” yet. The perceptron still needs to decide what that score represents.

Notice that, even though the two inputs have relatively similar values, hunger contributes more to the final score because it has the larger weight. This is exactly what we intended when we chose the weights: hunger should have a stronger influence on the decision than food availability.

At this point, however, the perceptron has not decided whether to eat. It has simply combined the available information into a single numerical score.

The next question is:

How does the perceptron turn this score into an actual decision?

Before we answer that, we need to introduce one more important ingredient: the bias.

5. Understanding the Bias

So far, we’ve learned that a perceptron combines its inputs using their corresponding weights to produce a single value called the weighted sum.

But what if we want to make it easier — or harder — for the perceptron to produce a positive prediction without changing the inputs or their weights?

This is where the bias comes in.

The bias is an additional value added to the weighted sum before the final decision is made. It allows us to shift the decision boundary, making the perceptron more or less likely to produce a positive output.

A useful analogy is a school exam.

Imagine a student takes a test and scores 68%.

If the passing grade is 70%, the student fails.

If the school lowers the passing grade to 60%, the very same student passes.

Notice that the student’s answers didn’t change — only the rule used to make the decision changed.

The bias plays a similar role in a perceptron. It doesn’t change the inputs or their weights. Instead, it shifts the final score, making a positive prediction easier or harder to produce.

Let’s return to our “Should I Eat?” example.

Suppose the weighted sum is 1.02.

If we add a bias of -0.4, the final score becomes:

1.02 + (-0.4) = 0.62

The inputs haven’t changed, but the score is now much lower, making the perceptron less likely to decide “Eat.”

On the other hand, if we use a bias of +0.4, the final score becomes:

1.02 + 0.4 = 1.42

Again, the inputs are exactly the same, but now the perceptron is more likely to produce a positive prediction.

You can think of the bias as a calibration knob. The weights determine how much each input influences the decision, while the bias determines how easy or difficult it is to produce a positive outcome.

In the next section, we’ll see how the perceptron uses this final score to make a simple binary decision using an activation function.

6. The Activation Function: Making the Final Decision

At this point, our perceptron has received the inputs, applied their corresponding weights, and adjusted the result using the bias. The outcome of these calculations is a single value called the weighted sum.

But our original goal wasn’t to calculate a number — it was to answer a simple question:

Should I eat?

The perceptron needs a way to convert this numerical score into a final decision. This is the purpose of the activation function.

An activation function is a mathematical function that takes the weighted sum as input and produces the perceptron’s output.

The original perceptron uses the simplest possible activation function: the step function.

The step function checks whether the final score is positive or negative. If the score is greater than or equal to zero, it returns 1. Otherwise, it returns 0.

In our example:

• 1 → Eat
• 0 → Don’t eat

Mathematically, the step function is defined as:

           { 1, if z ≥ 0
f(z) =     {
           { 0, if z < 0

where z is the weighted sum after adding the bias.

Example 1: Eat

Suppose we have the following inputs:

• Hunger = 0.9
• Food available = 0.8

Recall the weights and bias from the previous sections:

• Hunger weight = 0.8
• Food availability weight = 0.6
• Bias = -0.5

First, we calculate the weighted sum:

z = (0.9 × 0.8) + (0.8 × 0.6) - 0.5
  = 0.72 + 0.48 - 0.5
  = 0.70

Now we apply the step function:

f(z) = 1

Since 0.70 ≥ 0, the perceptron predicts:

Eat

Example 2: Don’t Eat

Now consider a different situation:

• Hunger = 0.2
• Food available = 0.3

Using the same weights and bias:

• Hunger weight = 0.8
• Food availability weight = 0.6
• Bias = -0.5

The weighted sum is:

z = (0.2 × 0.8) + (0.3 × 0.6) - 0.5
  = 0.16 + 0.18 - 0.5
  = -0.16

Applying the step function:

f(z) = 0

Since -0.16 < 0, the perceptron predicts:

Don’t eat

Notice that the weights and the bias remain the same in both examples. They define the perceptron itself. The only values that change are the inputs, representing different situations. This is exactly how a trained perceptron makes predictions: it applies the same learned parameters to every new input.

Again, the decision matches our intuition. You’re not very hungry, there isn’t much food available, and the combined evidence isn’t strong enough to justify eating.

Notice that the activation function doesn’t determine how hungry you are or how much food is available. Those factors have already been taken into account when calculating the weighted sum. Its only job is to transform the final score into a binary decision.

The step function was an excellent choice for the original perceptron because it naturally fits binary classification problems, where there are only two possible outcomes — for example, yes or no, true or false, spam or not spam, or eat or don’t eat.

Modern neural networks use more sophisticated activation functions, such as sigmoid, tanh, and ReLU, because they allow networks with many layers to learn much more complex patterns. However, for understanding how a perceptron works, the step function is both the simplest and the most intuitive choice.

Although the perceptron internally returns 0 or 1, you can interpret these values however you like. In our example, 1 means Eat, while 0 means Don’t eat. In other problems, they might represent Spam and Not Spam, Approved and Rejected, or any other pair of outcomes.

6. Putting It All Together: The Perceptron

We’ve now explored every component individually:

• Inputs represent the information available to make a decision.
• Weights determine how strongly each input influences the outcome.
• Bias adjusts the weighted sum, making a positive prediction easier or harder to produce.
• The activation function converts the final score into a binary decision.

When these components work together, they form a perceptron. A perceptron can be viewed as a simple pipeline: receive information, combine it, and make a decision.

A perceptron is the simplest type of artificial neuron. Every modern neural network — from image classifiers to large language models — is ultimately built by connecting many artificial neurons together. Understanding one perceptron is the first step toward understanding neural networks as a whole.

The complete process can be summarized in three simple steps:

Step 1: Compute the weighted sum

The perceptron multiplies each input by its corresponding weight, adds the results together, and includes the bias.

z = (w₁ × x₁) + (w₂ × x₂) + ... + b

For our example:

z = (0.9 × 0.8) + (0.8 × 0.6) - 0.5 = 0.70

Step 2: Apply the activation function

The weighted sum is then passed to the step function:

           { 1, if z ≥ 0
f(z) =     {
           { 0, if z < 0

Since z = 0.70, the activation function returns:

f(z) = 1

Step 3: Produce the prediction

Finally, we map the output to our original problem:

Hunger (0.9) ──► ×0.8 ─┐
                       │
Food (0.8) ─► ×0.6 ────┤
                       ▼
                   Weighted Sum
                       +
                     Bias
                       ▼
                Activation Function
                       ▼
                    Eat (1)

Key Takeaways

• A perceptron receives one or more inputs.
• Each input has a weight that determines its influence.
• The weighted sum combines the inputs.
• The bias adjusts the final score.
• The activation function converts the score into a prediction.
• Together, these components form the simplest artificial neuron.

Conclusion

Although our example is simple, this is exactly how a perceptron works. It transforms a set of numerical inputs into a prediction by following the same sequence of operations every time.

At this point, we’ve built a complete perceptron from scratch. We started with a simple everyday problem, introduced inputs, weights, the weighted sum, the bias, and finally the activation function. Together, these components allow a perceptron to make a binary decision.

Although we’ve built only a single artificial neuron, you’ve already learned the core ideas behind every neural network: combining inputs, weighting evidence, applying a bias, and making a prediction. Everything that comes next builds on these same principles.

In the next article, we’ll bring this perceptron to life by implementing it in code. We’ll translate the concepts we’ve learned into a simple program that takes inputs, computes the weighted sum, applies the activation function, and makes predictions just like the perceptron we’ve explored throughout this post.

After that, we’ll take the next step: teaching the perceptron how to learn. Instead of choosing the weights and bias ourselves, we’ll train the model using data and watch it improve its predictions over time.

This article is the first step in a series exploring the fundamentals of neural networks. Depending on where the journey takes us, we’ll continue building on these concepts to uncover how simple perceptrons evolved into the powerful deep learning models that power many AI applications today.

Thank you for reading! If you enjoyed this post, please like and follow. If you have any questions or suggestions, feel free to leave a comment.