Linear Algebra in Machine Learning

In my first blog, https://medium.com/@techkee/understanding-mathematics-in-machine-learning-math-made-simple-562c37f1f025, I had simplified the Mathematics in Machine Learning. Now, let's go a level deeper with all those concepts, starting with Linear Algebra.

Machine Learning models do three main things:

1. Represent data
2. Transform data
3. Extract patterns All these are done using vectors and matrices.

When you create a model and run this command:

model.fit(X, y)

Have you ever wondered what happens behind the scenes? You’re actually triggering thousands (sometimes millions) of matrix operations.

What is a Vector in Machine learning?

At its simplest, a vector is just a list of numbers. Example:

A person in a dataset:

• Height = 170
• Weight = 65
• Age = 25

We represent this as:

x=[170,65,25]

This is called a feature vector. Every data point in your dataset is a vector. If you have 1000 rows in your dataset → You have 1000 vectors.

Copyright © 2026 Keerthi K

1. What Does “Vector as an Arrow” Really Mean?

In mathematics, a vector like:

v=[3,4]

means, Move 3 units in the x-direction and Move 4 units in the y-direction This forms an arrow from the origin to the point (3,4).

Magnitude (Length)

The length of the arrow. For [3,4], the magnitude is:

|v| = √(3² + 4²) = √25 = 5

It tells us how large the vector is.

Direction

The angle at which the arrow points.

It tells us which way the vector is oriented.

2. Dot Product — The Core Operation in ML

Suppose,

x=[1,2,3]
w=[0.5,0.1,0.2]

The dot product is:

x⋅w = (1×0.5) + (2×0.1) + (3×0.2)

This is exactly what happens in:

• Linear Regression
• Logistic Regression
• Neural Networks

When we compute:

z= wᵀx + b

That’s a dot product. Dot product answers this question:

How aligned are these two vectors?

If they point in similar directions → high value If they are unrelated → small value If opposite → negative value

In NLP, cosine similarity (used in embeddings and search engines) is based on dot product.

So yes — even modern AI systems rely on this simple operation.

3. What is a Matrix?

If a vector represents one data point, a matrix represents the entire dataset. Example:

A = [[170, 65, 25],
     [180, 75, 30],
     [160, 55, 22]]

Rows → samples Columns → features

Every dataset in Machine Learning is stored as a matrix.

When you load data using pandas or NumPy, you are working with matrices.

4. Matrix Multiplication

Neural networks follow a simple rule:

Output=XW+b

Where, X → input matrix W → weight matrix b → bias

That’s it.

Deep learning is repeated matrix multiplication followed by activation functions.

Even CNNs and Transformers are optimized matrix machines.

At this point, it’s easy to confuse the Dot Product you read earlier and with the Matix Multiplication— but they are not the same thing.

Dot Product

Happens between two vectors and produces a single number (scalar)

Example: [1,2,3] ⋅ [4,5,6]= 1×4 + 2×5 + 3×6 = 32

|1 2 3| . | 4 |  
          | 5 |  = 1×4 + 2×5 + 3×6 = 32
          | 6 | 

Output is just one number.

Matrix Multiplication

Happens between matrices (or matrix and vector) and produces another matrix (or vector)

Example:

| 1  2 |   | 5 |   | (1×5 + 2×6) |   | 17 |
| 3  4 | x | 6 | = | (3×5 + 4×6)​ | = | 39 |

Output is a vector, not a single number.

5. Eigenvalues & Eigenvectors — Finding Important Directions

Now this sounds complex, but here’s the intuition. Imagine your data points are scattered in 2D space. Some directions show more variation than others.

In mathematical terms:

A v = λ v

Here, A is a matrix representing the transformation, v is the eigenvector, λ is the eigenvalue. “Applying the transformation A to the vector v just stretches it by λ, without changing its direction.”

Eigenvectors tell us:

The direction where variance is maximum means the direction where your data spreads the most

Eigenvalues tell us:

How important that direction is. Large eigenvalue → very important direction Small eigenvalue → less useful direction

Real World Applications in ML: PCA (Principal Component Analysis) Dimensionality Reduction Noise removal

Principal Component Analysis (PCA): Reducing the dimensions of huge datasets by finding the directions (principal components) with the most variance.

Copyright © 2026 Keerthi K

In the above image the original dataset has two features “Radius” and “Area” represented by the black axes. PCA identifies two new directions: PC₁ and PC₂ which are the principal components.

After calculating the Principal Components, top components that capture most of the variance like 95% are selected. Here, PC1 is the top component . So, this means we reduced the number of features (Area and Radius) to PC1 while keeping the important patterns in the data.

And that’s Linear Algebra in a nutshell — not as complicated as it first sounds.

In the next blog, we’ll continue this and look at Calculus in ML, and how models actually learn from data.

A message from our Founder

Hey, Sunil here. I wanted to take a moment to thank you for reading until the end and for being a part of this community. Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? We don’t receive any funding, we do this to support the community.

If you want to show some love, please take a moment to follow me on LinkedIn, TikTok, Instagram. You can also subscribe to our weekly newsletter. And before you go, don’t forget to clap and follow the writer️!