Vectors in Machine Learning: The Universal Language Every Model Speaks

Learn vectors in machine learning — how lists of numbers represent words, users, items, and images to power search, ML, and AI models.

1. Start simple (Zero → Basic)

Imagine you’re trying to describe a hotel to a friend who can only understand numbers. You can’t say “it’s cozy” or “the staff was lovely.” You have to say something like:

“Cleanliness: 9, Location: 8, Price: 6, Wi-Fi: 7, Breakfast: 5”

That little list — (9, 8, 6, 7, 5) — is a vector. Five numbers describing one hotel.

That’s the whole secret. A vector is just an ordered list of numbers that describes something. The “ordered” part matters: the first number always means the same thing (cleanliness), the second always means the same thing (location). If you scramble the order, you lose the meaning.

A scalar is one number (8). A vector is a row of numbers ([9, 8, 6, 7, 5]). That's the only difference. You already know what a vector is — you've used spreadsheets your entire career, and every row of a spreadsheet is secretly a vector.

The fancier definition you saw in the lecture — “an object with magnitude and direction, drawn as an arrow” — is the same thing viewed geometrically. If you take the vector (3, 4) and draw it on a graph as an arrow from the origin to the point (3, 4), that arrow has a length (about 5) and a direction (about 53° from the x-axis). Both views — list and arrow — are valid, and you'll switch between them constantly.

🎯 Active check 1: A user’s favorite-music preferences are stored as (rock: 0.9, jazz: 0.2, classical: 0.7, hiphop: 0.1). Is this a vector? What is its dimension?

(Answer at the end of section 2.)

2. Build intuition: Why does this concept exist?

Here is the single hardest truth in machine learning, and once you internalize it everything else gets easier:

A model cannot understand anything that is not a number.

A neural network does not “see” the word Bangalore. It does not “look at” a hotel photo. It does not “read” a customer review. All it can do is multiply, add, and compare numbers.

So if you want a model to do anything useful — recommend a hotel, translate a sentence, detect a fraudulent booking, answer a question — you first have to translate the real-world thing into a list of numbers. That list is a vector.

This is called representation, and it’s the bridge between the messy human world and the clean math world where models live.

Why does this concept exist?

1. Models do math, not English. Vectors are the only thing they can chew on.
2. Vectors let you measure similarity mathematically. Two similar hotels have similar vectors. Two unrelated words have very different vectors. This unlocks search, recommendations, clustering.
3. Vectors live in geometric space, so you can use centuries of geometry (distances, angles, projections) to reason about ML problems.

🎯 Answer to check 1: Yes, it’s a vector with dimension 4 (four ordered numbers). Each dimension represents one music genre’s preference score.

3. Core idea behind the concept

Here’s the central insight, in one sentence:

A vector turns meaning into geometry. Similar things end up nearby in space; unrelated things end up far apart.

If your encoding is good, then:

• Two similar hotels → their vectors point in nearly the same direction.
• The words “king” and “queen” → their vectors sit close together in their 300-dimensional space.
• A user who loves luxury hotels and a luxury hotel itself → their vectors align.
• A user who loves backpacker hostels and a 5-star resort → their vectors point in opposite directions.

The “thinking process” of an ML model is essentially:

1. Convert the input into a vector.
2. Compare it (via dot products, distances, angles) with other vectors it has learned.
3. Use those comparisons to produce an output.

Inputs become outputs by being projected, rotated, scaled, and added — and each of those operations is something you can do to a vector with simple arithmetic.

🎯 Active check 2: If a model has produced an embedding vector for the hotel “Taj Bangalore” and another for the hotel “ITC Gardenia”, what kind of math operation would you use to ask “how similar are these two hotels?”

(Answer at the end of section 4.)

4. Step-by-step working

Let’s trace one concrete example all the way through: a hotel recommendation.

Step 1 — Collect features. For each hotel you have ratings: cleanliness, location, price (normalized), wi-fi, breakfast. You assemble these into a vector. Example: Taj Bangalore = [0.9, 0.8, 0.4, 0.9, 0.9].

Step 2 — Encode the user the same way. From the user’s past bookings, you infer their preference vector: how much they care about cleanliness, location, low price, etc. Example user = [0.8, 0.9, 0.7, 0.5, 0.6]. Critically, the dimensions must align between the two vectors — position 0 must mean "cleanliness" for both, position 1 must mean "location" for both. Otherwise the comparison is meaningless.

Step 3 — Compute similarity. Dot product of user-vector with hotel-vector: 0.8×0.9 + 0.9×0.8 + 0.7×0.4 + 0.5×0.9 + 0.6×0.9 = 0.72 + 0.72 + 0.28 + 0.45 + 0.54 = 2.71. High score → recommend.

Step 4 — Rank. Compute that dot product for every hotel in your catalog. Sort descending. Show the top 10.

Step 5 — Normalize when comparing across users. A user who rates everything highly will get high dot products with everything. To strip out “loudness” and keep only “alignment,” divide by both vectors’ magnitudes — you’ve just computed cosine similarity. This is exactly the ‖V‖ formula from the lecture, used as a scaling discipline.

The whole pipeline is [encode] → [vector arithmetic] → [decision]. Almost every ML system follows this shape. The encoder and decision logic get more sophisticated, but the vector arithmetic in the middle stays remarkably constant.

🎯 Answer to check 2: Cosine similarity — dot product of the two vectors divided by the product of their magnitudes. It returns a number between -1 and 1 where 1 means identical direction, 0 means unrelated, -1 means opposite.

5. Math intuition

Three formulas from the lecture matter most. I’ll explain each in plain English first.

5.1 Dot product

v⋅w=v1​w1​+v2​w2​+…+vn​wn​

Plain English: “How much do these two vectors agree, dimension by dimension?” You multiply each pair of matching slots, then sum. Big positive number = they agree. Zero = unrelated. Big negative = they disagree.

Why it matters: Almost every “similarity score” in ML is some flavor of dot product.

5.2 Magnitude (length / norm)

∥v∥=sqrt(v12​+v22​+…+vn2​​​)

Plain English: “How long is the arrow?” It’s the Pythagorean theorem extended to n dimensions. The vector [3, 4] has length 5. The vector [1, 1, 1] has length √3.

Why it matters: Used to normalize vectors so you compare directions only. Also used in regularization (penalizing large weight vectors during training).

5.3 Cosine of angle between two vectors

cos(θ)=∥v∥×∥w∥/v⋅w​

Plain English: “Strip out length, give me pure directional similarity.” Always between -1 and 1. This is the workhorse formula of semantic search.

Why it matters: When you query “find me hotels like this one,” your vector database is computing this for thousands of candidate hotels and returning the top scores.

🎯 Active check 3: You compute the dot product of two user-preference vectors and get 0. What does that mean about the two users?

(Answer: their preferences are orthogonal — they care about completely different things. Not opposite, just unrelated.)

6. Code + Explanation

6.1 Beginner version — NumPy basics

python

import numpy as np
# Two hotels, described by [cleanliness, location, price, wifi, breakfast]
taj      = np.array([0.9, 0.8, 0.4, 0.9, 0.9])
itc      = np.array([0.85, 0.7, 0.5, 0.85, 0.95])
hostel   = np.array([0.4, 0.6, 0.9, 0.5, 0.3])
# A user's preferences
user = np.array([0.8, 0.9, 0.7, 0.5, 0.6])
# Dot product = how aligned is each hotel with the user?
print("Taj score:   ", np.dot(user, taj))
print("ITC score:   ", np.dot(user, itc))
print("Hostel score:", np.dot(user, hostel))

Line by line:

• import numpy as np — NumPy is Python's library for fast array math. Every ML codebase imports it. Think of it like the math utilities in java.lang.Math, but for whole arrays at a time.
• np.array([...]) — creates a vector. The numbers inside the brackets are the components.
• np.dot(user, taj) — computes the dot product. Equivalent to writing the loop yourself: sum(user[i] * taj[i] for i in range(len(user))), but much faster because NumPy uses vectorized C code under the hood.

Run it and you’ll see Taj and ITC scoring high (they match the user’s pattern), the hostel scoring low.

6.2 Slightly advanced — cosine similarity for fair comparison

python

import numpy as np
def cosine_similarity(a, b):
    dot   = np.dot(a, b)
    norms = np.linalg.norm(a) * np.linalg.norm(b)
    return dot / norms
user   = np.array([0.8, 0.9, 0.7, 0.5, 0.6])
hotels = {
    "Taj":    np.array([0.9, 0.8, 0.4, 0.9, 0.9]),
    "ITC":    np.array([0.85, 0.7, 0.5, 0.85, 0.95]),
    "Hostel": np.array([0.4, 0.6, 0.9, 0.5, 0.3]),
}
scores = {name: cosine_similarity(user, vec) for name, vec in hotels.items()}
ranked = sorted(scores.items(), key=lambda x: -x[1])
for name, score in ranked:
    print(f"{name:8s} → {score:.3f}")

What changed and why:

• Wrapped the math in a cosine_similarity function — reusable, testable. (As a backend engineer you'll recognize this as the right instinct: extract pure functions.)
• np.linalg.norm(a) computes ‖a‖. The linalg submodule of NumPy is where linear algebra lives — matrix inverses, eigenvalues, all of it.
• Dividing by ‖a‖ × ‖b‖ strips out vector length, so a hotel with overall high ratings doesn't get an unfair score boost.
• Iterating over a dict and ranking — this is the exact shape of code that runs inside production recommenders.

7. Connect to real-world AI systems

A non-exhaustive tour of where vectors show up in products you use daily:

• ChatGPT / Claude / Gemini: Every word you type is converted into a token, then into an embedding vector (usually ~4096 dimensions in modern LLMs). The entire conversation lives as a stack of vectors flowing through transformer layers. The attention mechanism — the magic of these models — is dot products at industrial scale.
• YouTube / Netflix / Spotify recommendations: Users and content are both embedded. The system serves you items whose vectors are closest to yours. Two users with similar viewing history end up close in vector space — “collaborative filtering” is geometry.
• Fraud detection: Each transaction becomes a feature vector (amount, location, time, device, merchant category, etc.). Models learn what fraudulent-vector neighborhoods look like and flag transactions landing in them.
• Self-driving cars: Camera frames become vectors of detected features; LIDAR sweeps become vectors of distances. The driving policy is a function from these vectors to steering/braking decisions.
• Healthcare AI: A patient’s record (symptoms, lab values, history) is encoded as a vector. Similar-patient retrieval and outcome prediction both ride on vector comparisons.
• Voice assistants (Alexa, Siri): Audio waveforms are encoded into vectors; intent classification is a function of those vectors.

8. Training vs Inference

For raw vectors (feature vectors you hand-craft like the hotel example), there’s no training — you just build them and use them.

For learned embedding vectors (the kind that power modern AI), there’s a clear two-phase story.

Training phase (slow, expensive, offline):

1. Initialize every entity (word, user, hotel, image) with a random vector — total noise.
2. Feed the model lots of examples: “users who booked Taj also booked ITC”, “the word queen appears near king in millions of sentences”, “this image is labeled ‘cat’.”
3. After each example, nudge the vectors so that things that should be similar end up with higher dot products, and things that shouldn’t end up with lower ones.
4. Repeat for millions or billions of examples. The vectors gradually self-organize into a meaningful geometric space.

This phase happens once (or periodically). It uses lots of GPUs. The output is a giant lookup table: entity_id → vector.

Inference phase (fast, cheap, online):

1. A request comes in: “recommend hotels for user 12345.”
2. Look up user 12345’s pre-trained vector.
3. Look up (or stream from a vector index) all candidate hotel vectors.
4. Compute dot products / cosine similarities.
5. Return top-K.

Inference is what runs in your production service in milliseconds. Training is what produces the vectors that inference looks up. Same vectors, two very different cost profiles.

🎯 Active check 5: During inference for a recommendation, is the model changing the vectors? Or just reading them?

(Answer: Just reading. The vectors are frozen after training. Online learning systems re-train periodically — say, nightly — to update vectors with new data.)

9. Advantages, limitations, trade-offs

Use vectors when:

• You need to measure similarity between things.
• Your inputs are not natively numerical (text, images, audio, categorical IDs).
• You want to feed data into any kind of neural network.
• You need fast nearest-neighbor search at scale.

Be cautious when:

• The vector dimensions don’t carry meaning. A random encoding gives you nothing.
• You’re comparing vectors from different encoders. A hotel embedding from model A is not comparable to one from model B — they live in different spaces.
• The data is highly structured and small. A SQL query over normalized tables may beat a vector approach for pure exact-match retrieval.

Trade-offs:

• Dimensionality: Higher dimensions capture more nuance but cost more memory and compute. A 4096-dim embedding for 100M documents is 1.6 TB of raw floats.
• Interpretability: Hand-built feature vectors ([cleanliness, location, ...]) are interpretable. Learned 768-dim BERT vectors are not — you have no idea what dimension 412 "means."
• Cold start: A brand-new hotel has no embedding until it gets enough interaction data. Vector-based systems struggle with brand-new entities.
• Drift: As behavior changes over time, old vectors become stale. Production systems need re-training schedules.

10. Common mistakes and misconceptions

1. “Vectors are 2D or 3D arrows.” They can have hundreds or thousands of dimensions. You just can’t draw them. The math doesn’t care.
2. “More dimensions = always better.” Diminishing returns kick in fast, and very high-dim spaces are computationally painful (the “curse of dimensionality”).
3. “Cosine similarity and dot product are the same.” Only true when both vectors are unit length. Otherwise dot product favors long vectors.
4. “I can mix vectors from different models.” A vector from OpenAI’s text-embedding-3 is meaningless in Cohere's embedding space, and vice versa. Treat them as different languages.
5. “Vectors must be the same dimension to be added.” Yes — you cannot add a 3-dim vector and a 5-dim vector. The lecturer was explicit about this; it’s a hard rule.
6. “Orthogonal means unrelated forever.” It means currently uncorrelated in this representation. Change the encoding and orthogonal vectors may become aligned.

11. Summary

• A vector is an ordered list of numbers. That’s it.
• In ML, vectors are how we represent real-world things (words, users, items, images) so models can do math on them.
• The dot product measures alignment between vectors. Almost all similarity scoring in ML reduces to it.
• The magnitude (norm) is the length of the vector. Used for normalization and regularization.
• Cosine similarity = pure directional alignment, length-independent. Workhorse of semantic search.
• Linear combinations (α₁v₁ + α₂v₂ + ...) are the core operation inside every neural network layer.
• Linear independence and orthogonality flag whether your features carry distinct information.
• Production vector systems: encode once, store, then run fast nearest-neighbor search at inference.

One-line intuition you’ll never forget:

A vector is meaning expressed as geometry — and ML is just geometry, performed at scale.

Thanks For Reading!!