Movie Recommender using Cosine Similarity

A recommendation system is a software that predicts what items a user might like and suggests them accordingly. Example Netflix suggestions show, Amazon showing “customer also bought”.

Content-based filtering: Recommends items based on the item’s own attributes (genre, keywords, tags etc).

Collaborative-based filtering: Recommends items based on patterns accross users. if people similar to you liked something, you’ll probably like it too. example if user1 and user2 are similar then favourite movies of usere1 is suggested to user2.

I’ve made a content based recommendation system using TMDB 5000 Movies & Credits dataset. The dataset is available on Kaggle.

first row of the dataset

Data Preprocessing & Feature Engineering

After Loading the dataset, merge the movies and credits dataset based on common title column. Handle missing and duplicate values using dropna and dropduplicated .

Select the relevant features drop the irrelevant features. Fetch the titles and names of individual in proper format. the names should not contain whitespaces in between because the vectorizer treats “sam” , “warthington” as completely separate independent tokens and not as one individual.

'Sam Warthington' -> 'SamWarthingtion'

Text Vectorization

Stemming (Porter Stemmer) is the process of reducing a word to its root/base form by chopping off prefixes or suffixes. Without stemming the vectorizer will treat “love”, “loved” and “loving” as three completely different words/tokens even though they all mean the same thing.

from nltk.stem.porter import PorterStemmer
ps = PorterStemmer()

ps.stem("loving")  # → 'love'
ps.stem("loved")   # → 'love'
ps.stem("loves")   # → 'love'

Bag-of-Words is one of the simplest ways to turn text into numbers so a computer can compare it. It just counts how many times each word appears. It’s like dumping all the words from a sentence into a “bag” and only caring about what’s in the bag and how many of each, not the order they went in.

CountVectorizer (from scikit-learn) automates exactly this process:

from sklearn.feature_extraction.text import CountVectorizer
cv = CountVectorizer(max_features=5000, stop_words='english')
vectors = cv.fit_transform(new_df['tags']).toarray()

Cosine Similarity

We need a way to measure how similar two movies are. This is where cosine similarity comes in. The most intuitive way people first think about comparing two points is by measuring the distance between them — like measuring how far apart two dots are on a map. But cosine similarity does something different: it measures the angle between two vectors, ignoring their length (magnitude) entirely.

from sklearn.metrics.pairwise import cosine_similarity
similarity = cosine_similarity(vectors)

Github Repo

Live demo