BOW(Bag of words) and N-gram(30 — day NLPchallenge)

day 6

When you type a search query, how does Google understand what words matter most? One of the earliest NLP techniques behind this is Bag of Words and N-grams.

It is an algorithm that turns text like a sentence, paragraph, or document into a collection of words and counts how often each word appears, but it ignores the order of words.

now will learn its variants

1 Unigram

(n=1) Single words treated individually. Standard BoW relies entirely on unigrams.

A Unigram is a single word/token. Example sentence: “I love NLP”

Unigrams:

• I
• love
• NLP

2. Bigram

(n=2) Sequences of two consecutive words, capturing immediate context.

A Bi-gram is a sequence of 2 consecutive words. Example sentence: “I love NLP”

Bigrams:

• I love
• love NLP

3. Trigram

(n=3) Sequences of three consecutive words

An N-gram is a sequence of N consecutive words. Examples using:“I love NLP.”

Diffrence b/w diffrent types of N — gram

Advantages of Bag of Words

• Simple and easy to implement
• Fast for small datasets
• Works well for basic classification tasks
• Easy to interpret

Limitations of N-gram

• High Memory Usage
• Data Sparsity Problem
• Lack of Context Understanding
• Fixed Window Size
• Poor Handling of Unseen Words
• Computationally Expensive for Large N
• No Semantic Understanding
• Word Order Dependency

“Where This Fails” Examples

Sentence 1: “I am happy”

Sentence 2: “I am not happy”

BoW may fail to capture negation properly. This creates critical thinking and a deeper understanding.

Implementation of the N-grams using the NLTK library

import nltk
from nltk import ngrams

# Download the necessary resources
nltk.download('punkt_tab')

# Sample text
text = "Implementation of N-gram is fundamental for natural language processing"

# 1. Tokenize the text (split into words)
tokens = nltk.word_tokenize(text.lower())

# 2. Generate Bi-grams (N=2)
bigrams_list = list(ngrams(tokens, 2))
print("Bi-grams:", bigrams_list)

# 3. Generate Tri-grams (N=3)
trigrams_list = list(ngrams(tokens, 3))
print("Tri-grams:", trigrams_list)