My Experience in Applied NLP & Language Intelligence — From Text Analytics to Generative AI

Exploring Natural Language Processing, Machine Learning, and Generative AI

As part of my learning journey in Artificial Intelligence and Data Science, I had the opportunity to work on Applied NLP (Natural Language Processing) & Language Intelligence. This experience helped me understand how machines process, analyze, and interpret human language using AI and machine learning techniques.

Through this training, I gained both theoretical knowledge and practical implementation experience in building NLP-based applications.

Understanding the Fundamentals of NLP

One of the first things I learned was the complete NLP workflow, starting from text preprocessing techniques used to clean raw textual data before applying machine learning models.

Text Preprocessing Techniques I Learned

The preprocessing stage included:

• Lowercasing text
• Removing punctuation
• Stopword removal
• Tokenization
• Stemming
• Lemmatization

These techniques helped me understand how unstructured text data is converted into a clean and machine-readable format.

Learning Text Representation Techniques

After preprocessing, I explored different feature extraction and text representation methods used in Natural Language Processing.

Techniques I Worked With

✔ Bag of Words (BoW) ✔ TF-IDF (Term Frequency–Inverse Document Frequency) ✔ N-grams ✔ Word Embeddings

These methods taught me how textual information can be transformed into numerical vectors for machine learning applications.

I also learned advanced embedding models such as:

• Word2Vec
• GloVe

These techniques improved my understanding of semantic relationships between words and contextual meaning in language processing.

Exploring Deep Learning in NLP

One of the most interesting parts of this learning experience was understanding how deep learning models process sequential text data.

Sequence Models I Explored

✔ LSTM (Long Short-Term Memory) ✔ GRU (Gated Recurrent Unit)

These models helped me understand how AI systems capture contextual information and dependencies in language.

Introduction to Transformers & Generative AI

I was also introduced to advanced NLP architectures such as:

✔ Transformers ✔ BERT (Bidirectional Encoder Representations from Transformers)

Learning these modern architectures gave me insight into how current AI systems and generative AI tools understand and generate human language.

This helped me understand the foundation behind:

• Chatbots
• AI assistants
• Text generation systems
• Intelligent language models

Final NLP Project — Spam Detection System

As part of the final NLP project, I implemented a Spam Detection System using TF-IDF and Naive Bayes Classification.

Project Workflow

The project included:

• Data preprocessing
• Text cleaning
• Feature extraction using TF-IDF
• Model training using Naive Bayes
• Model evaluation
• Prediction testing

I also evaluated the model using:

• Accuracy score
• Confusion Matrix

This practical implementation improved my: ✔ Python programming skills ✔ Machine learning understanding ✔ Confidence in building NLP-based applications

============================================================

SMS SPAM DETECTION PROJECT

============================================================

============================================================

IMPORT LIBRARIES

============================================================

import pandas as pd import numpy as np import re import seaborn as sns import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split from sklearn.feature_extraction.text import TfidfVectorizer from sklearn.naive_bayes import MultinomialNB

from sklearn.metrics import accuracy_score from sklearn.metrics import confusion_matrix from sklearn.metrics import classification_report

============================================================

LOAD DATASET

============================================================

dataset_path = r”D:\mini project\dataset\spam.csv”

df = pd.read_csv(dataset_path, encoding=’latin-1')

============================================================

SHOW FIRST 5 ROWS

============================================================

print(“\nFIRST 5 ROWS”) print(df.head())

============================================================

KEEP ONLY REQUIRED COLUMNS

============================================================

df = df[[‘v1’, ‘v2’]]

Rename columns

df.columns = [‘label’, ‘message’]

============================================================

LABEL ENCODING

ham = 0

spam = 1

============================================================

df[‘label’] = df[‘label’].map({ ‘ham’: 0, ‘spam’: 1 })

print(“\nDATASET INFO”) print(df.head())

============================================================

TEXT CLEANING FUNCTION

============================================================

def clean_text(text):

Convert to lowercase

text = text.lower()

Remove punctuation

text = re.sub(r’[^\w\s]’, ‘’, text)

Remove numbers

text = re.sub(r’\d+’, ‘’, text)

return text

Apply cleaning

df[‘message’] = df[‘message’].apply(clean_text)

============================================================

INPUT FEATURES AND LABELS

============================================================

X_text = df[‘message’]

y = df[‘label’]

============================================================

TF-IDF FEATURE EXTRACTION

============================================================

tfidf = TfidfVectorizer()

X = tfidf.fit_transform(X_text)

print(“\nTF-IDF SHAPE”) print(X.shape)

============================================================

TRAIN TEST SPLIT

============================================================

X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42 )

print(“\nTRAINING DATA SHAPE :”, X_train.shape) print(“TESTING DATA SHAPE :”, X_test.shape)

============================================================

MODEL TRAINING

============================================================

model = MultinomialNB()

model.fit(X_train, y_train)

print(“\nMODEL TRAINING COMPLETED”)

============================================================

PREDICTION

============================================================

y_pred = model.predict(X_test)

print(“\nPREDICTION COMPLETED”)

============================================================

ACCURACY

============================================================

acc = accuracy_score(y_test, y_pred)

print(“\nMODEL ACCURACY :”, acc)

============================================================

CONFUSION MATRIX

============================================================

cm = confusion_matrix(y_test, y_pred)

print(“\nCONFUSION MATRIX”) print(cm)

============================================================

CONFUSION MATRIX VISUALIZATION

============================================================

plt.figure(figsize=(6,5))

sns.heatmap( cm, annot=True, fmt=’d’, cmap=’Blues’ )

plt.title( “CONFUSION MATRIX”, fontsize=14, fontweight=’bold’ )

plt.xlabel( “PREDICTED”, fontsize=12, fontweight=’bold’ )

plt.ylabel( “ACTUAL”, fontsize=12, fontweight=’bold’ )

plt.show()

============================================================

CLASSIFICATION REPORT

============================================================

report = classification_report( y_test, y_pred )

print(“\nCLASSIFICATION REPORT”) print(report)

============================================================

CUSTOM MESSAGE PREDICTION

============================================================

sample = [“Congratulations! You won free cash”]

Clean sample text

sample = [clean_text(text) for text in sample]

Convert to TF-IDF

sample_vector = tfidf.transform(sample)

Predict

prediction = model.predict(sample_vector)

print(“\nCUSTOM MESSAGE PREDICTION”)

if prediction[0] == 1: print(“SPAM MESSAGE”) else: print(“HAM MESSAGE”)

Real-World Applications I Learned

Through this course, I understood how NLP is used in real-world applications such as:

• Sentiment Analysis
• Spam Detection
• Chatbots
• Customer Feedback Analysis
• Social Media Monitoring
• Intelligent AI Systems

💡 My Biggest Learning

NLP is not just about processing text — it is about enabling machines to understand human communication intelligently.

This experience significantly enhanced my knowledge in:

• Machine Learning
• Deep Learning
• Language Intelligence Systems
• Generative AI Concepts

Conclusion

Overall, working on Applied NLP & Language Intelligence — From Text Analytics to Generative AI was an extremely valuable learning experience for me.

It gave me hands-on exposure to: ✔ NLP preprocessing ✔ Text analytics ✔ Machine learning models ✔ Deep learning architectures ✔ Generative AI foundations

This journey strengthened my interest in Artificial Intelligence and motivated me to continue exploring advanced AI technologies and real-world intelligent systems.