Sentiment Analysis Using LSTM Model

Introduction

In this blog, we will build a sentiment analysis project using deep learning. Sentiment analysis means finding out if a text shows a positive or negative feeling. For example, a movie review can be happy or sad.

We will first look at the data, clean it, and prepare it for the model. Then we will use an LSTM (a type of Recurrent neural network) to train the model. LSTM helps the model understand the meaning of long sentences.

By the end of this blog, you will learn how to clean text data and build a deep learning model to find the sentiment of movie reviews.

Overview of the Dataset

For this project, we are using the IMDb Movie Reviews Dataset, which contains 50,000 movie reviews, equally split into training and testing sets. Each review is labelled with a sentiment: either positive or negative.

The dataset’s primary feature is the review text, which will be processed and tokenized to train the model. Our goal is to use an LSTM model to classify movie reviews based on their sentiment.

Importing Libraries

We have imported all the essential libraries that will be used throughout this project for data pre-processing, model building, training, evaluation, and visualization.

import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import re
import nltk
nltk.download('stopwords')
nltk.download('punkt')
nltk.download('punkt_tab')
from nltk.corpus import stopwords
from bs4 import BeautifulSoup          # BeautifulSoup is a useful library for extracting data from HTML and XML documents
from numpy import array
from tensorflow.keras.preprocessing.sequence import pad_sequences
from keras.layers import Activation, Dropout
from sklearn.model_selection import train_test_split
from tensorflow.keras.preprocessing.text import one_hot, Tokenizer
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Flatten, GlobalMaxPooling1D, Dense, Embedding, LSTM, GRU
import pandas.testing as tm

Loading the Dataset from Google Drive

We imported the dataset from Google Drive and used the head() function to peek into the first 5 rows of the dataset.

from google.colab import drive
drive.mount('/content/drive', force_remount=True)
import pandas as pd
file_path = '/content/drive/MyDrive/DSCproj/IMDB Dataset.csv'
movie_reviews = pd.read_csv(file_path)

# 5. Display the first 5 rows of the dataset
print(movie_reviews.head())

Starting with Exploratory Data Analysis (EDA)

We used EDA to check the shape of the data, identify null values, and examine data imbalance using the Seaborn library.

Key Findings

The dataset contains 50,000 rows and 2 columns, with no null values present. The analysis also reveals no data imbalance, as the plot shows an equal distribution of positive and negative reviews

Data imbalance check

Data Cleaning

Data cleaning is an essential step to ensure the quality and consistency of the text data. In this section, we focus on:

1. Removing HTML Tags: Using BeautifulSoup, we strip any HTML elements from the text.
2. Removing Punctuation: We eliminate unnecessary punctuation marks and extra spaces.
3. Removing Stopwords: Common words (like “the”, “and”, “is”) are removed, except for words like “not” and its contractions, which are important for sentiment analysis.

# removing the html strips
def strip_html(text):
    soup = BeautifulSoup(text, "html.parser")
    return soup.get_text()

# removing punctuations
def remove_punctuations(text):
    pattern = r'[^a-zA-Z0-9\s]'
    text = re.sub(pattern,'',text)
    text = re.sub(r"\s+[a-zA-Z]\s+", ' ', text)

    text = re.sub(r'\s+', ' ', text)
    return text

movie_reviews['review'] = movie_reviews['review'].apply(remove_punctuations)
updated_stopword_list = [
for word in stopword_list:
    if word=='not' or word.endswith("n't"):
        pass
    else:
        updated_stopword_list.append(word)
print(updated_stopword_list)

# removing the stopwords
def remove_stopwords(text, is_lower_case=False):
    # splitting strings into tokens (list of words)
    tokens = nltk.tokenize.word_tokenize(text)
    tokens = [token.strip() for token in tokens]
    if is_lower_case:
        filtered_tokens = [token for token in tokens if token not in updated_stopword_list]
    else:
        filtered_tokens = [token for token in tokens if token.lower() not in updated_stopword_list]
    filtered_text = ' '.join(filtered_tokens)
    return filtered_text

movie_reviews['review'] = movie_reviews['review'].apply(remove_stopwords)

Splitting the Data and Tokenizing Text

In this step, we prepare our data for the LSTM model by splitting it into training and testing sets, converting text into sequences, and applying padding.

the no of unique words found in data are 158927

# Convert sentiment labels to integers
movie_reviews['sentiment'] = movie_reviews['sentiment'].apply(lambda x: 1 if x=="positive" else 0)
X_train, X_test, y_train, y_test = train_test_split(movie_reviews['review'].values, movie_reviews['sentiment'].values,
                                                    test_size=0.20,
                                                    random_state=42)
len(X_train), len(X_test), len(y_train), len(y_test)

tokenizer = Tokenizer(num_words=5000)
tokenizer.fit_on_texts(X_train)
X_train_tok = tokenizer.texts_to_sequences(X_train)
X_test_tok = tokenizer.texts_to_sequences(X_test)

vocab_size = len(tokenizer.word_index) + 1
maxlen = 100
X_train_pad = pad_sequences(X_train_tok, padding='post', maxlen=maxlen, truncating='post')
X_test_pad = pad_sequences(X_test_tok, padding='post', maxlen=maxlen, truncating='post')
print ('number of unique words in the corpus:', vocab_size)

Embedding and Model Builduing

This model turns each review into a list of word vectors using the embedding layer, passes them through an LSTM layer to understand word order and context, and then predicts if the review is positive or negative using a dense output layer.

EMBEDDING_DIM = 32

print('Build model...')

model = Sequential()
model.add(Embedding(input_dim = vocab_size, output_dim = EMBEDDING_DIM, input_length=maxlen))
model.add(LSTM(units=40,  dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(1, activation='sigmoid'))

# Try using different optimizers and different optimizer configs
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train_pad, y_train, batch_size=128, epochs=5, validation_split=0.2)
print('Summary of the built model...')
print(model.summary())

Testing

After testing the model on the unseen data, we got the following results:

• Test Accuracy: 83.59% — This means the model correctly predicted the sentiment (positive/negative) of 83.59% of the test reviews.
• Test Loss: 0.6857 — The loss value represents how far the model’s predictions are from the actual labels. A lower loss indicates better performance.

Overall, the model performs well with an accuracy of over 80%, which is promising for a sentiment analysis task.

print('Testing...')
y_test = np.array(y_test)
score, acc = model.evaluate(X_test_pad, y_test, batch_size=128)

print('Test score:', score)
print('Test accuracy:', acc)

print("Accuracy: {0:.2%}".format(acc))