Translating Urdu Poetry to Roman Urdu with a BiLSTM Seq2Seq Model

Language is music — and poetry makes it even more magical. Urdu Ghazals, with their rich metaphors and flowing script, capture emotions like no other. But what happens when we try to teach a machine to transliterate Urdu text into Roman Urdu? This was the challenge I took on for my latest project: building a sequence-to-sequence (seq2seq) model with a bidirectional LSTM (BiLSTM) encoder-decoder to convert Urdu poetry into its Roman Urdu form.

Project Goal

The objective was simple but ambitious: Input: A line of Urdu text Output: Its Roman Urdu transliteration

Instead of using everyday text, I chose the urdu_ghazals_rekhta dataset — a collection of classical and contemporary Ghazals. This added an exciting twist: low-resource, poetic data where spelling variations and rhythmic patterns make transliteration more challenging.

Dataset link: urdu_ghazals_rekhta on GitHub

Data Preprocessing

Before any modeling, the dataset needed cleaning:

• Normalization: Removing extra punctuation and standardizing Urdu characters.
• Roman Urdu Preparation: The dataset contains multiple scripts (Urdu, Hindi, English transliteration). Where Roman Urdu was missing, I applied custom transliteration rules.
• Tokenization: I used subword tokenization (SentencePiece) to handle rare poetic words and spelling variations efficiently.

Model Architecture

For the model, I implemented a seq2seq architecture in PyTorch:

• Encoder: A 2-layer Bidirectional LSTM to capture context from both directions.
• Decoder: A 4-layer LSTM to generate Roman Urdu sequences step by step.
• Embedding Sizes: Experiments included 128, 256, and 512 dimensions.
• Regularization: Dropout layers helped prevent overfitting on the small dataset.

This setup allows the encoder to understand the full Urdu sentence before the decoder starts predicting Roman Urdu characters.

⚡ Training & Experiments

• Training: 50%
• Validation: 25%
• Test: 25%

Key training details:

• Loss Function: Cross-entropy
• Optimizer: Adam
• Batch Sizes Tested: 32, 64, 128
• Learning Rates: 1e-3 → 1e-4

To explore performance, I ran at least three experiments by varying:

• Embedding dimension
• Hidden size of LSTM layers
• Dropout rates

This helped identify the sweet spot for low-resource poetic data.

Evaluation Metrics

To measure model quality, I used:

• BLEU Score: To evaluate sequence accuracy.
• Perplexity: To check prediction confidence.
• Character Error Rate (CER): To capture small spelling mistakes.

Alongside metrics, I manually inspected sample outputs — because poetry deserves a human touch.

Example output:

Input (Urdu): “محبت میں یہ دل کہاں سنتا ہے” Prediction (Roman Urdu): “mohabbat mein yeh dil kahan sunta hai” Ground Truth: “mohabbat mein yeh dil kahan sunta hai”