Building an Urdu to Roman Urdu Translator using Neural Machine Translation (Byte Pair Encoding and Seq2Seq Architecture)

Introduction

Urdu is a widely spoken language in South Asia, but digital communication often happens in Roman Urdu (Urdu written using Latin characters). While Roman Urdu is common on social media, WhatsApp, and informal online writing, it lacks standard spelling and has limited linguistic resources. This poses an interesting problem for machine translation: How do we automatically translate Urdu text into Roman Urdu? Unlike traditional Urdu-to-English translation, this task requires handling orthographic variations rather than semantic differences.

In this blog, I will present my attempt at building a Urdu to Roman Urdu Translator using some neural machine translation methods.

Literature Review

Several approaches exist for handling this problem, ranging from simple to advanced:

1. Rule-Based Transliteration

• Mapping Urdu characters to fixed Roman equivalents.
• Pros: Simple, fast, requires no training.
• Cons: Cannot handle variations (khuda vs khudaah), dialectal spelling, or contextual disambiguation.

1. Dictionary-Based Mapping

• Uses bilingual lexicons for Urdu → Roman Urdu.
• Pros: Works for frequent words.
• Cons: Fails on out-of-vocabulary words and inconsistent spellings.

1. Statistical Machine Translation (SMT)

• Phrase-based translation using frequency and probability.
• Pros: Works with parallel corpora.
• Cons: Struggles with rare words and noisy data, requires alignment models.

1. Character-Level Neural Models

• Seq2Seq models trained directly on characters.
• Pros: Can handle unknown words.
• Cons: Training is slow, outputs often degenerate into gibberish for long sentences.

1. Subword-Level Neural Machine Translation (BPE + Seq2Seq)

• Uses Byte Pair Encoding (BPE) to split words into frequent subword units.
• Pros: Balances vocabulary size and coverage, handles spelling variations better, reduces out-of-vocabulary problem.
• Cons: Requires careful preprocessing, model still needs enough training data.

I chose BPE + Seq2Seq with LSTMs because it gave us the flexibility of handling morphological richness in Urdu while keeping the vocabulary manageable.

Methodology

For this project, a parallel corpus of Urdu–Roman Urdu pairs was prepared. To handle the morphological richness and spelling variations in the data, Byte Pair Encoding (BPE) is applied with 5,000 merges on both the source (Urdu) and target (Roman Urdu) sides. Special tokens such as <sos>, <eos>, <pad>, and <unk> were introduced to structure the sequences consistently during training and decoding. The dataset was then split into training, validation, and test sets to ensure fair evaluation. On the modeling side, sequence-to-sequence architecture was built where the encoder is a bi-directional LSTM designed to capture Urdu’s word order, while the decoder is a multi-layer LSTM trained with teacher forcing to stabilize learning. A bridging layer was included to map the encoder’s hidden states into a form compatible with the decoder’s initialization. The model was optimized using CrossEntropy loss with padding tokens ignored, ensuring the network focused only on meaningful content rather than padded positions.

Challenges Faced

During this work, several challenges were faced that shaped the direction of our approach. One major issue was the limited availability of parallel Urdu–Roman Urdu data, which made it difficult for the model to generalize effectively. Another challenge arose from Roman Urdu’s lack of standard spelling, where the same word could appear in multiple forms, adding noise to the training process. On the modeling side, we observed that the network often struggled in early epochs, producing repetitive or nonsensical outputs, which highlighted the need for careful preprocessing, teacher forcing, and gradient clipping to stabilize training. Difficulties in balancing the complexity of the model with the size of the dataset were encountered, as larger models risked overfitting while smaller models underperformed. Despite these challenges, the use of BPE and a Seq2Seq framework helped us achieve more consistent results compared to simpler tokenization or word-level approaches.

Results

The trained Seq2Seq model with BPE tokenization achieved a BLEU score of 0.1490, a perplexity of 10.32, and a character error rate (CER) of 0.278 on the test set. These scores indicate that the model not only generated fluent Roman Urdu but also maintained a relatively low error rate at the character level, which is particularly important given the spelling variability of Roman Urdu. The improvements over earlier baselines confirmed that subword-level tokenization combined with a multi-layer Seq2Seq architecture is well-suited for this translation task.

Live APP: https://jyn9dvztques6amfjseyhq.streamlit.app/

figure 1. Translation Results

Conclusion

This project shows that subword-level Seq2Seq models can learn Urdu → Roman Urdu translation, even under low-resource conditions. While results are still modest, the system demonstrates real progress compared to naive rule-based or dictionary-based approaches.

With more data and stronger architectures, this work can evolve into a practical Urdu–Roman Urdu translation tool for chatbots, social media platforms, and assistive writing applications.