Bhasha-Rupantarika: Bringing Multilingual AI Translation to Edge Devices Through Algorithm-Hardware Co-Design

Language is one of the biggest barriers to technology access across the world. While modern AI systems have made tremendous progress in multilingual translation, most of these models remain too large, power-hungry, and computationally expensive to deploy on edge devices.

This challenge is particularly important in countries like India, where there are hundreds of regional languages, multiple dialects, and millions of users who may need translation support in education, tourism, healthcare, government services, and business communication.

To address this challenge, we developed Bhasha-Rupantarika, a multilingual neural machine translation framework that combines algorithm-level innovation with hardware-aware optimization.

The goal is simple: enable real-time multilingual translation on resource-constrained devices without sacrificing translation quality.

Problem Statement

Modern multilingual neural machine translation systems are usually designed for cloud-scale environments. They rely on large transformer models, high-precision arithmetic, and powerful GPUs or datacenter accelerators.

Although these approaches provide excellent translation quality, they are often unsuitable for low-resource environments such as:

• Rural and remote areas
• Edge devices with limited compute capability
• IoT systems
• Portable translation devices
• Low-power embedded platforms
• FPGA-based AI accelerators

For example, in many real-world scenarios, a user may want to translate between Indian languages and foreign languages in real time. Existing solutions often use a chain of multiple models:

1. Speech-to-text
2. Indian language to English translation
3. English to foreign language translation
4. Text-to-speech generation

This approach is computationally expensive, introduces latency, and requires multiple models to remain active simultaneously.

As multilingual AI becomes more important, the key question becomes:

Can we build a single lightweight multilingual translation model that works across many languages and is efficient enough to run on edge hardware?

That is the central motivation behind Bhasha-Rupantarika.

Why Existing Methods Are Limited?

Most existing multilingual translation pipelines rely on separate models for each stage of translation. For example, an Indian language sentence may first be translated to English and then translated again into another international language.

This multi-stage process introduces several limitations:

• Higher latency
• Larger memory requirements
• More model parameters
• Increased energy consumption
• Poor suitability for edge devices
• Difficulty in supporting low-resource languages

Traditional multilingual models also tend to use FP32 or FP16 precision, which increases model size and memory footprint.

For many transformer-based models, this can quickly become a bottleneck because attention mechanisms, feed-forward layers, and activation functions consume large amounts of computation and memory.

Another challenge is that existing hardware accelerators for NLP often focus only on matrix multiplication and do not optimize the nonlinear activation functions used in transformers.

However, activation functions such as SoftMax, sigmoid, tanh, and GeLU can contribute significantly to overall resource consumption. In some cases, activation hardware can consume up to 20–25% of the area in large accelerators.

In addition, many current NLP accelerators are designed for cloud servers or GPUs rather than FPGA-based or edge-oriented deployment.

As a result, there is a clear need for a hardware-software co-design framework that can reduce model complexity while improving hardware efficiency.

Proposed Innovation

Bhasha-Rupantarika introduces a lightweight multilingual translation framework optimized for low-resource environments.

The key innovation lies in combining:

• Ultra-low precision quantization
• Transformer-aware accelerator design
• Hardware-software co-design
• FPGA-friendly computation
• Shared multilingual translation models

Instead of using multiple independent translation models, our approach uses a unified bidirectional model that can directly translate between Indian and international languages.

The framework is built around the distilled INT4 version of NLLB-200, a multilingual transformer model supporting around 200 languages.

To improve hardware efficiency, we explored multiple sub-octet precisions:

• FP8
• INT8
• INT4
• FP4

These low-precision formats reduce model size, memory bandwidth, and arithmetic complexity.

We also introduced a hardware-friendly multilingual NLP engine with SIMD-based matrix multiplication, reconfigurable activation functions, and a quantized transformer datapath.

The overall result is a framework that achieves significantly lower area, lower power, and higher throughput while maintaining strong translation performance.

Architecture and Methodology

The proposed translation pipeline combines multiple AI models for speech processing, translation, and text generation.

The overall framework includes:

• OpenAI Whisper Large-v3 for multilingual speech-to-text
• AI4Bharat IndicConformer for Indian language speech recognition
• NLLB-200 for multilingual translation
• IndicTrans2 for Indian language translation
• Coqui XTTS v3 for multilingual text-to-speech
• IndicParler TTS for Indian language speech synthesis
• Meta LLaMA 3.2 for advanced multilingual reasoning

At the core of the framework is a distilled NLLB-200 transformer model with around 600 million parameters.

The model uses:

• Transformer encoder-decoder architecture
• Multi-head attention
• Pre-normalized residual layers
• Feed-forward networks
• Mixture-of-Experts (MoE) layers

The MoE mechanism is especially important because it activates only the most relevant expert for a given token. This reduces unnecessary computation while maintaining translation quality.

On the hardware side, we designed a Natural Language Processing Engine (NLPE) consisting of:

• Control Unit (CU)
• Memory Read Unit (MRU)
• Memory Write Unit (MWU)
• SIMD Matrix Multiplication Engine (MME)
• Nonlinear MIMD Vector Array (NMV)

The matrix multiplication engine supports:

• 1× BF16 operations
• 3× FP8 operations
• 6× FP4 operations
• 6× INT4 operations

This SIMD capability allows the same hardware to process multiple lower-precision operations in parallel.

We also developed a CORDIC-based Floating-point Activation function unit for SoftMax, sigmoid, tanh, and other nonlinear functions.

This hardware reuse strategy is important because nonlinear activation functions often consume a large portion of the total area in transformer accelerators.

The proposed design was implemented in SystemVerilog and evaluated using FPGA synthesis on AMD ZCU104 hardware.

Results

The performance improvements achieved by Bhasha-Rupantarika are significant.

Using FP4 quantization, the model size was reduced to just 0.56 GB, corresponding to a 4.1× reduction compared to FP32.

Inference speed improved by 4.2×, while throughput reached 66 tokens per second.

Compared to prior NLP accelerator designs, the proposed architecture achieved:

• 1.96× reduction in LUT usage
• 1.65× reduction in flip-flops
• 2.2× higher throughput compared to OPU
• 4.6× higher throughput compared to HPTA
• Operating frequency of 250 MHz

The SIMD MAC architecture also demonstrated:

• Up to 90% lower LUT usage compared to prior posit-based designs
• 3× lower power consumption compared to previous SIMD compute units
• Up to 3.66× lower delay compared to prior architectures

Similarly, the proposed nonlinear activation unit delivered a better balance between frequency, area, and energy efficiency.

These results show that low-precision quantization combined with hardware-aware optimization can make multilingual AI far more deployable on edge platforms.

Real-World Applications & Impact

The biggest impact of Bhasha-Rupantarika is that it enables multilingual translation on low-cost, resource-constrained devices.

Potential applications include:

• Real-time tourist translation systems
• Portable multilingual communication devices
• Rural education platforms
• Healthcare translation assistants
• Government service kiosks
• Smart village initiatives
• Cross-border business communication
• Translation support in XR and wearable systems
• Speech assistants for low-resource languages

For India in particular, such systems could help bridge the communication gap between local communities and global users.

For example, a farmer speaking Marathi could communicate with an English-speaking agricultural expert. A tourist speaking Italian could interact with a local Hindi speaker. A student in a rural area could access educational material in multiple languages.

Beyond translation, the hardware innovations developed in this work can also be extended to future transformer accelerators, LLM inference systems, and edge AI platforms.

Authors: Mukul Lokhande, Tanushree Dewangan, Mohd Sharik Mansoori, Tejas Chaudhari, Akarsh J., Damayanti Lokhande, Adam Teman, and Santosh Kumar Vishvakarma

Paper PDF (ArXiv): https://arxiv.org/abs/2510.10676

GitHub: https://github.com/mukullokhande99/Bhasha-Rupantarika/

This work highlights how multilingual AI systems can become smaller, faster, and more deployable when algorithms and hardware are designed together.

For future research, this direction can be extended toward larger multilingual LLMs, edge-based speech assistants, wearable translators, and AI accelerators for real-time language processing.