Optimizing AI for Mobile: The Power of Model Quantization(4)

Part 4: Integrating the Quantized Model into a Mobile App

In the previous part, we quantized our Wav2Vec2-based speech model, calibrated it using real audio, exported it to Executorch format, and evaluated its performance on the TIMIT core test set. This article takes that model and completes the journey - integrating it into a mobile app built with ***Expo and [react-native-executorch](https://docs.swmansion.com/react-native-executorch/)***. The goal is simple: run the quantized model fully on-device, process live audio, and generate IPA phonemes with timestamps.

All mobile code referenced in this article is available at ***here***.

Hosting and Downloading the Model

Instead of embedding the .pte model directly inside the application bundle, we choose to host it remotely - Hugging Face in our case - and download it the first time the app runs. This approach keeps install sizes manageable and allows updates without waiting for app-store approval. If the quantized model ever changes, the app simply fetches the latest version.

The project downloads the model to the device’s document directory, checks its presence on each launch, and loads it through react-native-executorch once it’s available.

Recording Audio at 16 kHz

Because the quantized model expects one second of audio at 16,000 Hz, the app must feed audio in the same format. The mobile implementation uses react-native-audio-api, which streams raw PCM buffers directly from the microphone. This is ideal for on-device inference because it avoids creating temporary files and provides audio in Float32 format.

The recorder streams chunks of 2400 samples at exactly 16 kHz. Each time a chunk arrives, the app forwards it to the inference pipeline:

recorderRef.current = new RNAudioRecorder({
  sampleRate: 16000,
  bufferLengthInSamples: 2400,
});

This matches the model’s calibration settings from Part 3 and keeps the audio pipeline predictable and stable.

Preparing Audio for Inference

Since inference uses one-second windows (16,000 samples), the app must gather incoming chunks, normalize them, and merge overlapping sections to match the window size. The implementation maintains an overlap buffer and slides through audio in fixed increments to avoid artifacts between chunks.

The core logic lives in SpeechScreen.tsx and performs:

1. Collection of incoming Float32 audio chunks
2. Normalization of samples
3. Concatenation with the previous overlap
4. Sliding-window segmentation
5. Sending each window to the model for inference

The design ensures that the on-device pipeline matches the quantization behavior precisely.

Running the Quantized Model with Executorch

Once the model file is downloaded and stored locally, it can be loaded with:

const exec = useExecutorchModule({ modelSource: MODEL_LOCAL_URI });

Each inference step creates a tensor that mirrors the model’s input signature:

const tensor: TensorPtr = {
  dataPtr: inferenceChunk,
  sizes: [1, inferenceChunk.length],
  scalarType: ScalarType.FLOAT,
};

The forward call returns raw logits. These logits represent frame-by-frame probability distributions over the phoneme vocabulary, identical to the PTQ model evaluated in Part 3.

Decoding IPA Phonemes with Timestamps

Once logits are produced, the app decodes the phonemes using a simple CTC-style argmax decoder. The vocabulary mapping stored in vocab.json lets us resolve each ID into an IPA symbol, and because each model frame represents a slice of time, we can compute start and end timestamps for each segment.

The decoder walks through the logits, merges repeated symbols, removes blanks, and assigns timestamps based on sample count. The resulting list of {phoneme, start, end} entries is then displayed on-screen using PhonemeResult.tsx.

This gives the user a clean timeline of IPA phonemes aligned with their speech.

oʊ 1.92s → 1.94s
k  2.04s → 2.07s
eɪ 2.09s → 2.11s

Putting It All Together

The full mobile pipeline follows a clear sequence:

• Download the quantized model from a remote host
• Load it into the React Native app using Executorch
• Start streaming microphone audio at 16 kHz
• Normalize and segment audio into one-second inference windows
• Run each window through the .pte model
• Decode the logits into IPA phonemes
• Show each phoneme with accurate timestamps
• Track memory usage to ensure on-device stability

The implementation remains faithful to the behavior of the quantized model we built earlier, resulting in a consistent on-device speech recognition experience.

Wrapping Up

Thank you for following this multi-part series. We’ve taken a full journey - from understanding quantization fundamentals (Part 1), choosing a method tailored to our model (Part 2), executing quantization and evaluating the results (Part 3), to now embedding the quantized model in a mobile app (Part 4).

Thanks for reading - you didn’t waste your time & you invested it. 😎