Making Images Talk: Text Recognition in React Native (The Native Way)

JavaScript libraries let us down. ML Kit + native modules saved the day. Here’s how we made our React Native app read text from any image — handwritten notes, receipts, signboards, you name it.

The Problem Nobody Warned Us About

Picture this: your PM walks in and says, “Can we just scan a receipt and pull the text out? Should be easy, right?”

Famous last words.

We dove into the React Native ecosystem expecting a one-line npm install miracle. What we found instead:

• 🪦 Half-maintained JS libraries with last commits from 2020
• 🐢 Tesseract.js — works, but slow and bulky on mobile
• ❌ TensorFlow Lite — overkill for plain text recognition
• 🤷 No native-feeling, plug-and-play solution

So we did what every RN dev secretly dreads but eventually loves: we went native.

The Winning Stack: ML Kit + Native Modules

Google’s ML Kit runs on-device, works offline, and is free. The catch? You need to bridge it yourself. Here’s the 4-step playbook.

Step 1: Grab the Image

Use react-native-image-picker (or any camera lib you like):

import { launchCamera } from 'react-native-image-picker';

const result = await launchCamera({ mediaType: 'photo' });
const imageUri = result.assets[0].uri;

Pass that URI down to native. That’s the handoff.

Step 2: Android Module (Kotlin)

Add ML Kit to app/build.gradle:

implementation 'com.google.mlkit:text-recognition:16.0.0'

Then the magic method:

@ReactMethod
fun recognizeImage(filePath: String, promise: Promise) {
    try {
        val image = InputImage.fromFilePath(reactContext, Uri.parse(filePath))
        val recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)
        recognizer.process(image)
                    .addOnSuccessListener { visionText ->
                        promise.resolve(visionText.text)
                    }
                    .addOnFailureListener { e ->
                        promise.reject("OCR_FAILED", e)
                    }
            } catch (e: Exception) {
                promise.reject("ERROR", e)
            }
        }

That’s it. ~15 lines and Android is done.

3: iOS Module (Swift)

Add to your Podfile:

pod 'GoogleMLKit/TextRecognition', '3.2.0'

Then:

@objc(TextDetectionModule)
class TextDetectionModule: NSObject {
@objc
  func recognizeImage(_ filePath: String,
                      resolver: @escaping RCTPromiseResolveBlock,
                      rejecter: @escaping RCTPromiseRejectBlock) {
    guard let image = UIImage(contentsOfFile: filePath) else {
      rejecter("ERROR", "Invalid image", nil)
      return
    }
    let visionImage = VisionImage(image: image)
    visionImage.orientation = image.imageOrientation
    let recognizer = TextRecognizer.textRecognizer()
    recognizer.process(visionImage) { result, error in
      if let error = error {
        rejecter("OCR_FAILED", error.localizedDescription, error)
        return
      }
      resolver(result?.text ?? "")
    }
  }
}

Step 4: Call It From JavaScript

import { NativeModules } from 'react-native';
const { TextDetectionModule } = NativeModules;
const extractedText = await TextDetectionModule.recognizeImage(imageUri);
console.log(extractedText); // 🎉

That’s the entire pipeline. Image in, string out.

The Result

Snap a photo of a business card → name, email, and number land in your app as plain text. Point at a menu in a foreign language → feed the output to a translation API. Scan a whiteboard after a meeting → save the notes.

Once the bridge is in place, the use cases multiply fast.

Why This Approach Wins

On-device — no API calls, no latency, works offline

Free — no per-request billing like cloud OCR

Private — images never leave the phone

Fast — ML Kit is brutally optimized

Final Thought

React Native’s superpower isn’t that it does everything in JS. It’s that when JS can’t, you can still reach down and grab native APIs without rewriting the app.

Text recognition felt impossible for about 3 hours. Then it became a 50-line bridge.