How I Built a Secure, On-Device Face Attendance App with Flutter, ML Kit, and TensorFlow Lite

Building a low-latency, privacy-first facial recognition pipeline that runs 100% offline.

Intro & The Why

Processing real-time computer vision completely offline on mobile devices used to require complex C++ setups and heavy cloud infrastructure. Today, we can build low-latency, privacy-first facial recognition pipelines right inside Flutter.

The Why: Local-First & Privacy-First AI

Facial recognition for employee attendance or access control often relies on cloud APIs. While easy to implement, cloud-based vision systems come with significant drawbacks:

1. Latency: Sending raw images or high-res video streams over mobile networks adds noticeable lag.
2. Privacy Risks: Storing or transmitting raw biometric data off the device creates serious compliance and security headaches.
3. Bandwidth Costs & Connectivity: If the internet goes down, your attendance system stops working.

To solve this, I built **flutter_face_attence_app—a Flutter application that handles real-time face detection, embedding extraction, and matching 100% on-device**.

Here is a deep dive into how it works, the architectural choices, and the low-level optimizations that make it butter-smooth.

Tech Stack

The Tech Stack At a Glance

• Framework: Flutter (3.27+)
• Face Detection: google_ml_kit (Fast bounding-box extraction)
• Face Recognition: tflite_flutter (Custom MobileFaceNet model for embedding generation)
• Local Biometric Storage: hive_flutter (High-performance NoSQL key-value store)
• Live Feed: camera plugin with customized YUV/NV21 stream handling

Architecture & Pipeline

🔬 Architecture: The Camera & ML Pipeline

Building real-time computer vision apps on mobile requires balancing accuracy with battery and thermal limits. Running heavy deep-learning inference on every single frame at 60 FPS will heat up the phone and kill the battery within minutes.

Here is how the pipeline handles live frames efficiently:

1. Smart Frame Throttling

Camera feeds output 30 to 60 frames per second. We implement an asynchronous gate (_isProcessing) alongside a timing throttle:

• Registration Mode: Processes 1 frame every 800ms.
• Login Mode: Processes 1 frame every 500ms.

If a frame arrives while the TFLite engine is still calculating, it is discarded immediately.

2. Detection via Google ML Kit

We use Google ML Kit’s face detector configured with FaceDetectorMode.fast and classification disabled (enableClassification: false). We don't need to know if the user is smiling or blinking—we just need a tight, fast bounding box around the face.

Handling Multiple People: If multiple faces enter the camera frame, the app uses a simple reduce check to select the face with the largest bounding box width, ensuring we only evaluate the person directly in front of the screen.

3. Feature Extraction with MobileFaceNet & TFLite

Once cropped, the image region is resized to 112x112 pixels, normalized, and fed into our TensorFlow Lite MobileFaceNet model. The output isn’t a classification like “John” or “Sarah” — it’s an embedding vector (a multi-dimensional array of float values representing unique facial features).

// Conceptual snippet: Running inference on the cropped face tensor
var input = preprocessImage(croppedFaceImage); // Resized to 112x112
var output = List.filled(1 * 128, 0.0).reshape([1, 128]);

interpreter.run(input, output);

4. Noise Reduction: Multi-Sampling & L2 Normalization

Single-frame scans are prone to lighting glitches, motion blur, or awkward angles. To make registration bulletproof, we don’t save a single frame’s embedding:

• Registration: Requires 8 successful frames.
• Login: Requires 5 successful frames.

The application computes the mathematical average across these sample vectors, followed by L2 Normalization(_mlService.l2Normalize()). This normalized vector creates a rock-solid, stable signature for the user that is invariant to minor lighting variations.

5. Vector Comparison (Euclidean Distance)

To log a user in, we compare their newly captured vector against vectors stored in hive_flutter. We calculate the Euclidean Distance. If the distance drops below our threshold (0.7), we have a match!

Native Optimizations

Native Optimization & Build Pitfalls

When taking an ML-heavy Flutter application to production, raw Dart code isn’t enough. You have to handle native Android build configurations properly.

1. Demystifying jniLibs

Because TensorFlow Lite is written in optimized C++ for high-performance matrix operations, Flutter uses JNI (Java Native Interface) to bridge Dart code with native C++ .so libraries.

Ensure your build targets the proper CPU architectures:

• arm64-v8a (Modern Android devices)
• armeabi-v7a (Older 32-bit devices)
• x86_64 (Android Emulators)

If you ever hit an UnsatisfiedLinkError: dlopen failed: library not found, it almost always means the native .so binary for that specific architecture was missing from your build.

2. Android ProGuard & R8 Configuration

When building for release (flutter build apk --release), Android's R8 shrinker minifies and obfuscates the Java/Kotlin code. If R8 strips native JNI bindings or Hive reflection classes, your app will crash on startup.

To prevent this, add explicit keep rules in android/app/proguard-rules.pro:

Code snippet

# Prevent R8 from stripping TensorFlow Lite native bindings
-keep class org.tensorflow.** { *; }
-keepclassmembers class org.tensorflow.** { *; }

# Keep ML Kit Vision libraries
-keep class com.google.mlkit.** { *; }
-keep class com.google.android.gms.internal.mlkit_vision_** { *; }

# Retain Hive Models and Annotations
-keep class com.hive.** { *; }
-keepclassmembers class * {
    @com.hive.annotations.HiveField *;
}

Ensure proguard-rules.pro is referenced in your android/app/build.gradle:

Groovy

buildTypes {
    release {
        signingConfig signingConfigs.debug
        minifyEnabled true
        shrinkResources true
        proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
    }
}

Conclusion & Call to Action

Results & Key Takeaways

By combining ML Kit for quick face isolation, MobileFaceNet via TFLite for embedding generation, and Hive for fast local storage, we achieved:

• Sub-50ms inference time on modern midrange devices.
• Zero network latency and complete offline functionality.
• Maximum privacy, as raw face images are discarded instantly after vector extraction.

On-device AI in Flutter is no longer experimental — it’s production-ready.

Try it yourself!

If you’d like to inspect the code, contribute, or run the project locally on your device, check out the repository on GitHub!

Repo Link: https://github.com/kamalexe/flutter_face_attence_app