How We Built a “Truth-Based” Audit Layer for India’s Highway Network

Moving from "Trusting the Tag" to "Verifying the Reality" with Computer Vision.

India’s Electronic Toll Collection (FASTag) system is a marvel of modern infrastructure, handling millions of transactions daily with incredible speed. It relies on a simple, efficient protocol: the RFID tag on your windshield identifies you to the toll barrier, and the barrier lets you pass.

But this system has a hidden vulnerability: It is blind.

The toll system operates on a "Trust Protocol." If an RFID tag says "I am a Car," the system believes it—even if the vehicle carrying that tag is actually a 12-wheel commercial truck carrying 20 tons of cargo.

This simple discrepancy—where heavy vehicles use light-vehicle tags—creates a massive loop of Revenue Leakage, costing operators crores annually. The digital identity does not match the physical reality.

We call this the "Semantic Gap."

The Problem: Digital Trust vs. Physical Truth

In the current setup, there is no "Visual Audit Layer." The system processes the transaction based entirely on the digital data broadcast by the tag. It assumes the tag is on the correct vehicle.

But physics doesn't lie. A truck occupies more space, moves differently, and has a distinct visual footprint compared to a hatchback.

We realized that to stop this leakage, we didn't need to overhaul the entire banking infrastructure. We just needed to give the toll barrier a pair of eyes.

The Solution: Cross-Modal Verification

Our project proposes a Computer Vision-based Audit System designed to close this gap.

Instead of relying solely on the RFID sensor, our system introduces a second stream of data: Visual Class Verification.

Here is how the logic flows:

1. The Claim (RFID Input): As a vehicle approaches, the RFID reader picks up the tag ID. The database returns the claimed class (e.g., "Light Motor Vehicle").
2. The Reality (Vision Input): Simultaneously, a camera feeds live video to our Computer Vision engine. The system analyzes the vehicle's physical attributes—its shape, size, and structure—to determine its actual class (e.g., "Heavy Commercial Vehicle").
3. The Audit (The Handshake): The system compares the two inputs in real-time.

• If Claim == Reality: The barrier opens.
• If Claim != Reality: The system triggers a "Mismatch Alert," flagging the vehicle for manual inspection before it leaves the lane.

Why This Matters

This isn't just about catching a few cheaters. It represents a fundamental shift in how we design Intelligent Transport Systems (ITS). We are moving from a Trust-Based Model (where the system passively accepts data) to a Truth-Based Model (where the system actively verifies reality).

By deploying a lightweight, non-intrusive visual layer, we can protect infrastructure revenue and ensure that every vehicle pays its fair share—without adding a single second of delay to the commuter's journey.