Verifying a Face Without Ever Sending It Anywhere

How we check that the person taking the test is the person who applied, while their face stays on their own machine. Part 4 of a series on building DeepScreen, our final-year hiring platform.

Remote assessment has an obvious hole in it. A candidate gets shortlisted, then quietly asks a stronger friend to sit the coding test or the interview in their place. Nothing about the resume screening or the questions catches this, because the problem is not what is being answered. It is who is answering.

The fix sounds simple: check the person’s face. The trouble is that a face is exactly the kind of data you least want to collect and least want to leak. So we had two goals that pull against each other. Confirm identity reliably, and never become a service that hoards people’s biometrics. This article is about how we tried to do both, and where it is still imperfect.

The one decision that shaped everything

Here is the design rule the whole system is built around: the raw face image never leaves the candidate’s machine.

Every step that touches an actual camera frame happens locally, inside the desktop app. What leaves that boundary, if anything, is not a picture. It is an embedding, a list of numbers that summarizes the face. You cannot look at that list and see the person. You can only compare it to another list and get a similarity score.

Everything that touches the raw frame happens on the device. Only the embedding, a list of numbers, is ever stored or compared.

That one rule is what lets us do identity checks without building the thing nobody should build: a central pile of people’s face photos.

Turning a face into a number

The check runs in two steps, and it is worth separating them because they do different jobs.

The first step is detection. As the candidate sits in front of the camera, the app finds the face in the frame and tracks its features, drawing a box and points on screen so the person can see they are lined up correctly. This runs live, right in the app, and nothing is sent while it happens. No frame goes anywhere until the candidate actually chooses to submit.

The second step is the embedding. The detected face is cropped, resized to a small fixed size, and passed through a face-recognition model that outputs a 512-number vector. This runs in the app’s Rust backend, isolated from the browser part of the app. The model was trained with a method called ArcFace, and the useful property of ArcFace is this: it arranges its embeddings so that two pictures of the same person land close together and two pictures of different people land far apart. Because of that, a plain similarity measure between two embeddings is a reliable way to tell “same person” from “different person.” The model itself is small, well under a million parameters, and runs comfortably on a normal CPU with no graphics card, which is what makes it practical to run on a candidate’s laptop.

Two phases: enrol once, verify often

The candidate enrols one time, when they create their account. The app computes their embedding and stores that single list of numbers, keyed to their user ID. The raw image is discarded once the embedding exists. There is no photo kept, anywhere.

Then, at the start of each assessment stage, the app computes a fresh embedding from the current camera frame and compares it to the stored one. If the two are similar enough, the candidate is admitted. If not, the attempt is blocked and the recruiter is notified. That is the entire loop: enrol once, and re-check at every door.

The privacy detail I want to dwell on

There is a difference between two things that sound the same.

One is: “who is this face?” That is a search. You take a face and compare it against a database of everyone, looking for a match. It is how surveillance systems work, and it is exactly what we did not want to build.

The other is: “is this the same person as before?” That is a lookup. You take a face and compare it against one specific stored embedding, the one belonging to this user ID, and nothing else.

DeepScreen only ever does the second one. When it verifies you, it fetches your embedding by your ID and compares against that alone. It never runs your face against other candidates. This is not a small implementation choice, it is the difference between an identity check and a face-search engine. Building it as a direct lookup means the system structurally cannot identify you against other people, even by accident. It also happens to be faster, but the privacy property is the point.

About that threshold

The similarity has to clear a bar to count as a match, and we set that bar at 0.6. That number is not arbitrary. For this kind of ArcFace-trained model, embeddings of the same person typically score somewhere in the 0.7 to 1.0 range, and embeddings of different people typically land between 0.0 and 0.3. A cutoff of 0.6 sits in the comfortable gap between those two, which is why it separates genuine matches from impostors cleanly in normal conditions.

The phrase “in normal conditions” is doing real work in that sentence, and it leads straight to the part I am least comfortable glossing over.

Where this is still imperfect

Face technology has a well-documented failure mode: it does not perform equally across skin tones and lighting conditions. We set our threshold against the published behavior of this class of model, but we have not yet run our own evaluation across a diverse range of faces and lighting. Until we do, I would not claim the 0.6 bar is fair to everyone equally, and neither should anyone repeating these numbers. It is a known risk, we are naming it on purpose, and testing it is on the list before this could be relied on in production.

There is a gentler problem too. A mandatory face gate frustrates people with a bad webcam or awkward lighting through no fault of their own. A real deployment needs a calibration step and a supervised way to re-enrol when the automatic check keeps failing a legitimate person. Otherwise the security feature quietly becomes an accessibility barrier.

And as with the rest of DeepScreen, this gate flags rather than decides. A failed check blocks entry and alerts a human. It does not reject the candidate on its own. A person looks at the flag and decides what to do.

Data lifecycle, briefly

To close the loop on the privacy claims: the only face data stored is the embedding, never an image. It is kept alongside no picture. It is used only for that user’s own verification. And it is deleted when the account is closed. The goal throughout was to hold the least data that still does the job, and to hold it in the least revealing form.

Takeaways

• Decide the privacy boundary first, then build inward. “The raw image never leaves the device” shaped every other choice here.
• Store the embedding, not the image. A face embedding does the verification job without being a photo.
• A lookup is not a search. Comparing against one known ID is a fundamentally more private operation than matching against everyone, even if the math looks similar.
• Pick thresholds from known behavior, but do not claim fairness you have not measured.
• Keep a human in the loop, and give people a way to recover when the automatic check is wrong.

Next, and last in this series: how all of these pieces, the ranking, the Rust inference, the interview, and this verification gate, fit together into one system.

DeepScreen is our final-year project at the Department of Computer Science, Information Technology University, Lahore. Built by Bazil Suhail, Syed M. Abdullah Masood, and M. Ahmad Saleem, supervised by Dr. Mehwish Ghafoor.