The Psycholinguistic Layer: What Separates PMaps from a Generic LLM Wrapper

A few years ago I watched an ex-colleague at Coimbatore fail a personality test that she could have easily passed.

The item was fine in English. Something close to: I keep my workspace organized. Agree or disagree, five points. It had been validated, benchmarked, and used tens of thousands of times. Then it was translated into Tamil accurately, by any reasonable standard. A bilingual reader would have called it a good translation.

But, it stopped measuring the big 5 traits it originally measured, for instance ‘conscientiousness’.

The Tamil rendering carried a connotation closer to tidiness as a domestic virtue than orderliness as a work habit. It induced unconscious bias as women answered it differently than men. Candidates from households where that phrasing carries moral weight answered it differently than candidates from households where it doesn’t.

The item still produced a number. However, numbers just weren’t about her personality anymore.

That is the whole problem in the items in the question, and it is the reason I get uneasy when someone describes what we do as “an LLM wrapper with translation.”

Translation is easy now. Measurement was never the same thing.

I want to be fair to technology, because the temptation is real and I feel it too.

You can take a validated English assessment, pass every item through a frontier model, ask for Hindi, Marathi, Tamil, Bengali, and Kannada, and have a multilingual assessment suite by Thursday. The output will be fluent. It will be grammatical. A native speaker will read it and nod. If your success criterion is did the sentence survive the trip, you have won.

An employment assessment job is to make a specific, invisible trait produce a measurable signal, and to do it the same way for every person who encounters it. The English word organized is not the payload. The payload is the psychological distance between the person and the trait, how hard the item is to endorse, what it costs socially to endorse it, what it implies about you if you don’t.

Translation preserves the sentence, but does not preserve the distance.

This is not a new insight and I don’t want to claim it as one. Psychometricians have been arguing about measurement invariance since long before any of us had an API key — the International Test Commission’s Guidelines for Translating and Adapting Tests run to eighteen guidelines across six categories, and the second edition took a committee ten years to produce. Ten years. For a document about how to move a test between languages responsibly. What’s new is the speed at which you can now generate a plausible-looking violation of it, and the confidence with which it can be shipped.

Three ways a translated assessment quietly stops working

In twelve years of building these, the failures cluster. Almost everything we catch falls into one of three buckets.

Construct drift. The item survives translation but lands on a slightly different trait. Our Tamil conscientiousness item drifted toward domestic tidiness. An assertiveness item translated into Hindi can drift toward rudeness, because the honorific system does work that English handles with tone. You are now measuring something. You are not measuring the thing on the report.

Difficulty drift. This one is nastier because it hides inside cognitive tests, where people assume language is neutral. It isn’t. A numerical reasoning item’s difficulty depends partly on how much working memory the stem consumes before the candidate reaches the math. Languages differ in how compactly they express a conditional. Move an item from English to Marathi and the arithmetic is unchanged while the reading load moves, so the item gets harder, the benchmark no longer applies, and the candidate is penalized for the sentence rather than the sum. Under item response theory, the item’s difficulty parameter has shifted and nothing in the scoring pipeline knows it.

Response style drift. Different linguistic and cultural contexts carry different norms about extremity and agreement like how comfortable you are picking the end of a scale, how much you shade toward the socially approved answer. Two candidates with identical traits can produce different response patterns purely because of which language they are thinking in. Aggregate that across a hiring floor and you have built a bias into the very tool you bought to reduce bias.

None of the three announce themselves. Every one of them produces a clean report, a confident score, and a candidate who is being measured against a ruler that changed length in transit.

What we actually built, and why is it slower than it should be?

PMaps delivers assessments in 8+ Indian languages. I’d like to tell you that number is an achievement. It’s mostly a process achievement, and the process is deliberately, unglamorously human.

Every item that moves between languages passes two gates.

Gate one is linguistic. Language experts, not just fluent speakers, but people who work on register, connotation, and regional variance render the item. Their brief is not “translate this.” It is “produce the item that does the same psychological work in this language,” which sometimes means the words diverge substantially from the English. The goal is equivalent difficulty and equivalent social cost, not equivalent vocabulary.

Gate two is psychological. Our test design team reviews what came back against the construct it is supposed to measure. Does this still load on conscientiousness? Has the reading burden moved? Is there a socially correct answer in this language that didn’t exist in English? Items that fail get sent back, rebuilt, or cut. Plenty get cut.

Then the item has to earn its place in the data, behaving consistently across the population that actually takes it, against the benchmark it claims to belong to. Across 3M+ assessments, drift shows up in the numbers before anyone reports it as a complaint. That feedback loop is the part that can’t be shortcut, and it’s the part a wrapper structurally cannot have. You can generate an item in a second. You cannot generate the evidence that it measures what you say it measures, that’s criterion validity, and it costs time nobody can compress.

Hinglish broke my own framing

The cleanest illustration of why this is a layer and not a feature came from a large BPO client, where we ran what we believe was India’s first Hinglish AI voice interview.

I had been thinking about languages as containers like Hindi over here, English over there, and the work is moving items between them. Hinglish doesn’t work like that. It isn’t Hindi with English words. It’s a code-switching pattern with its own logic about which language carries which kind of content, and a candidate on a hiring floor in Mumbai will switch mid-clause, use the English number and the Hindi verb, and be entirely fluent in something that isn’t in either dictionary.

If you ask a semi-skilled candidate to conduct an interview in “proper” Hindi, you are not assessing them. You are assessing how well they perform a register they don’t use, which is a real skill and completely irrelevant to whether they’ll be good at the job. Every point you dock is a measurement error dressed up as a finding.

So EVA, our voice interviewer had to meet the candidate in the language they actually think in. That is not a translation problem. There is no source text.

It’s also why so much of our test library is deliberately visual rather than verbal. If the construct can be measured without leaning on language at all, none of the three drift problems get a foothold. That’s a design decision, not a UX one.

The question I’d ask any vendor, including us

If you are evaluating talent assessment platforms and multilingual delivery is on your requirements list, the checkbox will not help you. Everyone checks it. The technology to check it costs almost nothing now.

Ask this instead: show me an item you cut.

Ask which items failed the move into Marathi and why.

Ask who made the call, whether they were a linguist or a psychologist, and what the disagreement between them sounded like.

Ask how the benchmark was re-established for the translated version, or whether the English benchmark was simply reused because if it was reused, you are comparing candidates to a population that never took their test.

A vendor with a psycholinguistic layer will have specific, or maybe slightly boring answers. A vendor with a translation pipeline will tell you about their language coverage. That difference is the entire thing. It’s why our scores predict on-the-job performance across sectors and why we’re comfortable being measured on that.

Not because the model is good. Models are good now, and they’re good for everyone. Because there is a layer underneath the model whose only job is to notice when a number has quietly stopped meaning anything.

The ex-colleague of mine in Coimbatore deserved a test that measured her fairly and most essentially objectively. That’s my stand on the necessity of a psycholinguistic layer in candidate assessments, what’s yours?