Disasterous examples of Label Leaking in LLMs

Label leaking is a unique problem in datascience where in the question itself contains sufficient hints about the answer, in a way that doesn’t resemble real life data. This encourages models to cheat instead of deriving the solutions in the ideal way which would generalize to realworld data

Now I shall show 2 datapoints, find the one with label leak

sample 1) question=”1+1" answer=”2" and question =”0+1" answer =”1"

Sample 2) question=”1+1..” answet=”2" and question = “0+1.“ answer=”1"

Pause and think for a while before going for the answer!

Yes. Its the later, but why?? Sample 2 contains dots which can be easily used to count and compute the answer rather than learning to addition from scratch for a model.

This is a very simple example, more ridiculous examples follow:

[embed]

One can see over here that the presence of leading white spaces resulted in a correlation of 10% in chemistry, physics and math benchmarks

Now the most ridiculous of them all

[embed]*Mirage: The Illusion of Visual Understanding *Mohammad Asadi 1,∗ , Jack W. arxiv.org

When a student was writing code w.r.t evaluation of an LLM in a medical vision task, they forgot to pass the the images and the models was still able to compute the answers just straight out of memorized question answer pairs possibly from pretraining where the benchmark dataset got leaked into the pretraining data. And this was in medical benchmarks by the way!!

How to detect if your dataset is affected by it

A simple strategy would be is to mask the datapoints’ crucial information parts in all entries of a subset of the dataset (such that in ideal case labelshould not be derivable from the masked data, and then train/finetune a model on the same.) If the model despite masking gives accuracy greater than random baseline, then you have a case of label leaks!!