When Research Participants Lie: How to Spot Scammers on Panel Platforms

I remember my first impression of panel platforms like Respondent, User Interview — wow, with a platform like this you can find any audience for 95% of projects. All you need to do is set the filters, create a clear screening survey, and voilà: you can pick the most interesting and relevant “berries” from the field and put them in your basket.

For some projects, this really can work. But nowadays, there are so many scammers and opportunists, often not from the country that is needed, who create fake accounts and profiles and become “professional” research participants. The problem is that these people don’t have the real experience researchers are looking for, which means the data won’t be valid. They learn and experiment with strategies to get selected for studies, but once they’re in an interview it quickly becomes obvious that they’re not who they claim to be. I’ve had to stop interviews because of obvious lies and report the account, it’s not the end of the world, but unpleasant and it pushes the project off schedule.

That’s why you have to get creative and come up with new ways to avoid this.

I’ve come to a few hygiene rules (in addition to manually checking profiles and LinkedIn pages, if available):

• If interviews are conducted across different time zones, say, you’re in Europe and need participants from the other hemisphere, better to pay attention to when the screening survey was completed.
• Design a thoughtful screening survey. Sometimes it’s better to phrase questions so they look like simple preference clarifications, rather than obvious gatekeeping. That way, disqualifying answers aren’t too obvious.
• This strategy is especially useful when recruiting people with specific skills. The right person is much more likely to answer all professional questions correctly. You can even make all the listed options wrong and leave only an “Other” field, and a qualified participant, carefully reading the question, is more likely to write in the correct answer there. Then you can manually review and filter those responses. Someone without the necessary skills or knowledge likely won’t notice the trick at all.
• Include a couple of test questions from the screening survey at the beginning of the interview to compare the answers.
• And of course, make sure to ask participants in the screening survey whether they are willing to have their camera on.

What about you? Have you ever encountered tricky panel participants, and what strategies have you used to handle them?