Survey Protect: How We Identify and Prevent Clients From Survey Fraud
A lot of thought and effort goes into designing surveys and polls. But what if that time and effort is wasted because people cheat the system?
That happened in the 2020 presidential election when the industry encountered a historic polling error. It was the worst performance in four decades.
Most of the post-election analysis could not determine why the national polls so grossly overstated leads for Democrats. Over at our PollSmart product website, we've detailed our research on what happened in detail. Check it out; it's a sobering read.
The industry's most common but unconfirmed reasons are low cooperation rates among Trump-leaning voters. In short, most believe the problems were caused by the reticence of Trump voters to participate in or truthfully respond to polls.
The nonresponse issue is logical for phone polls where social desirability and nonresponse are real issues, but not for online polls where respondents answer anonymously, without human interaction, on a phone, tablet, or desktop screen.
Our analysis found that a high rate of bots and click farms answered the polls creating overstated Democrat support in 2020. We also discovered that the problem is more prevalent during presidential election years.
Like others in the industry, we purchase samples or respondents from sample companies who recruit and compensate those who complete a survey or poll.
Our analysis uncovered that nearly one in three respondents supplied by our vendors demonstrated some level of fraud. Click fraud is a huge problem for online advertising, and the market research and polling industries face the same issue. This activity can be very profitable in countries with weaker currencies vis-à-vis the dollar.
As a result, to ensure our polls and surveys only include real people, we've instituted the following measures to safeguard the quality of our results.
First, every respondent to a client survey must pass a rigorous fraud detection service typically used by banks and online retailers. We no longer rely on our partners, who have promised our industry, in writing, that they do this type of fraud detection work.
Second, we've built and installed our own in-house online survey platform to collect poll and survey responses. It's hosted on AWS services based in the U.S. to guarantee the reliability and security of our and our client's intellectual property.
Third, we have improved our use of trap questions in our surveys and polls.
Like others in the industry, we use trap questions (or "attention checks") in all of our studies to filter out bots and respondents who are not paying attention to our questions.
Just as artificial intelligence and other forms of software have improved, so too have the bots' ability to answer our traps correctly. We have created new and enhanced questions to improve our ability to capture actual human respondents who are paying attention to our questions.
We are dedicated to providing you with the best possible service and accurate, actionable data. Contact us today to learn more about our services!