What Every Politician Should Know about Survey Research

author
Dr. Louis Perron
blog post louis

Regardless of what they say in public, politicians need (and jump on) surveys like a junkie needs heroin. When I talk with clients, I am nevertheless often surprised about the prejudices regarding public opinion research, hence why in this post, we will clear up some of the basics.

Surveys that are published in the media often focus on the so-called horse race (“if the election were this Sunday, what party would win”). The way we use surveys in campaigns has a totally different purpose, namely to serve as a basis to make informed decisions regarding your chances to win, your message, targeting (low-hanging fruits) and spending (media buy, assessing campaign material).

But let’s start at the beginning: The bigger the sample size, the smaller the margin of error. Let’s assume you have a survey with a margin of error of +/- 3.5% and your candidate gets, say, 42% of the vote. This means that there is 95% chance that the vote of that candidate among the entire population is between 38.5% and 45.5%. Nothing more and nothing less.

There are people who don’t care about the margin of error and run surveys with ridiculously small sample sizes. The margin of error then becomes so big that the data may look scientific, but is practically meaningless. On the other hand, there are people who believe in huge sample sizes. This is not wrong in itself, but it becomes expensive. As a rule of thumb, 600 respondents is sort of the absolute strict minimum for a political survey in a locality. The advantage of a bigger sample size is that it allows more precise geographic and socio-economic targeting. What’s completely wrong is to get the minimum sample size, but then expect a detailed reading, for example by town.

Some politicians wrongly assume that the sample size is the only factor affecting the quality of a survey. In reality, there are plenty of other issues to consider: the quality of the questions, the sampling, the training and supervision of the interviewers, the extent of back-checking and the quality of translations. It’s important to get all these things right because a customer does not only pay for numbers, he or she pays to know the true numbers.

In the U.S., I learned that as a rule of thumb, it makes sense to allocate between 10% – 15% of the overall campaign budget to public opinion research. In a presidential campaign, this can mean daily tracking. In a local campaign and if resources are really scarce, this can mean one or two surveys. It depends on the budget.

A good survey can tell the awareness and the vote share a candidate gets at the present point in time. It can further assess the potential to grow a candidacy, determine the lowest hanging fruits regarding socio-economic targets, and the man or woman to beat. It can measure the impact of specific achievements or legislation and assess the power of endorsements. This being said, surveys are never a prediction of the result. In that sense, it just makes me giggle when the media shows surveys with Joe Biden leading the Democratic primary field for 2020. Early surveys merely reflect awareness and, yes, Biden is the most known of the potential Democratic candidates. But over the next months, primary voters will get to know the other candidates, absorb new information and everything might (and most likely will) change. And in particular voters who go to caucuses such as in Iowa are known to decide late.