Politicians often have sort of a love-hate relationship with surveys, mostly depending on whether results are encouraging for them or not. If results are positive, they sometimes ignore all limitations of market research. If results are negative, on the other hand, they dismiss the research in question all together. So here is part two of clearing up the basics.
The bigger the sample size, the smaller the margin of error. Let’s assume you have a survey with a margin of error of +/- 3.5% and your candidate gets, say, 42% of the vote. This means that the vote share of that candidate among the entire population is between 38.5% and 45.5%. In fact, since a confidence level of 95% has become sort of industry standard, it means that there is a probability of 95% that the vote is within the range of 38.5% and 45.5%, and a 5% chance that results are outside of that range. Nothing more and nothing less.
There are politicians who don’t care about the margin of error and run surveys with ridiculously small sample sizes. The margin of error then becomes so big that the data may look scientific but is practically meaningless. On the other hand, there are people who believe in huge sample sizes. This is not wrong in itself, but it becomes expensive. As a rule of thumb, 600 respondents is sort of the strict minimum in my view for a political survey in a locality. The advantage of a bigger sample size is that it allows more precise geographic and socioeconomic targeting. What’s completely wrong is to get the minimum sample size, but then expect a detailed reading for sub-categories, for example by town.
Some politicians wrongly assume that the sample size is the only factor affecting the quality of a survey. In reality, there are plenty of other issues to consider: the quality and flow of the questionnaire, the translations of it, and of course the sampling. The fieldwork is also crucially important: the training and supervision of the interviewers or the extent of back-checking. In the countries where I work, I have invested a lot of attention into building solid fieldwork partnerships. It’s important to get all these things right since planning a campaign is like walking in a labyrinth: if you take a wrong turn at the beginning (=base your strategy on flawed research), everything that follows will be wrong as well.