News Coverage of Campaigns - PresidentialCampaign2004 -The Messages and Their Analysis - William Benoit

News Coverage of Campaigns

Understanding Public Opinion Polls
Several important factors influence polls, what they mean, and how much we should trust the results.

Sampling. Polls rely on sampling, a procedure in which only some people are interviewed and their answers are taken to represent everyone’ answers. It would be too expensive and impossible in practice to ask all people which candidate they preferred, even in a primary state. However, we can make reasonably accurate projections based on small samples. But the fact that polls use samples means their sampling procedures–how do they decide who to interview–can make a difference. In 2004, for example, if a pollster interviewed mostly people who live in Vermont, where Howard Dean was governor, they would probably obtain different answers than if mostly Missourians were interviewed, because Dick Gephardt has been a Representative from Missouri for years.

Geographical location is only one factor in sampling. Another is the target population. Is the poll trying to represent everyone or a particular group of voters? Almost every poll is limited to people over 18, because only they are eligible to vote. It might be very interesting to learn how 16-year olds feel about the candidates, but that information wouldn’t be useful in predicting the election outcome. Other polls interview registered voters or likely voters. This restriction in sample follows the same basic logic: The only people whose opinion matters in predicting the election outcome are those who can vote (are registered) or who are likely to vote. In a primary campaign another concern is political party affiliation. In 2004 only the Democratic nomination is being contested. So, a poll of Democrats (or likely Democratic voters) might tell us more about the primary election outcome than a poll of all voters. To mention one concern that might not be obvious, some Republicans might answer a poll about Democratic candidates strategically. That is, they might not tell who they think would be the best Democratic candidate but who they think is most likely to lose to Republican President George W. Bush. So, understanding a poll means understanding the nature of the sample who were interviewed.

Sampling Error. Because a poll uses a sample, the results should reflect the attitudes of the target population (all voters, or likely voters, or Democrats) but still the sample might vary somewhat from the entire population. If sampling is done properly, statistical theory tells us that most polls (most samples) will cluster around the “real” attitudes of the target population. For example, if a poll finds that 36% of Democratic voters surveyed favor Howard Dean, 36% is the best guess about how many Democratic voters actually prefer Dean. But it could be 35% or 37%; it might be 34% or 38%. Many polls have a sampling error of plus or minus 4 percentage points. That means if a poll of a particular sample finds 36% favor Dean, it is very likely that between 32% and 40% of all Democratic voters favor Dean. At the risk of confusing things, Dean’s “real” popularity might be even lower 32% or higher than 40%, but it is most likely to be between 32% and 40%, with 36% being the best guess.

A Gallup Poll conducted between January 2-5, 2004 found Dean at 24% and Clark at 20% with a 5% margin of error. This means support for Dean could be as low as 19% and Clark could be as high as 25%. The numbers in bold font show the most likely attitudes in the population surveyed:

Dean 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Clark 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

In this case, the best guess is that Dean is slightly ahead of Clark, but they could be tied and it is even possible that Clark is really ahead of Dean. So, it is important to keep in mind the fact that a poll is best considered a projection (based on useful information, not a wild guess) but there is a margin of error.

Question Wording. The answer to a question in a poll is dependent on how the question was asked. For example, it was suggested above that some Republicans might answer a poll about Democratic candidates strategically. Democrats could do the same thing. Although many voters believe the candidate they like best will win, some people prefer a candidate whom they do not believe can win. Those voters might decide to vote for a candidate who is acceptable and electable rather than the one who is preferable but not electable. The following two questions could easily elicit the same answer from the first group of voters but different answers from those who believe their preferred candidate cannot win:

Q1. Which candidate do you like best?
Q2. Which candidate will get your vote?

Some polls ask about the most important problem. Most polls ask questions like “What is the most important problem facing America today?” In late 2003, the answer to that question was frequently jobs and unemployment. A few polls, however, asked a question that sounded similar: “What is the most important problem affecting you personally?” Many people who answered the first question with “unemployment” but who have a job could give a different answer to this question because they have a job and unemployment does not affect them personally. So, it is important to know precisely what question is being asked, because subtle differences in the wording of the question can change the results of a poll.

Undecided. One reason polls do not always reflect the outcome of an election is that many people are undecided until they actually get ready to vote. Obviously, if 35% prefer candidate A and 30% prefer candidate B and 30% are undecided, candidate B will win the election if most of the undecided ultimately choose candidate B at the polls (or if the undecided never decide and do not vote). But there is a more subtle problem here. Many pollsters do not like “undecided” or “don’t know” responses. For example, a voter who answers “I don’t know” to the question of which candidate is best may be pressured with follow up questions to pick someone: “Well, can you tell me which candidate you lean toward?” Some respondents continue to say they don’t know, but others will give an answer. Pollsters may lump the “leaners” in with those who have definitely decided to vote for a candidate. Of the 35% who favor candidate A above, if only 20% have decided on that candidate and 15% lean to that candidate, the “leaners” might very easily switch to candidate B when they vote. So undecided voters who decide, and “leaners” who switch when they decide, can make election results look different from poll results.

Gawiser and Witt (2002) discuss 20 questions journalists should ask about polls. (See References.)

Political party logos