Methodology

Election Polling

One of the most prominent applications of survey research is election polling. In election years, much of the Pew Research Center's polling focuses on elections, and even in the so-called "off years," many of our polls include questions gauging voter reactions to events or their future behavior.

Pre-election polling is one of the few times when pollsters can "check their work" by measuring how well their polls matched election outcomes. But, polls designed to measure vote intentions serve up some special challenges. How do you identify which respondents will actually vote? Are respondents honest when they tell us for whom they intend to vote? How will undecided voters make their final decision?

Although election polls attract a great deal of attention for their ability to predict the outcome of elections, their most important function is to help journalists and citizens to understand the meaning of the campaign and the election. Polls help to explain, among other things, what issues are important, which qualities of the candidates affect voters' decisions, or whether or not the winning candidate can claim a mandate for particular policy changes.

Identifying likely voters

One of the most difficult aspects of conducting election polls is determining whether a respondent will actually vote in the election. More respondents say they intend to vote than actually do so. As a consequence, pollsters do not rely solely upon a respondent's stated intention when classifying a person as likely to vote or not. Most pollsters use a combination of questions that measure intention to vote, interest in the campaign, and past voting behavior. Different pollsters use different sets of questions to help identify likely voters.

Pew's likely voter questions can be found in Pew Election Questions in red type. We use eight questions to assign each respondent a score on the likely voter scale in our final pre-election poll. Earlier in the campaign, we use a somewhat shorter version of the scale to identify likely voters.

To help identify the questions or combination of questions that were most accurate in predicting voting, Pew conducted an experiment in a closely contested mayoral race in Philadelphia in 1999. This report describes that experiment and what we learned from it. Along with the academic paper, which is linked to within the report, it also provides more detail about how the likely voter scale is created and used.

These reports contain the results of Pew's final presidential election forecast polls:

Back to top

Determining voter preference

Determining a voter's preference among the candidates running for office would appear to be a relatively simple task: just ask them who they are going to vote for. In fact, differences in how this question is asked, and where it is placed in the questionnaire, can affect the results. While most voters have usually made up their minds and are not likely to be affected by how the question is posed, many people have given less thought to the campaign or are genuinely ambivalent about the choices. For these voters, certain features of the question can make a difference.

The questions in Pew Election Questions in blue type were used by Pew in its final poll of the 2004 presidential election. The particular features of these questions reflect several choices:

  • Both the presidential and vice presidential candidates are included in the questions.
  • The party affiliation of each ticket is mentioned explicitly.
  • In states where Ralph Nader was on the ballot, he and his running mate are included in the choices read to respondents.
  • The order of presentation of the Democratic and Republican tickets is rotated from respondent to respondent; the Nader ticket always comes last.

These features are an effort to make the presentation of the options as similar as possible to what the voter would actually experience in his or her state. Most obviously, voters where Nader is not on the ballot would not be asked whether they favor him. And the rotation of the order of the two major party tickets simulates the fact that the order of the ballot may be different in different locations. However, this effort is not perfect, since not all states mention the party affiliation of the ticket, and not all states feature a rotation or random selection of the ballot order. In addition, there are often other candidates on the ballot in some states. Pew and most other national polling organizations make a judgment as to which third-party tickets it is important to include in their survey questions.

In addition to the problem of third-party tickets in presidential elections, pollsters face an even more difficult challenge in primary elections, where the number of candidates is often very large. Long lists of candidates can be very unwieldy in a telephone survey, and the effect of a candidate's position on the ballot can be even more consequential than in a general election contest where fewer candidates are listed.

Two other choices in Pew's questions are also important to note:

  • The trial heat questions are asked very early in the questionnaire, prior to any other substantive questions about politics other than registration, political engagement, and past voting history. This is done to avoid the possibility of affecting the voter's choice by raising considerations such as issues, candidate personalities, or other factors. While all of these may ultimately be relevant to the voter's choice, there is no guarantee that the things we mention will be the ones most important when a voter finally decides. Thus, it is important to make the choice as "clean" as possible.
  • Some voters will not express an initial choice; for these, a followup "leaner" question is asked (Q4b or Q4bX in Pew Election Questions) in an effort to obtain a preference. Answers to this question are typically included in the tabulation of voter preference.

The remaining questions in the series shown in Pew Election Questions are used to gauge strength and certainty of support (Q4a and Q4b), as well as to distinguish between "positive" and "negative" voting (questions Q5a and Q5b). One common analysis in Pew surveys is to document the size of the so-called "swing vote" - defined as voters who are either undecided, only leaning to a candidate, or who say they might change their mind before Election Day. The following report describes an analysis of the size of the swing vote in the late stages of the 2004 campaign and how swing voters identified in earlier surveys responded when re-contacted in mid-October:

One final issue in determining voter preference is the question of whether respondents will always answer honestly when asked for their choice in an election. It is usually assumed that voters who are reluctant to tell a pollster who they intend to vote for will simply decline to be interviewed or to answer the trial heat question. But a pattern of polling errors during the 1980s and 1990s in elections involving African American candidates raises the question of whether some people are reluctant to say that they are voting against a black candidate. Alternatively, there is the possibility that racially conservative respondents are disproportionately likely to refuse to be interviewed, thus producing a bias in the poll's estimate of the outcome.

This question has obvious relevance for the 2008 presidential election, which will match Barack Obama and John McCain. Pew's researchers have examined the question of whether polling still overstates support for black candidates. While the pattern was clear in the 1980s and earlier in the 1990s, more recent elections in 2006 showed little sign of the so-called "Bradley effect":

The issue is far from settled, of course. The failure of polls to predict Hillary Clinton's victory in the New Hampshire Democratic Primary occasioned considerable discussion about the possibility that the pattern might reappear. Pew's president Andrew Kohut provided one perspective in the New York Times:

Back to top

Gauging the accuracy of election polls

Although polls have occasionally failed to predict who will win an election (most recently in the 2008 Democratic primary election in New Hampshire), polling's track record is actually very good. For example, in 2004 the average of several major national polls taken in the days leading up to the election showed President George W. Bush with a 1.6 percentage point advantage over Senator John Kerry, just a slight understatement of Bush's actual margin of victory (2.4 percentage points). Of the major news organizations, only one had Kerry ahead, and that was by less than the poll's margin of sampling error.

 BushKerryDiffField Period
Actual 2004 Presidential Vote50.748.32.4 
Average of 7 polls48.947.3+1.6 
Pew5148+3Oct 27-30
Newsweek5044+6Oct 27-29
ABC/Washington Post4948+1Oct 28-31
Gallup/CNN/USAToday49490Oct 29-31
NBC/Wall Street Journal4847+1Oct 29-31
CBS/New York Times4947+2Oct 29-Nov 1
FOX4648-2Oct 30-31

The National Council for Public Polls provides an analysis of presidential election polling accuracy from 1936 to the present.

The good track record of final pre-election polls does not mean that all pre-election polls are reliable. Polls conducted early in an election season should be taken as snapshots in time, and obviously cannot capture the impact of the campaign and its events to come. This commentary takes a look at presidential election polls conducted well in advance of the election and attempts to gauge how predictive they are:

These commentaries provide a few tips to help in reading polls and deciding how much weight to give them:

National polls sometimes attempt to gauge how voters will vote in elections for the U.S. House of Representatives. Of course, there is no national election for the House; instead there are elections in each of the 435 congressional districts. But pollsters have found that the so-called "generic ballot test," which asks whether respondents intend to vote for the Republican or the Democratic candidate in their local race for the House, can provide an accurate estimate of the vote on which projections about party gains and losses in seats can be based. The following commentaries and analyses illustrate the use of the generic ballot test and some of the issues involved in using it:

Back to top

Additional topics on election polling

A number of Pew reports and commentaries provide additional examples of how election polls are conducted and the uses to which they are put. Reports in the following section illustrate in-depth pre-election polling where the goal is to analyze and classify voters, and to track changes in fundamental political attitudes and values over time:

In-depth voter analysis

Presidential election primary state polling

Primary elections present special challenges. These reports are based on polls in early primary states, or describe the issues involved in polling these contests.

The Pew Research Center has regularly polled voters after the election to gather reactions to the campaign and the outcome of the election. Most of these involve re-interviews with respondents from pre-election polls. These reports describe the findings from these surveys.

Voter reactions in the aftermath of the presidential election

Back to top