The number of surveys being conducted over the internet has increased dramatically in the last 10 years, driven by a dramatic rise in internet penetration and the relatively low cost of conducting web surveys in comparison with other methods. Web surveys have a number of advantages over other modes of interview. They are convenient for respondents to take on their own time and at their own pace. The lack of an interviewer means web surveys suffer from less social desirability bias than interviewer-administered modes. And Web surveys also allow researchers to use a host of multimedia elements, such as having respondents view videos or listen to audio clips, which are not available to other survey modes.
Although more surveys are being conducted via the Web, internet surveys are not without their drawbacks. Surveys of the general population that rely only on the internet can be subject to significant biases resulting from undercoverage and nonresponse. Not everyone in the U.S. has access to the internet and there are significant demographic differences between those who do have access and those who do not. People with lower incomes, less education, living in rural areas or age 65 and older are underrepresented among internet users and those with high-speed internet access (see the Pew Research Center’s Internet and American Life Project for the latest trends).
There also is no systematic way to collect a traditional probability sample of the general population using the internet. There is no national list of email addresses from which people could be sampled, and there is no standard convention for email addresses, as there is for phone numbers, that would allow random sampling. Internet surveys of the general public must thus first contact people by another method, such as through the mail or by phone, and ask them to complete the survey online.
Because of these limitations, researchers use two main strategies for surveying the general population using the internet. One strategy is to randomly sample and contact people using another mode (mail, telephone or face-to-face) and ask them to complete a survey on the web. Some of the surveys may allow respondents to complete the survey by a variety of modes and therefore potentially avoid the undercoverage problem created by the fact that not everyone has access to the web. This method is used for one-time surveys and for creating survey panels where all or a portion of the panelists take surveys via the web (such as the GfK KnowledgePanel and more recently the Pew Research Center’s American Trends Panel). Contacting respondents using probability-based sampling via another mode allows surveyors to estimate a margin of error for the survey (see Why probability sampling for more information).
The Pew Research Center has also conducted internet surveys of random samples of elite and special populations, where a list of the population exists and can be used to draw a random sample. Then, the sampled persons are asked to complete the survey online or by other modes. For example, see the scientist survey reported in Public Praises Science; Scientists Fault Public, Media.
Another internet survey strategy relies on convenience samples of internet users. Researchers use one-time surveys that invite participation from whoever sees the survey invitation online, or rely on panels of respondents who opt-in or volunteer to participate in the panel. These surveys are subject to the same limitations facing other surveys using nonprobability-based samples: the relationship between the sample and the population is unknown so there is no theoretical basis for computing or reporting a margin of sampling error and thus for estimating how representative the sample is of the population as a whole. (also see the American Association for Public Opinion Research’s (AAPOR) Non-Probability Sampling Task Force Report and the AAPOR report on Opt-In Surveys and Margin of Error). Many organizations are now experimenting with non-probability sampling in hopes of overcoming some of the traditional limitations these methods have faced. One example of this is sample matching, where a non-probability sample is drawn with similar characteristics to a target probability-based sample and the former uses the selection probabilities of the latter to weight the final data. Another example is sample blending whereby probability-based samples are combined with non-probability samples using specialized weighting techniques to blend the two. Here at the Pew Research Center we are closely following experiments with these methodologies, and conducting some of our own, to better understand the strengths and weaknesses of varying approaches.
- Public Sees U.S. Power Declining as Support for Global Engagement Slips December 3, 2013
- What the Public Knows – In Words, Pictures, Maps and Graphs September 5, 2013
- A Survey of LGBT Americans June 13, 2013
- Religion in Prisons – A 50-State Survey of Prison Chaplains March 22, 2012
- The Tablet Revolution October 25, 2011
- Global Survey of Evangelical Protestant Leaders June 22, 2011
- U.S. Seen as Less Important, China as More Powerful December 3, 2009
- Public Praises Science; Scientists Fault Public, Media July 9, 2009