May 15, 2006

The Cell Phone Challenge to Survey Research

National Polls Not Undermined by Growing Cell-Only Population

Summary of Findings

A growing number of Americans rely solely on a cell phone for their telephone service, and many more are considering giving up their landline phones. This trend presents a challenge to public opinion polling, which typically relies on a random sample of the population of landline subscribers. A new study of the issue finds that cell-only Americans — an estimated 7%-9% of the general public — are significantly different in many ways from those reachable on a landline. They are younger, less affluent, less likely to be married or to own their home, and more liberal on many political questions.

Yet despite these differences, the absence of this group from traditional telephone surveys has only a minimal impact on the results. Specifically, the study shows that including cell-only respondents with those interviewed from a standard landline sample, and weighting the resulting combined sample to the full U.S. public demographically, changes the overall results of the poll by no more than one percentage point on any of nine key political questions included in the study.

Estimates of the respondents’ likely congressional vote this fall, approval of President Bush, opinion about the decision to go to war in Iraq, and other important social and political measures are unaffected when cell-only respondents are blended into the sample. The relatively small size of the cell-only group, along with the demographic weighting performed when it is combined with the landline sample, accounts for the minimal change in the overall findings.
This research effort was undertaken by the Pew Research Center, in conjunction with the Associated Press and AOL, to assess the challenge posed by cell phones to random digit dial surveys. The project entailed a survey of 1,503 U.S. adults, with 752 interviewed in a conventional landline sample and 751 interviewed on their cell phones, using a sample drawn from a nationally representative cell telephone number database. The interviews were conducted March 8-28, 2006 and averaged about 11 minutes in length. Among those interviewed on their cell phones, 200 (27%) said that their cell phone was their only phone. Details about the survey, including response rates, costs, and other issues, are discussed in the body of the report below.

The Cell Phone Challenge

The number of people who have given up their landline telephones and rely solely on a cell phone has been increasing, both in the U.S. and internationally, for several years.

According to the U.S. Consumer Expenditure Survey, the percentage of households paying a cell phone bill but not a landline bill rose from 0.4% in 2000 to 7.8% in the first quarter of 2005.

The National Health Interview Survey estimated that, in the second half of 2005, 7.8% of adults lived in households with only a cell phone. And in the 2004 exit poll by the National Election Pool, 7.1% of voters said they relied solely on cell phones.

As the cell-only population has grown, telephone surveys by Pew and other organizations that rely on landline samples have experienced a sharp decline in the percentage of younger respondents interviewed in their samples. In Pew Research Center surveys over the past five years, the average percentage of those ages 18-34 in unweighted samples declined from 31% in 2000 to 20% through March 2006 (the population parameter was essentially unchanged through this period). This decline is consistent with the fact that the cell-only population is heavily tilted toward young people.

Profile of Cell-Only Respondents

Nearly half of the cell-only respondents in the survey (48%) are under age 30. This compares with just 14% in the landline sample (people reached on a landline) and 21% in the population as a whole, according to government statistics. Other characteristics associated with age are also distinctive in the cell-only population. Nearly three-in-ten (29%) cell-only respondents are married, compared with 57% in the landline sample. And only 24% say they own their own home; in the landline sample, 71% do so. The cell-only population also includes a higher proportion of minorities, especially Hispanics (14% vs. 6% among landline users).

The landline sample includes a higher proportion of college graduates than does the cell phone-only group (36% vs. 28%). But more cell-only users say they have some college experience compared with people who have landlines (by 33% vs. 24%); this may reflect the heavy reliance on cell phones among those currently attending college. The cell-only group also is significantly less affluent — more than half (53%) have annual family incomes of under $30,000, compared with just one-quarter (25%) among the landline sample.

Young Cell vs. Landline Users

Young people who rely exclusively on cell phones also are very different — in their lifestyles and family circumstances — from their landline counterparts of similar age. Far fewer cell-only people under age 30 are married, have children, or are homeowners when compared with landline users in this age category. Related to these factors, young cell-only respondents have significantly lower family incomes than young people in the landline sample.

But young cell-only users and landline users do not differ widely in their political attitudes and partisan affiliation. It is true that the cell-only young respondents are more likely to approve of Bush’s performance in office than are under-30 landline respondents (35% vs. 22%). On most other issues, however, they are more liberal and Democratic than their landline counterparts, though most of the differences do not achieve statistical significance. The modest nature of all of these differences suggests that young people — whether cell-only or not — are more similar than different politically.

Seniors Stick With Landlines

According to data collected by the National Center for Heath Statistics, 53% of Americans use both a landline and a cell phone; 37% have only a landline; and 8% rely only on a cell phone.

Like the cell-only population, Americans who rely solely on a landline are distinctive demographically. Fully 41% are ages 65 and older, compared with 16% of the general public. The landline-only group includes a greater proportion of whites than the general public (82% vs. 73%).

Among dual phone users, there are clear differences between those reached on a cell phone and those contacted on a landline. People who were interviewed on a cell phone are somewhat younger (24% under age 30 vs. 15% among those reached on a landline), more likely to be Hispanic (9% vs. 5%), and slightly more likely to have a child under 18 in the household (43% vs. 35%).
Politically, the landline-only and cell-only groups stand out as more Democratic — both in their congressional vote intention and party affiliation — than do those who have both types of phone service. Yet there are only modest differences in approval of President Bush among these four groups.

More striking is the wide divide in views about gay marriage. About half of the cell-only population (51%) favors allowing gay marriage, compared with 39% of the dual phone users and just a third of those who have only a landline phone (33%).

This difference mostly reflects the age patterns of these samples. Pew surveys have consistently found that young people — who make up about half of the cell-only population — are more supportive of gay marriage than are older Americans.

And Pew surveys show that people ages 65 and older, who make up a disproportionate share of the landline-only group, are the most opposed to gay marriage.

Patterns of Cell Phone Use

As might be expected, a solid majority of respondents in the cell phone sample who also have a landline (62%) say that they make more calls on their cell; nearly half (47%) say they make a lot more phone calls on their cell phone. Dual phone owners from the landline sample use landlines only somewhat more frequently than their cell phones; about half (48%) report making more of their calls on their landline while 42% say they make more calls on their cell phone.

Fully 91% of all respondents in the cell sample keep their cell turned on always or most of the time, compared with 73% of cell owners from the landline sample. A small but notable segment (12%) of cell owners from the landline sample say they rarely turn their cell on or do so only to make a call. Hardly anyone from the cell sample (2%) reported having their cell on this infrequently.

Consequently, heavy users of cell phones are more easily reached and interviewed on their cell phones than are lighter users, resulting in a potential bias on some types of measures. One illustration of this is the fact that 27% of respondents in the cell sample identified themselves as cell-only. But U.S. government estimates indicate that only about 13%-15% of cell owners (approximately 7%-9% of the general public) are cell-only.

People in the cell sample use more cell phone features and options than do cell owners from the landline sample. More people in the cell sample say they use a cell to send and receive text messages (45% cell sample vs. 30% landline sample), take still pictures (39% vs. 22%), and surf the web (18% vs. 13%). Three-quarters of those in the cell sample (75%) have personalized their cell phone by changing the wallpaper or ring tone, compared with 59% of cell owners in the landline sample.

Most people in both samples use only one cell phone, and most do not share their cell phone with others. About one-in-five (19%) of those reached in the cell sample say they regularly use more than one cell phone; the comparable number in the landline sample was 14%. And in each sample, 16% said that another adult regularly answers their cell phone.

Dropping Your Landline?

About a quarter of landline users (23%) say they are very (8%) or somewhat likely (15%) to stop using their landline and switch instead to using only a cell phone. A narrow majority (55%) says they are not likely at all to give up their landline in favor of a cell phone. As may be expected, far more young people than older Americans say they are at least somewhat likely to abandon their landline; 40% of those under age 30 say this compared with 19% of current landline users ages 30 and older.

Implications for Tech-Focused Surveys

Asked about their general opinion of computers and technology, cell-only respondents are much more positive toward computers and technology than are landline-only respondents, and somewhat more positive than other cell phone users who are accessible on a landline.

But there is little difference between the cell-only respondents and cell phone users reached on a landline in their use of the internet and their access to broadband. The only significant difference in internet use is how the respondent gets service: cell-only users are less likely than others to use DSL or a dial-up line.

Challenges of Cell Phone Interviews

In addition to providing a look at the cell-only population, this study was designed to assess the feasibility of conducting a telephone survey in a cell phone sampling frame. The conclusion is that such surveys are feasible, but they are more difficult and expensive to conduct than landline surveys.

Because most cell phone users have to pay for incoming calls (or use pre-paid minutes for them), a $10 incentive was offered only to respondents in the cell phone sample. Despite this inducement, gaining cooperation from people on cell phones was notably more difficult than for those on a landline phone.

The response rate was 30% in the landline frame but only 20% in the cell phone frame. It was actually easier to make contact with a respondent through the cell phone frame (the contact rate was 76% in the cell frame vs. 68% in the landline frame). But that greater accessibility did not translate into more cooperation. Half of the people reached in the landline sample (50%) cooperated with the interview, compared with roughly a quarter (28%) of those reached in the cell phone sample.

Aside from difficulties in gaining cooperation, the process of sampling cell phone numbers proved to be reasonably efficient. More of the cell phone numbers (59%) were connected to eligible respondents than were numbers in the landline sample (43%).
Interviewing people on cell phones presents several challenges that require new procedures and have implications for overall costs. Among the most important of these is the fact that federal law prohibits the use of automated dialing devices when calling cell phones; thus each number in the cell phone sample had to be dialed manually.

The $10 incentive offer incurred additional costs. An overwhelming majority of cell phone respondents who completed the interview (86%) accepted this offer and provided a mailing address to which the incentive was sent. In addition to the money paid to the respondent, the use of an incentive also incurs additional administrative work that raises the cost of the survey.

Results from the study suggest that interviews on a cell phone take about the same amount of time to complete as interviews on a landline phone. The same questionnaire was administered to both samples, and the median length was 11 minutes (mean = 11.8) for the cell phone sample and 10 minutes (mean = 10.2) for the landline respondents who reported owning a cell. Most of the small difference in average length between the two sampling frames is likely due to the extra time spent by the cell sample respondents in providing a mailing address for mailing the $10 incentive.

Cell phones tend to be personal devices, and many adolescents and younger children have their own phone. One consequence of this is that more people reached in the cell frame turned out to be ineligible because of their age than is typically the case in a household-based landline sample. Of people contacted in the cell phone frame, 45 cases were dropped from the study because the respondent was under 18. In the landline sample, only 6 cases were dropped because the sampled telephones were used exclusively by children.

Because people may not be accustomed to speaking with an unknown caller on their cell phone, two other modifications in Pew’s regular protocol were used. The survey introduction included the acknowledgement that the respondent had been reached on a cell phone, and an immediate question as to whether it was safe to do an interview at that time. If the interviewer reached voice mail, a message was left explaining the purpose of the survey along with a toll-free number for the respondent to call and complete the interview at their convenience. Approximately 20 of the 751 respondents in the cell phone survey completed the interview in this way.

Data collection costs (apart from overall study design, programming, and analysis costs) were slightly more than twice as high for the cell phone sample as for the landline sample. Adding in the costs of administering and paying the $10 incentive, the total costs of interviewing the cell phone sample were approximately 2.4 times the cost of the landline sample.

Cell Phone Respondents Not More Distracted

According to the interviewers working on the survey, the cell phone respondents were as focused and cooperative as those reached on a landline telephone. The vast majority (93%) of those surveyed on their cell phone demonstrated good or very good cooperation. This compares with 79% of those from the landline sample.

In addition to being cooperative, the cell phone respondents were also relatively focused on the survey task. In each sample only about 10% seemed somewhat or very distracted (8% cell phone vs. 11% landline, respectively), according to interviewers who conducted the survey. Likewise, when interviewers recorded whether it sounded as though the respondent had been doing another activity during the survey, results were quite similar for the two samples. About one-in-five of those from the cell phone sample (20%) and the landline sample (17%) were preparing a meal, watching television, shopping, exchanging comments with another person, or engaged in another activity.

Demographics of the Complete Cell and Landline Samples

People reached in the cell sample have a considerably different demographic profile from those reached in the landline sample, especially with respect to sex, race, age, education, and home ownership. On many variables, the landline sample was closer to the population parameter than the cell sample, though on some measures the cell sample picks up certain kinds of respondents that the landline samples under-represent.

A majority of those interviewed in the cell sample (55%) were men. Most landline surveys interview too few men, and require quotas or other techniques to obtain the proper proportion of men vs. women. As noted earlier, most landline surveys have too few young people in their samples (7% under age 25, vs. 13% in the population), but the cell phone sample had too many (21%). Conversely, the landline sample has too many older respondents (23% are 65 and older, vs. 16% in the population), while the cell phone sample had too few (just 8%).

The cell sample also proved to be effective at reaching African Americans, as 13% of the sample identified themselves as black. Landline samples often fall short of the population parameter (11%), though the landline sample in this project was very close (10%).

Although the survey was conducted only in English, fully 11% of the cell phone sample was Hispanic compared to just 6% of the landline frame sample. Hispanics constitute approximately 12% of the U.S. population.

Both samples include too many people with college experience, compared with the U.S. population. U.S. government figures show that 26% of the public has at least a four-year college degree, compared with 36% in the landline sample and 35% in the cell sample.

The people reached through these two samples differ in other ways as well. Over seven-in-ten (71%) of those interviewed from the landline sample report being a homeowner compared with closer to half (57%) of those reached on a cell phone. (The U.S. government estimates that 69% of the public are homeowners.)

In addition, fewer of the landline sample respondents were parents of children under 18 — a finding that likely reflects the presence of more young adults in the cell phone sample. At the same time, however, the samples were fairly similar in the percentage of respondents who were married (57% in the landline sample vs. 52% in cell sample — compared with 59% from U.S. government data), though the mix of unmarried people is very different in the two samples. One-third (33%) of the cell sample reported having never been married, compared with just 18% in the landline sample; according to the government, 25% of the adult population has never been married.

Cite this publication: “The Cell Phone Challenge to Survey Research.” Pew Research Center, Washington, D.C. (May 15, 2006) http://www.people-press.org/2006/05/15/the-cell-phone-challenge-to-survey-research/, accessed on July 23, 2014.