How Much Does Adding Cell Phone Sample Reduce Demographic Biases? d

How Much Does Adding Cell Phone
Sample Reduce Demographic Biases?
A Case Study
d
2011 IFD&TC
Scottsdale, AZ
Presented by Brian Harnisch
Wyoming Survey & Analysis Center (WYSAC), University of Wyoming
Outline
 Questions of interest
 Change in prevalence of cell phone-only households
 Demographic characteristics of adults living in CPO
HHs
 Wyoming Crime Victimization Survey, 2011
 Moving forward
Questions of interest…
 How much does adding
g cell p
phone sample
p to g
general
population surveys reduce the potential for
demographic biases?
 Can adding cell phone sample help avoid the need
ffor h
heavy weighting,
i hti
quota
t sampling,
li
etc.,
t while
hil
remaining a cost effective option?
The Changing Landscape
 National Health Interview Survey (NHIS)
 Estimates of the prevalence of cell phone-only households
produced and released by the National Center for Health
Statistics (NCHS) and the Center for Disease Control (CDC)
Wireless Substitution: State-level Estimates from the NHIS
(Blumberg & Luke, 2011)
 Wireless Subsitution: Early Release of Estimates from the NHIS
(Blumberg & Luke, 2010)

The Changing Landscape
 More than a quarter of American households
(26.6%) are now cell phone-only households

8x increase in the prevalence of cell phone-only HHs in the last
6 years
 At the same time, the number of landline-only
homes continues to decrease
The Changing Landscape
80.0%
70.0%
60.0%
58.1%
50.0%
Landline households with a wireless
telephone (Dual)
40.0%
Landline households without a wireless
telephone (LLO)
30 0%
30.0%
26.6%
Wireless-only households (CFO)
20.0%
10.0%
12.9%
0.0%
(Blumberg & Luke, 2011)
The Changing Landscape
Significant State
diff
differences…
Adults living in CFO HHs,
b state
by
 35.2% in AR
 22.3% in WY
 12.8% in RI
 Compared
p
to
26.6% nationally
(Figure via Blumberg & Luke, 2010)
Don’t forget about cell phone-mostly households!
 Add in the households with LLs that receive almost
all of their calls on a cell and some states now have
over 50% of all adults that are largely reachable only
b cell
by
ll phone
h



52.8% CPM adults in Texas
35 4% CPM adults in Wyoming
35.4%
24.9% CPM adults in South Dakota
General demographics of CPO HHs
5
51.3%
3 of all adults aged
g 25-29
5 9 lived in CPO HHs
 The same group accounts for 39.8% of all CPO adults
 26.2% of men lived in CPO, 23.7% of women
 But, the gender distribution is split evenly when looking only
at cell phone only HH adults
 Adults living in poverty more likely to be CPO
 Hispanic adults more likely to be living in CPO HHs
than non
non-Hispanic
Hispanic
Wyoming Crime Victimization
Survey, 2011
Statewide telephone survey
• Funded through the BJS State Justice Statistics
(SJS) project.
• Simple within-household random adult selection
for LL sample, no selection for cell sample
• 70% Landline sample (listed)
• 30% Cell phone sample
•
Why listed?
 In a perfect world, we would prefer RDD and Cell
 Proposed a listed LL and cell phone dual-frame instead of a
single frame RDD sample in an effort to meet budget
requirements.
 Cost/benefit
/
trade-off
Raw Completion Rates
Wave
Completion Rate
Calendar Days in field
1
25.20%
29
2
23 80%
23.80%
25
3
22.60%
18
4
20.80%
15
5
20 40%
20.40%
10
6
19.60%
8
7
17.00%
5
Wtd Avg
22 60%
22.60%
Cell 8
5.60%
30
Cell 9
5.90%
22
Cell 10
Cell 10
5.80%
11
Wtd Avg
5.70%
Completions
 1,451
,45 total completions
p
 171 from cell sample (11.8%), 1,280 from LL (88.2%)
 196
9 completed
p
on cell p
phone ((13.4%)
34 )
 2.1% of completions from LL sample on cell phone
 97.7% of completions from cell sample on cell phone
 Duall users not screened
d out
 17 minute average length
 No
N iincentives
ti
offered
ff d
 Typically very difficult to get completions male
and/or from younger respondents in WY
Completions totals by telephone status
Telephone
Status
Wyoming Wyoming
Estimates (NHIS)
LL Sample
Cell Sample
Survey Total
CPO
22 3%
22.3%
0 4%
0.4%
50 3%
50.3%
6 3%
6.3%
CPM
13.1%
12.7%
17.5%
13.2%
Dual‐use
22.0%
47.0%
25.1%
44.4%
LLM
5.9%
28.0%
7.0%
25.5%
LLO
35.5%
12.0%
‐‐
10.6%
98.8%
100.0%
100.0%
100.0%
Results by telephone status
Telephone
Status
Wyoming Wyoming
Estimates (NHIS)
LL Sample
Cell Sample
Survey Total
CPO
22 3%
22.3%
0 4%
0.4%
50 3%
50.3%
6 3%
6.3%
CPM
13.1%
12.7%
17.5%
13.2%
Dual‐use
22.0%
47.0%
25.1%
44.4%
LLM
5.9%
28.0%
7.0%
25.5%
LLO
35.5%
12.0%
‐‐
10.6%
98.8%
100.0%
100.0%
100.0%
Gender results
Gender
WY Adult Pop. Est.
American
CPO HHs
LL Sample
Cell Sample
Survey Total
Male
50.7%
50.9%
44.0%
59.0%
45.8%
Female
49.3%
49.1%
56.0%
41.0%
54.2%
100.0%
100.0%
100.0%
100.0%
100.0%
Age group distribution results
Age
WY Adult Pop. Est.
American
CPO HHs
LL Sample
Cell Sample
Survey Total
18‐24 years
14.5%
20.7%
1.3%
12.1%
2.5%
25‐34 years
18.2%
33.0%
5.7%
20.6%
7.4%
35‐44 years
y
15.6%
18.9%
9.6%
24.2%
11.3%
45‐64 years
35.6%
23.7%
44.8%
30.9%
43.2%
65 years and over
16.2%
3.7%
38.6%
12.1%
35.5%
100 0%
100.0%
100 0%
100.0%
100 0%
100.0%
100 0%
100.0%
100 0%
100.0%
Educational attainment results
Educational Attainment
WY Adult Pop. American CPO LL Sample
Est. HHs
Cell Sample
Survey Total
High school graduate or less
42.0%
43.3%
29.8%
34.1%
30.3%
Some college/AA/Tech
37.3%
32.0%
35.7%
32.4%
35.3%
College graduate/Graduate school
20.8%
24.7%
34.5%
33.5%
34.4%
100.0%
100.0%
100.0%
100.0%
100.0%
Hispanic or Latino
Hispanic or Latino, any race(s)
spa c o at o, a y ace(s)
WY Adult Pop. American CPO LL Sample
Sa p e
E
Est.
HH
HHs
Cell Sample
Ce
Sa p e
Survey T l
Total
Yes
7.5%
19.4%
3.7%
7.6%
4.2%
No
92.5%
80.6%
96.3%
92.4%
95.8%
100.0%
100.0%
100.0%
100.0%
100.0%
Adults living with children under age 18
Living with children under 18
WY Adult Pop. American CPO LL Sample
Est
Est. HHs
Cell Sample
Survey Total
Yes
31.5%
40.9%
21.7%
43.9%
24.3%
No
68.5%
59.1%
78.3%
56.1%
75.7%
100.0%
100.0%
100.0%
100.0%
100.0%
A few by telephone status - Age
Age
WY Adult Pop. Est. American CPO HHs
CPO/CPM
Dual‐use HH
LLO/LLM
18‐24
18
24 years
years
14.5%
20.7%
8.0%
1.3%
1.2%
25‐34 years
18.2%
33.0%
20.4%
6.3%
1.8%
35‐44 years
15.6%
18.9%
19.7%
13.2%
4.6%
45‐64 years
35.6%
23.7%
39.8%
53.0%
32.7%
65 years and over
16.2%
3.7%
12.0%
26.2%
59.7%
100.0%
100.0%
100.0%
100.0%
100.0%
A few by telephone status - Gender
Gender
Male
Female
WY Adult Pop. WY
Adult Pop American CPO American CPO
Est.
HHs
50.7%
50.9%
CPO
CPM
Dual‐use
LLM
LLO
46.2%
54.7%
45.7%
42.8%
42.2%
57.2%
%
57.8%
%
49.3%
%
49.1%
%
53.8%
%
45.3%
%
54.3%
%
100.0%
100.0%
100.0%
100.0%
100.0% 100.0% 100.0%
Other differences – our respondents
 Respondents
p
from our cell p
phone sample
p more likelyy
to have any internet connection (88%)
 More likely to have a high-speed internet connection
(76.1%)
 Of those with high-speed connections, respondents
f
from
th
the cell
ll sample
l were nott more likely
lik l tto prefer
f
to have taken the survey over the internet than those
from the listed LL sample.
sample
What does this all mean?
 So,, how much cell phone
p
sample
p would it have taken
to match the actual demographic distributions in
WY?
 Is it worth trying to reduce the potential need for
heavy weighting
h
i hti b
by adding
ddi more and
d more cell
ll
phone sample up front?
Eliminate simple weighting?
 Would take 6x more cell completions
p
to hit true WY
adult gender distribution without weighting.

Initial cell sample increases to over 18k necessary from the
original 3k,
3k holding LL sample constant at 7k.
7k
 Would take approximately 36x more initial cell
phone sample to bring to hit true WY adult age
distribution without weighting.

Now 108k cell sample required from original 3k, holding LL
sample constant at 7k.
Conclusions
 Understand the state-level CPO estimates - and plan
p
accordingly.
 Who is the population of interest?

Cost/benefit for client to pay for a population subgroup of little
interest?
 Non
Non-general
general pop
pop. surveys
 HHs with teens? 25-29 year olds? Hispanic/Latino? Renting
vs. owning? Unrelated roommates? Etc.
 Better position yourself for acceptable weighting
procedures…..
Conclusions
Weighting of dual-frame
dual frame telephone surveys will continue to
be a hot topic. Try and anticipate demographic
shortcomings and supplement with additional cell phone
sample - leading to less aggressive weighting procedures
and more reliable results.
Brian Harnisch
Assistant Research Scientist
W
Wyoming
i Survey
S
& Analysis
A l i C
Center
t
University of Wyoming
[email protected]
307 766 6103
307-766-6103
Lookout Lake, Near Laramie, WY
Thank
h k You!!
© Brett
B
Deacon
D
References




Blumberg SJ, Luke JV, Ganesh N, et al. Wireless substitution: State-level estimates from the National Health Interview
Survey, January 2007–June 2010. National health statistics reports; no 39. Hyattsville, MD: National Center for Health
Statistics. 2011.
Blumberg SJ, Luke JV. Wireless substitution: Early release of estimates from the National Health Interview Survey,
January–June 2010. National Center for Health Statistics. December 2010. Available from:
http://www.cdc.gov/nchs/nhis.htm.
2010 Census Redistricting Data (Public Law 94-171) Summary File, Tables P1, P2, P3, P4, H1.
2005-2009 American Community Survey 5-Year Estimates