Articles Measuring vaccine confidence

Articles
Measuring vaccine confidence: analysis of data obtained by a
media surveillance system used to analyse public concerns
about vaccines
Heidi J Larson, David M D Smith, Pauline Paterson, Melissa Cumming, Elisabeth Eckersberger, Clark C Freifeld, Isaac Ghinai, Caitlin Jarrett,
Louisa Paushter, John S Brownstein, Lawrence C Madoff
Summary
Background The intensity, spread, and effects of public opinion about vaccines are growing as new modes of
communication speed up information sharing, contributing to vaccine hesitancy, refusals, and disease outbreaks. We
aimed to develop a new application of existing surveillance systems to detect and characterise early signs of vaccine
issues. We also aimed to develop a typology of concerns and a way to assess the priority of each concern.
Methods Following preliminary research by The Vaccine Confidence Project, media reports (eg, online articles, blogs,
government reports) were obtained using the HealthMap automated data collection system, adapted to monitor
online reports about vaccines, vaccination programmes, and vaccine-preventable diseases. Any reports that did not
meet the inclusion criteria—any reference to a human vaccine or vaccination campaign or programme that was
accessible online—were removed from analysis. Reports were manually analysed for content and categorised by
concerns, vaccine, disease, location, and source of report, and overall positive or negative sentiment towards vaccines.
They were then given a priority level depending on the seriousness of the reported event and time of event occurrence.
We used descriptive statistics to analyse the data collected during a period of 1 year, after refinements to the search
terms and processes had been made.
Findings We analysed data from 10 380 reports (from 144 countries) obtained between May 1, 2011, and April 30, 2012.
7171 (69%) contained positive or neutral content and 3209 (31%) contained negative content. Of the negative reports,
1977 (24%) were associated with impacts on vaccine programmes and disease outbreaks; 1726 (21%) with beliefs,
awareness, and perceptions; 1371 (16%) with vaccine safety; and 1336 (16%) with vaccine delivery programmes. We
were able to disaggregate the data by country and vaccine type, and monitor evolution of events over time and location
in specific regions where vaccine concerns were high.
Interpretation Real-time monitoring and analysis of vaccine concerns over time and location could help immunisation
programmes to tailor more effective and timely strategies to address specific public concerns.
Funding Bill & Melinda Gates Foundation.
Introduction
Although immunisation has successfully reduced the
global burden of illness and death, a range of concerns
have converged to affect public confidence in vaccines.
When confidence in vaccination breaks down, hesitancy
can lead to delays and refusal, disrupting research and
delivery programmes, and sometimes leading to disease
outbreaks.1,2
The most serious example is the 2003–04 northern
Nigeria boycott of polio vaccination, which set the global
polio eradication initiative back sub­
stantially, cost
millions of US dollars, and led to a resurgence of the
disease.3–5 The boycott, driven mainly by politics and
unfounded fears of vaccine-induced sterilisation,
contributed to reinfection in 20 previously polio-free
countries, reaching as far as Indonesia. The fundamental
breakdown in public trust still affects polio eradication
efforts in Nigeria.6,7
Another example includes fear and refusal of the
measles, mumps, and rubella (MMR) vaccine, initially
ignited by Andrew Wakefield and colleagues’ nowretracted8,9 1998 study10 suggesting an association
between the vaccine and autism. The study’s findings
were amplified by the media, Wakefield’s own public
appearances advocating his research, and networks of
parents who felt that Wakefield finally had an answer to
the cause of their child’s autism. The effect of the media
and personal attention to the since-debunked research,
officially retracted 4 years after it was published and
deemed both unethical and fraudulent,11 resulted in a
substantial decline in MMR vaccine coverage12 that has
still not returned to the high of 92% before 1998.13
Meanwhile, MMR vaccine anxieties continue to circulate
worldwide. In 2009–10, high non-acceptance rates of the
pandemic influenza A H1N1 vaccine, including among
health-care professionals, were another example of the
potential effects of public distrust in vaccines.14
In such cases, the time between the prompting events
and their effect on public health outcomes is important—
eg, months or years can elapse, with extended periods of
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70108-7
Published Online
May 13, 2013
http://dx.doi.org/10.1016/
S1473-3099(13)70108-7
See Online/Comment
http://dx.doi.org/10.1016/
S1473-3099(13)70131-2
London School of Hygiene &
Tropical Medicine, London, UK
(H J Larson PhD,
D M D Smith DPhil,
P Paterson PhD,
E Eckersberger MPA, I Ghinai BSc,
C Jarrett MPH); Chatham House
Centre for Global Health
Security, Royal Institute of
International Affairs, London,
UK (H J Larson); ProMED-mail,
International Society for
Infectious Diseases, Brookline,
MA, USA (M Cumming MS,
L Paushter MPH,
Prof L C Madoff MD);
HealthMap, Boston Children’s
Hospital Informatics Program,
and Division of Emergency
Medicine, Boston Children’s
Hospital, Boston, MA, USA
(C C Freifeld MS,
J S Brownstein PhD);
Department of Biomedical
Engineering, Boston
University, Boston, MA, USA
(C C Freifeld); Department of
Paediatrics, Harvard Medical
School, Boston, MA, USA
(J S Brownstein); and University
of Massachusetts Medical
School, Worcester, MA, USA
(L C Madoff)
Correspondence to:
Dr Heidi J Larson, London School
of Hygiene &Tropical Medicine,
London WC1E 7HT, UK
[email protected]
1
Articles
For Google News see https://
news.google.co.uk/
For Google Blog Search see
http://www.google.com/
blogsearch
For more on Moreover
Technologies see http://www.
moreover.com/
For Google Translate see
http://translate.google.co.uk/
vaccine hesitancy or uncertainty. Various factors amplify
the spread of information, and misinformation, affecting
perceptions and behaviours and creating what Kasperson
and colleagues15 term the “social amplification of risk”, by
which they mean amplification of the spread of
information via social, cultural, and institutional
processes, and amplification of society’s response.
We postulated that media monitoring might provide
important information about perceived problems with
specific vaccines or immunisation programmes that
might take longer to register through official channels.
We therefore aimed to track the emergence and spread,
geographically and temporally, of media and social media
reports on vaccines by developing a new application for
rumour surveillance systems—typically designed to
detect signs of disease out­
breaks16–20—to detect and
characterise early signs of public concern about vaccines.
In doing so, we aimed to characterise and create a
typology of the content and to develop an approach to
assess the priority of each concern for further
investigation and intervention.
In January, 2010, The Project to Monitor Public
Confidence in Immunisation, now called The Vaccine
Confidence Project, of which we are members, was
launched to address three objectives. The first being to
establish a global information surveillance system to
detect emerging public concerns by monitoring media
and social media and building a global key informant
network. Second, to systematically code all reports to
identify positive or neutral versus negative content and
key areas of concern, to develop a typology of concerns.
And third, to develop a diagnostic method to prioritise
which reports need further investigation and intervention
on the basis of patterns of clustering or persistence of
reports, and when viewed against contextual and
historical factors that have contributed to their
amplification. This report focuses mainly on our first two
objectives. Concurrent research and analysis of the third
objective will be reported elsewhere.
Methods
Data collection
Early on in the project, an international advisory group21—
consisting of experts in vaccine-preventable diseases,
vaccine safety, risk and decision science, immunisation
programme management, and public health—identified
and discussed key examples of break­
downs in public
confidence in vaccines or vaccine programmes that
resulted in serious declines in vaccine acceptance.
Detailed retrospective analyses then identified crucial
factors that led to these breakdowns in acceptance, which
were then used as criteria for report categorisation.
Data collection began in April, 2010. Refinements to
search terms, collection, and coding were made
periodically between April, 2010, and April, 2011. Most
reports were collected with the HealthMap automated
data collection system,22 which was adapted to monitor
2
online reports about vaccines, vaccination programmes,
and vaccine-preventable diseases. An example of the
search criteria used is “intitle:vaccine OR intitle:rotavirus
OR intitle:measles”. Reports included online news articles,
blogs, website pages, public service announce­
ments,
government announcements, book reviews, and broadcast
media. The three primary acquisition channels were
Google News, Google Blog Search, and Moreover Public
Health, a global news aggregator service. The Moreover
content was culled from a broad range of RSS (rich site
summary) news feeds by use of health-related keywords
determined by Moreover. Foreign language reports were
translated using Google Translate. The non-English
languages that were translated were Chinese, Czech,
Danish, French, German, Greek, Hindu, Indonesian,
Italian, Japanese, Persian, Polish, Portuguese, Russian,
Spanish, Swedish, Thai, and Vietnamese.
Reports were passed through an exclusion filter to
remove any that were not vaccine-related or were related
only to animal vaccines. Any report that included a
reference to a human vaccine or vaccination campaign or
programme that was accessible online was included in
the analysis, which included articles about vaccinepreventable diseases whenever a vaccine was mentioned.
Reports not meeting these inclusion criteria were
excluded from analysis but logged and archived. The
articles were presented on a web-based user interface
system for rapid review, while the system automatically
forwarded the reports to analysts from the London
School of Hygiene & Tropical Medicine and the Program
for Monitoring Emerging Diseases (ProMED).
Data categorisation
Reports were automatically classified according to the
HealthMap algorithms every hour. Relevant reports were
classified by location and disease and entered into the
project database, with title, date, report source, URL, and
plain text in full automatically added, taking less than
1 min per report. The entry date and the team member
entering the report were also automatically recorded.
Analysts then reviewed and amended the autopopulated
fields, then allocated data categories, report source,
vaccine-preventable disease, vaccine, and vaccine
manufacturer, and disaggregated the automatically
identified locations into three fields: country of origin of
report, countries referred to, and country of origin of any
additional sources referenced. Additionally, a one-line
summary of the report content was created by the
analyst. After populating these fields, an overall
impression of the report tone (either positive or neutral,
or negative), and report priority level (high, medium, or
low) were decided on the basis of agreed criteria.
A report was coded as negative if it contained any
indication of concern about a vaccine or vaccination
programme, such as information about an adverse event
that occurred after immunisation, a vaccine suspension,
or any other factor that has a negative effect on a vaccine
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70108-7
Articles
Google News
Moreover Public Health
Translation of foreign language reports
Exclusion
terms
Application of inclusion and
exclusion criteria
Reject
Archive
Accept
Done every hour (automated)
Takes less than 1 min to complete
Done every day (manual)
Takes about 5 min per report
Populate fields
Date, title, full text, source, disease,
country (origin), country (about), URL
Web-based user interface
Include or exclude (relevance)
Reject
Archive
Accept
LSHTM/ProMED
Verify and amend autopopulated fields
Database
user guide
Data category, tone, vaccine,
manufacturer, summary, source type,
notes
Priority
High
Alert team
Medium or low
Data prioritisation
Report priority was classified as high if the report contained
any reference to a serious adverse event that occurred after
immunisation, such as death or hospitalisation, or
mention of vaccine refusal, sus­
pension, or withdrawal
within the previous 6 months. Priority was classified as
medium if the report did not contain reference to vaccine
refusal, but did contain public questioning or concerns
about vaccines or the immunisation programme occurring
within the previous 6 months. Priority was classified as
low whenever content was deemed positive or neutral in
nature, or when the event in question occurred more than
6 months before the report date (eg, if the report was
discussing the resolution of an older issue). The rationale
for the high, medium, and low coding was to identify
which reports needed priority investigation and their
content to be shared with relevant individuals and
institutions (ie, those that were coded as high priority);
reports coded as medium needed to be monitored closely,
and low-coded reports were archived for future reference
and as background information for changes in concern
level. The decision to code events reported as occurring
within the previous 6 months as either high or medium
priority was to distinguish recent events from articles
referencing historic events that were neither new nor
recurring, but that needed monitoring (medium priority).
Every report classified as high priority was automatically
sent to all team members and discussed at a weekly
teleconference. Interested individuals, Ministries of
Health, WHO, and UNICEF officials were also notified.
Duplicate reports with the same URL were auto­
matically deleted, but those with different URLs were
Google Blog Search
HealthMap
Search terms
Populate fields
programme. A report was coded as positive or neutral if it
contained no indication of public concern about a vaccine
or vaccination programme. A report that discussed an
adverse event that occurred after immunisation could be
coded as positive if, for instance, the article was about the
investigation of a reported adverse event that had been
confirmed to be unrelated to a vaccine. Other positive
reports included topics such as a country announcing the
introduction of a new vaccine or plans for a vaccination
campaign.
We used the definition of adverse events as any
untoward medical occurrence that follows immunisation
that does not necessarily have a causal relation with the
use of the vaccine, as described by the Council for
International Organizations of Medical Sciences and
WHO working group on vaccine pharmacovigilance.23
On average it took 5 min per report for this stage of the
data curation to be completed, although time taken varied
substantially depending on the length, complexity, and
clarity of the article. Within 24 h of receipt, each report was
read and assessed by one of the team of analysts trained
with a standardised database user guide developed by the
research team. Periodic quality assessment was done to
ensure consistency between reviewers (figure 1).
Add to database
Figure 1: Report collection, data entry, and coding in the database
The blue HealthMap section depicts the automated computer processes, and the green LSHTM/ProMed section
depicts the processes that involved data analysts from LSHTM and ProMed. LSHTM=London School of Hygiene &
Tropical Medicine. ProMED=Program for Monitoring Emerging Diseases.
recorded as separate reports, recognising the fact that
replicated reports show the spread of information, and
are therefore important to the analyses.
Statistical analysis
Summary statistics given here comprise simple counts
and percentages. All figures were generated in Matlab
version 8.0.0.783 (R2012b).
Role of the funding source
The sponsor of the study had no role in research design,
data collection, data interpretation, or writing of the
report. The corresponding author had full access to all
the data in the study and had final responsibility for the
decision to submit for publication.
Results
We analysed only the data that were entered into the
database between May 1, 2011, and April 30, 2012 (after all
refinements to the search terms and processes were
made), during which 10 380 reports originating from
144 countries were collected and coded. Of these reports,
7171 (69%) contained positive or neutral content about
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70108-7
3
Articles
See Online for appendix
vaccines and vaccination programmes and 3209 (31%)
contained negative content (figure 2).
More than one data category could be allocated to one
report. Of the 10 380 reports identified, 22 499 data
categories were recorded (14 145 for the 7171 positive or
neutral reports and 8354 for the 3209 negative reports).
During the analysis of the data, related data categories
were clustered into the following groups: beliefs,
awareness, and perceptions (eg, religious beliefs, risk
perceptions); contextual factors (eg, conflict or war);
impacts (eg, vaccine suspension, vaccine refusals); vaccine
delivery programme (eg, mass versus routine campaign,
vaccine supply, costs); vaccine safety (eg, reports or
concerns about adverse events occurring after im­
munisation, additives, or preservatives); vaccine
development and introduction (eg, vaccine research or
new products launched or approved); and recom­
mendations about vaccines, disaggregated according to
source (eg, national immunisation pro­gramme, religious
leader, non-governmental health organisation; appendix).
Of the positive and neutral reports, 4632 (33%) data
categories were associated with vaccine development and
introduction; 4332 (31%) with vaccine delivery pro­
grammes; 2141 (15%) with vaccine recommendations;
1475 (10%) with contextual factors; 970 (7%) with beliefs,
awareness, and perceptions of vaccines; 428 (3%) with
vaccine safety; and 167 (1%) with impacts on vaccine
programmes and disease outbreaks (figure 3). Of the
negative reports, 1977 data categories (24%) were
associated with impacts on vaccine programmes and
disease outbreaks; 1726 (21%) with beliefs, awareness,
and perceptions; 1371 (16%) with vaccine safety;
1336 (16%) with vaccine delivery programmes; 1119 (13%)
with vaccine recommendations; 582 (7%) with contextual
factors; and 243 (3%) with vaccine development and
introduction (figure 3).
Our grouping of data categories included the group
contextual factors, because some contextual factors can
amplify negative sentiments (eg, marginalised popu­
lations, negative high-profile individual) and others
attenuate them (eg, high vaccine acceptance rates,
positive high-profile individual), which is important for
the prioritisation of reported concerns. A more detailed
breakdown of the data categories allocated to the
10 380 reports is shown in the appendix.
We categorised the reported data by vaccine type—
worldwide and in five selected countries (China, Finland,
France, Nigeria, and Pakistan; figure 4). We chose these
countries because they had known or widely reported
issues with specific vaccines before or during the study
period, and we wanted to show that our data collection
was consistent with established concerns. We note that
the dominant vaccines discussed (either negatively or
positively) in the media in the five countries are consistent
with genuine issues that were reported concurrently by
WHO, providing some validation of the reliability of the
100%
Positive or neutral (%)
World
0%
Figure 2: Proportion of vaccine-related reports categorised as positive or neutral, by country
Based on analysis of all 10 380 reports. Of the 9655 reports (93%) that mentioned a country or countries, 11 535 countries were mentioned. Countries about which
there were fewer than ten vaccine-related reports are shaded grey. The world proportion (69%) is shown by the arrow on scale bar. Country border data are from the
Global Administrative Areas database.24
4
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70108-7
Articles
Vaccine development
and introduction
Vaccine delivery
programme
Recommendations
Contextual factors
Beliefs, awareness,
and perceptions
Vaccine safety
Positive or neutral
Negative
Impacts
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Number of associated data categories
Figure 3: Number of data categories allocated to all reports, per data category group
Data is from all 10 380 reports, obtained between May, 2011, and April, 2012, split according to whether the report was classified as positive or neutral, or negative.
Reports could be classified by more than one data category.
Varicella
Meningitis
H5N1
ella
Rub
infl Oth
uen er
za
s
Hep
asle
Me
lla
be
ps
sle
s
ea
asl
s
Ru
Me
Polio
Meningitis
er
es
Oth
Hepatitis B
Polio
es
asl
Me
V
World
r
Othe
HP
Mum
nu
ta
Sm
a
Per llpox
tus
Hepati sis
tis B
Polio
Pertussis
er
H1
Oth
N
Dip 1
hth
e
Menin ria
gitis
Tetanus
is B
Hepatit
er a
Oth uenz
infl
Polio
M
um
ps
atit
is B
la
H1N1
Other
influenza
Te
infl Othe
uen r
za
ies
s
osi
cul
Polio
Measles
Measles
ber
r
he
Ot
s
is
H1N1
Polio
Pakistan
China
mp
los
Tu
rcu
Polio
Nigeria
Mu
bel
Ru
be
eu
mo
c
nus occa
l
Meningitis
B
r
ssis
Teta
atitis
Pne
les
as
Pertu
Me
World
sA
Oth
HPV
Mumps
Hep
e
Oth
um
o
c
Measl
es occal
r
Yellow
fev
Malaria er
Tetanus
sles is
Mea
git
nin
Me
Polio
iti
at
p
He
Finland
er
Tu
Other
influenza
N1
H1
HPV
Ot
s
asle
Me
he
o
Poli
Teta
nus
Rotavirus
Pneumococcal
JE
N1
r
he
Ot
HPV
HP
H3N2
H1
r
he
Ot
Rotav
irus
H1N1
Malaria
lla
Rube itis B
pat heria
He
t
ph
Di
Pn
HPV
sA
France
Rubella
l
occa
moc
Pneu
V
HP 1
N
H5
1
Other
influenza
les
H5N
titi
B
China
Rotavirus
as
pa
Meningococcal
is
ingit
Men itis B
t
pa
e
H
o
Me
He
er
h
Ot
Poli
V
Finland
M
Polio
France
Rab
A
Nigeria
Pakistan
Figure 4: Proportionate number of times specific vaccine types were covered positively and negatively in the media, worldwide and in five selected countries
Vaccine types were reported positively (A) 9157 times worldwide, 45 times in France, 22 times in Finland, 171 times in China, 236 times in Nigeria, and 261 times in Pakistan. Vaccine types were
reported negatively (B) 4900 times worldwide, 154 times in France, 29 times in Finland, 94 times in China, 193 times in Nigeria, and 311 times in Pakistan. Country border data are from the Global
Administrative Areas database.24 HPV=human papillomavirus. JE=Japanese encephalitis.
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70108-7
5
Articles
May–July, 2011
May, 2011
China
Pakistan
New Delhi
Jaipur
Nepal
China
Pakistan
Bhutan
New Delhi
Jaipur
Lucknow
Nepal
Bhutan
Lucknow
Bangladesh
Bangladesh
Kolkata
India
Myanmar
(Burma)
Gujarat
India
Kolkata
Myanmar
(Burma)
Gujarat
Hyderabad
Hyderabad
Andhra Pradesh
Andhra Pradesh
Bangalore
Bangalore
Low priority
Medium priority
High priority
May–Sept, 2011
May–Nov, 2011
China
Pakistan
New Delhi
Jaipur
Nepal
China
Pakistan
Bhutan
New Delhi
Jaipur
Lucknow
Nepal
Bhutan
Lucknow
Bangladesh
India
Kolkata
Gujarat
Bangladesh
Myanmar
(Burma)
India
Gujarat
Kolkata
Myanmar
(Burma)
Hyderabad
Hyderabad
Andhra Pradesh
Bangalore
Bangalore
Low priority
Medium priority
High priority
Figure 5: Cumulative number of reports from India about human papillomavirus vaccination over time, by location and priority
Data consists of 48 location mentions from 31 reports obtained between May, 2011, and November, 2011. Country border data are from the Global Administrative
Areas database.24
data collection. For instance, polio vaccination dominates
in Nigeria and Pakistan, two of the three remaining
endemic countries, and the polio vaccination coverage in
China is probably a result of the 2011 spread of the disease
from Pakistan to China—the first cases of the disease in
China since 1999.25 The media coverage of H1N1 influenza
in Finland was due to the heightened concerns around the
2010–11 suspected links between narcolepsy and H1N1
influenza vaccination,26 and the attention to measles
vaccination in France is consistent with the 2011 outbreak.27
One of the important dimensions of the surveillance
system is the ability to monitor the evolution of concerns
over time and location. We mapped reports about human
papillomavirus (HPV) vaccination from India, from
May, 2011, to November, 2011, to monitor the effect of the
April, 2010, government suspension of the HPV vaccine
demonstration projects in Gujarat and Andhra Pradesh,
caused by pressure from activists (figure 5).28 Although
11 (28%) of the 39 reports about HPV in India obtained
during the study period (May 1, 2011, to April 30, 2012)
6
were positive or neutral about HPV vaccines, most,
28 (72%), were negative—with some already posted on
European and US anti-HPV vaccine websites—and were
still focused on the issues surrounding the suspension of
the HPV vaccine demonstration projects towards the end
of our data collection period (April 30, 2012), 2 years after
the government suspension of the project.
Discussion
33% of positive reports obtained worldwide were
categorised as being about vaccine development and
introduction, whereas only 3% of negative reports made
reference to that topic; this percentage point difference
was the largest noted between positive and negative
reports out of all topic categories. The category of beliefs,
awareness, and perceptions shows a similar large
difference, albeit in the opposite direction, with 3% of
positive reports making reference to this topic compared
with 21% of negative reports. These reports include those
discussing religious and philosophical beliefs about
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70108-7
Articles
vaccines, and perceptions about the motive behind
vaccination (eg, government, industry). More negative
reports discussed beliefs and perceptions than mentioned
vaccine safety (16% of negative reports); negative reports
discussing vaccine safety focused on adverse events and
concerns about additives such as thiomersal.
Clearly, the proportions of positive and negative reports
discussing each topic would change if examined at a
more local level or by specific vaccine. Disaggregation of
reports by positive or negative sentiment, country, and
vaccine type could identify countries where specific
vaccines might need more tailored engagement strategies
than are currently in place.
The subnational analysis of reports about HPV
vaccination from India (figure 5), is an example of an
analysis that could help to inform delivery and
engagement strategies. The HPV vaccine is available in
India only through the private sector; it has not yet been
included in India’s national immunisation schedule.
When the HPV vaccine is considered for inclusion in the
national immunisation schedule, this type of analysis,
which identifies the clustering of reports and their
sentiments in specific localities, might be useful to
inform the design of engagement strategies, which will
be particularly needed in Gujarat and Andhra Pradesh.
When faced with several concerns and limited human
and financial resources, this type of assessment could
help to prioritise attention and allocation of resources.
Despite the strengths of our surveillance system,
including the creation of a broad-spectrum and reflexive
typology of concerns, it also has limitations. First, we used
only English search terms to collect the initial data.
Although some reports from non-English speaking
countries were translated and included in the dataset, most
were published in English, and included English language
media from countries where the primary language is not
English. However, the primary search terms are now being
translated into five additional languages: Arabic, French,
Mandarin, Russian, and Spanish.
Although human curation was important in the
development and refinement of the system, it also
restricted the volume of reports that could be processed.
Efforts are focused on automating as much of the data
collection and classification as possible, using the
human-curated dataset as a training set. These changes
will result in an increase in the geographical reach,
volume, and speed at which reports can be classified and
analysed. The changes will also help to enhance the
typology of concerns, understand specific contextual
factors, and develop the risk assessment method. Future
iterations of the system will also expand the numbers of
sources that are included. This expansion will be achieved
by increasing the languages of our search terms and
using other modes of social media (eg, Twitter).
The nature of public concerns about vaccines is
complex and highly diverse. No single report can be
relied upon to signal a genuine public concern. Instead,
the strength and potential effects of a vaccine rumour or
public concern are determined by clusters and persistent
patterns of reports expressing similar concerns.
The surveillance system we present in this report allows
systematic monitoring and assessment of media reports
for vaccine sentiment, with the aim of detecting concerns
as they emerge and evolve in real time. We created a
typology of the scope of public concerns about vaccines
and a way to monitor the ecology of those concerns as they
develop temporally and spatially. Although this system
has not been running long enough to show its long-term
predictive value, it allows real-time characterisation of
vaccine sentiments by topic, negative or positive content,
location, time, and risk level. The third objective, to assess
which reported concerns merit priority attention, needs
longer-term study to validate the prioritisation methods.
Although international systems exist to monitor and
investigate adverse events that occur after immunisation,
and several local and vaccine-specific studies29 have
investigated factors affecting vaccine acceptance over
restricted periods of time (panel), no global systems are in
place to routinely monitor and investigate emerging
vaccine concerns that are not solely related to adverse
events that occur after immunisation. Adverse event
reporting systems are crucial to ensure vaccine safety, but,
as shown here, they do not adequately capture the multitude
of concerns that drive vaccine hesitancy and refusals.
The importance of listening to the public throughout the
design and implementation of vaccine programmes and
research trials cannot be understated. Even more
important is acting on what is learned and acting early.
Panel: Research in context
Systematic review
We did a systematic review with the search terms “vaccin*”, “immunis*”, “immuniz*”, AND
“trust”, “confidence”, and a set of additional related terms (eg, “hesitancy”, “anxiety”), with
no date restrictions, of both mainstream and regional databases including Medline, Embase,
PsycINFO, International Bibliography of the Social Sciences, Cochrane, Web of Science,
Cumulative Index to Nursing and Allied Health Literature, Index Medicus for the Eastern
Mediterranean Region, Literatura Latino Americana em Ciências da Saúde, and Africa-Wide
Information. We did the latest search on Nov 12, 2012. Within the results of the broader
search, we then searched for relevant research about online monitoring, and found four
relevant studies: an assessment of vaccination sentiments in social media that analysed
content and correlated the findings with vaccination rates estimated by the Centers for
Disease Control and Prevention;30 an analysis of discussions about the measles, mumps, and
rubella vaccine in an online chat forum for parents;31 a content analysis of top search engine
results for human papillomavirus vaccine information online;32 and a study that surveyed
web users and assessed the popularity of influenza-vaccine-related sites during an influenza
pandemic.33 None used worldwide data and none collected data about all types of vaccines.
Interpretation
This real-time information surveillance system uniquely captures both positive and
negative vaccine reports and sentiments as reported in both traditional news and social
media reports around the world. Because the system can categorise report content by
type and monitor the evolution of concerns spatially and temporally, the system could be
used to inform vaccine delivery and engagement strategies.
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70108-7
7
Articles
The retrospective analyses that informed this system’s
development showed that early signs of concern have often
been available well before their most serious effects
occurred, but were not acted on, largely because the
potential results were not expected. Although we show that
vaccine concerns can vary geographically, evidence of
global dissemination of vaccine concerns was apparent,
enhanced by internet-based communication. Thus a global
approach might help to detect confidence issues before
they become established in other geographical areas.
We now have a growing body of evidence of the potential
risks of the spread of unchecked rumours, and of failing to
address legitimate questions and concerns. Even if con­
cerns driven by misinformation or reported adverse events
are investigated and confirmed as not being caused by
vaccines, concerns and reputations in the public mind
need to be addressed. Although 69% of the global reports
were positive about vaccines, 31% were not. And, of these
31%, a large proportion of concerns are related to belief
systems. The public health community should not under­
estimate the implications of the global burden of belief.
Contributors
All authors contributed to study design, data analysis and interpretation,
and writing and approval of the manuscript. HJL was the principal
investigator. DMDS did the statistical analysis and produced all figures.
Conflicts of interest
One representative from the Bill & Melinda Gates Foundation, which
provided the funding, is a member of the project’s international advisory
group, which provided feedback over the course of the research, but the
research team had sole responsibility for all decisions about the conduct
of the research and analysis of the findings. Since submitting this
Article, Epidemico has been formed by Boston Children’s Hospital to
handle the licensing of HealthMap data for commercial companies.
JSB holds an equity stake in Epidemico. All other authors declare that
they have no conflicts of interest.
Acknowledgments
We thank the international advisory group of The Vaccine Confidence
Project for their valuable feedback throughout this research.
References
1 Larson HJ, Cooper LZ, Eskola J, Katz SL, Ratzan S. Addressing the
vaccine confidence gap. Lancet 2011; 378: 526–35.
2 Black S, Rappuoli R. A crisis of public confidence in vaccines.
Sci Transl Med 2010; 2: 61mr1.
3 Yahya M. Polio vaccines—“no thank you!” barriers to polio
eradication in Northern Nigeria. Afr Aff (Lond) 2007; 106: 185–204.
4 Kauffman JR, Feldbaum H. Diplomacy and the polio immunisation
boycott in Northern Nigeria. Health Affairs 2009; 28: 1091–101.
5 Heymann DL, Sutter RW, Aylward RB. Polio eradication:
interrupting transmission, towards a polio free-world. Future Virology
2006; 1: 181–88.
6 Larson HJ, Ghinai I. Lessons from polio eradication. Nature 2011;
473: 446–47.
7 Global Polio Eradication Initiative. Report by the WHO Secretariat
to the 60th World Health Assembly. Poliomyelitis: mechanism for
management of potential risks to eradication (May 24, 2008). http://
www.polioeradication.org/content/general/WHA61_Resolution_
English.pdf (accessed Jan 31, 2013).
8 Institute of Medicine. Immunisation safety review: vaccines and
autism. Washington, DC: The National Academies Press, 2004.
9 Honda H, Shimizu Y, Rutter M. No effect of MMR withdrawal on
the incidence of autism. J Child Psychol Psychiatry 2005; 46: 572–79.
10 Wakefield AJ, Murch SH, Anthony A, et al. Ileal-lymphoid-nodular
hyperplasia, non-specific colitis, and pervasive developmental
disorder in children. Lancet 1998; 351: 637–41.
8
11 Godlee F, Smith J, Marcovitch H. Wakefield’s article linking MMR
vaccine and autism was fraudulent. BMJ 2011; 342: c7452.
12 Health Protection Agency. Summary of trends in vaccination
coverage in the UK. Annual COVER report: 2005/06. http://www.
hpa.org.uk/webc/HPAwebFile/HPAweb_C/1194947367316 (accessed
Jan 31, 2013).
13 NHS. NHS immunisation statistics, England 2011–12. https://
catalogue.ic.nhs.uk/publications/public-health/immunisation/nhsimmu-stat-eng-2011-2012/nhs-immu-stat-eng-2011-12-rep.pdf
(accessed Jan 31, 2013).
14 Larson HJ, Heymann DL. Public health response to influenza A
(H1N1) as an opportunity to build public trust. JAMA 2010; 303: 271–72.
15 Kasperson RE, Renn O, Slovic P, et al. The social amplification of
risk: a conceptual framework. Risk Anal 1988; 8: 177–87.
16 Madoff LC. ProMED-mail: an early warning system for emerging
diseases. Clin Infect Dis 2004; 39: 227–32.
17 Public Health Agency of Canada. The Global Public Health
Intelligence Network (GPHIN). http://www.phac-aspc.gc.ca/gphin/
(accessed Jan 31, 2013).
18 WHO. Global Outbreak Alert & Response Network. http://www.
who.int/csr/outbreaknetwork/en/ (accessed Jan 31, 2013).
19 Surveillance Sans Frontières: Internet-based emerging infectious
disease intelligence and the HealthMap project. PLoS Med 2008;
5: e151.
20 Guy S, Ratzki-Leewing A, Bahati R, Gwadry-Sridhar F. Social media:
a systematic review to understand the evidence and application in
infodemiology. Lect Notes Inst Comput Sci Soc Inform Telecomm Eng
2012; 91: 1–8.
21 The Vaccine Confidence Project. Advisory group. www.
vaccineconfidence.org/advisorygroup.html (accessed Jan 31, 2013).
22 Freifeld CC, Mandl KD, Reis BY, Brownstein JS. HealthMap: global
infectious disease monitoring through automated classification and
visualization of Internet media reports. J Am Med Inform Assoc
2008; 15: 150–57.
23 Council for International Organizations of Medical Sciences.
Definition and application of terms for vaccine pharmacovigilance:
report of CIOMS/WHO Working Group on Vaccine
Pharmacovigilance. Geneva: Council for International
Organizations of Medical Sciences, 2012.
24 Global Administrative Areas (GADM). GADM database of global
administrative areas. http://www.gadm.org (accessed Feb 19, 2013).
25 Donaldson L, Sayed NE, Koplan J, et al, for the Independent
Monitoring Board of the Global Polio Eradication Initiative. Report,
October 2011. http://www.polioeradication.org/Portals/0/
Document/Aboutus/Governance/IMB/4IMBMeeting/
IMBReportOctober2011.pdf (accessed Jan 31, 2013).
26 Dauvilliers Y, Montplaisir J, Cochen V, et al. Post H1N1
narcolepsy-cataplexy. Sleep 2010; 33: 1428–30.
27 WHO, Regional Office for Europe. Measles outbreaks across
Europe show no sign of slowing. www.euro.who.int/en/what-wedo/health-topics/disease-prevention/vaccines-and-immunization/
news/news/2011/04/measles-outbreaks-across-europe-show-nosign-of-slowing (accessed Jan 31, 2013).
28 Larson HJ, Brocard P, Garnett G. The India HPV-vaccine
suspension. Lancet 2010; 376: 572–73.
29 Amarasinghe A, Black S, Bonhoeffer J. Effective vaccine systems in
all countries: a challenge for more equitable access to
immunization. Vaccine 2013; 31 (suppl): B108–14.
30 Salathé M, Khandelwal S. Assessing vaccination sentiments with
online social media: implications for infectious disease dynamics
and control. PLoS Comput Biol 2011; 7: e1002199.
31 Skea ZC, Entwistle VA, Watt I, Russell E. ‘Avoiding harm to others’
considerations in relation to parental measles, mumps and rubella
(MMR) vaccination discussions—an analysis of an online chat
forum. Soc Sci Med 2008; 67: 1382–90.
32 Madden K, Nan X, Briones R, Waks L. Sorting through search
results: a content analysis of HPV vaccine information online.
Vaccine 2012; 30: 3741–46.
33 Seeman N, Ing A, Rizo C. Assessing and responding in real time to
online anti-vaccine sentiment during a flu pandemic. Healthc Q
2010; 13: 8–15.
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70108-7
Comment
Inoculating communities against vaccine scare stories
The biggest threat facing the success of immunisation
might be public lack of confidence in vaccines, repeatedly
undermined by safety concerns promulgated in social
and news media.1 In The Lancet Infectious Diseases,
Heidi Larson and colleagues’ study examines how a
typology of concerns can be applied within an established
global surveillance system, HealthMap, to track and
characterise vaccine news stories.2,3 The usefulness of
systematically tracking online media stories was first
established for disease surveillance through a Canadian
project, the Global Public Health Information Network,4
followed by several other systems including HealthMap.
These systems offer the potential for efficient and early
signal detection.
Surveillance of different types of vaccine safety concerns
will be an important way to listen to communities, and
will also serve as a platform for research. Such surveillance
gives concerns a greater profile, and emphasises gaps in
the public health system’s responses to and preparedness
for vaccine safety events. Bad news stories damage
vaccination programmes as much as biological hazards,
and these stories evolve over minutes or hours, needing
immediate action. By the time a detailed scientific
analysis of a vaccine safety issue is completed, the story
is no longer newsworthy. Despite growing knowledge
about how to respond to vaccine safety events and the
availability of good guidance, confidence in vaccines
is falling.5,6 Much can be learnt from antivaccine
campaigners who make good use of engaging narrative
and social media to voice their ideas.7
Vaccine concerns do not inevitably or predictably
spread from one country to another. Because some
people are more resistant to scare stories than are
others, protecting against such events at the com­
munity level might be useful. With respect to the loss
of confidence in the measles, mumps, and rubella
(MMR) vaccine, UK public health bodies applied lessons
learnt from the whooping cough vaccine scare of the
1970s to good effect: MMR vaccine coverage only
dropped to a low of 80% compared with the pertussis
vaccine coverage low of 30%.8,9 Although MMR vaccine
coverage recovered as media interest declined,10 the
recovery was not sufficient to avert the end of measles
elimination in the UK, and the scare heralded a wider
re-emergence of measles in Europe.11
Because immunisation is so important for health,
public health systems need to move beyond passive
responses to vaccine safety events towards active
preparedness. Researchers need to discover how
to make communities resilient to bad science and
interest-driven scare stories to change public ideas and
norms and to promote belief in vaccines.12 However,
beliefs are hard to change, and correct information
might paradoxically entrench existing negative
attitudes and undermine positive ones.13 Pre-emptive
action in childhood to promote immunisation might
prime the minds of future parents, health-care
providers, journalists, and politicians with a positive
attitude towards immunisation. The benefits and risks
of immunisation could be discussed in schools as part
of history, politics, science, health, and ethics courses.
This approach might lead to a stronger sense of the
value of immunisation for both the individual and the
public.
Immunisation is a part of history and protects against
future disease outbreaks. Benefits of smallpox eradi­
cation continue to accrue—although they are invisible
to most people—but many challenges will need to
be overcome to reach polio and measles elimination
goals.14 Speed might be essential to achieve disease
eradication, but moving quickly enough will only
be possible by first building strong public belief and
confidence in immunisation.
Published Online
May 13, 2013
http://dx.doi.org/10.1016/
S1473-3099(13)70131-2
See Online/Articles
http://dx.doi.org/10.1016/
S1473-3099(13)70108-7
*Natasha Sarah Crowcroft, Kwame Julius McKenzie
Public Health Ontario, Toronto, ON, M5G 1V2, Canada (NSC);
Laboratory Medicine and Pathobiology and Dalla Lana School of
Public Health (NSC), and Department of Psychiatry (KJM),
University of Toronto, Toronto, ON, Canada; and Centre for
Addictions and Mental Health, Toronto, ON, Canada (KJM)
[email protected]
We declare that we have no conflicts of interest.
1
2
3
4
Dubé E, Laberge C, Guay M, Bramadat P, Roy R, Bettinger JA. Vaccine
hesitancy: an overview. Hum Vaccin Immunother 2013; published online
April 12. http://www.landesbioscience.com/journals/vaccines/
article/24657/.
Larson HJ, Smith DMD, Paterson P, et al. Measuring vaccine confidence:
analysis of data obtained by a media surveillance system used to analyse
public concerns about vaccines. Lancet Infect Dis 2013; published online
May 13. http://dx.doi.org/10.1016/S1473-3099(13)70108-7.
Brownstein JS, Freifeld CC, Reis BY, Mandl KD. Surveillance Sans Frontières:
internet-based emerging infectious disease intelligence and the HealthMap
project. PLoS Med 2008; 5: e151.
Mykhalovskiy E, Weir L. The Global Public Health Intelligence Network and
early warning outbreak detection: a Canadian contribution to global public
health. Can J Public Health 2006; 97: 42–44.
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70131-2
1
Comment
5
6
7
8
9
2
WHO Regional Office for Europe. Vaccine safety events: managing the
communications response. A guide for Ministry of Health EPI Managers and
Health Promotion Units. http://www.euro.who.int/en/what-we-do/healthtopics/disease-prevention/vaccines-and-immunization/publications/2013/
vaccine-safety-events-managing-the-communications-response (accessed
April 26, 2013).
Stefanoff P, Mamelund SE, Robinson M, et al, for the VACSATC working
group on standardization of attitudinal studies in Europe. Tracking parental
attitudes on vaccination across European countries: the Vaccine Safety,
Attitudes, Training and Communication Project (VACSATC). Vaccine 2010;
28: 5731–37.
Leask J. Target the fence-sitters. Nature 2011; 473: 443–45.
Gangarosa EJ, Galazka AM, Wolfe CR, et al. Impact of anti-vaccine movements
on pertussis control: the untold story. Lancet 1998; 351: 356–61.
UK Department of Health. NHS immunisation statistics 2003–04. http://
webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.
gov.uk/en/Publicationsandstatistics/Statistics/StatisticalWorkAreas/
Statisticalhealthcare/DH_4099576 (accessed April 25, 2013).
10 Smith A, Yarwood J, Salisbury DM. Tracking mothers’ attitudes to MMR
immunisation 1996–2006. Vaccine 2007; 25: 3996–4002.
11 European Centre for Disease Prevention and Control. Measles atlas:
progress towards elimination. http://emmageocase.ecdc.europa.eu/atlas/
measles/?t=4&m=7&x=8.08&y=49.68&l=3 (accessed April 25, 2013).
12 Casiday RE, Cox AR. Restoring confidence in vaccines by explaining vaccine
safety monitoring: is a targeted approach needed? Drug Saf 2006;
29: 1105–09.
13 Meszaros JR, Asch DA, Baron J, Hershey JC, Kunreuther H,
Schwartz-Buzaglo J. Cognitive processes and the decisions of some parents
to forego pertussis vaccination for their children. J Clin Epidemiol 1996;
49: 697–703.
14 Omer SB, Orenstein WA, Koplan JP. Go big and go fast—vaccine refusal and
disease eradication. N Engl J Med 2013; 368: 1374–76.
www.thelancet.com/infection Published online May 13, 2013 http://dx.doi.org/10.1016/S1473-3099(13)70131-2