Evaluating the Medical Literature Clista Clanton, MSLS, AHIP January 2009

Evaluating the
Medical Literature
Clista Clanton, MSLS, AHIP
January 2009
Start at
the top
and work
your way
down.
Haynes, RB. ACPJC Nov/Dec 2006;145(3):A8-9
Computer system
Clinical Evidence or PIER
(UpToDate)
ACP Journal Club, InfoPOEMS, Dynamed
Cochrane Library, PubMED Clinical
Queries, BMJUpdates, guidelines
Original Studies
If an original study is your only
option…….
Original Studies
IMRAD format
 Introduction: why the authors decided to
conduct the research.
 Methods: how they conducted the
research and analyzed their results.
 Results: what was found.
And
 Discussion: what the authors think the
results mean.
PP-ICONS







Problem
Patient or population
Intervention
Comparison
Outcome
Number of subjects
Statistics
Flaherty, Robert J. A simple method for evaluating the clinical literature. Fam Prac Mgt, May 2004;47-52. Available online at
http://www.aafp.org/fpm/20040500/47asim.html.
Scenario
 You just saw a nine-year old patient with
common warts on her hands. She is an
ideal candidate for cryotherapy. Her
mother has heard about treating warts
with duct tape and wants to know if you
would recommend this treatment.
Clinical Question
 What is your clinical question?
 PICO: Patient, Intervention/Comparison,
Outcome
 Write your question down, it can help
you to clarify exactly what you are
looking for.
“Is duct tape an effective treatment
for warts in children?”
Search
 After you have your clinical question, search
the appropriate databases:
 Dynamed, PIER, UpToDate, Cochrane, Clinical
Evidence, Essential Evidence Plus
 PubMed
 Focht DR 3rd, Spicer C, Fairchok MP.
The efficacy of duct tape vs cryotherapy in
the treatment of verruca vulgaris (the
common wart). Arch Pediatr Adolesc Med.
2002 Oct;156(10):971-4.

Abstract
OBJECTIVE: To determine if application of duct tape is as effective as cryotherapy
in the treatment of common warts.
DESIGN: A prospective, randomized controlled trial with 2 treatment arms for warts
in children.
SETTING: The general pediatric and adolescent clinics at a military medical center.
PATIENTS: A total of 61 patients (age range, 3-22 years) were enrolled in the study
from October 31, 2000, to July 25, 2001; 51 patients completed the study and were
available for analysis.
INTERVENTION: Patients were randomized using computer-generated codes to
receive either cryotherapy (liquid nitrogen applied to each wart for 10 seconds every
2-3 weeks) for a maximum of 6 treatments or duct tape occlusion (applied directly to
the wart) for a maximum of 2 months. Patients had their warts measured at baseline
and with return visits.
MAIN OUTCOME MEASURE: Complete resolution of the wart being studied.

RESULTS: Of the 51 patients completing the study, 26 (51%) were treated with duct
tape, and 25 (49%) were treated with cryotherapy. Twenty-two patients (85%) in the
duct tape arm vs 15 patients (60%) enrolled in the cryotherapy arm had complete
resolution of their warts (P =.05 by chi(2) analysis). The majority of warts that
responded to either therapy did so within the first month of treatment.
CONCLUSION: Duct tape occlusion therapy was significantly more effective than
cryotherapy for treatment of the common wart.
Problem (PP-ICONS)
 What is the clinical condition that was
studied in the article?
OBJECTIVE: To determine if application of duct tape
is as effective as cryotherapy in the treatment of
common warts.
 The problem studied should be sufficiently similar
to your clinical problem, or the results will not be
relevant.
Patient or Population (PPICONS)
 Is the study group similar to your patient
or practice?
SETTING: The general pediatric and adolescent
clinics at a military medical center.
PATIENTS: A total of 61 patients (age range, 3-22
years)
 If the patients in the study are not similar
to your patient (older, sicker, different
gender or more clinically complicated), the
results may not be relevant.
Intervention (PP-ICONS)
 Is the intervention the same as what you
are looking for?
 Could be a diagnostic test or a treatment
The patient’s mother has heard about treating
warts with duct tape and wants to know if you
would recommend this treatment.
Comparison (PP-ICONS)
 The comparison is what the treatment is
tested against.
 Could be a different diagnostic test,
another therapy, placebo, or no treatment
at all.
INTERVENTION: Patients were randomized using
computer-generated codes to receive either cryotherapy
(liquid nitrogen applied to each wart for 10 seconds every 23 weeks) for a maximum of 6 treatments or duct tape
occlusion (applied directly to the wart) for a maximum of 2
months.
Outcome (PP-ICONS)
 Disease-oriented outcomes (DOEs): usually
reflect changes in physiologic parameters.
 It has long been assumed that improving the
physiologic parameters of a disease will result in a
better outcome, but this is not always true.
 Patient-oriented evidence that matters
(POEMs): look at outcomes such as
morbidity, mortality and cost.
 Therefore, DOEs are interesting but of
questionable relevance, whereas POEMs are
very interesting and very relevant.
MAIN OUTCOME MEASURE: Complete resolution of the wart
being studied.
Number (PP-ICONS)
 Number of subjects in the study is
crucial in whether accurate statistics
can be generated from the data.
 Too few patients may not be enough to show that
a difference really exists between intervention and
comparison groups (power of a study).
 Many studies contain <100 subjects, which is
usually inadequate to provide reliable statistics.
 Good rule of thumb – 400 subjects needed.8
51 patients completed the study
Krejcie RV, Morgan DW. Determining sample size for research activities. Educational and Psychological Measurements. 1970;30:607-610.
Statistics (PP-ICONS)
 Relative risk reduction (RRR): the percent
reduction in events in the treated group
compared to the control group event rate.
 Not a good way to compare outcomes
 Amplifies small differences and makes insignificant
findings appear significant
 Doesn’t reflect the baseline risk of the outcome event
 Can make weak results look good, therefore
 Popular and will be reported in almost every journal
article
 Ignore – it can mislead you
RRR would be (85 percent – 60 percent/60 percent x 100 = 42 percent
I.e. 42 percent more effective than cryotherapy in treating warts
Statistics (PP-ICONS)
 Absolute risk reduction (ARR): the
difference in the outcome event rate
between the control group and the
experimental group.
ARR for the wart study is the outcome event rate (complete
resolution of warts) for duct tape (85 percent) minus the
outcome event rate for cryotherapy (60 percent) = 25 percent
 A better statistic to evaluate outcome, as it
does not amplify small differences, but
shows the true difference between the
experimental and control interventions.
Statistics (PP-ICONS)
 Number needed to treat (NNT): number of patients
who must be treated to prevent one adverse
outcome OR the number of patients who must be
treated for one patient to benefit
 Single most clinically useful statistic
 Easy to calculate, simply the inverse of the ARR.
For the wart study, the NNT is 1/25 percent = 1/0.25 = 4
4 patients need to be treated with duct tape for one to benefit more
than if treated by cryotherapy
 The lower the NNT, the better. For primary therapies, an
NNT of 10 or less is good, with less than 5 being very
good.
 For preventive interventions, the NNT will be higher. An
NNT for prevention of less than 20 might be particularly
good.
Intention to Treat Analysis
 Attrition: Were patients lost to follow-up, and if so, why?
Intention to treat: subjects are analyzed according to the
categories into which they were originally randomized.
– Benefits of a treatment are more difficult to demonstrate with
intention-to-treat analysis.
– Helps to mitigate differences by including subjects who are unlikely to
have experienced benefit from the intervention.
Six patients from cryotherapy group and 4 patients from the duct tape
group were lost to follow-up (16% of patients).
Worst case scenario: 6 cryotherapy patients had wart resolution and the 4
duct tape patients had residual wart.
Wart resolution would then be: duct tape 78% and cryotherapy 68% (95%
CI, -17 to 28) – therefore not a statistically significant difference between
the two treatments.
Christakis DA, Lehmann HP. Is duct tape occlusion therapy as effective as cryotherapy for the treatment of the common wart? Arch Pediatr Adolesc Med, Oct 2002; vol. 156;
975-977.
Best Type of Study for Your Question
Type of Question
Suggested Best Type of Study
Therapy
RCT > cohort > case control > case series
Diagnosis
Prospective, blind comparison to gold standard
Etiology / Harm
RCT > cohort > case control > case series
Prognosis
Cohort study > case control > case series
Prevention
RCT > cohort study > case control > case series
Clinical Exam
Prospective, blind comparison to gold standard
Cost
Economic analysis
Questions of therapy, etiology and prevention which can best be answered
by RCT can also be answered by a meta-analysis or systematic review.
Resources
 Presentation/handouts
available at:
http://southmed.usouthal.edu/library/ebmclass/rotationswinterspring.htm
Assignment





Identify a clinical problem with a patient
Formulate a clinical question using PICO
Search the literature for appropriate article(s)
Evaluate the article(s)
Complete the online assignment within two weeks
after date of lecture.
 http://southmed.usouthal.edu/library/ebmclass/rotationswinterspring.htm