Haresh Kirpalani

When is there enough evidence
to make therapy a standard?
Haresh Kirpalani
OUTLINE
• Thoughts about ‘evidence’
• Do we speak the same language
when defining outcomes?
• One perinatal examples
• What is the state of current
neonatal randomized trials?
• The case of probiotics:
importance of validity
• The case of red cell
transfusions: importance of size
Proposed general principles
1. Sound biological basis for experimental
practice
2. At least one large, valid,
randomized controlled trial – with a
reasonably precise estimate of the
risk:benefit ratio
3. Ideally, replication is performed
Usual Paradigm: The Evidence Pyramid
A SATIRE:
‘Parachute Use Needs no RCTs’
Parachute use to prevent death and major trauma related to gravitational
challenge: systematic review of randomised controlled trials
Gordon C S et al: Pell BMJ 2003;327;1459-1461
Propranolol for severe hemangiomas of infancy. Léauté‐Labrèze C, et al. N Engl J Med. 2008 Jun 12;358(24):2649‐51
“Can YOU Believe What Scientists
Publish?“September 14, 2007, Wall St Journal
Most published research is wrong, says
epidemiologist John Ioannidis.. cult-classic paper
‘Why Most Research Findings Are False’ (PLOS). …
You have millions of potential discoveries, but what is really true out of
The only
way findings
to get are
credibility
is to go for repeated
Whythat?
most published
research
false.
Ioannidis
JP.
replication,
again and
PLoS Med. 2005 Aug;2(8):e124.
again, with many different teams.
Replication is more important than discovery.
Why most published
research findings are
false.
Ioannidis JP.
PLoS Med. 2005
Aug;2(8):e124.
OUTLINE
• Thoughts about ‘evidence’
• Do we speak the same language
when defining outcomes?
• One perinatal examples
• What is the state of current
neonatal randomized trials?
• The case of probiotics:
importance of validity
• The case of red cell
transfusions: importance of size
Survival rates depend on denominator
2000-2011:
51 of 111 papers
reported denominator
39%
49%
12%
Guillen Ú et al; Am J Ob Gynecology 2011 PMID: 21741613
Attrition bias and developmental impairment 18-24 mos
2000-2011 20 of 41 papers
Only half report loss to FU rate
In Press: Guillen Ú, Arch Pediatr Adolesc Med 2011
OUTLINE
• Thoughts about ‘evidence’
• Do we speak the same language
when defining outcomes?
• One perinatal examples
• What is the state of current
neonatal randomized trials?
• The case of probiotics:
importance of validity
• The case of red cell
transfusions: importance of size
WHERE THE COCHRANE LOGO COMES FROM
USE OF A CUMULATIVE META-ANALYSIS
2500 OIS
Meta-analysis of RCTs of antenatal corticosteroid for prevention of respiratory distress
syndrome: discussion. Sinclair JC. Am J Obstet Gynecol. 1995;173:335
DISSEMINATING THE RESULTS OF THE 1994 NIH EXPERT
CONCENSUS ON ANTENATAL CORTICOSTEROIDS
RCT: Active dissemination 33% to 68% OR: 4.79
Usual dissemination 33% to 57% OR: 3.16
Onland W: Effects of ANCS prior to 26 weeks GA: a systematic
review : Am J Perinatology 2011: 28: 33
TOTAL NUMBER RANDOMIZED < 28 WEEKS IS ONLY 194
TOTAL NUMBER RANDOMZIED >28 WEEKS IS 1047
Mwansa-Kambafwile J et al: Antenatal steroids for prevention of neonatal deaths due to
complications of preterm birth. Int J Epidemiol2010;39(suppl 1):i122-33
RR 0.47
(0.36, 0.64
Middle Income Countries
RR 0.79
(0.65, 0.96
High Income Countries
Some conclusions on ANCS in
current practice
•Efficacious - standard of care
• Extrapolation to GA <26 weeks
uncertain, but likely beneficial
• Middle and low income countries
should adopt this therapy
OUTLINE
• Thoughts about ‘evidence’
• Do we speak the same language
when defining outcomes?
• One perinatal examples
• What is the state of current
neonatal randomized trials?
• The case of probiotics:
importance of validity
• The case of red cell
transfusions: importance of size
SIZE OF NEONATAL TRIALS
Percentile
10th
50th
90th
Number
18
53
248
JC Sinclair et al. Clin Perinatol 2003;30:285-304
Quality of reporting neonatal RCTs: high impact journals
DeMauro SB et al, Pediatrics 2011
Quality of RCTs and number of study centers
11
Number of quality crtieria
10
9
8
7
p < 0.0001
6
5
4
Median number of patients enrolled
N = 89
3
2
1
0
10
20
30
40
50
60
70
Number of centers
DeMauro SB, Pediatrics 2011
80
90
100
Probiotics to prevent NEC:
a case study on the
importance of the validity
of neonatal trials
RCTs of Probiotics: Outcome of definite NEC
Deshpande G et al:2010 Pediatrics
Pediatrics 2010;125;1068‐1070;
WILLIAM SILVERMAN MD (1942-2004)
“Retrolental Fibroplasia: A Modern Parable” 1980 N.Y.
http://www.neonatology.org/classics/parable/
The profession has sallied forth ….
carrying “truth” on a banner with
the same missionary spirit which
guided the New England clerics
when they invaded Tikopia with
Godly truth and stamped out the
Melanesian population-control of
infanticide. But the frantic
evangelism of modern perinatal
medicine is puzzling. ….
“What in God’s name is the hurry?
Are the results of the study valid?
Was assignment patients to treatments randomized?
Were all patients entering accounted for through study?
Was follow-up complete?
Were patients analyzed by groups randomized to?
Who was blinded?
Were groups at start-baseline similar?
Were groups treated the same, apart from experimental therapy?
What Were the Results?
How large was the effect? How precise was the estimate?
Will the results help me in caring for my patients?
Can results be applied to my patients?
Are all clinically important outcomes considered?
Are the likely benefits worth potential harms and costs ?
VALIDITY OF PAST TRIALS
WITH OUTCOME OF NEC
Validity
Concern
Number of
RCTs
Number of
Patients
Inadequate
blinding
5 of 11
45 %
1067 of 2176
49%
Exclusions of
outcomes after
randomization
2 of 11
18 %
654 of 2176
30%
What is the weight of the most valid RCTs
using outcome of definite NEC?
These four trials account for 26.3% of weight in the
pooled analysis
Deshpande G et al 2010 Pediatrics
When using only the most valid RCTs
– What is the outcome of definite NEC?
Number of infants <1000 g BW
Number
of
RCTs
Number of
Patients
7 of 11
63%
446 of 2176
20%
THE EXAMPLE OF PROBIOTICS TO PREVENT NEC
Six Studies use 1 probiotic
Three Studies use 2 probiotics
N
One Study Uses 4 probiotics
One Study Uses 3 probiotics
N
BB indicates Bifidobacterium breve; LB-GG, Lactobacillus GG; SB,
Saccharomyces boulardii; BI, Bifidobacteria infantis; ST, Streptococcus
thermophilus; BBB, Bifidobacterium bifidus; LB-A, Lactobacillus
acidophilus; LB-C, Lactobacillus casei; BB-L, Bifidobacterium lactis; BBLG, Bifidobacterium longum
Transfusions to improve
neurodevelopment:
a case study on the
importance of the size of
neonatal trials
PRIMARY OUTCOME PINT
Death, BPD, ROP, Brain Injury
Low Hb
High Hb
165/223
(74%)
159/228
(70%)
OR = 1.3
95% CI 0.8-2.0 p = 0.26
PINT-Outcome Study (PINT-OS)
Primary Outcome & a-priori components
OR
Composite
1.45 (0.94, 2.21)
p=0.09
Death
Cerebral Palsy
Cognitive Delay <70
1.74 (0.98, 3.11)
p=0.06
Blindness
Deafness
Favors Low Hgb Favors High Hgb
Whyte R et al: Pediatrics. 2009 123(1):207-13
PINT-Outcome Study (PINT-OS)
Post-Hoc Secondary Analysis
OR
Composite
1.71 1.12, 2.61
Death
p=0.013
Cerebral Palsy
Cognitive Delay <85
1.81 1.12, 2.93
p=0.016
Blindness
Deafness
Favors Low
Favors High
Whyte R et al: Pediatrics. 2009 ;123(1):207-13
Suggested sample size for ‘daughter’ of PINT
Primary outcome:
NDI or Death
Sample Size
Using 2SD cut-off for
cognitive score
1824
Using 1SD cut-off for
cognitive score
2008
8500 patients randomized to receive betablockade in 23 countries pre-operatively
TOTAL RANDOMIZED >10,000
Lancet 2008; 371: 1839–47
Proposed general principles
1. Sound biological basis for experimental
practice
2. At least one large, valid,
randomized controlled trial – with a
reasonably precise estimate of the
risk:benefit ratio
3. Ideally, replication is performed
DISCLOSURES
I have no conflicts
of interest to
declare
BETWEEN CENTER VARIABILITY OF RISK
ADJUSTED 28 DAY NEONATAL MORTALITY
MORTALITY %
NICU RANK
Variations in the quality of care for VLBW INFANTS: implications for
policy Rogowski JA Health Aff Millwood; 2004; 23:88