In its industry-wide petition

Section 505(d) of the Act requires “substantial evidence” of effectiveness before a drug can be
placed on the market and sold to consumers. In most instances, the Agency has interpreted this
to mean that a drug manufacturer must provide adequate and well-controlled clinical studies
distinguishing a drug’s effectiveness “from other influences, such as spontaneous change in the
course of the disease, placebo effect, or biased observation.” 21 C.F.R. § 314.126.
The Act’s language makes clear that homeopathic drugs fall under the definition of the term
“drug” as used broadly in the Act. Section 201(g)(1) of the Act defines the term “drug” as used
in the Act to include “articles recognized in the official United States Pharmacopeia [“USP”], the
official Homeopathic Pharmacopeia of the United States [“HPUS”], or official National
Formulary [“NF”], or any supplement to them; and articles intended for use in the diagnosis,
cure, mitigation, treatment, or the prevention of disease in man or other animals; articles (other
than food) intended to affect the structure or any function of the body of man or other animals;
and articles intended for use as a component of any articles” so specified. Whether or not they
are official homeopathic remedies, those products offered for the cure, mitigation, prevention, or
treatment of disease conditions are regarded as drugs within the meaning of Section 201(g)(l) of
the Act.
Section 400.400 of the FDA Compliance Policy Guidelines, “Conditions Under Which
Homeopathic Drugs May be Marketed,” states that a product’s “compliance with requirements of
the HPUS, USP, or NF does not establish that it has been shown by appropriate means to be safe,
effective, and not misbranded for its intended use.”
The plain meaning of the Act therefore permits the Agency to subject OTC homeopathic drugs to
the same effectiveness standards that apply to other OTC drugs.
The Agency has broad authority over the labeling of drugs. See Section 502 of the Act and 21
C.F.R. Part 201. Through its rulemaking authority, the Agency has required both testing for
effectiveness and appropriate labeling of certain classes of drugs. E.g., Labeling and
Effectiveness Testing; Sunscreen Drug Products for Over-the-Counter Human Use, 76 Fed. Reg.
35620 (June 17, 2011).
2.
The collective weight of the scientific evidence establishes that homeopathic
drugs are no more effective than placebos.
Homeopathic drugs are ineffective for at least two independent reasons, either of which is
sufficient to establish homeopathic drugs’ inefficacy in light of basic, well-established principles
of biochemistry and pharmacology.
First, there is no reliable scientific evidence that the alleged active ingredients in homeopathic
drugs cure the diseases and conditions for which they are indicated. Indeed, as discussed further
below, homeopathic “theory” is premised on the magical notion that very small doses of
substances that cause disease symptoms can alleviate those same symptoms.
Second, producers of homeopathic drugs typically dilute the drugs’ alleged active ingredients in
solution to the point that the solution contains, on average, not a single molecule of the alleged
2
active ingredient.1 In other words, what the producers and retailers of homeopathic drugs market
as “medications” are nothing more than solutions that are devoid of active ingredients (often the
solutions are simply water), which then have been combined with various inactive ingredients,
such as balls of ordinary sugar.
Homeopathy, which was developed and popularized in the late 18th century by the German
physician Samuel Hahnemann, purports to stimulate the body’s ability to heal itself by
administering vanishingly small doses of highly diluted substances.2 Homeopathy is based on
two principles, both of which contradict basic, well-established laws of modern chemistry and
pharmacology: the “principle of similars,” which holds that a disease can be cured by a substance
that produces similar symptoms in healthy people; and the “principle of dilutions,” which holds
that the lower the dose of the medication, the greater its effectiveness.3 Pursuant to
Hahnemann’s teachings, most homeopathic remedies sold today have been diluted repeatedly to
the extent that not a single molecule of the alleged healing substance remains.4
As one would expect, the scientific evidence establishes that any apparent benefits from
homeopathic drugs are mere placebo effects.
The medical journal The Lancet published a comprehensive review of homeopathic treatments in
2005.5 The authors analyzed every clinical investigation then published on the effects of
homeopathy. Their analysis identified 110 well-designed investigations into its effects. The
results of those trials were then compared to 110 well-designed investigations of conventional
medical therapies for the same ailments. The authors concluded:
Biases are present in placebo-controlled trials of both homoeopathy and conventional
medicine. When account was taken for these biases in the analysis, there was weak
1
Homeopaths denote a 1-to-100 dilution by the letter “C,” and a 1-to-10 dilution by the letter
“X.” For example, a “6X” homeopathic preparation is a 1-to-10 dilution repeated six times,
leaving the alleged active ingredient as one part per million. Homeopathic drugs designated as
24X, 12C or higher contain, on average, no molecules of the original substance from which they
were prepared. See generally Stehlin, Isadora. “Homeopathy: real medicine or empty
promises?” FDA Consumer, Dec. 1996. Available online at
http://findarticles.com/p/articles/mi_m1370/is_n10_v30/ai_18979004/.
2
National Institutes of Health (“NIH”) National Center for Complimentary and Alternative
Medicine Backgrounder: Homeopathy, An Introduction (July 2009). Available online at
http://nccam.nih.gov/health/homeopathy/.
3
Id.
4
Id.
5
Shang A, Huwiler-Müntener K, Nartey L, et al. “Are the clinical effects of homoeopathy
placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy.”
Lancet 2005; 366: 726-732. Available online at
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(05)67177-2/fulltext. (Attached
hereto as Exhibit A.)
3
evidence for a specific effect of homoeopathic remedies, but strong evidence for specific
effects of conventional interventions. This finding is compatible with the notion that the
clinical effects of homoeopathy are placebo effects.6
The highly esteemed Cochrane Collaboration has published reviews of homeopathic treatments
for the treatment of a number of diseases and medical conditions, including, among others,
dementia, chronic asthma, attention deficit hyperactivity disorder (ADHD), and influenza. In
each instance the reviews concluded that evidence for homeopathic drugs’ efficacy is
nonexistent, unreliable, or too scarce to warrant recommendations regarding homeopathic
treatment.7
Although most analyses have concluded that there is little evidence to support homeopathy as an
effective treatment for any specific condition, some studies have reported positive findings.8
Studies reporting positive findings, however, are generally poorly controlled and/or have not
been replicated.9
6
Id.
7
See, e.g., McCarney, R.W., et al., “Homeopathy for chronic asthma,” Cochrane Database of
Systematic Reviews 1999, Issue 1. Art. No.: CD000353. DOI:
10.1002/14651858.CD000353.pub2 (available online at
http://www2.cochrane.org/reviews/en/ab000353.html); Heirs, M. and Dean, M.E., “Homeopathy
for attention deficit/hyperactivity disorder or hyperkinetic disorder,” Cochrane Database of
Systematic Reviews 2007, Issue 4. Art. No.: CD005648. DOI:
10.1002/14651858.CD005648.pub2 (available online at
http://www2.cochrane.org/reviews/en/ab005648.html); McCarney, R.W., et al., “Homeopathy
for dementia,” Cochrane Database of Systematic Reviews 2003, Issue 1. Art. No.: CD003803.
DOI: 10.1002/14651858.CD003803 (available online at
http://www2.cochrane.org/reviews/en/ab003803.html); Vickers, A. and Smith, C.,
“Homoeopathic Oscillococcinum for preventing and treating influenza and influenza-like
syndromes,” Cochrane Database of Systematic Reviews 2009, Issue 3. Art. No.: CD001957.
DOI: 10.1002/14651858.CD001957.pub4 (available online at
http://www2.cochrane.org/reviews/en/ab001957.html).
8
NIH National Center for Complimentary and Alternative Medicine Backgrounder:
Homeopathy, An Introduction (July 2009). Available online at
http://nccam.nih.gov/health/homeopathy/.
9
For example, a 1997 meta-analysis of placebo-controlled trials of homeopathic drugs concluded
that patients receiving homeopathic drugs were more likely than patients receiving placebos to
show signs of improvement. Linde, K., et al., “Are the clinical effects of homeopathy placebo
effects? A meta-analysis of placebo-controlled trials.” Lancet, 1997; 350 (9081): 834-43.
Available online at http://www.ncbi.nlm.nih.gov/pubmed/9310601. The authors of the report
later re-examined their meta-analysis. In a follow-up paper that accounted for new, high-quality
trials showing negative results, the authors concluded that “in the study set investigated, there
was clear evidence that studies with better methodological quality tended to yield less positive
results . . . It seems likely, therefore, that our [1997] meta-analysis at least over-estimated the
4
The collective weight of scientific evidence demonstrates that homeopathic drugs yield no
benefit beyond the placebo effect. In addition, the premise upon which the effectiveness of
“homeopathic medicine” is founded—that highly diluted preparations of substances that cause
symptoms in healthy individuals will reduce similar symptoms in patients—has no basis in
reality and has been disproved repeatedly.10 Accordingly, major health and medical
organizations, including the American Medical Association, have concluded that there is no
convincing scientific evidence to support the use of homeopathic treatments in medicine.11
3.
The Agency’s failure to regulate homeopathic products has enabled a multimillion dollar industry to market ineffective products to unsuspecting, ill
consumers.
Although the Agency has the authority to require homeopathic products to undergo the
effectiveness testing that applies to prescription and new OTC drugs, it has to date declined to
exercise that authority. The Agency’s permissive regulatory policy has enabled the growth of a
multi-million dollar market for products for which there is no credible scientific evidence of
effectiveness beyond what is expected from the placebo effect.
Consumer sales of homeopathic treatments in the U.S. reached $870 million in 2009.12
According to a 2007 study conducted by the Center for Disease Control and Prevention
(“CDC”), an estimated 3.9 million U.S. adults and approximately 900,000 children used
homeopathic products in the previous year.13 Ill consumers who desire effective medical
treatment turn, unwittingly, to homeopathic drugs, using them for a broad range of health
concerns, including the treatment of diseases and conditions such as allergies, asthma, chronic
fatigue syndrome, depression, digestive disorders, ear infections, headaches, and skin rashes.14
effects of homeopathic treatments.” Linde, K., et al. “Impact of Study Quality on Outcome in
Placebo-Controlled Trials of Homeopathy,” Journal of Clinical Epidemiology 1999:52 (7): 631–
6. Available online at http://www.ncbi.nlm.nih.gov/pubmed/10391656 (Attached hereto as
Exhibit B.)
10
See, e.g., Ernst, E. “A systematic review of systematic reviews of homeopathy,” Br. J. Clin.
Pharmacol. V.54(6): 577-582 (Dec. 2002). Available online at
http://www.ncbi.nlm.nih.gov/pubmed/12492603. (Attached hereto as Exhibit C.)
11
AMA Council on Scientific Affairs. “Alternative Medicine: Report 12 of the Council on
Scientific Affairs (A–97)” page 8. American Medical Association (1997).
12
“Homeopathy prospers even as controversy rages.” J. Deardorff, Chicago Tribune, March 6,
2011. Available online at http://articles.chicagotribune.com/2011-03-06/news/ct-met-0306homeopathy-20110306_1_homeopathy-oscillococcinum-products.
13
NIH National Center for Complimentary and Alternative Medicine Backgrounder:
Homeopathy, An Introduction (July 2009). Available online at
http://nccam.nih.gov/health/homeopathy/.
14
Id.
5
Homeopathic drugs are typically marketed in ways that lead consumers to believe that they are
effective for preventing and treating illnesses and symptoms, some of which can lead to severe
complications, hospitalization, or death if improperly treated.
By way of example, the retail giant Wal-Mart Stores, Inc. (“Wal-Mart”) promotes the sale of the
homeopathic drug Boiron Oscillococcinum, a homeopathic drug produced by Boiron USA
(“Boiron”), through Wal-Mart’s website, www.walmart.com. The website indicates that Boiron
Oscillococcinum is indicated for the treatment of “flu-like symptoms.”15 The website16 also
features an image of the product’s package, which indicates that the product “Reduces [the]
Duration and Severity of Flu Symptoms,” including “Fever, Chills, Body Aches and Pains.”
Boiron itself is even bolder, engaging in advertising that is highly misleading as it seeks to
suggest this Agency has approved its product. Borion’s website for Oscillococcinum,
http://www.oscillo.com/about/facts-about-oscillo , maintains that four clinical studies show the
effectiveness of its product, and this claim is then immediately followed by the assertion that
“Oscillo complies with a well-established framework of guidelines, regulations, and quality
standards established by the FDA.”
Consumers undoubtedly purchase the product based on Wal-Mart’s and Boiron’s assurances that
Boiron Oscillococcinum is, in fact, effective for preventing and treating the flu—especially as
Boiron implies its product has been tested by this Agency.
Yet there is no credible scientific evidence to support the effectiveness of Boiron
Oscillococcinum beyond what is expected from the placebo effect.17 This is hardly surprising, as
Boiron Oscillococcinum’s alleged ingredient—liquefied duck liver and heart—was mistakenly
identified in the early 20th century as a possible homeopathic agent on the basis of the false
15
See http://www.walmart.com/ip/Boiron-Oscillococcinum-Homeopathic-Flu-Medicine6ct/10316942#ProductDetail.
16
Wal-Mart’s misleading promotion of this homeopathic drug as a treatment for flu is not
limited to the webpage on which the product is displayed. Consumers will reach this page only
after visiting Wal-Mart’s “Medicine Cabinet” page, http://www.walmart.com/cp/MedicineCabinet/976798, which assures customers that the products Wal-Mart carries will “fight colds
and the flu.” From there, website visitors will navigate to the “Cough, Colds & Flu Wellness
Shop” page, http://health.walmart.com/health-tips/cough-cold-flu/20, which promises to help the
customer “Stay on top of cold and flu season by learning about products that can help you and
your family stay well, relieve symptoms and recover fast.” In its “Cough, Cold, and Flu Buying
Guide,” http://health.walmart.com/health-advice/cough-cold-flu-buying-guide/689/, Wal-Mart
asserts that its products will provide the customer “with everything you and your family need for
battling a cold or the flu.”
17
Guo, R., et al., “Complementary Medicine for Treating or Preventing Influenza or Influenzalike Illness,” Am. J. Med., Vol. 120 Issue 11, 923-929 (2007). Available online at
http://www.amjmed.com/article/S0002-9343%2807%2900876-5/abstract.
6
belief that it contained “oscillococci,” microscopic bacteria that proved to be imaginary.18
Moreover, like most producers of homeopathic drugs, Boiron dilutes its alleged active ingredient
to the point that the likelihood that even a single molecule of the substance remains is
vanishingly small.19
The homeopathic industry’s marketing of homeopathic drugs, aided by major retailers such as
Wal-Mart, is a profound disservice to the public. Beyond leading consumers to squander
hundreds of millions of dollars annually on ineffective remedies, homeopathic drug producers
and retailers place the public health at risk by displacing proven, effective medical treatments.
Influenza, for example, is a serious illness that can lead to complications resulting in
hospitalization or even death, especially among the elderly, the very young, and individuals with
certain health conditions.20 Yet U.S. sales of the wholly ineffective homeopathic flu remedy
Oscillococcinum reached $15 million in 2008 alone.21
The Agency should act to ensure that consumers are not led to believe that effective preventive
and therapeutic measures can be ignored in favor of products that amount to mere placebos, at
best.
4.
As the Agency recognizes, accurate labeling is critical if
consumers are to make an informed choice.
As demonstrated by the Agency’s recently issued rule on labeling of sunscreen products,
Labeling and Effectiveness Testing; Sunscreen Drug Products for Over-the-Counter Human Use,
76 Fed. Reg. 35620 (June 17, 2011), this Agency recognizes its responsibility to ensure that
consumers are provided with all material information regarding a product so they can make an
informed choice. The choice of a sunscreen can be very important. At least equally important,
however, are choices relating to the treatment of serious diseases such as influenza. Homeopathic
drug producers and retailers are misleading consumers into believing that their products are
effective. As indicated by Boiron’s promotion of Oscillococcinum, manufacturers of
homeopathic products are not averse to implying that this Agency has approved their product.
18
Nienhuys, J. “The True Story of Oscillococcinum” (2003). Available online at
http://www.homeowatch.org/history/oscillo.html.
19
Boiron Oscillococcinum uses a “200C” homeopathic preparation, indicating a 1-to-100
dilution repeated two hundred times. By way of comparison, a mere “30C” preparation is so
dilute that a single molecule of the original substance dissolved would occupy a volume of
solution that is more than 30 billion times the size of the earth. See Barret, S. “Homeopathy: The
Ultimate Fake.” Available online at
http://www.quackwatch.org/01QuackeryRelatedTopics/homeo.html.
20
See, e.g., Centers for Disease Control and Prevention, “Seasonal Influenza: The Disease.”
Available online at http://www.cdc.gov/flu/about/disease/.
21
Park, Robert L. “Superstition: Belief in the Age of Science,” Princeton University Press. ed.,
pp. 146 (2008).
7
-
EXHIBIT A
-
I
on,o*
\
Are the clinical effects of homoeopathy placebo effects?
Comparative study of placebo-controlled trials of
homoeopathy and allopathy
AijingShong,KorinHu\Niter-Minte et, LindaNoftey,Petet)ini,StephanDotig,Jonotfion A a stcrte, DonicJPevJrner, Motthios
Egqer
5ummary
Ld^.et 2oo5; f66: 726-j2
5e€comnrertPage69l
Departm
en
t of social and
Preventive Medi.in€, Univebity
of B€rne, B€.ne. switu €rl.nd
(A5han9MD,
K
HuwileFMiintend MD,
LNarteyM0,
PJlniMD,5Doig,
D
PsnerMD,
PrcfMEggerMD); Medical
Re5ea,(h Coun.il
H.alth
S€ruicsResear.h
Background Homoeopathy is widely used, but specific effects ofhomoeopathic remedies seem implausible. Bias in
the conduct and reporting of trials is a possible explanation for positive findings oftrials ofboih homoeopathy and
conventional medicine. We analysed trials of homoeopathy and conventional medicine and estimated treatment
elTects in trials least likely to be affected by bias.
Methods placebo.controlled trials of homoeopathy were identified by a comprehensive litenture seatch, which
covered 19 electronic databases, reference lists ofrelevant papers, and contacts with experts. Trials in conventional
medicine matched to homoeopathy trials for disorder and gpe of outcome were randomly selected from the
Cochrane Controlled Trials Register (issue 1, 2003). Data were extracted in duplicate and outcomes coded so that
odds ratios below 1 indicated benefit. Trials described as double.blind, with adequate nndomisation, were assumed
to lre ofhigher methodological quality, Bias effects were examined in funnel plots and meta'regrcssion models.
Collabo.,tion, D€partdenl of
i.l Medicine, U n ive6 itY of
B itol, Brirtol, uK(PJ!ni,
J A C sten€ PhD, PotM E9ga4;
D€p:nment of Pharm:<olo9y,
UniveEity of Zitich, Zijtnh,
switu€ and(5 Dodg);and
Pr:di.e B ickfeld. Medix
G€n€lal Practi.€ Netwolk,
B.rn.,switzedand (D Pesner)
so.
Corspondence to
Prof
Manhes E99er, D€partnent
.f s..ial and Prekntlv€
Medicine,
UniBityof
Be..e,
8erne, switzedand
eqqer@ispm,unibe.ch
Findjngs 110 homoeopathy trials and 110 matched conventional-medicine trials were analysed. The median study
size wls 65 participants (range ten to 1573). 21 homoeopathy trials (19%) and nine (8%) conventional'medicine trials
were ofhigher quility. In both groups, smaller trials and those oflower quality showed more benefrcial treatment
effects thai larger and hlgh"r-qn"lity ttiul". When the analysis was restricted to large trials ofhigher quality, the odds
ratio was 0.8E (95% CI 0.65-i.19) for homoeopathy (eight trials) and 0 . 58 (0 39-4 .85) for conventional medicine
(six trials).
lnterpretation Biases are present in placebo-controlled tlials ofboth homoeopathy and conventional medicine. \Xrhen
account was taken for thise biases in the analysis, there was weak evidence for a specific e{fect ofhomoeopathic
remedies, but strong evidence for specific effecrs ofconventional interventions. This finding is compatible with the
notion that the clinical effects ofhomoeopathy are placebo effects.
to affect small than large studies;the smaller a study, the
lntroduction
the treatment eflect necessary for the results to be
larger
controversial
used
but
Homoeopathy is a widely
complementary or aliernative therapy.'' The basic statistically signifrcant, whereas lalge studies are more
premi." is that like is cured by like (sinilia simi,Iibus likely
curenfur)--iiseases can be treated by substances that
produce the same signs and symptoms in a healthy
individual.., The preplration ofremedies involves serial
dilution, commo;ly io the extent that no molecules of
the original substance temain, and vigorous shaking
between dilutions (potentisation). During this process
informaiion is thought to be transferred from the diluted
substance to the soivent,' which in the light of current
knowledge seems implausible. Many people theiefore
ih"t any effecis of homoeopathy must be non-
""",r-"
specific placebo effects.'
'Bias
to be
ofhigh methodological <luality and published
if their results are negative We examined the
effects of homoeopathy and conventional .medicine
observed in matched Pails of Placebo-conholled t als,
even
trial quality and the probability of publication
and lelated biases, and estimated results oflarge trials
assessed
least affected by such biases
Methods
Literature search and data sources
we
updated
a
prenous comprehensive search for
placebo'controlled tlials ofhomoeopathy, which covered
possible Publications up to August' 1995 " We searched
of placebo'controlled 19 electronic databases, including specialised
irithe conduct and rePorting oftrials is
a
explanation for positive 6"il"g.
trials of both homoeopathy and allopithy (conventional homoeopathic and comPlementary-medicine registries,
medicine).'" Publication bias
is
defined
as the
covering the Period from 1995
to |anuary' 2003:
preferential and more rapid publication of trials with MEDLINE, PIe-MEDLINE' EMBASE' DARE' CCTR'
statisticaliy significant and bineficial results than of CDSR, CINAHL, AMED, MANTIS, Toxline' PASCAL'
trials -itf,oit significant results.,o The low BIoL, science Citation Index, clscoM, British
methodological quality of many trials is another Homeopathic Library, the HomeoPathy A!:tracJ Pagel
important s'ource of bias." These biases are more likely Homlnform Homoeopathic library' NCCAM' and
726
vol365 Auqu5t27,2005
Arai.b,
SIGLE. The search terms
in MEDLINE were
(homeop'k
OR homoeop* OR llomeopathy (MeSH))
AND (placebo*
OR placebos {MeSH) OR placebo effect (MeSH) OR
sham). Search terms for the other databases were much
the same. We also checled the reference lists ofrelevant
papers, including teviews and meta-analyses of
homoeopathic interventions, and contacted experts in
pu
blicatio.5 ld.n tilied
the specialty. There were no language restrictions.
We searched the Cochrane Controlled Trials Registet
to identify placebo-controlled trials of conventional
medicine. This bibliographic database of controlled trials
is maintained by the Cochrane Collaboration. As part of
an international effort to search systematically health-
17
inj0fficient inlomation
14 inellgible study design
care journals worldwide and other sources of
information, the collaboration has combined results of
electronic searches and searches by hand io create a
2
rc homoeoprthlc inleNent on
comprehensive database oftrials.'r We searched issue 1,
2003, ofthe Cochrane Controlled Triais Register, which
included 353 809 billiographic references.
studyseledion
We defined inclusion and exclusion criteria a priori and
applied the same criteria to trials ofhomoeopathy and of
conventional medicine. lnclusion c te a were: that the
trial was controiled and of treatments o. preventive
measures with clinical outcomes; that it had a parallel-
group design with placebo control; that there was
nndom o! quasirandom assignment to treatment and
placebo groups; and t}lat a wyitten report (eg, journal
publication, abstract, thesis, conference proceeding,
unpublished repolt, book chapter, monograph) was
available with sufficient data to aliow the calculation of
odds ratios. Ve excluded trials of homoeopathic
"provings" in which remedies are given to healthy
individuals to assess thef effects, cross'over trials, and
figure 1j ldentifcation of 110 eligibl€ placebo<ontrolledtdakof
homoeopathyth.t.ould b€ makhed to an equal number of pla(ebocontrolled tdak of conventional medicine
population, intenertion, outcome measures, and trial
quality. Data wele ertracted independently by two
obsewers, and discrepancies were resolved by
consensus,
Homoeopathic interventions were defined
compiex homoeopathy, or
isopathy. Classical homoeopathy was defined
classical, clinical,
or
as
as
as
comprehensive homoeopathic history-taking, followed
by the prescription of a single individualised remedy,
possibly with subsequent change ofremedy in response
N-of-1trials.
to changing
Procedures
homoeopathic history was taken and all patienis
received a single. identical remedy, inreraenlions were
Ve
used prespecified criteria to identify outcomes for
inclusion in the analyses. The first choice was the main
outcome measure, defined as the outcome used for
sample-size calculations. If no main outcome was
specified, we selected other outcomes, in the order:
patients' overall assessment of
improvement;
symptoms.
If no
comprehensive
classified as clinical homoeopathy. Complex
homoeopathy was defined as the prescription of a
mixture of several different remedies. Interventions
were classified as isopathy if the agent that was judged
to be the cause of the disorder was used (for example,
pollen in pollinosis). hldications fo! tleatment were
physicians' overali assessment of improvement; and the
classified as acute or chronic or primary prevention or
clinically most relevant other outcome measule {for
example, the occurrence or duration of an illness).
prophylaxis {interventions
with the intention
if several were
ludged equally relevant. For each homoeopathy trial, we
identified matching trials of conventional medicine that
Respnatory-tn.t infe.tions
21(19%)
enrolled patients with similar disorders and assessed
6ynae(olog, and
obneti6
14(13%)
Outcomes were selected iandomly
simijar outcomes. We used computer-generated
random numbers to select one from several eligible
16(15%)
5urqeryand ana€sthet.s
12(11%)
72177%)
M!sculdkeletaldnode6
11(10%)
trials ofconventional medicine. Outcomes were selected
and triais matched without knowiedge oftrial results.
We used a piloted data-ertraction sheet, which
covered descriptive
information on the trial and study
wthelancet.com Vol356
AugUst27, 2oo5
10
(9*)
14{11%)
rbl" t: Dlrtrlbrtb*I p""-f
plac
of
I
-
I
o*,o*
Homoeopathytrials
{n=11o)
(10-1573)
(111)
55.5
777
1992(1966-2003)
Median ye& of publicaiion (.ange)
58{53%)
94(85%)
45 &1%)
lnEnglish
MEDLINE-indered jounal
54149%1
76124%)
77179%)
6(sv.)
ovedll a$e$ment of rerponse
o..urcn.e ordu.aiion of d sodet
A$e$m€ntofsrmptomt
Mearurcment of function or nate
ratios below 1.0 indicated a beneficial effect oftreatment
in ali cases. We used descriptive analyses to compare
characteristics of homoeopathy and conventionalmedicine trials. We examined heterogeneity between
trials with standard 1' tests and calculated I': statistics,
which measure the proportion ofvadation in treatment
effect estimates due to between'strldy heterogeneity.r6
We investigated the association between study size and
trial results in funnel plots, by plotting odds ratios on the
horizontal aJds (on a logarithmic scale) against thei. SE
on the vertical axis.'7 The extent to which studylevel
variables were associated wit}l log odds ratios was
examined by itting of univariable and multivariable
meta-regression models." The following variables were
considered: SE oflog odds ratio, Ianguage ofpublication,
indexing of the publication in MEDLINE, trial quality
Conventional-m.dicin.triak
(n-110)
65
02
1t67)
131(216)
1994(1974-2002)
94(8s%)
110{1oo%)
95 (85%)
49145t4
(24. )
26
26 Q4%)
6{5%)
A$e$nent of dini(al 5i9n,
D6dibed ar
double'blind
Etiof of allo.ation seq! en ce
Adequate co..ealm6t of alloGtion
Adeq
u
ate gene
Analysi, by intention to
Hiqherquality'
'fiah de{ibed 6
101{92%)
27 (25%)
t@at
96187%)
to t)7%)
49 (45%)
21(19%)
33 (30%)
40 (36%)
21(19%)
doubh-blind, with ad€qste
e
9eh!6ton ofall@tion 5eqwne
T06r. 2: chah<teri5ticr ofiola<abo-controlled
t
preventing
(8'")
and adequate rcn.ealment ot
all(ation.
alr othom@opathy and conventional medicine
the
occurrence
of a disorder
or
complication). The dwation of followup was measured
in weeks from the start of the treatment to the
assessment of outcomes.
Assessment of study quality focused on three key
domains of internal vaiidiry:'r'' randomisation
(generation of allocation sequence and concealment of
allocation), masking (of patients, therapists, and
outcome assessors), and data analysis (by intention to
treat or other). Random-number tables, computergenerated random numbers, minimisalion. coin-
{orwebappendi(es
.r
r'
1 and 2
benefit of the inteflention. Funnel-plot asymmetry was
measured by the asymmetry coemcient: the ratio ofodds
in
assignment envelopes, central
Role ofthe funding source
The funding sources had no role in the study design;
collection, analysis, or interpretation of data; or the
writing of the report. The corresponding author had fuli
access to all the data in the study and had final
responsibility for the decision to submit the paper lbr
randomisatron,
computerised
randomisation systems were classifled as adequate
methods of allocation concealment. Analysis by
intention to treat was assumed ifthe reported number of
r'.i .
than 1 0 corespond to a smaller odds ratio for trials
with the chalacteristic and hence a larger apparent
ratios per unit increase
identical appearance, and on-site
r
odds ratios, raiios of odds ratios, or asymmetry
coefficients with 95% CL Ratios of odds ratios of less
tossing, caid'shuffling, and lot-drawing were classified
as adequale methods for the generation ofthe allocation
sequence. Sealed, opaque, secluentially numbered
independendy prepared and coded drug packs of
See
{masking, generation of allocation se<1uence, concealment of allocation, intention-tolreat analysis), duration
offollow-up, and clinical topic. For homoeopathy trials,
we also examined whether effects varied between t,?es
ofhomoeopathy and tlpes ofindications {acute, chronic,
primary prevention, or prophylaxis).
We combined treatment effects from larger trials of
higher quality by use of standard random'effects metaanalysis and used meta-regression analysis to predict
treatment effects in trials as large as the largest trials
included in the study. Trials with SE in the lowest
quartile were defined as largertrials. Results are given as
analyses were done
in
SE of log odds ratio.'' All
Stata version 8.2.
palticipants randomised and the number analysed were
identical. Descriptions of other methods wele coded
either as inadecluate or unclear, depending on the
amount of detail provided. Trials described as doubleblind, with adequate methods for the generation of
allocation sequence and adequate concealment of
allocation, were classified as of higher methodological
60 reports. The commonesl reasons for exdusion were
insumcient information (precluding the calculation of
quality.
odds ratios), ineligible study design,
G raphical an d statistical a n alysis
We expressed results on the odds ratio scale and used
the method descdbed by Hasselblad and Hedges" to
convert difTerences in continuous outcomes to odds
ratios. We lecoded outcomes if necessary, so that odds
publication.
Results
We identified 165 potentially eiigible reports ofplacebo-
controlled
trials of
homoeopathy
and
excluded
multiple
publication, and inability to identify a matching trial of
conventional medicine (figure 1). We induded
105 publications that repor(ed on a total of
110 independent
trials ofhomoeopathy (webappendix
1)
and 110 publications of 110 matched trials of
conventional medrcine {webappendix 2).
{wwrhelancet.com vol355 Alqust27,2oo5
7
O,ai.l",
in
MEDLINE {mo!e
The clinical topics shrdied in paits oftdals ranged from
respiGtory infections to surgery and anaesthesiology
{table 1). The outcomes studied were closely matched;
overall assessments ofresponse were analysed in 49% of
homoeopathy trials and 45% of trials of conventional
other than English), indexing
medicine (table 2). More detailed information on
but did not reach statistical significance (table 3). There
was little evidence that treatment effects varied
outcomes is given in the webtable. The average sh-rdy size
was similar for the two groups, with a median of around
65 paticipants. Overall, str.rdy size ranged from ten to
1573 padicipants. Among homoeopathy tlials 48 (44%)
concemed dinical homoeopathy, 35 (32%) complex
homoeopathy, 18 (16%)dassical homoeopathy, and eight
al, the nature of the
\7%J isopathy. For the remaining
homoeopathic intervention was unclear. 101' 192%) ofrhe
t
conventional-medicine trials investigated drugs, eight
(7oZ) immunotherapy, and one a vaccine. The drugs most
frequently tested were non-steroidal anti"inflammatory
agents (11 trials), anti-allergy drugs (11 tiials), virostatic
drugs (11 trials), and antibiotics (seven trials).
53% of homoeopathy trials were published in English
compared with 85% of trials in conventional medicine.
50 homoeopathy trials were published in German or
French. The two groups of trials aiso differed in the
proponion published in MEDLINE indexed journals.
The two groups had similar methodological quality in
terms ofmasking, generation ofallocation sequence, and
analysis according to intention to treat, but a highe!
proportion of homoeopathy trials reported adequate
concealment of patients' allocation. 21 (19%) homoeopathy trials and nine (8olo) conventional-medicine trials
were ofhigher quality (table 2).
Most odds ratios indicated a beneficial effect of the
intervention {frgure 2). SE ranged from 0 12 to 1 65 for
homoeopathy trials and 0.11 to 1.52 for conventionalmedicine h'ials. Heterogeneity of trial results was less
plonounced for homoeopathy (heterogeneity x'=309, df
i09, p<0.0001) than for
beneficial effects
and indicators
I
in trials not indexed in MEDLINE),
oftriai quality (more beneficial
effects
in
trials of lower quality). The effects of these valiables
were generally similar fot conventional-medicine trials
according
to
duration
of follow'up (p:0.862
fot
homoeopathy, p=0 594 for conventional medicine) or
clinical topic {p=0 660 for homoeopathy, p=0'360 for
conventional medicine) or that effects differed between
different lypes of homoeopathy (p=0.636) or type of
indication (p=0 487). In multivariable analyses, the SE
of the log odds raiio (asymmetry coefEcient) was the
dominant variable in both groups. Coefficients ofother
l
:o
03
06
g
09
7)
15
conventional medicine
(heterogeneity x'=481, df 109, p<0 0001) This
difference is unlikely to be due to chance {p:0 011 by
F test). The proportion oftotal variation in the estlmates
of treatment effects due to between-study heterogeneity
(I?) '"
was
6 5
% for hom oeoparhy aad 77/o for conventional
06
medicine.
Funnel plots were asymmetrical, with smailer trials
(largo SE) in the lower part of the plot showing more
benefrcial treatment effects than iarger ttials (smaller SE,
figure 2). In meta-regression models, the association
between SE and treatment effects was similal for trials of
homoeopathy and conventional medicinel the respective
asymmetry coefficients were 0 17 l95Yo Cl 0 10-0 12)
and 0 21 {0 11-0 40). Therefore, with each unit increase
in the SE, the odds ratio decreased by a factor of0 17 for
homoeopathy and 0.21 for conventional medicine
g
a
09
\2
15
rO'
a'a
18
{table 3).
Other sources of heterogeneity between homoeopathy trials included the language of publication
(more beneficial effects in trials published in languages
[email protected] vol365 August27,2oos
t
ak
Funn.lplotof 11o homo.oP.thytials and 110 matched convehtionalmedkin€
solid lines indicate predktedt,eatmenteffectsfrom metajeg.ession, with dotted lines rcPr€senting the95% cl
Figurc 2:
I o*,0*
conventional medicine, are unspecific placebo or
R.tl"
(9516
Arymmetry .o€f6.
.ntt
(D
0.17(0.10-0.32)
073 (o 53-1oo)
0.69(0.50 0.94)
o 44
Con..alment of allo.arion
"f
(95%Ct)
(o.11-o.81)
"dd-"ii.*
<0.0001 0.21{0.11-0.40)
005
oo19
0 67 io
40-1 14)
103 (061-1.75)
0017 061(0.36-1.11)
o?4
098(065
146)
0.107
either type of medicine were oflow or uncertain quality.
In both groups, smaller trials and those ofiower quality
showed more beneficial treatmeni effects than larger
trials and those of higher quality. Between-trial
heterogeneity was less pronounced among homoe-
o91l
067(028-0.95)
o
0.78(0.57-1.07)
0117 076(o,E-116)
0193
1.25(0.87-1.80)
0.225 1.14(0.78-1.56)
o 506
0.62
0.011 0.61(0.34-1.09)
0.095
{043-0.90)
conterl effects.
In 1991, Kleilnen and colleagues')o argued that there is
no reason to believe that compared with homoeopathy
"the influence of publication bias, data massage, bad
methodology, and so on is much less in conventionai
medicine". Indeed, we fourd that trials ofhomoeopathy
tended to be of higher methodological quality than
conventional-medicine tlials, altlough most t als of
p
opathy trials. This finding might
be
expected
if
heterogeneity between homoeopathy trials is essentially
due to biased repoding and conduct oftdals, whereas in
belw 1.o @respondto a 5mall€r odris 6tio
sti. dividrd by od& Elio witho'rt {haad.'i3tk.
^atiorT'ials pubkh.d in lanluaqB other than E.qlish
lrirlr with .haadennk and hen(e a larqer apparenr h€n.fit ofihraNeniions.
show-r moft 6ei.n.ial trcatment elt .t th:n th6e publish ed in Ens li$, lor erample. rRario of odds htio p( lnit in.rca* in st
'od6atiowilhchac.te
for
the conventional-medicine sample treatment effects
represented an additional relevant source of heterogeneity. When we discussed resuits with practitioners of
homoeopathy, they contended that classicai homoeopathy and homoeopathic tteatment
I"bt"
homoeopathy rnd 110 matchid trials otronventioftl medicine
of
chronic
yield
specific effects. We addressed these points in additional
analyses but found no strong evidence in support of
disorders,
in tials with longer follow-up, would
these hypotheses.
variables, including srudy quality, were attenuated and
became non-signi6cant.
When the analysis was rest cted to the larger trials of
highe! repoded methodological quality, the odds ratio
from
random-effects meta-analysis
was
0
88
(0.65-1.19) based on eight trials of homoeopathy and
0 58 (0 19-0.85) based on six trials of conventional
medicine. Similarly, for prediction of treatment effects
in trials as large as the largest trials, the odds ratio was
0 96 {0.73-1 25) for homoeopathy and 0.67
(0 48-4.91) for conventional medicine.
This study directly compared the presence of biases
in homoeopathy
and conventional-medicine trials. Identical definitions
and their influence on effect estimates
were used, and data were abstracted independently by
two obseflers. The search ofhomoeopathic publications
was comprehensive, and we are confident that we
identified a near-complete set of published placebocontrolled trials of homoeopathy. The identification of
unpublished studies is notoriously difhcrit, and we
probably missed some of these t als. Conventionalmedicine
t
als were randomly selected from the largest
existing database
of clinical triais (the
Cochnne
compared the effects of homoeopathy and
conventional medicine that are seen in placebo'
controlled tials, examined the presence of bias
resulting from inadequate methods and selective
Controlled Trials Register) and were carefully matched
to homoeopathy triais fo! clinical subject and type of
outcome.
Diffelent sources of bias are difficult to disentangle.
The methodological quality of randomised rtials cannot
publication, and estimated results in trials least affected
be reiiably
Discussion
we
by these biases. We assumed that the effects observed
in
of homoeopathy could be
combination of methodological
deficiencies and biased reporting. Conve6ely, we
placebo-controlled trials
explained
by a
postulated that the same biases could not explain the
effects observed in comparable placebo.controlled t als
of conventional medicine. Our results confirm
these
hypotheses: when analyses were restricted to large trials
ofhigher quality there was no convincing evidence that
homoeopathy was superior to placebo, whereas for
conventionai medicine an important effect remained.
Our results thus provide support for the hypothesis that
the clinical effects of homoeopathy, but not those of
assessed
from published articles
because
reporting on impofiant features of the methods is
incomplete in many cases." Indeed, deficiencies in
methods of smaller trtals that were either not reported or
not assessed by us could also have conhibuted to the
asymmetrical shape of the funnel plot. We have argued
elsewhere that the funnel plot should be seen not oniy as
a means of detecting publication bias, but also as a
generic tool for examination of small-study effects*the
tendency for the smailer studies to show larger
treatment effects.l'] Ifreporting is inadequate, study size
can be a more plecise measure of t al quality than
formal assessments of trial quality. Ve addressed this
possibility by modelling the effects expected in
t
als as
ww.thelancet.(om Vol366 August27,20os
O,ai.b,
large as the largest trial included in our study; again, we
found litde evidence for an effect of homoeopathy but
stlonger evidence for conventional medicine. Another
limitation of our study is the exclusive focus on the
beneficial effects of homoeopathy and conventional
medicine, nther than on both benefits and risks.
However, the trials included in the study were smali and
lacked the powe! to reveal infrequent but important
adve$e effects. Futhermore, leporting on adverse
effects
is
inadequate even
in larger trials.,' A
comprehensive and vaiid assessment of adverse effects
wouid probably not have been possible within the
framework of this study.
A previous review, which did not include a metaanalysis, also found that many trials of homoeopathy
show beneficiai effects but are of low methodological
quality.'o A meta-analysis by Linde and co-workets', was
based on an extensive literahrre search, which we
updated for ou! study, but it did not include trials of
conventiooal medicine. These lesearchers concluded
that theii results were "not compatible with the
hypothesis that the clinical effects of homoeopathy are
completeiy due to placebo". However, in a subsequent,
more detailed analysis of the same data,,, they observed
that mole ligorous trials yielded smaller effect sizes and
tlat thefu meta-analysis" ptobably "at least
overestimated the ellects of homoeopathic treatmenls".
ln a separate study, the same group obse ed that many
trials
in
complementary medicine have important
methodological weaknesses.'zt Finally, a study of23 trials
of homoeopathy that were considered to be of high
methodological quality found that the few trials thar
used objective endpoints were all negative.To
Our study has implications beyond the question of
whether homoeopatlic remedies have specific effects.
First, an important point to keep in mind is that most
systematic reviews and meta-analyses ale based on
relativ+ few tdals. Simulation studies have shown that
detecdon of bias is difhcult when meta,analyses are
based on a small number oftrials." For example, for the
eight t als ofhomoeopathic remedies in acute infections
of the upper lespiratory tract that were included in our
sample,
the pooled effect indicated a
beneficial effect (odds ratio 0.36 195% CI
substantial
0.2
.500
and there was neither convincing evidence offunnel-plot
asymmetry nor evidence that ttre effect di{fered between
the trial classified as of higher leporled quality and the
remaining trials. Such sensitivity analyses might suggest
that there is robust evidence that the treatment under
investigation works. However, the biases that are
prevalent in these publications, as shown by our study,
might plomote the conclusion that the results cannot be
trusted. We submit that similar studies shouid be done in
other 9?es of both complementary and conventional
medicine. Such shrdies would "borow strength" from a
large number oftrials and provide empirical information
to assist reviewers and reade$ in the interyretatio4 of
wwwthelancet.com Vol 355 August27, 2oo5
findings from small meta-analyses that focus on
a
specifrc intervention and disorder. Second, although
impotant progress has been made lately,'r,7 further
research is needed to identify the dimensions of
methodological quality that aie important in different
clinical contexis, diflerent outcomes, and different types
ol trials. Finally, the relation between the probability of
publication of a study and its methodological <luality
should be examined in more detail.
We emphasise that our study, and the trials we
examined, exclusively addressed the narrow question of
whether homoeopathic remedies have specific effects.
Conte* effects can influence the effects ofinterventions,
and the relationship between patient and cater might be
a$ important ?athway mediating such effects.!*,,
Practitionels of homoeopathy can form powelful
alliances with their patients, because patients and carers
commonly shale strong beliefs about the treatment's
e{fectiveness, and other cultural beliefs, which might be
both empowering and restorative.b For some people,
therefore, homoeopathy could be another tool that
complements conventional medicine, whereas othen
might see it as pwposeful and antiscientific deception of
patients, which has no place in modern health care.
Clearly, rather than doing further placebo-controlled
trials of homoeopathy,' futr-rre research efforts should
focus on the nafure ofcontext effects and on the place of
homoeopathy in health-cale systems.
Our study powerfully illustates the interplay and
cumulative effect of different sources of bias. We
acknowledge that to prove a negative is impossible,', l:ut
we have shown that the effects seen in placebocontrolled tdals ofhomoeopathy are compatible with the
placebo hypothesis. By contnst, with identical methods,
we found that the beneits ofconventional medicine afe
unlikely to be explained by unspeciEc effects.
M Eggerconceived the studyand wrote the 6rst draft oftherelort.
All the authors contributed to the final draft. A ShanB
K HuwilerMiintener, L Naltey, S Dijrig, and P riini did the literature
searches, ideniified eligibie studies, and ext.acteddata. P jiini advised
on data enraction and quality assessnent. D Pewsner helped with daia
efiraction and classifcarion ofhomoeopathy irials. A Shang,
I A C Sterne, P Jiini, and M Egger did the siatistjcal analyses and
conlributed to daia inierpretation.
We dedare thatwehave no conflict ofinterest.
We thank
lritz
Grossenbacher iorvaluable helpwith literature
searches. This studywas funded by the Conrplementary Medicine
Evaluation Progiam (Programm €valuation der Komplementrrmedizin
IPEKI) of the Swiss Federal office ror Public Heahh. we thank
Maiianne Amiet and Florian Mitscherlich from the PIK coordinating
oltrceand Felir Curtne! froft the Fede.al Oifice ofPublic Health for
lheir suppon. Peter r0ni was supported by sranis from the Swiss
NationalScience Foundalion (grants no. 3231066177 and 1200066178).
1
risenberg DM, Davis RB, Ettner SL. etal. Trends in aLtemative
medicine use in the United States,1990-1997:results ofa follo*
up national suney. _IAMA 1998t 260: 1561-75.
I
7
I o*n,*
2
Emsl E. The role ofcomplementary and altemative medicine. BM.l
2000:321: l1l3-ls
Vandenbroucke ,P. Homoeoparhy t.ials:going nowhere. tara!
3
1997:350:824
Viclers A. Zouman C ABC of(omptemFnrary medkine:
homo€opafiy. 8Ml 1999: ll9. l1l5-18.
lonas w, jdcobs l. Heal.ns wifi horreopath]. New york. w"rrer
4
5
Bo.k(
6
190a.
Schulte J. tfie.ts of porenrizarion in aqueous sotufions.
tu Honeopath I 1999. EE: t5540.
7
Skrabanek P.Is homoeopathya placebo responsel
1107.
8
9
Gotzsche PC. T.ials ofhomeopathy. tana, 1991; t4t: 1513.
Rennie D. Fan conduct and fair repolting of dinicat trials. JAMA
I'bc.tt9g6tz.
17664a.
Egger M, Davey Smiti c. Meta-analysis: bias in locationand
sel€ction of studies. BMJ 1998j 116: 61-{6.
Schulz KI, Chalme6 I, Hayes Rl. Altman D. Empirical evidence of
t9
101-05.
20
Kleijrcn l, Knipschild p, ter
BMI 1991,t 302:31613.
2l
Schulz KF. Randomised trials, human narure, and reponinS
guidelines. lare, 1996; 3.1E: 596-98.
StemeJAC. Cavaghan Dt. Egger M. publ'cation and retated bias in
mela{nalysis. po$e! ol sr!-rsdcel lesrs dnd prevrten(e rn rhe
lle?turc J Clin Epd.tuial 2000: 53: Ilt !2i
JoaDnidis tP, ku t. completeness of safery.eporting in
randomized trials:an €valuation of7 medical areas. JAMA 20011
22
23
11
bias: dimensions of helhodological qualiry a$ociated with
estimates of treatment effects in conbolled trials..IAMA 1995;273:
408-1,2.
12
ll
14
Linde K, clausius N, Ram;ez c, et al. Are the clinicalellects of
honoeopathy placebo effectsi A meta-anelysjs ofplacebo,
controlled trials. tan rr l997i 350: 834-41.
Di&ersin K, Manheimer E. Wieland 5. Robinson KA. Lefebqe C.
McDonald S. Development ofthe Cochnne Collaboration's
CENTRAL Register of contolled clinicaltrials. E al H.atrt p/ot
ZOO2: 25. 1844.
Jijnr P. Alrman DC. Egger M. Assesrng the qualiry oj .onrrothd
clnical trials. 8Ml 2001:lZl: a2-a6.
15
16
17
l8
732
Hasselblad v, Hedges Lv. Meta-analysis ofscreening and
diagnostic tests. Prydrrol ErlL t995; t77: t67-78.
HiggiDs lPT, Thompson SG. Quantjrying hererogeneityin a metaanalylis. Sr4t Mrl 2002; 21: l519-58.
Steme tAC. Egger M. Funnelplots for deteding bias in metaanalysis: guid€iines on cho\.e of aas. I clin Epidemiol 2oot: S4l
Rier
c. CliDicaltrials ofhomoeopathy.
zES:4J743.
24
1999: 242:
10
StemerAC, EggerM. Davey-Smjth C. lnvesligating add dealing
with publication andotherbiases in meta-anatysis. BMJ2001; t3:
25
16
27
28
Linde K. Scholz M. R2mire c, Clausius N, Melchalt D,lonasWB.
lmpa.r ofstudy quahry on ourome in Dlrcebo-conLrolted lnats of
homeopathy.I Clin Eyid,hol 1999, 52: 6tl-36.
Linde K, ronas wB. Melchan D, wiilich S. The nethodological
quality of randomized controlled trials of homeopathy, her6al
medicines and acupuncture . Itt J Epid.hinl lWL 30:
526-31.
Morison B. trlfold R,. Imsi E. Mehodologka nsour end rpsutrs
of clinical lrials of homoeoparh'c remedies. p.tlstoa,,000j ll.
132-38.
Moher D. Pham B. lones A. fl al. Does queliry or relons of
!Jndomised rruls affed estrmales ofintFtuenl'on efficacy leported
in meta-analysesf lrrrr, 1998; 3 52: 609-13.
Di Blasi z, Harkness €, Ernst E, ceorsiou A, Kteiinen J. jnfluence
ofcontext effecis on health oulcomes: a systematic review. l4rrrt
20011 357,
29
75742.
KleiinenJ, de Craen Al. van Everdingen l, KJol L. placebo effecr in
doubl€-blind clinical bials: a rwiN of inleractio,s wirh
medications.
l0
31
bnet
1994:
!4L rJ4749.
KaplLhukTJ. Fiserberg DM. Thp pF (uanvF ppea ofahernarve
"
ntedicne Ann t^km M.d1998. l29 t0bl-65
Allrnan DC, Bland JM- Absence of evidence is nor evidence of
absence. BMl 1995; 311 a85.
1046_55.
Thompson SC. Sharp sJ Er?leining helerogenFiry in mera.nalysis:a comp!rison o[merhods. Srar M.d tr9q; lE:
2693-708.
M.thelancet.coh vol366 Auqun27,zoos
EXHIBIT B
ffi
I Clin EpidemiolVol. 52, No. ?, pp. 631-616, 1999
Copyright @ 1999 Eisevier Science Inc. All righb reserved.
0895-4356i99/$-see 6ont marter
P1l 50895.4356(99)00048-?
ET.sEVIER
Impact of Study Quality on Outcome in
Placebo-Controlled Trials of Homeopathy
Klnus Linde,l
'* Michnel
SchoLT,z
Gilbert Ramirez,3 Nicola Clausius,l Dieter Mebhart,l
andWayne B. lonasa
Mrorcrns Rrsranca, Depanrr.asxr or lNrgRNaL MrotcrNr Il, TecuNrscuo
2lls,rt].urr ron Msorcer Srerrsrlcs awo ErtoevloLocv, Trcnr.rIscH! IJNIVERSITiT
MuNcasN, Muwrcn, Genuerqv; lDeeexturur or PugI-rc HeeLtH aNo PnrvrNrve Meotcrvs, l-lvlvensl,tv or Nonrntnr Trxas
Hsarrg ScrrNcr Cruren, Fo*r Woern, Trxes; ar.ro 4Onrtce oe Arrsrwarrvp Meotctle, NartouaL Insrlrures or HeaLttt,
IMUNcHENER
MooELL
CeNrR-E eon CoN{prsrlrNrany
UNlvEFsrrAT MiNcHeN, MuNrcn, GeRMeNy;
Be.Iursoe, M.rnvu+r.ro
indicarors of methodological quality on study outcome in a set
in rhree dif{erenr ways: (l ) The results ofstudies meeting
placebo-controlled
trials
of
homoeopathy
clinical
of 89
single crireria (explicit statement of random allocation, allocation concealment, double,blinding, compieteness
of follow-up) of methodological quality were compared with those of studies not meeting the criteria in
univariate and multivariate analyses; (2) The results ofstudies scoring above and below predefined scores in two
quality assessment scales were compared; (3) Primary studies were consecutively entercd into a cumulative metaanalysis according to the summary scores derived from the quaiity assessment scales. All analyses were performed
lsing meta-regression methods. Studies that were explicitly €ndomized and were doubie-blind as well as studies
scoring above the cut-points yielded significantly less positive resuks than studies not meeting the criteria. ln rhe
cumulative meta-analyses, there was a rrend for increasing effect sizes when more studies wich lower-qualiry
scores were added. However, there was no linear relationship between quaLity scores and study outcome. !0e
conclude that in the study ser investigared, rhere was clear evrdence that studies with better methodological
ABSTRACT. !7e investigated the influence of
quality cended to yield less positive resuits. Because summarizing disparate study features into a singl€ score is
problemaric, meta-regression methods simultaneously invesrigating the influence of single study features seem
rhe best method {or investigating the impact of study quality on outcome. J cLrN EprDEMloL 52:7:631-636, 1999.
@ 1999 Elsevier Science Inc.
KEYWORDS. Study quality, meta,analysis, homeopathy, randomized controlled
INTRODUCTION
There is increasing evidence that more rigorous trials tend
ro yield less optimistic results than trials with less precau'
tions against bias [1-J]. ln a controversial area like complementary and alternative medicine, in which many trials are
performed by proponents with little research experience
and published in non-peer-reviewed journals, one might
expect that this problem could be even more pronounced.
Quality assessment, therefore, should be a mandatory element ot !yslematic revrews ofsuch rherapies
However, the assessment of clinical trial quality is problematic. Although a number of instruments designed for
this purpose have been used (see [4] for an overview), there
*Addres conespondence to: Dr. K. Linde, Centre for Complementarv
Medicine Research, Depanment o( Internat Medicine ll, Technische Univesiriir Miinchen, Kaiserstt. 9. 80801 Munich, Gerrnanv.
Accepted for publicarion on B March 1999.
crials, bias
is no widely accepted single method. There is not even a
uniform definition of quality. Most assessment methods focus on criteria dealing with the likelihood of bias such as
randomization, blinding, completeness of follow-up, and
intent-to-treat analysis. The "dimension"
assessed
with
such criteria is that of "methodological quality" or "internal
validity." There is also substantial controversy over whether
qualiry can be assessed with scoring systems that combine,
weight, and sum a catalogue of critetia, or whether the in'
fluence of single-quality components should be investigated separately [5,6].
We compared three different approaches to investigate
rhe impact of quality aspects on outcome in a published
meta,analysis of placebo-controlled trials of homeopathy
[7]: ( I ) investigating the influence of single-quality components on the outcome, (2) using cut-off points in quality
scores as inclusion criterion, and (3) entering trials into
meta-analysis consecutively according
(cumulative meta-analysis).
to quality
scores
632
K. Linde
METHODS
The general methods of our review have been described in
detail elsewhere [7] and are reported here only in a cursory
manner. In this article, we focus on the specific methods regarding study quality and outcome.
Selection Ctitetia ond Literatute Seatch
\fle included all available double'blind and/or randomized
clinical trials in which a homeopathic intervention and a
placebo had been compared for preventive or therapeutic
purposes. Eligible trials were identified *tough multiple sources
inciuding MEDLINE, Embase, complementary medicine
databases, contacts with researchers, and checking bibliographies of identified articles. A rotal of 1 19 studies meeting
the inclusion criteria were identified, and 89 presented suf-
er
al.
been used in a slightly modified version in other rcviews on
complementary medicine also [9,10], has seven items scor-
ing 0,0.5, or 1 point each. ltem I is a statement ofrandom
allocation; item 2 assesses the adequacy of randomization
concealment; item 3, baseline comparability; irem 4, blinding of patients; item 5, blinding of evaluators; item 6, the
likelihood of selection bias after allocation; and item 7, the
adequacy of the statistical analysis. Both scales were operationalized with detailed instructions (available from the authors). Al1 trials were assessed by at least two independent
reviewers. Each reviewer used both quality scales.
Stctisticol Anafses
Odds ratios (OR) and their respective 95olo confidence intervals (957o CI) were calculated for each trial, with values )
ficient data to be included in the quantitative meta-analy-
I
sis. Study characteristics and results were extracted by two
independent reviewers using a pretested form.
effective than placebo. For this analysis, the data from the
original meta-analysis l7l were reanallzed by using metaregression methods allowing arnong-trial heterogeneity.
As s es sment
of Methodotogical
First, ORs were naturai log-transformed before analysis.
Qtality
For assessing the methodological quality, the scale by Jadad
tailed description submitted for publication), which has
l.
treatment effect y, : ln(OR') ofstudy i, i =
1,...,n. \7e assumed the meta-rcgression model [11,12]
This results in
et ai. [8] and a system developed by one of us (KL) were
used. The Jadad scale is a scoring system that has three
items adding up to a maximum score offive points. Zero, 1,
or 2 points can be given for randomization (explicit statement that allocation was randomized and description of an
adequate generation of the random sequence), 0, 1, or 2
points for double-blinding (explicit statement that patients
and evaluators were blinded and thar treatments were indistinguishable), 0 or 1 point for the description of dropouts and withdrawals (numbers and reasons in all compared
groups given separately).
The second scale (lnternal Validity Scale = IVS; de-
TABLE
indicaring rhar the homeopathic rnterventron was more
a
Y=
x;F + bi+
ei,
(1)
where x, denotes a vector of known covariates for study i,
- N(0,r2) is a study-specific random effect reflecting the
residual among-trial heterogeneity and e, - N(0,o'z) with
the individual within-trial variance o,2. The variance o,z
was estimated by the known sample variance sj of the ith
trial. For b,, e, independent var(y,) equals .r2 * s'2. Under
this model assumption, the estimated linear predictor r,p
rcpresents the mean effect ofall triaLs having covariate vector x,, and b, is the ith rrial's deviation from this mean. Parameter estimares were obtained via a likelihood-based
method (resrricted maximum likelihood, REML [13]) imb,
Component and minimum 6core analysis
Not met
n
Component analysis (univariate)
Explicirly randomized
Adcquate concealment
Double-blinding
Complete follow-up
Component analysis (multivariate)
64
34
81
78
(1.81-2.75)
25
2.60)
(2.124.33)
z.tt (t.62-2.74)
Adequate concealment
2.08 (t.7 5-1.47 )
1.70 (t.97-3.72)
Double-blinding
Complete follow-up
Minimrlm score analysis (univariate)
Both
z.z3
2.00 (1.50-2.65) 55
2.18 (1.83
8
6l
3.03
2.rs (t.78-1.59)
Explicitly randomized
Jadad score > 2
lV scale > 4.5
OR (95olocl)
40
r.81
34
z6
1.95
t.'tz
(1.4r-2.32)
(1.50,2.59)
(1.28-2.3t\
49
55
63
oR (95ol.Cr)
ROR(95ol.CI)
Heterogeneity
3.4A (2.324.97)
2.83 (1.23-3-58)
9.14 (4.81-17.4)
2.29 (1_84-2.86)
0.66 (0.43-1.0r)
0.71 (0.49-1.02)
0.24 (a.t74.46)
0.41 (0.21-0.8?)
3.36 (2.374.78)
2.s0 (2.00-l.l l )
7.85 (4.10-15.0)
2.21 (1.81-2.69)
0.64
0.84
0.26
1.73
3.23 (2.534.13)
2.92 (2.29-3.73)
2.95 (2.37-3.67)
0.56 (0.40-0.79)
0.6? (0.474.97)
0.58 (0.40-0.79)
0.r9 (0.22-0.87)
r.r I (0.88-2.00)
0.31 (0.18-0.71)
0.44 (0.26-0.91)
(0.43 {.94)
0.28 (0.15-0.69)
(0.60-r.18)
(0.14-O.51)
(0.85-1.77)
0.34 (0.19-0.78)
o.4o (0.23-o.86)
0.36 (0.20-{.82)
Impact of Study Quality on Outcomes in Placebo.Controlled Trials
ofHomeoparhy
633
TABLE 2. Pooled odds ratios of shrdies with the same quality scores and cumulative meta.analysis according to quality
IV Score
(n studies)
7.0 (5)
6.5 (2)
IV
(n studies
1.55 (0.?7-3.10)
:7.0 (5)
5.5 (13)
4.6r (0.9\-23.4)
t -35 (0-74-Z.4sl
l.?4 (1.38-2.19)
5.0 (10)
2.95
4.s (e)
4.0 (13)
3.5 (9)
4.33 (1.86-r0.1)
3.0 (B)
1.97
2.5 (5)
3.91 (0.97-15.8)
2.0 (9)
2.47
(r.rt-5.52)
>3.0 (73)
>25 (78\
>2.0 (87)
1.0 (2)
6.92
(t.53-3t.41
all (89)
6.0 (4)
(2.404.34)
(2.00+.98)
2.82 (t.57-5.05)
3.16
(l.r+3.40)
Jadad
Cumulative
score
oR (9solocr)
(7
)
=6.5
>6.0 (11)
>5.5 (24)
>5.0 (34)
-_4.5 (43)
>4.0 (56)
=t.5
(65)
oR (95%Cr)
)
1.55 (0.77-J.r0)
2.AZ (r .06-3 .85\
1.67
1.7
(1.tL254t
| (1.36-2.15)
1.89 (1.s3-2.35)
2.09
(r.69 2.59)
2.29 (1.88-2.80)
2.35
(1.9+233)
2.29 (1.92_2.'t 4)
2.4212.00-2.9r)
2-44 (Z .02-2.93)
2.47 Q.a6-2.97\
Jadad
7.00 (1.3?-2.91)
t .42 (t .02-t .99)
5 (10)
4 (ll)
3 (1e)
1(r5)
t .83 (t A+2 .34)
3.24 (2.274.63)
3.58 (2.11-6.08)
0 (2t
6.92 (r.53,31.4)
z (32)
=5 (10)
>4 (Zt)
>3 (40)
z.0o
(t.37,2.9t)
1.68
(r.29-2.r8)
>z (72)
2.26 (1.87,2.',l3)
> I (8?)
all (89)
2.44 (Z.OZ-2.93)
r.7311.44-2.08)
2.47 (2.06-2.9?)
plemented in PROC MIXED of the SAS stem release 6.12
(SAS Institute, Cary, NC).
A likelihood ratio test was used to examine the null hypothesis of no residual among trial heterogeneity (r2 : 0)
2. Minimum score analysis: Comparison of pooled effect
sizes of studies scoring above or below predefined cutoff-points (>3 in the Jadad score, >5 in the IVS score,
by comparing the likelihood of this restricted model with
that of a model where rz is estimated. lf the restriction
holds, model (1) reduces to the fixed-effects model. The results are given either on the original OR-scale or in terms of
ratios of odds ratios (ROR). A ROR of 1.0 for a dichotomous quality criterion indicares no difference between the
two ORs. ln our coding lcheme, a ROR < 1,0 always reflecs that trials meeting the quality criterion had a smaller OR
compared with the trial not fulfilling this quality standard.
3. Cumulative meta-analysis: Primary studies were entered
consecutively into meta-analysis according to the quality scores received (Jadad and IV scale).
Approaches to Test the I'7'pact of Q1/,d,LitJ on Outcome
The relationship between study quality indicators and outcome was tested by three different approaches:
1. Cornponent analysis: Comparison of pooled effect sizes
of studies meeting a criterion and of studies not meeting
a criterion (explicit statement that allocation to groups
was randomized, adequate concealment of randomization, double-blinding, complete follow-up, or intent-totreat analysis). Both univariate and multivariate analyses
were performed.
separately and combined).
RESULTS
The mean quality score of the 89 trials was 2.58 (SD 1.29)
from a maximum offive on the Jadad scale and 4.20 (1.46)
from a maximum of seven on the IV scale. Forty (45%) of
the trials scored over the predefined quality cut-off (three
or higher) on the Jadad scale, and 34 (38%, five or higher)
on the IV scale. Twenty,six (29o/o\ vials scored above the
cut-off in both scales. Sixty-four (72%) trials explicitly
stated that allocation to groups was randomized; 21 (24olo)
made no ctear statement (only double-blinding mentioned);
and 4 (4ok) trials had used quasi-random methods. For 34
(38%) trials, an adequate method of allocation concealment (mostly consecutively numbered drug containers) was
reported; 50 (56010) did not describe the method; and in 5
(6%) concealment was inadequate. Eighty-ooe (917o) studies were double-blind;3 (3o ), single-blind; and in 5 (6Eo)
there was no statement about blinding. Twenty-eight
K. Linde
634
(37olo) trials either appeared to have included all patients
randomized into the analysis or did an intent-to-treat analysis; in 20 studies (22%), major bias seemed unlikely; in 19
(210,6) trials major bias seemed possible or likely; and in 22
(25olo), the item could not be rated owing to insufficient in-
formation.
The pooled mean random effects OR all 89 studies included was 2.45 (95Vo CI 2.05-2.93\. Because there was
strong heterogeneity Gt : 0.43;95okCI4.254.90; P = 2.4 x
l0-rl), only random effects analyses are reported here. [f
the analysis was restricted to trials that were explicitly randomized, adequately concealed, or double-blinded, the resulting mean ORs were 2.23 (a decrease of 9olo compared
with when all 89 trials were pooled), 2.00 ( 18% decrease),
and 2.18 (11"/" decrease) respectively (see Table 1). The
117o reduction of the overall OR by excluding eight nondouble-blind trials indicates that these studies had yielded
greatly inflated effect sizes and their removal considerably
decreased heterogeneity (see Table 1, last column). Trials
with complete follow-up yielded a slightly larger mean OR
rhan did rrials not meering rhis criterion.
Double.blinding had the strongest impact on outcome in
both the univariate and the multivariate analysis. The ratio
of odds ratios (ROR) were 0.24 (g5yocl 0.12-0.46; P <
0.0001) and 0.26 (95%CI 0.14-0.51; P = 0.0002), respectively. ln the mulitvariate analysis also the explicit statement of randomization reached significance (ROR 0.64;
95'kCl 0.434.94; P = 0.03). The influence of allocation
er
al.
concealment and complete follow-up was not statistically
significant.
If the analysis was restricted to trials that scored three or
higher on the Jadad scale, five or higher on the lV scale, or
both, the mean OR was decreased ro I.8I (1.41-2.32,28Vo
decrease), 1.97 (1.50-2.59,229o decrease), and 1.72 (1.28231, 3AVo decrease), respectively. The heterogeneity was
reduced most (350/o) in the multivariate component analysis.
\X4ren the primary studies were entered into the cumulative meta-analysis in descending order of IVS scores, there
was a trend for increasing effect sizes when studies with
lower quality were added (see Table 2, upper part). If only
the five studies scoring highest (seven points) were pooled,
the resulting OR was 1.55 (0.79-3.01), and for the studies
scoring more than two thirds of the iterns (>5 points) 1.89
(I.53-2.35\. When using the Jadad scale, the mean OR of
the highesr scoring trials was more positivc (2.00; 1.3?2.91) than if trials scoring four or three of the possible five
points were added (see Table 2, lower part).
When the OR of primary studies are plotted against quality scores, it becomes obvious that there is no clear linear
relationship between these two panmeters (Fig. I and 2).
DISCUSSION
Our analyses provide clear evidence that in the study set investigated more rigorous trials tended to yield smaller effect
sizes. The most plausible explanation ofthis finding is bias.
OR
100.0
e
I
10.0
B
6
H
6
A
fo,
2345
Jadad score
l. Scatterplot of odds ratios of 89 trials of homeopathy plotted against Jadad score (circle areas are scaled propordonal
to the variance of the trials odds ratio).
FIGURE
635
lmpact ofStudy Qualrry on Outcomes in Placebo-Controlled Trials of Homeopathy
have less,promising results. lt seems, therefore, likely that
out meta-analysis [7] at least overestimated the effects of
The results are comparable to those from similar analyses in
conventional medicine. However, the influence of dif{erent
quality factors depends on the context. Schulz et al. [1]
found in their analysis covering 250 trials from 33 meta'
analyses from the Cochrane Pregnancy and Childbirth database that aLlocation concealment was the most relevant
qualiry factor. Double-blinded trials also yielded significantly smaller effect sizes; however, the method ofsequence
generation for the allocation had no influence, and trials
with complete follow-up had slightly more positive out,
comes. In an analysis of 127 trials from I I meta-analyses on
a variety ofconditions by Moher et al. [3], the findings were
similar to those of Schulz et aI. for allocation concealment
and sequence genemtion, whereas double-blinding had no
effect. ln the homeopathy studies evaluated here, doubleblinding proved to be the most relevant influence factor. A
possible explanation is that bias from this factor might be
particularly important, as in these trials homeopathy was
mainly used for mild and chronic conditions for which
there are few objective outcome measures.
The evidence ofbias weakens the findings ofour original
meta-analysis [7]. Since we completed our literature search
in 1995, a considerable number of new homeopathy trials
have been published. The fact that a number of the new
high-quality trials (e.c. [14,15]) have negative results, and a
tecent update of our review for the most "original" subtype
of homeopathy (classical or individualized homeopathy
116]), seem to confirm ahe finding that more rigorous trials
homeopathic treatments.
The use of the predefined cut-off points
to
separate
higher- and lower-quality trials of homeopathy produced
mo.e conservative effect estimates than the use of single
components in the univariate analysis. Also, in the cumula'
tive meta-analyses, pooled effect sizes tended to increase
when studies with lower scores were entered. However, the
scatterplots reveal that a convincing linear relationship
does not exist. There are several possible explanations for
this result. ( 1 ) lf the overall hypothesis that all homeopathy
is placebo is inco[ect, our study set is clinically extremely
heterogeneous and the "true" effect sizes might vary strongly
between treatment and conditions. This heterogeneity
might obscure the slight correlation suggested by the cumulative meta analysis. Other explanations are (2) that the relationship between quality and outcome is rather weak or
(l) that the scores used ale not suitable to detect an existing relationship. ln a set of seven (clinically more homoge.
neous) meta.analyses applying the widely used scale by
Chalmers er al, [17], Emerson and coworkers [18] also failed
to find a clear relationship between quality and outcome.
This would support the explanations (2) and (3).
Quality scores have been criticized for mixing disparate
\rudy features rnto a nontranqparent quantitative e5timate
and inrroducing an arbitrary element into the analysis {191.
Empirical research providing sound evidence on the valid-
OR
100.0
10.0
o
o
€.-,e
o6
1.0
.o,
to_)
o
(o)
i:t
a
o
oc
o^
o
(.J
-id(i,
g
c\r'H
/-'j
UO
0.1
internal validity score
FIGURE 2.
Scatterplot of odds ratios of 89 trials of homeopathy plotted against intemal validity (IV) score (crfcle areas are
ofthe trials odds ratio).
scaled proportional to the variance
636
ity and comparability of the various instruments is almost
nonexistent [20]. Our results indicate that quality scores
might be useful and can lead to similar results as component analysis. Nevertheless, if feasible, meta-regression
with multivariate modeling of the influence of single components seems to us the first choice to test the influence of
quality on outcome, as this approach is transparent and has
a straighdotward rationale.
The smal number of primary studies in most meta-analyses
strongly limits the power of meta-regression. lt seems reason.
able to use minimum scores in such situations to investigate
whether high-quality studies yield resuls that are different fiom
lower-quality srudies. This approach has been used in the literature (e.g. [2,21]). A look on our scatter plot and the results by
Emerson et ai. [18] suggest *ut in small study sets dris might be
haz.ardous. Also, using quality scores to weight primary studies
[3] in meta-analysis seems problernatic based on such results.
'Whether
using summary scores or components, readers
should be aware that all quality assessments based on pub'
Lished reports are very crude. For example, a single phrase
(which even mighr have been inch.rded by the authors but
deleted after editorial review because of lack of space)
might be used to decide whether a trial is assessed as adequately concealed or not. Even if we assume that allocation
concealment is a relevant indicator, the presence or lack of
a statement on it does not necessary mean that a trial is
high or low quality. Guidelines such as the CONSORT
statement [22] should improve the reporting of key meth.
odological issues in future years.
There is a clear need to investigate the influence ofstudy
design factors and aspects of methodological quaLity on
study outcomes in more detail. The available tools for quality assessment are far from being perfect. In our opinion,
there is no reason why single-quality criteria and summary
scores should not be used in parallel. However, in the
present situation we recommend that investigators should
not rely on quality scoring methods alone when perforrning
sensitivity analyses. Such analyses should be interpreted
with great caution, as their accuracy and relevance may be
greatly influenced by context and reporting factors.
This studl was partiall:1 suppoftea bJ the Karl and Yetonica CartensSrrfr*ng and the NIAMS {ant 5 U24 -AR43346-a2. The Centre fm
Compbmenttrl Medicire Research is fundzd 11 *e Bavarian Parlanmt.
K. Linde
of intervention
efficacy reported
in
er
dl
meta-analysesl l,ancet
1998t 352: 649-613.
4. Moher
D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh
S. Assessing rhe quality of randomized controlled trials: An
annotated bibliography ofscales and checklisrs. Conrrol Clio
Trials 1995: 16:62-73.
5. Greenland S. Quality scores ate
useless and
polentially mis-
leading. Am J Epidemiol 1994; 140: 100-301.
6. Olkin l. lnvited commentary: Re: "A crirical look at
some
popular meta-analytic methods." Am J Epidemiol 1994; 140:
297
-299.
7. Linde K, Clausius N, Ramirez G, Melchart D, Eitel
F, Hedges
LV, et al. Are rhe clinical effects of homoeopathy placebo effects? A meta-analysis of placebo-controlled nials. l,ancet
1997;350, 83+843.
8. Jadad AR, Moore RA, Carrol D, Jenkinson C, Reynolds DJM,
Lavaghan DJ, et al. Assessing the qLrality ofreports ofrandomized clinical rrials: is blinding necessaryl Control Clin Trials
1996: 17:
1
12.
F, Srdr \f, lyiesner-Zechweister M, Pothenceuu R, Veinschitz T. Randomized clinical trials of acupunc-
9. Linde K, Worku
ture for asthma-a systematic revie\r. Fortschr Komplementiirmed 1996; 3: 148-155.
10. Linde K, Ramirez G, Mulrow CD, Pauls A, Weidenhammer
V, Melchart D. Sc John's wort for depression-an overview
and mera-analysis of randomised clinical ftials. BMJ 1996;
313:253
Z5B.
11. DerSimonian R, Laird N. Meta-analysis in clinical trials.
Control Clia Trials 1986;7: 177-188.
12. Berkey CS, Hoaglin DC, Mosteller F, Coldicz GA. A randomeffecrs regression model for mera-analysis. Stat Med 1995; 14:
39541r.
13. Laird NM, Ware JR. Random.effects models of longitudinal
data. Biometrics 1982;
38 963-97
4.
14. \0alach H, Haeusler W, Lowes T, Mussbach D, Schawell U,
Springer \0. Classical homeopathic trealment of chronic
headache. Cephalalgia 1997; 17: 119-126.
15. Whrrmarsh TE, Colescon-Shields DM, Sreiner TJ. Doubleblind randomized placebo-controlled study of homoeopathic
prophylaxis of migraine. Cephalalgia 1997; 1?: 600-604.
16. Linde K. Melcharr D. Randomized controlled trials of individualized homeopathy-a state-of,the-art review. J Alt Complement Med 1998; 4: 171-388.
17. Chalmers TC, Smith H, Blackbum B, Silverman B,
Schroeder B, Reirman D. A method for assessing the quality
of a randomized control crial. Control Clin Trials 19811 2:
I1-J9.
18. Emerson jD, Burdick E, Hoaglin DC, Mosteller F, Chalmers
TC. An empirical study of the possible relation of trearmenr
differences to quality scores in controlled randomized clinical
trials. Conttol Clin Trials 1990; I l, 339-352.
19. Greenland S. Mera,analysis. In: Rothman K, Greenland S.
Modern Epidemiology. Philadelphia, PA: Lippincott Raven;
199& 643473.
20. Moher D, Jadad AR, Tugwell P. Assessing the quality of ran-
References
1. Schulz KF, Chalmers l, Hayes RJ, Altman DG. Empirical evidence of bias. JAMA 1995 ; 27 3 : 40B4lZ.
2. Khan KS, Daya S, Jadad AR. The importance of quality of
primary studies in producing unbiased sysremalic reviews.
Arch lntern Med 1996;156 661 666.
Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et
al. Does quality of reports ofrandomised trials affect estimates
l.
domized controlled crials. Current issues and future directions.
J Technol Assessment Health Care 1996; 12:195-208.
Int
21. Nurmohamed MT, Rosendaal FR, Bueller HR, Dekker
E,
Howmes DV, Vandenbroucke JP. Low.molecular weight heparin versus standard heparin in general and orthopedic sur.
gery: A meta-anaLysis. Laacet 1997;340, 152-156.
17.. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin J.
Improving rhe quality of reporting of randomized trials: The
CONSORT statement. JAMA 1996; 27 6: 637 439.
EXHIBIT C
A systematic review of systematic reviews of homeopathy
E. Ernst
I lomcopachy renrains onc' of tht' urost controversial subjcccs in therapcutics. l"his
article is an irttenlpt lo clarifi its efkctiveness basecl on recent systemille revlcws.
Illectrouic dltabase's rvere searched l?cr syscenatic rt'views/rletn analysis on thc subjcct. Seventecn ar:ticles frrlfllled the irclusion/exclusion criceria. Six of thern related
to re-analyses of one lardnrark meta-analysis. Collectively they implied that the
overall positive result of tbis rreta-analysis is not supporred by a critical analysis of
the data. Eleven indcperdent systematic reviews rvere located. Collectively they
failcd to provide strong evidcrcc in favour of ho:ncopatirv In particular, tllcrc \4?J
no conclition which rt'sponds convincingly bctter to horlcopathic treatnrent thao
to pilcebo or other cortrcl interventions. Similarl_v, chcle rv.rs no honeopathic:
remcclv that w?s detrrolrstntecl ro vield clinical effects that are couvilcingly differelt
fronr piacebo. lt is corcluded thrt rhc best clinical evidence for honrcopathy available
to date does not warrant positivc r€conrmendations tbr its use in clinical practice.
Keytr,ords: akernative rnedicine, clinic:rl trials, horneopathy, nteta lnalysr'.
Intloduction
Honeopathy is a therapeutic mcthod using prepantior.s
of substalces whose effects when administered to healthv
subjects corresponcl to the marifestatiorx of the disorder
(symptoms, clinical signs, pathological states) il rhe individual patient. The method r'vas developed by Sanrnel
Hahncrnann (1755 1843) and is norv practised throughour the werld f 1]. Homeoparhy is based orr trvo nain
principals [1-3]. According to thr' 'Like cures like' principle, parients with particular sillDs xnd s),mptoms can be
helped by a homeopathic remedy that produces these
signs and syinpto*rs iu healthy individuals. Accorcling to
the second principle, homeopathic renreclies retain bioJogical activitl, after repeacecl cliltrtion lrrd sLrcussion even
rvhen cliluted beyoncl Avogaclrols nLrnrber.
Few therapies have atuacteci nrorc debate and contlo-i"hrougbout lts 200-year history,
vctsy thau homc'opathy.
critics have pointed out that its vely grriociplcs fly in tJ:e
face of science, while proponents have nairtained that
it is nar row minded to reject;ur overtly he$ful approach
to healing only because ooe canrrot explair how it rnigJrt
work [2|. Similarly, proponcnts bave quoted seemitrglv
Coresponden.er Professor E. Ernst, Deparrmenr of Complementary l4edicine,
Schooj ot Spon and Health scien.es, Universiry of Exeter, 25 Victoria Park
Road, Exeter EX2 4NI UK. E-mail: [email protected]
Re.etued
-
20U,
accepred
"
2002.
@ 2002 Bhckwell Science Ltd W J
Ah Phomo.ot.54.571-582
sy\tcnrtic
rigorous trials that snggest eflicacy, while critics had linle
trouble citing equally rigorous studies that implied the
opposite.
"fhc existence of contraclicring eviclence is not unusual
in therapeutics. Oue soltrtion to resoh.e srrch contradiction-\ is to c:onduct s),slcmatic levie\vs rnc{ nreta-analvses
of rrgorous studies. ln I997, Lindc et oi. [-jl dicl just thar.
fhe conclusions of this techr.rically superb neta-aoalysis
the notion that homeopathic nrcclicines are
more than mere placebos. The authors also stated that no
expressed
indication was identified in u,hich horncoprthy is clearly
supcrior to piaccbo. Despite this ard other caveatr,
homcopaths wor-lelrvidc celcbntcd thrs publicarion as the
ultirnate proof of their oeatnent. Silcc- thcn, a ilurry of
inceresl ilr honreopathy has errrerged, anti ser,'eral flrtlrcr
svstr:nrntic rsviews havc bren pLrblishcd.This article is an
atempt to criticalJy eraluatc' all such pepers published
since 1997 rvith a vic'w to defining the clinical effecriveness
of hou-reopatbic rnedicines.
Methods
l-iterarun sealchers weLe carricd out iLr the lbllor'virrg
databases: Metllirc (via Pubmc'cl), L.mbase, Amed,
CTSCOM (tiorr inception to October 2001). lhe serrch
terms used rvere hon-reopath..., homoeopath ...,
clinical trial, meta-analysis, s)/stenatic rcview, etlicacy,
cllectiveness. Io addition, other experts ir thc field
E. Ernsr
(t = 5) lvere consulted and my owtt, extensive files wcre
studied. The bibliographies of all articles thus iocared
were scamed for firther relevant teferences. No larrguage
restr ic!ions rvere applied.
Onl), systentatic reviervs (including rneta-anal1'ses) of
controlled clinical tlials of homcopachy r"r,ith hunuu
patienls or voluntee$ were includecl. Non-sysrematic
reviews, overvielvs, clinical trials and reviervs of nonclinical investigations were excluded. All arricles rverc
evaluated by the present author. The folLowine information rvas extracted fiom the original artidesi il)chrsion/
exclusioo criteria, cotal sanple size, assessnrcuc of rneth
odoiogical qualiry results of mcta-analyses. ovcrall corr,
clusicln of the arrrhors.
Results
Six re-analyses of Linde ef a/.t or:iginal me'ta-analysis [3]
u,'ere located [4-9]. Tlble 1 suturarizes kev dare froln
these prrblications. Tire results of these rc-anaiyses tlenr,
onstnte thar the more rigQrous trials arc associatcd rvith
srnaller effect sizes which. in turn, render the o\-emll cllect
insignificant 15, 6, 81. One re*analysis sussests rhar rbe
initial positive meta aralytic result [3'] was largely clue to
publication bias [9], a notion that had been considercd
by the original authors brlt rvas rejected by theli. Mosr
notably, pr:rhaps, the authors of thc originrl nrera,arulysis
l3l conclrrded that their rc' analysis 'weakent'cl the findings
of cheir original meta-analvsis' [6]. Coliectivel,v these reanalyses imply tl.ur the initial conclusions of L-irrde cl al.
u'as not sr.rpportc'd by critical evaluation of rheir data.
lrr additrou. ll irrdepcndenr sysrenr.rtic r'(\r,.\v5 wirl
located i10 20]. Table 2 sumrnarizes key data fiom these
publications. CoJlectively tlre findings do not pror.ide'
strong evidence iu favour of homeopathy'. Wirh the
exceptioD of postopc'rative iler.rs [:10] and influer.rza li7]
(see bclorv) tl.rerc is no condition for which honeopathy
is corvincingiy cfictive [1.0, 11, 13, 18-201. Arnica, thc
nlost fJequendy tested lloneopathic retrledy, is not
demonstrab)y diferent from pJaccbo f12, 151. Onc
hon.reopathic remecly (oscilJococcinun.r) rvas fbund to be
superior to placebo as a treahleDt lnd prcr.ention of
influcnza but thc eflect size r.vas small and thercfore of
debarablc clinical teJcv*lcc 117]. Morcover, rhc \'ololne
olthc evidence for osciilococcirrurr-r is srrall ancl rlrert'forc
Dot lirily collclurrvc. ()ttr rystclrrati.' r('vr'\\' ot \.,r'!,'u\
bon-reopathic m.:dicines for postoperative ileus prodlrccd
f3l
an over:all positivc result [10].Yet ser''erai caveats need to
be taken into account, most irnpor:tant)y the fict rhat thcr
clcfilitive study designed as a Drulticentre trial ro lcp-licatc
scveral of snrallcr: studies failed to dcmonstr:ltc a positive:
effecr f101. Onc indeperdent revierv of all honreopathic
l\C'l's
regardlcss of indication or rype of rcrred), ),ieldeil
a positive result [16l.Yet the statisticnl appro:rch to gen578
eete this r€sult r,r.'as of debatable va.lidity and the authors
are keen to point out that rheir overa.ll result is weak aud
not sufficieDt lbr definitivc recontmendations
Discrrssion
CoLlectivcly rhese data do rrot provide sound cvidencc
that homeopathic remedies are clinically diffcrcnt from
placebos. Horvcver, the present analysis has scveral limitatior'ls that should be kept
in rnind when interprering its
concL"rsions. Iiven thorrgh a thoroirgh sealch strategy was
adoptcd. tJrer:e is no absolute quaranree that all rclevanr
artjclcs wen' locltecl. Mrrry of tlre inclLrded reviews are
from thc present author:s team, ahd tbis c<luld havc inrnrduced bias. Finally the validity of conducrlng a sysrernatic
review of systetuatic rcviews has its limitations; most
imporcantly it does not create any ilrforrnation that was
not available before.
The clinical cvidence suormarized abovc is not dissir-r'lilar 1ion.r thc prcclirical data.Vickers rccently conducLed
a $ysternadc review of preclinical invcsrigations of home,
opachy l2l I. Er,en rhough 120 papers could be ilciuded
in the evaluation. tbis aurhor found that lack of indepcndent rcplications, sc'rious methodological flaws, and con,
tradicrory results precluded any firrn conclusion. This
systernatic revierv therefore casts considerirble doubt on
onc' of thc main assumprior-rs of honeoparhy, namelv that
houreopatiric rernedics tetain biological activity even
u,hcn dilurecl bc.vond Avogadro'.s ninber (sec abovc).
Pcrhaps tire nlost ri'cent trial cviclcncc, rror vet
includecl in systcnlatic reviervs, helps clarify rhe question
rvherher horreopathic remedies are more dran placebosSince the publication of the systematic reviervs. both
positive, e.g. 122 241. zs well as negative cLinical crials
c.g. [25-27] have eurerged. It seems drerefore unlikelv
tlut tllesc lew findings lr'ould substantially change rhe
results of an1, of the systernatic rsviews rvcrc they ro bc
up-dated.
The rccent observation of sqhrtc clusrcls in highly
diluted water has been inrerprcted by several homeopaths
as iDcreasing the plausibiliry oL bomeopathy [28]. This
novel finding rcqrires indcpendent replication. Furthcrnrore, this obscrl,atiolr (if confirmed) d(res not lend itself
ro explainirig hovu, solute clusters colrld havr' anv eficts
ou hun):lr healtb. TlrtLs both the clinical cvidc-nce rnd
tlre basic research rrnclcrpinnirg homeofathv rcntlrn
unconviocirrg-
lfone
accepts this conclusion, one mrghr ask what its
inlplicitions for future research may be. Tu,o opposiug
vieu,s edst. One holds that the dcfinitive trial of hone ,
opathy should be conductecl to once and for all settle tht:
cluesrion [29].'l lle other slales thal 'ne\,v trials . . . are no
longer a research priorify' and advocatcs'outcomr' srudies
to ev:llunte the individual treatment decisions . . . and
@ 2002 Blackwell Science Ltd B( J Clin Pharmo.ol, 54,577-Sa2
A systenatic rcview of systenoti. rcviews of homeopdthy
ti
!;i
3:
iE
EEi i=22
.E:;:
iii|z !i-
;;;€
tiFld
E!*
c g r i S;:
E t 3= q
ii
l=nA
c
;si=4
=
;i ,j
;: il .
€; lzi
3i ;
'€
i."
7..
!E
! i o=
1? = a
a a
= tL-z ",a
E
-- I .- | . ^
t =.
oc
-:'
=i i'i
:
x; 7'a!
iiE
?:; i<;
;"vi i iii
Lie
l-:
|
=
=
:i€e=
s?
iiE:=
:;:
iz,
iz.? 5'FZ:i tiZo
::Z
!: !='
,:E
=s:
lfii
ti=t
6.-. E:
'^"?Z<i
i22==i?2r:et2 i*z*;E
;:Zi
!
!
iz
i!
Zt
::ie
. y2
Z
"&1,i E:;E
rid:-
:.i <'
i y,
j.t
Eai" ?7i
?E:
F
J
_
E4i
., 1
i
':
=
;ia :iili:E
niiti;='.itzzi'tiia
?i1i
I
II
ii1;; j
: "n2ta==
3,i: 4 . '
a
9,8t)
-, E i!!:i,:
!:1.+
E
'",:
,"-3
.
a i
I c ='\).ii-!.3aj a !4.=':?
ai.t
=
a. e t\ i.i
f ! =
_iL;a
3l=
i : . a
i a ,,
*_
:1,, ,l;.ttj=:ii:::i
7:? i ,1i: i; i;: i ;iB*
-
E
a''c
;ZE
i-*{t
: =:. !.
=
5
2
-=
?
F
V
lt
+
a
g
t::
.:':*
e
I
a?
ti a
,9
c>.
;!
5
F
!
F
I
g!
@ 2002 Bla(kwell Science Ltd
|2
l:
* I An
PhdtnacoL
5
4, 377 -587
579
ii :i:
:.i:,-=
iria
i
r,ii:t
i:=7
l?zi
4i;
ti
ii
iii .ii! EiE ii i=iil
" a=2
'Li:t:
az;;€r
.;;
+'t
-"'|=iE
tZaE :ii i
ii
iz
liZi
i:F izz
jr
:zz clF:-iez 4.zziz*3 F{:E'i=re
;
:
d
7
2
E
y=
i..
3;g ;:
F
a
F
,r7=-t=i!1
F E ! zs: E
e i:;ziZ2illir;
ZZ
Z
i".iilZ J =i Za,;r
,.rr+
,=''-
,i9i
le
4.
; r€ts - *
-{
't
!
;
!
c
EE Li4E2-E-.. Et:rE?Zi?iEe7
..E= t22".>i4a
g=i;
i!{?.+j:;?;
at"
,TEttEr=i !iE== i?55i
-a;i=-
:i2i
if :,E:-i?3j'r*
;Sf9;+ iiiL
..;ei:l2z:::ii
z 5 u 4 !. ), -! , . -Jl-a!iL
=
=
?ii1==,i7i.a:.i
I': ;l : ! - ji
!_
z+Jr-iEi;114 1E;iiE
rF-trzz
G
- ?!.
ilil^
2
E
J
n.
't:.i2
e:;d:;e.:.ii
t;4x
:i*z
!I_
h3
=
49
Y- it
i:=
s;i
2?
,7'.
i,; =f
-!."
4a
in
vi
::=
.:! ' t^
::.
l::
o=
:i
-; P
! i=
:4:
"=
a- 2 ::
i=
2-n
c5
- :: ^
>zzzzz
i=9
=Tca
:: 7
'2
E;
I
P
";
;E
c
:
=^8.
?."n 2"
L==:
i*l
?:-c -.;'i.
'-t4 r
=,
!-!+
!:;
i
?
,i2
E
4
v
'j
z
o:::
l!^-{
=
7
b
;
i
L .'
-t;:;
:r a aZ a -
2
T
T;:Ei
; r'l^:
r
',!
!
--:
?-.r.s t1 l!
as
a
i
= t,4.
tc
P
o
E
.,
P.
;
;
+
z ca
I
H.
tl
i
ts
i!
t
I
z
t:
€
E
lt
! i::
d.
i
E
e
:s
lt
c
*
!1
d
580
1.6
@ 2002 Bla.kwell Science
Lvl
kI
Ain F'lrcrnacol, 54.577-582
A syslemoti. review ofsystematjc .ewews of homeopathy
compare outconles to orthodox trcetntelrt' f3l]1. Sr"rch
outcome studies exist. They are btrdcned rvitb a :rvriad
of methodological w.eaknt'sses, rnost importanrly a pronc,
ness to selcction bias, ancl r:suallv rcporr findings r,vhich
are convincinglv in fivour of the horneoparhic apprr>ach
[31j.Thrs could inrplv rh.rt rhc iudiviclua]izc'd. cnrFarhtic
and tinre-intensir,c rpproach nrosr hortreepat]rs rclopr to
healthcare viclds eood clinicel resr.rlrs.This onrphasizt's rhe
lt
12
13
1:l
Progress.
16
In conclusion. the hyporhcsis thlt any givcn homccr
pathic rc'Drecly lcads co clinicll t'llects th:rt ale rclcvrrltiv
clil-ierent fronr placcbo or superior to ofher ccurtrol ir)terventiom for any Dledicil conilitioD, is not suPported by
evidence lion systematic rcvicws. Until nrore cornpcllinEl
11
(irrllrrt of irrtllrrt: Thc rLrtlror ir .r trainccl
intcrcrts in this.rrca.
horrrc.opirth:
ot
phccbo-corrnallcd trials. P./ftq.,,r 1993t 1 1 : :1*8.
Ernsr l, Pitdcr MrI. [f]icac,v ol-honlcoparhic arnicr. A
systematic ,clirjN of plact-bo corrtlollcd clinical trials ,4rd
.S,iU 1!)')ili 133: llll7 1190.
Lntst E. HoruocoprrhJc plophvlrls ol Icrd.r,-lrt's rLrd
nigrainci A lvst!'nuti. rcvrli\\'. I l' h Sy,l1pt i\,Idnaf(n 1t).)t):
18: -353 357
importance ofthe therapcuric elrcouDtcr and is in accordance with a wc'aith of inlbrrnation in this area f321. lr
does not, howevcr, ansrver the'placebo qLrestion'. I insisc
that this quesLioo cloes re'cluirc an answcr - fbr thc' sake
of scielltific honestv arrtl possibly in the namc ofcliuicrl
results arc available. honrcopltlly c:lrr)ot be vie,.vcel as an
evidcncc basecl lourr oi rhcral.y.
Erlst E, l3arncs.f Arc homoeoprrhic rcrrrcc|cs r{lcctile lirr
JrLy.'l-orr- nr,rsl. sorcn..- A \\:,.,, ,r, (v,('.
l5
Ernst E. Classieal homoeoir.rthv vt'rsus conventror:rl
trertnrents: a sy\tenmti. rcvr.w Pcrli:ion 1999; 12:13 15.
Liidtke I\, Wilkcns J. I{liriscire Wirksrr[bcitsstudien zu
Arnica i:r hornocopathischcn Zubercurrngcn. In (-ir,5lr,?r
StiJiunq Oanpanl' Rq,orr. Escn. Gernuny: Cirrstcns Srifrurg,
i999.
Cuchcrrt M. ll.rugh Nt(]. (;oorh M. tloissel l 1, Eridence
of ciinical eflic:rc1 oi:lmnrcop:rthy A rrct.r-,rralyrLs oi clLnrcal
irirh. It,r / OIn
Iri
Phannatol 2o01)i 56:
27 :l.i
Vickers AJ. Snrith Cl. lJomcoprthic oscrlloco..rlrLlrn tir
prevcnting :rnd rreature inlluc-nza ud inllutnz;r Lkc
syndrorrres. Corhntnt' l-ibnt11 2001; 1: 1 10.
Linde I{.Jobst KA. Hon!'oprtlw fi)r chronir asthma. Clodrralt
l-tlr#l l99ti: l: I
7.
(1. l-lornrop:rthv rnd rhcrurr:rtic
-kurs Wll, Linde I{, Rrnrirtz
drscre. ,lllrcrar .1)ii a,lnr \i,(lr .1n 2t)ltt):26: 111 123
Long L, err\i Ii. Honrroprthrc rcrDedrrs lbr tht rrL,,liurcnr of
osr.orrthritrs: .r \ysld1$tL. rcvrr:rv Ilr Harrrr-opatlr i l1)0li 90:
he hrs uo
Ilnitnci
37
21
References
I
I turrtia , D,.liutaty ol llowopdthy Erlnbureh:
(lhurchll Llvi Stort, ?ll(r0
2 Ernst E. Hor!reop!L,'. Fi\t prcscr:t hrtLrrc. Ilr I Clin Phdrndan
3
ftiur rfid serres b lltd.l )t)|r):721: It'1 416.
CrorL!'r ll Homcoprrhi,: rlcatment
of.rcr.rte otitjs uredu rn childrcrr: .r prc'lLnrrrrirry randonizeci
placebo conlrolled tnll. Pt:JiM lifut Dis J 2li\)1.20 117-
24
irt rc!ic\v../,4,
()bc'rbaurn M, Vrrrii l, tlerr Gal Y, cr a/. A nndonriscd,
controlled clinital tli,rl cif thc honrcoprtlric rrcdic,rtion
Tl\AUMEEI- Si irr thc trc,rtrlrcrrt oi circr othcrrpl-indLrcect
N, Mclch;rlt I).
stoDl:rn!is ir) .hiLlrel Lrudcrgoing stcnr
( )|L@ 2l)t )1. 92: 68.1-6'l(l
(,
indivrdLralired honrcoprtlrv: .r \!arr of:thc
0\nt)le tttlt r\tud 1r)98:4: -l:is.
Lrnde K. Scholz M, l\rnlrez (1, (lhLrsrLrs
oi stud1, qrrrlitl' on outcomc Ln placebo
conuollcd trirls ol homoeoprthv.J Clut Epn!etnd l')99;52:
Jona\ WB. lr4);rct
183.
t5
631-63(r.
Morrison B. Lilfi)rd I\J, Ir.rlst E. Mcrhodoloeic.rl rigour rnd
rc-sults of clirucrl lri;rls of h(nrocol):rthic r(xllcdics. Pl'4;/r,r,
3
132 liS.
Ernst E, ['littlrr M]i i\c':urJysis of prcvior.rs rncLr rn.Ll]:ris of
clinrcal tli.rh ofhomcop.rrhy J C/rr i+i,/,'rnn l 2{)tl{l; 53: 1188
9
Sterne J, Egger
(lerer).
M. Snrrrh (lD. Invcstrq;rring ancl de:rling with
publication end otlrcr brirscs. In .S),-vorlrtir' Rd,t.,r,r i,'
Heahhcare: Ment attulj\ii i Ci tt:xt, c'ds Eggcr M, Snith GD,
Altmu D(i, pp. 189-20u. l-ordon: ISMJ Publishing ciroup,
2tt
^n
on riiluriorr. Chtn OantmLr ?0t)1; l)ct:2221-2223.
l'anccstcr T, Vickers A. I-:rger nials r.cdcd. 8r.\4cdJ 2ir00;
llitrnc\ J. Rcsrjr KL. E|rrsr [. I Ionrcopirthv tb. Po\topcr;uivr:
Jlcus.J Clrr Cnnnc ri/Jl 19r)7; 25: 62ti {r3-1.
321: 4'16
Clin Phomacot,
54.577
5A7
.r
honrcop:rthic trernrcnr of .rdcnoicl vrqctanorrs. Iff -,1 (;ca
.P,11,r 2al00i 7: 48' 54.
Lewith (il-, V,'rtknN AIl. llvl.rrrds ML, rt,r/. Usc ol
ultrarnoleculrr potcrri:ies ofaliergen to uret rsthruh. propld
;rll(rqic to housc dust rrritcr do hie blilrd r.rndonrrscd
co[trollcd dirucr] trirl. B/ l,/r(/ / 2002t 32: 520 523.
\.rr.r.rl S. (;rrlcl, r l{1.I rr. r1'cir"d'ol.r. i& n.$",
rr
rvater
I
trLnsl,l.rnr.rnon.
Wahch l{,l.orves T, Musl';i(h l), r,1 The long rcrnr c*lc$
of homcop,rrhi. trertnrcrr ol chronrc herd.rchc,i: one ye;rl
fo]lor-r-uf and snglc c.rse tirrc sclies analvsir. I3r'IlonreopatlJ
2001.
@ 2002 Blackweil Science Ltd Bt
li:ll
2()01:90:63-12.
Flicsc K-i-I, Fcuchrer U. l,iid*c ll, Morllcr Il. ll-csLrlts of
rrrrdoruscrl prcspci'ti!r doublt lnind clinrcal trr:rl or thc
20001 13:
1()
wellyn-Joncs ILII, McSharrv C,
Jacobs -i, Springcr 1)A,
5
7
L.1t
Ai!cl)i\on TC. l{audou)rscd contK)ll.d trial ol homcopath,v
versus phc'bo rn percurri.rl rllergic rhinitrs lith orcrlrov of
23
clla'cts
of homoeopathy placebo ell'ects7 A urcie ilnel_\,sis of p[crt]o
conlrolled lrialr. Inrcr 1')97;350: 8-14 843.
Ernst E. Arc lighly dihtc horlocopathic remedies phccbos?
PerJ :ion 1<)98; 11 2L)1-292.
L;rdc K. Mclclnr! I) lllrdornrzcd controllcd trals of
4
Tirlbt MA. Rcil[l l).
Sw:ryn. J.
1991.44 135-43i
Linde K, Clrusrus N, R.ur)irez (]. fr d/ Ar. thc cli lc:rl
43.
Vickers AJ. hrdcpeldent rcpllc:rtion ofpre clinic;11 rescltrch in
homeopathv:,r systen.tii. rer,it'rv. I:orrb Konplc'ktltannd
1999: 6:311-12().
58r
E. Ernsl
3tl
31
Feder G, Katz T. I{xndotriscd co troiled triils for
horlcopirthy. Ilr Med J 2Ot)2; 324 4t)8-41)t).
f\iley D, Fisher M, Singh 13, l'lridvogl M, I{cgi.r M.
llorlcopathy i:nd conveutional nrfdicinc: .rn outcornes strrdv
conrparing effectiveness in
Cotry emenrar A4cd
582
r prxr.rri,carc
2tJiI;1:149
sertnlit.
32 Di
Illasi Z. Halkncss F.. Ernst Ii, Ccorgiorr A, Klerjncn
InilLreni:c ofcorrrert effccrs on hcalth ourconrcs: r
rcview.
l*rcr
J.
\trr.nrrrir
20011357:731-762.
-/.4ll
159.
O 2002 Blackwell
Science Ltd
B.l Ain Phamacal,
54.577-SA1