Prospective Multicenter Study of Quantitative Pretest Probability

CARDIOLOGY/ORIGINAL RESEARCH
Prospective Multicenter Study of Quantitative Pretest Probability
Assessment to Exclude Acute Coronary Syndrome for Patients
Evaluated in Emergency Department Chest Pain Units
Alice M. Mitchell, MD, MS
J. Lee Garvey, MD
Abhinav Chandra, MD
Deborah Diercks, MD
Charles V. Pollack, MD, MA
Jeffrey A. Kline, MD
From the Department of Emergency Medicine, Carolinas Medical Center (Mitchell, Garvey, Kline),
the Department of Emergency Medicine, Duke University Medical Center (Chandra), Durham, NC;
the Department of Emergency Medicine, University of California–Davis, Sacramento, CA (Diercks);
and the Department of Emergency Medicine, Pennsylvania Hospital, Philadelphia, PA (Pollack).
Study objective: We compare the diagnostic accuracy of 3 methods—attribute matching, physician’s
written unstructured estimate, and a logistic regression formula (Acute Coronary Insufficiency-Time
Insensitive Predictive Instrument, ACI-TIPI)— of estimating a very low pretest probability (ⱕ2%) for
acute coronary syndromes in emergency department (ED) patients evaluated in chest pain units.
Methods: We prospectively studied 1,114 consecutive patients from 3 academic EDs, evaluated for
acute coronary syndrome. Physicians collected data required for pretest probability assessment
before protocol-driven chest pain unit testing. A pretest probability greater than 2% was considered
“test positive.” The criterion standard was the outcome of acute coronary syndrome (death,
myocardial infarction, revascularization, or ⬎60% stenosis prompting new treatment) within 45 days,
adjudicated by 3 independent reviewers.
Results: Fifty-one of 1,114 enrolled patients (4.5%; 95% confidence interval [CI] 3.4% to 6.0%)
developed acute coronary syndrome within 45 days, including 4 of 991 (0.4%; 95% CI 0.1% to 1.0%)
patients, discharged after a negative chest pain unit evaluation result, who developed acute coronary
syndrome. Unstructured estimate identified 293 patients with pretest probability less than or equal
to 2%, 2 had acute coronary syndrome, yielding sensitivity of 96.1% (95% CI 86.5% to 99.5%) and
specificity of 27.4% (95% CI 24.7% to 30.2%). Attribute matching identified 304 patients with pretest
probability less than or equal to 2%; 1 had acute coronary syndrome, yielding a sensitivity of 98.0%
(95% CI 89.6% to 99.9%) and a specificity of 26.1% (95% CI 23.6% to 28.7%). ACI-TIPI identified 56
patients; none had acute coronary syndrome, yielding sensitivity of 100% (95% CI 93.0% to 100%)
and specificity of 6.1% (95% CI 4.7% to 7.9%).
Conclusion: In a low-risk ED population with symptoms suggestive of acute coronary syndrome,
patients with a quantitative pretest probability less than or equal to 2%, determined by attribute
matching, unstructured estimate, or logistic regression, may not require additional diagnostic testing.
[Ann Emerg Med. 2006;47:438-447.]
0196-0644/$-see front matter
Copyright © 2006 by the American College of Emergency Physicians.
doi:10.1016/j.annemergmed.2005.10.013
INTRODUCTION
Background and Importance
The introduction of chest pain evaluation units has provided
a partial solution to the problem of evaluating low-risk patients
for acute coronary syndrome. Proponents postulate that chest
pain units provide efficient protocol-driven diagnostic testing
for patients who would otherwise be admitted to a more
438 Annals of Emergency Medicine
expensive hospital setting or be discharged with an incomplete
evaluation for myocardial ischemia, the most common cause of
sudden death.1 Indeed, data from the Physicians Insurers
Association of America indicate that 26% of all money paid in
closed malpractice claims in emergency medicine from 1985 to
2003 was for patients who had a chief complaint of “chest
pain.”2 Emergency physicians have begun to use chest pain
Volume , .  : May 
Mitchell et al
Quantitative Pretest Probability Assessment for Acute Coronary Syndrome
Editor’s Capsule Summary
What is already known on this topic
Emergency physicians are conservative with respect to
disposition of patients with potential acute coronary
syndromes because of poor outcomes in patients with
missed acute coronary syndromes.
What question this study addressed
This study addressed whether “attribute matching” (a
computerized matching of the individual patient being
evaluated with previous patients who had similar
presenting characteristics) can identify patients admitted
to chest pain units who are at less than 2% risk of acute
coronary syndrome.
What this study adds to our knowledge
Of 304 patients who entered the chest pain unit
identified by attribute matching to have a less than 2%
risk of acute coronary syndrome, only 1 (0.3%) patient
had acute coronary syndrome diagnosed within 45 days.
However, attribute matching did not outperform the
Acute Coronary Insufficiency-Time Insensitive Predictive
Instrument and the physician’s unstructured estimate of
pretest probability.
How this might change clinical practice
Use of an objective system, such as attribute matching, to
risk stratify chest pain unit patients at low risk of acute
coronary syndrome might enable physicians to forgo
further evaluation of coronary artery disease in this
extremely low-risk cohort.
units to evaluate patients with diminutively low pretest
probability for acute coronary syndrome; some centers have
reported that 98% of all chest pain unit testing result are
negative.3– 6 The consequences of testing very-low-risk patients
include overuse of costly resources and the risk of a false-positive
chest pain unit protocol that leads to unnecessary cardiac
catheterization and its 1% risk of serious morbidity.7
Goals of This Investigation
To address the problem of overtesting for acute coronary
syndrome, researchers have proposed the use of pretest
probability to exclude very-low-risk patients from a chest pain
unit protocol. The central hypothesis states that the risks of
testing will exceed the benefits of testing in patients with a
quantitative pretest probability below a very low, predefined test
threshold computed from the method of Pauker and Kassirer.8,9
Patients with a point estimate pretest probability below the test
threshold are unlikely to benefit from further testing. Kline et al 10
previously estimated the test threshold for the evaluation of
acute coronary syndrome to be 2.0%. In the present study, we
examine 3 methods of estimating the pretest probability: the
Volume , .  : May 
physician’s unstructured written estimate, the Acute Coronary
Insufficiency-Time Insensitive Predictive Instrument (ACITIPI),11,12 and a novel method, attribute matching.10 The
primary aim of this study was to compare 3 quantitative
methods of estimating pretest probability less than or equal to
2% to rule out acute coronary syndrome in low-risk patients
undergoing evaluation for acute coronary syndrome in a chest
pain unit at 3 US emergency departments (EDs). The specific
comparisons were the proportions of patients categorized as
having a pretest probability less than or equal to 2% and the
proportions of patients with a pretest probability less than or
equal to 2% who developed acute coronary syndrome in the
next 45 days.
MATERIALS AND METHODS
Study Design
The study was a prospective, noninterventional, multicenter
study of diagnostic accuracy.
Setting
Patients were recruited from the EDs at Carolinas Medical
Center, Duke University Medical Center, and the University of
California, Davis. All 3 centers have established, nationally
accredited chest pain units and residencies in emergency
medicine and are staffed around the clock by board-certified
emergency physicians. The study protocol was approved by the
institutional review boards at the 3 participating institutions.
Selection of Participants
All patients evaluated for acute coronary syndrome in the ED
chest pain unit at Carolinas Medical Center (April 2003
through July 2004), Duke University Medical Center (April
2004 through June 2004) and the University of California,
Davis (December 2003 through July 2004) were eligible for this
study. Consecutive patients were approached by a study
associate (a registered nurse, a physician, or a premedical
student clerk) immediately on decision to transfer to the chest
pain unit. Written informed consent was obtained from patients
at Carolinas Medical Center and Duke University Medical
Center. Waiver of written consent was obtained at the
University of California, Davis. Patients were excluded for
cocaine use within the previous 3 days, determined by either
patient disclosure or urine testing and performed at the
discretion of the evaluating physician. We also planned in
advance to exclude patients who eloped before completing the
chest pain unit study protocol.
All patients with a constellation of clinical symptoms and
findings prompting a chest pain unit evaluation for potential
acute coronary syndrome were eligible for the protocol. Typical
symptoms included, but were not limited to, anterior chest
pain, dyspnea, “indigestion,” neck or arm discomfort, syncope,
fatigue, and dizziness. All patients were initially evaluated in the
ED, and the decision to undergo protocol testing was made by
the attending physician, independent of this study. The initial
Annals of Emergency Medicine 439
Quantitative Pretest Probability Assessment for Acute Coronary Syndrome
evaluation included medical history and physical examination,
ECG, chest radiograph, and assay of cardiac biomarkers of
myocardial necrosis, hemoglobin, and electrolytes. Biomarkers
included troponin T, measured with third-generation Roche
Elecsys 2010 analyzer (Indianapolis, IN); (troponin T ⬍0.04
ng/mL normal, threshold based on 10% imprecision coefficient
of variability); and creatine kinase with MB fractionation (⬍5%
or ⬍8 ng/mL normal). Patients with greater than 0.1 mV STsegment deviation on ECG indicative of ischemia or infarction
or diagnostic biomarkers acute indicative of coronary syndrome
were ineligible for chest pain unit testing and were admitted to
the hospital. Patients with borderline abnormal biomarkers
explained by other conditions (eg, a troponin T concentration
between 0.04 and 0.08 ng/mL in a patient with end-stage renal
failure) were eligible per the protocol of the evaluating
institution. Patients with electrolyte abnormalities or a
hemoglobin concentration less than 8.0 gm/dL required
treatment before continuation of the protocol. Eligible patients
were transferred to the intradepartment chest pain unit, where
they were monitored by 12-lead telemetry for evidence of
pathological ST-segment changes (ST Guard; Marquette,
Milwaulkee, WI) for at least 8 hours, followed by a second
measurement of cardiac biomarker concentrations. Patients with
negative ST-segment monitoring results and 2 sets of negative
biomarker results received provocative testing. The choice of
provocative test was site dependent and provisionally
determined by the evaluating emergency physician and
prospectively approved by a cardiologist supervising the test. All
provocative tests were interpreted by board-certified
cardiologists or nuclear medicine specialist radiologists. Patients
with positive provocative test results were admitted to the
hospital. In the absence of prohibitive medical or social
conditions, patients with negative testing results were discharged
to home and referred for outpatient follow-up. Patients with
nondiagnostic testing were discharged or consulted for
admission at the discretion of the evaluating emergency
physician.
Data Collection and Processing
For all patients evaluated in the chest pain unit, in real time
and before admission of a patient to the chest pain unit,
clinicians completed a mandatory, structured, 1-page data form
(Appendix E1; available online at http://www.annemergmed.com).
Clinicians were given instructions in lecture format at several
meetings at each center. Clinicians were informed of the study
purpose and how to complete the data collection form.
Additionally, the physicians’ names and levels of training were
also recorded.
Clinicians who completed the data forms included
emergency medicine attending physicians and second- and
third-year emergency medicine residents. The first question on
the data form asked explicitly, “What is your estimate to the
nearest 1 (whole) % of the probability that this patient will
require coronary angioplasty, coronary stenting, or bypass
440 Annals of Emergency Medicine
Mitchell et al
grafting, or will suffer myocardial infarction or die within the
next 45 days? ___.” This numeric estimate represented the
physician’s unstructured, written, quantitative estimate of
pretest probability of acute coronary syndrome.
The data collection forms also collected variables needed to
compute the pretest probability using the back-transformed
ACI-TIPI logit equation solved for “P.” This equation includes
the following independent variables: chest pain or left arm pain,
history of coronary artery disease, change in stable angina
symptoms, and ECG characteristics.11,12 All of these elements
were included in the data form (Appendix E1; available online
at http://www.annemergmed.com).
Attribute matching requires a patient profile of 8
independent clinical variables, such as age, historical features,
ECG findings, and physical findings (PREtest Consult ACS,
Breathquant Medical Systems Inc., Charlotte, NC; available
online at http://www.pretestconsult.com). Using this profile, a
computer program queries a 14,800 patient database of ED
patients who were previously evaluated for acute coronary
syndrome and who had 30-day follow-up. The program returns
only the patients who exactly match the profile for the patient
under evaluation. The percentage of matched patients who had
acute coronary syndrome equals the pretest probability. In a
previous study, when this attribute-matching system estimated
the pretest probability of acute coronary syndrome to be less
than or equal to 2.0%, the observed 45-day outcome prevalence
of acute coronary syndrome was 1.7% (95% confidence interval
[CI] 1.1% to 2.4%) in a population with an 8% prevalence of
acute coronary syndrome.10
The details of the method of follow-up are described
separately.13 Briefly, we prospectively asked patients to provide a
primary and secondary telephone number. We recorded the
numbers in writing on the data form, and after writing the
numbers, we read them back to the patient to confirm accuracy.
Starting at 45 days after enrollment, we initiated attempts at
telephone contact. On making telephone contact with the
patient or a proxy, we asked a series of 10 scripted questions
listed on a second paper data collection form.13 We dialed both
numbers at least 5 times at different times on different days
until telephone contact was made. If both numbers failed to
yield valid follow-up after 5 attempts, we searched public
telephone directories for possible alternative numbers. If the
patient or a suitable proxy could not be contacted by this
method, this was considered a failure of telephone follow-up.
For all patients, we searched the electronic medical record for
evidence of return visits, death, or the development of acute
coronary syndrome. Additionally, we searched the Social
Security Death Index to confirm the patient’s survival. Details
of this method have been previously published.13
Data collection forms, results of biomarker tests, provocative
testing results, telephone follow-up, and medical record and
Social Security Death Index searches were compiled into
spreadsheet form (Microsoft Excel, Seattle, WA).
Volume , .  : May 
Mitchell et al
Quantitative Pretest Probability Assessment for Acute Coronary Syndrome
Outcome Measures
The criterion standards for a positive outcome of acute
coronary syndrome included death; myocardial infarction (as
defined by the European Society of Cardiology)14; the need for
revascularization, including coronary artery stenting,
angioplasty, or coronary artery bypass grafting; and a newly
documented coronary artery stenosis greater than 60% that
prompted new medical therapy. Deaths within 45 days from
causes that could not be determined were to be considered a
positive outcome. The criterion standard outcomes were
determined by 2 reviewers who were given explicit definitions of
the outcomes and who were blinded to each other’s
interpretation of standardized versions of the medical records
and follow-up data to determine the presence or absence of an
acute coronary syndrome defining event within 45 days after
enrollment. On request, reviewers were given original
photocopies of all relevant medical records and research
documents for any patient to assist in their decisions.
Discordances were adjudicated by a third physician reviewer. A
positive criterion standard outcome required the consensus of 2
out of 3 reviewers.
Primary Data Analysis
We compared the proportions of patients categorized as
having a pretest probability less than or equal to 2% to the
proportions of patients with a pretest probability less than or
equal to 2% who developed acute coronary syndrome in the
next 45 days using 95% CIs from the exact binomial formula.
The sensitivity and specificity of each pretest probability
method were computed from 2⫻2 table analysis, where a
pretest probability greater than 2% was considered a “positive”
pretest assessment, and the criterion standard was the
adjudicated outcome after 45 days’ follow-up. Receiver
operating characteristic curve analysis was performed using
STATSDirect Version 3.3 (Cheshire, England). Areas under the
curve were estimated using the Wilcoxon method.
The primary comparison was the 3 proportions of patients
categorized as having a pretest probability less than or equal to
2.0% by 3 methods. Based upon preliminary data available at
the time the sample size was computed, we expected to see a
minimum of a 7% difference between the frequency of pretest
probability estimates less than or equal to 2.0% for attribute
matching versus either ACI-TIPI or the physician’s estimate.
Using ␣⫽0.01 (adjusted for multiple comparisons) and a power
of 80% (␤⫽0.2) to detect a difference of 7% for a 2-sided test
of independent proportions required an equal sample size of
347 patients for each of 3 groups (ie, 3 methods of pretest
probability estimate), thus requiring a minimum of 1,041
patients total.
RESULTS
Characteristics of Study Subjects
Data were collected for 1,127 patients evaluated in
participating chest pain centers. Of these, 13 patients were
excluded: 10 did not complete the chest pain unit protocol and
Volume , .  : May 
were unable to be contacted for follow-up, and 3 patients were
excluded for cocaine use (Figure 1).
Main Results
The first 2 adjudicators agreed on the outcome of 1,087 out
of 1,114 patients (97.6%), requiring a third adjudicator for 27
cases (Cohen’s ␬ statistic 0.73; 95% CI 0.67 to 0.79). Two
adjudicators agreed that 51 patients (4.6%; 95% CI 3.4% to
6.0%) had acute coronary syndrome and thus had acute
coronary syndrome based on the criterion standard. Fifty
patients had more than 1 qualifying outcome; 32 (2.9%; 95%
CI 2.0% to 4.0%) patients required revascularization (N⫽27
stenting, N⫽5 coronary artery bypass grafting, N⫽2 both); of
these 32 patients who had revascularization, 18 patients also had
criteria for acute myocardial infarction. Of these 18 patients, 15
patients developed an acute myocardial infarction within 48
hours of presentation. One patient had a second myocardial
infarction within the follow-up period. Nineteen (1.7%; 95%
CI 1.0% to 2.7%) additional patients had a greater than 60%
coronary artery stenosis that resulted in new medical
management but did not develop an acute myocardial infarction
or require revascularization. These data are summarized in
Figure 1. Nine hundred ninety-one patients were discharged
after a negative chest pain unit evaluation result, and 4 (4/991,
0.4%; 95% CI 0.1% to 1.0%) patients developed acute
coronary syndrome within 45 days. No deaths were attributed
to a cardiac cause; 1 patient, who had underlying chronic
myelogenous leukemia, died of sepsis during the study period.
Table 1 summarizes clinical characteristics of patients
without and with acute coronary syndrome. Patients with acute
coronary syndrome were older, were more likely to have
diabetes mellitus.
Initial diagnostic testing, including cardiac catheterization
and rates of provocative testing, are summarized in Table 2. Of
the 1,114 study patients, 96 (8.6%; 95% CI 7.0% to 10.4%)
patients underwent cardiac catheterization within 24 hours of
enrollment. Five patients with an acute coronary syndrome
defining event diagnosed within the study period did not receive
a catheterization within 24 hours of enrollment but instead
underwent catheterization and revascularization later within the
45-day follow-up period. A total of 50 patients (4.7%; 95% CI
3.5% to 6.2%) received a cardiac catheterization demonstrating
normal coronaries or nonsignificant coronary artery disease.
Telephone follow-up was successful for 976 patients,
representing 88% (95% CI 86% to 90%) of all patients who
did not undergo cardiac catheterization. Of the 138 remaining
patients not contacted by telephone and who did not undergo
cardiac catheterization, we performed a systematic search of the
Social Security Death Index for evidence of death, as described
previously.13
The descriptive statistics and diagnostic indexes for each
method of pretest probability assessment are summarized in
Table 3. None of the methods produced a pretest probability
that was normally distributed (Figure 2A-C). Clinicians who
provided data for pretest probability estimates (including the
Annals of Emergency Medicine 441
Quantitative Pretest Probability Assessment for Acute Coronary Syndrome
Mitchell et al
Patients Enrolled
(n = 1,127)
Excluded
Cocaine Use (n = 3)
Incomplete Testing (n = 10)
Complete CPU Protocol
(n = 1,114)
45 Day Follow-up
Telephone Contact + Electronic Medical Record
(n = 976)
Electronic Medical Record
(n = 138)
Adjudication
2 Agreed
(n = 1,187)
2 of 3 Agreed
(n = 27)
ACS Total
(n = 51)
4.6% (3.4 to 6.0)
Revascularization*
(n = 32)
2.9% (2.0 to 4.0)
> 60% Coronary Artery Stenosis†
(n = 19)
1.7% (1.0 to 2.7)
Myocardial Infarction*
(n = 18)
1.6% (1.0 to 2.5)
Figure 1. Summary of enrollment, exclusions, follow-up, and outcome adjudication. ACS, Acute coronary syndrome; CPU,
chest pain unit.
*All patients with myocardial infarction underwent revascularization. Fifteen of 18 patients had a myocardial infarction
within 48 hours of enrollment.
†
Patients with coronary stenosis but did not have myocardial infarction or revascularization.
unstructured method) included 31 attending physicians, who
completed 51.9% of the forms, and 21 postgraduate-year 3
residents and 19 postgraduate-year 2 residents who completed
the remainder of the forms.
The median pretest probability produced by the attributematching method was significantly lower than the median of the
other methods. Attribute matching and physician’s unstructured
estimate identified a similar number of patients with a pretest
probability lower than the predefined testing threshold (ⱕ2%):
442 Annals of Emergency Medicine
304 patients (27.3%; 95% CI 24.7% to 30.0%) versus 293
patients (26.3%; 95% CI 23.7% to 29.0%), respectively. One
patient (1/304, or 0.3%; 95% CI 0% to 1.8%) with a pretest
probability less than or equal to 2%, determined by attribute
matching, and 2 patients (2/293, or 0.7%; 95% CI 0% to
2.4%) with a pretest probability less than or equal to 2% by
unstructured estimate developed an acute coronary syndrome.
The ACI-TIPI method identified 56 patients with a pretest
probability less than or equal to 2%, representing a significantly
Volume , .  : May 
Mitchell et al
Quantitative Pretest Probability Assessment for Acute Coronary Syndrome
Table 1. Comparison of demographic and diagnostic data for patients with and without an acute coronary syndrome.*
Characteristic
Mean age, y (SD)
Sex
Male, %
Race
Asian, %
Hispanic, %
Black, %
White, %
Other/unknown, %
Risk factors
History of CAD, %
Current/recent smoking, %
Hypertension, %
Diabetes mellitus, %
Family history of CAD, %
ACS Negative
(nⴝ1063)
ACS Positive
(nⴝ51)
Difference (95% CI)
50.3 (11.1)
58.4 (12.8)
⫺8.1 (⫺11.2 To ⫺5.0)
57.9
23.5
34.4 (20.9–44.5)
1.4
2.6
47.2
44.2
4.5
3.9
7.8
27.5
49.0
11.8
⫺2.5 (⫺11.8 To 0.5)
⫺5.2 (⫺15.9 To –0.3)
19.7 (⫺6.0 To 30.6)
⫺4.8 (⫺18.4 To 8.7)
⫺7.3 (⫺18.9 To 0.8)
11.4
31.6
50.5
16.2
45.3
43.1
39.2
64.7
37.3
56.9
⫺31.7 (⫺45.9 To 19.0)
⫺7.6 (⫺21.6 To 4.9)
⫺14.2 (⫺26.3 To ⫺0.1)
⫺21.1 (⫺35.0 To ⫺8.9)
⫺11.6 (⫺25.4 To 2.4)
CAD, Coronary artery disease.
*With the exception of age, data are reported as percentages.
Table 2. Initial diagnostic tests performed, by outcome, including catheterization and provocative testing rates (data reported as
percentages).
Diagnostic Test*
Cardiac catheterization, %
Exercise ECG, %
Exercise echocardiogram, %
Dobutamine echocardiogram, %
Other, %†
ACS Negative
(nⴝ1063)
ACS Positive
(nⴝ51)
Difference (95% CI)
4.7
34.6
41.8
11.8
9.6
90.2
23.5
23.5
7.8
21.6
⫺85.7 (⫺91.2 To 74.3)
11.1 (⫺2.4 To 21.1)
18.3 (4.7–28.3)
4.0 (⫺6.9 To 9.2)
⫺12.0 (⫺25.1To⫺2.7)
*Performed within 24 hours of enrollment.
†
Adenosine myoview, thallium stress test, or cardiac magnetic resonance imaging (varies by center).
lower proportion (5.0%; 95% CI 3.8% to 6.5%) compared
with attribute matching or unstructured estimate. No patient
with an ACI-TIPI pretest probability less than or equal to 2%
developed an acute coronary syndrome (0%; 95% CI 0% to
6.5%).
Sensitivity Analyses
The sensitivities of the 3 methods were similar: 96.1% (95%
CI 86.5% to 99.5%) for unstructured estimate, 98.0% (95% CI
89.6% to 99.9%) for attribute matching, and 100% (95% CI
93.0% to 100%) for ACI-TIPI. The specificities, however, were
higher for both the attribute matching (27.4%; 95% CI 24.7%
to 30.2%) and unstructured estimate (26.1%; 95% CI 23.6%
to 28.7%) versus ACI-TIPI (6.1%; 95% CI 4.7% to 7.9%).
Figure 3 displays the prevalence of acute coronary syndrome,
stratified by pretest probability, using 4 predefined categories for
each of these 3 methods. Areas under receiver operating
characteristic curves were 0.78 (95% CI 0.70 to 0.86), 0.73
(95% CI 0.65 to 0.78), and 0.51 (95% CI 0.44 to 0.58) for
unstructured estimate, attribute matching, and ACI-TIPI,
respectively.
Volume , .  : May 
LIMITATIONS
We studied a low-risk population with a very low prevalence
of disease. As a consequence of a low prevalence of acute
coronary syndrome, the misclassification of a small number of
outcomes would result in significant changes. Thus, we cannot
exclude small but important differences between the 3 methods.
This is reflected in relatively wide CIs.
Our study was designed and completed before the
publication of the standardized reporting guidelines for studies
evaluating risk stratification of ED patients with potential acute
coronary syndromes.15 However, with the exception of
cholesterol status and amphetamine use (although cocaine use,
specifically, was an exclusion criteria), all of the core data are
reported for this study. Additionally, we did use 60% coronary
artery stenosis as an outcome criterion instead of 50% or 70%,
as recommended by these guidelines.
This study was also designed and initiated before the
publication of the Standards for Reporting Diagnostic Accuracy
(STARD) initiative, which delineates criteria for studies of
diagnostic accuracy.16 However, all applicable criteria have been
reported in this manuscript.
Annals of Emergency Medicine 443
Quantitative Pretest Probability Assessment for Acute Coronary Syndrome
Mitchell et al
Attribute Matching
250
200
200
Frequency
Frequency
Physician's Unstructured Estimate
250
150
100
50
150
100
50
0
0
0
10
20
30
40
50
60
70
80
90
100
0
10
20
Pretest Probability (%)
30
40
50
60
70
80
90
100
Pretest Probability (%)
ACI-TIPI
250
Frequency
200
150
100
50
0
0
10
20
30
40
50
60
70
80
90
100
Pretest Probability (%)
Figure 2. Distribution of pretest probability assessment data for A, physician’s unstructured estimate; B, attribute
matching; and C, ACI-TIPI. A pretest probability lower than the predefined testing threshold (ⱕ2%) was identified in 293
patients (26.3%; 95% CI 23.7% to 29.0%) by physician’s unstructured estimate, 304 (27.3%; 95% CI 24.2% to 30.0%) by
attribute matching, and 56 (5.0%; 95% CI 3.87% to 6.5%) by ACI-TIPI.
Data were collected on patients evaluated within chest pain
centers at academic centers. We cannot extrapolate to the general
community practice of emergency medicine. We do believe that,
because of the size and multiple populations represented, the
database allows attribute matching to be useful in urban
populations.10 More data from the community setting are needed
to test whether or not the database engine used by our attributematching system needs to be replaced or augmented.
The adoption of methods of pretest probability determination
in clinical practice has been limited. Using conventional definitions,
this study elevates attribute matching to the third of 5 levels of
validity.17 Even if fully validated, quantitative pretest probability
assessment by attribute matching (or any method) may not be
adopted by physicians. Baxt et al 18 derived and validated a neural
network aid that produced outstanding sensitivity and specificity,
88.1% and 86.2%, respectively. Yet, attempts at implementing the
444 Annals of Emergency Medicine
system were met unenthusiastically by emergency clinicians at the
home institution. We speculate that multiple behavioral, social, and
intellectual reasons contributed to this resistance. Additionally, use
of pretest probability to obviate formal diagnostic testing may have
psychological effects on physicians or patients that we did not
measure.19,20
DISCUSSION
In this prospective, multicenter study, we tested whether
quantitative pretest probability could function as a safe and
efficient method to rule out acute coronary syndrome in
patients selected for chest pain unit evaluation, provided that
the pretest probability estimate was at or below 2.0%. We
examined 3 methods of quantifying pretest probability: the
physician’s unstructured estimate, the attribute-matching
method, and a logistic regression equation, ACI-TIPI. When
Volume , .  : May 
Mitchell et al
Quantitative Pretest Probability Assessment for Acute Coronary Syndrome
Proportion with ACS, %
25
20
Physician’s unstructured estimate
Attribute matching
ACI-TIPI
15
10
5
2
0
<2%
2 to 10%
>10%
Predicted Pretest Probability
Figure 3. Observed proportions (with 95% CIs) of patients
with acute coronary syndrome, stratified into 3 categories
of pretest probability predicted by each of the methods.
The diagnosis of acute coronary syndrome for the observed
proportions was from adjudicated review of the results of a
chest pain unit protocol and 45 day follow-up. The dotted
arrow shows the test threshold.
compared to a criterion standard that included paired biomarker
measurements, a provocative test, and 45-day follow-up, all 3
methods had high sensitivity. The physician’s unstructured
written estimate and attribute matching forecast more than one
quarter of all patients to have a 2% or lower probability of
developing acute coronary syndrome. The observed frequency
of acute coronary syndrome in this subgroup was 0.3% (95%
CI 0% to 1.8%) and 0.7% (95% CI 0% to 2.4%) for attribute
matching and unstructured estimate, respectively. In
comparison, 0.4% (95% CI 0.1% to 1.0%) of patients
experienced acute coronary syndrome after a negative chest pain
unit protocol result.
This study has several differences from previous studies of
predictive instruments for acute coronary syndrome. The
Goldman Predicative Instrument was derived from a large
sample of patients studied in 1988 but was based on the
primary outcome of myocardial infarction at 72-hour follow-up
but did consider other important patient-oriented endpoints
such as need for urgent revascularization within the subsequent
weeks.21 Because many ED patients do not have reliable access
to medical care,22,23 coupled with new standards of treatment
for unstable angina,24 we submit that this is not an acceptably
rigorous criterion standard for the low-risk ED population.
Additionally, the Goldman et al 21 study was not specific to
low-risk patients; approximately 40% of the patient population
developed a myocardial infarction within 72 hours of
enrollment. Reilly et al 25 demonstrated that the Goldman rule
does improve the triage of patients admitted from the ED to a
cardiac care unit versus a chest pain evaluation unit or
monitored floor bed. However, this does not address the clinical
question posed in this study: how to determine which patients
require a diagnostic testing for acute coronary syndrome versus
Volume , .  : May 
those for whom diagnostic testing is unnecessary or even
harmful.
Walker et al 26 and Marsan et al 27 derived and tested an
implicit categoric decision rule for the evaluation of chest pain
in younger patients (24 to 39 years old). This rule uses a normal
ECG and clinical data to categorize approximately 30% of
patients younger than 40 years into a low-risk category defined
as a less than 1% risk of acute coronary syndrome at 30 days. In
their study, this rule substantially reduced the frequency of
disease from 5.4% to 0.14%.27 Our patient population
included 160 patients (14.4% of the population) younger than
40 years, 114 of whom were low risk by the Marsan et al 27 rule,
which would have decreased testing to a greater degree than the
use of the ACI-TIPI method. We did not specifically ask about
hypercholesterolemia, as is required by the Marsan et al 27 rule.
One of these 114 low-risk, young patients in our study went on
to develop an acute coronary syndrome.
Quantitative methods of pretest probability assessment offer
advantages over decision rules that do not categorize patients
with a discrete percentage estimate. Quantitative assessment
allows a more precise comparison of pretest probability to the
precomputed test threshold, which defines an evidence-based
threshold. In this report, a positive test result occurred when the
pretest probability exceeded 2%, and a negative test was 2% or
less. Another theoretical advantage of a discrete estimate of
pretest probability also allows the user to combine the pretest
probability with a likelihood ratio associated with a positive or
negative provocative test to generate a discrete percentage
estimate of posttest probability. Similarly, we anticipate that
physicians may consider the use of a combination of methods of
pretest probability determination with or without the
measurement of troponin or other cardiac markers. These
results can then be documented and used to provide an accurate
reference point for discussions with the patient about the next
step after provocative testing. We did not test this theoretical
aspect of pretest probability.
Previously published decision rules were derived based on the
common belief that physician judgment alone is inadequate in
distinguishing patients with acute coronary syndrome from
those without acute coronary syndrome. In the present study,
clinicians documented that their unstructured pretest
probability of acute coronary syndrome was less than or equal to
2% in one quarter of patients but still thought that chest pain
evaluation unit testing was warranted in all. These data imply
that physicians either do not trust their own impression of very
low risk for acute coronary syndrome as a sufficient diagnostic
test to rule out acute coronary syndrome or are unaware of an
appropriate or well-established test threshold. Physicians may
use an indefinable threshold generally considered to be “safe.”
The actual frequency of acute coronary syndrome was very low,
at 0.7% for patients with a pretest probability less than or equal
to 2%, determined by physician’s unstructured estimate, but the
upper limit of the 95% CI exceeded 2.0%. Previous studies
have concluded that physician judgment was unreliable
Annals of Emergency Medicine 445
Quantitative Pretest Probability Assessment for Acute Coronary Syndrome
Table 3. Summary of diagnostic indexes for pretest probability
assessment.
Estimated PTP
Physician’s unstructured
estimate*
⬎2%
ⱕ2%
Total
Attribute matching†
⬎2%
ⱕ2%
Total
ACI-TIPI‡
⬎2%
ⱕ2%
Total
ACS
Positive
ACS
Negative
Total
49
2
51
772
291
1,063
821
293
1,114
50
1
51
760
303
1,063
810
304
1,114
51
0
51
1,007
56
1,063
1,058
56
1,114
AUC, Area under the curve; IQR, interquartile ratio; PTP, pretest probability.
*Median PTP, %: 4 (range 0-100, first to third IQR⫽2-10). Sensitivity: 96.1%
(95% CI 86.5% to 99.5%). Specificity: 27.4% (95% CI 24.7% to 30.2%). Likelihood ratio negative: 0.14 (95% CI 0.04 to 0.48). AUC: 0.78 (95% CI 0.70 to
0.86). Potential decrease in testing, %: 26.3 (95% CI 23.7% to 29.0%).
†
Median PTP, %: 5 (range 0-100, first to third IQR⫽1 to 8). Sensitivity: 98.0%
(95% CI 89.6% to 99.9%). Specificity: 27.4% (95% CI 23.6% to 28.7%). Likelihood ratio negative: 0.07 (95% CI 0.01 to 0.36). AUC: 0.71 (95% CI 0.65 to
0.78). Potential decrease in testing, %: 27.3 (95% CI 24.7% to 30.0%).
‡
Median PTP, %: 10 (range 0.5 to 85.6, first to third IQR⫽5 to 17). Sensitivity:
100% (95% CI 93.0% to 100%). Specificity: 6.1% (95% CI 4.7% to 7.9%). Likelihood ratio negative: 0 (95% CI 0 to 1.33). AUC: 0.51 (95% CI 0.44 to 0.58).
Potential decrease in testing, %: 5.0 (95% CI 3.8% to 6.5%).
Mitchell et al
Supervising editor: Judd E. Hollander, MD
Author contributions: This study was designed by JAK, who
also obtained research funding. AMM, AC, DD, and JAK
supervised and conducted the study, including the recruiting
of study participants, collection of data, and conduction of
follow-up. JLG, AC, and CVP performed adjudication of study
outcomes. Primary data analysis was performed by AMM, who
drafted the manuscript. JAK and CVP provided advice on
design and statistical analysis. All authors contributed
significantly to the revisions of the manuscript. JAK takes
responsibility for the paper as a whole.
Funding and support: This study was funded by an Emergency
Medicine Foundation–Riggs Policy grant 2003-2004. Jeffrey A.
Kline is an inventor on a patent (pending) related to attribute
matching and owns stock in BreathQuant Medical Systems Inc.
Publication dates: Received for publication May 17, 2005.
Revisions received August 3, 2005, and September 20,
2005. Accepted for publication October 5, 2005.
Reprints not available from the authors.
Address for correspondence: Jeffrey A. Kline, MD, Department
of Emergency Medicine, Carolinas Medical Center, PO Box
32861, Charlotte, NC 28323-2861; 704-355-7092, fax 704355-7047; E-mail [email protected]
REFERENCES
inasmuch as physicians erroneously discharged 2% to 5% of all
ED patients with acute coronary syndrome.6,28,29 However,
these studies asked physicians to classify patients into the binary
categories of “acute coronary syndrome” or “no acute coronary
syndrome,” a different method than asking physicians to
estimate a patient’s percentage probability of developing acute
coronary syndrome during the next 45 days. Additionally, these
studies examine discharges in the entire population with acute
coronary syndrome as opposed to discharges among patients
with a prevalence of acute coronary syndrome that is lower than
2%. It remains unknown whether the use of any method of
determining pretest probability will have an effect on the overall
rate of missed acute coronary syndrome.
Among low-risk ED patients referred for chest pain unit
evaluation for possible acute coronary syndrome, attribute
matching and physician’s unstructured estimate categorized
significantly more patients than the ACI-TIPI logistic regression
equation as having a pretest probability less than or equal to 2%.
Patients who had a quantitative pretest probability less than or
equal to 2% determined by any method had an extremely low rate
of acute coronary syndrome on 45-day follow-up (Table 3).
The funding for this study was provided by an Emergency
Medicine Foundation–Riggs grant. We would also like to thank the
Emergency Medicine Cardiac Research Group for access to the
Internet Tracking of Acute Coronary Syndromes Study Group
database.
446 Annals of Emergency Medicine
1. Storrow AB, Gibler WB. Chest pain centers: diagnosis of acute
coronary syndromes. Ann Emerg Med. 2000;35:449-461.
2. Physician Insurers Association of America. PIAA Claim Trend
Analysis. Rockville, MD: Physicians Insurers Association of
America; 2004.
3. Reilly BM, Evans AT, Schaider JJ, et al. Triage of patients with
chest pain in the emergency department: a comparative study of
physicians’ decisions. Am J Med. 2002;112:95-103.
4. Freas GC. Medicolegal aspects of acute myocardial infarction.
Emerg Med Clin North Am. 2001;19:511-521.
5. Pilote L, Granger C, Armstrong PW, et al. Differences in the
treatment of myocardial infarction between the United States and
Canada: a survey of physicians in the GUSTO trial. Med Care.
1995;33:598-610.
6. Graff LG, Dallara J, Ross MA, et al. Impact on the care of the
emergency department chest pain patient from the chest pain
evaluation registry (CHEPER) study. Am J Cardiol.1997;80:563-568.
7. Davis C, VanRiper S, Longstreet J, et al. Vascular complications
of coronary interventions. Heart Lung. 1997;26:118-127.
8. Pauker SG, Kassirer JP. The threshold approach to clinical
decision making. N Engl J Med. 1980;302:1109-1117.
9. Pauker SG, Kassirer JP. Decision analysis. N Engl J Med. 1987;
316:250-258.
10. Kline JA, Johnson CL, Pollack CV, et al. Pretest probability
assessment derived from attribute matching. BMC Med Inform
Decis Mak. 2005;5:26-37.
11. Pozen MW, D’Agostino RB, Selker HP, et al. A predictive
instrument to improve coronary-care-unit admission practices in
acute ischemic heart disease: a prospective multicenter clinical
trial. N Engl J Med.1984;310:1273-1278.
12. Lau J, Ioannidis JP, Balk EM, et al. Diagnosing acute cardiac
ischemia in the emergency department: a systematic review of
the accuracy and clinical effect of current technologies. Ann
Emerg Med. 2001;37:453-460.
Volume , .  : May 
Mitchell et al
Quantitative Pretest Probability Assessment for Acute Coronary Syndrome
13. Kline JA, Mitchell AM, Runyon MS, et al. Electronic medical
record review as a surrogate to telephone follow-up to establish
outcome for diagnostic research studies in the emergency
department. Acad Emerg Med. 2005;12:1127-1132.
14. Alpert JS, Thygesen K, Antman E, et al. Myocardial infarction
redefined: a consensus document of the Joint European Society
of Cardiology/American College of Cardiology Committee for the
Redefinition of Myocardial Infarction. J Am Coll Cardiol. 2000;36:
959-969.
15. Hollander JE, Blomkalns AL, Brogan GX, et al. Standardized
reporting guidelines for studies evaluating risk stratification of
emergency department patients with potential acute coronary
syndromes. Ann Emerg Med.2004;44:589-598.
16. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and
accurate reporting of studies of diagnostic accuracy: the STARD
Initiative. Ann Intern Med2003;138:40-44.
17. McGinn TG, Guyatt GH, Wyer PC, et al. Users’ guides to the medical
literature: XXII: how to use articles about clinical decision rules:
Evidence-Based Medicine Working Group. JAMA. 2000;284:79-84.
18. Baxt WG, Shofer FS, Sites FD, et al. A neural network aid for the
early diagnosis of cardiac ischemia in patients presenting to the
emergency department with chest pain. Ann Emerg Med.2002;
40:575-583.
19. Sox HC Jr, Marguelies I, Sox CH. Psychologically mediated effects
of diagnostic tests. Ann Intern Med. 1981;95:680-685.
20. Goodacre S, Nicholl J. A randomized controlled trial to measure
the effect of chest pain unit care upon anxiety, depression, and
health-related quality of life. Health Qual Life Outcomes. 2004;2:
39.
21. Goldman L, Cook EF, Brand DA, et al. A computer protocol to
predict myocardial infarction in emergency department patients
with chest pain. N Engl J Med. 1988;318:797-803.
22. Richardson LD, Hwang U. Access to care: a review of the
emergency medicine literature. Acad Emerg Med. 2001;8:10301036.
23. US General Accounting Office. Hospital Emergency Departments:
Crowded Conditions Vary Among Hospitals and
Communities.Washington, DC: US General Accounting Office;
2003. Publication GAO-03-460.
24. Braunwald E, Antman EM, Beasley JW, et al. ACC/AHA 2002
guideline update for the management of patients with unstable
angina and non–ST-segment elevation myocardial infarction: a
report of the American College of Cardiology/American Heart
Association Task Force on Practice Guidelines (Committee on the
Management of Patients With Unstable Angina): 2002. Available
at: http://www.acc.org/clinical/guidelines/unstable/unstable.pdf.
Accessed April 5, 2005.
25. Reilly BM, Evans AT, Schaider JJ, et al. Impact of a clinical
decision rule on hospital triage of patients with suspected acute
cardiac ischemia in the emergency department. JAMA. 2002;288:
342-350.
26. Walker NJ, Sites FD, Shofer FS, et al. Characteristics and
outcomes of young adults who present to the emergency
department with chest pain. Acad Emerg Med. 2001;8:703708.
27. Marsan RJ Jr, Shaver KJ, Sease KL, et al. Evaluation of a clinical
decision rule for young adult patients with chest pain. Acad
Emerg Med. 2005;12:26-31.
28. Christenson J, Innes G, McKnight D, et al. Safety and efficiency
of emergency department assessment of chest discomfort. CMAJ.
2004;170:1803-1807.
29. Miller CD, Lindsell CJ, Khandelwal S, et al. Is the initial
diagnostic impression of “noncardiac chest pain” adequate to
exclude cardiac disease? Ann Emerg Med. 2004;44:565-574.
Did you know?
You can personalize the new Annals of Emergency Medicine Web site to meet your individual needs.
Visit www.annemergmed.com today to see what else is new online!
Volume , .  : May 
Annals of Emergency Medicine 447
What is your estimate to the nearest 1 (whole)% of the of the probability that this patient will require
coronary angioplasty, coronary stenting, or bypass grafting, or will suffer myocardial infarction or die
within the next 45 days?_______________
Age:
_______
Gender: M F
First symptom_______________
Race: B W A H Other
How long ago did it start?____________
Characteristic
Yes
No
Chest pain chief complaint
Left arm pain chief complaint
Diaphoresis
Chest Pain still present
Chest pain worse than usual angina
Dyspnea
Nausea or vomiting
Chest pain reproduced with palpation
Current smoker
History of CAD
Diabetes Mellitus
Hypertension
Positive Family History of CAD
Recent Cocaine use
ED visit for similar symptoms within 7 days
ECG Characteristics
2 or more leads
1 Lead
No Leads
ST depression in >1 lead?:
0.5-1 mm
1 – 2 mm
> 2 mm
T wave inversion in >1 lead?:
Flattening
1 – 5 mm
> 5 mm
Pathological Q waves?
aVL or II
aVF or III
Precordial
Hyperacute Twaves?
aVL or II
aVF or III
Precordial
Appendix E1. Data collection form completed by evaluating physician prior to chest pain unit testing. This form was
photocopied and sealed when the patient was enrolled in the study. This form was used to record the physician’s
unstructured estimate of ACS and to create and attribute matching profile and calculate the ACI-TIPI score.
447.e1 Annals of Emergency Medicine
Volume 47, .  : May 