Error and its Meaning in Forens¡c Science*

.l l'.t¡r¿nsic.lcr. Januar¡, 2014, Vol.
\\,itlì
iììlilgc.
Itrcl¡¡.n.U'c'ss/lìl sevii¡.
ninittion n¡4
Sctts Ng¡¡a¡
on ol it
tkri:
PAPER
]tsic Sci
lnt
No.
1
I
227.s
Availlblc olllinc ¿t: orrlirrcliltrat y.u,i lc;'.ç,r,'rt
GENERAL
M. Cltri.stcn.;t'n,t Ph.D.: Chri.stiutt M. Crov,rler,2 Ph.D.; Ste¡then D.
Httttt'k." Plt.D.
M.
Mttt
A¡,(,i
tcla-
-59.
10. I I I I/1.5.5(r-.1029.
Otr,sle¡,,3
Ph.D.;
un¿
Error and its Meaning in Forens¡c Science*
br'l\\/ocrl intc(lbscÍvcr cnols. uncc|tairrty, variati()n, ancl nlistakcs.
KIÌYWORI)S:
lìrrcnsic scicncc. crtot. lintitation, fìrcnsic anthlo¡lology. 1)r¿rrl¡r¡1, ¡lristako
Drscussion rcganling "crror"
in
I'rrrcnsic scicncc analyscs
gairrctl rnorrrentum lìlllowing Ll'tc l)uttbcrt rulitrg (I) ancl h¡s
intcnsil'iecl with thc National Acaclerny ol' Scienccs' National
Rescalcl-t Cor¡ncil Re¡rort ,ttrerr,r¡thettirt¡4 I'i¡ren.tit' Scietrcc itt Ílta
UtÌitKl Stut(s: A Pttlt l;'t¡nwtnl (2). Thc lolc ol' scicncc within
the ludicial system is nothing novcl: howevel', the l'ocus has
shillcd
to
incluclc the evaluation
ol'llethotls lnd
tcchnic¡ucs
Iathcr than simply the cxpcrt's interplctation ol'thc lcsults.
Estlblishing scicntil'ic valiclity is challcnging rvithin thc lolensic
scicnccs consiclering that thc conccl)t of elrol has clif'lèl'ent lncanings ancl functions in the courlroorl conrpalctl with thc rcsealclt
settirrg (3). Estintating lncthorl validity ancl untlcrstanding clxrr
iìle ¡rìrportant, however, rcgarcllcss oJ' whcthcl co¡tclusions encl
The conce¡rt ol' en'or has bccn problcnratic, lncl too ofìen, thc
courts ¿rs wcll as lbl'cnsic plactitioncls rr-risundcl'stancl thc tncaning ol' crror as it rclatcs to fìrrcnsic scicncc rcscalch, ¡tlocctlurcs,
ittttl tcchnic¡ues. Elrot'can be defined in ¿r nuntbcl ol'rvays
inclurling the lbllowing: arì itct, asscl'tion. ol bclicl' that unilltctìtionllly rleviates lì'ont what is corrcct. r'ight, ol tnlci thc cotì(litiorr ol' having incon'ect or lirlse knowlerlge; thc ¿ìct or iìn
instlrnce ol' cleviating lionr an lccc¡ttccl codc ol' behaviur; <lr a
mistakc. Mathcntatically and statistically. crlol nray rcf'cl to the
l(
ìcolgc Mason [.llrivclsily. Fairlìrx. Vr\.
of'Chiel-Mcclical IÌxanrincr. Ncl, York (litv. NY.
'\lercylnrrst Ullivcrsir),. l-.r'ic. l)¡\.
'l)cplltnrent ol' Iìorcrrsic Scic¡¡ccs. CÌrlrsolirlalctl lìo¡cnsic l,aboluloly,
-Of l'ice
Wlslri¡rgton. tX'.
'i'l'resented at thc 63¡tl Ânnual Mccting ol thc Antcricarr Acatlcnty of'
lìt¡r'rtsic Scicnccs, l-lblLrirly 20 26. 2()l l. irr Chicago, IL. 'l-he lc'scllch
¡tre\ctlletl ilt lhis rrrrllruseli¡rl rvas n0t c0lrtltrctcrl rIltlcr thc aus¡ticcs ol'tltc Ncrv
Yrrrk ('il),Ollicc ol ('hir.l Mcrlical Flx¡rnincr (NYC-OCME.) or the Del)iut¡lttrlt ol'Forensic Scienccs (Dtis). T'hc o¡tiniotrs cx¡rrcssctl hclein arc lhosc ol'
llìc ,urthors antl tlo not tcllcct thc opinions ol
tlìc ÑYC-OCME ol rhe DFS.
lìeccivcrl 2
l0I
lìllcnsic scicnce lealtìl catì rcsult lì'onr ¿ì nuntbcr ol' rlil'l'el'ent
contributing to thc complexity in urrderstancling thc
c¿ìr.lses.
potcntial soul'cc(s) ol' et'rol'. The convcrgcncc ol' science alìcl law
has maclc thc idcntification ancl interpletation ol'error in lhc
courtroonì arr even _greater challcngc. cs¡tccially as ol'ficers ol'thc
coult typically lack a scicntific backgrouncl ¿rn(l spccifìc knowlcclgc rcgalcling cn'(n' analysis. Thus. thc conccpt of et'ror is often
vaguc ancl subject to a variety ol'intelpretalions.
Achnissibility criteria f'ol cx¡rert tcstirnony in thc Unitctl States
wcrc lcclcl'inccl in thc 1993 Suprcnrc Cc¡ttl| Duubctr clecision (l)
in signilìcant changcs in ancl intcr'¡rretation of the
Fcclclal Rulcs of Evirlcncc Rule 702 (4). The Dttttl¡ert critct ia
rl,clc intcnclecl to plovicle guiclelines l'or arhnitting scientifìc
cxl)crt tcstinlony Lo cnsurc its rcliability ancl validity. In f'cclcral
cascs irncl in statcs that havc adoptccl tlte l)nubert ct'itel'ia, trial
iuclgcs might consirler the follorving lictors to asscss thc aclrlrissibility ol'scientif ic or tcchnical cxpcfl testimony: (i) rvhethcl thc
thcot'y ot' tcchnic¡uc in rprcstion can bc (ancl has been) scientifically tcstccl. (ii) whether it has been sub.jectecl Io ¡tccr rcvic'uv
lesulting
up irr court.
^
()cr.
ilil'lèrence l'retwecn ¿l colnputc(l clr rneasurecl valuc antl a truc or
thcorctically corrcct valuc.
Consiclcling thcsc clcf initions, it is apparent that error in thc
JL¡ne
2012;
lntl ill
lei,isctl lìrr'¡n 2(r Scpt. 2012: irccc¡rtctl
(iii) its known or potcntial effol rrtc, (iv) thc
While the turnult surlounding the ¡rotential im¡ract ol'thc
I)uubcrt ruling on lhe lìlcnsic scicnccs sccnlingly began to clissipatc ovcr thc ycals, thc challenge to tlre folensic science'conlnlunity r.vas lenervecl u,ith the release ol' the National Acaclenry
of' Scicnces' National Rcscalch Council Rcport Slrcrr.r¡thcttin¡4
Ft¡ren.tit' Scietu't' in tltc llni¡cd SÍarc.ç: A Paîh Fonwnl (2). This
clocunrcnt outlinccl thc scicntific ancl tcchnical challcngcs tìrat rnust
be nlet in olclel lìrl the fìlrensic science entcr¡rrisc in the Unitccl
States to ol)cratc ¿rt its lirll ¡rotcntial. In the Cor¡ncil's opinion,
sonrc disciplirìcs wcrc lìluncl
2.
!. 2t)13 ¡\ntclicalr Acurlclny
27
ancl publication,
cxistcncc ancl tnainlenance ol' standalcls contlolling its opclatiort.
ancl (v) whcthcr it has attractccl widespread acceptance lvithin a
rclcvant scientific comnrunity ( I :-593-94).
ol [-ìr'cnsic Scicllccs
to lack scie ntilic ligur. lcatling
123
124
JOURNAL
oF FoHENSIC SCIENCES
leconìnrcrìcl¿rtions thlt enr¡rhasizctl tlte llcecl lbt' itlct'casecl
anrl inr¡'rxrvcd lcse¿rrch. In ¡ralticular, [ì.ccotrtnicntlation three
statcs that rcsea|ch is nccclccl tO cslablish the valiclity ()f'íbrcnsic
mcthods. rlcvckr¡r anil cstablish quanti{'iablc l.ncasul'cs of the
lcliability anrl acculacy of fblensic analyscs, ancl rlcvclo¡t qualltifiablc rneasules ol r¡ncert¿rinty in thc conclusions of lbrensic
to
¡
analyscs.
Whilc thc NAS lcport specifically calls ¿ittention to the lack
of scicntil'ic tcsting
ancl clevelopment ol' known crlor lates fìtr'
nlarì)/ fblcnsic methocls, ntorc gcnclally, the report rclrinrls us
that as scientists lr'¿ ,,1¡t.r/ do .ytiettcc l,¿1i. Furthcrtlorc. as fbrensic plactitioncrs, wc lìlust be cognizant of' thc concerns ol the
legal conrrnunity. This inclLrdcs unclcrstancling how thc Coults
view ancl evalr¡atc scicntiJ'ic cviclence. being proactivc in eclucating thc lcgal conrnrunity aboul the scicntific l)roccss, arrcl being
¡rrcplrcrl to nritigate rnisintcr'¡tlctation ancl lnisuntlcrstanrling of
scientific results.
Unclerstanding "I4rror"
Prirrr tcr |l'tc I)uttl¡crÍ clecision, llcthocl r'eliability and valiclity
rìot specif ically rcc¡uilccl tcl be considcrccl; thc admissibility
of'scientif ic cviclcnce was a mattcr ol'tlte general acceptancc tcst
r¡nrlcr' 1in,¿, (,5). In thc Douhert ilecision, "reliability" was usccl
lepcatcdly [o nlcarì "clepenclability," which incorporates both scicntil'ic leliability ancl valiclity. "lìcliabiliry" in thc scicntific sense
is olien ex¡rrcssccl in hor.v clil'{èrcnt obsclvcl's tìleasure or scot'c
thc sanrc phenornenon clif'fircntly, ancl there are a variety ol' statistical nrcasurcs to quantily reliatrility rlc¡rcnding on the typc of
obsclvation (6). Highly lclia[rlc obselvations shorv vcry low or
no interobscrvcl valiability and high lepeatability. Reliability,
horvcvcr. is not suf'l'icicnt lìrl cstablishing validity.
Valiclity can bcst be thought of' as thc ovcrall probability ol'
\¡,/ere
lcaching the
corre
ct conclusion,
give
na
spccif'ic nlcthocl
ancl
clata. Methocls that alc consiclel'cd "valicl" give us the corrcct
conclusion rnore ofien than chance, but st¡lle havc highel valiility than othcrs ancl will give us lhc wrong answers lcss olicn. In
this scnse, "validity" is clcally what the Coul't harl in lnincl when
it enrplrasizccl "r'cliability." Inclcccl, since the l)uuhert rlccision,
c¡ucstionablc rÌcthod rcliability/valiclity has bccn thc rìlost fì'c-
(lucntly citccl leason fbr cxclucling
or limiting
tcst¡rìlony on
lìrlcnsic iclentif icatinn scicnces (7).
While thcrc itrc rnally aspccts of el'ror that will influcnce
valiclity, the known rate of en'or proviclcs a scientif ic lncasul.c of
a n.tcthocl's valiclity ancl that is likely why it ri,as incor'¡tor.atecl as
pitrt of fhe l)uul¡ert guiclelines. Of' oourse, error rates are nol
knowrr, brrt cstirnatecl; tl.tc potentiul error l.¿ìte of any rncthocl is
100%. Thc error ratc guicleline, howcvcr, has of tcn cr.catcd ntor-e
conlìtsion than clarif ication. We attempt hcrc to simplify the cliscussion by dcscribing the cli{'f'cr.cnccs between several gcnct.ally
recognizccl potential s()ut'ccs of error.: practitioncr. er.r.or.. instr.unìent cl'rot'. stalistical ct.r.Or., ancl tìlctho(l en.(n..
Plactitioner el'rot' rcf'crs to a nristakc ()r opcrator (human) error.
It nray bc l'anckrm or systenlatic, may be relatecl to ncgligerrce tlr
inconr¡rctcnce, ancl is, fbr thc most part, unintcntional and unr¡uantil'iablc (8,9). Pr.actitioncl. en.or nray rclir. to blunder.s such
as trans¡tosirrg nuntbcl.s whcn rccol.cling clata. incol.r.ect instr.u_
Inent use. scleclion of inappropriate rnethocls, or. iurpnrper.
nlcthocl application. Practitioncr. cr.ror. rììay also be intcniioial,
suclr as tìauclulent behavior. whilc practiti.rìc' crrol. is ccrtainry
a co'cel'n of the courts,
Practitioncr
'ecluccd
c'or
it is not c'ro' in thc scientif ic scnsc.
is cxccccringly clif icurt k) estirì1atc but ca. bc
thr.ugh c¡'alily ass.r.a'cc systetìls. tr.aining. pr..ficicncy
aclhe'ing
lÏlilì,'ï;,'iîî-,'i,î,',".i..1]"'
t.
T
r'aliclatctl ¡r*rt.c'ls
^n.,
I
lnstnìrìlent (or technological) crrol can be clef ined as thc .l;" I
1èr'errce betwccn an inrlicatecl instlurnent valuc and,h",,.,ulì
I
(tluc) r,aluc. Instrurtrents shoulcl be caliblatctl agirinst ,, rt.,¡,la,,i
I
(i.e., a certifìed ref'erence nratclial), but cvcn whcn ¡lo¡rcr.ly c,,¡i
I
bratccl, thcy typically havc a prcsclibed. acceptable ,,rn,,,,,,, n, I
cll'or l,hich has been cletet'rninecl by thc instrurncnt's nìirrrLrlac- |
tut'el'. lrìslnlnlerìt en'or is nreasuretl in val'ious witys stittist¡eallv
I
lunrl cun llc nrilrinlizcrl hy ¡rlo¡rcl nririntcniurcc irntl crrlibtali,,r¡ 0f
t
instl't¡rncnts as a p¿Ìrt o1' a laboratory c¡uality ¿rssut¿ìtìcc l)n)!ratt.
I
ltut sorlrc acceptable iìrr()unI of'crrrlr is unclcrstoorl lrnrl thcr.clìlre I
rccogrrizctl ttr cxist.
I
Statistical elnrl is the cleviation betwcen actu¿rl ¿rnrl ¡rrctlicterl I
values, gerrerirlly estinratecl by thc stantlanl clror ol otllel nlcl\qre I
ol unccltainty in plcdictiorr, lìll exant¡lle when a ¡rrcrlictiotr irrter- I
val with an cxplicit plobability is spccif icrl. Statisticll en'ot trlis¡ |
nrclely expresscs norrnal variability ancl is inhelent in nleasure- I
lnents ancl estinratcs bccause they are basecl on thc ltrrr¡tcrtics of n I
sanrple. lt is possible th¿r( thc actual value ol ¿ì measurcnlerìr or
I
estimate nray firlloutsirlc ol'the preclictiorl intcrval.
I
I-astly. nrcthocl (or techniquc) crrol rclatcs to i¡therent li¡nita- I
tions that ltavc nothing to clo with practitioner error or brcaktlori ns I
in tcchnology (8 l0). Method cn'ol is oficn a lirnctit'lt ol'how I
nleasurerìlents cll traits clvclla¡r antong clil'l'elent gror.tl)s or to tlte I
licr¡ucncy ol'the obselvccl tlait(s) in thc ¡topulation at lnlge. WIrile I
these limitations arc not ttre¡nselves "errors," they al'll.ct thc sr.¡lsiI
tivity or lcsolving power, proltative valuc. ancl ultirrrltcly valirlity I
of the rnethocl. 'Ihc nlclt'c rarc tl'ìat a trait or suitc ol ttrits is in ¿r I
po¡rulation. thc rnole scnsitivc that nlcthocl is for associatirrg the I
trait(s) to a particular inctivictual, itent r¡r grou¡r. F-ol exarn¡rlc. I
nuclcal DNA has grcater resolving ¡rowcl lbr cleternrinirrg itlcntiry I
than nltDNA lrccausc the sarne r¡tDNA occuls nlore frequently in I
thc ¡ropulation at large. Thc pelvis has gt'eater lesolvirrg ¡t<twcr' I
than the skull in clctcrntining sex fì'orl skclctal lcnluins bccausc
I
thcre is grcater overlap bctwccn thc scxcs in l'c¿rturcs of the skull I
(becausc thc pclvis is rnorc sexually climot'phic than the skull).
I
Eslimations of'ntcthocl cn'om are the llost fànliliar'. A lì'cc¡rrcn- |
tist cstinration of an elror l'ate is lrasctl on prcvious lcsults trlonc: I
a 997o valicl nletho(l, lbl cxantple, woukl h¡vc a lrl¡ enol late in I
thc long run. A lìayesian cstintation of an errol rirtc is buscd orr I
prcvious resrrlts anrl the specific otrservations ol clata bcing ana- |
lyzctl. so the estil]latc(l crror ratc applics only to the ctse at
is no way trl ntinillizc method ellrtr' (with. lbr
hand. There
I
I
exanrplc, acklitional training ol calibration) it sinrply cxists as l
I
lìrnction of inhel'ent valiation in thc rnatclial itscll. Such linrita- |
tions, horvever, shoultl bc acknowìcclgcd ancl colnltrt¡ltictterl in
I
le¡xrlts unil tcstinì()ny.
I
The estimatcd (known or potenlial) l'atc ol' error to which thc
I
Duttl¡efl guiclelines relir can inclucle a nuntbcl of things sr¡ch ts
I
the conficlencc intclval, the statistical significance ol u lcsult, tlr
I
thc probability that iì rcportecl conclusion is incorrcct. This nrly
I
involvc any or a contbination ol' thc lìrctors cliscusscd above. hut I
most often lalgely involvcs statislical cl'r'ol' and rnetlrotl cl'lc¡r'. I
The selection of goocl lcsearcl.r dcsigns ancl appropriate stntisticul I
moclcls is impclativc to procluce valicl scicntil'ic nìeth()cls with
I
low cstinratcil ratcs of crror.
I
Thc impoltance ol' inrplcnrcnting nleasulos to llrinintize alltl
I
account fbl crror ancl limitations in folensic scienccs lnctlìorls I
shoultl now bc apparcnt. Unclerstancling ancl appropliatcly conr- |
rnunicating thcsc isst¡es can rescllvc confìrsion ovcl thc sigtrili- |
cancc ol'rcsults ancl can prcvcrìt intcntional or.unilltc-lttionul
ttlisusc
ol
e¡
rrlr'.
I
I
CHRISTENSEN ET
ols
ilncl
lhe (lìl-
)
¿tc(tìîl
L¿t¡1diìfcl
'ly cati-
lunt
oi"
anuJac-
sticaìty
Ltiul oi
ogf¿uì,
erelì¡.e
3dictc(l
ìc¿Ìsuì11
ì inlet-
l oiiel
)asulc-
:s
o1'
¡
enl or
irÌitalowns
f'how
to the
Whilc
seDsi-
rlidity
sina
g thc
rnplc,
rntily
tly ìn
)aLìse
skull
IUen-
lone:
te in
d o¡l
a¡lil-
cat
J'or'
asa
'rita-
I
in
the
I ¿ts
,of
ìrÂy
bLì
t
l-of,
ical
¡i¿h
rnd
)ds
Illifì-
r¡l
I,lislr ll(lcl'st¿rtld¡¡rg
,l'llc lllisuse
"llt'l'ot'"
ol elror
h¿ìs sclioLls
r¿rìllilìcltions
jll
both legal
('trìfcc\
,,r¡t iertlilir c,'rnDlultilic'. Wc It¡vc irlclttilìr..i IIìrcc
( rr,'r l.ll(i lrrl
¡ir
,'l
rlrc
crrL,r:
ulrril¡tll!
.l
/(¡,,
lllistls(
i¡1, ',linu
¡ i.. lrDiquc. tiit .ll,irnin¡ lllirl itlr r'rr('r r'rlc uirllll"l hc c:'lillllrlc,i.
.,
lnd (ìii) attcnptrug to citlculate el.lù-ralcs /r..) t/./à./¿) liom ¡ctivitìes lh¡l werc not inteDcled lbì tlrût pulPose sr.rch as ¡rtolicicncy
tcsts" Prolessional cxclrjscs, or- othcr sttìdies. RctìsolÌs behind
('ll iìol ulldcÌstan(liDg the llleâDj¡lg of an
thcsr: nristtscs range ft
.'crror !-¿ltc," to ìr)lìlrq)er {r¿linìng iD st¿ìlistios lrn(l the scientilic
nxrthod. lo corìceflrs that curleDl lnethods will bc cxposed as
liìcliir)g ân cnlDilicll bâsis
Sorre lorensic prôctitiolìers ìrâvc cl¡i¡rlcd lhat the elror r¿rlc
Jbr rhcìr lechûique or Illcthocl is zero (see (1Ì) lol one exarr¡rle).
l.ir cxâDrple, the lollowing tcs{ilùolly w¿ls ploviclecl by a 1ìngcr'
prinl cxâDriner it Peoplc t' Gon¡¿2. (12) cxplairìng the reasoling
bchìnd lhc zcro eùor'lale cl¡illl:
And we p|ofess âs fiDgclprint cx¿n'liDels tllat thc rate oI
euor is zcro. And the reason wc ln¿tkc tllât bold stntcìnellt
is bec¿ruse we kDow brsed on 100 ycars oJ rose¿Ìrcl] that
evclybody's liDgeryriÍrts ârc uniqLìl:. ¡lld in natt¡rc il is
nevel going {o repcal itscll'again.
'1hc lìrlìacy rn the cxpeIL's reasoning
lotion
()1-
is twolblcl, FiIst, lhc
is Prob¿ìLrjlìstic and
uniclucness rn 1ìlrcnsic science
iDll)ossible to provc ìn ¿r scientific seusc, iìnd úis forln ol logic
Jìtle\\,s indLìclive |easoning (13). Scconcl, this praolitioner' lails
ro rìndorstaud that dcspite llle strength o1'thc b¿rsis lbr fì gerPrini
iÌssociation (thrt there is a low plobability lor two i(lcntìc¡l l'rnir.crprìr)l palterns k) cxist), cùoì, (' litnitâliolls ll'l¿ly sllll exisl ill
thc colÌDarison nlelhodology. Ëlrol liìtes lelâte both to thc.li"
tlt.tctlcy ol a llârticuìar lrâil(s), as well as to how 4¿c¡l¡¿f¿ the
rììethods o1 ooDrpalison are iD dclclrl1ining an associatiol'ì or
cxclusion. llveD i1 a leâtllrc ìs '1tnique," it docs ¡1ol nrear) thal
co¡)rÞarìso¡ rÌethods cân iDlallibly dctcrntinc \\4lethcl two síìuì
l)les origiuatocl
lìun the san]c
source. Suoh rc¿rsoning olte!1
rcsults lìonl the beli0l lh¿ìt hnoltns llìây cl.I l]Llt lbrensic lcchniqlles clo r)ot; ¿l srLggestiou thâl ìs both eneneous rìn(l ulllalsifi-
rìtrlc (10). Ad¡nittedly,
cliscìplines (such as finget¡ttinl
courParison) wllcrc tllc lnelhod is prinìarily the.jLìdgtllcìtt ol tllc
cx¿DriDcl, jt cau be ilìpossiblc te disentaDgle ¡rlelhod elrol lìonl
ÞriìctiLioner euols (10). Even so, thcrc ìs ¿rlways a nonzclo probâbility oJ crror, ¿ìrd to cldiD'r iìn erÍor rate ol zclo is iûherclltly
in
r¡DscicDlilìc.
,AlterDativcly, sollre plactjtioncrs clai¡n that ¿l¡1 orlor latc silnc¿ìnnot be eslinl¿ìlcd. This misgLticled philosoplty likcly
rcsults tom r lâck ol' proPer lcslil'lg 1o dotelmlne wh¿Ìt thc
l)ly
known
nre(hocL
li
rjtirtions or potelltill lale
ol crror ll'lay be,
i¡rsr.rflìcient statìslic¡ìl t|ainir)g o1' Praclilioners, or ì'llistìDdcrstaDding rhc nrcaning oi- error altogclher. Fol liteDsic lllacLi(ioners
{haL llìâke llìis cl¿ìiDt, therc is likely a liar'thal by ackrlowledgi g
rrethod lirlìiLâtior)s ¿uì(l Polorllial error' ¡âles' thc Powcr d tlÌe
âualysis iu the coLìrt!-ü) woulcl be clininished Othel {aclics
iDvolvc cllelilìy selcctcd liìr)guagc to avoid llle issue ol el.l1¡'
ln loalily, ho\\,cvcr, iI ¿ì r]etho(l can bc lPPlicd, errol lll¿ly exist
¿ilcl shc¡uld bc aclinou'lcclgecl
ln the abseDcc ol known ot Potcntjûl error l¿ìtes l¡r a lllclhod.
it is nol ¡cccÞt¿rblc to derivc crlol rales lìoll'l Practilìollcr Proli-
cier)cy lests, llrolessioniÌl exelcjscs. or sLtìdies tllll were llot
desìgncrl to estiDl¡te lncthod elrol l¿ìles Prolìciellcy tests âre
typic¿ìlly (lesigned lo nonilor Pcrfblrl]¿lllce iln(l dclnonshille thiìt
AL.
.
ERROR IN FORENSIC
SCIENCE 125
personncl prodùce leliabìe wo¡k lìl(l lh¿lt ualytic¿ìl procedurcs
¡re con([ìclcd wilhìn estâblishe(l cliteri¡. l]xelcises dcsig¡rcd to
cxplorc the ability ol plactitìoncrs {o pct-lonl1 cerl¿ìilÌ aniìlyses
¡ltay not cl ¿rlily irs 4lrLre l:Jrolicielrcy lest, urcl caution shoulcl be
exercisc(l \\4ren rcsults ol such stt(lics iue cDrployed outsidc o1'
{lreil irìteuclcd usc.
As one ex¿unplc, ths A¡lericân Bo¡rcl of irorcnsic Oclontology
(AllF'O) in]pleDrculcd iÌ stucly developc(l as a¡ "cducalio¡¡l cxercise \\¡rosc ÞriDl¡ry pl posc wirs .lesig ed to sulvey the dcgree
o1' ¿lgrccrùent (or (lisagrcelnent) bclwcen Jbo¿r'd-ce[lilìed ABITOI
diplolÌ¿ìtcs coDlìorted wì{h cases of vatyìng îìùoLìnts ¡rd qualjty
ol llitc¡lârk cviclence" (ì4). llsìng clata ftoln thìs cxcrcjse, ¿l
lbreDsic o(lontologisl ìcÞoÌted the iìrllowing in ân ¡1fid¿ìvjt 1or
2002 Srìpreme Court ol Mississippi tlial (15):
¿i
On avcragc.63.5% 01 llìe examiucrs coulrrli(ted làlse Þositive eûols ¿rcross the lcsl clscs. I1 lhis rcllects thoir Dcrlbl.
r-ì¿urcc in âctLìiì cases, thcn ìncuþaloly opinicins by
lble¡sic dcDLisfs ¿ùc n)orc likcly to be wroìlg thâll l]glÌt.
'l'his cl¡in] misrcplcsenls bo(h prol'icicucy lesting and
crt('
tls
ol thc stu.ìy do l)ol plovide a ltLìe lncasura oI eitheÍ.
Ncither Lhe accumoy of the rùethod nor the reiiability ol its
applicâtion is illLrmi¡a¡ed Jìù1ì thìs study, yct the rcsults lÌâve
l)ecu cxlr¡polate(i inlo iìn "error r¡te" lbt an elÌtile ¡Ìlethod. 'l'hc
legal aDd scicntìlic cour!ìrìDìlies âre left not knowiug auy nrote
¿ìboul lhe method's !rìljdjty or rcli¿ìbilily desPite ot pelhaps
beciìuse oJlintcl pretalions lil(c tllìs,
Aìrolher exalnplc d'i¡lcolrectly cxtr¿rllolating an erlor Iatc js
the usc ol rcsults torn rÌ 2001 paper by Ilouck and Buclowle
lhe lcsults
(16). Although a novcl study ()1 lhc colcord¡nce betwecD Phctlo-
type ¿Ìnd geìrdype in lruurtn hairs by lwo lolellsic lnethods
(nricrosco¡ry ancl mtDNA), the resulls are olten used in statel]]enls of "crror ì¡tes" lor Ilrìcìoscollical lrail cxa¡rrjnalions The
paper rcviewed 170 c¿rscs ìn which nricloscoPìcal 4ncl mtDNA
cxirffìníìtior)s weto coDdtìcled o¡ human hait sanlples in fcltcùsic
c¿rscwork. Ol' 170 c¿ìscs, 133 wcrg stìllicienl lirl analysis; ol'
thcsc. nine c¿rscs wcre lound wheÍe thc htirs h¡cl a sit¡ilat nlictoscopic appcatance (phcnotyPe), but cìiffcrcnt nrtDNA scquellces
(gcr)otype). lntclprel¿f ions ol' these results by colÌlrnentakns alld
pr¿rclitioncls clelnonsllate thc ¡'llâlheln¡tical gylllllastjcs solÌle go
throlgh lo îorçc the Ilunlbers into being a llte of elror:
One way to rcport such data is to síly that of lhe 26 cascs
iìì which Lhe rÌtDNA lbulld an exclu$ion, the ex¿llllillcrs
t¡\itìt llrL vi\uilì lr¡¡rlol'tlr.rrllcJ ¿lt lr:'.'r.i¡llion nillc lilllcs
¡ Type I falsclositìvc clror r¿ltc ol'
(912.ll. A¡othcr wiìy to ìook at the data is to rcPorl
Thcsc (lâta ill.iicrtc
,15(/a
thirt nine tiìÌes oùt of the 78 times lhat visu¡l exllnillers
declarod iìn âssociation (I2a/a), lhe mIDNA locht)ique
shou,ccì ¡in exchrsiclu. (17)
Houck ¡ncl Budowlc IoLrncl a lalse positivo rate oI lifl.r or
35%, dcpcndirrg on how or)c cîlculfìtes thc false Positive
lalc. (18)
...1oI cxanrple, an FBI sttìcly llr¿Ìl four)d that 1/8 of h¡ir
sarrples saicl lo "be assocjaled" ir¿lscd on nrjcloscopic colùll¿rrison \!ere sLlbsequcntly found to come fi-onr difielclll
Þcople b¡scd on DNA ¡nal),sis. (19)
Thcsc
"cr(n
lrc calculaljons
lrased on a tnisunderstiurding
str¡dy ând putting tlìe lrLnnbers 10 â
which lhcy \\,cre not .lesigncd.
r¿rlcs"
ol'lhe niìtrìlc ol thc orìgilr.ìì
Lìsc
lbl
126
JOURNAL OF FORFNSIC SCIENCÈS
Conclusiolr"^
It is irìll)er'¿Ìtìve that resc¿ìrchor-s ¿ìn(l prÂctilioìlels hiÌve â tholough undclstrrrding ol-the v¿tlioùs conccpts ol cIJ-oI in or'der 1()
clesign ¡rlc¡re| reseârch lo ¡rrocl!ce vaììcl fbrcnsic sciencc ¡'lolhods, as wcll âs to bc ilblc to l)roperly rcport ând expl¿rìn rcsults.
Wc strurgly recoullnel]cl th¿t cc[ìc¿ìtiona] p|oglans in litrensic
sciences rs rvell ¿rs tr¡ining proglams lol llr¿ìclìlioners tdcltcss
elor a¡ld clror'analysis. We nust ¡lso consicle¡-the leg¿Lì coDlcxt
as judgcs ancl l¿ìwyers typically do not undeìsl¿rncl how elt<¡r'
l¿ìtes alc clcl-ived or Lhe cot'lll)lcxily iu sepalTtillg "uristakes" lìol],]
uncertairìly.
Studies havc exhibitecl v¡r'iccl succcss in prol)erìy cvaluafìng
thc leljltlilily ol ccrt¿ìin tì¡its nscd iD analyscs. lìese¡¡chc|s ¡rc
applying rno|e soPhisticatc(ì meiìsutcì'ltc¡)t tccht)iqres ¿ììld st¿ltistical anaìyscs to cvalLìale 1o¡ensic cviclence irnal ¿ìtc ¡ìlso increâsiugly lincling rvays oI <¡Lrautilying tr¿rits thal h¿ìve hislorically
ienraired Iaìr'1y subicclive iìncl tìlought to be unc¡uunlil'ìable (see
(20) lbr an exârnple lìon1 aD{hropology. (21) for an exanr¡rlc
fìooì l'ìr)gelprìnts, ancl (22) ftrl an cxan':ple in fblcnsic ìrlir comPâriso]r). For the most patL, coDtcr¡porary rcsc¿fch preseuts ctror
vallìcs, but tlÌe lcl1tì js oftetÌ not dclinc(l. ¿lnd the polcDti¿il cl'fect
o¡ eviclen{iary cxiÙnin¡tiolt is Dot addresse(L (23). Ìn recctrt
ycârs, we hâve also sccD clisci¡rlines worl(ing tow¡rcl thc developll]ent oJ standards ¡nd bosL llrâcttces aD(l adv¿ÌnceDleltt ol'
lbrcnsic sta¡cla|cls âr)d tcohniclLles throLìglì the lbflllttiou ol'
Sc¡entilic ¿ncl'l.ecluricaì Working Ctoups (24). ln âcìclitìoìr to
rccoD]lÌe ding bcst pr¿rctices lor tcchDìqucs aûcl clullity assur'a ce n]c¡sL¡lcs to redrìce l)lâc{jtjonel âlld trsttt¡nÌcnl et.ror., guìiielincs also ol'tcl incluclc Ieco¡¡¡tc¡clifiols lor propel.research
.lesign aDd the selectiorì ol ¿rpprol)riate st¡lislic¿ìl nlethods.
l'oo olLen, thc tcr'Dl "errol" is a source of confrrsion lDcl eve¡
l]]isused iD the courLlootÌt atÌd iD lolensic science. This hîs
occuilecl desltite tìte incteasecj ¡tlo1ìle of ancl l.elianco on the concepl ol clr.or lbllowiDg thc DaÍðer./ guiclclines ancl Lhc NAS
Ììeporl.
lbrc¡tsic scieDLists, we mLìsl bc co[cctrod with the
^s
0larìty, rcliabilily,
aDd validìty of oLrr nrethods. Dùe lo our
involvelÌìent with the leg¡l syslclù, we slìollld ilso be l]loilclive
iD educating the lcgal coulnuuity about thc (lìllelcnccs bct\\,een
scientilìc clrol, lnethod ìin'litations, uncclt¿ìiutics. and ¡l'listakcs
be pleparecl to ¡lliligate issues rcl¿ìted t¡¡ eÛ.or'. This can best
be accclrr¡rlishetl by cnsLrling that wc underslrìncl, âckno\!lcdgc.
üDcl
âDd colnrnlrnjcotc Dretho.l lilììi{ttious ancl ¡toLcntìal solìtces ol'
crrol-in our reseâtch iÌnd Iorerrsic analyscs.
.. r'' ln iI((,'| I.lcrìrl)rI rl¡ N¡c,l\ ',r ll, l.'rrr,.i. S.i..h(.¡,.r ,r ,.
rìrr). Ni,r'iìi, l(.\:,rLl, ( .'Urh il.',){rr. \ i j.trIcIiIJ l,'rrr : r .. i. r... r,.
I Iirr,t 5ri,t.* i' f,,tl, tl,rú ,r,1. \\'.flì,rìtt. f. l¡ , l,'ll, n.1 ¡.r"f,.,r,,]l
l1ù\. ì¡) /
l. llir{l SJ. Scjcrìtific cert intyt reseâìah ve11\us tìrcrìsic l)crspccli\,0s.
rhù
J lrorcnsic Sci 2001i46(4):978 81.
J L.1 rtl
11,1,:.
5. ltt.rt \'. Unil(!
6.
t
1
,,:1
ht 'l,'75:2rrr,.
Stnt?.\ 5¿{ App. D.C.4(r.293 lr. ì0ll11923)
Iìâììd DJ. MeasureDrcnt lìrcory rìr(l pr¡crice: rhe woì (l lhroLrgh qUrìntilì.
crìti()ì. L,on(Ìon. tJ.K.: Arxnd, 200¿t.
7. P.ìge M, Ì'ryloì J, BlcnkìD M. Iìr'eosic i(lenljlic¡rion \ciencc c!idoìc;
..ir.l. /t,,¡,/ 1/ lJ,rr | :, ,I,ri,Lri|.,ri\" :,rrl).r. ,,t jlrL ,r, ll,i.,r,,;
lìÌensic idenrilicrrioìì scicncc cvidcncc. .l lorcnsic :jcì 20 :56
(5):1180 4.
8. (lhrislcnssì AM. C r$(ìcr (lM, llo ck MM, Ouslcy SI) Ilrr\)f, cn.ol
râtes iìD(l thcir nreanings ìrì lòrcnsìc scicùcc. Pr1)ceedings ol lììe 6l
Annrì¡l Mccljng ol the
Âcâ(ìcnry of lìorensic Scic|ccs, 20tl
Itb 2l 26, Chjcngo. Il,.^ììcric¡o
&norìrb Sprirrgs, COr Anrcricrìn
ot.
lìorcDsìc Scicrccs. 20 j j
^ciìdcn\
9. Dìoì lll, Chrrlt( D. Why expcrls rÌ¡ke crlors..l Irorensic Idcnr
(4)í)0
l0 Srk¡
t6.
MJ, Koehìcr
lJ. 'lhc conring pâmdi-sln shiil in lÌnc¡s c
lion scicnccs. ScicDce 2005:109:lì92
We woulcl like 1() thank scvcr'¡l of olÌr coìlciÌgLìes
levìcwecl ealliel vclsions of this Paltcr'.
\\rho
tt. U¡titd Sklt¿t¡ t A4itci,r,Il I¿trs tr.3d 5?2 3d Cir. (1998).
I'q,l( | Aot't..,99C1:ì 0391 (2002J.
12.
l-'ì. Prge M,-lûyloì J, Blcûkin M. Uniqrencss iìr lhe lòrcnsic i(lcntìJic lion
scienccs l'r¡ct or fìcLionl Forcnsic Scl ì|t 20ll:2(X):12 8.
14. ADcriciìrì lloÛ11 ol ii)rcrrsic Odontdogy (ABfO). l)osìti(nì prìpcr orì
iritcnÌìrk \\,(lkshoD 4. ASIrO Ncws 2001:22(2)r5.
|5. Bk 1t)t t,..t¡r¿(.tlI9 So.2d I169 (2002).
l6 Iknìck Mi\4, Bu(lo\\4c Il. Co rlîlion ol llricroscopic ilìrd nrirochondrirìt
chlrrclcrìslics in hLrnlrn lìîiß. J Forellsic Sci 2002ì47(5):964 7.
i7.
S.ìks M, Kochlcr J. IìcsDonsc Sci 2006;3 ì l(576l):í)6.
Cole S. Morc thrn zcro: ilccoul]Ling lbr crl) iÌì lâtenr fìngcì.j,.rìllr idcntilicûtion. J Crinì l-il\\,2005,95(3):985 1078.
19. l-tlDder lls. lcstiDr(J¡y bcf¡re llìe UDircd Stîtcs Senale CoDrnìilcc on
ConrÌrerce, Sc {rrìcc, rìù(l Space. ]'hc Scicucc rìnd St¡nd.ìrrls ol ì.ì)tcnsics.
Mrìrch 2il. 2{112.
20 Chrislensen
i'csljùg lhe rcliabiliLy ol iorìtîl sinus oLìrìincs in pcrs( al idcntìlìcâtion.
^1\4 J Iìorensic Sci 2005;50(l)rl8 22.
2l Neùm.ìnn Cl. ChNDrtnd C, Puch-Soìis l{, Ègli N, Anthoûioz A. llrorììigc
Grillilhs,{. (l)nrpLrtNtion of likelihood r'âtios in linger.Jr.inl idcnliticrìtioI
lo| configùr.rti(rìs ol lny nLìrÌbcr ()1 nìi¡rUtj¿ìc J lrorensic Sci 2007i52
llì.
(l):-51 ó4.
22. Brooks D. (brìl)er B, McNârìght l, Iìobcrlsorì J. Dìgírâì i|ì¡gjng ù(l
inr¿rge llllrìlysis rptlì¿d to nunrericrìl âl)plicatiorìs iD li)rcnsìc hrir cxlnri¡.ìlion. Sci Justicc 201 li5 i( l):28 37.
23 Crou,tìer C. ìDgvoklsllrd M. ObscNer c|ioì. tìcnds in lìnclsìc rìnlhìolxtogy I)roccedlngs ol thc 6lst ADDual Mccting oJ the
Ac¡dcn]y
(blolr(to 5^prirìgs,
ol fircrìsrc Sc cr)ccsi 20(19 l.tb 16.21; Dcnver, CO.^nrcricrìll
CO: ADrcric¿ìll
ol lìfensic Scictrces,2009.
24. Nâtionîl |rstitLrlc
^câ(lcrìry
of Justice. Scicnrific Workìng Crl)uDst h[l)/ \'\rw¡ii
gov/L(ìr ics/lìn ensics/hl)-opcl'lrti(nìs/st.ìn(llll{Vscien
Lìlïc- uo|k iI g g]oùfs.hlìì
infìn nÌrrion ¡nd rcpri t
^(ldilìoìì¡l
M ClhììsLensen. Ph D.
(ìeoìgc
^rgi M¡son Univcrsi{y
1400 UniveNìry Dri!e
FdiìJlx, VA 22030. llSA
Il nrrìij: rchris
Rcfcr.cnccs
L
Daul)en v. Mc,r'cll l)o$' Ph.ìr'Ìnacer)ticals. Inc., 509 tj.S. 579 (t993).
idenlìJic¿ì-
5.
(¡cccsscd Scptcnrlrcr 14, 2012).
Acl¡¡u¡¡tIetlgntcnIs
2004)i56
[email protected]
reqLrcsrs: