Arsenal/Zone Rating: A PitchF/X based pitcher projection system

Arsenal/ZoneRating:
APitchF/Xbasedpitcherprojectionsystem
PeiZheShu
1.
Introduction
Pitcherperformanceprojectionisafundamentalareainbaseballanalysis.Traditionalprojection
systems,likePECOTA,ZiPSandSteamer,arebasedoneitherERA(orRA/9),whichcannotseparate
pitcherperformancefromfielderdefensewell,orK%,BB%,HR%andbattedballdata,whichare
hardtoderivecrucialinformationlikeBABIP.Thesesystemsleavealottobedesired.
Fortunately,PitchF/Xwasintroducedtobaseball,andbyutilizingitwecananalyzepitcherswith
muchbetteraccuracy.Thispaperintroducesanewandimprovedmethodforpitcherprojection,
whichwecall“Arsenal/Zonerating”,usingPitchF/Xdata.Theideaisthatpitcherperformancecan
bemostlyjudgedandpredictedfromtwoaspects:arsenalrating,whichcorrespondstothespeed
andmovementofthepitch,andzonerating,whichisrelatedtothelocationthepitchwithregardto
thestrikezone.
Ourarsenalratingandzoneratingmodelaretrainedandpredictedonper-pitchleveldata,which
hasaprettydecentsamplesize(over2millionavailablepitchesin7seasonsofdata)andrelatively
richcontent(speed,movement,location,pitchcount,etc.),whichallowsustoconstructadetailed
model.
Ourcombinedprojectionstat,whichwecallarsenal-zone-combinedrating,notonlyhasa
comparableresultwithmainstreamprojectionsystemswhenjudgedbyR^2andRMSEtoactual
performancedata,butalsoexcellentforpredictingbreakoutandbreakdownpitcherscomparedto
thosesystems.Andthere’sanevenbetterresult:afterlinearlycombineourratingtobasicpitching
statslikeK%,BB%,HR%,oursystemismuchbetterthanallmainstreamprojectionsystems.We’ll
describethemethodofoursystembelow.
2.
Method
2.1. Mainmodel
Weconstructamultilevelregressionmodeltoincorporateandestimatearsenalratingandzone
rating.Ourmainmodel,whichisonper-pitchlevel,andonrunscale,whichcanbedescribedas:
ri ~ a( spi , xmovei , zmovei , typei , hand i ) + z ( xloci , zloci , typei , hand i ) + rb + deft + pf pa Here ri istheactualresult(runexpectancy)ofthispitch, a( spi , xmovei , zmovei , typei , hand i ) is
thearsenalratingofthepitch,witheachcovariaterepresentingspeed,x-movement,z-movement,
pitchtype,andhandness(samehandedoropphandedbatter/pitcher)ofthepitchrespectively,
z ( xloci , zloci , typei , handi ) iszoneratingofthepitch,witheachcovariaterepresentingx-location,
2016ResearchPapersCompetition
Presentedby:
1
z-location,pitchtypeandhandnessofthepitch, rb istherunexpectancyofthebatter, deft isthe
pitcher’steamdefense, pf pa istheparkfactor.We’llexplaineachparameterofthismodelbelow.
2.2. RunExpectancyonpitchlevel
Wefollowthewidelyusedfangraphs.comwOBAformulaandthecorrespondingleagueaverage
wOBAandwOBAscale[1]forrunexpectancyonplateappearance(PA)level.Weusethefollowing
rulestodeterminepitchlevelrunexpectancy ri :
(1) Fornon-PAendingpitches,therunexpectancyistheoneofnewpitchcountminustheoneof
oldpitchcount.
(2) For PA ending pitches, the run expectancy is the one of PA result minus the one of old pitch
count.
Forpitchcountrunexpectancy,wecalculatetheexpectationofthePAresultrunexpectancy
conditionedoneachpitchcount.ThismethodscattersthevalueofthePAresultrelativelyequally
oneachpitchofPA.
2.3. ZoneRating
Asboththezoneratingfunctionandpitchlocationdistributionarerelativelysmooth,weusea
simplifiedmodeltodescribezonerating:weseparatethestrikezoneintosquaregrids,anduse
eachgridasaseparatecovariateinourmodeltoobtainitszonerating.Alocalregressionis
performedontherawresulttoobtainasmoothedone.Thegraphbelowshowsanexampleofthe
finalzonerating.
Todescribelocationofapitchwithregardtothestrikezone,weneedto“standardize”thestrike
zone,namelytoadjustthestrikezoneonbatter’sheightandhandness.We’lldescribethis
adjustmentlaterinsection2.7.
2016ResearchPapersCompetition
Presentedby:
2
Nowthatwe’veobtainedthezoneratingofeachstrikezonelocation,weneedtocalculatethezone
ratingofeachpitcher.Asaresult,weneedtoknowthedistributionofeachpitcher’spitchlocation.
Weassumethedistributionofeachpitcher’spitchlocationisacombinationof“circleswith
identicalsize”,i.e.mixtureofGaussianwitheachcomponent’scovariancematrixbeingidentical
scalarmatrix.Belowisagraphshowingtherawdataandthesmoothedpitchlocationdata.
2.4. ArsenalRating
Weuseageneralizedadditivemodel(GAM)[2]todescribethearsenalrating.Weassumethata
pitch’sarsenalratingissumofthreecomponents:speed-relatedrating,x-movement-relatedrating
andz-movement-relatedrating.Eachratingwouldthenbedescribedbyacombinationof
multiple(usuallyseveraltofifty)thinplatesplines[3].
Usingourscoutingknowledge,weknowthatdifferenttypeofpitcheswillhavedifferentproperties.
Forexample,fastballwithfasterspeedishighlylikelytobebetter,whilecurveball’sspeedhasless
(ifany)positiveimpact.Soweuseseparatearsenalratingmodelfordifferentpitchtypes,thuspitch
classificationisessentialtoourmodel.
Also,evenforsamepitchtype,identicalpitchesshouldhavedifferent“power”ondifferent
batter/pitcherhandness.Forexample,“lateralmoving”slidersareknowntobebetteragainstsame
handedbatter.Sowealsouseseparatemodelsforsamehanded/opphandedbatter.Wealsouse
separatezoneratingmodelfordifferentpitchtypesandsamehanded/opphandedbatter,although
thishaslesssignificantimpactonourresult.
Weusea“soft”pitchtypeclassification,whichmeanseachpitchhasaprobabilityrangingfrom
[0,1]ofbeingsomepitchtype.Tocalculatethearsenalratingofapitcher,weaveragethearsenal
ratingofeverypitchthrownbyhim,whichisthispitch’sratingwhentreatedasaspecificpitch
type,weightedbyitsprobabilityofbeingthispitchtype.We’llshowhowtoclassifypitchesin
section2.5.
2016ResearchPapersCompetition
Presentedby:
3
Seethefollowinggraphforanexampleofarsenalrating,whichshowsslidersforbothsamehanded
andopphandedbatters.
2.5. PitchTypeClassification
WefirstintroducethefollowingGaussianMixturemodel(GMM)foraspecificpitcher’spitches:
!
p( spi , xmovei , zmovei ) = ∑π t Ν( spi , xmovei , zmovei | µt , Σt ) t
Here ( spi , xmovei , zmovei ) isa3dvectorcontainsspeed,x-movementandz-movementofpitchb
!
thrownbypitchera, π t istheweightofpitchclustert, µt and Σ t isa3dcentervectorand3*3
covariancematrix,respectively,forpitchclustertthrownbythisspecificpitcher.1
Thewholemodelcanbedescribedas:apitcher’sspeedandmovementdistributionisthe
combinationofseveralanyshaped3d-ellipsoid.WefoundthisGMMmodelagooddescriptionofa
specificpitcher’spitches,withoneorseveralpitchclustersbelongingtooneactualpitchtype.
1Hereweassumethatthecovariancematrix Σ t canbeanypositivedefinitematrix,whichmeans
thecontourofGaussiancanbean3d-ellipsoidwithanyshape.
2016ResearchPapersCompetition
Presentedby:
4
Toclassifypitches,weslightlyalterourassumptionsintoastrongerone,whichisthefollowing
model:
!
p( spi , xmovei , zmovei ) = ∑ π t Ν ( spi , xmovei , zmovei | µt + off p , Σ t ) t
Whichmeanseverypitcher’spitchfollowsthesameGMMmodel,nowwitheachtrepresentinga
pitchtype.Theotherdifferenceisthe“offset”vector off p ,whichisthesameforanypitchthrowing
byaspecificpitcher,regardlessofpitchtype.
Welabeledatrainingsetfortheclassificationtask.Fromthetrainingsetandthemodelabovewe
getaNaïveBayesclassificationresultusingthismodel,andweaddintherawspeedandmovement
data,toformamultinomiallogisticregressionmodel,toobtainthefinalresult.Seethegraphbelow
foranexampleofourclassificationresult.
Wefollowthepitchtypesystemofbrooksbaseball.net.Asaresult,ourlabelsetclassifypitchesinto
6types:FF(fastball),SI(sinker),CH(changeup,includingsplitter),FC(cutter),SL(slider),CU(curve,
includingfasteronesandslowerones).
Wefoundthemodelabovehasgoodaccuracyforclassifyingpitches:Weobtained88.3%accuracy
onourtestset.Whilethismightnotseemtobetoohigh,wefoundnearlyalloftheclassification
errors“borderlinepitches”,namelyFF/SI,FC/SLorSL/CU.Thosepitchtypesarenaturallyhardto
classify,andourlabelsetsometimesessentiallyprovidesanarbitrarylabel.
Thismultinomiallogisticregressionmodelalsonaturallygivesprobabilisticclassificationresult,
whichweusetocalculatearsenalrating,aswediscussedinsection2.4.Thisprobabilisticresult
providesanicealternativesolutiontotheborderlinepitches.
2.6. PitchMovementCalculationandPitchCalibration
WefollowA.Nathan’smethod[4]tocalculatepitchmovement.Weremovedrageffectfromthe
data,andremovegravityeffectfromdataaswell.Asaresult,weonlyconsidertheMagnuseffectof
thepitch.
Also,PitchF/Xrawdatahasaknownissue:itispoorlycalibrated[5].Preciselyspeaking,insome
ballparksand/orsomegames,itwillsystematicallyreportaslightlyhigherorlowernumberof
2016ResearchPapersCompetition
Presentedby:
5
someattribute(includingvelocityandmovement)ofeverypitchthrown.Thisiswhatwewantto
correct,aswecanseebelow,therawdatacanbeprettyweird.
Weusethemodelinsection2.5forclusteringpitchestosolvethisproblem,alsowithan“offset”
vector.Thistimeweuseanoffsetforeachgame.Seethefollowinggraphforanexample.
2.7. StrikeZone
Weuseasimpledefinitionofthestrikezone.Aclosedcurvewhichsatisfiesthefollowingcondition:
apitchlocatesonthecurvehas50%probabilityofbeingcalledastrike.Followingthisdefinition,
weusealogisticregressiononrawcalled-strike/balldatatodeterminetheactualstrikezone.
Aswe’reactuallymoreconcernedabouttheimpactofheightandhandnesstothestrikezonerather
thanitsshape,wecanjustapproximatethestrikezonewitharectanglewithsamecenterofmass
andsamearea.Addintheassumptionthatthecenterofmassofthez-sideoftherectanglehasa
linearrelationshipwithbatterheight,andthex-sideparametersoftherectangleaswellaslengthof
thez-sideareirrelevantwithbatterheight,wearriveatthefollowingstrikezoneformula:
Right-handedbatter:
− 0.991 <= x <= 0.946 and 0.906 + 0.1281* height <= z <= 2.591 + 0.1281* height Left-handedbatter:
− 1.139 <= x <= 0.755 and 0.617 + 0.1729 * height <= z <= 2.306 + 0.1729 * height Aswecansee,thisisprettysimilarwithMikeFast’sstrikezoneformula[6].Weusethisresultto
adjustapitch’sstrikezonelocationaccordingtobatterheightandhandness.
2.8. Otherparameters
Weuseasimplelinearregressionfortheprojectionofbothteamdefenseandparkfactor:the
projectedvalueisthelinearcombinationofpreviousseveralyearsofobservedvalue.Weusethe
averageofUZRandDRSscoreforteamdefense,andobservedwOBA-basedparkfactorasraw
value.Forparkfactor,5yearsofpreviousdataareused,whileforteamdefense,3yearsareused,as
teamdefensehasamuchhigheryear-to-yearvariance,especiallywhenusingrawmodelslikeUZR
andDRS.
2016ResearchPapersCompetition
Presentedby:
6
WeassumeaGaussianpriordistributionforrunexpectancyofbatters2.Thisistoeliminatethe
effectofeachpitcher’scontext,ordifferent(combined)abilityofthebattershepitchedto.
2.9. Adjustmenttoourmainmodel
Theoutputofourmodeltillnowisapitcher’szoneratingandarsenalratingbasedonhis
actual(past)dataofaspecificperiod.Combiningthemtogether,andaddintheprojectedteam
defenseandparkfactor,weobtainarawcombinedrating.Whilethisisalreadyadecentestimateof
pitcher’sfutureperformance,wefeeltheneedtodosomerelativelyminorbutstillimportant
adjustment,whichcanbedescribedusingthefollowingcombinedformula:
arp = ( a + z + deft + pf pa ) * pitchp + l _ avg + hand p + pc _ valuep ∑ ∑
Here arp isthecombinedresultofourmodel,whichwewillcall“arsenal-zone-combinedrating”
fromnowon.
∑ a isthesumofthearsenalrating,discussedinsection2.4, ∑ z isthesumofthe
zonerating,discussedinsection2.3. deft istheteamdefense, pf pa istheparkfactor,bothofwhich
arediscussedinsection2.8. pitch p istheestimatevalueofpitchesthrownbythispitcherper9
innings. l _ avg istheleagueaverageRA/9. hand p isanadjustmentbasedonpitcherhandness.
pc _ value p isanadjustmentbasedonpitcher’spastpitch-countdistribution.We’llexplaineachof
thoseparametersbelow.
pitch p :Thisisusedtoscaletheratingfromrun-per-pitchscaletoRA/9scale.Topredictthe
estimatevalueofpitchesthrownbythispitcherper9innings,weusetheobservationthatthis
valueisinfactthemultiplicationoftwofactors:pitchesperPA,andPAper9innings(roughlysame
as9timesWHIP).Wejustusethepreviousseason’svalueforpitchesperPA,andusealinearmodel
forPAper9innings,witharsenalratingandzoneratingasparameters.
l _ avg :Thezoneratingandarsenalratingaretrainedonmulti-yeardata,andthebaselineis
“leagueaverage”.SowhenwescaletheseratingbacktoRA/9,weshoulduseadifferentleague
averageRA/9foreachyearandforAL/NL.Weusealinearmodel(ofpreviousseveralyears’RA/9)
topredictnextyear’sRA/9,topreventoverfitting.
hand p :Weusedifferentmodelsforsame-handed/opp-handedbatter,butduetolackofdataon
leftypitchers,wecombinedrightyandleftypitchersinthesamearsenalratingmodel,whichisnot
verysatisfactory.Ouradjustmentonthisistogiveallleftiesa“bonus”.Weuseout-of-sample
data(weuse2008-11datatopredict2012-14)toobtainthebonusvalue,topreventoverfitting.
pc _ value p :Wesumthezoneandarsenalratingofeachpitch(andthus,eachpitchtype)ofa
pitchertoobtainhisfinalrating.Butaccordingtoourscoutingknowledge,combinationofseveral
typesof“strong”pitchprobablywouldleadtoevenhigherlevelofperformancethansummingthe
pitches’arsenalandzoneratingsalone.Wehaveseveralwaystocompensateforthis,andafter
2Thismethodisoftencalled“mixedmodel”or“hierarchicalmodel”.
2016ResearchPapersCompetition
Presentedby:
7
testingitondata3,wefoundapitch-count-distributionbasedadjustmentisthebestone.Thatis,
pitchersruninto“good”pitchcountmoreoftenwillleadtoevenbettercombinationresult.
2.10.
Adjustmentforpredictionpurpose
Forpredictionpurposes,pastdataisnotexactlyequaltofuturedata.Althoughaspecificpitcher’s
arsenalandzoneratingarelargelystablethroughyears(we’lldiscussthisinsection3.3),apitcher’s
arsenalratingisprobablydecreasingeveryyear(atleastafteraroundage24-25),duetoadvancing
ageandaccumulatedpitch“miles”.Weuseadirectage-basedadjustment,basedonkernel
regressionresultof2008-2011data.
Regressiontothemeanisalsoawidelyadoptedmethodinprojectionsystems.Wealsoutilizethis
idea,andregresseachpitchertoleagueaverageRA/9accordingtohisinningspitchedinlast
year(theyearwecalculatehisarsenalandzonerating).Theregressioncoefficientispickedby
cross-validationonbetween-seasonarsenal-zone-combinedrating4.
3.
ResultandDiscussion
3.1. Predictivepowertest
Usingourmethoddescribedabove,we’reabletoproducetheresultofthearsenalratingandzone
ratingmodels,andeachpitcher’sarsenal-zone-combinedrating.ThisresultisonRA/9scale,and
canbeusedtopredictpitcher’sfutureperformance.
WeusePitchF/XfromMLB.comandplay-by-playdatafromretrosheet.orgfromseason2008to
season2014totrainourarsenalratingandzoneratingmodels.Toclassifypitches,weadditionally
requirethepitcherinvolvedtohaveatleast1500pitches(roughly95innings)pitchedduringa
singleseason,whichrestrictouranalysistostartersonly.Thefull7-seasondatasetcontainsaround
4.9millionpitches,andwiththeadditional1500-pitchrequirement,thedatasetcontainsaround
2.4millionpitches.
TotestthepredictivepowerofourcombinedRA/9scaleresult,weuse2012-2014data,withthe
additionalrequirementofpitcherpitchingatleast50inningintherespectiveseason,andpitchfor
onlyoneteamduringboththecurrentseasonandpreviousseason5.Wehaveatotalof181
qualifiedseasonsofpitchers,allofthemstarters.Wepredictpitcher’sseasonperformanceusing
thepreviousseason’sarsenal-zone-combinedratingonly,withtheonlyadjustmentbeingtheage
adjustmentandregressiontothemeandescribedinsection2.10.
Weusefourprojectionsystemsasourcomparisonbaseline:Marcel,PECOTA,ZiPS,andcontextbased-FIP(cFIP)[7].TheMarcelprojectionsystemisapurereplacement-level“baseline”system,
3Werunapartialleastsquares(PLS)regressionof2008-2011datawithalotofpotentialfeatures
andchoosethemaincomponent(whichalmostonlycontainspitch-count-distribution).
4Weusecross-validationtomaximizetheloglikelihoodofthedataafterregression-to-the-mean.
Theresultisthatthepriorcountsasaround20innings.
5Thisnot-changing-team-for-two-seasonsrequirementismainlybecauseweuseparkfactorand
teamdefensetocombineourfinalrating,andweknowtheexactteameachpitcherpitchedforin
thesecondseason,butwe’renotsurehoweachprojectionsystemhandlesthisissue,sotoavoid
unfairadvantageforourrating,wejustremovethosepitcherswhochangedteam.
2016ResearchPapersCompetition
Presentedby:
8
butisdesignedtohaveagoodweightedRootMeanStandardError(RMSE)6.ThePECOTA
projectionsystemandZiPSsystemaretwomainstreamsystems,bothofwhichutilize“comparable
pitchers”topredictpitcher’sfutureperformance.ZiPSconsistentlyrankhighlyinpitcherprojection
comparison[8].ThosethreesystemsaboveareonERAscale,sowelinearlyscaleittoRA/9scale.
cFIPisasystemthatadjustK%,BB%andHR%accordingtoopponent,park,catcherandumpire,
andusethoseresulttocombineanFIP-likerating.Thisratingisoriginallyona100-pointscale,but
itcanalsobelinearlyscaledtoRA/9scale.Also,thisratingisnotacompleteprojectionsystem,asit
iscalculatedonlyononeseasonofdata.Wealsousepreviousseason’scFIPonlytoprojectnext
season,withanageadjustmentandregressiontothemean,bothwithsimilarmethoddescribedin
section2.10.
Resultofprojectionsystem
system
Marcel PECOTA
ZiPS
cFIP
arsenal-zone-combined
R^2withnext
0.1866 0.1975 0.2043 0.2183
0.2017
season'sRA/9
RMSEwithnext
0.860
0.881
0.874
0.867
0.864
season'sRA/9
Thetableaboveshowsthecorrelation(R^2,highermeansbetter)andRMSE(lowermeansbetter)7
withrealRA/9dataforoursystemandfourbaselinesystems.Wecanseethatoursystemis3rdin
R^2and2ndinRMSE,andnottoomuchbehindtheleaders.Thesecomparisonareonthose181
qualified2012-2014startingpitchers,andthesamesetofdataareusedforallR^2andRMSE
comparisonsbelow.
Breakout-Breakdownpredictionresultusingarsenal-zone-combinedrating
Breakout
Breakdown
Baseline
Bottom
Bottom
Percentage Top1/3 Middle
Middle Top1/3
Sytem
1/3
1/3
PECOTA 20%
58%
25%
17%
44%
50%
6%
33%
58%
27%
15%
45%
43%
12%
ZiPS
20%
56%
17%
28%
44%
42%
14%
33%
57%
17%
27%
40%
43%
17%
cFIP
20%
50%
28%
22%
56%
28%
17%
33%
47%
25%
28%
50%
33%
17%
Thetableaboveshowsthe“breakout-breakdown”predictionofourrating.Thispredictionworks
asfollow(usingZiPSbaselineasexample):sortall2012-2014qualifiedstartingpitchersaccording
to“arsenal-zone-combinedratingminusZiPSprojection”.Checkthepitcherswiththe
highest(incidatingbreakdown,orlowest,indicatingbreakout)20%(or33%)result,toseewhere
their“realRA/9minusZiPSprojection”fallswithinallpitchers:topthird,middle,orbottomthird.
Theresultshowsoursystemcanstatisticallysignificantlypickoutthebreakoutandbreakdown(or
underestimatedandoverestimatedbyeachbaselineprojectionsystem,respectively)pitcherswith
6Marcelsystem’sgoodRMSEpropertycanalsobefoundin[8].
7Weallowprojectionsystemstohavean“offset”whencomputingtheRMSE.
2016ResearchPapersCompetition
Presentedby:
9
regardtoeverybaselineprojectionsystem,asineverybreakoutprediction,the“top1/3”
percentage(red)iswaybiggerthan“bottom1/3”(green),andviceversainbreakdownprediction.
Thisbreakout-breakdownpredictionshowsoursystemshouldhavesomedistinctadvantageover
thebaselineprojectionsystems.Weusethefollowingtesttoconfirmthisobservation.
3.2. CombinationwithK%,BB%andHR%
Asourarsenal-zone-ratingsystemhaven’tutilizethemoststablecomponentstatsofpitchers(K%,
BB%,HR%adjustedbyparkorestimatedHR%basedonFB%),andeveryprojectionsystemhave
alreadyusedthose,wenaturallyarriveatthefollowingguess:ifwecombinethearsenal-zoneratingwiththesestablepitchingcomponents,weshouldobtainabettercombinedmodelthan
combinethebaselinesystemswiththesecomponents.
ResultofcombinedsystemwithxFIP
arsenal-zoneIndex(weight) Marcel+xFIP PECOTA+xFIP ZiPS+xFIP cFIP+xFIP
combined+xFIP
R^2(50/50)
0.2263
0.2315
0.2293
0.2124
0.2528
RMSE(50/50)
0.834
0.834
0.836
0.846
0.819
R^2(optimal)
0.2268
0.2315
0.2298
0.2209
0.2529
RMSE(optimal)
0.833
0.834
0.836
0.845
0.819
Thetableaboveshowsthisguessistrue.Afterlinearlycombining8everymodelwithxFIP(canbe
seenasalinearmodelofK%,BB%andestimatedHR%basedonFB%)inpreviousseason,our
modelcomesoutontop,withaprettybigadvantage(R^2advantageissimilartoZiPSvsMarcel).
Ofcourse,cFIPhasanaturaldisadvantageinourcombinationtestabove,asit’sbasicallyan
adjustedFIPwithalmostnothingelse.ButcFIPcanalsobeenviewedasalinearmodelofstable
pitchingcomponent(infact,itscomponentsaremorestablethanrawK%,BB%,andHR%),sowe
cancombinecFIPwithourarsenal-zone-combined-rating,andwitheverybaselineprojection
system(otherthancFIPitself).Thisisshowninthetablebelow,andourmodelcomesoutontop
again,withasimilarbigadvantage.
ResultofcombinedsystemwithcFIP
arsenal-zoneIndex(weight)
Marcel+cFIP PECOTA+cFIP ZiPS+cFIP
combined+cFIP
R^2(50/50)
0.2499
0.2484
0.2486
0.2693
RMSE(50/50)
0.822
0.829
0.829
0.812
R^2(optimal)
0.2499
0.2487
0.2486
0.2693
RMSE(optimal)
0.822
0.8287
0.829
0.812
8Weusea50/50weightforeverymodelcombinationforthefollowingreason:50/50isextremely
closetoMAPestimationofthispriorweight(linearcombinationofmodelcanbeseenasaBayesian
averagingofmodels)ofeverypairofmodels.Wealsouseaoptimalpriorweight(basedoncrossvalidationonR^2),andfoundtheoptimalweightformostmodelcombinationpairsarecloseto
50/50inmostcases(everypairisin45/55-55/45range,exceptcFIP+xFIP)aswell.Weshowthe
resultforboth.
2016ResearchPapersCompetition
Presentedby:
10
3.3. Year-to-yearcorrelation
Wearealsointerestedintheyear-to-yearcorrelationofsamepitcher’sarsenal-zone-combined
rating,withorwithouttheregressiontothemeanadjustment.Wefoundevenwithoutthe
regressiontothemean,ithasamuchhigherR^2thanthatofcFIP(whichisinturnmuchhigher
thaneverybasicpitchingstatandlinearmodelsbasedonit,suchasxFIPandSIERA,asshownin
[7]),andhasasimilarR^2withthatofZiPS,onlylowerthanPECOTA,asshowninthetablebelow,
duringseasons2012-2014.Soit’saprettystableratingacrossyears.
Year-to-yearcorrelationofprojectionsystem
system
arsenal-zonecombined
arsenal-zone(w/oregression
combined
tothemean)
year-to-year
R^2
0.6628
0.6787
cFIP
(w/oregression
tothemean)
cFIP
PECOTA
ZiPS
0.4106
0.4475
0.8327
0.6703
3.4. Discussion
Allourresultaboveshowsagoodprojectionsystemforpitcherswithenoughpreviousseason’s
data(around95innings).Thislimitsourresulttoqualifiedstartingpitchers.However,asthe
arsenal-zone-combinedratingisstableenoughacrossyears,andisbasedonper-pitchleveldata
whichhasabiggersamplesizethanper-PAdata,nottomentionarsenalrating(themainpartofthe
model)itselfusesfeatures(speedandmovement)thathavewaylessvariancethanotherper-pitch
statslikeplatedisciplinestats,itshouldtheoreticallygeneralizewelltostartingpitcherswithless
previousdata,aswellasreliefpitchers,althoughsomeadjustmentmustbemade.Themain
problemprobablyliesonpitchclassification,asoursystemrequiresGMMandNaïve-Bayes
classificationresult,whichfitwellforlargedatasetbutcanbeproblematicwithsmallerones.
Luckilythere’realreadymanually-labeledpitchclassificationdata(forexample,theoneon
Brooksbaseball.net),soit’spossibletoovercomethisproblem.
4.
Conclusion
WepresentaPitchF/Xbasedmodel:Arsenal/Zonerating,todescribepitcher’struepitchingability,
andtopredictpitcher’sfutureperformance.
Thefinalmodel,whichwecallarsenal-zone-combinedrating,isagoodpitchingperformance
predictorbyitself.Ithascomparableresultwithmainstreamprojectionsystems,andithasdistinct
advantageinpickingoutthebreakoutandbreakdownpitchersthanthosemainstreamsystems.
Whenlinearlycombinedwithstablepitchingstats(K%,BB%andHR%adjustedbyparkor
estimatedHR%basedonFB%),thearsenal-zone-combinedratingismuchbetterthanthe
mainstreamprojectionsystems,andcanbeseenasoneofthebestpitchingprojectionsystem
currentlyavailable.
2016ResearchPapersCompetition
Presentedby:
11
References
[1]Fangraphs.com,“wOBA”,http://www.fangraphs.com/library/offense/woba/
[2]Hastie,T.J.andTibshiraniR.J.,“Generalizedadditivemodels”,1990
[3]Wood,S.N.,“Thin-plateregressionsplines”,JournaloftheRoyalStatisticalSociety(B),2003
[4]Nathan,A.,“DeterminingPitchMovementfromPITCHf/xData”,
http://baseball.physics.illinois.edu/Movement.pdf,2012
[5]Greenhouse,J.“TheYearinPITCHf/xCalibration”,
http://baseballanalysts.com/archives/2010/12/the_year_in_pit.php,2010
[6]Fast,M.,“AZoneofTheirOwn”,
http://www.baseballprospectus.com/article.php?articleid=14572,2011
[7]Judge,J.,“FIP,InContext”,http://www.hardballtimes.com/fip-in-context/,2015
[8]Larson,W.,“2014ProjectionReview”http://www.fangraphs.com/community/2014projection-review-updated/;“Evaluating2013Projections”,
http://www.fangraphs.com/community/evaluating-2013-projections/;“Evaluating2012
Projections”,http://www.fangraphs.com/community/evaluating-2012-projections/,2012-2014
2016ResearchPapersCompetition
Presentedby:
12