Lecture 24 - Wharton Statistics Department

Administrative Notes
Statistics 111 - Lecture 24
•  Homework 7 posted. Due in recitation on Friday,
April 22
Fitting Curves to Data: Application to
Fielding in Baseball
April 14, 2016
Stat 111 - Lecture 24 - Baseball!
Current Methods and Data
Bayesball Model
•  No recitation on Friday, April 15th
1
SAFE
April 14, 2016
Future
Current Methods and Data
Stat 111 - Lecture 24 - Baseball!
Bayesball Model
2
SAFE
Future
Quantifying Fielding Performance in Baseball
Overall goal: accurate evaluation of the fielding
performance of each major league baseball player
Measuring Fielding in Baseball: Present
and Future
Historical Method: Errors
Errors only punishes for bad plays, no corresponding
reward for good plays
No accounting for relative difficulty of each play
Shane T. Jensen
Historical Method: Fielding Percentage
Department of Statistics, The Wharton School,
University of Pennsylvania
Percentage of time a player properly handles the ball
Ambiguity in the denominator: players with poor range
could have high FP due to less opportunities
April 14, 2016
Need to take into account the relative difficulty of
individual balls-in-play (BIP)
Current Methods and Data
Bayesball Model
SAFE
Future
Available Data
Each season has ≈120000 balls-in-play (BIP)
I have worked with BIP data from 2002-08 (seven seasons)
Three BIP types: 42% grounders, 33% flys, 25% liners
BIP velocity information as ordinal category
Flyballs Caught by CF
Flyballs Not Caught by CF
400
400
300
300
Y Coordinate
Y Coordinate
Bayesball Model
SAFE
Current Methods: Ultimate Zone Rating
Ball-in-play data available from Baseball Info Solutions
200
100
200
100
0
0
−300
Current Methods and Data
−200
−100
0
X Coordinate
100
200
300
−300
−200
−100
0
X Coordinate
100
200
300
Ultimate Zone Rating: divides field up into zones and
tabulates success/failures of each fielder within zones
Future
Current Methods and Data
Bayesball Model
SAFE
Future
Current Methods: Ultimate Zone Rating cont’d
Current Methods and Data
Bayesball Model
SAFE
Future
Other Current Methods
Plus-Minus system (John Dewan): uses zones like UZR
Average success rate sk calculated within each zone k
Fielder gets credit of 1 − sk for each successful play, debit
of −sk for each unsuccessful play in zone
Aggregating over zones gives plus-minus value
Version with run values: defensive runs saved (DRS)
Difference between fielders success rate and average
success rate calculated for each zone
Differences weighted by run value and then aggregated
zone for overall rating
Probabilistic Model of Range (David Pinto): uses angles
to represent BIP direction (instead of zones)
Advantage: UZR zones are proxy for difficulty of BIP
Predicted outs for each direction calculated over all players
Actual outs for each direction calculated for individual
players and compared to predicted
Different PMR charts for grounders vs. liners vs. flys
Advantage: runs saved/cost is an easy to interpret scale
Disadvantage: zones are an ad hoc discretization of the
continuous fielding surface
Big Zone Metric (Peter Jensen):
Uses publicly available MLB Gameday data instead of BIS
Data is less resolute, so larger zones are used
Current Methods and Data
Bayesball Model
SAFE
Future
Continuous Fielding Curves
(x , y )ij location, velocity Vij and type of the BIP
Allows for principled sharing of information within and
between individual players
Bayesball Model
These probability functions will be smooth parametric
curves that can vary between different players
SAFE
Future
Representation for Different BIP Types
Flyballs and Liners
Y Coordinate
Y Coordinate
Grounder
Trajectory
CF Location at (0,324)
BIP
Location
100
150
SS
Location
100
Right
−200
−100
0
X Coordinate
100
200
"
log
−50
#
= βi0 + βi1 Dij + βi2 Dij Fij + βi3 Dij Vij
Logistic regression for grounders:
θ Angle
50
−100
pij
1 − pij
Dij = distance to BIP, Vij = vel, Fij = 1 if forward (vs. back)
Left
0
0
SAFE
Logistic regression for fly-balls/liners:
log
200
Distance
Bayesball Model
Logistic regression used to model smooth curves for
probability pij of successfully fielding BIP j by player i
Grounders
400
Forward
Current Methods and Data
Logistic regression for each smooth curve
Two-dimensional curves needed for flys/liners: success
depends on velocity, direction and distance to BIP
One-dimensional curves needed for grounders: success
depends on velocity, direction and angle to BIP
200
Future
Observed successes and failures are modeled as Binary
outcomes from an underlying probability pij
Each pij is a function of available data for that BIP:
Even more sophisticated approach embeds smooth
fielding curves within a Bayesian hierarchical model
300
SAFE
The outcome of each play is either a success or failure:
!
1 if the j th BIP hit to the i th player leads to out
Sij =
0 if the j th BIP hit to the i th player leads to hit
High-resolution data could also be used to fit smooth
fielding curves to the continuous playing surface
Backward
Bayesball Model
Count Data
Zone-based methods break up the field into discrete bins
for computational convenience
Current Methods and Data
Current Methods and Data
0
X Coordinate
50
100
"
pij
1 − pij
#
= βi0 + βi1 θij + βi2 θij Lij + βi3 θij Vij
θij = angle to BIP, Vij = velocity, Lij = 1 if left (vs. right)
Future
Current Methods and Data
Bayesball Model
SAFE
Future
Individual Grounder Curves
Current Methods and Data
Bayesball Model
SAFE
Future
Individual Fly/Liner Curves
Compare curves of individual fielders β$i of to aggegrate
model β$+ for all fielders at that position
Compare curves of individual fielders β$i of to aggegrate
model β$+ for all fielders at that position
P(Success) for Everett, Jeter vs. average SS
1.0
0.8
Average
Jeter
Everett
P(Success)
0.6
0.4
0.2
0.0
3rd Base
22.5
SS Location
7.5
2nd Base
7.5
22.5
1st Base
37.5
52.5
67.5
Degrees from SS
SAFE
Future
Numerical Summary of Overall Performance
Current Methods and Data
Bayesball Model
SAFE
Future
Differential Weighting in SAFE
Our full aggregation also weights grid points by BIP
frequency, run value, and shared consequence
Beyond comparing curves between players, we can derive
an overall numerical estimate of fielder performance
(a) P(Success) for Jeter vs. Average
(b) Density Estimate of Grounder Angle
1.0
0.8
P(Success)
SAFE: Spatial Aggregate Fielding Evaluation
For each player, aggregate differences between individual
curve (based on β i ) and overall curve (based on µ )
Average
Jeter
0.6
Density
Bayesball Model
0.4
0.2
0.0
3rd Base SS Location
22.5
Aggregation done by numerical integration over fine grid
of values (1D grid for grounders, 2D grid for flys/liners)
7.5
7.5
2nd Base
22.5
3rd Base SS Location
1st Base
37.5
52.5
67.5
22.5
7.5
7.5
2nd Base
22.5
1st Base
37.5
52.5
Degrees from SS
Degrees from SS
(c) Run Consequence for Grounders
(d) Shared Responsibility of SS
0.65
67.5
1.0
0.8
Responsibility Fraction
Current Methods and Data
Runs
0.60
Estimates and standard errors of β i gives us the mean and
95% confidence interval of SAFE for each player
0.55
0.50
0.6
0.4
0.2
0.0
3rd Base SS Location
22.5
7.5
7.5
2nd Base
22.5
1st Base
37.5
52.5
67.5
Degrees from SS
3rd Base SS Location
22.5
7.5
7.5
2nd Base
22.5
1st Base
37.5
52.5
67.5
Degrees from SS
SAFE value: runs saved/cost of fielder vs. average
Current Methods and Data
Bayesball Model
SAFE
Future
Results for Corner Infielders: Best/Worst Posterior SAFE values
Ten Best 1B Player-Years
Name and Year
Mean
95% Interval
Doug Mientkiewicz , 2007
7.2
( 2.8 , 11.3 )
Andy Phillips , 2007
7.1
( 2.6 , 11.4 )
Rich Aurilia , 2007
6.6
( 2.7 , 10.2 )
Albert Pujols , 2007
5.5
( 3.1 , 8.2 )
Doug Mientkiewicz , 2006
5.5
( 1.8 , 9.1 )
Albert Pujols , 2006
5.1
( 1.9 , 8.1 )
Kendry Morales , 2006
5.0
( -0.5 , 10.3 )
Ken Harvey , 2003
5.0
( 1.5 , 8.0 )
Howie Kendrick , 2006
4.5
( -0.8 , 9.6 )
Albert Pujols , 2008
4.1
( 1.0 , 6.8 )
Ten Best 3B Player-Years
Name and Year
Mean
95% Interval
Marco Scutaro , 2003
12.6
( 10.0 , 16.6 )
Mark Bellhorn , 2004
10.4
( 4.0 , 17.1 )
Hank Blalock , 2002
10.0
( 4.2 , 16.5 )
Sean Burroughs , 2004
8.9
( 3.4 , 14.2 )
David Bell , 2003
7.4
( 1.7 , 13.3 )
Scott Rolen , 2002
7.4
( 1.9 , 12.1 )
Hank Blalock , 2002
7.3
( 1.4 , 11.3 )
Damian Rolls , 2005
7.2
( 0.1 , 13.6 )
Pedro Feliz , 2002
7.1
( 0.5 , 13.3 )
Joe Crede , 2002
7.0
( 0.0 , 15.8 )
Ten Worst 1B Player-Years
Name and Year
Mean
95% Interval
Richie Sexson , 2002
-4.9
( -8.2 , -1.9 )
Robert Fick , 2002
-5.0
( -11.3 , 2.0 )
Mo Vaughn , 2002
-5.1
( -9.7 , -0.3 )
Dmitri Young , 2003
-5.5
( -9.9 , 0.1 )
Tony Clark , 2005
-6.3
( -11.7 , -1.6 )
Fred McGriff , 2002
-6.4
( -9.4 , -2.8 )
Mike Jacobs , 2002
-6.4
( -9.4 , -2.9 )
Ben Broussard , 2005
-6.7
( -10.4 , -2.2 )
Nomar Garciaparra , 2003
-7.2
( -11.1 , -3.5 )
Jason Giambi , 2003
-7.7
( -13.4 , -3.2 )
Ten Worst 3B Player-Years
Name and Year
Mean
95% Interval
Eric Munson , 2003
-7.1
( -12.4 , -2.8 )
Michael Cuddyer , 2005
-7.3
( -11.4 , -2.9 )
Michael Cuddyer , 2004
-7.4
( -14.1 , -2.3 )
Garrett Atkins , 2007
-7.8
( -12.4 , -2.4 )
Fernando Tatis , 2002
-8.1
( -14.2 , -2.0 )
Chone Figgins , 2006
-8.8
( -18.7 , -1.4 )
Travis Fryman , 2002
-9.4
( -15.2 , -4.4 )
Joe Randa , 2006
-9.8
( -17.3 , -2.8 )
Ryan Braun , 2007
-10.9
( -17.4 , -2.9 )
Jose Bautista , 2006
-11.6
( -17.4 , -5.9 )
Current Methods and Data
Bayesball Model
SAFE
Future
Results for Middle Infielders: Best/Worst Posterior SAFE values
Ten Best 2B Player-Years
Name and Year
Mean
95% Interval
Julius Matos , 2002
18.1
( 12.4 , 22.1 )
Erick Aybar , 2007
17.6
( 10.0 , 24.6 )
Junior Spivey , 2005
14.5
( 4.7 , 27.1 )
Tony Graffanino , 2006
14.1
( 4.6 , 27.6 )
Adam Kennedy , 2008
11.3
( 1.7 , 18.6 )
Willie Bloomquist , 2005
10.9
( 4.3 , 17.8 )
Jose Valentin , 2006
10.9
( 4.2 , 17.9 )
Chase Utley , 2008
10.8
( 5.7 , 17.5 )
Chase Utley , 2005
10.8
( 3.1 , 17.7 )
Craig Counsell , 2005
10.8
( 5.3 , 18.0 )
Ten Best SS Player-Years
Name and Year
Mean
95% Interval
Pokey Reese , 2004
22.6
( 12.0 , 31.2 )
Adam Everett , 2007
20.4
( 10.4 , 27.4 )
Adam Everett , 2006
17.1
( 9.0 , 21.8 )
Craig Counsell , 2006
14.7
( 6.9 , 21.1 )
Jorge Velandia , 2003
14.2
( 3.0 , 24.0 )
Alex Cora , 2005
14.1
( 3.0 , 24.6 )
Alex Rodriguez , 2003
13.5
( 3.5 , 24.4 )
Maicer Izturis , 2004
13.2
( 3.8 , 22.2 )
Marco Scutaro , 2008
13.0
( 4.0 , 20.1 )
Brent Lillibridge , 2008
11.8
( 5.0 , 19.1 )
Ten Worst 2B Player-Years
Name and Year
Mean
95% Interval
Ronnie Belliard , 2008
-9.8
( -19.5 , 2.6 )
Geoff Blum , 2005
-10.2
( -17.5 , -1.7 )
Miguel Cairo , 2004
-10.9
( -17.9 , -3.1 )
Terry Shumpert , 2002
-11.0
( -22.2 , 0.7 )
Roberto Alomar , 2003
-12.1
( -19.3 , -4.6 )
Enrique Wilson , 2004
-12.3
( -18.9 , -6.2 )
Alberto Callaspo , 2008
-12.4
( -20.4 , -4.5 )
Dave Berg , 2002
-13.5
( -25.1 , -2.4 )
Luis Rivas , 2002
-13.8
( -20.9 , -6.4 )
Bret Boone , 2005
-15.4
( -22.4 , -8.1 )
Ten Worst SS Player-Years
Name and Year
Mean
95% Interval
Erick Almonte , 2003
-13.8
( -26.9 , 2.3 )
Derek Jeter , 2007
-13.9
( -21.7 , -5.8 )
Michael Morse , 2005
-14.2
( -23.0 , -4.5 )
Damian Jackson , 2005
-14.5
( -30.6 , -3.5 )
Brandon Fahey , 2008
-15.1
( -22.4 , -8.2 )
Marco Scutaro , 2006
-15.1
( -22.0 , -10.0 )
Derek Jeter , 2003
-15.6
( -24.8 , -6.4 )
Michael Young , 2004
-15.6
( -23.6 , -7.2 )
Josh Wilson , 2007
-15.8
( -26.5 , -6.4 )
Derek Jeter , 2005
-18.5
( -29.1 , -9.2 )
Current Methods and Data
Bayesball Model
SAFE
Future
Results for Outfielders: Best/Worst Posterior SAFE values
Ten Best Center Fielders
Name and Year Mean 95% Interval
J Michaels , 05
17.9 ( 3.3 , 32.5 )
C Figgins , 03
15.5 ( 3.8 , 31.2 )
J Hairston Jr. , 05 13.7 ( 0.3 , 28.6 )
A Jones , 05
11.8 ( 2.2 , 20.7 )
D Glanville , 04
11.1 ( -3.1 , 30.5 )
J Payton , 05
10.2 ( 0.0 , 17.8 )
J Edmonds , 05
10.1 ( -0.5 , 20.5 )
J Gathright , 05
10.1 ( -6.6 , 25.0 )
D Erstad , 03
10.0 ( -1.2 , 20.7 )
C Patterson , 04
9.8 ( 1.9 , 17.9 )
Ten Best Right Fielders
Name and Year
Mean 95% Interval
G Matthews Jr. , 02 14.4 ( 5.7 , 22.3 )
D Mohr , 05
11.8 ( 2.3 , 28.0 )
T Nixon , 05
11.5 ( 3.3 , 18.1 )
G Matthews Jr. , 05 10.5 ( 2.6 , 19.0 )
R Langerhans , 05
10.5 ( 4.6 , 19.3 )
T Nixon , 04
9.4 ( 0.7 , 19.4 )
A Escobar , 03
8.7 ( -0.1 , 19.3 )
A Ochoa , 02
8.7 ( -2.3 , 20.9 )
E Marrero , 02
8.7 ( -0.9 , 20.7 )
J Drew , 03
8.4 ( -1.4 , 20.8 )
Ten Worst Left Fielders
Name and Year Mean 95% Interval
K Mench , 02
-10.7 ( -19.4 , -2.3 )
K Mench , 03
-11.1 ( -14.9 , -7.3 )
A Piatt , 02
-12.4 ( -16.8 , -6.9 )
L Berkman , 05 -13.0 ( -16.7 , -8.2 )
M Ramirez , 07 -13.5 ( -19.1 , -5.4 )
R Sierra , 03
-13.8 ( -16.2 , -10.3 )
B Kielty , 06
-15.2 ( -16.7 , -9.1 )
T Womack , 05 -17.1 ( -25.0 , -5.0 )
L Berkman , 02 -18.2 ( -18.9 , -17.0 )
M Ramirez , 06 -19.5 ( -24.8 , -13.5 )
Ten Worst Center Fielders
Name and Year Mean 95% Interval
C Hermansen , 02 -9.5 ( -23.6 , 4.3 )
D Roberts , 05
-9.8 ( -21.0 , 2.2 )
R Ledee , 02
-10.0 ( -19.6 , 0.5 )
K Griffey Jr. , 04 -12.5 ( -24.4 , -1.3 )
B Williams , 04 -13.2 ( -24.5 , -3.1 )
S Green , 05
-13.3 ( -28.3 , 2.8 )
L Terrero , 05
-13.6 ( -29.4 , 5.8 )
B Williams , 05 -14.2 ( -23.4 , -5.3 )
M Grissom , 05 -20.3 ( -34.2 , -9.4 )
J Cruz , 05
-22.4 ( -36.2 , -5.4 )
Ten Worst Right Fielders
Name and Year
Mean 95% Interval
G Kapler , 05
-8.1 ( -13.2 , -1.6 )
L Walker , 05
-8.2 ( -17.2 , 1.2 )
J Guillen , 05
-8.6 ( -17.0 , 0.7 )
K Mench , 02
-8.6 ( -17.7 , 0.7 )
E Kingsale , 04
-9.2 ( -13.1 , -2.9 )
W Pena , 02
-9.7 ( -17.6 , 0.2 )
C Wilson , 02
-11.1 ( -22.1 , -2.6 )
J Gonzalez , 05
-13.2 ( -16.4 , -10.4 )
M Tucker , 03
-14.1 ( -21.8 , -8.1 )
Sheffield , 04
-14.7 ( -21.6 , -9.5 )
Bayesball Model
SAFE
Bayesball Model
SAFE
Future
Publicity and Feedback
BIP data allows more detailed examination of differences
between players
Parametric approach: smooth probability function
reduces variance of results by sharing information between
all points near to a fielder
SAFE run value aggregates individual differences while
weighting for BIP frequency, run value, and shared
consequence between positions
Current Methods and Data
Bayesball Model
SAFE
Video-based Data
New system tracks players and BIPs with video cameras
Boston Globe (Gideon Gil, 02/16/08):
“Numbers tell a glove story”
Wired (Greta Lorge, 02/16/08):
“Statistics in the Outfield”
AP (Randolph E. Schmid, 02/16/08):
“Baseball’s top fielders ranked in
new statistical system”
New York Post had different take on study:
“You’ve Got To Be Kidding!”
Jeter himself responded in NY Post:
“Must have been a computer glitch”
Current Methods and Data
Bayesball Model
SAFE
Field F/X cont’d
How will Field F/X improve fielding estimation?
Real starting positions and speed for each player
Real hang time on flys/liners instead of current proxies
based on distance/velocity
Real trajectories on all BIPs: was that liner to the
shortstop 10 feet (catchable) or 20 feet (uncatchable) off
the ground?
Issue is availability of data. Current access limited to only
a few people (I’m not currently one of them).
Future
Summary of Our Approach
Ten Best Left Fielders
Name and Year Mean 95% Interval
E Brown , 07
14.4 ( 2.2 , 27.9 )
D Dellucci , 06
13.7 ( 5.7 , 20.4 )
R Johnson , 05
12.1 ( 2.3 , 21.0 )
C Crisp , 05
11.2 ( 4.1 , 17.8 )
S Hairston , 07
11.1 ( 1.1 , 23.5 )
S Podsednik , 07 11.1 ( 6.1 , 17.8 )
M Byrd , 05
10.7 ( 0.5 , 22.7 )
G Vaughn , 02
10.7 ( 1.9 , 16.9 )
O Palmeiro , 02 10.6 ( 0.8 , 22.2 )
T Long , 04
10.3 ( -0.8 , 21.7 )
Current Methods and Data
Current Methods and Data
This data will revolutionize the estimation of fielding ability
Future
Future