Going for Three - MIT Sloan Sports Analytics Conference

Transcription

Going for Three:
Predicting the Likelihood of Field Goal
Success with Logistic Regression
Torin Clark, Aaron Johnson, Alexander Stimpson
Massachusetts Institute of Technology
Can you spot the differences
between these two field goals?
Photos
Vinatieri: http://images.patriots.com/Vinatieri-snow.jpg
Kaeding: Joel Auerbach/Getty Images North Americn (http://www.zimbio.com/photos/Nate+Kaeding)
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
Buffalo Bills at Miami Dolphins
December 23, 2012 (Week 17)
Miami – 21, Buffalo – 3
3rd quarter, 4:22
Photos
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
3rd quarter, 4:22
Photos
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
3rd quarter, 4:22
Photos
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
3rd quarter, 4:22
Photos
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
3rd quarter, 4:22
Photos
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
3rd quarter, 4:22
Photos
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
3rd quarter, 4:22
Photos
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
3rd quarter, 4:22
Photos
3rd quarter, 4:22
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
Oakland Raiders at New England Patriots
January 19, 2002 (Divisional Playoffs)
Oakland – 13, New England – 10
4th quarter, 0:32
Photos
3rd quarter, 4:22
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
4th quarter, 0:32
Photos
3rd quarter, 4:22
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
4th quarter, 0:32
Photos
3rd quarter, 4:22
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
4th quarter, 0:32
Photos
3rd quarter, 4:22
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
4th quarter, 0:32
Photos
3rd quarter, 4:22
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
4th quarter, 0:32
Photos
3rd quarter, 4:22
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
4th quarter, 0:32
Photos
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
All 45 yard field goals are
not created equal.
3rd quarter, 4:22
4th quarter, 0:32
Photos
45 yards
45 yards
Warm
Cold
Sunny
Snowy
Calm
Windy
Regular season
Postseason
No pressure
High pressure
All 45 yard field goals are
not created equal.
3rd quarter, 4:22
Which of these factors really
make a difference?
4th quarter, 0:32
Photos
Literature Review
• Environmental factors
– Advanced NFL Stats – temperature, wind, altitude
– The Complete Guide to Kickology – playing indoors, field surface
– FootballOutsiders – altitude, temperature, playing indoors
• Situational and psychological factors
– Football Freakonomics – “icing the kicker”
Literature Review
• Limitations of these studies
– None test for statistical significance
– Average their data over multiple potentially important factors
Literature Review
• Limitations of these studies
– None test for statistical significance
– Average their data over multiple potentially important factors
• We aim to build a comprehensive binary logistic regression model that
includes all statistically significant explanatory variables and estimates the
probability that a field goal attempt will be successful
Research Aims
1. Determine what factors influence the likelihood that a field
goal attempt will be successful.
2. Develop a comprehensive logistic regression model to
quantify this likelihood and the difficulty of field goal
attempts.
3. Use this model to more accurately evaluate and compare
individual kickers, seasons, and stadiums.
4. Apply techniques to classify field goal attempts as either
makes or misses based upon relevant factors.
Model Construction
• Analyzed all 11,896 field goal attempts from the
2000-2011 NFL seasons1
• Considered the following variables:
Environmental
•
•
•
•
•
•
Distance
Temperature
Field surface
Altitude
Precipitation
Wind
Humidity
Psychological / Situational
•
•
•
•
1Complete
Postseason
In-game pressure
Home or away
Icing the kicker
play-by-play dataset obtained from ArmchairAnalysis.com
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
Cold temperature (<50°F)
βcold = -0.341
p<0.0005
Field surface (artificial turf)
βturf = 0.299
p<0.0005
Altitude (≥ 4000ft)
βalt = 0.694
p<0.0005
Precipitation (rain, snow, etc.)
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
High situational pressure
p=0.539
Away game
p=0.501
“Icing the kicker” (TO before)
p=0.118
α = 0.05
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
Psychological
/ Situational
Environmental
Model Construction
Variables
Coefficient
Significance
Constant
β0 = 5.953
p<0.0005
Distance (yards)
βdist = -0.106
p<0.0005
βcold = -0.341
p<0.0005
βturf = 0.299
p<0.0005
βalt = 0.694
p<0.0005
βprecip = -0.280
p=0.005
Windy (≥ 10mph)
βwind = -0.140
p=0.011
Humid (≥ 60%)
p=0.844
Postseason
p=0.196
p=0.539
Away game
p=0.501
p=0.118
The Model
𝑋1 = 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 (𝑦𝑎𝑟𝑑𝑠)
𝑋2−6 =
0 𝑖𝑓 𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 𝑁𝑂𝑇 𝑝𝑟𝑒𝑠𝑒𝑛𝑡
1 𝑖𝑓 𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 𝑝𝑟𝑒𝑠𝑒𝑛𝑡
Coefficients
β0 = 5.953
βdist = -0.106
βcold = -0.341
βturf = 0.299
βalt = 0.694
βprecip = -0.280
βwind = -0.140
Using the Model
45 yards
Grass
No wind
Sea-level
Warm
Coefficients
β0 = 5.953
βdist = -0.106
βcold = -0.341
βturf = 0.299
βalt = 0.694
βprecip = -0.280
βwind = -0.140
No precipitation
Using the Model
45
45 yards
Grass
0
0
0
0
0
No wind
Sea-level
Warm
Coefficients
β0 = 5.953
βdist = -0.106
βcold = -0.341
βturf = 0.299
βalt = 0.694
βprecip = -0.280
βwind = -0.140
No precipitation
Using the Model
45
45 yards
Grass
0.77
0
0
0
0
0
No wind
Sea-level
Warm
Coefficients
β0 = 5.953
βdist = -0.106
βcold = -0.341
βturf = 0.299
βalt = 0.694
βprecip = -0.280
βwind = -0.140
No precipitation
Effect Sizes and Directions
Multiple Factors
Photos
Multiple Factors
Photos
Multiple Factors
Photos
Multiple Factors
Photos
Ranking Kickers and Seasons
Added Points metric
• Allows us to rank kickers while accounting for the difficulty of
each field goal attempt
– 3 * [actual outcome of FG – model’s predicted likelihood of success]
• Actual outcome: 1 – make, 0 – miss
Added Points metric
Likelihood = 0.71
Added Points metric
Likelihood = 0.71
Miss: 3(0-0.71) = -2.13
Added Points metric
Likelihood = 0.71
Miss: 3(0-0.71) = -2.13
Make: 3(1-0.71) = 0.87
Added Points metric
Likelihood = 0.71
Miss: 3(0-0.71) = -2.13
Make: 3(1-0.71) = 0.87
– Assesses kicker’s performance relative to an average kicker given the
same opportunities
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
FA
SD
----------
Make %
87.7
86.8
86.2
86.1
85.4
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Only “career” kickers: > 50 career attempts
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
By Added Points
Rank
Kicker
1
2
3
4
5
13
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Connor Barth
Current
Team
NO
---CHI
TEN
HOU
KC
51
52
53
54
55
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
FA
SD
----------
Make %
Rank
87.7
86.8
86.2
86.1
85.4
83.9
-7
-7
1
3
-12
10
72.2
71.9
71.0
70.7
66.1
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
By Added Points
Rank
Kicker
1
2
3
4
5
13
14
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Connor Barth
John Kasay
Current
Team
NO
---CHI
TEN
HOU
KC
NO
51
52
53
54
55
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
FA
SD
----------
Make %
Rank
87.7
86.8
86.2
86.1
85.4
83.9
83.9
-7
-7
1
3
-12
10
10
72.2
71.9
71.0
70.7
66.1
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
By Added Points
Rank
Kicker
1
2
3
4
5
13
14
24
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Connor Barth
John Kasay
Dan Carpenter
Current
Team
NO
---CHI
TEN
HOU
KC
NO
MIA
51
52
53
54
55
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
FA
SD
----------
Make %
Rank
87.7
86.8
86.2
86.1
85.4
83.9
83.9
82.2
-7
-7
1
3
-12
10
10
19
72.2
71.9
71.0
70.7
66.1
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
13
14
24
48
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Connor Barth
John Kasay
Dan Carpenter
Steve Christie
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
KC
NO
MIA
---FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
83.9
83.9
82.2
75.0
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
10
10
19
-3
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Career Rankings
By Make %
Rank
Kicker
1
2
3
4
5
13
14
24
48
51
52
53
54
55
Garrett Hartley
Matt Stover
Robbie Gould
Rob Bironas
Shayne Graham
Connor Barth
John Kasay
Dan Carpenter
Steve Christie
Dave Rayner
Nick Novak
Tim Seder
Jose Cortez
Wade Richey
Current
Team
NO
---CHI
TEN
HOU
KC
NO
MIA
---FA
SD
----------
By Added Points
Make %
87.7
86.8
86.2
86.1
85.4
83.9
83.9
82.2
75.0
72.2
71.9
71.0
70.7
66.1
Rank
-7
-7
1
3
-12
10
10
19
-3
1
0
0
0
0
1
2
3
4
5
8
9
17
50
51
52
53
54
55
Current
Team
Rob Bironas
TEN
Robbie Gould
CHI
Connor Barth
KC
John Kasay
NO
Dan Carpenter
MIA
Garrett Hartley
NO
Matt Stover
---Shayne Graham
HOU
Dave Rayner
FA
Steve Christie
---Nick Novak
SD
Tim Seder
---Jose Cortez
---Wade Richey
---Kicker
AP /
Attempt
0.262
0.204
0.195
0.160
0.134
0.108
0.101
0.0705
-0.170
-0.200
-0.201
-0.342
-0.405
-0.467
Make percentage is not a good metric to use
when ranking kickers, as it is completely
ignorant of the difficulty of each attempt
Overrated
Underrated
Career Rankings
Rank
Kicker
1
2
3
4
5
51
52
53
54
55
Sebastian Janikowski
Dan Carpenter
Ryan Succop
Josh Scobee
Mason Crosby
Lawrence Tynes
Shayne Graham
Gary Anderson
Mike Vanderjagt
Stephen Gostkowki
Current
Team
OAK
MIA
KC
JAC
GB
NYG
HOU
------NE
Degree Underrated /
Overrated by Make %
25
19
16
15
14
-11
-12
-13
-17
-20
Ranking Seasons
Rank
Kicker
Season
Team
1
2
3
4
5
341
342
343
344
345
Neil Rackers
Rob Bironas
Mike Vanderjagt
Kris Brown
Neil Rackers
Kris Brown
Wade Richey
Seth Marler
2009
2005
2011
2011
2003
2001
2001
2009
2001
2003
OAK
ARI
OAK
TEN
IND
PIT
CIN
HOU
SD
JAC
Total Added
Points
19.4
18.7
18.4
17.8
16.6
-15.1
-15.2
-15.7
-16.4
-20.0
Only “full” seasons: > 25 attempts
Ranking Seasons
Rank
Kicker
Season
Team
1
2
3
4
5
341
342
343
344
345
Neil Rackers
Rob Bironas
Mike Vanderjagt
Kris Brown
Neil Rackers
Kris Brown
Wade Richey
Seth Marler
2009
2005
2011
2011
2003
2001
2001
2009
2001
2003
OAK
ARI
OAK
TEN
IND
PIT
CIN
HOU
SD
JAC
Total Added
Points
19.4
18.7
18.4
17.8
16.6
-15.1
-15.2
-15.7
-16.4
-20.0
Ranking Seasons
Rank
Kicker
Season
Team
1
2
3
4
5
341
342
343
344
345
Neil Rackers
Rob Bironas
Mike Vanderjagt
Kris Brown
Neil Rackers
Kris Brown
Wade Richey
Seth Marler
2009
2005
2011
2011
2003
2001
2001
2009
2001
2003
OAK
ARI
OAK
TEN
IND
PIT
CIN
HOU
SD
JAC
Total Added
Points
19.4
18.7
18.4
17.8
16.6
-15.1
-15.2
-15.7
-16.4
-20.0
Ranking Seasons
Rank
Kicker
Season
Team
1
2
3
4
5
341
342
343
344
345
Neil Rackers
Rob Bironas
Mike Vanderjagt
Kris Brown
Neil Rackers
Kris Brown
Wade Richey
Seth Marler
2009
2005
2011
2011
2003
2001
2001
2009
2001
2003
OAK
ARI
OAK
TEN
IND
PIT
CIN
HOU
SD
JAC
Total Added
Points
19.4
18.7
18.4
17.8
16.6
-15.1
-15.2
-15.7
-16.4
-20.0
Ranking Seasons
Rank
Kicker
Season
Team
1
2
3
4
5
341
342
343
344
345
Neil Rackers
Rob Bironas
Mike Vanderjagt
Kris Brown
Neil Rackers
Kris Brown
Wade Richey
Seth Marler
2009
2005
2011
2011
2003
2001
2001
2009
2001
2003
OAK
ARI
OAK
TEN
IND
PIT
CIN
HOU
SD
JAC
Total Added
Points
19.4
18.7
18.4
17.8
16.6
-15.1
-15.2
-15.7
-16.4
-20.0
Ranking Seasons
Rank
Kicker
Season
Team
1
2
3
4
5
341
342
343
344
345
Neil Rackers
Rob Bironas
Mike Vanderjagt
Kris Brown
Neil Rackers
Kris Brown
Wade Richey
Seth Marler
2009
2005
2011
2011
2003
2001
2001
2009
2001
2003
OAK
ARI
OAK
TEN
IND
PIT
CIN
HOU
SD
JAC
Total Added
Points
19.4
18.7
18.4
17.8
16.6
-15.1
-15.2
-15.7
-16.4
-20.0
• 3/5 of the best five seasons occurred between 2008-2011
• 4/5 of the worst five seasons occurred between 2000-2003
Ranking Seasons
Rank
Kicker
Season
Team
1
2
3
4
5
341
342
343
344
345
Neil Rackers
Rob Bironas
Mike Vanderjagt
Kris Brown
Neil Rackers
Kris Brown
Wade Richey
Seth Marler
2009
2005
2011
2011
2003
2001
2001
2009
2001
2003
OAK
ARI
OAK
TEN
IND
PIT
CIN
HOU
SD
JAC
Total Added
Points
19.4
18.7
18.4
17.8
16.6
-15.1
-15.2
-15.7
-16.4
-20.0
• 3/5 of the best five seasons occurred between 2008-2011
• 4/5 of the worst five seasons occurred between 2000-2003
• Are kickers getting better?
Are kickers getting better?
• Yes. There is a statistically-significant upward trend
+0.017 added points per attempt/season, t(11)=5.34, p<0.0005
• Yes. There is a statistically-significant upward trend
+0.017 added points per attempt/season, t(11)=5.34, p<0.0005
Why?
• Kickers are becoming more experienced
– Average kicker experience is increasing
– Adding kicker experience (# of seasons) to the model
improved the prediction of FG success likelihood
(coefficient=+0.017/season of experience, Wald Statistic=3.21, p=0.001)
• Kickers are becoming more experienced
– Average kicker experience is increasing
– Adding kicker experience (# of seasons) to the model
improved the prediction of FG success likelihood
(coefficient=+0.017/season of experience, Wald Statistic=3.21, p=0.001)
• There is another effect at work (better kickers
entering the league, improved training techniques)
– Adding season to the model also improved the prediction
(coefficient=+0.037, Wald Statistic=4.86, p<0.0005 )
Ranking Current Stadiums
Rank
Stadium
Team
2
4
5
6
7
39
41
42
44
45
Lambeau Field
Heinz Field
Cleveland Browns Stadium
Soldier Field
Arrowhead Stadium
Georgia Dome
Mall of America Field at H.H.H. Metrodome
Mercedes-Benz Superdome
Ford Field
Sports Authority Field at Mile High
GB
PIT
CLE
CHI
KC
ATL
MIN
NO
DET
DEN
Avg. Likelihood
Relative to Mean
-0.063
-0.049
-0.049
-0.047
-0.042
0.0461
0.0461
0.0461
0.0461
0.0957
Classification
• Given particular kicking conditions, can we predict
outcome?
Support Vector Machines (SVMs)
Ensemble Methods (Adaboost)
Classification
• Challenges of dataset
– High proportion of makes
– Inherent noise
• Misclassification rates marginally better than
maximum likelihood prediction (all makes)
• Probabilistic analysis more applicable to this type of
dataset
Model Use
Fans
Media
Coaches
Better understanding
the game
Evaluating kickers and
field goal attempts
In-game decision
making
Photos
Fans: http://www.steeleraddicts.com/blog/wp-content/uploads/2011/01/steelerfans.jpg
MNF: http://www.freesworld.com/wp-content/uploads/2012/10/swtg-mike-tirico-gruden.jpg
Belichick: http://bradlaidman.com/wp-content/uploads/bill-belichick.jpg
Conclusions
• A comprehensive model of field goal attempts allows us to
investigate what factors really have an effect on the outcome
– No evidence to support icing the kicker or high pressure situations (end of
close games, playoffs, or on the road) have any effect
– Poor weather conditions (cold, precipitation, or windy) make attempts more
difficult to convert.
– Altitude improves attempt chances more than any other variable studied.
Conclusions
Photos
Reid: http://juniordsports.com/wp-content/uploads/2012/10/timeout.jpg
Conclusions
Photos
Reid: http://juniordsports.com/wp-content/uploads/2012/10/timeout.jpg
Dawson: http://cdn0.sbnation.com/legacy_images/dawgsbynature/images/admin/07week15dawson.jpg
Conclusions
• Accounting for attempt difficulty allows us to better rank and
investigate kicker ability
– Rob Bironas has had the best career and Sebastian Janikowski the best season
(2009) of any kicker from 2000-2011 when accounting for kick difficulty.
– On average, over the last 11 seasons, kickers have been improving when
accounting for kick difficulty.
Photos
Bironas: http://assets.nydailynews.com/polopoly_fs/1.1181661.1350026120!/img/httpImage/image.jpg_gen/
derivatives /landscape_635/rob-bironas-titans.jpg
Janikowski: http://www.toledoblade.com/image/2011/09/13/800x_b1_cCM_z_cT/Sebastian-Janikowski-kicksrecord-field-goal-for-raiders.jpg
Now you know what to really
worry about
Photo
References
• Burke, Brian. “Temperature and Field Goals.” AdvancedNFLStats.com.
Advanced NFL Stats, 17 Jan 2012. Web. 16 May 2012.
• Burke, Brian. “Altitude and Field Goals.” AdvancedNFLStats.com. Advanced
NFL Stats, 9 Jan 2013. Web. 12 Jan 2012.
• Dubner, Stephen J. “Why Even Ice a Kicker?” Football Freakonomics
Episode 2. NFL, 13 Nov 2011. Web. 16 May 2012.
• Herman, Mike. The Complete Guide to Kickology. 3rd edition. Footballguys,
2009. Web. 16 May 2012.
• Schatz, Aaron. “Methods to Our Madness: Special Teams.”
FootballOutsiders.com. Web. 16 May 2012.
Other Academic Papers
•
Berry, D.A., Berry T.D. “The Probability of a Field Goal: Rating Kickers.”
– Built logistic regression model with kicker specific distance parameter (no other factors) using
1983 season data
•
Morrison, D.G., Kalwani, M.U. “The Best NFL Field Goal Kickers: Are They Lucky or Good?”
– Found kicker’s performance in one year did not predict performance in another year, even
when accounting for distance (i.e. kickers are lucky) using 1989-1991 season data
•
Bilder, C.R., Loughin, T.M. “ ‘It’s Good!’ an Analysis of the Probability of Success for
Placekicks”
– Built logistic regression using FGs and PATs using distance, environmental, and psychological
factors
– Contrary to our results, they found pressure, PAT, and time remaining to be significant and
altitude, precipitation, surface, temperature, and wind to not be significant possibly because
only used 1995-1996 season data and included PATs
• Berry, S.M. “A Geometry Model for NFL Field Goal Kickers.”
– Distance model that used length and precision components
Backup
Photo
R. Allan Schnoor (http://www.examiner.com/slideshow/photos-from-49ers-practice-readying-forpackers#slide=57399031)
Categorization of Pressure
Time Remaining
4th quarter
4th quarter, < 2 minutes
1st-3rd quarters
4th quarter, > 2 minutes
Overtime
th
4 quarter, < 2 minutes
Score
Differential
> |21|
> |8|
< -7
Any
< |21|
5, 6, 7, 8
-4, -5, -6
4
3
1, 2
0
0 (Any)
-3
-1, -2
Effect of Kick
No effect
No effect
No effect
Regular effect
“Close” 4th quarter
Helps seal game
Come within 3
Opponent needs TD
Opponent needs TD
Opponent needs TD
Win. If miss, OT
Win. If miss, more OT
OT. If miss, lose
Win. If miss, lose
6-category
Pressure
No
No
No
Low
Medium
Medium
Medium
Medium-high
Medium-high
High
Higher
Higher
Highest
Highest
2-category
Pressure
Low
Low
Low
Low
Low
Low
Low
High
High
High
High
High
High
High
Details of Categorization
•
•
•
•
•
All environmental conditions are at kickoff and not specific to the time of each
individual kick.
The only games played at altitudes greater than 4,000 ft were those in Denver or
Mexico City (Oct. 2, 2005).
A “chance of rain” is categorized as no precipitation. The model was tested with
the alternative categorization, and this had a negligible impact on the value and
significance of the coefficient.
Neutral site games were categorized based upon official distinctions. Treating
neutral site games as a third category was still not significant.
A timeout called by either head coach was considered “icing the kicker.”
Categorizing “icing” as either no timeout, timeout by opposing coach, or timeout
by own coach was still not significant.
Model Building
Model Selection criteria:
Complexity
Accuracy
2800
AIC
2750
Selection Criteria Values
Schwarz’s BIC: ~5 predictor variables
AIC: ~10 predictor variables
Schwarz's BIC
-2 LogLikelihood
2700
2650
2600
2550
1
2
3
4
5
6
7
8
9
Number of Predictor Variables (p)
10
11
12
• Kicker is not significant
including a coefficient for
every kicker
– But there seemed to be
large differences between
some of the kickers
– Grouping similar kickers
might help*
• How many groups?
– Use kickers’ raw make
percentages for long kicks
– K-means clustering, select
with SBC/AIC
Criterion Value
Kicker effect?
3150
3100
3050
3000
2950
2900
2850
2800
2750
2700
AIC
SBC
0
10
20
30
40
50
Number of Kicker Groups
There is evidence for differences
between kickers, but not enough
to include individual kickers in
the model.
Model Diagnostics
• Logistic regression has few diagnostics
1
Diagnostic Residual Plot: Goodness of Fit:
Residual
0 0.5
-1 -0.5 RESIDUALS
1.0
0.5
Detection of Influential Observations:
Cook’s distance: standardized change in
fitted response vector when ith case is
deleted
0.0
-0.5
-1.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
0
0.5
1
Estimated Probability
• Lowess smoothing fit near
horizontal with 0 intercept
indicates a good fit
• Hosmer-Lemeshow Goodness of Fit
Test confirms a good fit (χ2(8)=8.03,
p=0.403)
PRED
• Cases with Cook’s dist > 0.05
temporarily removed and log reg refit
• Results not appreciably different from
those obtained from the full data set,
so the cases were retained
Model Diagnostics (2)
• Half-Normal Probability Plot with Simulated Envelope
3
2.5
Deviance Residuals
• Use model predicted
probabilities to simulate 19
sets of DVs
• Fit model to 19 sets of DVs
• Calculate “deviance residuals”
• Sort absolute dev residuals
• Calculate min, max, and
average of abs dev residuals
• Plot vs. expected value
2
1.5
1
0.5
0
• Dev residual from actual
observations should fit within
simulated 95% confidence
bounds
0
1
2
3
Expected Value
• Linear model is appropriate
• No outlying deviance residuals
4
Cook’s Distance Details
• Cook’s distance statistic: help identify influential
observations by measuring the standardized change
in the fitted response vector πhat when the ith case
is deleted
– Requires n maximizations of the likelihood, so an
approximation is used:
No rules of thumb for
logistic regression
outliers, use visual
assessment
hii is the ith diagonal
of the H (or hat)
matrix
(approximately)
Converts n observations
into n predictions
p = number of explanatory variables = 6
Kicker Improvement

Going for Three - MIT Sloan Sports Analytics Conference

Transcription

Similar documents

YLC JAN2-4 brochure - Greater Forsyth County Young Life

Dukono - Carlyle Collective

Specifications: Illude

KICKER KX120 - TuningMania.com.br

Lil Rascals.indd

Clausing 8520 Vertical Milling Machine Catalog

MetroABQ Real Estate Newsletter What Is Going On At The ABQ

pdf version

S.Pellegrino Sparkling Mineral Water