Introduction

In football, the term “icing the kicker” refers to a play immediately before a field goal attempt in which the defensive team calls a timeout in the hope that it will affect the kicker’s ability to make the field goal. This is important to look at because it is often a game deciding play. For example, in week 14 of the 2011 season, the New York Giants were winning the game against the Dallas Cowboys 37-34, and there was just enough time for one possession. The possession team, the Dallas Cowboys, decided to attempt a field goal to tie the game because there were just 6 seconds left. By the time the Cowboy’s kicker, Dan Bailey, was about to kick, the New York Giants called a strategic timeout hoping that he would miss, and he did indeed miss, giving the win to the Giants.

The research question we are going to focus on is: Does icing the kicker (i.e., calling a timeout before a field goal) impact a kicker’s ability to make the field goal? To analyze it, we need to account for the fact that the decision of calling a timeout is deliberately made, and therefore, the data we are going to work with is non-random observational data. Given the nature of our data, we will approach our research question from a causal inference perspective. Causal inference is a strategy to test whether a particular explanatory variable, called a treatment, causes a particular output through a simulated randomized experiment. The objective of causal inference is to learn about the causal effect of the application of the treatment relative to the control.

In our instance, the treatment and therefore the experimental variable is iced kicks, and the set of potential controls is all non-iced kicks. We will select out controls through a process referred to as matching, as discussed in the Methods Section. We want to measure the effect of the treatment (icing a kick) on the outcome, which is whether or not the kicker makes the field goal attempt. This is analogous to the effect of getting a vaccine, which would be the treatment, to prevent disease infection, which would be the outcome.

Data

To work on this project, we downloaded NFL play-by-play data using the nflfastR package in R. This data set contains data from the 1999 season to the 2021 season.

Given our interest in iced kicks, which we considered to be any field goal attempt that was immediately preceded by the opposing team on defense calling a timeout, we filtered out how many kicks during those 22 seasons were iced, and out of 23259 field goal attempts, only 1393 were iced.

In addition, we created a separate data set that contained only variables that we found to potentially be related to a kicker’s ability to make a kick. This new data set included variables such as game_half, game_seconds_remaining, half_seconds_remaining, goal_to_go, yardline_100, ydstogo, kick_distance, score_differential, kicker_player_name, kicker_player_id, wp, vegas_wp, and ep. There is a more informative description of all variables contained in the Appendix.

To get a better understanding of the data set we were working with, we performed EDA (Exploratory Data Analysis). First, we wanted to observe the number of field goals and iced field goals that happened each season. We wondered if there was kind of a constant tendency on the number of total field goals per year or if it varied a lot between them. In addition, we also looked at the proportion of field goals that were iced during each season. To do so, we created the following three figures.

As we expected, the number of field goals per season varied throughout the years, and the proportion of iced kicks during each season was relatively constant. In addition, it is important to mention how the number of field goals and the number of iced kicks have an increasing tendency, because that could mean that more people are using them because they believe they work.

Next, we wanted to observe the proportion of iced and non-iced kicks that were made, missed or blocked, and to do so, we created the next figure.

Here, we see that roughly 77% of iced and 83% of non-iced kicks were made, with non-iced kicks having a slightly higher proportion. We can also appreciate how the proportion of iced kicks that were missed, approximately 20%, is greater than the proportion of non-iced kicks that were missed, which was approximately 15%. Although there is a (roughly) 6% increase in the proportion of made non-iced kicks relative to iced kicks, we are interested if this difference can be attributed to the act of icing the kicker (treatment). Therefore, as mentioned previously, we will employ a causal framework to isolate the treatment effect for any differences in these proportions while controlling for a host of additional variables.

Then, we also wanted to see when in the game field goals and iced kicks were happening, so we created the following two histograms. i

By looking at these plots, we observed that the amount of field goals and iced kicks that happen at the end of each half is greater than that of the rest of the game.

After that, we created density plots to explore the distributions of kick distance, half seconds remaining, score differential, win probability and expected points. The aim of creating these plots was to see if any of these variables is associated with our treatment variable, icing the kicker, in order to determine the “causality” of icing the kicker to the ability of a kicker to make the field goal.

The first density plot we created was the kick distance plot shown below. On it, we can appreciate how there are a lot more non-iced kicks taken in a distance smaller than 30 yards, and a lot more iced kicks taken at a distance higher than 45 yards. This plot shows that kick distance highly affects the decision of icing a kicker, since for short distance kicks (<30 yards), coaches decide not to ice the kicker because the kick is almost automatic. In contrast, for bigger kick distances (>45 yards), coaches are more likely to call a timeout because it is harder to kick a field goal.

The next density plot that we created was half seconds remaining. By looking at the figures, we can see that at the end of the first half, there were significantly more iced kicks taken than non-iced ones. The same trend can be observed at the end of the second half, but with a smaller difference between them. However, during overtime, by visual inspection, it does not seem like there is a significant difference between the amount of iced and non-iced field goals taken.

Our next density plot was the one of score differential. In this figure we can appreciate that most field goals are happening when the game is within about 10 points. In the first half, we observed that there were significantly more non-iced than iced kicks taken when the score differential was within roughly 10 points. In the second half, we observed a different trend, and more iced than non-iced kicks were taken within a score differential of approximately 5 points.

Second to last, we created the density plot of win probability, which shows that when a play involving a kick has a higher win probability, the kicker is more likely to be iced than not. This aligns with our expectations, as coaches tend to use the strategy of icing the opponent’s kicker when the game is close and the kick could give the other team a lead.

By last, we made a expected points density plot. This plot shows that the density of iced and non-iced kicks follow a similar distribution, the only significant difference is that non-iced kicks have a higher density when the score differential is around 3.

Methods

In an ideal world where we could randomly assign whether a kick is iced or not, it would be much easier to figure out whether icing affects the success of a kick. Unfortunately for us, and for the sake of fairness in sports, that cannot happen, especially at the level of the NFL. Because icing a kicker is not randomly decided and is tied to several other explanatory variables, we need to take a causal inference approach to this problem.

To estimate the effect of icing the kicker on modeling whether the kicker made the field goal or not, we replicated a randomized experiment by matching treatment and controls. In our case, controls are kicks that are not iced and treatments are kicks that were iced, the treatment being the icing.

We did the following:

  1. First, we used all data without matching and ran logistic regression using a generalized additive model (GAM) and a generalized linear model (GLM) for predicted success of a field goal using only kick distance as an explanatory variable and the binary for an iced kick as an indicator variable. Kick distance was the only explanatory variable used for simplicity and because it is probably the variable that most affects a kicker’s ability to make a field goal or extra point. After looking at both the GLM and GAM models, we chose to stick to the GAM model because of the choice of a smooth function of distance.

  2. Second, we matched on win probability in a proportion of 1 to 10, meaning that for every treatment (iced kick) we have 10 controls (non-iced kick). We decided to match only on win probability because it takes into account many explanatory variables including kick distance, time remaining, score differential, and more. Here is one example of a match that was made:

  1. Then, we refit the model and ran generalized additive model (GAM) again.

  2. Next, we created the propensity score to match off of. The propensity was for an iced kick based off of kick distance, win probability, and seconds remaining in the half. We decided to match on these variables because they are the ones that we thought could be related to the ability of the kicker to make a field goal after performing EDA. We created the score by fitting a GAM.

  3. Then we refit the model using a GAM for kick success using the propensity score as the response variable.

  4. Last, we looked at the coefficients, confidence intervals, and p-values of all three models to see if icing the kicker does actually make a difference on the ability of the kicker to make the field goal.

Results

After going through the previously mentioned first five steps, we considered the results for the three GAM models based on the following data set: (1) unmatched data (i.e., all data), (2) matched data on win probability, and (3) matched data on the propensity score. By visual inspection, it does not seem that the predicted probabilities differed between the GAM models (figure below shows only unmatched and propensity score but we observe a similar figure for matched on win probability).

Then, we looked at the coefficients, confidence intervals and p-values for the iced kick indicator variable across the different models, yielding the following table of results:

Summary of the Three Models
Data Coefficient St.Deviation p.value
Unmatched Data -0.06695 0.07216 0.353
Matched Data -0.07265 0.07398 0.326
Propensity Score -0.06833 0.0735 0.353

The coefficient, standard deviation and p-values do not differ a lot between models, which indicates that our estimated effect from icing the kicker is roughly the same regardless of matching.

By last, we created a coefficient plot using the values shown in the previous table. This plot shows the coefficients found for the treatment of icing a kicker for each of the three logistic regression models we fit. All the coefficients were found to be negative, suggesting that icing a kicker has a negative impact on a kicker’s ability to make a field goal. However, when looking at the 95% confidence interval of the coefficients, all three intervals include zero, which indicates it could make no difference at all.

Discussion

Overall, we have not found evidence that a kicker’s ability is affected by timeouts. We have just focused on variables such as win probability, score differential or half seconds remaining, but for future improvement, we think that variables such as weather or stadium could be relevant. It is not the same to attempt a field goal at an indoor facility than at an outdoor one. The same happens with the type of surface, rain, sun, temperature, and wind conditions.

Another interesting thing to account for in the future is whether the clock is running or not, because a running clock might cause uncertainty for the offensive team. We also plan on looking at the effect of icing the kicker across each year separately to see if maybe icing the kicker had an effect before but it has changed. In addition, in our future work, we also want to consider kicker effects, which could be modeled using random effects.

Even though we have not found evidence that proves that calling a timeout right before the kicker attempts the field goal affects his ability of making it or not, this could change with future work. We are also basing our project on our definition of what icing the kicker means for us but maybe if that definition changes, the outcome could be completely different. One example of an alternative definition for icing the kicker could be to just consider field goals within the final three minutes of the second half in close games.

Acknowledgements

This project was proposed by Dr. Ryan Elmore, an associate professor at University of Denver in cooperation with Carnegie Mellon University Statistics and Data Science department, and has been supervised by Dr. Ron Yurko. We would like to thank Dr. Elmore and Dr. Yurko for their guidance throughout the process. In addition, we thank Dr. Mike Lopez, Senior Director of Football Data and Analytics at the National Football League, for his advice. Last but not least, we also appreciate the help that the teaching assistants from the CMSA program provided us during the process.

References

Greifer, N. (2022, May 18). Matchit: Getting started. Retrieved July 27, 2022, from https://cran.r-project.org/web/packages/MatchIt/vignettes/MatchIt.html

(n.d.). Retrieved July 27, 2022, from https://www.nflfastr.com/

Appendix

Description of Variables

Variable Names Description
play_id Numeric play id that when used with game_id and drive provides the unique identifier for a single play
game_id Ten digit identifier for NFL game
old_game_id Legacy NFL game ID
home_team String abbreviation for the home team
away_team String abbreviation for the away team
play_type String indicating the type of play: pass (includes sacks), run (includes scrambles), punt, field_goal, kickoff, extra_point, qb_kneel, qb_spike, no_play (timeouts and penalties), and missing for rows indicating end of play
timeout Binary indicator for whether or not a timeout was called by either team
posteam String abbreviation for the team with possession
defteam String abbreviation for the team on defense
prev_def_team_to Prev_def_team_to
is_iced_kick is_iced_kick
desc Detailed string description for the given play
half_seconds_remaining Numeric seconds remaining in the half
game_seconds_remaining Numeric seconds remaining in the game
quarter_end Binary indicator for whether or not the row of the data is marking the end of a quarter
sp Binary indicator for whether or not a score occurred on the play
qtr Quarter of the game (5 is overtime)
goal_to_go Binary indicator for whether or not the posteam is in a goal down situation
yrdln String indicating the current field position for a given play
yardline_100 Numeric distance in the number of yards from the opponent’s endzone for the posteam
ydstogo Numeric yards in distance from either the first down marker or the endzone in goal down situations
field_goal_result String indicator for result of field goal attempt: made, missed, or blocked
kick_distance Numeric distance in yards for kickoffs, field goals, and punts
score_differential Score differential between the posteam and defteam at the start of the play
score_differential_post Score differential between the posteam and defteam at the end of the play
field_goal_attempt Binary indicator for field goal attempt
kicker_player_name String name for the kicker on FG or kickoff
kicker_player_id Unique identifier for the kicker on FG or kickoff
end_clock_time Game time at the end of a given play
result Equals home_score - away_score and means the game outcome from the perspective of the home team
success Binary indicator wheter epa > 0 in the given play
special Binary indicator if the play was a special teams play
game_half String indicating which half the play is in, either Half1, Half2, or Overtime
wp Estimated win probabiity for the posteam given the current situation at the start of the given play
ep Using the scoring event probabilities, the estimated expected points with respect to the possession team for the given play
vegas_wp Estimated win probabiity for the posteam given the current situation at the start of the given play, incorporating pre-game Vegas line