Cited by 7Sports Analytics and PerformanceSports Performance and TrainingStatistical Methods and Bayesian Inference

nflWAR: a reproducible method for offensive player evaluation in football

Open access
Yurko, Ronald, Ventura, Samuel, Horowitz, Maksim·2019·Journal of Quantitative Analysis in Sports

Abstract

Existing methods for player evaluation in American football rely heavily on proprietary data, are often not reproducible, lag behind those of other major sports, and are not interpretable in terms of game outcomes. We present four contributions to the study of football statistics to address these issues. First, we develop the R package nflscrapR to provide easy access to publicly available play-by-play data from the National Football League (NFL). Second, we introduce a novel multinomial logistic regression approach for estimating the expected points for each play. Third, we use the expected points as input into a generalized additive model for estimating the win probability for each play. Fourth, we introduce our nflWAR framework, using multilevel models to isolate the contributions of individual offensive skill players in terms of their wins above replacement (WAR). We assess the uncertainty in WAR through a resampling approach specifically designed for football, and we present results for the 2017 NFL season. We discuss how our reproducible WAR framework can be extended to estimate WAR for players at any position if researchers have data specifying the players on the field during each play. Finally, we discuss the potential implications of this work for NFL teams.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1515/jqas-2018-0010

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Yurko et al. (2019) estimate win probability in the NFL using a generalized additive model (GAM), as part of the nflscrapR package and nflWAR framework.

    “In addition, Yurko et al. (2019) estimate win probability in the NFL using a generalized additive model (GAM), as part of the nflscrapR package (Horowitz et al., 2020) and nflWAR framework.”

    From Big ideas in sports analytics and statistical tools for their investigation (2023) · cites this paper as Yurko et al. (2019)

  • Yurko et al. (2019)'s win probability model is fully reproducible and uses publicly available data.

    “A vital highlight of Yurko et al. (2019)'s win probability model is that it is fully reproducible and uses publicly available data.”

    From Big ideas in sports analytics and statistical tools for their investigation (2023) · cites this paper as Yurko et al. (2019)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 31

  1. Alamar, B. 2010. "Measuring Risk in NFL Playcalling." Journal of Quantitative Analysis in Sports 6(2). https://doi.org/10.2202/ 1559-0410.1235. · DOI
  2. Albert, J. 2006. "Pitching Statistics, Talent and Luck, and the Best Strikeout Seasons of All-Time." Journal of Quantitative Analysis in Sports 2(1). https://doi.org/10.2202/1559-0410.1014. · DOI
  3. Balreira, E. C., B. K. Miceli, and T. Tegtmeyer. 2014. "An Oracle Method to Predict NFL Games." Journal of Quantitative Analysis in Sports 10:183-196.
  4. Bates, D., M. Machler, B. Bolker, and S. Walker. 2015: "Fitting Linear Mixed-Effects Models Using lme4." Journal of Statistical Software 67:1-48.
  5. Baumer, B., and P. Badian-Pessot. 2017. "Evaluation of Batters and Base Runners". Pp. 1-37 in Handbook of Statistical Methods and Analyses in Sports, edited by J. Albert, M. E. Glickman, T. B. Swartz and R. H. Koning, Boca Raton, Florida: CRC Press.
  6. Baumer, B., S. Jensen, and G. Matthews. 2015. "Openwar: An Open Source System for Evaluating Overall Player Performance in Major League Baseball". Journal of Quantitative Analysis in Sports 11:69-84.
  7. Becker, A. and X. A. Sun. 2016. "An Analytical Approach for Fan- tasy Football Draft and Lineup Management". Journal of Quantitative Analysis in Sports 12:17-30.
  8. Cafarelli, R., C. J. Rigdon, and S. E. Rigdon. 2012. "Models for Third Down Conversion in the National Football League." Journal of Quantitative Analysis in Sports 8(3). https://doi.org/10.1515/ 1559-0410.1383. · DOI
  9. Clark, T. K., A. W. Johnson, and A. J. Stimpson. 2013. "Going for Three: Predicting the Likelihood of Field Goal Success with Logistic Regression". MIT Sloan Sports Analytics Conference.
  10. Deshpande, S. K. and S. T. Jensen. 2016. "Estimating an NBA Player's Impact on his Team's Chances of Winning". Journal of Quantitative Analysis in Sports 12:51-72.
  11. Eager, E. A., G. Chahrouri, and S. Palazzolo. 2017. "Using PFF Grades to Cluster Quarterback Performance". Pro Football Focus Research and Development Journal 1:4-14.
  12. Franks, A. M., A. D'Amour, D. Cervone, and L. Bornn. 2017. "Meta- Analytics: Tools for Understanding the Statistical Properties of Sports Metrics". Journal of Quantitative Analysis in Sports 12:151-165.
  13. Gelman, A. and J. Hill. 2007. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge, United Kingdom: Cambridge University Press.
  14. Goldner, K. 2012. "A Markov Model of Football: Using Stochastic Processes to Model a Football Drive". Journal of Quantita- tive Analysis in Sports 8(1). https://doi.org/10.1515/1559- 0410.1400. · DOI
  15. Goldner, K. 2017. "Situational Success: Evaluating Decision-Making in Football". Pp. 183-198 in Handbook of Statistical Methods and Analyses in Sports, edited by J. Albert, M. E. Glickman, T. B. Swartz, and R. H. Koning. Boca Raton, Florida: CRC Press.
  16. Gramacy, R. B., M. A. Taddy, and S. T. Jensen. 2013. "Estimat- ing Player Contribution in Hockey with Regularized Logis- tic Regression". Journal of Quantitative Analysis in Sports 9:97-111.
  17. Grimshaw, S. D. and S. J. Burwell. 2014. "Choosing the most Pop- ular NFL Games in a Local tv Market". Journal of Quantitative Analysis in Sports 10:329-343.
  18. Jensen, J. A. and B. A. Turner. 2014. "What if Statisticians Ran Col- lege Football? A Re-Conceptualization of the Football Bowl Subdivision". Journal of Quantitative Analysis in Sports 10:37-48.
  19. Jensen, S., K. E. Shirley, and A. Wyner. 2009. "Bayesball: A Bayesian Hierarchical Model for Evaluating Fielding in Major League Baseball". The Annals of Applied Statistics 3:491-520.
  20. Kubatko, J., D. Oliver, K. Pelton, and D. T. Rosenbaum. 2007. "A Starting Point for Analyzing Basketball Statistics". Journal of Quantitative Analysis in Sports 3(3). https://doi.org/10.2202/ 1559-0410.1070. · DOI · Paper page
  21. Lock, D. and D. Nettleton. 2014. "Using Random Forests to Estimate Win Probability before Each Play of an NFL Game". Journal of Quantitative Analysis in Sports 10:1-9.
  22. Macdonald, B. 2011. "A Regression-Based Adjusted Plus-Minus Statistic for NHL Players". Journal of Quantitative Analysis in Sports 7(3). https://doi.org/10.2202/1559-0410.1284. · DOI
  23. Martin, R., L. Timmons, and M. Powell. 2017. "A Markov Chain Analysis of NFL Overtime Rules". Journal of Sports Analytics 4:95-105.
  24. Mulholland, J. and S. T. Jensen. 2014. "Predicting the Draft and Career Success of Tight Ends in the National Football League". Journal of Quantitative Analysis in Sports 10:381-396.
  25. Pasteur, D. and K. Cunningham-Rhoads. 2014. "An Expectation- Based Metric for NFL Field Goal Kickers". Journal of Quantitative Analysis in Sports 10:49-66.
  26. Piette, J. and S. Jensen. 2012. "Estimating Fielding Ability in Base- ball Players Over Time". Journal of Quantitative Analysis in Sports 8(3). https://doi.org/10.1515/1559-0410.1463. · DOI
  27. Romer, D. 2006. "Do Firms Maximize? Evidence from Professional Football". Journal of Political Economy 114:340-365.
  28. Snyder, K. and M. Lopez. 2015. "Consistency, Accuracy, and Fair- ness: A Study of Discretionary Penalties in the NFL". Journal of Quantitative Analysis in Sports 11:219-230.
  29. Thomas, A. C., S. L. Ventura, S. T. Jensen, and S. Ma. 2013. "Competing Process Hazard Function Models for Player Ratings in Ice Hockey". The Annals of Applied Statistics 7:1497-1524.
  30. Wakefield, K. and A. Rivers. 2012. "The Effect of Fan Passion and Oflcial League Sponsorship on Brand Metrics: A Longitudi- nal Study of Oflcial NFL Sponsors and Roo". MIT Sloan Sports Analytics Conference.
  31. Yurko, R., S. Ventura, and M. Horowitz. 2017. "Nfl Player Evaluation Using Expected Points Added with Nflscrapr". Presented at the Great Lakes Sports Analytics Conference.

Cited by

Sorted by citations

Authors
Yurko, R., Ventura, S., & Horowitz, M.
Affiliations
Carnegie Mellon University (US)
Venue
Journal of Quantitative Analysis in Sports
Published
2019-04-16
Volume
15
Issue
3
Pages
163-183
ISSN
  • 1559-0410 (ISSN-L)
  • 2194-6388
OpenAlex
W2787615756
Cited by
7
References
0
Citations by year
2026: 6 · 2023: 1
FWCI
0.00
Type
journal article
Language
en
Open access
Locations
  • Journal of Quantitative Analysis in Sports (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Carnegie Mellon University
Topics
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Sports Analytics and Performance
  • Health Sciences › Medicine › Orthopedics and Sports Medicine › Sports Performance and Training
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
player performance evaluation, American football, NFL, wins above replacement, offensive players, winning probability, play-by-play data, multinomial logistic regression, generalized additive models, multilevel models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review