What was lost? A causal estimate of fourth down behavior in the National Football League
Open accessAbstract
In part driven by academic research, perception in the sports analytics community asserts that coaches in the National Football League are too conservative on fourth down. Using 13 years of data, we confirm this premise and quantify the unobserved benefit that teams have missed out on by not utilizing a better fourth down strategy. Formally, teams that went for it are paired to those who did not go for it via a nearest neighbor matching algorithm. Within the matched cohort, we estimate the additional number of wins that each NFL team would have added by implementing a basic but more aggressive fourth down strategy. We find that, on average, a better strategy would have been worth roughly an extra 0.4 wins per year for each team. Our results better inform decision-making in a high-stakes environment where standard statistical tools, while informative, have possibly been confounded by extraneous factors.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The two win probability frameworks proposed by Lock and Nettleton (2014) and Yurko et al. (2019) were also implemented in Yam and Lopez (2019) with minimal modifications.
“The two win probability frameworks proposed by Lock and Nettleton (2014) and Yurko et al. (2019) were also implemented in Yam and Lopez (2019) with minimal modifications.”
From Big ideas in sports analytics and statistical tools for their investigation (2023) · cites this paper as Yam and Lopez (2019)
Section 3.4 (Outcome) says Yam and Lopez replicated Lock and Nettleton’s random-forest model, and Appendix A.1 says their only deviation was how they trained it. But section 3.4 identifies the second model as a generalized additive model adapted from Horowitz (2016), not Yurko et al. (2019). (LLM, conf=0.92)
Experimental — accuracy not yet measuredYam and Lopez (2019) estimate that a better 4th down strategy would be worth about 0.4 wins per season on average.
“In addition, they estimate that a better strategy would be worth about 0.4 wins per season on average, a substantial amount comparable to the effect size reported by Pelechrinis et al. (2019) above.”
From Big ideas in sports analytics and statistical tools for their investigation (2023) · cites this paper as Yam and Lopez (2019)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210 · see citing works
Flagged first, then by citations
Research works printed but not linked: 13
- Ahmed, A., Perry, G.J., Fleg, J.L., Love, T.E., Goff, D.C., & Kitz- man, D.W., 2006, Outcomes in ambulatory chronic systolic and diastolic heart failure: A propensity score analysis, Amer- ican Heart Journal, 152, 956-966.
- Cochran, W.G., & Chambers, S.P., 1965, The planning of obser- vational studies of human populations, Journal of the Royal Statistical Society. Series A (General), 128, 234-266.
- Dehejia, R.H., & Wahba, S., 1999, Causal effects in nonexperimen- tal studies: Reevaluating the evaluation of training programs, Journal of the American statistical Association, 94, 1053- 1062.
- Genuer, R., Poggi, J.-M., & Tuleau, C., 2008, Random Forests: Some methodological insights, arXiv preprint arXiv:0811.3619. · arXiv
- Lock, D., & Nettleton, D., 2014, Using random forests to estimate win probability before each play of an NFL game, Journal of Quantitative Analysis in Sports, 10, 197-205.
- Lopez, M.J., & Gutman, R., 2017, Estimation of causal effects with multiple treatments: A review and new ideas, arXiv preprint arXiv:1701.05132. · arXiv
- Moskowitz, T., & Wertheim, L.J., 2012, Scorecasting: The Hidden Influences Behind How Sports Are Played and Games Are Won, New York, NY, USA: Three Rivers Press.
- Romer, D., 2006, Do firms maximize? Evidence from professional football, Journal of Political Economy, 114, 340-365.
- Rubin, D.B., 1974, Estimating causal effects of treatments in ran- domized and nonrandomized studies, Journal of Educational Psychology, 66, 688.
- Sekhon, J.S., 2008, Multivariate and propensity score matching software with automated balance optimization: The matching package for R, Journal of Statistical Software, Forthcoming.
- Stuart, E.A., 2010, Matching methods for causal inference: A review and a look forward, Statistical Science: A Review Journal of the Institute of Mathematical Statistics, 25, 1.
- Tversky, A., & Kahneman, D., 1991, Loss aversion in riskless choice: A reference-dependent model, The Quarterly Journal of Economics, 106, 1039-1061.
- Urschel, J.D., & Zhuang, J., 2011, Are NFL coaches risk and loss averse? Evidence from their use of kickoff strategies, Journal of Quantitative Analysis in Sports, 7.
Cited by
Sorted by citations
Big ideas in sports analytics and statistical tools for their investigation
Baumer, B. S., Matthews, G. J., & Nguyen, Q. (2023) · WIREs Computational Statistics · doi:10.1002/wics.1612 · Cited by 44 · see citing works
- Authors
- Yam, D. R., & Lopez, M. J.
- Affiliations
- Brown University (US)
- Skidmore College (US)
- Corresponding author
- Lopez, Michael J.
- Venue
- Journal of Sports Analytics
- Publisher
- SAGE Publications
- Published
- 2019-06-24
- Volume
- 5
- Issue
- 3
- Pages
- 153-167
- ISSN
- 2215-020X (ISSN-L)
- 2215-0218
- OpenAlex
- W2953855120
- Cited by
- 33
- References
- 1
- Citations by year
- 2026: 6 · 2025: 5 · 2024: 5 · 2023: 7 · 2022: 3 · 2021: 1 · 2020: 5 · 2019: 1
- FWCI
- 2.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Journal of Sports Analytics (publishedVersion, cc-by-nc)
- SSRN Electronic Journal (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Sports Analytics and Performance
- Health Sciences › Medicine › Orthopedics and Sports Medicine › Sports Performance and Training
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- National Football League, football coaches, aggressive strategy, winning probability, causal effect estimation, nearest neighbor matching, sports analytics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
