The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
A general approach for predicting the behavior of the Supreme Court of the United States
Open accessAbstract
Building on developments in machine learning and prior work in the science of judicial prediction, we construct a model designed to predict the behavior of the Supreme Court of the United States in a generalized, out-of-sample context. To do so, we develop a time-evolving random forest classifier that leverages unique feature engineering to predict more than 240,000 justice votes and 28,000 cases outcomes over nearly two centuries (1816-2015). Using only data available prior to decision, our model outperforms null (baseline) models at both the justice and case level under both parametric and non-parametric tests. Over nearly two centuries, we achieve 70.2% accuracy at the case outcome level and 71.9% at the justice vote level. More recently, over the past century, we outperform an in-sample optimized null model by nearly 5%. Our performance is consistent with, and improves on the general level of prediction demonstrated by prior work; however, our model is distinctive because it can be applied out-of-sample to the entire past and future of the Court, not a single term. Our results represent an important advance for the science of quantitative legal prediction and portend a range of other potential applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
Classification And Regression Trees
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017) · doi:10.1201/9781315139470 · Cited by 11,417
An empirical comparison of supervised learning algorithms
Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143865 · Cited by 2,780
Online Learning and Online Convex Optimization
Shalev-Shwartz, S. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000018 · Cited by 1,246
The Supreme Court and the Attitudinal Model Revisited
Segal, J. A., & Spaeth, H. J. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511615696 · Cited by 1,165
Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999
Martin, A. D., & Quinn, K. M. (2002) · Political Analysis · doi:10.1093/pan/10.2.134 · Cited by 1,144
Online Learning and Online Convex Optimization
Shalev-Shwartz, S. (2011) · now publishers, Inc. eBooks · doi:10.1561/9781601985477 · Cited by 1,095
The Supreme Court and the attitudinal model revisited
(2003) · Choice Reviews Online · doi:10.5860/choice.40-6706 · Cited by 857
Aletras, N., Tsarapatsanis, D., Preoţiuc-Pietro, D., & Lampos, V. (2016) · PeerJ Computer Science · doi:10.7717/peerj-cs.93 · Cited by 810
How Public Opinion Constrains the U.S. Supreme Court
Casillas, C. J., Enns, P. K., & Wohlfarth, P. C. (2011) · American Journal of Political Science · doi:10.1111/j.1540-5907.2010.00485.x · Cited by 281
On-Line Learning in Neural Networks
MTW, & Saad, D. (2000) · Journal of the American Statistical Association · doi:10.2307/2669811 · Cited by 260
Ruger, T. W., Kim, P. T., Martin, A. D., & Quinn, K. M. (2004) · Columbia Law Review · doi:10.2307/4099370 · Cited by 200
The Influence of Stare Decisis on the Votes of United States Supreme Court Justices
Segal, J. A., & Spaeth, H. J. (1996) · American Journal of Political Science · doi:10.2307/2111738 · Cited by 199
Surden, H. (2014) · Colorado Law Scholarly Commons (University of Colorado Colorado Springs) · Cited by 166
Automatically classifying case texts and predicting outcomes
Ashley, K. D., & Brüninghaus, S. (2009) · Artificial Intelligence and Law · doi:10.1007/s10506-009-9077-9 · Cited by 158
Competing Approaches to Predicting Supreme Court Decision Making
Martin, A. D., Quinn, K. M., Ruger, T. W., & Kim, P. T. (2004) · Perspectives on Politics · doi:10.1017/s1537592704040502 · Cited by 157
Evaluation of machine-learning protocols for technology-assisted review in electronic discovery
Cormack, G. V., & Grossman, M. R. (2014) · Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval · doi:10.1145/2600428.2609601 · Cited by 149
Large-scale structure of time evolving citation networks
Leicht, E. A., Clarkson, G., Shedden, K., & Newman, M. E. J. (2007) · The European Physical Journal B · doi:10.1140/epjb/e2007-00271-7 · Cited by 148
On-Line Learning in Neural Networks
Saad, D. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9780511569920 · Cited by 140
Katz, D. M. (2012) · SSRN Electronic Journal · Cited by 130
Self-citationDoes Legal Doctrine Matter? Unpacking Law and Policy Preferences on the U.S. Supreme Court
Bailey, M. A., & Maltzman, F. (2008) · American Political Science Review · doi:10.2139/ssrn.1121547 · Cited by 130
Of Time and Consensual Norms in the Supreme Court
Calderia, G. A., & Zorn, C. (1998) · American Journal of Political Science · doi:10.2307/2991733 · Cited by 114
Assessing Preference Change on the US Supreme Court
Martin, A. D., & Quinn, K. M. (2007) · The Journal of Law Economics and Organization · doi:10.1093/jleo/ewm028 · Cited by 83
Statistical Mechanics of the US Supreme Court
Lee, E. D., Broedersz, C. P., & Bialek, W. (2015) · Journal of Statistical Physics · doi:10.1007/s10955-015-1253-6 · Cited by 82
Justice Blocks and Predictability of U.S. Supreme Court Votes
Guimerà, R., & Sales‐Pardo, M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027188 · Cited by 79
McShane, B. B., Watson, O. P., Baker, T., & Griffith, S. J. (2012) · Journal of Empirical Legal Studies · doi:10.1111/j.1740-1461.2012.01260.x · Cited by 31
Choice of Law: An Empirical Analysis
Sanga, S. (2014) · Journal of Empirical Legal Studies · doi:10.1111/jels.12059 · Cited by 25
Law on the Market? Evaluating the Securities Market Impact of Supreme Court DecisionsPreprint
Katz, D. M., Bommarito, M. J., Soellinger, T., & Chen, J. M. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2649726 · Cited by 9
Self-citation
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Using machine learning to predict decisions of the European Court of Human Rights
Medvedeva, M., Vols, M., & Wieling, M. (2020) · Artificial Intelligence and Law · doi:10.1007/s10506-019-09255-y · Cited by 429
Legal Judgment Prediction via Topological Learning
Zhong, H., Guo, Z., Tu, C., Xiao, C., Liu, Z., & Sun, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1390 · Cited by 344
Iteratively Questioning and Answering for Interpretable Legal Judgment Prediction
Zhong, H., Wang, Y., Tu, C., Zhang, T., Liu, Z., & Sun, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5479 · Cited by 113
LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
Chalkidis, I., Jana, A., Hartung, D., Bommarito, M. J., Androutsopoulos, I., Katz, D. M., & Aletras, N. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3936759 · Cited by 111
Self-citation
Metadata
- Authors
- Katz, D. M., Bommarito, M. J., & Blackman, J.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-04-12
- Volume
- 12
- Issue
- 4
- Pages
- e0174698
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W3122334902
- Cited by
- 528
- References
- 31
- FWCI
- 206.93
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.