On the Existence of Simpler Machine Learning Models
Open accessThe DOI matches its Crossref record; 53 of the 55 references it cites were found in a registry or an index.
Abstract
It is almost always easier to find an accurate-but-complex model than an accurate-yet-simple model. Finding optimal, sparse, accurate models of various forms (linear models with integer coefficients, decision sets, rule lists, decision trees) is generally NP-hard. We often do not know whether the search for a simpler model will be worthwhile, and thus we do not go to the trouble of searching for one. In this work, we ask an important practical question: can accurate-yet-simple models be proven to exist, or shown likely to exist, before explicitly searching for them? We hypothesize that there is an important reason that simple-yet-accurate models often do exist. This hypothesis is that the size of the Rashomon set is often large, where the Rashomon set is the set of almost-equally-accurate models from a function class. If the Rashomon set is large, it contains numerous accurate models, and perhaps at least one of them is the simple model we desire. In this work, we formally present the Rashomon ratio as a new gauge of simplicity for a learning problem, depending on a function class and a data set. The Rashomon ratio is the ratio of the volume of the set of accurate models to the volume of the hypothesis space, and it is different from standard complexity measures from statistical learning theory. Insight from studying the Rashomon ratio provides an easy way to check whether a simpler model might exist for a problem before finding it, namely whether several different machine learning methods achieve similar performance on the data. In that sense, the Rashomon ratio is a powerful tool for understanding why and when an accurate-yet-simple model might exist. If, as we hypothesize in this work, many real-world data sets admit large Rashomon sets, the implications are vast: it means that simple or interpretable models may often be used for high-stakes decisions without losing accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“One recent theory suggests that when the prediction problem is heavily influenced by randomness (e.g., whether someone will commit a crime within 2 y could depend on any number of circumstances, and is a noisy process), there are many approximately equally predictive models, and in that case, it is likely that at least one of these models is interpretable (34).”
From Interpretable algorithmic forensics (2023) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
A Tutorial on Support Vector Machines for Pattern Recognition
Burges, C. J. C. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009715923555 · Cited by 16,429 · cited 3×
“Then the margin d is a sum of these distances d = d+ + d− [7].”
BackgroundInfluentialRudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“However, because simple models are essential for many high-stakes decisions (Rudin, 2019), perhaps we should return to the goal of aiming directly for simpler models.”
BackgroundSelf-citationProceedings of the 21st International Conference on Neural Information Processing Systems
Koller, D., Schuurmans, D., Bengio, Y., & Bottou, L. (2008) · Cited by 4,461
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · cited 3×
“We hypothesize that these three observations can all be explained by the same phenomenon: the “Rashomon effect,” which is the existence of many almost-equally-accurate models (Breiman et al., 2001).”
BackgroundMethodOn the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
Vapnik, V. N., & Chervonenkis, A. Y. (2015) · Measures of Complexity · doi:10.1007/978-3-319-21852-6_3 · Cited by 3,105 · cited 6×
“The VC dimension [49] shows the expressive power of a hypothesis space for any data set including an extreme arrangement of data points and labels.”
BackgroundInfluentialBoosting the margin: a new explanation for the effectiveness of voting methods
Bartlett, P., Freund, Y., Lee, W. S., & Schapire, R. E. (1998) · The Annals of Statistics · doi:10.1214/aos/1024691352 · Cited by 2,306
Meinshausen, N., & Bühlmann, P. (2010) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00740.x · Cited by 2,215 · cited 1×
“Rashomon sets: Rashomon sets have been used for various purposes (Breiman et al., 2001; Srebro et al., 2010; Fisher et al., 2019; Coker et al., 2021; Tulabandhula and Rudin, 2014b; Meinshausen and Bühlmann, 2010; Letham et al., 2016; Nevo and Ritov, 2017).”
Method
Research works printed but not linked: 9
- 10.1023/A:1022873112823 · doi:10.1023/a:1022873112823 · Paper page
- Olivier Bousquet and André Elisseeff. 2002. Stability and generalization. Journal of Machine Learning Research 2, Mar (2002), 499–526.
- Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. 2016. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836 (appeared at ICLR 2017) (2016). · arXiv
- 10.5555/2968618.2968674 · doi:10.5555/2968618.2968674
- 10.5555/3524938.3525566 · doi:10.5555/3524938.3525566
- Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. 2013. Learning with noisy labels. In Advances in Neural Information Processing Systems, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Vol. 26. Curran Associates, Inc.
- Lesia Semenova Cynthia Rudin and Ronald Parr. 2019. A study in Rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning. arXiv preprint arXiv:abs/1908.01755(2019).
- 10.5555/2567709.2567726 · doi:10.5555/2567709.2567726
- 10.5555/211359 · doi:10.5555/211359
Cited by
Explainable Artificial Intelligence by Genetic Programming: A Survey
Mei, Y., Chen, Q., Lensen, A., Xue, B., & Zhang, M. (2023) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2022.3225509 · Cited by 164
“This can be achieved when the different machine learning models perform similarly on the dataset (the Rashomon set is large [54]).”
MethodBock, K. W. D., Coussement, K., Caigny, A. D., Słowiński, R., Baesens, B., Boute, R. N., Choi, T., Delen, D., Kraus, M., Lessmann, S., et al. (2023) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.09.026 · Cited by 211
“According to Semenova, Rudin, & Parr (2022), they also provide simple, accurate models that can be superior to accurate, more complex models in terms of understandability.”
BackgroundWhy black box machine learning should be avoided for high-stakes decisions, in brief
Rudin, C. D. (2022) · Nature Reviews Methods Primers · doi:10.1038/s43586-022-00172-0 · Cited by 127
“A hypothesis for this remarkable phenomenon is the Rashomon set theory, that datasets tend to admit many accurate models 10 , among which are simpler, more interpretable models.”
BackgroundSelf-citationModel Multiplicity: Opportunities, Concerns, and Solutions
Black, E., Raghavan, M., & Barocas, S. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533149 · Cited by 85
Boosting: Empowering Citizens with Behavioral Science
Herzog, S. M., & Hertwig, R. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-020924-124753 · Cited by 92
Metadata
- Authors
- Semenova, L., Rudin, C., & Parr, R.
- Affiliations
- Duke University (US)
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 1827-1858
- OpenAlex
- W4283168572
- Cited by
- 90
- References
- 55
- Citations by year
- 2026: 23 · 2025: 23 · 2024: 22 · 2023: 19 · 2022: 2
- FWCI
- 9.12
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- Rashomon set, model simplicity, sparse models, interpretable models, hypothesis space, model complexity, decision tree, decision sets, NP-hardness, statistical learning theory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On the Existence of Simpler Machine Learning Models. https://app.cochat.ai/papers/10.1145/3531146.3533232
- The paper
- Semenova, L., Rudin, C., & Parr, R. (2022). On the Existence of Simpler Machine Learning Models. 2022 ACM Conference on Fairness Accountability and Transparency. https://doi.org/10.1145/3531146.3533232
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.