On the Existence of Simpler Machine Learning Models
Open accessThe DOI matches its Crossref record; 53 of the 55 references it cites were found in a registry or an index.
Abstract
It is almost always easier to find an accurate-but-complex model than an accurate-yet-simple model. Finding optimal, sparse, accurate models of various forms (linear models with integer coefficients, decision sets, rule lists, decision trees) is generally NP-hard. We often do not know whether the search for a simpler model will be worthwhile, and thus we do not go to the trouble of searching for one. In this work, we ask an important practical question: can accurate-yet-simple models be proven to exist, or shown likely to exist, before explicitly searching for them? We hypothesize that there is an important reason that simple-yet-accurate models often do exist. This hypothesis is that the size of the Rashomon set is often large, where the Rashomon set is the set of almost-equally-accurate models from a function class. If the Rashomon set is large, it contains numerous accurate models, and perhaps at least one of them is the simple model we desire. In this work, we formally present the Rashomon ratio as a new gauge of simplicity for a learning problem, depending on a function class and a data set. The Rashomon ratio is the ratio of the volume of the set of accurate models to the volume of the hypothesis space, and it is different from standard complexity measures from statistical learning theory. Insight from studying the Rashomon ratio provides an easy way to check whether a simpler model might exist for a problem before finding it, namely whether several different machine learning methods achieve similar performance on the data. In that sense, the Rashomon ratio is a powerful tool for understanding why and when an accurate-yet-simple model might exist. If, as we hypothesize in this work, many real-world data sets admit large Rashomon sets, the implications are vast: it means that simple or interpretable models may often be used for high-stakes decisions without losing accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“One recent theory suggests that when the prediction problem is heavily influenced by randomness (e.g., whether someone will commit a crime within 2 y could depend on any number of circumstances, and is a noisy process), there are many approximately equally predictive models, and in that case, it is likely that at least one of these models is interpretable (34).”
From Interpretable algorithmic forensics (2023) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
A Tutorial on Support Vector Machines for Pattern Recognition
Burges, C. J. C. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009715923555 · Cited by 16,429 · cited 3×
“Then the margin d is a sum of these distances d = d+ + d− [7].”
BackgroundInfluentialRudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“However, because simple models are essential for many high-stakes decisions (Rudin, 2019), perhaps we should return to the goal of aiming directly for simpler models.”
BackgroundSelf-citationProceedings of the 21st International Conference on Neural Information Processing Systems
Koller, D., Schuurmans, D., Bengio, Y., & Bottou, L. (2008) · Cited by 4,461
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · cited 3×
“We hypothesize that these three observations can all be explained by the same phenomenon: the “Rashomon effect,” which is the existence of many almost-equally-accurate models (Breiman et al., 2001).”
BackgroundMethodOn the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
Vapnik, V. N., & Chervonenkis, A. Y. (2015) · Measures of Complexity · doi:10.1007/978-3-319-21852-6_3 · Cited by 3,105 · cited 6×
“The VC dimension [49] shows the expressive power of a hypothesis space for any data set including an extreme arrangement of data points and labels.”
BackgroundInfluentialBoosting the margin: a new explanation for the effectiveness of voting methods
Bartlett, P., Freund, Y., Lee, W. S., & Schapire, R. E. (1998) · The Annals of Statistics · doi:10.1214/aos/1024691352 · Cited by 2,306
Meinshausen, N., & Bühlmann, P. (2010) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00740.x · Cited by 2,215 · cited 1×
“Rashomon sets: Rashomon sets have been used for various purposes (Breiman et al., 2001; Srebro et al., 2010; Fisher et al., 2019; Coker et al., 2021; Tulabandhula and Rudin, 2014b; Meinshausen and Bühlmann, 2010; Letham et al., 2016; Nevo and Ritov, 2017).”
MethodRademacher and Gaussian Complexities: Risk Bounds and Structural Results
Bartlett, P. L., & Mendelson, S. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_15 · Cited by 1,949 · cited 1×
“The generalization bound follows Bartlett and Mendelson (2002).”
BackgroundReconciling modern machine-learning practice and the classical bias–variance trade-off
Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1903070116 · Cited by 1,595 · cited 3×
“We even build neural networks that are so complex that they can achieve zero training error, and try afterwards to determine why they generalize Belkin et al. (2019); Nakkiran et al. (2021).”
BackgroundFisher, A., Rudin, C., & Dominici, F. (2018) · arXiv · doi:10.48550/arxiv.1801.01489 · Cited by 1,165
Self-citationAn Information Measure for Classification
Wallace, C. S., & Boulton, D. M. (1968) · The Computer Journal · doi:10.1093/comjnl/11.2.185 · Cited by 933 · cited 2×
“Based on a minimum-message-length argument (Wallace and Boulton, 1968), several works claim that flat loss functions lead to better generalization due to a robustness to noise around the minimum (Hochreiter and Schmidhuber, 1997; Keskar et al., 2016; Chaudhari et al., 2019).”
BackgroundDeVore, R. A. (1998) · Acta Numerica · doi:10.1017/s0962492900002816 · Cited by 877 · cited 2×
“f ∈ Lp (I ), where I ⊂ Rd is a cube in Rd , ∥ · ∥W r (Lp (I ))—Sobolev semi norm Pr—space of polynomials of order r in d , constant C depends on r infp∈Pr ∥f − p ∥Lp (I ) ≤ C |I |r /d |f |W r (Lp (I )) DeVore [15]”
Angluin, D., & Laird, P. (1988) · Machine Learning · doi:10.1007/bf00116829 · Cited by 827 · cited 3×
“REFERENCES [1] Dana Angluin and Philip Laird.”
BackgroundHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.1.1 · Cited by 676 · cited 3×
“Based on a minimum-message-length argument (Wallace and Boulton, 1968), several works claim that flat loss functions lead to better generalization due to a robustness to noise around the minimum (Hochreiter and Schmidhuber, 1997; Keskar et al., 2016; Chaudhari et al., 2019).”
BackgroundMethodDeep double descent: where bigger models and more data hurt*
Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., & Sutskever, I. (2021) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac3a74 · Cited by 565 · cited 1×
“We even build neural networks that are so complex that they can achieve zero training error, and try afterwards to determine why they generalize Belkin et al. (2019); Nakkiran et al. (2021).”
BackgroundAdvanced Lectures on Machine Learning
(2004) · Lecture Notes in Computer Science · doi:10.1007/b100712 · Cited by 562
Bartlett, P. L., Bousquet, O., & Mendelson, S. (2002) · Cited by 533
Empirical Margin Distributions and Bounding the Generalization Error of Combined Classifiers
Koltchinskii, V., & Panchenko, D. (2002) · The Annals of Statistics · doi:10.1214/aos/1015362183 · Cited by 451 · cited 3×
“…(Srebro et al., 2010; Kakade et al., 2008) and Gaussian complexities (Kakade et al., 2008), PAC-Bayes theorems (Langford and Shawe-Taylor, 2002), covering numbers bounds (Zhou, 2002), and margin bounds (Vapnik and Chervonenkis, 1971; Schapire et al., 1998; Koltchinskii and Panchenko, 2002).”
BackgroundAlgorithmic Stability and Sanity-Check Bounds for Leave-One-Out Cross-Validation
Kearns, M., & Ron, D. (1999) · Neural Computation · doi:10.1162/089976699300016304 · Cited by 407
Underspecification Presents Challenges for Credibility in Modern Machine Learning
D'Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. (2020) · arXiv · doi:10.48550/arxiv.2011.03395 · Cited by 398 · cited 5×
“[13] Alexander D’Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, et al.”
BackgroundInfluentialBartlett, P. L., Bousquet, O., & Mendelson, S. (2005) · The Annals of Statistics · doi:10.1214/009053605000000282 · Cited by 372 · cited 6×
“…(as the Rashomon ratio does not rely on robustness of an algorithm with respect to changes in the data), it differs from local Rademacher complexity (Bartlett et al., 2005) (as the Rashomon ratio does not measure the ability of the hypothesis space to handle random changes in targets and actually…”
BackgroundMethodInfluentialThe covering number in learning theory
Zhou, D.-X. (2002) · Journal of Complexity · doi:10.1006/jcom.2002.0635 · Cited by 316 · cited 1×
“…(Srebro et al., 2010; Kakade et al., 2008) and Gaussian complexities (Kakade et al., 2008), PAC-Bayes theorems (Langford and Shawe-Taylor, 2002), covering numbers bounds (Zhou, 2002), and margin bounds (Vapnik and Chervonenkis, 1971; Schapire et al., 1998; Koltchinskii and Panchenko, 2002).”
BackgroundSupersparse linear integer models for optimized medical scoring systems
Ustun, B., & Rudin, C. (2016) · Machine Learning · doi:10.1007/s10994-015-5528-6 · Cited by 313 · cited 1×
“…and a more complex space, consider F 2 to be the space of linear models with real-valued coefficients in a space of d dimensions, and consider F 1 to be the space of scoring systems (Ustun and Rudin, 2016), which are sparse linear models, with at most d (cid:48) nonzero integer coefficients, d d .”
BackgroundSelf-citationEntropy-SGD: biasing gradient descent into wide valleys*
Chaudhari, P., Choromanska, A., Soatto, S., LeCun, Y., Baldassi, C., Borgs, C., Chayes, J., Sagun, L., & Zecchina, R. (2019) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ab39d9 · Cited by 291 · cited 2×
“Based on a minimum-message-length argument (Wallace and Boulton, 1968), several works claim that flat loss functions lead to better generalization due to a robustness to noise around the minimum (Hochreiter and Schmidhuber, 1997; Keskar et al., 2016; Chaudhari et al., 2019).”
BackgroundNoise Tolerance Under Risk Minimization
Manwani, N., & Sastry, P. S. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2223460 · Cited by 258 · cited 2×
“As this calculation holds for every model f from the true Rashomon set This proof uses similar technique as Theorem 1 in the work of (Manwani and Sastry, 2013), where the purpose is to show noise tolerance of the optimal model.”
MethodInterpretable classification models for recidivism prediction
Zeng, J., Ustun, B., & Rudin, C. (2017) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/rssa.12227 · Cited by 220
Self-citationThe age of secrecy and unfairness in recidivism prediction
Rudin, C., Wang, C., & Coker, B. (2020) · Harvard Data Science Review · doi:10.1162/99608f92.6ed64b30 · Cited by 177
Self-citationA Few Notes on Statistical Learning Theory
Mendelson, S. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-36434-x_1 · Cited by 161 · cited 2×
“The use of approximating sets, as used extensively in this paper, is used throughout the literature on learning theory (Lecué, 2011; Lugosi and Wegkamp, 2004; Schapire et al., 1998; Mendelson, 2003).”
MethodOn the degree of polynomials that approximate symmetric Boolean functions (preliminary version)
Paturi, R. (1992) · Proceedings of the twenty-fourth annual ACM symposium on Theory of computing - STOC '92 · doi:10.1145/129712.129758 · Cited by 152 · cited 1×
“,xn P (d )—algebraic polynomials of degree d ∥f − P (d ) ∥∞ ≤ O( √ n(n − Γ(f ))) Paturi [38]”
A Finite Sample Distribution-Free Performance Bound for Local Discrimination Rules
Rogers, W. H., & Wagner, T. J. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344196 · Cited by 132 · cited 2×
“[39] William H Rogers and Terry J Wagner.”
BackgroundSharp Minima Can Generalize For Deep Nets
Dinh, L., Pascanu, R., Bengio, S., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04933 · Cited by 125 · cited 9×
“Flat minima or wide valleys: The concept of flat minima (wide valleys) has been explored in the deep learning literature as a possible way to understand convergence properties of the complicated, non-convex loss functions that deep networks traverse during training [9, 16, 20, 23].”
BackgroundMethodInfluentialExploring the cloud of variable importance for the set of all good models
Dong, J., & Rudin, C. D. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00264-0 · Cited by 113 · cited 1×
“For example, let us consider criminal recidivism data, whose Rashomon sets have been studied Fisher et al. (2019); Dong and Rudin (2020) and that admit simple-yet-accurate models Zeng et al. (2017); Rudin et al. (2020).”
BackgroundSelf-citationThe estimate for approximation error of neural networks: A constructive approach
Cao, F., Xie, T., & Xu, Z. (2007) · Neurocomputing · doi:10.1016/j.neucom.2007.07.024 · Cited by 90 · cited 1×
“[8] Feilong Cao, Tingfan Xie, and Zongben Xu.”
BackgroundComplexity Regularization via Localized Random PenaltiesPreprint
Lugosi, G., & Wegkamp, M. H. (2002) · Cited by 66
Robust Optimization using Machine Learning for Uncertainty SetsPreprint
Tulabandhula, T., & Rudin, C. D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.1097 · Cited by 62 · cited 2×
“The empirical Rashomon set (or simply Rashomon set) is a subset of models of the hypothesis space F that have training performance close to the best model in the class, according to a loss function (Breiman et al., 2001; Srebro et al., 2010; Fisher et al., 2019; Coker et al., 2021; Tulabandhula and Rudin, 2014b).”
BackgroundMethodSelf-citationComplexity regularization via localized random penalties
Lugosi, G., & Wegkamp, M. (2004) · The Annals of Statistics · doi:10.1214/009053604000000463 · Cited by 55 · cited 1×
“The use of approximating sets, as used extensively in this paper, is used throughout the literature on learning theory (Lecué, 2011; Lugosi and Wegkamp, 2004; Schapire et al., 1998; Mendelson, 2003).”
MethodSmoothness, Low Noise and Fast RatesPreprint
Srebro, N., Sridharan, K., & Tewari, A. (2010) · arXiv (Cornell University) · Cited by 52
A Theory of Statistical Inference for Ensuring the Robustness of Scientific Results
Coker, B., Rudin, C. D., & King, G. (2021) · Management Science · doi:10.1287/mnsc.2020.3818 · Cited by 42 · cited 4×
“The empirical Rashomon set (or simply Rashomon set) is a subset of models of the hypothesis space F that have training performance close to the best model in the class, according to a loss function (Breiman et al., 2001; Srebro et al., 2010; Fisher et al., 2019; Coker et al., 2021; Tulabandhula and Rudin, 2014b).”
BackgroundMethodInfluentialSelf-citationApproximation of monomials by lower degree polynomials
Newman, D. J., & Rivlin, T. J. (1976) · Aequationes Mathematicae · doi:10.1007/bf01835995 · Cited by 35 · cited 1×
“f ∈ {xk , k ∈ N } P (n)—polynomials of degree at most n ∈ N ∥f − P (n) ∥∞ ≤ 1 2k−1 ∑ j>(n+k )/2 (k j ) Newman and Rivlin [37] f ∈ C[0, 1] is a non-constant symmetric boolean function on x1,.”
On combining machine learning with decision making
Tulabandhula, T., & Rudin, C. D. (2014) · Machine Learning · doi:10.1007/s10994-014-5459-7 · Cited by 32 · cited 9×
“…set (or simply Rashomon set ) is a subset of models of the hypothesis space F that have training performance close to the best model in the class, according to a loss function (Breiman et al., 2001; Srebro et al., 2010; Fisher et al., 2019; Coker et al., 2021; Tulabandhula and Rudin, 2014b).”
BackgroundMethodInfluentialSelf-citationPredictive Multiplicity in Classification
Marx, C. T., Calmon, F. P., & Ustun, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06677 · Cited by 28 · cited 3×
“[31] Charles Marx, Flavio Calmon, and Berk Ustun.”
BackgroundResultLecué, G. (2011) · HAL · Cited by 20
Algorithms and Error Bounds for Multivariate Piecewise Constant Approximation
Davydov, O. (2010) · Springer proceedings in mathematics · doi:10.1007/978-3-642-16876-5_2 · Cited by 13 · cited 1×
“F2 F1 δ (depends on parameters in bounds below) Source f ∈ L∞(Ω), ∥f ∥∞ ∈ [m, M ] sN ∈ S (Ω), sN—piecewise constant, N—number of constants ∥f − sN ∥∞ ≤ M−m 2N Davydov [14], DeVore [15] f ∈W 1 p (Ω), 1 ≤ p ≤ ∞, whereW 1 p is a Sobolev space s∆(f ) ∈ S (Ω), s∆—piecewise constant, ∆—fixed partition, Ω = (0, 1)d , N—numbe…”
Detecting Underspecification with Local EnsemblesPreprint
Madras, D., Atwood, J., & D’Amour, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09573 · Cited by 13 · cited 1×
“Madras et al. (2019) proposed a post-hoc local-ensemble method that measures underspecification for a given test datum.”
MethodCharacterizing Fairness Over the Set of Good Models Under Selective Labels
Coston, A., Rambachan, A., & Chouldechova, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00352 · Cited by 13 · cited 3×
“[12] investigates predictive disparities for algorithmic fairness.”
BackgroundIdentifying a minimal class of models for high-dimensional dataPreprint
Nevo, D., & Ritov, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00494 · Cited by 10 · cited 2×
“Rashomon sets: Rashomon sets have been used for various purposes [6, 10, 19, 27, 32, 36, 44, 47].”
MethodMachine Learning with Operational CostsPreprint
Tulabandhula, T., & Rudin, C. D. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1112.0698 · Cited by 3 · cited 8×
“For a useful example of a simple and a more complex space, consider F 2 to be the space of linear models with real-valued coefficients in a space of d dimensions, and consider F 1 to be the space of scoring systems (Ustun and Rudin, 2016), which are sparse linear models, with at most d (cid:48) nonzero integer coefficient…”
MethodInfluentialSelf-citationIdentifying a minimal class of models for high-dimensional data
NevoDaniel, & RitovYa'acov (2017) · Journal of Machine Learning Research · doi:10.5555/3122009.3122033 · Cited by 3
Prediction uncertainty and optimal experimental design for learning dynamical systems
Rudin, C. D., Letham, B., Browne, E. P., Letham, P. A., Browne, E. P., Letham, B., Rudin, C. D., & Letham, P. A. (2020) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/azht-0z93 · Cited by 2
Self-citation(2008) · ScholarlyCommons (University of Pennsylvania)
Research works printed but not linked: 9
- 10.1023/A:1022873112823 · doi:10.1023/a:1022873112823 · Paper page
- Olivier Bousquet and André Elisseeff. 2002. Stability and generalization. Journal of Machine Learning Research 2, Mar (2002), 499–526.
- Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. 2016. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836 (appeared at ICLR 2017) (2016). · arXiv
- 10.5555/2968618.2968674 · doi:10.5555/2968618.2968674
- 10.5555/3524938.3525566 · doi:10.5555/3524938.3525566
- Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. 2013. Learning with noisy labels. In Advances in Neural Information Processing Systems, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Vol. 26. Curran Associates, Inc.
- Lesia Semenova Cynthia Rudin and Ronald Parr. 2019. A study in Rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning. arXiv preprint arXiv:abs/1908.01755(2019).
- 10.5555/2567709.2567726 · doi:10.5555/2567709.2567726
- 10.5555/211359 · doi:10.5555/211359
Cited by
Explainable Artificial Intelligence by Genetic Programming: A Survey
Mei, Y., Chen, Q., Lensen, A., Xue, B., & Zhang, M. (2023) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2022.3225509 · Cited by 164
“This can be achieved when the different machine learning models perform similarly on the dataset (the Rashomon set is large [54]).”
MethodBock, K. W. D., Coussement, K., Caigny, A. D., Słowiński, R., Baesens, B., Boute, R. N., Choi, T., Delen, D., Kraus, M., Lessmann, S., et al. (2023) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.09.026 · Cited by 211
“According to Semenova, Rudin, & Parr (2022), they also provide simple, accurate models that can be superior to accurate, more complex models in terms of understandability.”
BackgroundWhy black box machine learning should be avoided for high-stakes decisions, in brief
Rudin, C. D. (2022) · Nature Reviews Methods Primers · doi:10.1038/s43586-022-00172-0 · Cited by 127
“A hypothesis for this remarkable phenomenon is the Rashomon set theory, that datasets tend to admit many accurate models 10 , among which are simpler, more interpretable models.”
BackgroundSelf-citationModel Multiplicity: Opportunities, Concerns, and Solutions
Black, E., Raghavan, M., & Barocas, S. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533149 · Cited by 85
Boosting: Empowering Citizens with Behavioral Science
Herzog, S. M., & Hertwig, R. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-020924-124753 · Cited by 92
Metadata
- Authors
- Semenova, L., Rudin, C., & Parr, R.
- Affiliations
- Duke University (US)
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 1827-1858
- OpenAlex
- W4283168572
- Cited by
- 90
- References
- 55
- Citations by year
- 2026: 23 · 2025: 23 · 2024: 22 · 2023: 19 · 2022: 2
- FWCI
- 9.12
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- Rashomon set, model simplicity, sparse models, interpretable models, hypothesis space, model complexity, decision tree, decision sets, NP-hardness, statistical learning theory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.