The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
An AUC-based permutation variable importance measure for random forests
Open accessAbstract
BACKGROUND: The random forest (RF) method is a commonly used tool for classification with high dimensional data as well as for ranking candidate predictors based on the so-called random forest variable importance measures (VIMs). However the classification performance of RF is known to be suboptimal in case of strongly unbalanced data, i.e. data where response class sizes differ considerably. Suggestions were made to obtain better classification performance based either on sampling procedures or on cost sensitivity analyses. However to our knowledge the performance of the VIMs has not yet been examined in the case of unbalanced response classes. In this paper we explore the performance of the permutation VIM for unbalanced data settings and introduce an alternative permutation VIM based on the area under the curve (AUC) that is expected to be more robust towards class imbalance. RESULTS: We investigated the performance of the standard permutation VIM and of our novel AUC-based permutation VIM for different class imbalance levels using simulated data and real data. The results suggest that the new AUC-based permutation VIM outperforms the standard permutation VIM for unbalanced data settings while both permutation VIMs have equal performance for balanced data settings. CONCLUSIONS: The standard permutation VIM loses its ability to discriminate between associated predictors and predictors not associated with the response for increasing class imbalance. It is outperformed by our new AUC-based permutation VIM for unbalanced data settings, while the performance of both VIMs is very similar in the case of balanced classes. The new AUC-based VIM is implemented in the R package party for the unbiased RF variant based on conditional inference trees. The codes implementing our study are available from the companion website: http://www.ibe.med.uni-muenchen.de/organisation/mitarbeiter/070_drittmittel/janitza/index.html.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Data mining and knowledge discovery: making sense out of data
Feyyad, U. M. (1996) · IEEE Expert · doi:10.1109/64.539013 · Cited by 4,612
A study of the behavior of several methods for balancing machine learning training data
Batista, G. E. A. P. A., Prati, R. C., & Monard, M. C. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007735 · Cited by 4,203
Unbiased Recursive Partitioning: A Conditional Inference Framework
Hothorn, T., Hornik, K., & Zeileis, A. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x133933 · Cited by 4,021
Bias in random forest variable importance measures: Illustrations, sources and a solution
Strobl, C., Boulesteix, A.-L., Zeileis, A., & Hothorn, T. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-25 · Cited by 3,688
Self-citationThe class imbalance problem: A systematic study1
Japkowicz, N., & Stephen, S. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6504 · Cited by 3,295
The Statistical Evaluation of Medical Tests for Classification and Prediction
Pepe, M. S. (2003) · doi:10.1093/oso/9780198509844.001.0001 · Cited by 2,198
The Statistical Evaluation of Medical Tests for Classification and Prediction
(2005) · Technometrics · doi:10.1198/tech.2005.s278 · Cited by 1,899
Machine Learning for the Detection of Oil Spills in Satellite Radar Images
Kubat, M., Holte, R. C., & Matwin, S. (1998) · Machine Learning · doi:10.1023/a:1007452223027 · Cited by 1,243
A Multiple Resampling Method for Learning from Imbalanced Data Sets
Estabrooks, A., Jo, T., & Japkowicz, N. (2004) · Computational Intelligence · doi:10.1111/j.0824-7935.2004.t01-1-00228.x · Cited by 1,014
Cited by
Predictors of public climate change awareness and risk perception around the world
Lee, T. M., Markowitz, E. M., Howe, P. D., Ko, C.-Y., & Leiserowitz, A. A. (2015) · Nature Climate Change · doi:10.1038/nclimate2728 · Cited by 1,356
Variable importance analysis: A comprehensive review
Wei, P., Lu, Z., & Song, J. (2015) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2015.05.018 · Cited by 525
Lydersen, J. M., North, M. P., & Collins, B. M. (2014) · Forest Ecology and Management · doi:10.1016/j.foreco.2014.06.005 · Cited by 161
A peek into the black box: exploring classifiers by randomization
Henelius, A., Puolamäki, K., Boström, H., Asker, L., & Papapetrou, P. (2014) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0368-8 · Cited by 98
Wolfe, M. B., Schagat, T. L., Paulsen, M. T., Magnuson, B., Ljungman, M., Park, D., Zhang, C., Campbell, Z. T., Goldstrohm, A. C., & Freddolino, P. L. (2020) · RNA · doi:10.1261/rna.077362.120 · Cited by 22
Metadata
- Authors
- Janitza, S., Strobl, C., & Boulesteix, A.-L.
- Venue
- BMC Bioinformatics
- Published
- 2013-04-05
- Volume
- 14
- Issue
- 1
- Pages
- 119-119
- ISSN
- 1471-2105 (ISSN-L)
- OpenAlex
- W2120153268
- Cited by
- 263
- References
- 31
- FWCI
- 14.36
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An AUC-based permutation variable importance measure for random forests. https://app.cochat.ai/papers/10.1186/1471-2105-14-119
- The paper
- Janitza, S., Strobl, C., & Boulesteix, A.-L. (2013). An AUC-based permutation variable importance measure for random forests. BMC Bioinformatics. https://doi.org/10.1186/1471-2105-14-119
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.