Bringing Transparency to Predictive Analytics: A Systematic Comparison of Predictive Modeling Methods in Higher Education
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Colleges have increasingly turned to predictive analytics to target at-risk students for additional support. Most of the predictive analytic applications in higher education are proprietary, with private companies offering little transparency about their underlying models. We address this lack of transparency by systematically comparing two important dimensions: (1) different approaches to sample and variable construction and how these affect model accuracy and (2) how the selection of predictive modeling approaches, ranging from methods many institutional researchers would be familiar with to more complex machine learning methods, affects model performance and the stability of predicted scores. The relative ranking of students’ predicted probability of completing college varies substantially across modeling approaches. While we observe substantial gains in performance from models trained on a sample structured to represent the typical enrollment spells of students and with a robust set of predictors, we observe similar performance between the simplest and the most complex models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Stereotype threat and the intellectual test performance of African Americans.
Steele, C. M., & Aronson, J. (1995) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.69.5.797 · Cited by 9,020
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
A Brief Social-Belonging Intervention Improves Academic and Health Outcomes of Minority Students
Walton, G. M., & Cohen, G. L. (2011) · Science · doi:10.1126/science.1198364 · Cited by 1,992 · cited 4×
“Furthermore, biased models can lead institutions to intervene disproportionately with students from underrep-resented backgrounds and may reinforce existing psychological barriers that students encounter, including feelings of social isolation and anxiety (Walton & Cohen, 2011).”
BackgroundInfluentialMitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
Can You Recognize an Effective Teacher When You Recruit One?
Rockoff, J. E., Jacob, B. A., Kane, T. J., & Staiger, D. O. (2011) · Education Finance and Policy · doi:10.1162/edfp_a_00022 · Cited by 390
The Impact of Price Caps and Spending Cuts on U.S. Postsecondary Attainment
Deming, D., & Walters, C. (2017) · doi:10.3386/w23736 · Cited by 181 · cited 1×
“…pressure, at broad-access institutions attended by most undergraduates, the level of resources available to invest in completion strategies has declined considerably over time as states have reduced their appropria-tions to public higher education (Deming & Walters, 2017; Ma et al., 2017).”
BackgroundHerzog, S. (2006) · New Directions for Institutional Research · doi:10.1002/ir.185 · Cited by 138 · cited 1×
“…social and computational sciences to predict the probability of credential attainment within 6 years (Attewell & Monaghan, 2015; Hand et al., 2001; Herzog, 2006): OLS, logistic regression, Cox proportional hazard (CPH) survival analysis, random forest, gradient boosted machines (XGBoost), and…”
Background
Cited by
Rane, N. L. (2026) · Journal of Artificial Intelligence and Robotics · doi:10.61577/jaiar.2024.100006 · Cited by 278
Gándara, D., Anahideh, H., Ison, M. P., & Picchiarini, L. (2024) · AERA Open · doi:10.1177/23328584241258741 · Cited by 67
Global justice and the use of AI in education: ethical and epistemic aspects
Vučković, A., & Sikimić, V. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02076-x · Cited by 22
“…that is concerned with the educational institution as a whole (e.g., to predict which courses are going to be the most beneficial for students in their future careers) and to make predictions regarding individual students (e.g., to identify those who are at risk of dropping out) (Bird et al. 2021).”
Metadata
- Authors
- Bird, K. A., Castleman, B. L., Mabel, Z., & Song, Y.
- Venue
- AERA Open
- Publisher
- SAGE Publications
- Published
- 2021-01
- Volume
- 7
- ISSN
- 2332-8584 (ISSN-L)
- OpenAlex
- W3184767488
- Cited by
- 46
- References
- 7
- FWCI
- 5.73
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Bringing Transparency to Predictive Analytics: A Systematic Comparison of Predictive Modeling Methods in Higher Education. https://app.cochat.ai/papers/10.1177/23328584211037630
- The paper
- Bird, K. A., Castleman, B. L., Mabel, Z., & Song, Y. (2021). Bringing Transparency to Predictive Analytics: A Systematic Comparison of Predictive Modeling Methods in Higher Education. AERA Open. https://doi.org/10.1177/23328584211037630
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.