Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity
The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Modern machine learning increasingly supports paradigms that are multi-institutional (using data from multiple institutions during training) or cross-institutional (using models from multiple institutions for inference), but the empirical effects of these paradigms are not well understood. This study investigates cross-institutional learning via an empirical case study in higher education. We propose a framework and metrics for assessing the utility and fairness of student dropout prediction models that are transferred across institutions. We examine the feasibility of cross-institutional transfer under real-world data- and model-sharing constraints, quantifying model biases for intersectional student identities, characterizing potential disparate impact due to these biases, and investigating the impact of various cross-institutional ensembling approaches on fairness and overall model performance. We perform this analysis on data representing over 200,000 enrolled students annually from four universities without sharing training data between institutions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The large-scale collection of learner data raises concerns regarding ethics, privacy, fairness, and trustworthiness [7,22].”
From Federated Learning Analytics: Investigating the Privacy-Performance Trade-Off in Machine Learning for Educational Analytics (2024) · cites this paper as [7]
In the citing paper
“A recent study in the educational field investigated a transfer learning approach and voiced concerns regarding the relevance of decentralised approaches for education [7].”
From Federated Learning Analytics: Investigating the Privacy-Performance Trade-Off in Machine Learning for Educational Analytics (2024) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Crenshaw, K. (2018) · Living with Contradictions · doi:10.4324/9780429499142-5 · Cited by 7,232
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Sheller, M. J., Edwards, B., Reina, G. A., Martin, J., Pati, S., Kotrotsou, A., Milchenko, M., Xu, W., Marcus, D., Colen, R. R., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-69250-1 · Cited by 1,646 · cited 4×
“[67] Micah J Sheller, Brandon Edwards, G Anthony Reina, Jason Martin, Sarthak Pati, Aikaterini Kotrotsou, Mikhail Milchenko, Weilin Xu, Daniel Marcus, Rivka R Colen, et al.”
BackgroundMethodInfluentialBaker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993
Evaluating Learning Algorithms: A Classification Perspective
Japkowicz, N., & Shah, M. (2011) · doi:10.1017/cbo9780511921803 · Cited by 767
Yang (2019) · doi:10.1145/3339474 · Cited by 632
Sheller, M. J., Reina, G. A., Edwards, B., Martin, J., & Bakas, S. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11723-8_9 · Cited by 609
A comparative study of fairness-enhancing interventions in machine learning
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287589 · Cited by 601
The Measure and Mismeasure of Fairness
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2018) · arXiv · doi:10.48550/arxiv.1808.00023 · Cited by 444
Distributed deep learning networks among institutions for medical imaging
Chang, K., Balachandar, N., Lam, C., Yi, D., Brown, J., Beers, A., Rosen, B., Rubin, D. L., & Kalpathy-Cramer, J. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy017 · Cited by 340
Research works printed but not linked: 61
- 10.1145/2976749.2978318 · DOI · Paper page
- Orevaoghene Ahia, Julia Kreutzer, and Sara Hooker. 2021. The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation. arXiv preprint arXiv:2110.03036 (2021). · arXiv
- Izzat Alsmadi, ZW Taylor, and Joshua Childs. 2020. US News & World Report Best Colleges rankings: Which institutional metrics contribute to sustained stratification?Scientometrics 124, 3 (2020), 1851–1869.
- 10.1145/2983323.2983351 · DOI · Paper page
- Noah Arthurs and AJ Alvero. 2020. Whose Truth is the “Ground Truth”? College Admissions Essays and Bias in Word Vector Evaluation Methods. In Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020). 342–349.
- Lovenoor Aulck, Dev Nambi, Nishant Velagapudi, Joshua Blumenstock, and Jevin West. 2019. Mining University Registrar Records to Predict First-Year Undergraduate Attrition. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019). 9–18.
- 10.35542/osf.io/pbmvz · DOI
- 10.1007/s11162-019-09546-y · DOI · Paper page
- 10.1177/23328584211037630 · DOI · Paper page
- 10.1007/978-3-319-19773-9_6 · DOI
- Andrew P Bradley. 1997. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition 30, 7 (1997), 1145–1159.
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77–91.
- 10.1145/3170358.3170410 · DOI · Paper page
- 10.1145/3442188.3445879 · DOI
- 10.1145/3306618.3314236 · DOI · Paper page
- Andrew Cotter, Maya Gupta, Heinrich Jiang, Nathan Srebro, Karthik Sridharan, Serena Wang, Blake Woodworth, and Seungil You. 2019. Training well-generalizing classifiers for fairness metrics and other data-dependent constraints. In International Conference on Machine Learning. PMLR, 1397–1405.
- Gerben W. Dekker, Mykola Pechenizkiy, and Jan M. Vleeshouwers. 2009. Predicting students drop out: A case study. In Proceedings of the 2nd International Conference on Educational Data Mining (EDM 2009). 41–50.
- 10.1007/978-3-030-52237-7_11 · DOI · Paper page
- 10.1145/3303772.3303794 · DOI
- 10.1145/3303772.3303838 · DOI · Paper page
- Zilah EISESTEIN. 1979. Combahee River collective: A Black Feminist Statement. Capitalist Patriarchy and the Case for Socialist Feminism (1979).
- James Fogarty, Ryan S Baker, and Scott E Hudson. 2005. Case studies in the use of ROC curve analysis for sensor-based estimates in human computer interaction. In Proceedings of Graphics Interface 2005. 129–136.
- 10.1145/3303772.3303791 · DOI · Paper page
- Joshua P Gardner Zoran Popovi and Ludwig Schmidt. [n. d.]. Subgroup Robustness Grows On Trees: An Empirical Baseline Investigation. In Advances in Neural Information Processing Systems.
- James A Hanley and Barbara J McNeil. 1982. The meaning and use of the area under a receiver operating characteristic (ROC) curve.Radiology 143, 1 (1982), 29–36.
- Tatsunori B Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. 2018. Fairness without demographics in repeated loss minimization. arXiv preprint arXiv:1806.08010 (2018). · arXiv
- Nicholas W Hillman, David A Tandberg, and Jacob PK Gross. 2014. Performance funding in higher education: Do financial incentives impact college completions?The journal of higher education 85, 6 (2014), 826–857.
- Sara Hooker, Nyalleng Moorosi, Gregory Clark, Samy Bengio, and Emily Denton. 2020. Characterising bias in compressed models. arXiv preprint arXiv:2010.03058 (2020). · arXiv
- Qian Hu and Huzefa Rangwala. 2020. Towards Fair Educational Data Mining: A Case Study on Detecting At-risk Students. In Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020). 431–437.
- Stephen Hutt, Margo Gardner, Angela L. Duckworth, and Sidney K. D’Mello. 2019. Evaluating Fairness and Generalizability in Models Predicting On-Time Graduation from College Applications. In Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019).
- 10.18608/jla.2014.11.3 · DOI · Paper page
- Erik Jones, Shiori Sagawa, Pang Wei Koh, Ananya Kumar, and Percy Liang. 2020. Selective Classification Can Magnify Disparities Across Groups. arXiv preprint arXiv:2010.14134 (2020). · arXiv
- Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, 2019. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 (2019). · arXiv
- 10.1109/ICDMW.2011.83 · DOI · Paper page
- Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems 30 (2017).
- Fereshte Khani and Percy Liang. 2020. Feature Noise Induces Loss Discrepancy Across Groups. In International Conference on Machine Learning. PMLR, 5209–5219.
- René F. Kizilcec and Hansol Lee. 2022. Algorithmic fairness in education. In The Ethics of Artificial Intelligence in Education. Routledge, 174–202.
- 10.1145/3328526.3329621 · DOI · Paper page
- 10.1002/aehe.3205 · DOI · Paper page
- 10.1145/3386527.3406755 · DOI
- 10.1145/3152714 · DOI
- Hansol Lee and René F. Kizilcec. 2020. Evaluation of Fairness Trade-offs in Predicting Student Success. arxiv:2007.00088 [cs.CY] · arXiv
- Xingyu Li Difan Song Miaozhe Han Yu Zhang and René F Kizilcec. 2021. On the limits of algorithmic prediction across the globe. (2021). https://arxiv.org/abs/2103.15212 · arXiv
- 10.18653/v1/W19-4401 · DOI · Paper page
- Yuetian Luo and Zachary A. Pardos. 2018. Diagnosing University Student Subject Proficiency and Predicting Degree Completion in Vector Space. In Proceedings of the Eighth AAAI Symposium on Educational Advances in Artificial Intelligence. New Orleans, LA, USA.
- 10.1086/669570 · DOI · Paper page
- Julian Matschinske, Julian Späth, Reza Nasirigerdeh, Reihaneh Torkzadehmahani, Anne Hartebrodt, Balázs Orbán, Sándor Fejér, Olga Zolotareva, Mohammad Bakhtiari, Béla Bihari, 2021. The FeatureCloud AI Store for Federated Learning in Biomedicine and Beyond. arXiv preprint arXiv:2105.05734 (2021). · arXiv
- Giulio Morina, Viktoriia Oliinyk, Julian Waton, Ines Marusic, and Konstantinos Georgatzis. 2019. Auditing and achieving intersectional fairness in classification problems. arXiv preprint arXiv:1911.01468 (2019). · arXiv
- 10.1111/bjet.12156 · DOI · Paper page
- Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson. 2018. Scalable private learning with pate. arXiv preprint arXiv:1802.08908 (2018). · arXiv
- Dana Pessach, Tamir Tassa, and Erez Shmueli. 2021. Fairness-Driven Private Collaborative Machine Learning. arXiv preprint arXiv:2109.14376 (2021). · arXiv
- Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. 2019. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731 (2019). · arXiv
- Mark Schneider and Lu Yin. 2011. The High Cost of Low Graduation Rates: How Much Does Dropping Out of College Really Cost?Technical Report. · doi:10.1037/e537282012-001
- 10.1145/3442188.3445865 · DOI · Paper page
- Kai Ming Ting and Ian H Witten. 1997. Stacked Generalization: when does it work? (1997).
- 10.3102/00346543045001089 · doi:10.3102/00346543045001089 · Paper page
- S. Truth. 1851. Ain’t I A Woman?Speech delivered at Women’s Rights Convention, Akron, Ohio.
- David H Wolpert. 1992. Stacked generalization. Neural networks 5, 2 (1992), 241–259.
- Forest Yang, Moustapha Cisse, and Sanmi Koyejo. 2020. Fairness with Overlapping Groups. arXiv preprint arXiv:2006.13485 (2020). · arXiv
- 10.1145/3430895.3460139 · DOI · Paper page
- Renzhe Yu, Qiujie Li, Christian Fischer, Shayan Doroudi, and Di Xu. 2020. Towards Accurate and Fair Prediction of College Success: Evaluating Different Sources of Student Data. In Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020). 292–301.
Cited by
van Haastrecht, M., Brinkhuis, M., & Spruit, M. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-64299-9_5 · Cited by 13
Metadata
- Authors
- Gardner, J., Yu, R., Nguyen, Q., Brooks, C., & Kizilcec, R.
- Affiliations
- University of Washington, USA
- Teachers College, Columbia University, USA
- University of British Columbia, Canada
- University of Michigan, USA
- Cornell University, USA
- Venue
- 2023 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2023-06-12
- Pages
- 1664-1684
- Cited by
- 22
- References
- 14
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity. https://app.cochat.ai/papers/10.1145/3593013.3594107
- The paper
- Gardner, J., Yu, R., Nguyen, Q., Brooks, C., & Kizilcec, R. (2023). Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity. 2023 ACM Conference on Fairness Accountability and Transparency. https://doi.org/10.1145/3593013.3594107
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.