The DOI matches its Crossref record; 95 of the 95 references it cites were found in a registry or an index.
Machine Learning in Environmental Research: Common Pitfalls and Best Practices
Abstract
Machine learning (ML) is increasingly used in environmental research to process large data sets and decipher complex relationships between system variables. However, due to the lack of familiarity and methodological rigor, inadequate ML studies may lead to spurious conclusions. In this study, we synthesized literature analysis with our own experience and provided a tutorial-like compilation of common pitfalls along with best practice guidelines for environmental ML research. We identified more than 30 key items and provided evidence-based data analysis based on 148 highly cited research articles to exhibit the misconceptions of terminologies, proper sample size and feature size, data enrichment and feature selection, randomness assessment, data leakage management, data splitting, method selection and comparison, model optimization and evaluation, and model explainability and causality. By analyzing good examples on supervised learning and reference modeling paradigms, we hope to help researchers adopt more rigorous data preprocessing and model development standards for more accurate, robust, and practicable model uses in environmental research and applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
An Introduction to Statistical Learning
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7 · Cited by 12,106
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638
A general and simple method for obtaining R 2 from generalized linear mixed‐effects models
Nakagawa, S., & Schielzeth, H. (2013) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2012.00261.x · Cited by 10,356
Building Predictive Models in R Using the caret Package
Kuhn, M. (2008) · Journal of Statistical Software · doi:10.18637/jss.v028.i05 · Cited by 8,863
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Small sample size effects in statistical pattern recognition: Recommendations for practitioners
Raudys, S. J., & Jain, A. K. (1991) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.75512 · Cited by 1,384
Foundations of Machine Learning
Singh, A. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3399990 · Cited by 1,123
Drug discovery with explainable artificial intelligence
Jiménez-Luna, J., Grisoni, F., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00236-4 · Cited by 1,063
Machine Learning: New Ideas and Tools in Environmental Science and Engineering
Zhong, S., Zhang, K., Bagheri, M., Burken, J. G., Gu, A., Li, B., Ma, X., Marrone, B. L., Ren, Z. J., Schrier, J., et al. (2021) · Environmental Science & Technology · doi:10.1021/acs.est.1c01339 · Cited by 945
Self-citationHe, J., Gong, S., Yu, Y., Yu, L., Wu, L., Mao, H., Song, C., Zhao, S., Liu, H., Li, X., et al. (2017) · Environmental Pollution · doi:10.1016/j.envpol.2017.01.050 · Cited by 750
Handbook of Statistical Analysis and Data Mining Applications
(2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374765-5.x0001-0 · Cited by 700
Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges
Molnar, C., Casalicchio, G., & Bischl, B. (2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-65965-3_28 · Cited by 677
Integration of artificial neural network and MADA methods for green supplier selection
Kuo, R. J., Wang, Y. C., & Tien, F. C. (2010) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2010.03.020 · Cited by 654
Ahmad, M. W., Reynolds, J., & Rezgui, Y. (2018) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2018.08.207 · Cited by 642
Estimating PM2.5 Concentrations in the Conterminous United States Using the Random Forest Approach
Hu, X., Belle, J. H., Meng, X., Wildani, A., Waller, L. A., Strickland, M. J., & Liu, Y. (2017) · Environmental Science & Technology · doi:10.1021/acs.est.7b01210 · Cited by 629
Dou, J., Yunus, A. P., Tien Bui, D., Merghadi, A., Sahana, M., Zhu, Z., Chen, C.-W., Khosravi, K., Yang, Y., & Pham, B. T. (2019) · Science of The Total Environment · doi:10.1016/j.scitotenv.2019.01.221 · Cited by 615
Qi, Y., Li, Q., Karimian, H., & Liu, D. (2019) · Science of The Total Environment · doi:10.1016/j.scitotenv.2019.01.333 · Cited by 601
Chen, G., Li, S., Knibbs, L. D., Hamm, N. A. S., Cao, W., Li, T., Guo, J., Ren, H., Abramson, M. J., & Guo, Y. (2018) · Science of The Total Environment · doi:10.1016/j.scitotenv.2018.04.251 · Cited by 591
39 Dimensionality and sample size considerations in pattern recognition practice
Jain, A. K., & Chandrasekaran, B. (1982) · Handbook of Statistics · doi:10.1016/s0169-7161(82)02042-2 · Cited by 560
Data-Driven Machine Learning in Environmental Pollution: Gains and Problems
Liu, X., Lu, D., Zhang, A., Liu, Q., & Jiang, G. (2022) · Environmental Science & Technology · doi:10.1021/acs.est.1c06157 · Cited by 548
Assessing PM 2.5 Exposures with High Spatiotemporal Resolution across the Continental United States
Di, Q., Kloog, I., Koutrakis, P., Lyapustin, A., Wang, Y., & Schwartz, J. (2016) · Environmental Science & Technology · doi:10.1021/acs.est.5b06121 · Cited by 510
Chen, W., Peng, J., Hong, H., Shahabi, H., Pradhan, B., Liu, J., Zhu, A.-X., Pei, X., & Duan, Z. (2018) · Science of The Total Environment · doi:10.1016/j.scitotenv.2018.01.124 · Cited by 478
Rodriguez-Galiano, V., Mendes, M. P., Garcia-Soldado, M. J., Chica-Olmo, M., & Ribeiro, L. (2014) · Science of The Total Environment · doi:10.1016/j.scitotenv.2014.01.001 · Cited by 465
Short-term wind power forecasting based on support vector machine with improved dragonfly algorithm
Li, L.-L., Zhao, X., Tseng, M.-L., & Tan, R. R. (2020) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.118447 · Cited by 463
Zhu, X., Li, Y., & Wang, X. (2019) · Bioresource Technology · doi:10.1016/j.biortech.2019.121527 · Cited by 431
Santos, M. S., Soares, J. P., Abreu, P. H., Araujo, H., & Santos, J. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2866730 · Cited by 402
Mapping flood susceptibility in mountainous areas on a national scale in China
Zhao, G., Pang, B., Xu, Z., Yue, J., & Tu, T. (2018) · Science of The Total Environment · doi:10.1016/j.scitotenv.2017.10.037 · Cited by 389
Developers, T. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.4724125 · Cited by 384
Xiao, Q., Chang, H. H., Geng, G., & Liu, Y. (2018) · Environmental Science & Technology · doi:10.1021/acs.est.8b02917 · Cited by 376
Machine Learning: An Algorithmic Perspective
Marsland, S. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/b17476 · Cited by 353
Randomness in neural networks: an overview
Scardapane, S., & Wang, D. (2017) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1200 · Cited by 352
Kaab, A., Sharifi, M., Mobli, H., Nabavi-Pelesaraei, A., & Chau, K.-W. (2019) · Science of The Total Environment · doi:10.1016/j.scitotenv.2019.02.004 · Cited by 349
Review of soft sensor methods for regression applications
Souza, F. A. A., Araújo, R., & Mendes, J. (2016) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2015.12.011 · Cited by 345
PySwarms: a research toolkit for Particle Swarm Optimization in Python
James V. Miranda, L. (2018) · The Journal of Open Source Software · doi:10.21105/joss.00433 · Cited by 343
Chen, J., de Hoogh, K., Gulliver, J., Hoffmann, B., Hertel, O., Ketzel, M., Bauwelinck, M., van Donkelaar, A., Hvidtfeldt, U. A., Katsouyanni, K., et al. (2019) · Environment International · doi:10.1016/j.envint.2019.104934 · Cited by 342
Hosseini, F. S., Choubin, B., Mosavi, A., Nabipour, N., Shamshirband, S., Darabi, H., & Haghighi, A. T. (2020) · Science of The Total Environment · doi:10.1016/j.scitotenv.2019.135161 · Cited by 342
Satellite-Derived 1-km-Resolution PM1 Concentrations from 2014 to 2018 across China
Wei, J., Li, Z., Guo, J., Sun, L., Huang, W., Xue, W., Fan, T., & Cribb, M. (2019) · Environmental Science & Technology · doi:10.1021/acs.est.9b03258 · Cited by 338
Introduction to Machine Learning with Python
Müller, A. (2016) · Cited by 336
Xue, T., Zheng, Y., Tong, D., Zheng, B., Li, X., Zhu, T., & Zhang, Q. (2019) · Environment International · doi:10.1016/j.envint.2018.11.075 · Cited by 311
Zhan, Y., Luo, Y., Deng, X., Grieneisen, M. L., Zhang, M., & Di, B. (2018) · Environmental Pollution · doi:10.1016/j.envpol.2017.10.029 · Cited by 310
scikit-optimize/scikit-optimize
Head, T., Kumar, M., Nahrstaedt, H., Louppe, G., & Shcherbatyi, I. (2021) · Zenodo · doi:10.5281/zenodo.1157319 · Cited by 302
Gayen, A., Pourghasemi, H. R., Saha, S., Keesstra, S., & Bai, S. (2019) · Science of The Total Environment · doi:10.1016/j.scitotenv.2019.02.436 · Cited by 295
Yang, H., Huang, K., Zhang, K., Weng, Q., Zhang, H., & Wang, F. (2021) · Environmental Science & Technology · doi:10.1021/acs.est.1c02479 · Cited by 289
Revealing Drivers of Haze Pollution by Explainable Machine Learning
Hou, L., Dai, Q., Song, C., Liu, B., Guo, F., Dai, T., Li, L., Liu, B., Bi, X., Zhang, Y., et al. (2022) · Environmental Science & Technology Letters · doi:10.1021/acs.estlett.1c00865 · Cited by 288
Doing Data Science: Straight Talk from the Frontline
Schutt, R., & O’Neil, C. (2013) · O'Reilly Media, Inc. eBooks · Cited by 287
Elmolla, E. S., Chaudhuri, M., & Eltoukhy, M. M. (2010) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2010.02.068 · Cited by 282
Bui, D. T., Tsangaratos, P., Ngo, P.-T. T., Pham, T. D., & Pham, B. T. (2019) · Science of The Total Environment · doi:10.1016/j.scitotenv.2019.02.422 · Cited by 279
Rodriguez-Galiano, V. F., Luque-Espinar, J. A., Chica-Olmo, M., & Mendes, M. P. (2018) · Science of The Total Environment · doi:10.1016/j.scitotenv.2017.12.152 · Cited by 275
Park, Y., Cho, K. H., Park, J., Cha, S. M., & Kim, J. H. (2015) · Science of The Total Environment · doi:10.1016/j.scitotenv.2014.09.005 · Cited by 272
Reid, C. E., Jerrett, M., Petersen, M. L., Pfister, G. G., Morefield, P. E., Tager, I. B., Raffuse, S. M., & Balmes, J. R. (2015) · Environmental Science & Technology · doi:10.1021/es505846r · Cited by 256
Chen, C., Xiong, R., Yang, R., Shen, W., & Sun, F. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.06.273 · Cited by 252
Zhan, Y., Luo, Y., Deng, X., Zhang, K., Zhang, M., Grieneisen, M. L., & Di, B. (2018) · Environmental Science & Technology · doi:10.1021/acs.est.7b05669 · Cited by 250
Prediction Modeling and Mapping of Groundwater Fluoride Contamination throughout India
Podgorski, J. E., Labhasetwar, P., Saha, D., & Berg, M. (2018) · Environmental Science & Technology · doi:10.1021/acs.est.8b01679 · Cited by 248
Wang, B., Waters, C., Orgill, S., Gray, J., Cowie, A., Clark, A., & Liu, D. L. (2018) · Science of The Total Environment · doi:10.1016/j.scitotenv.2018.02.204 · Cited by 244
ChatGPT and Environmental Research
Zhu, J.-J., Jiang, J., Yang, M., & Ren, Z. J. (2023) · Environmental Science & Technology · doi:10.1021/acs.est.3c01818 · Cited by 243
Self-citationVoukantsis, D., Karatzas, K., Kukkonen, J., Räsänen, T., Karppinen, A., & Kolehmainen, M. (2011) · Science of The Total Environment · doi:10.1016/j.scitotenv.2010.12.039 · Cited by 240
Wang, D., Wei, S., Luo, H., Yue, C., & Grunder, O. (2017) · Science of The Total Environment · doi:10.1016/j.scitotenv.2016.12.018 · Cited by 236
Beckerman, B. S., Jerrett, M., Serre, M., Martin, R. V., Lee, S.-J., van Donkelaar, A., Ross, Z., Su, J., & Burnett, R. T. (2013) · Environmental Science & Technology · doi:10.1021/es400039u · Cited by 231
Predicting Daily Urban Fine Particulate Matter Concentrations Using a Random Forest Model
Brokamp, C., Jandarov, R., Hossain, M., & Ryan, P. (2018) · Environmental Science & Technology · doi:10.1021/acs.est.7b05381 · Cited by 229
Huang, K., Xiao, Q., Meng, X., Geng, G., Wang, Y., Lyapustin, A., Gu, D., & Liu, Y. (2018) · Environmental Pollution · doi:10.1016/j.envpol.2018.07.016 · Cited by 227
Qi, C., Fourie, A., Chen, Q., & Zhang, Q. (2018) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2018.02.154 · Cited by 225
Zhai, B., & Chen, J. (2018) · Science of The Total Environment · doi:10.1016/j.scitotenv.2018.04.040 · Cited by 225
A hybrid algorithm for carbon dioxide emissions forecasting based on improved lion swarm optimizer
Qiao, W., Lu, H., Zhou, G., Azimi, M., Yang, Q., & Tian, W. (2020) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.118612 · Cited by 225
Parshetti, G. K., Chowdhury, S., & Balasubramanian, R. (2014) · Bioresource Technology · doi:10.1016/j.biortech.2014.03.087 · Cited by 222
Barzegar, R., Fijani, E., Asghari Moghaddam, A., & Tziritis, E. (2017) · Science of The Total Environment · doi:10.1016/j.scitotenv.2017.04.189 · Cited by 218
Huser, B. J., Egemose, S., Harper, H., Hupfer, M., Jensen, H., Pilgrim, K. M., Reitzel, K., Rydin, E., & Futter, M. (2016) · Water Research · doi:10.1016/j.watres.2015.06.051 · Cited by 205
Meteorological factors had more impact on airborne bacterial communities than air pollutants
Zhen, Q., Deng, Y., Wang, Y., Wang, X., Zhang, H., Sun, X., & Ouyang, Z. (2017) · Science of The Total Environment · doi:10.1016/j.scitotenv.2017.05.049 · Cited by 198
Golafshani, E. M., & Behnood, A. (2018) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2017.11.186 · Cited by 198
Ransom, K. M., Nolan, B. T., A. Traum, J., Faunt, C. C., Bell, A. M., Gronberg, J. A. M., Wheeler, D. C., Z. Rosecrans, C., Jurgens, B., Schwarz, G. E., et al. (2017) · Science of The Total Environment · doi:10.1016/j.scitotenv.2017.05.192 · Cited by 195
Revolutionizing Membrane Design Using Machine Learning-Bayesian Optimization
Gao, H., Zhong, S., Zhang, W., Igou, T., Berger, E., Reid, E., Zhao, Y., Lambeth, D., Gan, L., Afolabi, M. A., et al. (2022) · Environmental Science & Technology · doi:10.1021/acs.est.1c04373 · Cited by 190
Wilkes, G., Edge, T. A., Gannon, V. P. J., Jokinen, C., Lyautey, E., Neumann, N. F., Ruecker, N., Scott, A., Sunohara, M., Topp, E., et al. (2011) · Water Research · doi:10.1016/j.watres.2011.06.021 · Cited by 184
Wetland loss due to land use change in the Lower Paraná River Delta, Argentina
Sica, Y. V., Quintana, R. D., Radeloff, V. C., & Gavier-Pizarro, G. I. (2016) · Science of The Total Environment · doi:10.1016/j.scitotenv.2016.04.200 · Cited by 184
Chen, G., Wang, Y., Li, S., Cao, W., Ren, H., Knibbs, L. D., Abramson, M. J., & Guo, Y. (2018) · Environmental Pollution · doi:10.1016/j.envpol.2018.07.012 · Cited by 178
Egea, J. A., Henriques, D., Cokelaer, T., Villaverde, A. F., MacNamara, A., Danciu, D.-P., Banga, J. R., & Saez-Rodriguez, J. (2014) · BMC Bioinformatics · doi:10.1186/1471-2105-15-136 · Cited by 176
Zhao, Y., Zhao, J., Huang, Y., Zhou, Q., Zhang, X., & Zhang, S. (2014) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2014.06.018 · Cited by 176
Linear and nonlinear modeling approaches for urban air quality prediction
Singh, K. P., Gupta, S., Kumar, A., & Shukla, S. P. (2012) · Science of The Total Environment · doi:10.1016/j.scitotenv.2012.03.076 · Cited by 174
Khataee, A. R., Dehghan, G., Ebadi, A., Zarei, M., & Pourhassan, M. (2010) · Bioresource Technology · doi:10.1016/j.biortech.2009.11.079 · Cited by 170
Aghav, R. M., Kumar, S., & Mukherjee, S. N. (2011) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2011.01.067 · Cited by 166
Estimating future burned areas under changing climate in the EU-Mediterranean countries
Amatulli, G., Camia, A., & San-Miguel-Ayanz, J. (2013) · Science of The Total Environment · doi:10.1016/j.scitotenv.2013.02.014 · Cited by 161
Ma, J., Ding, Y., Cheng, J. C. P., Jiang, F., Tan, Y., Gan, V. J. L., & Wan, Z. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.118955 · Cited by 161
Jeong, N., Chung, T.-H., & Tong, T. (2021) · Environmental Science & Technology · doi:10.1021/acs.est.1c04041 · Cited by 157
Cho, K. H., Sthiannopkao, S., Pachepsky, Y. A., Kim, K., & Kim, J. H. (2011) · Water Research · doi:10.1016/j.watres.2011.08.010 · Cited by 156
Çavaş, L., Karabay, Z., Alyürük, H., Doğan, H., & Demir, G. K. (2011) · Chemical Engineering Journal · doi:10.1016/j.cej.2011.04.030 · Cited by 155
Yang, W., Wang, J., Lu, H., Niu, T., & Du, P. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.03.036 · Cited by 153
Chen, T., Chang, Q., Clevers, J. G. P. W., & Kooistra, L. (2015) · Environmental Pollution · doi:10.1016/j.envpol.2015.07.009 · Cited by 142
Leakage in data mining: formulation, detection, and avoidance
Kaufman, S., Rosset, S., & Perlich, C. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020496 · Cited by 137
Wavelet and ANN combination model for prediction of daily suspended sediment load in rivers
Rajaee, T. (2011) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2010.11.028 · Cited by 125
Zhu, J., Dressel, W., Pacion, K., & Ren, Z. J. (2021) · Environmental Science & Technology · doi:10.1021/acs.est.0c07551 · Cited by 122
Self-citationXu, R., Cao, J., Ye, T., Wang, S., Luo, J., Ni, B., & Fang, F. (2022) · Water Research · doi:10.1016/j.watres.2022.118975 · Cited by 117
Data Analytics for Environmental Science and Engineering Research
Gupta, S., Aga, D. S., Pruden, A. J., Zhang, L., & Vikesland, P. J. (2021) · Environmental Science & Technology · doi:10.1021/acs.est.1c01026 · Cited by 103
A survey on pre-processing techniques: Relevant issues in the context of environmental data mining
Gibert, K., Sànchez–Marrè, M., & Izquierdo, J. (2016) · AI Communications · doi:10.3233/aic-160710 · Cited by 87
Yang, M., Zhu, J., McGaughey, A. L., Zheng, S., Priestley, R. D., & Ren, Z. J. (2023) · Environmental Science & Technology · doi:10.1021/acs.est.2c06382 · Cited by 71
Self-citationDeep Learning Optimization for Soft Sensing of Hard-to-Measure Wastewater Key Variables
Zhu, J., Borzooei, S., Sun, J., et al. (2022) · ACS ES&T Engineering · doi:10.1021/acsestengg.1c00469 · Cited by 52
Self-citation
Cited by
Deep learning for water quality
Zhi, W., Appling, A. P., Golden, H. E., Podgorski, J., & Li, L. (2024) · Nature Water · doi:10.1038/s44221-024-00202-z · Cited by 364
Metadata
- Authors
- Zhu, J.-J., Yang, M., & Ren, Z. J.
- Venue
- Environmental Science & Technology
- Publisher
- American Chemical Society (ACS)
- Published
- 2023-06-29
- Volume
- 57
- Issue
- 46
- Pages
- 17671-17689
- ISSN
- 0013-936X (ISSN-L)
- 1520-5851
- OpenAlex
- W4382536268
- Cited by
- 672
- References
- 95
- FWCI
- 132.63
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.