A genetic algorithms approach to modeling the performance of memory-bound computations
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Benchmarks that measure memory bandwidth, such as STREAM, Apex-MAPS and MultiMAPS, are increasingly popular due to the "Von Neumann" bottleneck of modern processors which causes many calculations to be memory-bound. We present a scheme for predicting the performance of HPC applications based on the results of such benchmarks. A Genetic Algorithm approach is used to "learn" bandwidth as a function of cache hit rates per machine with MultiMAPS as the fitness test. The specific results are 56 individual performance predictions including 3 full-scale parallel applications run on 5 different modern HPC architectures, with various CPU counts and inputs, predicted within 10 % average difference with respect to independently verified runtimes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, they rarely provide insights into how to improve performance of programs, compilers, or computers [1] or they can be hard to use by non-experts [27].”
From Roofline: an insightful visual performance model for multicore architectures (2009) · cites this paper as [27]
Citations
Cites
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,002
Genetic Algorithms in Search, Optimization and Machine Learning
Goldberg, D. E., & Robson, D. (1988) · Cited by 18,391
Goldberg, D. E., & Shakespeare, W. (2002) · Cited by 16,088
Bailey, D. H., Barszcz, E., Barton, J. T., Browning, D. S., Carter, R. L., Dagum, L., Fatoohi, R. A., Frederickson, P. O., Lasinski, T. A., Schreiber, R. S., et al. (1991) · The International Journal of Supercomputing Applications · doi:10.1177/109434209100500306 · Cited by 2,395
Hitting the memory wall: implications of the obvious
Wulf, W. A., & McKee, S. A. (1995) · ACM SIGARCH Computer Architecture News · doi:10.1145/216585.216588 · Cited by 1,624
Hitting the Memory Wall: Implications of the Obvious
Wulf, W. A., & McKee, S. A. (1995) · Cited by 1,379
An oceanic general circulation model framed in hybrid isopycnic-Cartesian coordinates
Bleck, R. (2002) · Ocean Modelling · doi:10.1016/s1463-5003(01)00012-9 · Cited by 1,343
Efficiently exploring architectural design spaces via predictive modeling
Ïpek, E., McKee, S. A., Caruana, R., de Supinski, B. R., & Schulz, M. (2006) · Proceedings of the 12th international conference on Architectural support for programming languages and operating systems · doi:10.1145/1168857.1168882 · Cited by 323
Converting thread-level parallelism to instruction-level parallelism via simultaneous multithreading
Lo, J. L., Emer, J. S., Levy, H. M., Stamm, R. L., Tullsen, D. M., & Eggers, S. J. (1997) · ACM Transactions on Computer Systems · doi:10.1145/263326.263382 · Cited by 233
Cross-architecture performance predictions for scientific applications using parameterized models
Marin, G., & Mellor-Crummey, J. (2004) · Proceedings of the joint international conference on Measurement and modeling of computer systems · doi:10.1145/1005686.1005691 · Cited by 228
Research works printed but not linked: 15
- McCalpin J., Memory bandwidth and machine balance in current high performance computers, IEEE Technical Committee on Computer Architecture Newsletter.
- 10.5555/1032659.1034232 · doi:10.5555/1032659.1034232
- Hoke C. C., 2004, Applied Aerodynamics Conference and Exhibit.
- Buning P. G., 2003, Langley Research Center
- J. H. Holland Adaptation in Natural and Artificial Systems. University of Michigan Press 1975 J. H. Holland Adaptation in Natural and Artificial Systems. University of Michigan Press 1975
- McCalpin J., Memory bandwidth and machine balance in current high performance computers, IEEE Technical Committee on Computer Architecture Newsletter.
- Ballansc R. S., 1962, New York
- Abandah G., 1996, Proceedings Int'l Parallel Processing Symposium, 249
- Hosie A., 1999, Proceedings of Frontiers of Massively Parallel Computing '99
- 10.5555/1762418.1762521 · doi:10.5555/1762418.1762521
- 10.5555/1114280.1114372 · doi:10.5555/1114280.1114372
- Snir M., 2005, Technical Report No. UIUCDCS-R-2005-2611
- X. Gao. PhD Thesis. 2006. University of California Computer Science Department. X. Gao. PhD Thesis. 2006. University of California Computer Science Department.
- Chen Y., 2007, Dongarra Editor, 2
- Phansalkar A., 2006, Proceedings of SPEC Benchmark Workshop
Cited by
Williams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
Williams, S., Waterman, A., & Patterson, D. (2009) · doi:10.2172/1407078 · Cited by 110
PSINS: An Open Source Event Tracer and Execution Simulator for MPI Applications
Tikir, M. M., Laurenzano, M. A., Carrington, L., & Snavely, A. E. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-03869-3_16 · Cited by 94
Self-citationPerformance projection of HPC applications using SPEC CFP2006 benchmarks
Sharkawi, S., DeSota, D., Panda, R., Indukuru, R., Stevens, S., Taylor, V., & Wu, X. (2009) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2009.5161057 · Cited by 51
Meswani, M. R., Carrington, L., Unat, D., Snavely, A. E., Baden, S. B., & Poole, S. (2012) · The International Journal of High Performance Computing Applications · doi:10.1177/1094342012468180 · Cited by 49
Self-citationDiagnosis, Tuning, and Redesign for Multicore Performance: A Case Study of the Fast Multipole Method
Chandramowlishwaran, A. M., Madduri, K., & Vuduc, R. W. (2010) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc.2010.19 · Cited by 44
Modeling memory concurrency for multi-socket multi-core systems
Mandal, A., Fowler, R., & Porterfield, A. (2010) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass.2010.5452064 · Cited by 38
An idiom-finding tool for increasing productivity of accelerators
Carrington, L., Tikir, M. M., Olschanowsky, C. M., Laurenzano, M. A., Peraza, J., Snavely, A. E., & Poole, S. (2011) · International Conference on Supercomputing · doi:10.1145/1995896.1995928 · Cited by 33
Self-citationData Partitioning on Heterogeneous Multicore Platforms
Zhong, Z., Rychkov, V., & Lastovetsky, A. (2011) · IEEE International Conference on Cluster Computing (CLUSTER) · doi:10.1109/cluster.2011.64 · Cited by 24
Performance Analysis of HPC Applications on Low-Power Embedded Platforms
Stanisic, L., Videau, B., Cronsioe, J., Degomme, A., Marangozova‐Martin, V., Legrand, A., & Méhaut, J. (2013) · Design, Automation, and Test in Europe · doi:10.7873/date.2013.106 · Cited by 24
PSINS: An Open Source Event Tracer and Execution Simulator
Tikir, M. M., Laurenzano, M. A., Carrington, L., & Snavely, A. E. (2009) · DoD High Performance Computing Modernization Program Users Group Conference (HPCMP-UGC) · doi:10.1109/hpcmp-ugc.2009.73 · Cited by 20
Self-citationPerformance Analysis of HPC Applications on Low-Power Embedded Platforms
Stanisic, L., Videau, B., Cronsioe, J., Degomme, A., Marangozova‐Martin, V., Legrand, A., & Méhaut, J. (n.d.) · Cited by 20
Machina, J., & Sodan, A. C. (2009) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2009.5161233 · Cited by 19
Inferring Large-Scale Computation Behavior via Trace Extrapolation
Carrington, L., Laurenzano, M. A., & Tiwari, A. (2013) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2013.137 · Cited by 18
Self-citationMeswani, M. R., Carrington, L., Unat, D., Snavely, A. E., Baden, S. B., & Poole, S. (2012) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2012.226 · Cited by 17
Self-citationKramer, W. (2012) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1145/2370816.2370850 · Cited by 17
PATHA: Performance Analysis Tool for HPC Applications
Yoo, W., Koo, M. S., Cao, Y., Sim, A., Nugent, P., & Wu, K. J. (2015) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/pccc.2015.7410313 · Cited by 16
Using an Adaptive HPC Runtime System to Reconfigure the Cache Hierarchy
Totoni, E., Torrellas, J., & Kalé, L. V. (2014) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc.2014.90 · Cited by 16
Percu: a holistic method for evaluating high performance computing systems
Demmel, J. W., & Kramer, W. (2008) · Cited by 15
Accurate memory signatures and synthetic address traces for HPC applications
Weinberg, J., & Snavely, A. E. (2008) · International Conference on Supercomputing · doi:10.1145/1375527.1375536 · Cited by 14
Self-citationImproving SMT performance: an application of genetic algorithms to configure resizable caches
Álvarez, J. D., Hidalgo, J. I., Fernández, F. V., Garnica, Ó., & López, S. M. (2009) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/1570256.1570271 · Cited by 13
Characterizing the Relation Between Apex-Map Synthetic Probes and Reuse Distance Distributions
Ibrahim, K. Z., & Strohmaier, E. (2010) · Proceedings of the International Conference on Parallel Processing · doi:10.1109/icpp.2010.43 · Cited by 12
Self-citationHPC-Smart Infrastructures: A Review and Outlook on Performance Analysis Methods and Tools
Muhammed, T., Mehmood, R., Albeshri, A. A., & Alsolami, F. J. (2019) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-030-13705-2_18 · Cited by 12
Fine-Grained Energy Consumption Characterization and Modeling
Olschanowsky, C. M., Rosing, T., Snavely, A. E., Carrington, L., Tikir, M. M., & Laurenzano, M. A. (2010) · DoD High Performance Computing Modernization Program Users Group Conference (HPCMP-UGC) · doi:10.1109/hpcmp-ugc.2010.54 · Cited by 12
Self-citationPrediction of Software Quality Model Using Gene Expression Programming
Singh, Y., Kaur, A., & Malhotra, R. (2009) · Lecture notes in business information processing · doi:10.1007/978-3-642-02152-7_5 · Cited by 11
CHARACTERIZING LARGE-SCALE HPC APPLICATIONS THROUGH TRACE EXTRAPOLATION
Carrington, L., Laurenzano, M. A., & Tiwari, A. (2013) · Parallel Processing Letters · doi:10.1142/s0129626413400082 · Cited by 11
Self-citationParametric optimization of storage systems
Zadok, E., Arora, A., Cao, Z. Z., Chaganti, A., Chaudhary, A., & Mandal, S. (2015) · USENIX conference on Hot topics in storage and file systems · Cited by 11
Graphics performance analysis using Amdahl's law
Issa, J. A., & Figueira, S. M. (2010) · International Symposium on Performance Evaluation of Computer and Telecommunication Systems · Cited by 10
On Predictable Reconfigurable System Design
Voss, N., Kwaadgras, B., Mencer, O., Luk, W. W., & Gaydadjiev, G. N. (2021) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3436995 · Cited by 9
Stanisic, L., Schnorr, L. M., Degomme, A., Heinrich, F. C., Legrand, A., & Videau, B. (2017) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2017.125 · Cited by 9
Stanisic, L. (2015) · theses.fr (ABES) · doi:10.70675/af2d6929zbbe2z4eddzbf13z0cb9fa9cf837 · Cited by 9
Dai, F., Chen, Y., Huang, Z., Zhang, H., Zhang, H., & Xia, C. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-022-04945-y · Cited by 8
BOPS, Not FLOPS! A New Metric and Roofline Performance Model For Datacenter ComputingPreprint
Wang, L., Zhan, J., Gao, W., Yang, K., Jiang, Z., Ren, R., He, X., & Luo, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.09212 · Cited by 7
Prediction-based load balancing and resolution tuning for interactive volume raycasting
Bruder, V., Frey, S., & Ertl, T. (2017) · Visual Informatics · doi:10.1016/j.visinf.2017.09.001 · Cited by 7
Fickenscher, J., Reiche, O., Trautmann, J., Hannig, F., & Teich, J. (2016) · Proceedings · doi:10.1109/hldvt.2016.7748257 · Cited by 6
Power and energy management of modern architectures in adaptive HPC runtime systems
Totoni, E. (2014) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 6
Characterizing the Performance of Hybrid Memory Cube Using ApexMAP Application Probes
Ibrahim, K. Z., Fatollahi-Fard, F., Donofrio, D., & Shalf, J. M. (2016) · International Symposium on Memory Systems (MEMSYS) · doi:10.1145/2989081.2989090 · Cited by 6
Real-time performance prediction and tuning for interactive volume raycasting
Bruder, V., Frey, S., & Ertl, T. (2016) · SIGGRAPH Asia Symposium on Visualization · doi:10.1145/3002151.3002156 · Cited by 6
Викторович, Л. С., Козелков, А. С., Yalozo, A. V., Gerasimov, V. Y., & Зеленский, Д. К. (2017) · Journal of Applied Mechanics and Technical Physics · doi:10.1134/s0021894417070069 · Cited by 5
Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters
Yoo, W., Koo, M. S., Cao, Y., Sim, A., Nugent, P., & Wu, K. J. (2016) · Springer eBooks · doi:10.1007/978-3-319-33742-5_7 · Cited by 4
Roofline Model Based Performance-Aware Energy Management for Scientific Computing
Wang, Y., Zhao, T., Li, L., Hou, Z., & Gu, J. (2018) · International Symposium on Parallel Architectures, Algorithms and Programming · doi:10.1109/paap.2018.00020 · Cited by 4
Performance and power-consumption analysis of Mobile Internet Devices
Issa, J. A., & Figueira, S. M. (2011) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/pccc.2011.6108116 · Cited by 4
A performance estimation model for GPU-based systems
Issa, J. A., & Figueira, S. M. (2012) · International Conference on Advances in Computational Tools for Engineering Applications · doi:10.1109/ictea.2012.6462883 · Cited by 4
Efficiency analysis of parallel implementation of SIMPLE algorithm on multi-processor computers
Викторович, Л. С., Козелков, А. С., Yalozo, A. V., Gerasimov, V. Y., & Зеленский, Д. К. (2016) · Computational Continuum Mechanics · doi:10.7242/1999-6691/2016.9.3.25 · Cited by 4
Use of model‐based architecture attributes to construct a component‐level trade space
McKean, D., Moreland, J. D., & Doskey, S. C. (2019) · Systems Engineering · doi:10.1002/sys.21478 · Cited by 3
Quantitative Analysis of Graph Algorithms: Models and Optimization Methods
Wang, X., Zhu, Y., & Chen, Y. (2016) · IEEE International Conference on Big Data Security on Cloud, IEEE International Conference on High Performance and Smart Computing, and IEEE International Conference on Intelligent Data and Security (BigDataSecurity/HPSC/IDS) · doi:10.1109/bigdatasecurity-hpsc-ids.2016.42 · Cited by 3
Modeling and Analysis for Performance and Power
Choi, J. W., & Vuduc, R. W. (2012) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2012.304 · Cited by 3
Selecting linear algebra kernel composition using response time prediction
Hurault, A., Baek, K., & Casanova, H. (2014) · Software Practice and Experience · doi:10.1002/spe.2307 · Cited by 3
Performance Modeling Tools for Parallel Sparse Linear Algebra Computations
Cicotti, P., Li, X. S., & Baden, S. B. (2010) · Advances in parallel computing · doi:10.3233/978-1-60750-530-3-83 · Cited by 3
Processor performance modeling using regression method
Issa, J. A. (2016) · IEEE Mediterranean Electrotechnical Conference (MELECON) · doi:10.1109/melcon.2016.7495404 · Cited by 3
Multi-spectral Reuse Distance: Divining Spatial Information from Temporal Data
Cabrera, A. M., Chamberlain, R. D., & Beard, J. C. (2019) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec.2019.8916398 · Cited by 3
HPC application address stream compression, replay and scaling
Olschanowsky, C. R. M. (2011) · eScholarship (California Digital Library) · Cited by 3
Automated performance characterization of applications using hardware monitoring events
Yoo, W. (2012) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 2
Antunes, B. A., & Hill, D. R. C. (2023) · Research Reports on Computer Science · doi:10.37256/rrcs.2220233134 · Cited by 2
Towards Modeling and Simulation of Exascale Computing Platforms
Stanisic, L. (2012) · HAL · Cited by 2
How to Evaluate Various Commonly Used Program Classification Methods?
Qi, X., Yuan, Y., Chen, J., & Dong, Y. Q. (2020) · Communications in computer and information science · doi:10.1007/978-981-15-8135-9_17 · Cited by 1
Worley, P. H. (2010) · Chapman & Hall/CRC computational science series/Chapman & Hall/CRC computational science · doi:10.1201/b10509-18 · Cited by 1
Prediction of Processor Utilization for Real-Time Multimedia Stream Processing Tasks
Krawczyk, H., Proficz, J., & Daca, B. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-36071-8_22 · Cited by 1
A Workload Prediction Model for 3D Textured Meshes in Webgl Context
Meynet, G., Iehl, J., Delanoy, J., Lavoué, G., & Dupont, F. (2024) · international conference on 3D web technology · doi:10.1145/3665318.3677156 · Cited by 1
Easy-to-use SAP sizing based on evolutionary generated scalability models
Tertilt, D., & Krcmar, H. (2012) · Procedia Technology · doi:10.1016/j.protcy.2012.09.018 · Cited by 1
A novel method to estimate performance for a high performance computation workload
Issa, J. A. (2014) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/pccc.2014.7017027 · Cited by 1
A performance estimation model for Ls-Dyna computation workload
Issa, J. A. (2016) · Future Technologies Conference · doi:10.1109/ftc.2016.7821755 · Cited by 1
Methodology for complex dataflow application development
Voss, N. (2020) · doi:10.25560/91314 · Cited by 1
Velho, P. A. M. D. C. (2011) · theses.fr (ABES) · doi:10.70675/5feb40fez2fcez4968z8fe7z2384a70c0bfd · Cited by 1
Application-Specific Memory Subsystem Benchmarking
Lakshminarasimhan, M. (2019) · ScholarWorks (Boise State University) · doi:10.18122/td/1534/boisestate
Antunes, B. A. (2024) · theses.fr (ABES) · doi:10.70675/5de5da49z74e0z4d79zaeaaz57d00e6c9ac2
Scalability Analysis and Optimization for Large-Scale Deep Learning
Pumma, S. (2020) · VTechWorks (Virginia Tech)
Developing a Parameterized Performance Proxy for Sequential Scientific Kernels
Shan, H., & Strohmaier, E. (2010) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc.2010.50
Self-citationGraphics Processor performance analysis for 3D applications
Issa, J. A., & Figueira, S. M. (2012) · International Conference on Advances in Computational Tools for Engineering Applications · doi:10.1109/ictea.2012.6462881
Characterization and Performance Analysis for 3D Benchmarks
Issa, J. A. (2012) · Journal of Computers · doi:10.4304/jcp.7.11.2702-2710
Soft Co-Processor Based Hardware Acceleration for Image Blending
Shapiro, D., Parri, J., Desmarais, J.-M., Kouri, A., Bergeron, J.-P., & Bolić, M. (2011) · uO Research (University of Ottawa)
Power Efficient Last Level Cache for Chip Multiprocessors
Dani, A. M. (2013)
Performance Projections of HPC Applications on Chip Multiprocessor (CMP) Based Systems
Sharkawi, S., & Sh, S. (2012) · OakTrust (Texas A&M University Libraries)
Large Scale Parallel Simulation For Extended Discrete Element Method
Chekaraou, M., & Chekaraou, A. W. M. (2020) · Open Repository and Bibliography (University of Luxembourg)
Performance quantification of volume visualization
Bruder, V. (2016) · OPUS Publication Server of the University of Stuttgart (University of Stuttgart) · doi:10.18419/opus-9608
Predicción de Perfiles de Comportamiento de Aplicaciones Científicas en Nodos Multicore
Corredor, J., & Operatius, U. A. D. B. D. D. D. C. I. S. (2011) · Dipòsit Digital de Documents de la UAB (Universitat Autònoma de Barcelona)
Metadata
- Authors
- Tikir, M. M., Carrington, L., Strohmaier, E., & Snavely, A.
- Affiliations
- San Diego Supercomputer Center (US)
- Lawrence Berkeley National Laboratory (US)
- Venue
- Proceedings of the 2007 ACM/IEEE conference on Supercomputing
- Publisher
- ACM
- Published
- 2007-11-10
- Pages
- 1-12
- ISSN
- 1063-9535 (ISSN-L)
- OpenAlex
- W2109811664
- Cited by
- 66
- References
- 36
- Citations by year
- 2024: 1 · 2023: 1 · 2022: 1 · 2020: 1 · 2019: 4 · 2018: 2 · 2017: 3 · 2016: 8 · 2015: 2 · 2014: 4 · 2013: 6 · 2012: 11
- FWCI
- 3.94
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE Conference on Supercomputing (SC) (publishedVersion)
- Funders
- National Science Foundation (406312, 516162)
- Office of Science (SciDAC2)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Keywords
- memory bandwidth, performance prediction, genetic algorithm, STREAM benchmark, cache hit rate, von Neumann bottleneck, HPC applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.