Long-term Effectiveness of mHealth Physical Activity Interventions: Systematic Review and Meta-analysis of Randomized Controlled Trials
Open accessThe DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Abstract
Background: Mobile health (mHealth) interventions can increase physical activity (PA); however, their long-term impact is not well understood. Objective: The primary aim of this study is to understand the immediate and long-term effects of mHealth interventions on PA. The secondary aim is to explore potential effect moderators. Methods: We performed this study according to the Cochrane and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We searched PubMed, the Cochrane Library, SCOPUS, and PsycINFO in July 2020. Eligible studies included randomized controlled trials of mHealth interventions targeting PA as a primary outcome in adults. Eligible outcome measures were walking, moderate-to-vigorous physical activity (MVPA), total physical activity (TPA), and energy expenditure. Where reported, we extracted data for 3 time points (ie, end of intervention, follow-up ≤6 months, and follow-up >6 months). To explore effect moderators, we performed subgroup analyses by population, intervention design, and control group type. Results were summarized using random effects meta-analysis. Risk of bias was assessed using the Cochrane Collaboration tool. Results: Of the 2828 identified studies, 117 were included. These studies reported on 21,118 participants with a mean age of 52.03 (SD 14.14) years, of whom 58.99% (n=12,459) were female. mHealth interventions significantly increased PA across all the 4 outcome measures at the end of intervention (walking standardized mean difference [SMD] 0.46, 95% CI 0.36-0.55; P<.001; MVPA SMD 0.28, 95% CI 0.21-0.35; P<.001; TPA SMD 0.34, 95% CI 0.20-0.47; P<.001; energy expenditure SMD 0.44, 95% CI 0.13-0.75; P=.01). Only 33 studies reported short-term follow-up measurements, and 8 studies reported long-term follow-up measurements in addition to end-of-intervention results. In the short term, effects were sustained for walking (SMD 0.26, 95% CI 0.09-0.42; P=.002), MVPA (SMD 0.20, 95% CI 0.05-0.35; P=.008), and TPA (SMD 0.53, 95% CI 0.13-0.93; P=.009). In the long term, effects were also sustained for walking (SMD 0.25, 95% CI 0.10-0.39; P=.001) and MVPA (SMD 0.19, 95% CI 0.11-0.27; P<.001). We found the study population to be an effect moderator, with higher effect scores in sick and at-risk populations. PA was increased both in scalable and nonscalable mHealth intervention designs and regardless of the control group type. The risk of bias was rated high in 80.3% (94/117) of the studies. Heterogeneity was significant, resulting in low to very low quality of evidence. Conclusions: mHealth interventions can foster small to moderate increases in PA. The effects are maintained long term; however, the effect size decreases over time. The results encourage using mHealth interventions in at-risk and sick populations and support the use of scalable mHealth intervention designs to affordably reach large populations. However, given the low evidence quality, further methodologically rigorous studies are warranted to evaluate the long-term effects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Hodkinson, A., Kontopantelis, E., Adeniji, C., van Marwijk, H., McMillan, B., Bower, P., & Panagioti, M. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.12895 · Cited by 55
RetractedThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681
Lee, I.-M., Shiroma, E. J., Lobelo, F., Puska, P., Blair, S. N., & Katzmarzyk, P. T. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61031-9 · Cited by 8,215
Using Pedometers to Increase Physical Activity and Improve Health: A Systematic Review
Bravata, D. M., Smith-Spangler, C., Sundaram, V., Gienger, A. L., Lin, N., Lewis, R., Stave, C. D., Olkin, I., & Sirard, J. R. (2007) · JAMA · doi:10.1001/jama.298.19.2296 · Cited by 2,064
Large-scale physical activity data reveal worldwide activity inequality
Althoff, T., Sosič, R., Hicks, J. L., King, A. C., Delp, S. L., & Leskovec, J. (2017) · Nature · doi:10.1038/nature23018 · Cited by 1,000
Schoeppe, S., Alley, S., Van Lippevelde, W., Bray, N. A., Williams, S. L., Duncan, M. J., & Vandelanotte, C. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0454-y · Cited by 949 · cited 2×
“Current evidence draws an inconclusive picture: existing studies have found no effects on PA when interventions are scalable [24,30] (ie, mHealth interventions without human-to-human interactions), stronger effects when interventions are nonscalable (ie, mHealth interventions with human-to-human interactions) [27,32],…”
BackgroundMcAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02786-0 · Cited by 812
Choi, K. W., Chen, C.-Y., Stein, M. B., Klimentidis, Y. C., Wang, M.-J., Koenen, K. C., Smoller, J. W., & Major Depressive Disorder Working Group of the Psychiatric Genomics Consortium (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.4175 · Cited by 744
Reis, R. S., Salvo, D., Ogilvie, D., Lambert, E. V., Goenka, S., & Brownson, R. C. (2016) · The Lancet · doi:10.1016/s0140-6736(16)30728-0 · Cited by 719
Brickwood, K.-J., Watson, G., O'Brien, J., & Williams, A. D. (2019) · JMIR mHealth and uHealth · doi:10.2196/11819 · Cited by 689
What is the clinical value of mHealth for patients?
Rowland, S. P., Fitzgerald, J. E., Holme, T., Powell, J., & McGregor, A. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0206-x · Cited by 454 · cited 1×
“Most governments and health organizations are still hesitant about rolling out mHealth PA programs, as clear evidence for the effectiveness of mHealth interventions for sustainable behavior change is lacking [16,17].”
BackgroundThe impact of digital technology on psychological treatments and their dissemination
Fairburn, C. G., & Patel, V. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.08.012 · Cited by 448 · cited 1×
“Owing to the increasing dissemination and ubiquity of mobile technology, mobile technology–based interventions, that is, mobile health (mHealth), have been discussed as a solution for overcoming scalability challenges [10,11].”
BackgroundFinkelstein, E. A., Haaland, B. A., Bilger, M., Sahasranaman, A., Sloan, R. A., Nang, E. E. K., & Evenson, K. R. (2016) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(16)30284-4 · Cited by 402
Direito, A., Carraça, E., Rawstorn, J., Whittaker, R., & Maddison, R. (2017) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9846-0 · Cited by 337 · cited 3×
“minimal or no intervention control groups and control groups receiving an alternative intervention [18-20,22-29,31].”
BackgroundResultTudor-Locke, C., Bell, R. C., Myers, A. M., Harris, S. B., Ecclestone, N. A., Lauzon, N., & Rodger, N. W. (2004) · International Journal of Obesity · doi:10.1038/sj.ijo.0802485 · Cited by 321
Talbot, L. A., Gaines, J. M., Huynh, T. N., & Metter, E. J. (2003) · Journal of the American Geriatrics Society · doi:10.1046/j.1532-5415.2003.51113.x · Cited by 315
Martin, S. S., Feldman, D. I., Blumenthal, R. S., Jones, S. R., Post, W. S., McKibben, R. A., Michos, E. D., Ndumele, C. E., Ratchford, E. V., Coresh, J., et al. (2015) · Journal of the American Heart Association · doi:10.1161/jaha.115.002239 · Cited by 267
Gal, R., May, A. M., van Overmeeren, E. J., Simons, M., & Monninkhof, E. M. (2018) · Sports Medicine - Open · doi:10.1186/s40798-018-0157-9 · Cited by 257
Demeyer, H., Louvaris, Z., Frei, A., Rabinovich, R. A., de Jong, C., Gimeno-Santos, E., Loeckx, M., Buttery, S. C., Rubio, N., Van der Molen, T., et al. (2017) · Thorax · doi:10.1136/thoraxjnl-2016-209026 · Cited by 256
de Blok, B. M. J., de Greef, M. H. G., ten Hacken, N. H. T., Sprenger, S. R., Postema, K., & Wempe, J. B. (2006) · Patient Education and Counseling · doi:10.1016/j.pec.2005.02.005 · Cited by 231
Lyons, E. J., Swartz, M. C., Lewis, Z. H., Martinez, E., & Jennings, K. (2017) · JMIR mhealth and uhealth · doi:10.2196/mhealth.6967 · Cited by 220
Franssen, W. M. A., Franssen, G. H. L. M., Spaas, J., Solmi, F., & Eijnde, B. O. (2020) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-00955-2 · Cited by 193
Internet intervention for increasing physical activity in persons with multiple sclerosis
Motl, R. W., Dlugonski, D., Wójcicki, T. R., McAuley, E., & Mohr, D. C. (2011) · Multiple Sclerosis Journal · doi:10.1177/1352458510383148 · Cited by 186 · cited 3×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundMaher, C., Ferguson, M., Vandelanotte, C., Plotnikoff, R., De Bourdeaudhuij, I., Thomas, S., Nelson-Field, K., & Olds, T. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4086 · Cited by 171
Lynch, B. M., Nguyen, N. H., Moore, M. M., Reeves, M. M., Rosenberg, D. E., Boyle, T., Vallance, J. K., Milton, S., Friedenreich, C. M., & English, D. R. (2019) · Cancer · doi:10.1002/cncr.32143 · Cited by 171 · cited 5×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundMethodInfluentialPromoting physical activity through hand-held computer technology
King, A. C., Ahn, D. K., Oliveira, B. M., Atienza, A. A., Castro, C. M., & Gardner, C. D. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.09.025 · Cited by 160 · cited 3×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundCan implementation intentions and text messages promote brisk walking? A randomized trial.
Prestwich, A., Perugini, M., & Hurling, R. (2010) · Health Psychology · doi:10.1037/a0016993 · Cited by 158
Wijsman, C. A., Westendorp, R. G., Verhagen, E. A., Catt, M., Slagboom, P. E., de Craen, A. J., Broekhuizen, K., van Mechelen, W., van Heemst, D., van der Ouderaa, F., et al. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2843 · Cited by 156
van der Weegen, S., Verwey, R., Spreeuwenberg, M., Tange, H., van der Weijden, T., & de Witte, L. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4579 · Cited by 154
Richard, E., Moll van Charante, E. P., Hoevenaar-Blom, M. P., Coley, N., Barbera, M., van der Groep, A., Meiller, Y., Mangialasche, F., Beishuizen, C. B., Jongstra, S., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30153-0 · Cited by 150 · cited 1×
“Owing to the increasing dissemination and ubiquity of mobile technology, mobile technology–based interventions, that is, mobile health (mHealth), have been discussed as a solution for overcoming scalability challenges [10,11].”
BackgroundGuertler, D., Vandelanotte, C., Kirwan, M., & Duncan, M. J. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4339 · Cited by 147
Dlugonski, D., Motl, R. W., Mohr, D. C., & Sandroff, B. M. (2012) · Psychology, Health & Medicine · doi:10.1080/13548506.2011.652640 · Cited by 133 · cited 4×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundResultInfluentialde Vries, H. J., Kooiman, T. J. M., van Ittersum, M. W., van Brussel, M., & de Groot, M. (2016) · Obesity · doi:10.1002/oby.21619 · Cited by 123
Accelerometer-Determined Steps/Day and Metabolic Syndrome
Sisson, S. B., Camhi, S. M., Church, T. S., Tudor-Locke, C., Johnson, W. D., & Katzmarzyk, P. T. (2010) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2010.02.015 · Cited by 116 · cited 3×
“Previous research found that 1000 incremental steps per day can result in a 10% lower risk of having metabolic syndrome (MetS) and a 6% risk reduction of all-cause mortality, substantiating that mHealth interventions can result in significant J Med Internet Res 2021 | vol. 23 | iss.”
BackgroundInfluentialKing, A. C., Bickmore, T. W., Campero, M. I., Pruitt, L. A., & Yin, J. L. (2013) · Journal of Health Communication · doi:10.1080/10810730.2013.798374 · Cited by 114
Ashton, L. M., Morgan, P. J., Hutchesson, M. J., Rollo, M. E., & Collins, C. E. (2017) · Nutrition Journal · doi:10.1186/s12937-017-0227-8 · Cited by 111 · cited 4×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundInfluentialPoirier, J., Bennett, W. L., Jerome, G. J., Shah, N. G., Lazo, M., Yeh, H.-C., Clark, J. M., & Cobb, N. K. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5295 · Cited by 111 · cited 4×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundInfluentialStep counter use in type 2 diabetes: a meta-analysis of randomized controlled trials
Qiu, S., Cai, X., Chen, X., Yang, B., & Sun, Z. (2014) · BMC Medicine · doi:10.1186/1741-7015-12-36 · Cited by 106 · cited 4×
“minimal or no intervention control groups and control groups receiving an alternative intervention [18-20,22-29,31].”
BackgroundInfluentialBarwais, F. A., Cuddihy, T. F., & Tomson, L. (2013) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-11-183 · Cited by 104 · cited 7×
“However, the funnel plot analysis revealed 3 outlier studies [54,82,102].”
BackgroundMethodResultInfluentialNolan, C. M., Maddocks, M., Canavan, J. L., Jones, S. E., Delogu, V., Kaliaraju, D., Banya, W., Kon, S. S. C., Polkey, M. I., & Man, W. D.-C. (2017) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201607-1372oc · Cited by 104
Cruz (2016) · doi:10.1177/1479972315619574 · Cited by 97
Harris, T., Kerry, S. M., Limb, E. S., Furness, C., Wahlich, C., Victor, C. R., Iliffe, S., Whincup, P. H., Ussher, M., Ekelund, U., et al. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002526 · Cited by 95 · cited 5×
“In 1 study [75], 2 mHealth interventions (scalable and nonscalable) were combined.”
BackgroundInfluentialShcherbina, A., Hershman, S. G., Lazzeroni, L., King, A. C., O'Sullivan, J. W., Hekler, E., Moayedi, Y., Pavlovic, A., Waggott, D., Sharma, A., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30129-3 · Cited by 90 · cited 1×
“Within our sample, most scalable mHealth interventions leveraged basic technologies (eg, texting, pedometers, or accelerometers), without taking advantage of more advanced mobile technologies (eg, automated individualized coaching, social comparison, and mobile apps), which could have further increased intervention ef…”
BackgroundVan Blarigan, E. L., Chan, H., Van Loon, K., Kenfield, S. A., Chan, J. M., Mitchell, E., Zhang, L., Paciorek, A., Joseph, G., Laffan, A., et al. (2019) · BMC Cancer · doi:10.1186/s12885-019-5427-5 · Cited by 85
Kramer, J.-N., Künzler, F., Mishra, V., Smith, S. N., Kotz, D., Scholz, U., Fleisch, E., & Kowatsch, T. (2020) · Annals of Behavioral Medicine · doi:10.1093/abm/kaaa002 · Cited by 84 · cited 1×
“Within our sample, most scalable mHealth interventions leveraged basic technologies (eg, texting, pedometers, or accelerometers), without taking advantage of more advanced mobile technologies (eg, automated individualized coaching, social comparison, and mobile apps), which could have further increased intervention ef…”
BackgroundSelf-citationCompernolle, S., Vandelanotte, C., Cardon, G., De Bourdeaudhuij, I., & De Cocker, K. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3402 · Cited by 80 · cited 4×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundInfluentialLi, L. C., Sayre, E. C., Xie, H., Clayton, C., & Feehan, L. M. (2017) · JMIR mHealth and uHealth · doi:10.2196/mhealth.7863 · Cited by 74 · cited 3×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundStacey, F. G., James, E. L., Chapman, K., & Lubans, D. R. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0372-z · Cited by 74 · cited 5×
“All but one study [40] reported end-of-intervention results, 33 studies reported short-term follow-up results [40-72], and only 8 studies reported long-term”
BackgroundResultInfluentialVandelanotte, C., Duncan, M. J., Maher, C. A., Schoeppe, S., Rebar, A. L., Power, D. A., Short, C. E., Doran, C. M., Hayman, M. J., & Alley, S. J. (2018) · Journal of Medical Internet Research · doi:10.2196/11321 · Cited by 68 · cited 3×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundOliveira, J. S., Sherrington, C., Paul, S. S., Ramsay, E., Chamberlain, K., Kirkham, C., O'Rourke, S. D., Hassett, L., & Tiedemann, A. (2019) · Journal of Physiotherapy · doi:10.1016/j.jphys.2018.11.005 · Cited by 63 · cited 5×
“All but one study [40] reported end-of-intervention results, 33 studies reported short-term follow-up results [40-72], and only 8 studies reported long-term”
BackgroundResultInfluentialBennett, J. A., Young, H. M., Nail, L. M., Winters-Stone, K., & Hanson, G. (2008) · Nursing Research · doi:10.1097/01.nnr.0000280661.34502.c1 · Cited by 57
Simons, D., De Bourdeaudhuij, I., Clarys, P., De Cocker, K., Vandelanotte, C., & Deforche, B. (2018) · JMIR mHealth and uHealth · doi:10.2196/10003 · Cited by 56 · cited 5×
“All but one study [40] reported end-of-intervention results, 33 studies reported short-term follow-up results [40-72], and only 8 studies reported long-term”
InfluentialThorsteinsen, K., Vittersø, J., & Svendsen, G. B. (2014) · International Journal of Telemedicine and Applications · doi:10.1155/2014/746232 · Cited by 50 · cited 3×
“Table 1 summarizes all results, and Multimedia Appendices 7 [41-154], 8 [41-154], and 9 [41-154] provide detailed forest plots for each analysis.”
BackgroundKangasniemi, A. M., Lappalainen, R., Kankaanpää, A., Tolvanen, A., & Tammelin, T. (2015) · BMC Public Health · doi:10.1186/s12889-015-1604-x · Cited by 37
Warren, F. C., Stych, K., Thorogood, M., Sharp, D. J., Murphy, M., Turner, K. M., Holt, T. A., Searle, A., Bryant, S., Huxley, C., et al. (2014) · Trials · doi:10.1186/1745-6215-15-134 · Cited by 25
Cited by
Santos, A., Braaten, K., MacPherson, M., Vasconcellos, D., Vis-Dunbar, M., Lonsdale, C., Lubans, D., & Jung, M. E. (2023) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-023-01535-w · Cited by 59
“For example, support through concurrent mHealth, eHealth, and activity tracker interventions [245–248], the implementation of behavior change techniques [249, 250], and/or the development of unsupervised interventions grounded on theoretical frameworks [251] are just some strategies that have shown…”
BackgroundKracht, C. L., Hutchesson, M., Ahmed, M., Müller, A. M., Ashton, L. M., Brown, H. M., DeSmet, A., Maher, C. A., Mauch, C. E., Vandelanotte, C., et al. (2021) · Obesity Reviews · doi:10.1111/obr.13331 · Cited by 37
Metadata
- Authors
- Mönninghoff, A., Kramer, J. N., Hess, A. J., Ismailova, K., Teepe, G. W., Tudor Car, L., Müller-Riemenschneider, F., & Kowatsch, T.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2021-04-30
- Volume
- 23
- Issue
- 4
- Pages
- e26699
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W3140890558
- Cited by
- 183
- References
- 56
- FWCI
- 6.42
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.