The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index; 1 of the 14 sources its claims cite were checked for support.
RoBANS 2: A Revised Risk of Bias Assessment Tool for Nonrandomized Studies of Interventions
Open accessAbstract
Assessment of the risk of bias is an essential component of any systematic review. This is true for both nonrandomized studies and randomized trials, which are the main study designs of systematic reviews. The Risk of Bias Assessment Tool for Nonrandomized Studies (RoBANS) was developed in 2013 and has gained wide usage as a risk-of-bias assessment tool for nonrandomized studies. Four risk-of-bias assessment experts revised it by reviewing existing assessment tools and user surveys. The main modifications included additional domains of selection and detection bias susceptible to nonrandomized studies of interventions, a more detailed consideration of the comparability of participants, and more reliable and valid outcome measurements. A psychometric assessment of the revised RoBANS (RoBANS 2) revealed acceptable inter-rater reliability (weighted kappa, 0.25 to 0.49) and construct validity in which intervention effects of studies with an unclear or high risk of bias were overestimated. The RoBANS 2 has acceptable feasibility, fair-to-moderate reliability, and construct validity. It provides a comprehensive framework for allowing authors to assess and understand the plausible risk of bias in nonrandomized studies of interventions.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
RoBANS 2 is a revised outcome-based risk-of-bias tool for nonrandomized studies of interventions, adding selection and detection bias domains and more detailed consideration of participant comparability and outcome measurement. In psychometric testing it showed acceptable feasibility, fair-to-moderate inter-rater reliability (weighted kappa 0.25–0.49), and construct validity, with unclear/high risk-of-bias studies overestimating intervention effects.
Key findings
- RoBANS 2 revised the original RoBANS by adding domains for selection and detection bias, separating participant selection into comparability of participants and target group selection, and revising outcome assessment to consider blinding of outcome assessors and reliable/valid outcome measurement methods.
- Feasibility: mean time to complete risk-of-bias assessment was 8.72±5.00 minutes per article, with a range from 20 seconds to 36.35 minutes.
- Inter-rater reliability: all eight RoBANS 2 domains had fair agreement or higher, with weighted kappa statistics ranging from 0.25 to 0.49.
- Construct validity: among 112 nonrandomized studies included for inter-rater reliability, 77 were included for construct validity; studies conducted inadequately for each risk-of-bias domain were likely to report low odds ratios in seven of eight domains, indicating that intervention effects with unclear or high risk of bias were overestimated.
Methods
- The revision reviewed existing risk-of-bias/critical appraisal checklists (SIGN, Newcastle-Ottawa Scale, AHRQ checklist, RTI Item Bank) and held consultative meetings with five systematic review methodologists; a sample of nonrandomized studies was compiled via the National Evidence-based Healthcare Collaborating Agency, Health Insurance Review and Assessment Service, PubMed, and the Cochrane Library.
- Inclusion criteria for psychometric evaluation were studies with no intervention or placebo control, dichotomous outcome data except before-and-after studies, and designs including cohort, case-control, cross-sectional, or before-and-after studies; 112 studies were selected: 45 cohort, 16 case-control, 25 cross-sectional, and 26 before-and-after. The minimum required by two raters was 85 studies, based on 80% power to detect a kappa of 0.70 against a null kappa of 0.40.
- Paired assessors with doctoral degrees and at least 10 years of systematic review experience independently rated studies after pilot testing; analyses used weighted kappa for reliability and compared effect sizes by risk-of-bias domain using Cohen’s d for continuous outcomes and Hasselblad and Hedges’s transformation for odds ratios, with SAS 9.4, Stata SE 16.0, and RevMan 5.4; tests were two-sided at α=0.05.
Limitations
- Stated in the paper: further refinement and extensive feedback from RoBANS 2 users are required; for controlled before-after studies, interrupted time series, and interrupted time series with comparisons, assessors need to determine domain judgments considering study design nature from epidemiological experts; the authors recommend Cochrane risk-of-bias tools for RCTs, non-RCTs, or quasi-experimental trials; further research is needed to compare inter-rater agreement and usability of RoBANS 2 and ROBINS-I for cohort-type designs.
- Evident from the text: reliability was only fair-to-moderate (weighted kappa 0.25–0.49), not substantial or almost perfect; construct validity analysis excluded 26 before-and-after studies without comparison and nine studies unable to extract data, leaving 77 of 112 studies; no direct comparison with ROBINS-I was performed in this study.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 of 14 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured
- Findingown data
All eight domains of RoBANS 2 had fair agreement or higher.
“All domains of eight RoBANS 2 had fair agreement or higher, ranging from 0.25 to 0.49 κ statistics.”
Quote in the paperSubstantial claim - Findingown data
The inter-rater reliability of RoBANS 2 ranged from 0.25 to 0.49 weighted kappa.
“All domains of eight RoBANS 2 had fair agreement or higher, ranging from 0.25 to 0.49 κ statistics.”
Quote in the paperSubstantial claim - Findingown data
Studies conducted inadequately for each domain of the risk of bias were likely to report low ORs in seven of eight domains.
“Our findings revealed that studies conducted inadequately for each domain of the risk of bias were likely to report low ORs in seven of eight domains (Table 3).”
Quote in the paperSubstantial claim - Findingnone given
The tools RoBANS 2 and ROBINS-I overlap substantially in terms of the risk of bias domains.
“However, the tools overlap substantially in terms of the risk of bias domains (Appendix 3).”
Quote in the paperNot a substantial claim - Study design and conductcitation
The minimum number of studies required by the two raters was 85, based on a dichotomous variable with 80% power, to detect a kappa of 0.70, at a proportion of positive ratings of 0.70.
“The minimum number of studies required by the two raters was 85, based on a dichotomous variable with 80% power, to detect a kappa of 0.70, at a proportion of positive ratings of 0.70.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [12] source
- Interpretationown data
Intervention effect studies with an unclear or high risk of bias were overestimated.
“In other words, intervention effect studies with an unclear or high risk of bias were overestimated.”
- Also stated in: § Abstract · plain
“intervention effects of studies with an unclear or high risk of bias were overestimated”
Quote in the paperSubstantial claim - Also stated in: § Abstract · plain
- Premisecitation
Differential or non-differential misclassification of the outcome data could introduce detection bias in NRSI.
“Differential or non-differential misclassification of the outcome data could introduce detection bias in NSRI [15].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [15] source
- Premisecitation
Confounding by indication or severity introduces systematic bias, leading to either over- or underestimation of treatment effects depending on the treatment decision mechanism.
“Consequently, confounding by indication or severity introduces systematic bias, leading to either over- or underestimation of treatment effects depending on the treatment decision mechanism [14].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [14] source
- It may state two separate claims as one. 9
- Probably not a substantial claim. 9
It may state two separate claims as one.
PremisecitationA crucial limitation of observational studies on intervention effects and adverse reactions is that the intervention of interest is not randomly assigned, blinding is lacking, and there is often no comparison.
“However, a crucial limitation of observational studies on intervention effects and adverse reactions is that the intervention of interest is not randomly assigned, blinding is lacking, and there is often no comparison.”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [5] source
- cites [6] source
The 'Inclusion of non-randomised studies in systematic reviews' section says non-randomised intervention studies face biases that can make their effect estimates inaccurate, supporting the concern about non-random assignment. It does not say that observational studies generally lack blinding or often have no comparison group. (LLM, conf=0.86)
Experimental — accuracy not yet measured
It may state two separate claims as one.
Findingown dataThe mean time to complete the risk of bias assessment using RoBANS 2 was 8.72±5.00 minutes per article, with a range of 20 seconds to 36.35 minutes.
“The time spent assessing each study ranged from 20 seconds to 36.35 minutes, with a mean of 8.72±5.00 minutes per article.”
Quote in the paperSubstantial claimIt may state two separate claims as one.
Findingnone givenThe main modifications in RoBANS 2 included additional domains of selection and detection bias susceptible to nonrandomized studies of interventions, a more detailed consideration of the comparability of participants, and more reliable and valid outcome measurements.
“The main modifications included additional domains of selection and detection bias susceptible to nonrandomized studies of interventions, a more detailed consideration of the comparability of participants, and more reliable and valid outcome measurements.”
- Also stated in: § DISCUSSION · plain
“The main modifications included additional domains of selection and detection bias susceptible to the NRSI, a more detailed consideration of the comparability of participants, and more reliable and valid outcome measurements.”
Quote in the paperSubstantial claim- Also stated in: § DISCUSSION · plain
Probably not a substantial claim.
Findingnone givenSimilar to the original RoBANS, the assessments in RoBANS 2 were related to the risk of bias in the estimates of the intervention effect for a single outcome or endpoint rather than at the study level.
“Similar to the original RoBANS, the assessments in RoBANS 2 were related to the risk of bias in the estimates of the intervention effect for a single outcome or endpoint rather than at the study level.”
Quote in the paperSubstantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Findingnone givenSimilar to the original version, RoBANS 2 is an outcome-based checklist.
“Similar to the original version, RoBANS 2 is an outcome-based checklist.”
Quote in the paperNot a substantial claimProbably not a substantial claim.
FindingreasoningThe domains of blinding of outcome assessors, outcome assessment, and incomplete outcome data can be treated as result-based evaluations because they are classified as patient-reported outcomes or objective outcome measures.
“Additionally, the domains of blinding of outcome assessors, outcome assessment, and incomplete outcome data can be treated as result-based evaluations because they are classified as patient-reported outcomes or objective outcome measures.”
Quote in the paperSubstantial claimIt may state two separate claims as one.
InterpretationreasoningRoBANS 2 has construct validity and can detect significant differences in effect size estimates according to the risk of bias.
“Therefore, the RoBANS 2 has construct validity and can detect significant differences in effect size estimates according to the risk of bias.”
Quote in the paperSubstantial claimProbably not a substantial claim.
Study design and conductown dataPaired assessors independently evaluated the risk of bias of the included studies using RoBANS 2 after pilot testing a sample of included studies.
“Paired assessors of the review team independently evaluated the risk of bias of the included studies using RoBANS 2 after pilot testing a sample of included studies.”
Quote in the paperSubstantial claimIt may state two separate claims as one.
PremisecitationRoBANS was developed in 2013 and has gained wide usage as a risk-of-bias assessment tool for nonrandomized studies.
“The Risk of Bias Assessment Tool for Nonrandomized Studies (RoBANS) was developed in 2013 and has gained wide usage as a risk-of-bias assessment tool for nonrandomized studies.”
- Also stated in: § INTRODUCTION · plain
“The Risk of Bias Assessment Tool for Nonrandomized Studies (RoBANS), published in 2013 [7], is a widely used bias tool.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [7] source
- Also stated in: § INTRODUCTION · plain
Probably not a substantial claim.
PremisecitationIn nonrandomized studies, selection bias occurs when participants chosen for the intervention of interest have different characteristics from those allocated to the alternative intervention (or not treated) because the choice of a given intervention might be affected by the discretion of the treating clinician or patient preference, patient characteristics, and clinical history.
“In nonrandomized studies, selection bias occurs when participants chosen for the intervention of interest have different characteristics from those allocated to the alternative intervention (or not treated) because the choice of a given intervention might be affected by the discretion of the treating clinician or patient preference, patient characteristics, and clinical history [13].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [13] source
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Study design and conductnone givenEach study was randomly assigned to paired assessors using computer-based random number generation.
“Each study was randomly assigned to paired assessors using computer-based random number generation.”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
Interpretationown dataRoBANS 2 had acceptable feasibility, fair to moderate reliability, and construct validity.
“In conclusion, RoBANS 2 had acceptable feasibility, fair to moderate reliability, and construct validity.”
- Also stated in: § Abstract · plain
“The RoBANS 2 has acceptable feasibility, fair-to-moderate reliability, and construct validity.”
Quote in the paperSubstantial claim- Also stated in: § Abstract · plain
Probably not a substantial claim.
PremisecitationReal-world evidence has become increasingly critical for identifying the effects and safety of healthcare interventions.
“Real-world evidence has become increasingly critical for identifying the effects and safety of healthcare interventions.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4] source
It may state two separate claims as one.
PremisecitationThe findings of nonrandomized studies of interventions can be generalized to the population and provide clinical evidence regarding the benefits or risks of healthcare interventions.
“The findings of nonrandomized studies of interventions (NRSI) can be generalized to the population and provide clinical evidence regarding the benefits or risks of healthcare interventions [1].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [1] source
Probably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationThese reviews may allow for the evaluation of adverse events and long-term effects after exposure to healthcare interventions.
“Moreover, these reviews may allow for the evaluation of adverse events and long-term effects after exposure to healthcare interventions [3].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [3] source
It may state two separate claims as one.
PremisecitationThe study findings of observational studies are susceptible to confounding and selection biases, which could result in biased estimates of intervention effects compared with smaller RCTs.
“Thus, the study findings are susceptible to confounding and selection biases, which could result in biased estimates of intervention effects compared with smaller RCTs [5,6].”
Quote in the paperSubstantial claim0 of 2 sources checkedProbably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationResearchers have increasingly included NRSI in systematic reviews to examine various interventions such as medications, hospital procedures, community health interventions, and health systems.
“Researchers have increasingly included NRSI in systematic reviews to examine various interventions such as medications, hospital procedures, community health interventions, and health systems [2].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [2]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
It may state two separate claims as one.
PremisecitationBias can occur when outcome assessors are aware of the intervention status, if different methods are used to assess outcomes in different intervention groups, or if measurement errors are related to the intervention status or effects.
“Bias can occur when outcome assessors are aware of the intervention status, if different methods are used to assess outcomes in different intervention groups, or if measurement errors are related to the intervention status or effects [16].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [16] source
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,586
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681 · cited 1×
“24) The RoBANS 2 is a comprehensive checklist instrument for assessing the risk of bias in cohort studies, case-control studies, cross-sectional studies, and before and after studies of interventions with user guidance to support educational purposes and improve inter-rater agreement of the…”
MethodShea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
The kappa statistic in reliability studies: use, interpretation, and sample size requirements
Sim, J., & Wright, C. C. (2005) · Physical Therapy · doi:10.1093/ptj/85.3.257 · Cited by 4,093
Norderyd, O. M., Koch, G., Papias, A., Köhler, A. A., Helkimo, A. N., Brahm, C., Lindmark, U., Lindfors, N., Mattsson, A., Rolander, B., et al. (2015) · PubMed · Cited by 3,584
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
Kim, S. Y., Park, J. E., Lee, Y. J., Seo, H.-J., Sheen, S.-S., Hahn, S., Jang, B.-H., & Son, H.-J. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.09.016 · Cited by 1,165
Self-citationEffect-Size Indices for Dichotomized Outcomes in Meta-Analysis.
Sánchez‐Meca, J., Marín‐Martínez, F., & Moscoso, S. C. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.448 · Cited by 653
Meta-analysis of screening and diagnostic tests.
Hasselblad, V., & Hedges, L. V. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.117.1.167 · Cited by 579
Confounding by Indication in Clinical Research
Kyriacou, D. N., & Lewis, R. J. (2016) · JAMA · doi:10.1001/jama.2016.16435 · Cited by 565
Kim, H.-S., Lee, S., & Kim, J. H. (2018) · Journal of Korean Medical Science · doi:10.3346/jkms.2018.33.e213 · Cited by 370
(2013) · doi:10.1016/j.jclinepi.2011.05.008 · Cited by 264
Monti, S., Grosso, V., Todoerti, M., & Caporali, R. (2018) · Rheumatology · doi:10.1093/rheumatology/key109 · Cited by 170
Introduction to real-world evidence studies
Chodankar, D. (2021) · Perspectives in Clinical Research · doi:10.4103/picr.picr_62_21 · Cited by 153
Harnessing Real-World Evidence to Advance Cancer Research
Tang, M., Pearson, S.-A., Simes, R. J., & Chua, B. H. (2023) · Current Oncology · doi:10.3390/curroncol30020143 · Cited by 88
Validity and Inter-Rater Reliability Testing of Quality Assessment Instruments [Internet]
Hartling, L., Hamm, M. P., Milne, A., Vandermeer, B., Santaguida, P. L., Ansari, M., Tsertsvadze, A., Hempel, S., Shekelle, P., & Dryden, D. M. (2012) · Cited by 29
Desai, R. J., Levin, R., Lin, K. J., & Patorno, E. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.120.016906 · Cited by 29
Evidence for Detection Bias by Medication Use in a Cohort Study of Breast Cancer Survivors
Wirtz, H. S., Calip, G. S., Buist, D. S. M., Gralow, J. R., Barlow, W. E., Gray, S. L., & Boudreau, D. M. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kww242 · Cited by 22
Measurement error in clinical research, yes it matters
Groenwold, R. H. H., & Dekkers, O. M. (2020) · European Journal of Endocrinology · doi:10.1530/eje-20-0550 · Cited by 17
Recent Advance in Clinical Practice Guideline Development Methodology
Kim, S. Y. (2022) · Korean Journal of Family Medicine · doi:10.4082/kjfm.22.0178 · Cited by 7
Self-citation
Flagged first, then by citations
Research works printed but not linked: 4
- c2023, Checklists [Internet]
- Wells, c2021, The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses [Internet]
- Warrens, 2013, Weighted kappas for 3×3 tables, 325831
- Review Manager (RevMan): version 5.4 [Internet]
Cited by
Sorted by citations
Characteristics of fragility hip fracture-related falls in the older adults: A systematic review
Lim, S.-K., Choi, K., Heo, N. H., Kim, Y., & Lim, J. (2024) · The journal of nutrition health & aging · doi:10.1016/j.jnha.2024.100357 · Cited by 47
What is the role of physical exercise in the era of cancer prehabilitation? A systematic review
Bianco, N. D., Borsati, A., Toniolo, L., Ciurnielli, C., Belluomini, L., Insolda, J., Sposito, M., Milella, M., Schena, F. F., Pilotto, S., et al. (2024) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2024.104350 · Cited by 41
Bae, M. (2024) · BMC Sports Science Medicine and Rehabilitation · doi:10.1186/s13102-024-01025-x · Cited by 37
Wan, K., Dai, Z., Wong, P., Huang, Y., Lei, E. F., Little, J. P., Lin, F., & Tam, B. T. (2025) · Scandinavian Journal of Medicine and Science in Sports · doi:10.1111/sms.70114 · Cited by 35
Ferreira, F. A., Santos, C. C., Palmeira, A. L., Fernandes, R. J., & Costa, M. J. (2024) · Journal of Functional Morphology and Kinesiology · doi:10.3390/jfmk9030158 · Cited by 25
Grotto, G., & Buja, A. (2025) · Digital Health · doi:10.1177/20552076251336973 · Cited by 24
Wu, C., Zhang, C., Li, X., Ye, C., & Astikainen, P. (2024) · Memory · doi:10.1080/09658211.2024.2423812 · Cited by 21
Physiological, Physical and Technical Demands During Sided Soccer Game Formats: a Review
Clemente, F. M., Martinho, D. V., Silva, A. F., Trybulski, R., Sánchez-Sánchez, J., Fernandez, A. R., Beato, M., & Afonso, J. R. (2025) · International Journal of Sports Medicine · doi:10.1055/a-2591-6995 · Cited by 17
Samama, M., Ueno, J., Veiga, E. C. D. A., Piscopo, R. C. C. P., Ikeda, F., de Lemos, N. P., Bidinotto, L. T., da Silva, M. G., Bella, Z. I. K. D. J. D., & Entezami, F. (2025) · Life · doi:10.3390/life15091382 · Cited by 17
Seol, J. H., Song, J., Kim, S. J., Ko, H., Na, J. Y., Cho, M.-J., Choi, H. J., Lee, J. S., Oh, K. J., Jung, J. W., et al. (2024) · Hepatology Communications · doi:10.1097/hc9.0000000000000580 · Cited by 17
Self-citationBayly-McCredie, E., Treisman, M., & Fiorenza, S. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25179736 · Cited by 16
A systematic review on anti-Xa monitoring in the therapeutic use of low-molecular-weight heparins
Jaspers, T. C. C., Remmelzwaal, P. C., Weersink, E. P. S., Meijer, K., & Khorsand, N. (2025) · Journal of Thrombosis and Haemostasis · doi:10.1016/j.jtha.2025.06.025 · Cited by 15
Shimmyo, K., & Obayashi, S. (2024) · Biomedicines · doi:10.3390/biomedicines12030623 · Cited by 15
Xiao, W. L., Yuan, H., Wang, D., Zhao, J., & Fang, F. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1603670 · Cited by 14
Safety of Gadolinium-based Contrast Agents in Children: A Systematic Review and Meta-Analysis
Yu, O. J., Kim, P. H., Yoon, H. M., Jung, A. Y., Cho, Y. A., Lee, J. S., Suh, C. H., Hwang, J., Ryu, Y. J., Kang, H., et al. (2025) · Radiology · doi:10.1148/radiol.241224 · Cited by 13
Lim, S. J., Suh, P. S., Suh, C. H., Kim, P. H., Park, K. J., Park, H. J., & Lee, C. W. (2024) · European Radiology · doi:10.1007/s00330-024-11114-7 · Cited by 13
Sousa, H., Abade, E., Maia, F., Costa, J. A., & Marcelino, R. (2025) · Sport Sciences for Health · doi:10.1007/s11332-024-01310-9 · Cited by 13
Park, Y., Chang, S. J., Kim, H. J., & Jeong, H. N. (2024) · Journal of Korean Gerontological Nursing · doi:10.17079/jkgn.2024.00353 · Cited by 12
Song, H. J., Seo, H., Choi, E. J., Lee, J. S., & Choi, Y. (2024) · Journal of Cancer Survivorship · doi:10.1007/s11764-024-01661-9 · Cited by 12
Self-citationDysphagia development in heart failure patients: A scoping review
Suzuki, M., Saino, Y., Nagami, S., Ueshima, J., Inoue, T., Nagano, A., Kawase, F., Kobayashi, H., Murotani, K., & Maeda, K. (2024) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2024.105728 · Cited by 12
Choi, Y., Noh, S., Seo, H., & Yoon, J. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-087528 · Cited by 10
Self-citationLiu, S., Augustin, G., Ning, N., & Ota, E. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105134 · Cited by 9
Ku, N. S., Lee, Y., & Park, D. W. (2025) · The Korean Journal of Internal Medicine · doi:10.3904/kjim.2025.037 · Cited by 9
Self-citationOral and Gingival Crevicular Fluid Biomarkers for Jawbone Turnover Diseases: A Scoping Review
Fadli, N. A., Rahman, M. A., Karsani, S. A., & Ramli, R. (2024) · Diagnostics · doi:10.3390/diagnostics14192184 · Cited by 9
Jannah, W., Khoiry, Q. A., Alfian, S. D., & Abdulah, R. (2025) · Patient Preference and Adherence · doi:10.2147/ppa.s492461 · Cited by 8
A Systematic Review on Health Impact of Electronic Cigarettes in South Korea
Cheon, E., Kim, H., Kang, N., Park, S., Cho, S., & Hwang, J. (2024) · Journal of the Korean Society for Research on Nicotine and Tobacco · doi:10.25055/jksrnt.2024.15.2.29 · Cited by 8
Li, Q., Zhang, Y., Wang, X., Dai, L., & Zhao, W. (2025) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2025.1444793 · Cited by 7
Senorski, R. H., Högberg, J., Teow, K., Nordenholm, A., Kaarre, J., Snæbjörnsson, T., Musahl, V., Samuelsson, K., & Senorski, E. H. (2025) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1002/ksa.70069 · Cited by 7
Lee, J. M., Suh, H.-W., Lee, H., Choi, M., Kim, J. S., Lee, K., Kim, S.-H., Sohn, J. W., Yoon, H. J., Paek, Y., et al. (2025) · Cancers · doi:10.3390/cancers17223623 · Cited by 7
Self-citationAndriolo, L., Fermín, T. M., Gaggia, G. M. M. C., Serner, A., Kon, E., Papakostas, E., Massey, A., VERDONK, P. C. M., & Filardo, G. (2024) · Cartilage · doi:10.1177/19476035231224951 · Cited by 7
Physiological and Biomechanical Characteristics of Inline Speed Skating: A Systematic Scoping Review
Zheng-zhi, W. U., Cardoso, F., Pyne, D. B., Goethel, M. F., & Fernandes, R. J. (2025) · Applied Sciences · doi:10.3390/app15147994 · Cited by 6
Alrasheed, A. S., Almaqboul, T. M., Alshamrani, R. A., AlMohish, N. M., & Alabdali, M. M. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13226713 · Cited by 6
Renzaho, A. M. N., Abdulai, R., Rahaman, K. S., Das, S. K., Noor, R. A., Salami, B. O., & Chitekwe, S. (2025) · Agriculture & Food Security · doi:10.1186/s40066-025-00554-z · Cited by 5
Abdelhameed, F., Lagojda, L., Kite, C., Dallaway, A., Mustafa, A., Than, N. N., Kassi, E. N., Randeva, H. S., & Kyrou, I. (2025) · Frontiers in Endocrinology · doi:10.3389/fendo.2025.1574842 · Cited by 5
Strong, K. M., Emdin, F., Orubu, S., Katwyk, S. R. V., Ganshorn, H., Grimshaw, J., & Poirier, M. J. P. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02829-9 · Cited by 5
Kowalski, R., Park, A. N., Abdulgader, L., Shelawala, N., Don, R., Ryan, K. A., Johnson, A. M., McArdle, P. F., Terhune, J. H., Kuo, J. H., et al. (2025) · Thyroid · doi:10.1177/10507256251393520 · Cited by 4
Lee, T.-K., Lee, S.-Y., & Cho, E. (2026) · Health and Quality of Life Outcomes · doi:10.1186/s12955-026-02494-z · Cited by 4
Muscle collagen accumulation is not a universal feature of human aging: a systematic review
Schweitzer, A. M., Abercrombie, M. J., Losciale, J. M., & Mitchell, C. J. (2026) · American Journal of Physiology-Cell Physiology · doi:10.1152/ajpcell.00789.2025 · Cited by 4
Local Salvage Therapy Alone for Local Recurrence of Prostate Cancer After Radiotherapy
Miszczyk, M., Mastroleo, F., Kujawska, A., Gurwin, A., Marvaso, G., Kói, T., Calleris, G., Rasing, M. J. A., Chin, J. L., Fuller, D. A., et al. (2026) · JAMA Oncology · doi:10.1001/jamaoncol.2026.1023 · Cited by 4
Guidelines for the Treatment of Laryngeal Cancer from the Korean Society of Head and Neck Surgery
Jang, J. Y., Kim, G., Kim, S., Kim, M.-S., Lee, D. K., Kwon, M., Ahn, D., Ban, M. J., Kang, Y. A., Won, H.-R., et al. (2025) · Clinical and Experimental Otorhinolaryngology · doi:10.21053/ceo.2025.00009 · Cited by 4
Self-citationBrassington, R. J., Mara, J. K., Ball, N. B., Waddington, G. S., Paul, K., & Cooke, J. (2025) · Biology of Sport · doi:10.5114/biolsport.2026.153306 · Cited by 4
Ghatak, T., Gupta, B., Samanta, S., Dube, B. N., & Halemani, K. (2025) · Annals of Cardiac Anaesthesia · doi:10.4103/aca.aca_16_25 · Cited by 4
Emdin, F., Ong, S. W. X., McGall, C., Leung, V., Schwartz, K. L., Langford, B. J., Brown, K. A., Massarella, S., & Daneman, N. (2025) · BMC Veterinary Research · doi:10.1186/s12917-025-04722-y · Cited by 3
Sex differences in association football: a scoping review
Xu, W., Zhang, D., & Zhang, X. (2025) · PeerJ · doi:10.7717/peerj.19976 · Cited by 3
Seo, Y. B., Lee, H., Paek, Y., Lee, H., Lee, C. M., Lee, E. S., Kimm, H., An, H.-J., Bae, E.-J., Kim, J. S., et al. (2025) · Journal of Korean Medical Science · doi:10.3346/jkms.2025.40.e299 · Cited by 3
Self-citationSingh, A., Pokarram, S. S., & Podder, S. (2026) · Journal of Prosthetic Dentistry · doi:10.1016/j.prosdent.2025.12.019 · Cited by 3
Themistocleous, I.-C., Michael, C., Hadjisavvas, S., Papamichael, E., Efstathiou, M. A., Michailidou, C., & Stefanakis, M. (2026) · Journal of Functional Morphology and Kinesiology · doi:10.3390/jfmk11020164 · Cited by 3
Peng, W., Zhang, Y., Wang, D., Sarmento, H., Vilas-Boas, J. P., & Ribeiro, J. (2026) · Frontiers in Physiology · doi:10.3389/fphys.2026.1818415 · Cited by 3
Shukla, N., Akram, Z., Kumar, P. G. N., Khairnar, M., Jadhav, S. K., & Priyadarsini, S. (2024) · Evidence-Based Dentistry · doi:10.1038/s41432-024-01092-6 · Cited by 3
Wu, C., Zhang, C., Li, X., & Astikainen, P. (2025) · Cognition & Emotion · doi:10.1080/02699931.2025.2568559 · Cited by 3
Ryoo, S., Choi, M., Choi, N., Shin, H., Woo, J. H., Park, B., & Oh, S. (2024) · Osong Public Health and Research Perspectives · doi:10.24171/j.phrp.2023.0325 · Cited by 3
Halim, C., Howen, M., Fitrisubroto, A. A. N. B., Pratama, T., Harahap, I. R., Ganesh, L. J., & Siahaan, A. M. P. (2024) · PLoS ONE · doi:10.1371/journal.pone.0301564 · Cited by 3
Occupational risk factors for lumbar spinal stenosis: a systematic review
Kim, S. Y., Yang, M., Salo, S., Sund, R., Sirola, J. P., Kröger, H., Yoo, H., & Kang, M. (2025) · Occupational Medicine · doi:10.1093/occmed/kqaf005 · Cited by 3
Self-citationKurniati, A. M., Partan, R. U., Lestari, P. M., Liberty, I. A., Zulkarnain, M., Effendi, K. Y., & Indra, B. (2025) · PubMed Central · doi:10.34763/jmotherandchild.20252901.d-25-00006 · Cited by 2
Halim, C., Teruna, B. P., Harahap, I. R., Fernando, E. T., & Nugroho, L. B. C. (2025) · Global Cardiology Science and Practice · doi:10.21542/gcsp.2025.41 · Cited by 2
Kim, E., Lee, H. I., Roh, T. H., Kim, Y. Z., Lim, D. H., Cho, S. J., Hwang, K., & Wee, C. W. (2025) · Journal of Neuro-Oncology · doi:10.1007/s11060-025-05265-w · Cited by 2
Prehospital Interventions Provided by Helicopter Emergency Medical Services Teams: A Scoping Review
Hu, X., Wu, D., Jiang, W., & Chen, C. (2025) · Journal of Emergency Nursing · doi:10.1016/j.jen.2025.10.011 · Cited by 2
Clinical pharmacokinetics on canagliflozin: a systematic review of in-vitro and in-vivo studies
Eden, N. E., Zamir, A., Saeed, H., Alqahtani, F., & Rasool, M. F. (2025) · Expert Review of Clinical Pharmacology · doi:10.1080/17512433.2025.2597512 · Cited by 2
Belsey, J., Reid, A., Hannah, S., Johnson, L., & Faulkner, J. (2026) · Clinical Rehabilitation · doi:10.1177/02692155251411864 · Cited by 2
Mulder, P. P. G., van Hooren, M., Bumbuc, R. V., Hooijmans, C. R., Korkmaz, H. I., & Boekema, B. K. H. L. (2026) · Frontiers in Immunology · doi:10.3389/fimmu.2026.1793945 · Cited by 2
Association of the ACTN3 Genotype with Muscle Function: A Systematic Review and Meta-analysis
Kamiya-Saito, M., Zempo, H., de Almeida, K. Y., Homma, H., Mochizuki, Y., Deguchi, M., Kozuma, A., Tashiro, S., & Kikuchi, N. (2026) · International Journal of Sports Medicine · doi:10.1055/a-2799-1026 · Cited by 2
Sánchez‐Marco, M., Montoro‐Pérez, N., Rubio‐Aparicio, M., Cabañero‐Martínez, M. J., Lozano‐Casanova, M., & Escribano, S. (2026) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2026.112230 · Cited by 2
Digital health interventions for improving nutritional behaviors in older adults: a scoping review
Buja, A., & Grotto, G. (2026) · Frontiers in Nutrition · doi:10.3389/fnut.2026.1743189 · Cited by 2
Rijnhart, J. J. M., Rabbers, A., & Rizzuto, S. (2024) · International Journal of Epidemiology · doi:10.1093/ije/dyae147 · Cited by 2
Kim, Y. R., Lee, C. H., Park, H., Kim, J.-H., & Kim, C. H. (2025) · Scientific Reports · doi:10.1038/s41598-024-84636-1 · Cited by 2
Exploring the Pharmacokinetics of Drugs in Disabled Saudi Patients: A Systematic Review
Alqahtani, F., Awadh, S. A. A., & Rasool, M. F. (2025) · Pharmaceuticals · doi:10.3390/ph18040582 · Cited by 2
Indra, B., Qodir, N., Pramudhito, D., Legiran, L., Hafy, Z., & Yusran, A. M. I. (2025) · Journal of the Egyptian National Cancer Institute · doi:10.1186/s43046-025-00293-z · Cited by 2
Ahn, D., Kwak, J. H., Lee, G. J., & Sohn, J. H. (2024) · European Radiology · doi:10.1007/s00330-024-10709-4 · Cited by 1
Tănase, B. C., Horvat, T., Davidescu, M. D., Nistor, C., Muntean, C., Cozma, G. V., Nicola, A. C., Bratosin, F., Laitin, S. M. D., & Burlacu, A. I. (2025) · Healthcare · doi:10.3390/healthcare13080957 · Cited by 1
Messina, S., Ferro, S., Santonocito, C., Minardi, C., Zawadka, M., Drago, F., Noto, A., & Sanfilippo, F. (2025) · Anaesthesia Critical Care & Pain Medicine · doi:10.1016/j.accpm.2025.101579 · Cited by 1
Matsumoto, T., Yoshimatsu, R., Osaki, M., Shibata, J., Miyatake, K., Yamanishi, T., & Yamagami, T. (2025) · Japanese Journal of Radiology · doi:10.1007/s11604-025-01874-7 · Cited by 1
Belfield, K. D., Walter, K. S., Marvin, J. E., Bonner, R. W., Carlson, R., & Szempruch, K. R. (2025) · Transplantation Reviews · doi:10.1016/j.trre.2025.100970 · Cited by 1
Hu, X., Jiang, W., Liu, S., Wu, D., & Chen, C. (2025) · BMC Emergency Medicine · doi:10.1186/s12873-025-01419-1 · Cited by 1
Smith, M. L., Oldenhof, E., Byrne, J. E. M., McNeill, I. M., Sweetman, A., & Staiger, P. K. (2026) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2026.102269 · Cited by 1
Familial and genetic overlap between Sjögren’s disease and other autoimmune diseases
Minagi, H. O., Ohnishi, T., Atyeo, N., Okuno, K., Burbelo, P. D., & Chiorini, J. A. (2026) · Frontiers in Immunology · doi:10.3389/fimmu.2026.1740360 · Cited by 1
Yoon, J., Jun, Y., Lee, H. S., Yoo, J. H., & Park, J. J. (2026) · Inflammatory Bowel Diseases · doi:10.1093/ibd/izag040 · Cited by 1
Self-citationMustafa, A., Dallaway, A., Lagojda, L., Kite, C., Abdelhameed, F., Chatha, K. K., Suresh, A., Randeva, H. S., & Kyrou, I. (2026) · Frontiers in Nutrition · doi:10.3389/fnut.2026.1778393 · Cited by 1
Histopathological Evidence of Neurodegenerative Pathology in Epilepsy: A Systematic Review
Hashmi, S. A., Łuniewski, A., Smith, V. L., McCord, M., Kapur, J., Quigg, M., Joshua, C., & Zawar, I. (2026) · Annals of Clinical and Translational Neurology · doi:10.1002/acn3.70486 · Cited by 1
Circulating CTRP1 in Adults with Type 2 Diabetes Mellitus: A Systematic Review and Meta-Analysis
Suresh, A., Dallaway, A., Mustafa, A., Lagojda, L., Kite, C., Kyrou, I., & Randeva, H. S. (2026) · International Journal of Molecular Sciences · doi:10.3390/ijms27156775 · Cited by 1
The protective effects of hyperbaric oxygen on ionising radiation injury: A systematic review
Holyer, A., Stewart, T., & Ashworth, E. T. (2025) · Acta Astronautica · doi:10.1016/j.actaastro.2025.03.011 · Cited by 1
Systematic Review of the Diagnostic Imaging Evaluation of Pulsatile TinnitusPreprint
Jairam, M. P., Kidanemariam, S., Malik, A., Corrales, C. E., Suh, C. H., & Guenette, J. P. (2025) · medRxiv · doi:10.1101/2025.02.25.25322858 · Cited by 1
Park, S. A., & Kim, H. (2026) · medRxiv · doi:10.64898/2026.02.26.26347155 · Cited by 1
Lee, H. R., Kim, M.-S., Shin, S., Jang, J. Y., Park, J.-O., Baek, S., & Cho, K. J. (2025) · Korean Journal of Otorhinolaryngology - Head and Neck Surgery · doi:10.3342/kjorl-hns.2025.00052
Self-citationJo, E., & Kim, S. (2025) · Asian Oncology Nursing · doi:10.5388/aon.2025.25.2.39
Self-citationFlemban, A., Bakhor, H. A., Almuzaini, R., Alshahrani, T., Alahmadi, R. A., Alshurbi, R., Althobiti, K., Alwadai, H., Maghrabi, S., Alkhairy, N., et al. (2025) · Cureus · doi:10.7759/cureus.90352
Alharbi, S. M. A., Alsanosi, A. A., Ardi, T. E., Alahmari, Y. D., Assiri, M., Alhamoud, M. A. S., & Alshehri, M. A. (2025) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-025-09792-0
Nallamuthu., P., & Gandhimathi, S. N. S. (2025) · Forum for Linguistic Studies · doi:10.30564/fls.v7i12.12159
Choi, Y., Seo, H., Lee, S. J., Park, E., Yoon, J., & Noh, S. (2025) · Health Education Research · doi:10.1093/her/cyaf050
Self-citationRelación entre la enfermedad periodontal y el cáncer de mama: revisión sistemática
Barrantes-Rafael, J. Y., Flaviano, V. S. D., Egido‐Moreno, S., Roselló-Llabrés, X., Marí-Roig, A., & López-López, J. (2025) · Scientific Electronic library online (Sciences Carlos III Health Institute)
Samhat, H. (2025) · eScholarship@McGill (McGill) · doi:10.82308/32711
Stanberry, B., Deloose, K. R., Gouëffic, Y., Goyault, G., Isernia, G., Jackson, R., Kucher, N., Lichtenberg, M. K. W., McCaslin, J. E., Migliara, B., et al. (2026) · Journal of Endovascular Therapy · doi:10.1177/15266028261467201
Takatani, S., Nakaguchi, H., Honda, J., Soejima, T., Kitao, M., Lin, Q., & Nishimura, N. (2026) · Cureus · doi:10.7759/cureus.101377
Park, Y., Yun, J.-S., Kim, S., & Park, J.-H. (2026) · Australian Critical Care · doi:10.1016/j.aucc.2025.101529
Self-citationVásquez-Rojas, S. P., Moscol-Zapata, G. E., & Santamaría, R. O. M. (2026) · F1000Research · doi:10.12688/f1000research.175064.1
Outpatient versus inpatient thyroidectomy: A systematic review and meta‐analysis
Mehmood, Y., AlShammari, A. O. K., Alanazi, A. H. A., Alanazi, H. A. K., Alanazi, J. F., Alaradi, T. S. E., Alhazmi, Z. A. H., Alenezi, A. B. N., Almafadhilah, S. F. S., Alanazi, A. A. S., et al. (2026) · Surgical Practice · doi:10.1111/1744-1633.70050
Nwankwo, L., Maidment, I., Periselneris, J. N., Jackson, D. J., Agarwal, R., Asano, K., Nuh, A., Ni, M. Z., Shah, A., & Armstrong-James, D. P. H. (2026) · Systematic Reviews · doi:10.1186/s13643-026-03113-0
Kang, E., Cha, J. M., Kang, S. Y., Lee, K., Kim, S. Y., Kim, Y.-H., Seo, A. N., Kang, H.-J., Jang, J. K., Ko, K., et al. (2026) · Journal of Korean Medical Association · doi:10.5124/jkma.26.0002
Self-citationHazewinkel, A., Curnow, E., Tilling, K. M., & Hughes, R. A. (2026) · BMC Medical Research Methodology · doi:10.21203/rs.3.rs-9283156/v1
Larassagita, A. F., Widantara, H., Romadhona, E. I., Aslia, A., Aliah, R. S., Anjani, T. P., Bawono, Y. P., Nuraini, D. M., & Andityas, M. (2026) · Journal of Advanced Veterinary Research/Journal of Advanced Veterinary Research
PEREIRA, J. H. O. S., Ferreira, F. A., & Lima, V. (2026) · Healthcare · doi:10.3390/healthcare14080998
Metadata
- Authors
- Seo, H.-J., Kim, S. Y., Lee, Y. J., & Park, J.-E.
- Affiliations
- Chungnam National University (KR)
- Hallym University (KR)
- Jaseng Medical Foundation (KR)
- Corresponding author
- Kim, Soo Young
- Venue
- Korean Journal of Family Medicine
- Published
- 2023-09-20
- Volume
- 44
- Issue
- 5
- Pages
- 249-260
- ISSN
- 2005-6443 (ISSN-L)
- 2092-6715
- PMC
- PMC10522469
- OpenAlex
- W4383550757
- Cited by
- 128
- References
- 23
- Citations by year
- 2026: 55 · 2025: 52 · 2024: 21
- FWCI
- 13.06
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Korean Journal of Family Medicine (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- Health Insurance Review and Assessment Service
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- risk of bias assessment, non-randomized studies, selection bias, detection bias, outcome measure validity, inter-rater reliability, construct validity, psychometric properties, systematic review methodology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
