Sampling Issues in Bibliometric Analysis
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Another possibility for analyzing larger data sets is to use additional online tools, such as InCites for WoS data and SciVal for Scopus data.
“Another possibility for analyzing larger data sets is to use additional online tools, such as InCites (for WoS data) and SciVal (for Scopus data) [26,219].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [219]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,385 · see citing works
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828 · see citing works
What do citation counts measure? A review of studies on citing behavior
Bornmann, L., & Daniel, H. (2008) · Journal of Documentation · doi:10.1108/00220410810844150 · Cited by 1,607 · see citing works
Self-citationWilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S. A., Jones, R. D., Kain, R. J. P., Kerridge, S., Thelwall, M. A., et al. (2015) · Digital Education Resource Archive (University College London) · doi:10.13140/rg.2.1.4929.1363 · Cited by 738 · see citing works
Schubert, A. P., & Braun, T. (1986) · Scientometrics · doi:10.1007/bf02017249 · Cited by 450 · see citing works
The citation process: The role and significance of citations in scientific communication
Cronin, B. (1984) · International Linear Collider · Cited by 358 · see citing works
Bornmann, L., Loet Leydesdorff, A., & Mutz, R. (2012) · Journal of Informetrics · doi:10.1016/j.joi.2012.10.001 · Cited by 205 · see citing works
Self-citationStatistical Inference for Apparent Populations
Berk, R. A., Western, B., & Weiss, R. E. (1995) · Sociological Methodology · doi:10.2307/271073 · Cited by 180 · see citing works
Tobias Opthof, A., & Loet Leydesdorff, A. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.02.003 · Cited by 173 · see citing works
Bornmann, L. (2014) · Research Evaluation · doi:10.1093/reseval/rvu002 · Cited by 124 · see citing works
Self-citationSchneider, J. (2014) · Scientometrics · doi:10.1007/s11192-014-1251-5 · Cited by 111 · see citing works
Bornmann, L., & Marx, W. (2012) · EMBO Reports · doi:10.1038/embor.2013.9 · Cited by 92 · see citing works
Self-citationLassen sich Signifikanztests auf Vollerhebungen anwenden? Einige essayistische Anmerkungen
Behnke, J. (2005) · Politische Vierteljahresschrift · doi:10.1007/s11615-005-0240-y · Cited by 90 · see citing works
Bornmann, L. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22792 · Cited by 87 · see citing works
Self-citationBornmann, L., & Williams, R. A. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2013.02.005 · Cited by 76 · see citing works
Self-citationApparent and Nonapparent Significance Tests
Bollen, K. A. (1995) · Sociological Methodology · doi:10.2307/271074 · Cited by 71 · see citing works
Bornmann, L., Mutz, R., Marx, W., Schier, H., & Daniel, H. (2011) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2011.00689.x · Cited by 70 · see citing works
Self-citationCaveats for using statistical significance tests in research assessments
Schneider, J. (2011) · J. Informetrics · doi:10.1016/j.joi.2012.08.005 · Cited by 70 · see citing works
Caveats for using statistical significance tests in research assessments
Schneider, J. W. (2013) · RePEc: Research Papers in Economics · Cited by 68 · see citing works
On the Interpretation of Censuses as Samples
Deming, W. E., & Stephan, F. F. (1941) · Journal of the American Statistical Association · doi:10.1080/01621459.1941.10502070 · Cited by 55 · see citing works
The problem of citation impact assessments for recent publication years in institutional evaluations
Bornmann, L. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2013.05.002 · Cited by 44 · see citing works
Self-citationWill Bayesian Inference Help? A Skeptical View
Firebaugh, G. (1995) · Sociological Methodology · doi:10.2307/271075 · Cited by 42 · see citing works
Williams, R. A., & Bornmann, L. (2014) · Springer eBooks · doi:10.1007/978-3-319-10377-8_12 · Cited by 18 · see citing works
Self-citationThe advantage of the use of samples in evaluative bibliometric studies
Bornmann, L., & Mutz, R. (2012) · Journal of Informetrics · doi:10.1016/j.joi.2012.08.002 · Cited by 17 · see citing works
Self-citationReply to Bollen, Firebaugh, and Rubin
Berk, R. A., Western, B., & Weiss, R. E. (1995) · Sociological Methodology · doi:10.2307/271077 · Cited by 16 · see citing works
Bayes, Neyman, and Calibration
Rubin, D. B. (1995) · Sociological Methodology · doi:10.2307/271076 · Cited by 13 · see citing works
On the Interpretation of Censuses as Samples
Deming, W. E., & Stephan, F. F. (1941) · Journal of the American Statistical Association · doi:10.2307/2278811 · Cited by 6 · see citing works
(2009)
Flagged first, then by citations
Research works printed but not linked: 3
- Levy, P., & Lemeshow, S. (2008). Sampling of population -methods and applications (4 ed.). New York, NY, USA: Wiley.
- Schneider, J. (2012). Testing university rankings statistically: why this is not such a good idea after all. Some reflections on statistical power, effect sizes, random sampling and imaginary populations. In E. Archambault, Y. Gingras & V. Lariviere (Eds.), The 17th International Conference on Science and Technology Indicators (pp. 719-732). Montreal, Canada: Repro-UQAM.
- StataCorp. (2013). Stata power and sample size reference manual Release 13. College Station, Texas: Stata Press.
Cited by
Sorted by citations
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696 · see citing works
Bornmann, L., & Haunschild, R. (2018) · PLoS ONE · doi:10.1371/journal.pone.0197133 · Cited by 143 · see citing works
Self-citationThree practical field normalised alternative indicator formulae for research evaluation
Thelwall, M. A. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2016.12.002 · Cited by 104 · see citing works
Rupp, M., Schneckenburger, M., Merkel, M., Börret, R., & Harrison, D. K. (2021) · Journal of Open Innovation Technology Market and Complexity · doi:10.3390/joitmc7010068 · Cited by 90 · see citing works
Bornmann, L., & Williams, R. A. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.06.001 · Cited by 79 · see citing works
Self-citationWondimagegn, D., Whitehead, C. R., Cartmill, C., Rodrigues, E., Correia, A., Salessi Lins, T., & Costa, M. J. (2023) · BMJ Global Health · doi:10.1136/bmjgh-2022-011656 · Cited by 66 · see citing works
Normalisation of citation impact in economics
Bornmann, L., & Wohlrabe, K. (2019) · Scientometrics · doi:10.1007/s11192-019-03140-w · Cited by 62 · see citing works
Self-citationThree practical field normalised alternative indicator formulae for research evaluation
Thelwall, M. A. (2017) · RePEc: Research Papers in Economics · Cited by 57 · see citing works
Bornmann, L., & Haunschild, R. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.01.010 · Cited by 39 · see citing works
Self-citationBornmann, L., Wagner, C., & Leydesdorff, L. (2018) · PLOS ONE · doi:10.1371/journal.pone.0194805 · Cited by 39 · see citing works
Self-citationInterrater reliability and convergent validity of F 1000 P rime peer review
Bornmann, L. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23334 · Cited by 34 · see citing works
Self-citationWhat are we measuring? Refocusing on some fundamentals in the age of desktop bibliometrics
Rowlands, I. (2018) · FEMS Microbiology Letters · doi:10.1093/femsle/fny059 · Cited by 31 · see citing works
Matthew effects in science and the serial diffusion of ideas: Testing old ideas with new methods
Farys, R., & Wolbring, T. (2021) · Quantitative Science Studies · doi:10.1162/qss_a_00129 · Cited by 33 · see citing works
Ribeiro-Navarrete, B., Saura, J. R., & Simón-Moya, V. (2023) · Review of Managerial Science · doi:10.1007/s11846-023-00663-8 · Cited by 31 · see citing works
The nexus of artificial intelligence and entrepreneurship research: Bibliometric analysis
Mumi, A., Ngammoh, N., & Suwanpakdee, A. (2025) · Sustainable Futures · doi:10.1016/j.sftr.2025.100688 · Cited by 29 · see citing works
Workplace spirituality: exploring past, present and future through bibliometric analysis
Barik, S., & Nayak, T. (2023) · Benchmarking An International Journal · doi:10.1108/bij-04-2022-0251 · Cited by 24 · see citing works
Wang, X., & Feng, X. (2022) · Library Hi Tech · doi:10.1108/lht-09-2021-0296 · Cited by 23 · see citing works
Mutz, R., & Daniel, H. (2019) · Journal of Informetrics · doi:10.1016/j.joi.2019.03.007 · Cited by 22 · see citing works
Sustainability in the global value chain—a scientometric analysis
Jacob, N. R., Aggarwal, S., Saini, N., Wahid, R., & Sarwar, S. (2023) · Environmental Science and Pollution Research · doi:10.1007/s11356-023-29381-0 · Cited by 20 · see citing works
Conceptual difficulties in the use of statistical inference in citation analysis
Waltman, L. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.09.012 · Cited by 16 · see citing works
Confidence intervals for Journal Impact Factors
Bornmann, L. (2017) · Scientometrics · doi:10.1007/s11192-017-2365-3 · Cited by 15 · see citing works
Self-citationStefanis, C., Stavropoulou, E., Stavropoulos, A., Gyriki, D., Nikolaidis, C. G., Vassos, V., Nena, E., Tsigalou, C., & Bezirtzoglou, E. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-07653-3 · Cited by 14 · see citing works
The accuracy of confidence intervals for field normalised indicators
Thelwall, M. A., & Fairclough, R. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.03.004 · Cited by 13 · see citing works
Sharek, A. S., & Shah, K. U. (2021) · Journal of Environmental Studies and Sciences · doi:10.1007/s13412-021-00681-6 · Cited by 9 · see citing works
Transformation and overview of audit in past 20 years: a bibliometric analysis
Cumbana, D. A. P. T., & Ventura, J. B. (2024) · Cogent Business & Management · doi:10.1080/23311975.2024.2395434 · Cited by 8 · see citing works
There should not be any mystery: A comment on sampling issues in bibliometrics
Claveau, F. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.09.009 · Cited by 8 · see citing works
Schneider, J. W. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.09.011 · Cited by 8 · see citing works
Bornmann, L. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23546 · Cited by 7 · see citing works
Self-citationGreen Infrastructure for Urban Flooding: Knowledge Domains and Research Evolution (2015–2024)
Kim, J.-P., & Kim, J.-O. (2025) · Land · doi:10.3390/land14050921 · Cited by 6 · see citing works
Chen, G., Chen, S., Chen, Z., Xiao, L., & Hu, J. (2025) · Scientometrics · doi:10.1007/s11192-025-05335-w · Cited by 6 · see citing works
Klona, M., Menzies, J. L., & Zutshi, A. (2021) · International Journal of Conflict Management · doi:10.1108/ijcma-12-2020-0200 · Cited by 6 · see citing works
Tost, C. M., & Rindermann, H. (2017) · Psychologische Rundschau · doi:10.1026/0033-3042/a000325 · Cited by 5 · see citing works
Barile, D., Lorenzi, M. C. D., Latino, M. E., & Secundo, G. (2026) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.70430 · Cited by 4 · see citing works
Hasija, T., Ramkumar, K. R., Kaur, A., & Bali, M. S. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01269-5 · Cited by 3 · see citing works
Night Shift Work and Weight Gain among Female Filipino Nurses
Apellido, R. (2018) · Journal of Obesity and Overweight · doi:10.15744/2455-7633.4.106 · Cited by 3 · see citing works
Thelwall, M. A. (2017) · RePEc: Research Papers in Economics · Cited by 3 · see citing works
Федорук, О. (2025) · Society Document Communication · doi:10.69587/sdc/1.2025.55 · Cited by 2 · see citing works
Some further aspects of sampling: Comment on Williams and Bornmann
Mutz, R. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.09.007 · Cited by 2 · see citing works
To infer or not to infer? A comment on Williams and Bornmann
Nane, G. F. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.09.010 · Cited by 2 · see citing works
Khany, R., Fahimpour, Z., & Teo, T. P. (2025) · Research Synthesis in Applied Linguistics · doi:10.1080/29984475.2025.2522452 · Cited by 2 · see citing works
Thelwall, M. A. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.09.002 · Cited by 4 · see citing works
Normalization of zero-inflated data: An empirical analysis of a new indicator familyPreprint
Haunschild, R., & Bornmann, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.02211 · Cited by 2 · see citing works
Self-citationBibliometric Analysis of New Zealand Research Performance: Measurement and Classification IssuesPreprint
Jaffe, A. B., & Preston, K. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3187107 · Cited by 2 · see citing works
Colliander, C. (2014) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 2 · see citing works
Singh, D., Syed, R. T., Tariq, U. M., Dalwai, T. A. R., & Ananda, S. (2025) · Sustainable Futures · doi:10.1016/j.sftr.2025.101032 · Cited by 1 · see citing works
Bornmann, L., Bauer, J., & Schlagberger, E. M. (2018) · Collnet Journal of Scientometrics and Information Management · doi:10.1080/09737766.2017.1332610 · Cited by 1 · see citing works
Self-citationBornmann, L. (2018) · Online Information Review · doi:10.1108/oir-08-2015-0276 · Cited by 1 · see citing works
Self-citationTomada de decisão em sistemas ciber-físicos: uma análise bibliométrica
Santana, H. L., & Assumpção, M. R. P. (2021) · Exacta · doi:10.5585/exactaep.2021.13163
Tomada de decisão em sistemas ciber-físicos: uma análise bibliométrica
Santana, H. L., & Assumpção, M. R. P. (2021) · Exacta · doi:10.5585/exactaep.0.13163
Alptekin, H. D., & DOĞAN, S. (2022) · Pazarlama ve Pazarlama Araştırmaları Dergisi · doi:10.15659/ppad.15.3.1068856
Analyzing public–private partnerships in the social economy framework: a bibliometric perspective
Montesinos-Sansaloni, E. X., Soriano, D. R., & Botella-Carrubí, D. (2025) · Review of Managerial Science · doi:10.1007/s11846-025-00930-w
Analysis on Research Situation of Soybean Quality Evaluation Based on Bibliometrics
Gao, Y., Tang, P., Tang, X., Wang, D., Luo, J., & Wu, J. (2025) · Big Data · doi:10.1177/2167647x251399053
Singh, P. D., Kour, K., Hasija, T., & Ketti, R. R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6302999
Almadhloum, S. B., Mushtaha, E. S. N., & Ma'bdeh, S. (2026) · ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS) · doi:10.1109/icetsis68266.2026.11549370
Nguyen, Q., Nguyen, P. V., & Hansen, U. E. (2026) · SAGE Open · doi:10.1177/21582440261466354
Artificial Intelligence for Data Analysis
Ribeiro, E. C., González-Padilla, P., & Hernández-Tamurejo, Á. (2026) · Contributions to management science · doi:10.1007/978-3-032-22360-9_12
Colors of hydrogen: A scientometric and topic modeling assessment of hydrogen energy research
Paudel, T., Pan, C.-T., Dhakal, T., Chen, C.-Y., Li, W.-Y., & Tiwari, S. (2026) · Energy Reports · doi:10.1016/j.egyr.2026.109761
- Authors
- Williams, R., & Bornmann, L.
- Affiliations
- University of Notre Dame (US)
- Max Planck Society (DE)
- Corresponding author
- Williams, Richard
- Venue
- Journal of Informetrics
- Publisher
- Elsevier BV
- Published
- 2016-11
- Volume
- 10
- Issue
- 4
- Pages
- 1225-1232
- ISSN
- 1751-1577 (ISSN-L)
- OpenAlex
- W1847453300
- Cited by
- 58
- References
- 27
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 1 · 2023: 4 · 2022: 2 · 2021: 8 · 2019: 2 · 2018: 8 · 2017: 9 · 2016: 5 · 2015: 2 · 2014: 1
- FWCI
- 7.47
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Journal of Informetrics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- citation data, sample size determination, power analysis, statistical inference, significance testing, confidence intervals, effect size, research resources
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
