Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level
Open accessPublished version of a preprint
The DOI matches its Crossref record; 87 of the 91 references it cites were found in a registry or an index; 1 of the 30 sources its claims cite were checked for support.
Abstract
Despite their recognized limitations, bibliometric assessments of scientific productivity have been widely adopted. We describe here an improved method to quantify the influence of a research article by making novel use of its co-citation network to field-normalize the number of citations it has received. Article citation rates are divided by an expected citation rate that is derived from performance of articles in the same field and benchmarked to a peer comparison group. The resulting Relative Citation Ratio is article level and field independent and provides an alternative to the invalid practice of using journal impact factors to identify influential papers. To illustrate one application of our method, we analyzed 88,835 articles published between 2003 and 2010 and found that the National Institutes of Health awardees who authored those papers occupy relatively stable positions of influence across all disciplines. We demonstrate that the values generated by this method strongly correlate with the opinions of subject matter experts in biomedical research and suggest that the same approach should be generally applicable to articles published in all areas of science. A beta version of iCite, our web tool for calculating Relative Citation Ratios of articles listed in PubMed, is available at https://icite.od.nih.gov.
Verification finished with gaps
Some checks didn’t finish. The results below cover the steps that did.
- Registry recordVerified
- Full textRead the full text
- Reference list87 references found
- Reference check87 of 87 references checked
- StatisticsRecomputed
- Claims86 claims anchored to a quote
- Cited sourcesChecked
- Self-checkDidn’t finish — the rest of the report still applies.
- VerificationFindings verified
Overview
CoChat Summary
## TL;DR
The paper introduces Relative Citation Ratio (RCR), an article-level, field-normalized citation metric built from co-citation networks and benchmarked to a peer group. The authors report that RCR correlates strongly with expert judgments, behaves better than several existing bibliometrics in their tests, and is available through the iCite web tool.
## Key findings
- Co-citation networks were used to define an article’s field; in a validation study, diagnostic words were more likely to be shared between an article and papers in its co-citation network than between that article and papers in the same journal, based on 1,397 articles and 249,981 pairwise comparisons within co-citation networks versus 28,516,576 comparisons with same-journal articles.
- In a benchmark set of 35,837 papers published in 2009 by NIH R01 recipients, FCRs based on co-citation networks had lower variance than FCRs based on cited or citing networks, and lower variance than ACRs or JIFs (p < 10−4 for the variance comparisons reported).
- RCR correlated well with subject matter expert ratings across three validation datasets: 2,193 Faculty of 1000-rated articles, 430 STPI-rated articles, and 290 NIH-rated articles.
- In an NIH case study of 88,835 articles from 3,089 continuously funded R01 principal investigators, 53% of investigators in the top RCR quintile stayed there across two 4-year periods, and 53% of those in the bottom quintile stayed at the bottom; among more than 200,000 articles analyzed for FCR stability, only 0.2% had an RCR drop of 0.1 or more between 2012 and 2014, and less than 2% dropped at all.
## Methods
- The metric was built from an article’s co-citation network: papers citing the reference article were used to assemble the set of co-cited papers, whose journal citation rates were averaged to form the field citation rate (FCR); the article’s article citation rate (ACR) was then normalized by FCR and benchmarked to a peer group to yield RCR.
- Citation analyses used the Thomson Reuters Web of Science dataset from 2002–2012 (extended to 2014 for FCR stability analyses), focusing on journals in which NIH R01-funded researchers published; grant and PI data came from NIH RePORTER, SPIRES, and IMPAC-II.
- Validation included cosine-similarity content analysis with IN-SPIRE on 1,397 articles from six journals, expert-comparison analyses, and regression/variance tests; RCR benchmarking used OLS for large-scale analyses and quantile regression as an alternative for smaller, skewed settings.
## Limitations
- The paper states that RCR cannot be calculated immediately after publication; enough citations must accrue first, and the authors note that 93% of co-citation-network-based FCRs stabilize after a paper has been cited five times.
- The authors explicitly say they have not determined whether small differences in RCR, such as 1.0 versus 1.2, reliably reflect meaningful differences in influence.
- The paper also states that citation metrics cannot fully capture article value, are not a substitute for reading the paper, and RCR is not designed as a measure of long-term impact or as a metric for applied research; these are stated limitations of scope.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



![Fig 4 RCRs correspond with expert reviewer scores. (A–C) Bubble plots of reviewer scores versus RCR for three different datasets. Articles are binned by reviewer score; bubble area is proportionate to the number of articles in that bin. (A) F1000 scores for 2,193 R01-funded papers published in 2009. Faculty reviewers rated the articles on a scale of one to three (“good,” “very good,” and “exceptional”, respectively); those scores were summed into a composite F1000 score for each article (S3 Fig). (B) Reviewer scores of 430 HHMI and NIH-funded papers collected by STPI. (C) Scores of 290 R01-funded articles reviewed by experts from the NIH Intramural Research Program. Black line, linear regression. (D) Box-and-whisker plots illustrating the distribution of journal impact factors (JIFs) citations per year (CPY) and RCRs for two areas of NIH-funded research from 2007–2011. Cell biology, n = 5,936; neurological function, n = 5,417. *** p < 0.001, Kruskal-Wallis with Dunn’s multiple comparison test. n.s., not significant. Mean represented by a “+.” (E) Comparison of RCRs (orange) and Thompson Reuters ratios (blue) [17,28] for the same 544 articles with a low denominator. Data points are partially transparent to allow coordinates with multiple papers (darker) to be more clearly identified.](https://figures.cochat.ai/paper-figures/e792b94297fa4418b200b0579a2ea0b9/relative-citation-ratio-rcr-a-new-metric-that-uses-citation-rates-to-measure-figure-4.jpg)


![Fig 7 Properties of RCRs at the article and investigator level. (A, B) Frequency distribution of article-level RCRs (A) and JIFs (B), from 88,835 papers (authored by 3,089 R01-funded principal investigators [PIs]) for which co-citation networks were generated. Article RCRs are well fit by a log-normal distribution (R2 = 0.99), and JIFs less so (R2 = 0.79). (C) Box-and-whisker plots summarizing JIFs for the same papers, binned by impact factor quintile (line, median; box, 25th–75th percentiles; whiskers, 10th–90th percentiles). (D) RCR for the same papers using the same bins by JIF quintile (same scale as C). Although the median RCR for each bin generally corresponds to the impact factor quintile, there is a wide range of article RCRs in each category. (E) Box-and-whisker plots summarizing RCRs of these same papers published in selected journals. In each journal, there are papers with article RCRs surpassing the median RCR of the highest impact factor journals (left three). The impact factor of each journal is shown above. (F, G) Frequency distribution of investigator-level RCRs (F) and JIFs (G), representing the mean values for papers authored by each of 3,089 R01-funded PIs. Dashed line in (F), mode of RCR for PIs.](https://figures.cochat.ai/paper-figures/7111239ea73541aeace7efe3f230790e/relative-citation-ratio-rcr-a-new-metric-that-uses-citation-rates-to-measure-figure-7.jpg)

Verification
Claims the paper makes
1 Valid · 1 Not valid · 18 Not checked · 66 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Strikingly, this analysis showed that diagnostic words are much more likely to be shared between an article and the papers in its co-citation network than between that same article and the papers that appear alongside it in its journal of publication (Fig 2).”
§ Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level > Results > Co-citation Networks Represent an Article’s Area of Influence
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We demonstrate that the values generated by this method strongly correlate with the opinions of subject matter experts in biomedical research”
§ Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level > Abstract
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“though journals with the highest impact factors have the highest median RCR”
§ Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level > Results > Quantifying How Past Influence Predicts Future Performance We next undertook a large case study of all 88,835 articles published by NIH investigators who maintained continuous R01 funding from fiscal year (FY) 2003 through FY2010 to ask how the RCR of publications from individual investigators changed over this 8-y interval. Each of these investigators had succeeded at least once in renewing one or more of their projects through the NIH competitive peer review process. In aggregate, the RCR values for these articles are well matched to a log-normal distribution (Fig 7); in contrast, as noted previously by others, the distribution of impact factors of the journals in which they were published is non-normal (Fig 7A and 7B) [53,54]. Sorting into quintiles based on JIF demonstrates that, though journals with the highest impact factors have the highest median RCR, influential publications can be found in virtually all journals (Fig 7C and 7D); these data call into question the assertion that journal of publication is the strongest quality signal for researchers who are young or otherwise unknown to a field [55]. Focusing on a dozen representative journals with a wide range of JIFs further substantiates the finding that influential science appears in many venues and reveals noteworthy departures from the correlation between JIF and median RCR (see S1 Text). For example, NIH-funded articles in both Organic Letters (JIF = 4.7) and the Journal of the Acoustical Society of America (JIF = 1.6) have a higher median RCR than those in Nucleic Acids Research (JIF = 7.1; Fig 7E). As part of this case study, we also calculated the average RCR and average JIF for papers published by each of the 3,089 NIH R01 principal investigators (PIs) represented in the dataset of 88,835 articles. In aggregate, the average RCR and JIF values for NIH R01 PIs exhibited log-normal distributions (Fig 7F and 7G) with substantially different hierarchical ordering (S8 Fig). This raised a further question concerning PIs with RCR values near the mode of the log-normal distribution (dashed line in Fig 7F): as measured by the ability to publish work that influences their respective fields, to what extent does their performance fluctuate? We addressed this question by dividing the 8-y window (FY2003 through FY2010) in half. Average RCRs in the first time period (FY2003 through FY2006) were sorted into quintiles, and the percentage of PIs in the second time period (FY2007 through FY2010) that remained in the same quintile, or moved to a higher or lower quintile, was calculated (Fig 8). The position of PIs in these quintiles appeared to be relatively immobile; 53% of PIs in the top quintile remained at the top, and 53% of those in the bottom quintile remained at the bottom (Fig 8A). For each PI, we also calculated a weighted RCR (the number of articles multiplied by their average RCR); comparing on this basis yielded almost identical results (Fig 8B). It is worth noting that average FCRs for investigators were extremely stable from one 4-y period to the next (Pearson r = 0.92, Table 2), Since FCRs are the quantitative representation of co-citation networks, this further suggests that each co-citation network is successfully capturing the corresponding investigator’s field of research. Another possible interpretation of the above data is that PI RCRs perform an unbiased random walk from their initial state with a large diffusion rate. Considered from this frame, it could be said that 47% of PIs who started in the top quintile moved out of it during the second 4-y period we analyzed. To test this hypothesis directly, we performed a mean reversion test, which determines whether or not the set of values under consideration will return to an average, or mean, value over time. If drift in PI RCR were simply a random walk, then the change in RCR should by definition be independent of starting RCR, and plotting these two values against each other should result in a straight line with a slope of zero. However, the results show that change in RCR is dependent on starting RCR value (p < 0.001, linear regression analysis, n = 3,089, Fig 8C). Furthermore, randomly shuffling PI RCRs from the second 4-y period gives a slope that is significantly different than that observed for the real data (p < 0.001, extra sum-of-squares F-test, n = 3,089, Fig 8C), ruling out the possibility that these values are randomly sampled from the same distribution in each time interval.
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The RCR metric is ranking invariant when the same number of uncited articles is added to two groups of equal size (Supporting Equations S7–S9 in S1 Text).”
§ Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level > Results > Expert Validation of RCR as a Measure of Influence For the work presented here, we chose as a benchmark the full set of 311,497 RAs published from 2002 through 2012 by NIH R01 awardees. To measure the degree of correspondence between our method and expert opinion, we compared RCRs generated by OLS benchmarking of ACR/FCR values with three independent sets of postpublication evaluations by subject matter experts (details in S1 Text). We compared RCR with expert rankings (Fig 4) for 2,193 articles published in 2009 and evaluated by Faculty of 1000 members (Fig 4A and S3 Fig), as well as rankings of 430 Howard Hughes Medical Institute- or NIH-funded articles published between 2005 and 2011 and evaluated in a study conducted by the Science and Technology Policy Institute (STPI, Fig 4B and S4 Fig), and finally, 290 articles published in 2009 by extramurally funded NIH investigators and evaluated by NIH intramural investigators in a study of our own design (Fig 4C; S5, S6 and S7 Figs). All three approaches demonstrate that RCR values are well correlated with reviewers’ judgments. We asked experts in the latter study to provide, in addition to an overall score, scores for several independent subcriteria: likely impact of the research, importance of the question being addressed, robustness of the study, appropriateness of the methods, and human health relevance. Random Forest analysis [38] indicated that their scores for likely impact were weighted most heavily in determining their overall evaluation (S6 Fig). In addition to correlating with expert opinion, RCR is ranking invariant, which is considered to be a desirable property of bibliometric indicators [39,40]. In short, an indicator is ranking invariant when it is used to place two groups of articles in hierarchical order, and the relative positions in that order do not change when uncited articles are added to each group. The RCR metric is ranking invariant when the same number of uncited articles is added to two groups of equal size (Supporting Equations S7–S9 in S1 Text). RCR is also ranking invariant when the same proportion of uncited articles is added to two groups of unequal size (Supporting Equations S10–S11 in S1 Text). This demonstrates that the RCR method can be used effectively and safely in evaluating the relative influence of large groups of publications.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The average TR ratio for these papers is 1.67, and the average RCR is 0.67.”
§ Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level > Results > Comparison of RCR to Existing Bibliometrics
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
50 more claims
Create a free account to see all 86 claims and to verify the 29 references still open.
- Probably not a substantial claim. 18
- It may state two separate claims as one. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The RCR methodology was validated using citation data from about 90,000 published papers emanating from NIH-funded research and comparing calculated RCRs to the opinions on manuscript reach of recognized experts in selected fields.”
From Structure, function, and productivity from the National Dental Practice-Based Research Network. (2022) · cites this paper as [11]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As one of several methods to quantify publication quality or its impact on the field, the RCR has performed better than journal impact factor, citations per year, or the Thompson–Reuters ratio [11,26,27].”
From Structure, function, and productivity from the National Dental Practice-Based Research Network. (2022) · cites this paper as [11]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The wRCR was considered a measure of influence, with higher values representing the most cited publications 22 .”
From Global mapping of randomized controlled trials in dentistry (2024) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“the Relative Citation Ratio (RCR) was developed to quantify the influence of a research article that is article level and independent of the scientific field [11].”
From Structure, function, and productivity from the National Dental Practice-Based Research Network. (2022) · cites this paper as [11]
The cited source says what the claim attributes to it.
In the citing paper
“That the median RCR of Network publications exceeded 1.0 and the weighted RCR substantially exceeded the total number of publications upon which the analysis was based, demonstrate that the network’s publications have had a greater-than-average influence in their fields [11].”
From Structure, function, and productivity from the National Dental Practice-Based Research Network. (2022) · cites this paper as [11]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
This check didn’t finish.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
(1999) · Cited by 12,854
An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669 · cited 4×
“In recent years, decisionmakers have increasingly turned to numerical approaches such as counting first or corresponding author publications, using the impact factor of the journals in which those publications appear, and computing Hirsch or H-index values (2).”
BackgroundMethodInfluentialCo‐citation in the scientific literature: A new measure of the relationship between two documents
Small, H. (1973) · Journal of the American Society for Information Science · doi:10.1002/asi.4630240406 · Cited by 5,345 · cited 1×
“There are a priori three types of article-linked citation networks [35].”
BackgroundKoenker, R. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511754098 · Cited by 4,551
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828 · cited 2×
“These authors [41] also found that RCR is better correlated with expert opinion than citation percentiles, which is the measure bibliometricians have previously recommended as best for evaluating the impact of published work [43].”
BackgroundGarfield, E. (1972) · Science · doi:10.1126/science.178.4060.471 · Cited by 2,788 · cited 4×
“For reasons of algorithmic parsimony we used the JCRs for the year each article in the co-citation network was published; a different choice at this step would be expected to have little if any effect, since almost all JCRs are quite stable over time ( Supporting Figure S1; Supporting Table S1 ).”
MethodInfluentialThe History and Meaning of the Journal Impact Factor
Garfield, E. (2006) · JAMA · doi:10.1001/jama.295.1.90 · Cited by 2,770 · cited 1×
“Journal impact factor (JIF) was for a time seen as a valuable indicator of scientific quality because it serves as a convenient, and not wholly inaccurate, proxy for expert opinion [8].”
BackgroundWhy the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447
Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market
Salganik, M. J., Dodds, P. S., & Watts, D. J. (2006) · Science · doi:10.1126/science.1121066 · Cited by 2,231 · cited 2×
“The results of a landmark study on the relationship between quality and success in a competitive market suggest that the ultimate winners in such contests are determined not only by the intrinsic value of the work but also by more intangible social variables [66].”
Background
Cited by
Items 101–200 of 515
Patel, P. A., Patel, K. K., Gopali, R., Reddy, A. P., Bogorad, D. D., & Bollinger, K. E. (2021) · Seminars in Ophthalmology · doi:10.1080/08820538.2021.1953543 · Cited by 25
Patel, P. A., Gopali, R., Reddy, A. P., & Patel, K. K. (2021) · Annals of Medicine and Surgery · doi:10.1016/j.amsu.2021.103021 · Cited by 29
“This metric theoretically compensates for the limitations of the h-index by providing an article-level score derived from dividing the total number of citations a publication has received by the average annual number of citations for NIH-funded publications in that paper’s field, which is determined by the article’s c…”
BackgroundMurphy, L. S.-L., Kraus, C. K., Lotfipour, S., Gottlieb, M. A., Langabeer, J. R., & Langdorf, M. I. (2018) · Western Journal of Emergency Medicine · doi:10.5811/westjem.2018.9.38213 · Cited by 26
Tantengco, O. A. G., Aquino, I. M. C., Asis, J. L. B., Tan, J. J. E., Uy, M. N. A. R., & Paz-Pacheco, E. T. (2021) · Diabetes & Metabolic Syndrome Clinical Research & Reviews · doi:10.1016/j.dsx.2021.102202 · Cited by 24
Llewellyn, N. M., & Nehl, E. J. (2022) · Clinical and Translational Science · doi:10.1111/cts.13251 · Cited by 25
Biomedical publishing: Past historic, present continuous, future conditional
Sever, R. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3002234 · Cited by 32
“In both cases, numbers are hard to compare across disciplines, despite attempts to normalize for this [64], and attention is not always an indication of quality.”
BackgroundHigh cost of bias: Diminishing marginal returns on NIH grant funding to institutionsPreprint
Wahls, W. P. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/367847 · Cited by 24
“The RCR value, which is being used by the NIH to assess portfolio performance and to guide funding decisions (e.g., Lauer, 2016a, 2016b, 2016d, 2017), is a time-normalized, field-normalized metric for citation impact (Hutchins et al., 2016).”
BackgroundMethodInfluentialSharing GWAS summary statistics results in more citations
Reales, G., & Wallace, C. (2023) · Communications Biology · doi:10.1038/s42003-023-04497-8 · Cited by 26
How to Choose Where to Publish Your Work
Abbott, J. H. (2017) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2017.0102 · Cited by 27
Cavinatto, L., Bronson, M. J., Chen, D. D., & Moucha, C. S. (2018) · International Orthopaedics · doi:10.1007/s00264-018-4175-5 · Cited by 22
From Gutenberg to Open Science: An Unfulfilled Odyssey
Triggle, C. R., & Triggle, D. J. (2016) · Drug Development Research · doi:10.1002/ddr.21369 · Cited by 22
“The Office of Portfolio Analysis at the NIH has recently proposed the Relative Citation Ratio, RCR that arguably addresses these issues; however, it has yet to be vigorously used [Hutchins et al., 2015, 2016].”
BackgroundMethodDardas, L. A., Sallam, M., Woodward, A. L., Sweis, N. S., Sweis, N., & Sawair, F. (2023) · Publications · doi:10.3390/publications11010005 · Cited by 24
The Rare Diseases Clinical Research Network: a model for clinical trial readiness
Lumsden, J. M., & Urv, T. K. (2023) · Therapeutic Advances in Rare Disease · doi:10.1177/26330040231219272 · Cited by 28
Top-100 Most Cited Dental Articles with Authors from Brazil
Gonçalves, A. P., Plá, A. L., Rodolfo, B., Nahsan, F. P. S., Corrêa, M. B., & Moraes, R. R. (2019) · Brazilian Dental Journal · doi:10.1590/0103-6440201902529 · Cited by 21
Lauer, M. S., Roychowdhury, D., Patel, K., Walsh, R., & Pearson, K. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/142554 · Cited by 24
“Outcomes: The “Relative Citation Ratio” (RCR) is a bibliometric measure developed by the NIH Office of Portfolio Analysis that compares citation influence of articles while accounting for field of study and time of publication.(9) Briefly, RCR compares actual and expected citation rates by assembling and inspecting a…”
MethodSciCV, the Swiss National Science Foundation’s new CV formatPreprint
Strinzel, M., Kaltenbrunner, W., van der Weijden, I., von Arx, M., & Hill, M. R. H. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.03.16.484596 · Cited by 23
“The inclusion of the two metrics H-index and RCR were less enthusiastically received by applicants and reviewers but still considered useful with ratings of 2.96 and 3.24 out of 5 (both moderately useful) for the RCR and the H-index for applicants, respectively, and an RCR scoring 2.96 out of 5 and H-index 3.21 out of…”
MethodInfluentialBaig, F., Kassem, A. H., Akhter, M. Z., Kabeer, S., Faiz, M. A., Baig, M. F., & Sherif, M. M. (2025) · Gondwana Research · doi:10.1016/j.gr.2025.07.022 · Cited by 21
A brief review on article-, author-, and journal-level scientometric indices
Mondal, H., & Mondal, S. (2022) · Indian Dermatology Online Journal · doi:10.4103/idoj.idoj_729_21 · Cited by 22
Yu, H., Marschke, G., Ross, M. B., Staudt, J., & Weinberg, B. A. (2022) · The Journal of Human Resources · doi:10.3368/jhr.59.2.1219-10630r1 · Cited by 20
“sought to extract more information from citation records than is afforded by simple citation counts (for example, Wang, Song, and Barabasi 2013; Hutchins et al. 2016). Some of this work has been focused specifically on new citation-based metrics of novelty. Acemoglu, Akcigit, and Celik (2015) used a range of rich char…”
Shift in the paradigm towards next-generation microbiology
Stres, B., & Kronegger, L. (2019) · FEMS Microbiology Letters · doi:10.1093/femsle/fnz159 · Cited by 21
“students as a good mentor? All layers of the relevant information concerning a published paper, such as the underlying value of a study or the extent of improvement after solving the problem at hand, cannot be captured by citation-based metrics (Hutchins et al. 2016).”
BackgroundInfluential methods reports for group-randomized trials and related designs
Murray, D. M. (2022) · Clinical Trials · doi:10.1177/17407745211063423 · Cited by 19
“The RCR uses relative citation rates to measure influence at the article level, standardized across areas of science defined by the relative citation network (the articles cited by other articles that also cite the reference article).(70) In brief, the citation rate of the reference article is compared to the citation…”
MethodAkudinobi, E. A., & Kilmarx, P. H. (2022) · BMJ Global Health · doi:10.1136/bmjgh-2022-009466 · Cited by 19
Llewellyn, N. M., Carter, D. R., DiazGranados, D., Pelfrey, C. M., Rollins, L. S., & Nehl, E. J. (2019) · Evaluation & the Health Professions · doi:10.1177/0163278719839435 · Cited by 19
“For each matched publication as of the time of data retrieval, the iCite data set included the number of times cited and the relative citation ratio (RCR; Hutchins et al., 2016).”
MethodDobránszki, J., & da Silva, J. A. T. (2019) · Scientometrics · doi:10.1007/s11192-019-03190-0 · Cited by 21
High-impact and transformative science (HITS) metrics: Definition, exemplification, and comparison
Staudt, J., Yu, H., Light, R. P., Marschke, G., Börner, K., & Weinberg, B. A. (2018) · PLoS ONE · doi:10.1371/journal.pone.0200597 · Cited by 19
Conservation-focused biobanks: A valuable resource for wildlife DNA forensics
Pérez‐Espona, S. (2021) · Forensic Science International Animals and Environments · doi:10.1016/j.fsiae.2021.100017 · Cited by 22
The Association Between Altmetric Attention Scores and Public Engagement in the Medical Literature
Mullins, C. H., Boyd, C. J., & Ladowski, J. M. (2023) · Journal of Surgical Research · doi:10.1016/j.jss.2023.07.051 · Cited by 18
Impacting Capabilities: A Conceptual Framework for the Social Value of Research
Hicks, D., Stahmer, C. G., & Smith, M. (2018) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2018.00024 · Cited by 18
On the interplay between normalisation, bias, and performance of paper impact metrics
Dunaiski, M., Geldenhuys, J., & Visser, W. (2019) · Journal of Informetrics · doi:10.1016/j.joi.2019.01.003 · Cited by 21
Citations and Gamed Metrics: Academic Integrity Lost
da Silva, J. T. (2021) · Academic Questions · doi:10.51845/34s.1.18 · Cited by 20
“What is now needed is a way to adjust ABMs and JBMs downward to accommodate for self-citations and retractions while simultaneously seeking to emphasize article-based metrics 24 rather than JBMs.”
MethodEvolution of coeloconic sensilla in the peripheral olfactory system of Drosophila mojavensis
Nemeth, D. C., Ammagarahalli, B., Layne, J. E., & Rollmann, S. M. (2018) · Journal of Insect Physiology · doi:10.1016/j.jinsphys.2018.08.003 · Cited by 18
Evaluation of US Orthopaedic Surgery Academic Centers Based on Measurements of Academic Achievement
Namavar, A. A., Loftin, A. H., Khahera, A. S., Stavrakis, A. I., Hegde, V. V., Johansen, D., Zoller, S. D., & Bernthal, N. M. (2018) · Journal of the American Academy of Orthopaedic Surgeons · doi:10.5435/jaaos-d-16-00536 · Cited by 17
Ke, Q., Gates, A. J., & Barabási, A.-Ĺ. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2309378120 · Cited by 27
Early-Career Setback and Future Career ImpactPreprint
Wang, Y., Jones, B. F., & Wang, D. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3353841 · Cited by 19
African neuroscience on the global stage: Nigeria as a model
Maina, M. B., Mohammed, Y. G., Bukar, A. M., Ahmad, U., Tijjani Salihu, A., Ibrahim, H. A., Abdurrazak, M., Hamidu, S. K., Yaro, A. U., Awadelkareem, M. A., et al. (2019) · European Journal of Neuroscience · doi:10.1111/ejn.14372 · Cited by 16
Transforming and extending library services by embracing technology and collaborations: A case study
Wheeler, T. R., Delgado, D., Albert, P. J., Maamar, S. B., & Oxley, P. R. (2022) · Health Information & Libraries Journal · doi:10.1111/hir.12439 · Cited by 16
“The Wood Library also has a one-click bibliometric report generation tool (Dutta, 2022a) which can produce a document describing any WCM researcher's publication impact, benchmarking the researcher against peers using NIH's iCite service (Hutchins et al., 2016).”
MethodPredicting substantive biomedical citations without full text
Hoppe, T. A., Arabi, S., & Hutchins, B. I. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2213697120 · Cited by 17
“First, we included information about the citing and referenced articles’ Relative Citation Ratio (RCR), which is a field-and time-normalized metric of an article’s citation rate (33, 34).”
Self-citationTechnology assisted research assessment: algorithmic bias and transparency issues
Thelwall, M. A., & Kousha, K. (2023) · Aslib Journal of Information Management · doi:10.1108/ajim-04-2023-0119 · Cited by 19
Barnett, A. G., Zardo, P., & Graves, N. (2018) · PLoS ONE · doi:10.1371/journal.pone.0195613 · Cited by 19
TIGA: target illumination GWAS analytics
Yang, J. J., Grissa, D., Lambert, C. G., Bologa, C., Mathias, S. L., Waller, A., Wild, D. J., Jensen, L. J., & Oprea, T. I. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab427 · Cited by 16
Wong, C.-S. (2021) · Energies · doi:10.3390/en14154573 · Cited by 19
Haunschild, R., & Bornmann, L. (2023) · Journal of Informetrics · doi:10.1016/j.joi.2023.101394 · Cited by 18
Neophilia ranking of scientific journals
Packalén, M., & Bhattacharya, J. (2016) · Scientometrics · doi:10.1007/s11192-016-2157-1 · Cited by 17
“…a few, try out a new idea, this 1 See e.g. Abbott et al. (2010), Adam (2002), Chapron and Husté (2006), Egghe (2006), Engemann and Wall (2009), Frey and Katja (2010), Garfield (1972), Hirsch (2005), Moed (2008) Palacios-Huerta and Volij (2004) (2014), Tort et al. (2012) and Hutchins et al. (2015).”
Han, X., Williams, S., & Zuckerman, B. L. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196545 · Cited by 18
Bibliometric Analysis of Erectile Dysfunction Publications in Urology and Sexual Medicine Journals
Rezaee, M. E., Johnson, H. A., Munarriz, R. M., & Gross, M. S. (2018) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2018.08.004 · Cited by 16
NIH metric that assesses article impact stirs debate
Bloudoff-Indelicato, M. (2015) · Nature · doi:10.1038/nature.2015.18734 · Cited by 15
Liu, M., Chien, T., & Chou, W. (2022) · Medicine · doi:10.1097/md.0000000000031609 · Cited by 17
“Second, to measure the research achievements shown in Figure 3, the authors used the article RCRs instead of traditional citations.”
MethodResultInfluentialLlewellyn, N. M., Weber, A. A., Pelfrey, C. M., DiazGranados, D., & Nehl, E. J. (2023) · Academic Medicine · doi:10.1097/acm.0000000000005225 · Cited by 18
Authorship inequalities in global health research: the IeDEA Southern Africa collaboration
Skrivankova, V. W., Hossmann, S., Cornell, M., Ballif, M., Dupont, C., Huwa, J. M., Seintaridis, K., Kalua, T., Wandeler, G., Kassanjee, R., et al. (2023) · BMJ Global Health · doi:10.1136/bmjgh-2023-013316 · Cited by 18
Information Systems Research: Thinking Outside the Basket and Beyond the Journal
Fitzgerald, B. F., Dennis, A. R., An, J., Tsutsui, S., & Muchhala, R. C. (2019) · Communications of the Association for Information Systems · doi:10.17705/1cais.04507 · Cited by 19
Evaluating medical conferences: the emerging need for a quality metric
Lang, R., Porter, K., Krentz, H. B., & Gill, M. J. (2019) · Scientometrics · doi:10.1007/s11192-019-03291-w · Cited by 15
Thelwall, M. A., Kousha, K., & Abdoli, M. (2017) · Scientometrics · doi:10.1007/s11192-017-2292-3 · Cited by 16
“The purpose of the RCR is to overcome many of the limitations of other indicators through the use of a complex algorithm to minimise the risk that articles are unfairly treated in citation comparisons.”
BackgroundInfluentialBornmann, L., Tekles, A., & Loet Leydesdorff, A. (2019) · Scientometrics · doi:10.1007/s11192-019-03071-6 · Cited by 16
The Olfactory Chemosensation of Hematophagous Hemipteran Insects
Liu, F., Chen, Z., Ye, Z., & Liu, N. (2021) · Frontiers in Physiology · doi:10.3389/fphys.2021.703768 · Cited by 18
Measuring the Impact of Scientific Research
Silva, P. U. K. D., & Vance, C. K. (2017) · Fascinating life sciences · doi:10.1007/978-3-319-50627-2_7 · Cited by 20
Importance of Quality Assessment in Clinical Research in Japan
UEDA, R., Nishizaki, Y., Homma, Y., Sanada, S., Otsuka, T., Yasuno, S., Matsuyama, K., Yanagisawa, N., Nagao, M., Fujibayashi, K., et al. (2019) · Frontiers in Pharmacology · doi:10.3389/fphar.2019.01228 · Cited by 15
A Bibliometric analysis of folate receptor research
Didion, C. A., & Henne, W. A. (2020) · BMC Cancer · doi:10.1186/s12885-020-07607-5 · Cited by 16
Rank analysis of most cited publications, a new approach for research assessments
Rodríguez‐Navarro, A., & Brito, R. (2024) · Journal of Informetrics · doi:10.1016/j.joi.2024.101503 · Cited by 17
Nwaichi, P. I., Ridzuan, N. B., Nwaichi, E. O., Gbonhinbor, J., & Agi, A. (2024) · SPE Nigeria Annual International Conference and Exhibition · doi:10.2118/221589-ms · Cited by 15
The accuracy of confidence intervals for field normalised indicators
Thelwall, M. A., & Fairclough, R. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.03.004 · Cited by 18
Bibliometric Evaluation of U.S. Neurosurgery Subspecialties and Academic Rank Using RCR Index
Asfaw, Z. K., Kalagara, R., Li, A. Y., Hannah, T. C., Schüpper, A. J., McCarthy, L., Hrabarchuk, E. I., Quinones, A., Durbin, J. R., Morgenstern, P. F., et al. (2021) · World Neurosurgery · doi:10.1016/j.wneu.2021.10.148 · Cited by 13
Newgard, C. D., Morris, C. D., Smith, L., Cook, J. N. B., Yealy, D. M., Collins, S. P., Holmes, J. F., Kuppermann, N., Richardson, L. D., Kimmel, S., et al. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2018.06.018 · Cited by 13
“The Relative Citation Ratio is a freely accessible, article-level metric that is field normalized and benchmarked to a peer comparison group, providing an article-level and field-independent measure of influential articles.(8) The Relative Citation Ratio benchmarking process is arranged so that the mean article has a…”
Henderson, M. N., Sojitra, B., Burke, O., & Prenner, J. L. (2023) · Ophthalmology Retina · doi:10.1016/j.oret.2023.01.002 · Cited by 13
D’Ambrosi, R., Kambhampati, S. B. S., Vishwanathan, K., Vaish, A., & Vaishya, R. (2023) · Indian Journal of Orthopaedics · doi:10.1007/s43465-023-00977-5 · Cited by 14
“The RCR is a newly developed metric that was introduced to prevent disparity between young researchers in comparison to senior researchers and also to correctly gauge the influence of the scientific work by comparing it with other published work in the same field of work [29, 30].”
ResultPerception, prestige and PageRank
Zeitlyn, D., & Hook, D. W. (2019) · PLoS ONE · doi:10.1371/journal.pone.0216783 · Cited by 15
“[18, 21] suggests interesting possibilities for clustering and local rankings within academic “families”.”
BackgroundNye, J., D’Souza, M. P., Hu, D. J., & Ghosh, D. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e06005 · Cited by 13
“The first is the RCR, a field normalized metric that measures a research article's influence [23].”
BackgroundMethodInfluentialPizzo, D. M., Imam, N., Sudah, S. Y., Faccone, R. D., Akinrodoye, M. A., Manzi, J. E., Menendez, M. E., & Gesell, M. W. (2022) · The Journal of Arthroplasty · doi:10.1016/j.arth.2022.07.022 · Cited by 13
External Tests of Peer Review Validity Via Impact Measures
Gallo, S. A., & Glisson, S. R. (2018) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2018.00022 · Cited by 20
“These ex post studies are related to the above ex ante results in that some literature has indicated that one of the strongest predictors of future citation performance is prior citation performance (Kaltman et al., 2014; Hutchins et al., 2016).”
NCI Alliance for Nanotechnology in Cancer – from academic research to clinical interventions
Grodzinski, P. A., Liu, C. H., Hartshorn, C. M., Morris, S. A., & Russell, L. M. (2019) · Biomedical Microdevices · doi:10.1007/s10544-019-0360-6 · Cited by 12
Asfaw, Z. K., Young, T., Brown, C. S., & Germano, I. M. (2024) · Journal of Neuro-Oncology · doi:10.1007/s11060-024-04778-0 · Cited by 11
“The relative citation ratio (RCR) of each manuscript included in this study was queried from the National Institute (NIH) of Health iCite website (https://icite.od.nih.gov) to analyze the influence of the article relative to other NIH-funded studies [17].”
InfluentialMachine learning-driven processes in architectural building design
Lystbæk, M. S. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106379 · Cited by 15
McNamara, J. K., Henderson, M. N., Sudah, S. Y., Faccone, R. D., Michel, C., Dijanic, C. N., Menendez, M. E., & Ruskin, J. B. (2022) · Cureus · doi:10.7759/cureus.25362 · Cited by 14
“The RCR has been validated in a dataset of over 88,000 publications, demonstrating consistency with the expert opinion of research quality [13].”
BackgroundMethodInfluentialBibliographic coupling networks reveal the advantage of diversification in scientific projects
Li, H., Wu, M., Wang, Y., & Zeng, A. (2022) · Journal of Informetrics · doi:10.1016/j.joi.2022.101321 · Cited by 13
Davis, F. M., Obi, A. T., Gallagher, K. A., & Henke, P. K. (2019) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2019.09.036 · Cited by 11
Analyzing the Scholarly Impact of Cardiothoracic Surgery Research Using the Relative Citation Ratio
Wang, H., Bajaj, S. S., Williams, K. M., O’Donnell, C. T., Heiler, J. C., Krishnan, A., Pickering, J. M., Sanchez, M., Manjunatha, K., Kumar, S. S., et al. (2022) · Journal of Surgical Research · doi:10.1016/j.jss.2022.02.007 · Cited by 11
An Arbitrary Line in the Sand: Rising Scientists Confront the Impact Factor
Depellegrin, T. A., & Johnston, M. (2015) · Genetics · doi:10.1534/genetics.115.182261 · Cited by 11
Metaresearch study highlights the gender gap in randomized controlled trials in dentistry
Prado, M. C., Dotto, L., Agostini, B. A., & Sarkis-Onofre, R. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.08.005 · Cited by 10
Yu, F., Patel, T., Carnegie, A., & Dave, G. (2023) · Journal of Clinical and Translational Science · doi:10.1017/cts.2022.530 · Cited by 10
“The Spearman’s rho testing shows that (1) FWCI, cites (i.e., Scopus citation counts), CB, RCR, PlumX-Citations, PlumX-Captures, and PlumX-Social Media counts are all positively correlated with each other (p< .05); (2) STPP is positively correlated with FWCI, Cites, CB, RCR, PlumX-Citations, PlumX-Mention, and PlumX-So…”
MethodInfluentialCurrent landscape of treating different cancers using nanomedicines: Trends and perspectives
Salvador-Morales, C., & Grodzinski, P. A. (2023) · Wiley Interdisciplinary Reviews Nanomedicine and Nanobiotechnology · doi:10.1002/wnan.1927 · Cited by 11
Nwaichi, P. I., Ridzuan, N. B., Nwaichi, E. O., Umunnawuike, C., Nyah, F., Abutu, D., Money, B., & Agi, A. (2025) · Circular Economy and Sustainability · doi:10.1007/s43615-025-00684-3 · Cited by 11
Qualitative Analysis of Faculty Opinions on and Perceptions of Research Impact Metrics
Bakker, C., Cooper, K., Langham-Putrow, A., & McBurney, J. (2020) · College & Research Libraries · doi:10.5860/crl.81.6.896 · Cited by 14
Klopfenstein, D. V., & Dampier, W. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1456 · Cited by 11
Mandibular Bone Resorption Following Chin Augmentation: A Systematic Review
Yeung, A. W. K., & Wong, N. S. M. (2022) · Frontiers in Surgery · doi:10.3389/fsurg.2022.815106 · Cited by 11
“(3) Mean number of follow-up years seemed to have no effect on the prevalence of bone resorption as reported by the studies (Figure 3D).”
BackgroundQuantifying knowledge from the perspective of information structurization
Wang, X., Kang, H., Fu, L., Yao, L., Ding, J., Wang, J., Gan, X., Zhou, C., & Hopcroft, J. E. (2023) · PLoS ONE · doi:10.1371/journal.pone.0279314 · Cited by 17
“The number of times an article is cited by other articles, books, etc. Relative Citation Ratio [15] The field-normalized citation compared with the other papers that appear alongside it in reference lists.”
InfluentialZang, D., & Liu, C. (2023) · Scientometrics · doi:10.1007/s11192-023-04634-4 · Cited by 11
“RCR (The relative citation ratio) is an indicator developed by the National Institutes of Health (NIH) in 2015 (Hutchins et al., 2016).”
search.bioPreprint: a discovery tool for cutting edge, preprint biomedical research articles
Iwema, C. L., LaDue, J., Zack, A., & Chattopadhyay, A. (2016) · F1000Research · doi:10.12688/f1000research.8798.1 · Cited by 11
Williams, M. J., Mullane, K., & Curtis, M. J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-804725-5.00005-7 · Cited by 13
Rainbow ranking: an adaptable, multidimensional ranking method for publication sets
Stoupas, G., Sidiropoulos, A., Γώγογλου, Α., Katsaros, D., & Manolopoulos, Y. (2018) · Scientometrics · doi:10.1007/s11192-018-2731-9 · Cited by 10
Accelerating a paradigm shift: The Common Fund Single Cell Analysis Program
Roy, A. L., Conroy, R., Smith, J. M., Yao, Y. M., Beckel-Mitchener, A. C., Anderson, J. M., & Wilder, E. L. (2018) · Science Advances · doi:10.1126/sciadv.aat8573 · Cited by 10
“The mean Relative Citation Ratio [RCR; (23)] for these papers is 2.”
BackgroundSelf-citationInterdisciplinarity Metric Based on the Co-Citation Network
Hernández, J. M., & Dorta–González, P. (2020) · Mathematics · doi:10.3390/math8040544 · Cited by 10
Responsible Science Assessment: downplaying indexes, boosting quality
Kowaltowski, A. J., Silber, A. M., & de Oliveira, M. F. (2021) · Anais da Academia Brasileira de Ciências · doi:10.1590/0001-3765202120191513 · Cited by 11
Using bibliometrics to evaluate outcomes and influence of translational biomedical research centers
Bragg, K. M., Marchand, G. C., Hilpert, J. C., & Cummings, J. L. (2021) · Journal of Clinical and Translational Science · doi:10.1017/cts.2021.863 · Cited by 10
“A co-citation network consists of other papers appearing in the reference lists alongside the RA, a citing network is the collection of papers citing the RA, and a cited-by network is the collection of papers in the reference list of the RA (20).”
BackgroundSayavedra, N., Hogle, J. A., & Moberg, D. P. (2017) · Journal of Clinical and Translational Science · doi:10.1017/cts.2018.1 · Cited by 10
“NIH percentile Percentile rank amongst NIH-funded publications [13] Total citations Total citations found in the citation database [13] Citations per year Citations per full calendar year after publication, through the end of their records [13]”
BackgroundMethodInfluentialDevelopment of science and education in the Western Balkan countries: competitiveness with the EU
Živković, Ž., & Panić, M. (2020) · Scientometrics · doi:10.1007/s11192-020-03554-x · Cited by 11
Valderrama, P., Baca, P., Solana, C., & Ferrer-Luque, C. M. (2021) · Antibiotics · doi:10.3390/antibiotics10111412 · Cited by 13
The Relative Citation Ratio: An Impartial Assessment of Productivity in Academic Otolaryngologists
Mendiratta, D., Herzog, I., Vought, R., Vought, V., Singh, R., Kodali, N., Patel, P. S., & Eloy, J. A. (2023) · The Laryngoscope · doi:10.1002/lary.30876 · Cited by 9
“11,12 RCR scores are calculated for each article, and author research productivity can be determined using the mean and weighted RCRs, which take the mean and sum of all the articles produced by that individual, respectively.”
BackgroundMethodRiley, W. T., Bibb, K., Hargrave, S. L., & Fearon, P. (2020) · PLoS ONE · doi:10.1371/journal.pone.0242271 · Cited by 9
“Evaluations of study sections using the Relative Citation Ratio (RCR) [16] had shown that the RCRs from some BSSR-focused study sections may not be as high as for some more biomedically focused BSSR removed above from full model due to its inclusion in BasicApplied neither level; Exp(β): Hazard Ratio (HR) per predicto…”
MethodInfluentialBMCR: A collaborator recommendation model based on Scholars’ academic level
Qin, H., Zhao, M., Ma, X., Sun, H., & Wei, W. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.09.018 · Cited by 9
Morisawa, F., Nishizaki, Y., Devos, P., Yanagisawa, N., Matsuyama, K., Homma, Y., UEDA, R., Sekine, M., Daida, H., Minamino, T., et al. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.978174 · Cited by 10
“This means that funded clinical research papers, even if they are not published in high impact journals, can be highly cited (26).”
BackgroundOtolaryngology Education: Recent Trends in Publication
Cass, N. D., Okland, T. S., Rodriguez, K. D., & Mann, S. E. (2016) · Otolaryngology · doi:10.1177/0194599816684098 · Cited by 9
Metadata
- Authors
- Hutchins, B. I., Yuan, X., Anderson, J. M., & Santangelo, G. M.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Hutchins, B. Ian
- Yuan, Xin
- Anderson, James M.
- Santangelo, George M.
- Venue
- PLoS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-09-06
- Volume
- 14
- Issue
- 9
- Pages
- e1002541
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC5012559
- OpenAlex
- W2953304564
- Cited by
- 500
- References
- 91
- Citations by year
- 2026: 43 · 2025: 51 · 2024: 52 · 2023: 62 · 2022: 68 · 2021: 48 · 2020: 44 · 2019: 45 · 2018: 36 · 2017: 32 · 2016: 17
- FWCI
- 40.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/publicdomain/zero/1.0/ (vor)
- Locations
- PLoS Biology (publishedVersion, public-domain)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- Conselho Nacional de Desenvolvimento Científico e Tecnológico (573871/2008-6)
- Fundação de Amparo à Pesquisa do Estado de Minas Gerais (573871/2008-6)
- National Institutes of Health (Z99 OD999999)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- relative citation ratio, citation analysis, co-citation networks, field normalization, iCite
- MeSH terms
- Authorship, Humans, National Institutes of Health (U.S.), United States, Bibliometrics, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.