Over-optimization of academic publishing metrics: observing Goodhart's Law in action
Open accessThe DOI matches its Crossref record; 70 of the 77 references it cites were found in a registry or an index.
Abstract
Background: The academic publishing world is changing significantly, with ever-growing numbers of publications each year and shifting publishing patterns. However, the metrics used to measure academic success, such as the number of publications, citation number, and impact factor, have not changed for decades. Moreover, recent studies indicate that these metrics have become targets and follow Goodhart's Law, according to which, “when a measure becomes a target, it ceases to be a good measure.” Results: In this study, we analyzed >120 million papers to examine how the academic publishing world has evolved over the last century, with a deeper look into the specific field of biology. Our study shows that the validity of citation-based measures is being compromised and their usefulness is lessening. In particular, the number of publications has ceased to be a good metric as a result of longer author lists, shorter papers, and surging publication numbers. Citation-based metrics, such citation number and h-index, are likewise affected by the flood of papers, self-citations, and lengthy reference lists. Measures such as a journal's impact factor have also ceased to be good metrics due to the soaring numbers of papers that are published in top journals, particularly from the same pool of authors. Moreover, by analyzing properties of >2,600 research fields, we observed that citation-based metrics are not beneficial for comparing researchers in different fields, or even in the same department. Conclusions: Academic publishing has changed considerably; now we need to reconsider how we measure success.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures















![Figure 16: The mean number of citations [Citation Number/Documents Number (2 years)] over time. The mean number of citations values have almost doubled in the past 18 years; additionally, their distributions have changed considerably.](https://figures.cochat.ai/paper-figures/03933677bbea423b8340291d60efccd5/over-optimization-of-academic-publishing-metrics-observing-goodhart-s-law-in-figure-16.jpg)






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As articles that describe miR functions have also been shown to be highly cited (60)”
From Identification of human gene research articles with wrongly identified nucleotide sequences (2022) · cites this paper as [60]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For example, the number of academic papers published per year is more than 7 million (Fire and Guestrin, 2019).”
From ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models (2025) · cites this paper as Fire and Guestrin, 2019
In the citing paper
“Scientists likely feel increased pressure to hype their results because productivity metrics have taken on a greater role in scientific advancement (23).”
From Misinformation in and about science (2021) · cites this paper as [23]
In the citing paper
“Much of the present pathology of hype, hyperbole, and publication bias is associated with an overreliance on productivity metrics (23).”
From Misinformation in and about science (2021) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Observation of Gravitational Waves from a Binary Black Hole Merger
Abbott, B. P., Abbott, R., Abbott, T. D., Abernathy, M. R., Acernese, F., Ackley, K., Adams, C., Adams, T., Addesso, P., Adhikari, R. X., et al. (2016) · Physical Review Letters · doi:10.1103/physrevlett.116.061102 · Cited by 14,151 · cited 1×
“For example, the recent Laser Interferome-ter Gravitational-Wave Observatory paper [37] listed > 1,000 authors [38].”
An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669 · cited 2×
“Another common metric to measure a researcher’s output or a journal’s impact is the h-index, which measures an author’s or a journal’s number of papers that have at least h citations each [3].”
BackgroundMethodThe History and Meaning of the Journal Impact Factor
Garfield, E. (2006) · JAMA · doi:10.1001/jama.295.1.90 · Cited by 2,770
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447 · cited 1×
“Even with their many known shortcomings [8, 24, 55, 68– 70], measures such as the impact factor, citation number, and h-index are still widely used.”
ArnetMiner: extraction and mining of academic social networks
Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1402008 · Cited by 2,231 · cited 1×
“The AMiner open academic graph dataset [77] contains data from > 154 million papers.”
Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., Petersen, A. M., Radicchi, F., Sinatra, R., Uzzi, B., et al. (2018) · Science · doi:10.1126/science.aao0185 · Cited by 1,497 · cited 1×
“This research is a large-scale scientometrics study (also referred to as the “science of science” [30]).”
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388 · cited 2×
“…of the measures can be compromised, and unwelcome and unethical behaviors may develop, such as salami publications [17], ghost author-ships [18], p-hacking [19], metrics manipulation [20], faking research data [21], faking of peer reviews [22], and even plagiariz-ing by a peer reviewer [23].”
Does the h index have predictive power?
Hirsch, J. E. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0707962104 · Cited by 973 · cited 1×
“It has been shown that the h-index can predict academic achievements [57].”
Assessing the impact of planned social change
Campbell, D. T. (1979) · Evaluation and Program Planning · doi:10.1016/0149-7189(79)90048-x · Cited by 957 · cited 2×
“Good-hart’s Law, and the closely related Campbell’s Law [11], influence many systems in our everyday life, including educational [11], biological [12], and other decision-making systems [13, 14].”
The “Gender Gap” in Authorship of Academic Medical Literature — A 35-Year Perspective
Jagsi, R., Guancial, E. A., Worobey, C. C., Henault, L. E., Chang, Y., Starr, R., Tarbell, N. J., & Hylek, E. M. (2006) · New England Journal of Medicine · doi:10.1056/nejmsa053910 · Cited by 870 · cited 2×
“Additionally, Gwilym Lockwood observed that “articles with positively-framed titles, interesting phrasing, and no word-play get more attention online” [45].”
Is science becoming more interdisciplinary? Measuring and mapping six research fields over time
Porter, A. L., & Rafols, I. (2009) · Scientometrics · doi:10.1007/s11192-008-2197-2 · Cited by 785 · cited 1×
“In 2009, Porter and Rafols [51] used bibliometric indicators alongside a new index of interdis-ciplinarity to measure how the degree of interdisciplinarity has changed between 1975 and 2005 for 6 research domains.”
Edwards, M. A., & Roy, S. (2017) · Environmental Engineering Science · doi:10.1089/ees.2016.0223 · Cited by 688 · cited 4×
“Studies suggest that publication patterns have changed as a result of Goodhart’s Law, according to which, “When a measure becomes a target, it ceases to be a good measure” [9, 10].”
InfluentialAn Overview of Microsoft Academic Service (MAS) and Applications
Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J. (., & Wang, K. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742839 · Cited by 658 · cited 1×
“InthisstudyweprimarilyusedtheMAG[72],whichwasreleased as part of the 2016 KDD Cup [73].”
The Metric Tide: Independent Review of the Role of Metrics in Research Assessment and Management
Wilsdon, J. (2015) · doi:10.4135/9781473978782 · Cited by 610 · cited 2×
“s escalation, metrics such as the number of papers, number of citations, impact factor, h-index, and altmetrics are being used to compare the impact of papers, researchers, journals, and universities [2, 3, 4, 5]. Using quantitative metrics to rank researchers contributes to a hypercompetitive research environment, wh…”
BackgroundComparison of SCImago journal rank indicator with journal impact factor
Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008) · The FASEB Journal · doi:10.1096/fj.08-107938 · Cited by 555 · cited 1×
“Likewise, the SJR indicator [42] and simple citation distributions [43] are offered as alternatives to the impact factor.”
MethodThe STM Report: An overview of scientific and scholarly journal publishing
Ware, M., & Mabe, M. A. (2015) · Lincoln (University of Nebraska) · Cited by 528
Cronin, B. (2001) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.1097 · Cited by 500 · cited 1×
“This phenomenon is known as hyperauthorship [34] or author inflation [35] and is common across research fields, where the majority of papers with > 1,000 authors are produced in the physical sciences [36].”
Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey
Wislar, J. S., Flanagin, A., Fontanarosa, P. B., & DeAngelis, C. D. (2011) · BMJ · doi:10.1136/bmj.d6128 · Cited by 478 · cited 1×
“Wislar et al. found such evidence in biomedical journals [23], and similar findings were observed by Kennedy et al. [24] and by Vera-Badillo et al. [25].”
ResultDetecting h-index manipulation through self-citation analysis
Bartneck, C., & Kokkelmans, S. (2011) · Scientometrics · doi:10.1007/s11192-010-0306-5 · Cited by 301 · cited 5×
“…of the measures can be compromised, and unwelcome and unethical behaviors may develop, such as salami publications [17], ghost author-ships [18], p-hacking [19], metrics manipulation [20], faking research data [21], faking of peer reviews [22], and even plagiariz-ing by a peer reviewer [23].”
InfluentialLehmann, S., Jackson, A. D., & Lautrup, B. E. (2006) · Nature · doi:10.1038/4441003a · Cited by 277 · cited 3×
“Even with their many known shortcomings [8, 24, 55, 68– 70], measures such as the impact factor, citation number, and h-index are still widely used.”
The Google scholar experiment: How to index false papers and manipulate bibliometric indicators
Delgado López‐Cózar, E., Robinson‐García, N., & Torres‐Salinas, D. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23056 · Cited by 272 · cited 1×
“All types of manipulative methods are used, such as increasing the number of self-citations [15], increasing the number of publications by slicing studies into the smallest measurable quantum acceptable for publication [38], indexing false papers [39], and merging papers on Google Scholar [40].”
MethodTowards a new crown indicator: an empirical analysis
Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2011) · Scientometrics · doi:10.1007/s11192-011-0354-5 · Cited by 228
Beat it, impact factor! Publishing elite turns against controversial metric
Callaway, E. (2016) · Nature · doi:10.1038/nature.2016.20224 · Cited by 220 · cited 1×
“Senior employees at several leading science publishers called upon journals to restrain from using the impact factor and suggested replacing it with simple citation distributions [43, 44].”
BackgroundA simple proposal for the publication of journal citation distributions
Larivière, V., Kiermer, V., MacCallum, C. J., McNutt, M., Patterson, M., Pulverer, B., Swaminathan, S., Taylor, S., & Curry, S. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/062109 · Cited by 213 · cited 2×
“Senior employees at several leading science publishers called upon journals to restrain from using the impact factor and suggested replacing it with simple citation distributions [43, 44].”
BackgroundMethodHecht, F., Hecht, B. K., & Sandberg, A. A. (1998) · Cancer Genetics and Cytogenetics · doi:10.1016/s0165-4608(97)00459-7 · Cited by 206 · cited 1×
“Similarly, the altmetric [65] was proposed as an alternative metric to the impact factor and h-index.”
Researchers’ Individual Publication Rate Has Not Increased in a Century
Fanelli, D., & Larivière, V. (2016) · PLOS ONE · doi:10.1371/journal.pone.0149504 · Cited by 205 · cited 3×
“In 2016, Fanelli and Larivi`ere [52] analyzed the publication patterns of > 40,000 researchers for more than a century.”
InfluentialAuthorship and citation manipulation in academic research
Fong, E. A., & Wilhite, A. W. (2017) · PLOS ONE · doi:10.1371/journal.pone.0187394 · Cited by 205 · cited 3×
“Fong and Wilhite’s findings revealed that the majority of researchers disapprove of this type of metric manipulation,yetmanyfeelpressuredtoparticipate;otherresearchers blandly state “that it is just the way the game is played” [58].”
Peer-Review Fraud — Hacking the Scientific Publication Process
Haug, C. J. (2015) · New England Journal of Medicine · doi:10.1056/nejmp1512330 · Cited by 163 · cited 1×
“…when measures become targets in academic publishing, the effectiveness of the measures can be compromised, and unwelcome and unethical behaviors may develop, such as salami publications [12], ghost author-ships [13], p-hacking [14], metrics manipulation [15], and even faking of peer reviews [16].”
BackgroundThe meaning of the Impact Factor
Garfield, E. (2003) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 160
Have the “mega-journals” reached the limits to growth?
Björk, B.-C. (2015) · PeerJ · doi:10.7717/peerj.981 · Cited by 149 · cited 2×
“Mega-journals accelerate review and publication times to 3-5 months and usually have high acceptance rates of over 50% [1].”
BackgroundWhat's in a title? Numbers of words and the presence of colons
Lewison, G., & Hartley, J. R. (2005) · Scientometrics · doi:10.1007/s11192-005-0216-0 · Cited by 145 · cited 1×
“Lewison and Hartley [44] analyzed how papers’ titles have changed over time.”
Watch out for cheats in citation game
Biagioli, M. (2016) · Nature · doi:10.1038/535201a · Cited by 140 · cited 4×
“Studies suggest that publication patterns have changed as a result of Goodhart’s Law, according to which, “When a measure becomes a target, it ceases to be a good measure” [9, 10].”
InfluentialThe w‐index: A measure to assess scientific impact by focusing on widely cited papers
Wu, Q. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21276 · Cited by 130 · cited 1×
“For example, the q-index [15] and w-index [41] are alternatives to the h-index.”
MethodGoodhart's Law: its origins, meaning and implications for monetary policy
Chrystal, A. (2003) · Edward Elgar Publishing eBooks · doi:10.4337/9781781950777.00022 · Cited by 100
Salami publication: definitions and examples
Šupak Smolčić, V. (2013) · Biochemia Medica · doi:10.11613/bm.2013.030 · Cited by 91 · cited 1×
“Recent studies indicate that when measures become targets in academic publishing, the effectiveness of the measures can be compromised, and unwelcome and unethical behaviors may develop, such as salami publications [12], ghost authorships [13], p-hacking [14], metrics manipulation [15], and even faking of peer reviews…”
A Century of Science: Globalization of Scientific Collaborations, Citations, and Innovations
Dong, Y., Ma, H., Shen, Z., & Wang, K. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098016 · Cited by 89 · cited 2×
“From the perspective of impact and innovations, Dong et al. observed “diversifica-tion of scientific development across the planet over the past century” [53].”
Free journal-ranking tool enters citation market
Butler, D. (2008) · Nature · doi:10.1038/451006a · Cited by 76 · cited 1×
“To better understand trends in journal publications, we used the SCImago Journal Ranking (SJR) open dataset [52].”
MethodAn Analysis of the Microsoft Academic Graph
Herrmannova, D., & Knoth, P. (2016) · D-Lib Magazine · doi:10.1045/september2016-herrmannova · Cited by 75 · cited 4×
“The volume of papers has increased sharply from < 1 million papers published in 1980 to > 7 million papers published in 2014 [2].”
InfluentialWeighted citation: An indicator of an article's prestige
Yan, E., & Ding, Y. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21349 · Cited by 73
From bibliometrics to altmetrics: A changing scholarly landscape
Roemer, R. C., & Borchardt, R. (2012) · College & Research Libraries News · doi:10.5860/crln.73.10.8846 · Cited by 71 · cited 2×
“Additionally, an overview of the changing scholarly landscape can be found in Roemer and Borchardt’s study [2].”
BackgroundMethodQuality Measures for Dialysis: Time for a Balanced Scorecard
Kliger, A. S. (2015) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.06010615 · Cited by 64 · cited 1×
“[16] Recent studies indicate that when measures become targets in academic publishing, the effectiveness of the measures can be compromised, and unwelcome and unethical behaviors may develop, such as salami publications [17], ghost author-ships [18], p-hacking [19], metrics manipulation [20],…”
Weighted citation: An indicator of an article's prestige
Yan, E., & Ding, Y. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.4876 · Cited by 60 · cited 1×
“However, not all citations are equal [33].”
Honorary and Ghost Authorship in Nursing Publications
Kennedy, M. S., Barnsteiner, J., & Daly, J. W. (2014) · Journal of Nursing Scholarship · doi:10.1111/jnu.12093 · Cited by 58 · cited 1×
“Wislar et al. found such evidence in biomedical journals [40], and similar findings were observed by Kennedy et al. [41] and by Vera-Badillo et al. [42].”
Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey
(2011) · BMJ · doi:10.1136/bmj.d7677 · Cited by 55
The Google Scholar Experiment: How To Index False Papers And Manipulate Bibliometric Indicators
López‐Cózar, E. D., Robinson‐García, N., & Torres‐Salinas, D. (2013) · Institutional Repository of the University of Granada (University of Granada) · doi:10.5281/zenodo.1042920 · Cited by 53
The rising trend in authorshipPreprint
Aboukhalil, R. (2014) · doi:10.15200/winn.141832.26907 · Cited by 52 · cited 1×
“Robert Aboukhalil measured this trend [39] and discovered that the mean number of authors of academic papers has increased sharply since the beginning of the 20th century.”
The continuing evolution of publishing in the biological sciences
Kelly, S. L. (2018) · Biology Open · doi:10.1242/bio.037325 · Cited by 48 · cited 2×
“Recently, Steven Kelly observed an unexpected increase in the mean number of authors of papers in the biological sciences [4].”
A Century of Science: Globalization of Scientific Collaborations, Citations, and InnovationsPreprint
Dong, Y., Ma, H., Shen, Z., & Wang, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.05150 · Cited by 46 · cited 2×
“From the perspective of impact and innovations, Dong et al. observed “diversifica-tion of scientific development across the planet over the past century” [53].”
(2005) · Nature Materials · doi:10.1038/nmat1305 · Cited by 45 · cited 1×
“All types of manipulative methods are used, such as increasing the number of self-citations [20], increasing the number of publications by slicing studies into the smallest quantum acceptable for publication [59], indexing false papers [60], and merging papers on Google Scholar [61].”
Central Banking, Monetary Theory and Practice
(2003) · Edward Elgar Publishing eBooks · doi:10.4337/9781781950777 · Cited by 43
Implications of Goodhart's Law for monitoring global biodiversity loss
Newton, A. C. (2011) · Conservation Letters · doi:10.1111/j.1755-263x.2011.00167.x · Cited by 36 · cited 2×
“Good-hart’s Law, and the closely related Campbell’s Law [11], influence many systems in our everyday life, including educational [11], biological [12], and other decision-making systems [13, 14].”
H-index manipulation by merging articles: Models, theory, and experiments
van Bevern, R., Komusiewicz, C., Niedermeier, R., Sorge, M., & Walsh, T. (2016) · Artificial Intelligence · doi:10.1016/j.artint.2016.08.001 · Cited by 36
Dear Plagiarist: A Letter to a Peer Reviewer Who Stole and Published Our Manuscript as His Own
Dansinger, M. L. (2016) · Annals of Internal Medicine · doi:10.7326/m16-2551 · Cited by 33 · cited 1×
“…of the measures can be compromised, and unwelcome and unethical behaviors may develop, such as salami publications [17], ghost author-ships [18], p-hacking [19], metrics manipulation [20], faking research data [21], faking of peer reviews [22], and even plagiariz-ing by a peer reviewer [23].”
Lockwood, G. (2016) · doi:10.15200/winn.146723.36330 · Cited by 33 · cited 2×
“Additionally, Gwilym Lockwood observed that “articles with positively-framed titles, interesting phrasing, and no word-play get more attention online” [45].”
Honorary and ghost authorship in reports of randomised clinical trials in oncology
Vera-Badillo, F., Napoleone, M., Krzyzanowska, M. K., Alibhai, S. M. H., Chan, A., Ocaña, A., Templeton, A. J., Šeruga, B., Amir, E., & Tannock, I. F. (2016) · European Journal of Cancer · doi:10.1016/j.ejca.2016.06.023 · Cited by 32 · cited 1×
“Robert Aboukhalil measured this trend [39] and discovered that the mean number of authors of academic papers has increased sharply since the beginning of the 20th century.”
Kupferschmidt, K. (2018) · Science · doi:10.1126/science.361.6403.636 · Cited by 30 · cited 2×
“…of the measures can be compromised, and unwelcome and unethical behaviors may develop, such as salami publications [17], ghost author-ships [18], p-hacking [19], metrics manipulation [20], faking research data [21], faking of peer reviews [22], and even plagiariz-ing by a peer reviewer [23].”
Growth in the number of references in engineering journal papers during the 1972–2013 period
Úcar, I., López-Fernandino, F., Rodríguez‐Ulibarri, P., Sesma-Sanchez, L., Urrea-Micó, V., & Sevilla, J. (2013) · Scientometrics · doi:10.1007/s11192-013-1113-6 · Cited by 30 · cited 1×
“In addition to paper title lengths increasing, Ucar et al. have found lengthening reference lists for engineering journal articles, such as those published in Biomedical Engineering and Information Theory [46].”
Revised h index for biomedical research
Romanovsky, A. A. (2012) · Cell Cycle · doi:10.4161/cc.22179 · Cited by 28 · cited 2×
“To tackle this issue, various measures such as the c-index [24] and revised h-index [25] have been suggested.”
Ghost and Guest Authors: You Can't Always Trust Who You Read
Schofferman, J. A., Wetzel, F. T., & Bono, C. M. (2014) · Pain Medicine · doi:10.1111/pme.12579 · Cited by 26 · cited 3×
“7), and some top journals have become “old boys’ clubs” that mainly publish papers from the same researchers (see Figs 17 and 18).”
Academe's Unspoken Ethical Dilemma: Author Inflation in Higher Education.
Bergen, C. W. V., & Bressler, M. S. (2017) · Research in higher education journal · Cited by 26
Post, A., Li, A. Y., Dai, J., Maniya, A. Y., Haider, S. F., Sobótka, S., & Choudhri, T. F. (2018) · Cureus · doi:10.7759/cureus.2629 · Cited by 15 · cited 3×
“To tackle this issue, various measures such as the c-index [24] and revised h-index [25] have been suggested.”
Publication trends in top-tier journals in higher education
Johnson, M., Wagner, N. J., & Reusch, J. (2016) · Journal of Applied Research in Higher Education · doi:10.1108/jarhe-01-2015-0003 · Cited by 14 · cited 1×
“Johnson et al. [49] studied publication trends in top-tier journals of higher education.”
Colavizza, G., Franssen, T., & van Leeuwen, T. N. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.11.006 · Cited by 12
What bioRxiv’s first 30,000 preprints reveal about biologists
Learn, J. R. (2019) · Nature · doi:10.1038/d41586-019-00199-6 · Cited by 10 · cited 1×
“Additionally, > 1 million papers are now downloaded from bioRxiv every month [32].”
Scientific publishing: The inside track
Aldhous, P. (2014) · Nature · doi:10.1038/510330a · Cited by 7 · cited 1×
“Peter Aldhous analyzed publications in the National Academy of Sciences (PNAS) journal, to consider the influence of an“old boys’ club” mentality [32].”
BackgroundALIAS: Author Disambiguation in Microsoft Academic Search Engine Dataset
Pitts, M., Savvana, S., Roy, S. B., & Mandava, V. (2014) · doi:10.5441/002/edbt.2014.65 · Cited by 6
‘Altmetrics’! Can you afford to ignore it?
Griffin, S. A., Oliver, C. W., & Murray, A. M. (2017) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-098258 · Cited by 6 · cited 1×
“The altmetric [45] is a generalization of article-level metrics and considers other aspects of the impact of the work, such as the number of downloads, article views, mentions in social media, and more.”
BackgroundByrne, A. (2017) · Nature · doi:10.1038/548s22a · Cited by 4 · cited 1×
“Similarly, the altmetric [65] was proposed as an alternative metric to the impact factor and h-index.”
Castelvecchi, D. (2017) · Nature · doi:10.1038/nature.2017.22786 · Cited by 2 · cited 1×
“For example, the recent Laser Interferome-ter Gravitational-Wave Observatory paper [37] listed > 1,000 authors [38].”
Fire, M., & Guestrin, C. (2019) · GigaDB · doi:10.5524/100587
Self-citation
Research works printed but not linked: 6
- 2003, Central Banking, Monetary Theory and Practice: Essays in honour of Charles Goodhart, 221
- Francescani, 2012, NYPD report confirms manipulation of crime stats, Reuters
- Economist, 2016, Why research papers have so many authors, The Economist
- Gálvez, 2000, Scientific publication trends and the developing world: What can the volume and authorship of scientific articles tell us about scientific progress in various regions?, American Sci, 88, 526
- 2016, KDD Cup 2016: Whose papers are accepted the most: towards measuring the impact of research institutions
- Low, 2014, Graphlab: A new framework for parallel machine learning, 1006.4990
Cited by
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584
Misinformation in and about science
West, J. D., & Bergstrom, C. T. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912444117 · Cited by 391
“Much of the present pathology of hype, hyperbole, and publication bias is associated with an overreliance on productivity metrics (23).”
BackgroundThe strain on scientific publishing
Hanson, M. A., Barreiro, P. G., Crosetto, P., & Brockington, D. (2024) · Quantitative Science Studies · doi:10.1162/qss_a_00327 · Cited by 345
“The difficulty is that “quality” is hard to define (17, 18, 23), and some metrics are at risk of abuse per Goodhart’s law : “when a measure becomes a target, it ceases to be a good measure” (24).”
BackgroundThe strain on scientific publishing
Hanson, M. A., Barreiro, P. G., Crosetto, P., & Brockington, D. (2023) · arXiv · doi:10.48550/arxiv.2309.15884 · Cited by 345
“The difficulty is that “quality” is hard to define (17, 18, 23), and some metrics are at risk of abuse per Goodhart’s law : “when a measure becomes a target, it ceases to be a good measure” (24).”
BackgroundThe sustainable development goals of the United Nations: A comparative midterm research review
Sorooshian, S. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142272 · Cited by 205
Metadata
- Authors
- Fire, M., & Guestrin, C.
- Affiliations
- Ben-Gurion University of the Negev (IL)
- University of Washington (US)
- Corresponding author
- Fire, Michael
- Guestrin, Carlos
- Venue
- GigaScience
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-05-30
- Volume
- 8
- Issue
- 6
- ISSN
- 2047-217X (ISSN-L)
- PMC
- PMC6541803
- OpenAlex
- W2890107153
- Cited by
- 351
- References
- 77
- Citations by year
- 2026: 52 · 2025: 74 · 2024: 65 · 2023: 65 · 2022: 46 · 2021: 18 · 2020: 20 · 2019: 9
- FWCI
- 15.56
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- GigaScience (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Washington Research Foundation
- University of Washington
- Amazon Web Services
- Microsoft Azure Research Award
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- Goodhart's law, citation metrics, impact factor, h-index, publication counts, self-citation, author list, research evaluation, biology research, bibliometric analysis
- MeSH terms
- Publishing, Bibliometrics, Benchmarking, Journal Impact Factor
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.