Reference Accuracy in the General Surgery Literature
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Background: Reference inaccuracy in scientific articles brings the scientific validity of the research into question and may create difficulty when accessing the cited background data. The objective of this study was to examine the reference accuracy in the general surgery literature and its correlation with the journal impact factor. Methods: Five general surgery journals were chosen with varying impact factors. From the year 2007, one issue was randomly chosen from each journal, and from each issue 180 citations were randomly chosen for review. Three investigators evaluated the chosen references for primary, citational, and quotational errors. The impact factor of each journal was compared to the percentage of errors detected. Results: The total number of errors per journal ranged from 31.3 to 39.3%, with a total of 35.4% of all citations reviewed containing some type of error. The most common error type detected was incorrect citation of the primary source supporting a statement, the incidence of which ranged from 13.8 to 25.2%, depending on the journal, and accounting for 53.6% of the total errors found. Citational errors, which included incorrect author names, pagination, dates, and issue and volume numbers, ranged from 1.8 to 18.1% and accounted for 20.4% of the total errors detected. Qualitative errors, which occurred when the author misquoted another author’s written assertions or conclusions, ranged from 7.4 to 16.0% and accounted for 34.7% of the total errors detected. Quantitative errors (misquoted numerical data) ranged from 3.1 to 8.6% and accounted for 17.9% of the total errors detected. No association between impact factor and error rate was demonstrated. Conclusions: Reference inaccuracy is common in the general surgery literature. The impact factor has no clear association with the error rate, demonstrating that journal quality does not necessarily correlate with reference quality. Further investigation into potential methods for improving reference accuracy in the general surgery literature is warranted.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Generally, each assertion was evaluated by multiple experts and the classification of quotation error was determined by consensus.[19, 20, 22, 27, 31, 32]”
From Accuracy of cited "facts" in medical research articles: A review of study methodology and recalculation of quotation error rate (2017) · cites this paper as [19]
In the citing paper
“Yet, several studies also noted some degree of difficulty,[20, 27, 31] subjectivity,[31, 32] and disagreement[19, 22, 27, 31] among the authors while evaluating assertions.”
From Accuracy of cited "facts" in medical research articles: A review of study methodology and recalculation of quotation error rate (2017) · cites this paper as [19]
Citations
Cites
Impact factor: a valid measure of journal quality?
(2003) · Europe PMC (PubMed Central) · Cited by 584
Association of Journal Quality Indicators With Methodological Quality of Clinical Research Articles
Lee, K. P. (2002) · JAMA · doi:10.1001/jama.287.21.2805 · Cited by 284
Quotational and reference accuracy in surgical journals. A continuing peer review problem.
(1990) · JAMA · Cited by 176
Quotational and Reference Accuracy in Surgical Journals: A Continuing Peer Review Problem
Evans, J. T. (1990) · JAMA · doi:10.1001/jama.1990.03440100059009 · Cited by 129
Eichorn, P., & Yankauer, A. (1987) · American Journal of Public Health · doi:10.2105/ajph.77.8.1011 · Cited by 114 · cited 1×
“One of these parameters was a high total number of references cited in an article [3, 15, 16 ].”
BackgroundThe accuracy of citation and quotation in otolaryngology/head and neck surgery journals
Fenton, J. E., Brazier, H., Souza, A., Hughes, J. P., & McShane, D. P. (2000) · Clinical Otolaryngology · doi:10.1046/j.1365-2273.2000.00322.x · Cited by 94
Risk factors for citation errors in peer‐reviewed nursing journals
Lok, C. K. W., Chan, M. T. V., & Martinson, I. M. (2001) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2001.01748.x · Cited by 73 · cited 3×
“This finding is not in line with the findings of a previous study which examined the risk factors for Citation errors in peer-reviewed nursing journals [ 15 ].”
BackgroundResultInfluentialTrends in the Quality of Highly Cited Surgical Research Over the Past 20 Years
Brooke, B. S., Nathan, H., & Pawlik, T. M. (2008) · Annals of Surgery · doi:10.1097/sla.0b013e31819291f9 · Cited by 68
Quotation and Reference Accuracy of Three Nursing Journals
Schulmeister, L. (1998) · Image: the Journal of Nursing Scholarship · doi:10.1111/j.1547-5069.1998.tb01269.x · Cited by 53 · cited 1×
“Other investigators however, have failed to show this correlation [6, 17 ].”
ResultAccuracy of references in general surgical journals — An old problem revisited
Reddy, M. S., Srinivas, S., Sabanayagam, N., & Balasubramanian, S. P. (2008) · The Surgeon · doi:10.1016/s1479-666x(08)80067-4 · Cited by 41
Citation indexes do not reflect methodological quality in lung cancer randomised trials.
Berghmans, T., Meert, A. P., Mascaux, C., Paesmans, M., Lafitte, J. J., & Sculier, J. P. (2003) · Annals of Oncology · doi:10.1093/annonc/mdg203 · Cited by 38 · cited 1×
“Other investigators have found no correlation between citation indexes and methodological quality, at least for lung cancer randomized trials [ 14 ].”
Background(2004) · PubMed · Cited by 31
(2005) · PubMed · Cited by 28
Inappropriate attribution: the “lazy author syndrome”
GAVRAS, H. (2002) · American Journal of Hypertension · doi:10.1016/s0895-7061(02)02989-8 · Cited by 27
HOW ACCURATE ARE REFERENCES IN THE AUSTRALIAN AND NEW ZEALAND JOURNAL OF SURGERY?
Kee, W. D. N., Roach, V. J., & Lau, T. K. (1997) · Australian and New Zealand Journal of Surgery · doi:10.1111/j.1445-2197.1997.tb02005.x · Cited by 19
Citation Accuracy in the Oral and Maxillofacial Surgery Literature
Mohammad, A. E., & Laskin, D. M. (2008) · Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.joms.2007.06.682 · Cited by 17
Celayir, A. C., Sander, S., & Celayir, S. (2003) · Journal of Pediatric Surgery · doi:10.1053/jpsu.2003.50154 · Cited by 11
[The accuracy of references in Turkish journals in orthopedics and traumatology].
Çakır, V., İlhan, F., Kiliçoğlu, G., Balkan, Y., Veziroğlu, A., & Günal, İ. (2003) · PubMed · doi:10.3944/aott.v37i4.816 · Cited by 5
Research works printed but not linked: 1
- 10.1001/jama.263.10.1353 · DOI · Paper page
Cited by
Impact factor: outdated artefact or stepping-stone to journal certification?
Vanclay, J. K. (2012) · Scientometrics · doi:10.1007/s11192-011-0561-0 · Cited by 370
Improving the peer review of narrative literature reviews
Byrne, J. A. (2016) · Research Integrity and Peer Review · doi:10.1186/s41073-016-0019-2 · Cited by 159
“Many different types of citation errors can be identified in the research literature [21, 22], and these may occur regardless of the journal impact factor [22].”
BackgroundQuotation accuracy in medical journal articles—a systematic review and meta-analysis
Jergas, H., & Baethge, C. (2015) · PeerJ · doi:10.7717/peerj.1364 · Cited by 88
“In total, we included 28 studies in our main analysis (De Lacey, Record & Wade, 1985; Lowry, 1985; Eichorn & Yankauer, 1987; Neihouse & Priske, 1989; Evans, Nadjari & Burchell, 1990; Hobma & Overbeke, 1992; Puttermann, 1992; Goldberg et al., 1993; George & Robbins, 1994; Hansen & McIntire, 1994; Warren et al., 1997; S…”
BackgroundInfluentialMogull, S. A. (2017) · PLoS ONE · doi:10.1371/journal.pone.0184727 · Cited by 75
Let not thy food be confused with thy medicine: The Hippocratic misquotation
Cárdenas, D. (2013) · e-SPEN Journal · doi:10.1016/j.clnme.2013.10.002 · Cited by 62
Metadata
- Authors
- Awrey, J., Inaba, K., Barmparas, G., Recinos, G., Teixeira, P. G. R., Chan, L. S., Talving, P., & Demetriades, D.
- Affiliations
- University of Southern California (US)
- Division of Trauma Surgery and Surgical Critical Care Los Angeles County + University of Southern California (LAC+USC) Medical Center 1200 North State Street, Inpatient Tower (C), Room C5L100 90033 Los Angeles California USA
- Corresponding author
- Inaba, Kenji
- Venue
- World Journal of Surgery
- Publisher
- Wiley
- Published
- 2010-12-16
- Volume
- 35
- Issue
- 3
- Pages
- 475-479
- ISSN
- 0364-2313 (ISSN-L)
- 1432-2323
- OpenAlex
- W2014957631
- Cited by
- 33
- References
- 17
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 4 · 2023: 3 · 2022: 2 · 2021: 5 · 2020: 2 · 2019: 2 · 2018: 1 · 2017: 1 · 2016: 2 · 2015: 1 · 2014: 1 · 2013: 3
- FWCI
- 11.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- World Journal of Surgery (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- citation accuracy, citation errors, journal impact factor, misquotation, journal quality, bibliographic errors
- MeSH terms
- Bibliographies as Topic, Humans, Periodicals as Topic, Quality Control, Sensitivity and Specificity, General Surgery, Bibliometrics, Journal Impact Factor
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.