CoChat
Sign Up
Cited by 6Academic Writing and Publishingscientometrics and bibliometrics researchPublishing and Scholarly Communication

Reference accuracy of primary studies published in peer-reviewed scholarly journals: A scoping review

Logan, Samuel W, Hussong-Christian, Uta, Case, Layne, Noregaard, Samantha·2024·Journal of Librarianship and Information Science
DOI 10.1177/09610006231177715
Open this paper in CoChat — free account

Abstract

Objective. The objective of this scoping review was to identify key characteristics of primary research studies in peer-reviewed journals about reference accuracy. Characteristics of studies include the publication year, field of study, journal name and publication year of verified references, number of references verified, reference elements examined, definition of major, intermediate, and minor errors, whether punctuation or typographical errors were counted, overall error rate, major, intermediate, and minor error rates, and most common error type. Introduction. Reference accuracy has been studied across several decades and remains a persistent issue in peer-reviewed publications. Reference accuracy is typically reported as an error rate that represents the proportion of references with at least one error. Inclusion criteria for this scoping review included primary studies published in peer-reviewed journals that verified references in articles. Methods. Library, Information Science and Technology Abstracts (LISTA), Medline via EBSCOhost, and Web of Science were searched with final searches conducted in all three databases on November 26, 2022. No grey literature sources were searched given the focus of the project on peer-reviewed scholarly journals. Results. A total of 105 studies were included. The overall error rate across all studies was 32.7%. The error rates within specific fields of study were relatively similar. There is a lack of clarity about whether studies counted punctuation or typographical errors. The most common error was related to author names. Conclusions. There is a lack of studies to understand the most effective means to increase reference accuracy that are feasible to implement on a wide scale. Future research is warranted to understand how best to improve reference accuracy across fields of study.

Integrity signals

Registry

Crossref · 10.1177/09610006231177715

Cites

117 of 117 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

25 of 114 cited sources checked · 16 supported · 2 flagged · accuracy not yet measured

  • Premisecitation

    Accurate references are critical for the correct indexing of articles, including author information, in electronic databases.

    “Accurate references are critical for the correct indexing of articles, including author information, in electronic databases (Gupta and Babel, 2018).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Gupta and Babel, 2018 source
  • Premisecitation

    There are only two published reviews on reference accuracy.

    “Despite the number of studies about reference accuracy, there are only two published reviews on the topic (Gupta and Babel, 2018;Wager and Middleton, 2008).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Gupta and Babel, 2018 source

      The Abstract identifies Gupta and Babel’s paper as a review of reference accuracy in medical-science literature, so it supports identifying this paper as one published review. Neither the Abstract nor the passages retrieved about other reference-accuracy work establish that only two reviews had been published. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites Wager and Middleton, 2008 source
  • Premisereasoning

    As a result of Gupta and Babel's review, it is unclear how many studies were included in the review.

    “As a result, it is unclear how many studies were included in the review.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Gupta and Babel, 2018 source

      The Background says the review groups studies into subcategories, and the subject-specific sections describe individual studies, but the passages found do not state how many studies Gupta and Babel included. Searches for an inclusion count and selection criteria found no passage that establishes whether the total is specified elsewhere in the review. (LLM, conf=0.67)

      Experimental — accuracy not yet measured
  • Findingcitation

    In Lawson and Fosker (1999), 36.0% of verified references in three psychiatric journals contained at least one error.

    “No 36.0%Major: 3% Minor: 33%”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Lawson and Fosker (1999) source

      The Findings and Table 1 report errors among 147 verified references from three psychiatric journals, and the Comment reports a 3% prevalence of major citation errors. The paper does not report that 36.0% of verified references had at least one error; its listed error counts do not establish that percentage of distinct references. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • Findingcitation

    In Khodabakhshi et al. (2021), 75.9% of verified references in 20 obstetrics and gynecology journals contained at least one error.

    “Unclear 75.9% Major: 92.3% Minor: 7.9%”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Khodabakhshi et al. (2021) source
  • The quote only partly states this claim. 12
  • Classified as probably not a substantial claim. 54
  • May combine two claims. 24

  • The quote only partly states this claim.

    Findingcitation

    In Genzinger and Wills (2017), 25.8% of verified references in three library science journals contained at least one error.

    “Unclear 25.8% N/A Article title”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Genzinger and Wills (2017) source

      The Citation Accuracy section reports 472 references, of which 366 (77.5%) were error-free; thus 106 (22.5%) contained at least one error. Its 25.8% figure is 122 total errors divided by 472 references, not the percentage of references with an error. The paper studies the three journals named in the claim, but the claimed percentage describes the wrong measure. (LLM, conf=0.90)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured

  • The quote only partly states this claim.

    Findingcitation

    In Montenegro et al. (2020), 15.5% of verified references in three spine journals contained at least one error.

    “Yes 15.5% Major: 4.5%.Minor: 11.0%”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Montenegro et al. (2020) source

      The Abstract (Methods) says Montenegro et al. examined four neurosurgery journals, not three spine journals. The Abstract (Results) reports 62.1% of articles with a minor citation error and 8.33% with a major citation error, rather than 15.5% of references with an error (4.5% major and 11.0% minor). (LLM, conf=0.88)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Major and minor errors are not always defined the same among studies, but the intention of the distinction between major and minor errors remains the same (whether an error makes finding an original source difficult).

    “Major and minor errors are not always defined the same among studies, but the intention of the distinction between major and minor errors remains the same (i.e.whether an error makes finding an original source difficult to find).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites de Lacey et al., 1985 source

      The Materials and methods passage defines major citation errors as those preventing immediate identification of the source; all remaining citation errors are minor. This supports the claimed purpose of the distinction, but the paper does not establish that studies define major and minor errors differently. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Reference accuracy has been studied across several decades and remains a persistent issue in peer-reviewed publications.

    “Reference accuracy has been studied across several decades and remains a persistent issue in peer-reviewed publications.”
    • Also stated in: § Introduction · plain
      “Reference accuracy has been a topic of debate through commentaries (Dobell, 1939;Garfield, 1990), letters to the editor (Crichton, 1989;Haworth, 1985), and descriptive and empirical studies for many, many decades (Gupta and Babel, 2018;Kronick, 1958).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Gupta and Babel, 2018 source
    • cites Kronick, 1958

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Reference accuracy is typically reported as an error rate that represents the proportion of references with at least one error.

    “Reference accuracy is typically reported as an error rate that represents the proportion of references with at least one error.”
    • Also stated in: § Introduction · plain
      “Reference accuracy is commonly reported as an error rate (Gupta and Babel, 2018;Key and Roland, 1977;Stull et al., 1991;White, 1987aWhite, , 1987b)).”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites Gupta and Babel, 2018 source
    • cites Key and Roland, 1977

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Stull et al., 1991 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Gupta and Babel (2018) reported that reference accuracy varies widely amongst journals and subfields within the medical literature.

    “Gupta and Babel (2018) reported that reference accuracy varies widely amongst journals and subfields within the medical literature.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Gupta and Babel, 2018 source

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    A reference list serves as a roadmap for how scholarship about a topic develops over time.

    “A reference list also serves as a roadmap for how scholarship about a topic develops over time (Hinchcliff et al., 1993;White, 1987a).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Hinchcliff et al., 1993 source
    • cites White, 1987a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisereasoning

    Gupta and Babel's review used a narrative style and did not use a systematic approach (e.g., PRISMA guidelines) to conduct a literature search or report findings.

    “It used a narrative style and did not use a systematic approach (e.g.PRISMA guidelines) to conduct a literature search or report findings.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Gupta and Babel, 2018 source

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    One of the most basic functions of a reference list is to give credit for the work of other authors.

    “One of the most basic functions of a reference list is to give credit for the work of other authors (George and Robbins, 1994;Hinchcliff et al., 1993;Oermann et al., 2001;Taylor, 1998).”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites George and Robbins, 1994 source
    • cites Hinchcliff et al., 1993 source
    • cites Oermann et al., 2001 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning

    In Gupta and Babel's review, results were often discussed separately for each study and were not placed within a broader summary or trend of findings, which limits the usefulness of the review.

    “Results were often discussed separately for each study and were not placed within a broader summary or trend of findings which limits the usefulness of the review.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Gupta and Babel, 2018 source

      Section 2 (Medical Science Literature; page unavailable) discusses studies individually, as the claim describes. But the Background says the review groups studies into subcategories, and the Abstract gives a broad finding that reference errors occur even in peer-reviewed, high-impact journals. The source does not state that its approach limits its usefulness. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    This scoping review followed the PRISMA Extension for Scoping Reviews.

    “This is a scoping review that followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Extension for Scoping Reviews (Tricco et al., 2018).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Tricco et al., 2018 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The search strategy was peer-reviewed by a librarian professor using the Peer Review of Electronic Search Strategies guidelines.

    “The search strategy was peer-reviewed by a librarian professor using the Peer Review of Electronic Search Strategies guidelines (McGowan et al., 2016).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites McGowan et al., 2016 source

  • Classified as probably not a substantial claim.

    Premisecitation

    An identified purpose of a scoping review includes to identify key characteristics or factors related to a concept.

    “An identified purpose of a scoping review includes “. ..to identify key characteristics or factors related to a concept” (Munn et al., 2018: 2).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Munn et al., 2018 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Findingcitation

    In Hinchcliff et al. (1993), 29.8% of verified references in six veterinary medicine journals contained at least one error.

    “Unclear 29.8% (error rate only reported for major errors)”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Hinchcliff et al. (1993) source

      The Summary (p. 1) reports that 88 of 295 located references (29.8%) contained major errors across six veterinary journals. It separately reports 68 minor errors, so 29.8% is the rate for references with at least one major error, not an established rate for references with any error. (LLM, conf=0.90)

      Experimental — accuracy not yet measured

  • The quote only partly states this claim.

    Findingcitation

    In Ghane (2016), 36.6% of verified references in 97 Iranian-accredited journals contained at least one error.

    “Unclear 36.6%N/A Author”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Ghane (2016) source

      The Results report a 36.6% mean percentage error rate across articles in a sample of 97 journals. Table 2 lists 74,577 checked references and 27,313 reference errors, but it does not establish that 36.6% of individual references each contained at least one error. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    In McLellan et al. (1992), 50.3% of verified references in four anesthesia journals contained at least one error.

    “Unclear 50.3%Not reported Article title”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites McLellan et al. (1992) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Genzinger and Wills (2017) reported an overall error rate of 25.8%.

    “Genzinger and Wills (2017) reported an overall error rate of 25.8% yet also report that all articles with incorrect citations were found through a Google search.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Genzinger and Wills (2017) source

  • Classified as probably not a substantial claim.

    Premisecitation

    Genzinger and Wills (2017) reported that all articles with incorrect citations were found through a Google search.

    “Genzinger and Wills (2017) reported an overall error rate of 25.8% yet also report that all articles with incorrect citations were found through a Google search.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Genzinger and Wills (2017) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Major, intermediate, and minor errors were reported as either an error rate or as a proportion of all errors.

    “Major, intermediate, and minor errors were reported as either an error rate (i.e. % of references with at least one type of error) or as a proportion of all errors (i.e.proportion of error type relative to total number of errors).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    About 6344 articles were initially identified, including 12 articles from a snowball search, and potentially met inclusion criteria.

    “About 6344 articles were initially identified (including 12 articles from a “snowball search”) and potentially met inclusion criteria.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    There is a lack of clarity about whether studies counted punctuation or typographical errors.

    “There is a lack of clarity about whether studies counted punctuation or typographical errors.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingcitation

    In Gehanno et al. (2005), 2.9% of verified references from 1998 in two occupational medicine journals contained at least one error.

    “No 1994: 3.6% 1998: 2.9% N/A Author”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gehanno et al. (2005)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Publication year of when references were verified was usually within 5 years of publication, although 12 studies examined reference accuracy trends across more than 5 years.

    “Publication year of when references were verified was usually within 5 years of publication, although 12 studies examined reference accuracy trends across more than 5 years.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    There were fewer studies in library and information science (n = 9; 36.4%) and other non-medical or health fields (n = 14; 29.6%).

    “There were fewer studies in library and information science (n = 9; 36.4%) and other non-medical or health fields including fashion, entomology, environmental science, social work, business, general science, counseling, psychology, physics, chemistry, and plant science (n = 14; 29.6%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Within the 82 medical and health-related studies, there was a focus on general medical sciences and medicine (n = 22; 29.3%), surgery (n = 14, 31.1%), and nursing (n = 9; 32.9%).

    “Within these 82 studies, there was a focus on general medical sciences and medicine (n = 22; 29.3%), surgery (n = 14, 31.1%), and nursing (n = 9; 32.9%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The field of library and information science has the highest error rate, but this should be interpreted with caution given the small number of studies.

    “Interestingly, the field of library and information science has the highest error rate but this should be interpreted with caution given the small number of studies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Descriptive statistics of the number of references verified across studies include Mean = 571 references; Standard Deviation = 833; and Range 47-74,577 references.

    “Descriptive statistics of the number of references verified across studies include Mean = 571 references; Standard Deviation = 833; and Range 47-74,577 references.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Journal names aligned with the field of study stated by authors.

    “Journal names aligned with the field of study stated by authors.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    The full text of 137 articles were reviewed further, and 105 articles met inclusion criteria and were included in the review.

    “The full text of 137 articles were reviewed further, and 105 articles met inclusion criteria and were included in the review.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Authors typically reported the overall error rate.

    “Authors typically reported the overall error rate.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The publication of reference accuracy studies was most common from 2000 to 2009 and appears to have increased across the decades with a recent decline.

    “The publication of reference accuracy studies was most common from 2000 to 2009 and appears to have increased across the decades with a recent decline.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    In Gupta et al. (2005), 39.2% of verified references in Indian Pediatrics contained at least one error.

    “Unclear 39.2% Major: 3.2% Minor: 37.5%”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gupta et al. (2005)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    In Kristof (1997), 30.1% of verified references in five entomology journals contained at least one error.

    “Yes 30.1% N/A Article title”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kristof (1997) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    In George and Robbins (1994), 41% of verified references in four dermatology journals contained at least one error.

    “No 41% Major: 6% Minor: 35% %s represent error rate Author”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites George and Robbins (1994) source

  • The quote only partly states this claim.

    Findingcitation

    In Gupta (2019), 33.0% of verified references in the Malaysian Journal of Library and Information Science contained at least one error.

    “Yes 33.0%Major: 49% Minor: 51%”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gupta (2019)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Findingcitation

    In Lopresti (2010), 24.4% of verified references in five environmental science journals contained at least one error.

    “No 24.4% N/A Author”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lopresti (2010) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Findingcitation

    In Goldberg et al. (1993), 27.5% of verified references in three emergency medicine journals contained at least one error.

    “Unclear 27.5% Author, year published”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Goldberg et al. (1993) source

  • The quote only partly states this claim.

    Findingcitation

    In Lee and Lee (1999), 28.5% of verified references in two dermatology journals contained at least one error.

    “No 28.5% Major: 1.5% Minor: 27%”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lee and Lee (1999) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In Nicoll et al. (2022), the overall error rate was not reported.

    “No Not reported N/A Author”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Nicoll et al. (2022)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • The quote only partly states this claim.

    Findingcitation

    In Mohammad and Laskin (2008), 21.0% of verified references in five oral and maxillofacial surgery journals contained at least one error.

    “Yes 21.0% (unclear how this was calculated)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mohammad and Laskin (2008) source

  • May combine two claims.

    Findingown data

    Author names was the most common error type (n = 57) followed by article title (n = 22).

    “Author names was the most common error type (n = 57) followed by article title (n = 22).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Findingcitation

    In Oermann et al. (2001), 41.6% of verified references in four pediatric nursing journals contained at least one error.

    “Unclear 41.6% Major: 28.9%.Minor: 13.7%”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Oermann et al. (2001) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    One study reported no difference between major and minor errors.

    “while one study reported no difference between major and minor errors.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    In O'Connor and Kristof (2001), 15.0% of verified references contained at least one error.

    “Yes 15.0%Not reported Article title”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites O'Connor and Kristof (2001) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    In Mitchell-Williams et al. (2017), 32.6% of verified references in five social work journals contained at least one error.

    “Yes 32.6%Not reported Author”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mitchell-Williams et al. (2017)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Empirical studies about reference accuracy have been conducted since the 1970s, primarily in health-related fields such as medical/medicine, surgery, nursing, and rehabilitation.

    “Empirical studies about reference accuracy have been conducted since the 1970s (Key and Roland, 1977;Poyer, 1979), and primarily in health-related fields such as medical/medicine (de Lacey et al., 1985;Ngan Kee et al., 1997a, 1997b), surgery (Awrey et al., 2011;Jackson et al., 2003), nursing (Oermann et al., 2002a(Oermann et al., , 2002b;;Taylor, 1998), and rehabilitation (Gosling et al., 2004;Unv”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Key and Roland, 1977

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Poyer, 1979

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    A reference list reflects upon the author's quality of work and scholarship.

    “A reference list, whether accurate or inaccurate, reflects upon the author's quality of work and scholarship (American Psychological Association, 2020; George and Robbins, 1994;Taylor, 1998;Wyles, 2004), credibility (Foreman and Kirchhoff, 1987;Karabulut, 2017;Lok et al., 2001), thoroughness (Stull et al., 1991), and quality of writing (Wilks et al., 2017).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites American Psychological Association, 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites George and Robbins, 1994 source
    • cites Taylor, 1998 source

  • May combine two claims.

    Premisecitation

    Correct indexing is necessary for documenting the frequency an article is cited, which contributes to a journal's impact factor.

    “Correct indexing is necessary for documenting the frequency an article is cited which contributes to a journal's impact factor (Davids et al., 2010;Fenton et al., 2000;Serenko et al., 2021).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Davids et al., 2010 source
    • cites Fenton et al., 2000 source
    • cites Serenko et al., 2021 source

  • May combine two claims.

    Premisecitation

    Incorrect references can cost librarians time and result in delayed document delivery, and increased costs on a library's budget may eventually be passed on to authors or result in reduced resources, services, and staff.

    “Incorrect references can cost librarians time and result in delayed document delivery, and the increased costs on a library's budget may eventually be passed on to authors or result in reduced resources, services, and staff (Oermann et al., 2001;Taylor, 1998).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Oermann et al., 2001 source
    • cites Taylor, 1998 source

  • May combine two claims.

    Premisecitation

    Wager and Middleton conducted a systematic review and reported a 30% error rate across more than 27,000 references in 66 reference accuracy studies in peer-reviewed biomedical journals, with a median error rate of 38% and a range of 4% to 67%.

    “Wager and Middleton conducted a systematic review and reported a 30% error rate across >27,000 references in 66 reference accuracy studies in peer-reviewed biomedical journals (Median error rate: 38%; Range: 4%-67%).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wager and Middleton, 2008 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Correct indexing of author information is important for the author's metrics of scholarship such as the h-index.

    “Lastly, correct indexing of author information is important for the author's metrics of scholarship such as the h-index (Montenegro et al., 2021;Serenko et al., 2021).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Montenegro et al., 2021 source
    • cites Serenko et al., 2021 source

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    An inaccurate reference list will frustrate the reader.

    “An inaccurate reference list will also frustrate the reader (Asano et al., 1995a;George and Robbins, 1994;Lok et al., 2001;Wyles, 2004).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Asano et al., 1995a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites George and Robbins, 1994 source
    • cites Lok et al., 2001 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A reference list reflects upon the author's credibility.

    “A reference list, whether accurate or inaccurate, reflects upon the author's quality of work and scholarship (American Psychological Association, 2020; George and Robbins, 1994;Taylor, 1998;Wyles, 2004), credibility (Foreman and Kirchhoff, 1987;Karabulut, 2017;Lok et al., 2001), thoroughness (Stull et al., 1991), and quality of writing (Wilks et al., 2017).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Foreman and Kirchhoff, 1987 source
    • cites Karabulut, 2017

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Lok et al., 2001 source

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Accurate references in research studies are essential for the transmission of scientific knowledge.

    “In the broadest sense, and simply put, “Accurate references in research studies are essential for the transmission of scientific knowledge. . .” (p. 497, Vargas-Origel et al., 2001).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Vargas-Origel et al., 2001

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    An accurate reference list is the best strategy to ensure readers can find and consult the original source to learn more about a topic and verify the authors' representation and interpretation of previous work.

    “An accurate reference list is the best strategy to ensure readers can find and consult the original source to learn more about a topic and verify the authors' representation and interpretation of previous work (American Psychological Association, 2020;Foreman and Kirchhoff, 1987;Key and Roland, 1977;Satyanarayana and Ratnakar, 1989;Stull et al., 1991;Sutherland et al., 2000).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites American Psychological Association, 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Foreman and Kirchhoff, 1987 source
    • cites Key and Roland, 1977

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    An inaccurate reference may be repeated throughout the literature, resulting in a lasting impact on the authors, journal, and field.

    “It is also possible that an inaccurate reference may be repeated throughout the literature (Biebuyek, 1992;Lichter, 1989), resulting in a lasting impact on the authors, journal, and field.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Biebuyek, 1992

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Lichter, 1989 source

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    A reference list reflects upon the author's quality of writing.

    “A reference list, whether accurate or inaccurate, reflects upon the author's quality of work and scholarship (American Psychological Association, 2020; George and Robbins, 1994;Taylor, 1998;Wyles, 2004), credibility (Foreman and Kirchhoff, 1987;Karabulut, 2017;Lok et al., 2001), thoroughness (Stull et al., 1991), and quality of writing (Wilks et al., 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wilks et al., 2017

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    A reference list reflects upon the author's thoroughness.

    “A reference list, whether accurate or inaccurate, reflects upon the author's quality of work and scholarship (American Psychological Association, 2020; George and Robbins, 1994;Taylor, 1998;Wyles, 2004), credibility (Foreman and Kirchhoff, 1987;Karabulut, 2017;Lok et al., 2001), thoroughness (Stull et al., 1991), and quality of writing (Wilks et al., 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Stull et al., 1991 source

  • May combine two claims.

    Premisecitation

    An inaccurate reference list may reflect negatively on the journal it is published in and on the broader field of study.

    “An inaccurate reference list may also reflect negatively on the journal it is published in (George and Robbins, 1994;Lok et al., 2001;Sutherland et al., 2000), as well as the broader field of study (Mitchell-Williams et al., 2017).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites George and Robbins, 1994 source
    • cites Lok et al., 2001 source
    • cites Sutherland et al., 2000 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Articles were either included or excluded following full text review in consensus between the first and third authors.

    “Articles were either included or excluded following full text review in consensus between the first and third authors.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Two authors reached 100% agreement for included studies.

    “Two authors reached 100% agreement for included studies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The research question, eligibility criteria, and methods for this scoping review were specified in advance and documented for internal use and are not formally published.

    “The research question, eligibility criteria, and methods for this scoping review were specified in advance and documented for internal use and is not formally published.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The librarian who peer-reviewed the search strategy has 17 years of experience in the field and extensive teaching experience on conducting literature reviews.

    “The librarian has 17 years of experience in the field and extensive teaching experience on conducting literature reviews.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The first and third authors resolved any discrepancies through discussion.

    “The first and third authors resolved any discrepancies through discussion.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The third author independently reviewed the extracted data.

    “The third author independently reviewed the extracted data.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    There is a lack of studies to understand the most effective means to increase reference accuracy that are feasible to implement on a wide scale.

    “There is a lack of studies to understand the most effective means to increase reference accuracy that are feasible to implement on a wide scale.”
    • Also stated in: § Conclusion · plain
      “There is a lack of studies to understand the most effective means to increase reference accuracy that are feasible to implement on a wide scale.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisecitation

    Reference accuracy is critical for timely access to knowledge about results of medical studies because human lives may depend on it and contributes to the development of evidence-based medicine.

    “Reference accuracy is critical for timely access to knowledge about results of medical studies because human lives may depend on it (Wyles, 2004) and contributes to the development of evidence-based medicine (Armstrong et al., 2018).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Wyles, 2004 source
    • cites Armstrong et al., 2018 source

  • Classified as probably not a substantial claim.

    Interpretationown data

    The lack of clarity about whether punctuation or typographical errors are counted makes it challenging to understand the impact of these types of errors on error rates.

    “The lack of clarity makes it challenging to understand the impact of these types of errors on error rates.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    One limitation of the current study is the institutional access to search tools used to find reference accuracy studies.

    “One limitation of the current study is the institutional access to search tools used to find reference accuracy studies.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. The quote only partly states this claim.

    Study design and conductreasoning

    By employing a snowball search strategy along with the database searches, the authors are confident that the included studies allowed for scoping the body of literature on the topic.

    “However, by employing a”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The distinction between major and minor errors in reference accuracy research may be outdated because it is often unsubstantiated and may no longer be an issue given current tools for searching the literature.

    “However, this distinction may be outdated because it is often unsubstantiated and may no longer be an issue given the current tools for searching the literature.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The database selection decision was partially driven by the institutional availability to the author conducting the searches.

    “The database selection decision was partially driven by the institutional availability to the author conducting the searches.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Another limitation is that the authors categorized major and minor errors the same way as previous research and did not examine how different errors were related to findability of the article.

    “Another limitation is that we categorized major and minor errors the same way as previous research and did not examine how different errors were related to findability of the article.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The problem of reference accuracy has been recognized for at least 100 years.

    “The problem of reference accuracy has been recognized for at least 100 years (Place, 1916).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Place, 1916

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

74 of 74

  • The paper relies on this without support.

    Overall error rates reported by the included studies are comparable enough to be pooled into a single overall error rate despite differences in definitions and methods.

    “Reference accuracy studies are descriptive and inhibit quantitative synthesis across studies, with the exception of sample size (i.e.number of references verified) and overall error rate.”
    “The overall error rate across all studies was 32.7%.”

  • The paper relies on this without support.

    The overall error rates reported by the included studies are comparable enough to be pooled into a single mean.

    “Descriptive statistics of overall error rates across studies include total error rates reported (n = 118), Mean = 32.7%,Standard Deviation = 15.5%, and Range = 2.9%-83.3%.”

  • The paper relies on this without support.

    Web of Science does not comprehensively index references across fields but instead focuses on high-impact journals in each field.

    “While Web of Science is a cross-disciplinary database, it does not comprehensively index references across fields but instead focuses on high-impact journals in each field.”

  • The paper relies on this without support.

    Two databases (Web of Science and LISTA) were not included in the previous two systematic reviews of reference accuracy.

    “Although our search strategy was limited to three databases, two (Web of Science and LISTA) were not included in the previous two systematic reviews of reference accuracy.”
4 of 4

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Logan's annual reference counts differ between the narrative and table

    Minor

    The narrative gives 396 references for 2009 and 506 for 2019, while Table 1 gives 395 and 507. Both versions total 1,341, but the annual counts disagree.

  • Two different decades are named as the publication peak

    Minor

    The results first identify 2000-2009 as the most common period for publication, then rank the 1990s ahead of the 2000s.

  • Figure 3's included-study count does not match its exclusions

    Minor

    The caption says 102 of 105 studies appear in the plot, but names four studies that were excluded. That leaves 101.

  • The stated near-universal element coverage conflicts with Poyer's entry

    Minor

    The results say 104 studies examined all six listed elements and identify Nicoll as the sole exception. Table 1 also says Poyer did not examine article titles.

4 of 4

Cites

Flagged first, then by citations

Research works printed but not linked: 25

  1. *Adhikari P, 2009, Nepal Medical College Journal: NMCJ, 11, 130
  2. *Adhikari P, 2011, Internet Journal of Medical Update, 6, 47
  3. *Bahar Z, 2012, HealthMED, 6, 2066
  4. *Benning SP, 1993, Bulletin of the Medical Library Association, 81, 56
  5. *Boyce BR, 1979, RQ, 18, 349
  6. DOI Handbook (1997) Available at: https://www.doi.org/the-identifier/resources/handbook/
  7. Garfield E, 1990, Current Comments, 41, 367
  8. *Gehanno JF, 2005, Journal of the Medical Library Association JMLA, 93, 118
  9. *Gupta P, 2005, Indian Pediatrics, 42, 140
  10. *Gupta VK, 2019, Journal of Indian Library Association, 55, 27
  11. Gupta VK, 2018, IRA-International Journal of Management & Social Sciences, 12, 62
  12. *Holt S, 2000, The New Zealand Medical Journal, 113, 416
  13. Kronick DA, 1958, Bulletin of the Medical Library Association, 46, 219
  14. *Nuckles DB, 1993, Operative Dentistry, 18, 28
  15. Place F, 1916, Atlanta Journal - Record of Medicine, 63, 278
  16. *Pope NN, 1992, RQ, 32, 240
  17. *Poyer RK, 1979, Bulletin of the Medical Library Association, 67, 396
  18. *Putterman C, 1991, Israel Journal of Medical Sciences, 27, 109
  19. Rudolph J, 1990, Journal of Forests, 88, 39
  20. *Satyanarayana K, 1989, European Science Editing, 36, 5
  21. *Siebers R, 1999, New Zealand Journal of Medical Laboratory Science, 53, 46
  22. *Ullah M, 2008, Pakistan Armed Forces Medical Journal, 58, 299
  23. *Yankauer A, 1990, CBE Views: Official Publication of the Council of Biology Editors, 13, 38
  24. *Yankauer A, 1991, CBE Views, 14, 23
  25. *Zasa M, 2015, Science & Sports, 30

Cited by

Sorted by citations

Related

Authors
Logan, S. W., Hussong-Christian, U., Case, L., & Noregaard, S.
Affiliations
  • Oregon State University (US)
  • California State University, Chico (US)
Corresponding author
Logan, Samuel W
ORCID
Venue
Journal of Librarianship and Information Science
Publisher
SAGE Publications
Published
2023-06-07
Volume
56
Issue
4
Pages
896-949
ISSN
  • 0022-2232 (ISSN-L)
  • 0961-0006
  • 1741-6477
DOI
10.1177/09610006231177715
OpenAlex
W4379647706
Semantic Scholar
afc344240aca87503ca0d98396adad4fb8f0f10f
Cited by
6
References
104
Citations by year
2026: 2 · 2025: 1 · 2024: 3
FWCI
5.41
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
Journal of Librarianship and Information Science (publishedVersion)
Topics
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Publishing and Scholarly Communication
Keywords
citation accuracy, citation errors, bibliographic references, error rate, author name errors, punctuation errors, typographical errors, reference verification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review