CoChat
Sign Up
Cited by 4Wikis in Education and Collaborationscientometrics and bibliometrics researchWeb visibility and informetrics

The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index; 11 of the 30 sources its claims cite were checked for support.

Dataset of first appearances of the scholarly bibliographic references on Wikipedia articles

Open access
Kikkawa, Jiro, Takaku, Masao, Yoshikane, Fuyuki·2022·Scientific Data
DOI 10.1038/s41597-022-01190-zPMC PMC8921307
Open this paper in CoChat — free accountOpen PDF

Abstract

Referencing scholarly documents as information sources on Wikipedia is important because it supports or improves the quality of Wikipedia content. Several studies have been conducted regarding scholarly references on Wikipedia; however, little is known of the editors and their edits contributing to add the scholarly references on Wikipedia. In this study, we develop a methodology to detect the oldest scholarly reference added to Wikipedia articles by which a certain paper is uniquely identifiable as the “first appearance of the scholarly reference.” We identified the first appearances of 923,894 scholarly references (611,119 unique DOIs) in 180,795 unique pages on English Wikipedia as of March 1, 2017 and stored them in the dataset. Moreover, we assessed the precision of the dataset, which was highly precise regardless of the research field. Finally, we demonstrate the potential of our dataset. This dataset is unique and attracts those who are interested in how the scholarly references on Wikipedia grew and which editors added them.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This paper presents a dataset and methodology for identifying the first appearances of scholarly bibliographic references on English Wikipedia articles, defined as the oldest reference by which a paper is uniquely identifiable. It reports 923,894 first appearances (611,119 unique DOIs) across 180,795 pages as of March 1, 2017, with overall detection precision of 93.3%.

Key findings

  • The dataset contains 923,894 scholarly reference first appearances (611,119 unique DOIs) in 180,795 unique English Wikipedia pages as of March 1, 2017; these were added by 74,456 users, 63 bots, and 37,748 IP editors.
  • Manual validation of 1,100 sampled records found overall precision of 93.3% (1,026/1,100); precision was highest at 98.0% in clinical medicine, environment/ecology, and psychiatry/psychology, and lowest at 86.0% in chemistry and 84.0% in physics.
  • Compared with the mwcites dataset, common DOI links accounted for 77.84% of the mwcites dataset and 91.90% of this dataset; timestamps were identical in 51.0% of common pairs, older in this dataset in 49.0%, and newer in 0.01%, with an average lag of 723.2 days and median of 1.5 days.
  • ProteinBoxBot was the top overall editor, adding 99,150 references (10.7% of the total) and ranking first in 6 of 22 research fields; the largest month was 2007-12 with 64,833 references, of which ProteinBoxBot added 54,991 (84.8%).

Methods

  • The basic dataset was built from English Wikipedia dump files as of March 1, 2017: DOI links were extracted from main namespace pages, non-Crossref DOIs and non-individual content types were excluded, Crossref metadata were obtained, other identifiers (PMID, PMCID, Bibcode) were associated, and ESI research fields were assigned.
  • First appearances were identified by extracting all revision histories for the page IDs and detecting candidate revisions when one or more identifiers, the full paper title or first five title words, or a Levenshtein-distance similarity threshold matched; the oldest matching revision was selected, and editors were classified as User, Bot, or IP.
  • Precision was assessed by manual diff checking of random samples of 50 records per research field (1,100 total) by the first author; an experiment on title-word length found the best precision (84.6%) when the first five words were used.

Limitations

  • The paper states that precision was relatively low in chemistry (86.0%) and physics (84.0%) because citation conventions in these fields often use information other than paper title and identifiers, such as author, journal, volume, issue, or pages; these errors are described as unavoidable for the methodology, and future work could add factors such as journal names and published years.
  • The paper states that the dataset covers English Wikipedia only and specific snapshots (March 1, 2017, and October 1, 2021), excludes non-Crossref DOIs (0.3% of unique DOI names), and does not distinguish reference roles such as evidence, mention, or further reading.
  • The text makes evident that bot identification is heuristic and complex, relying on user groups, categories, name strings, edit counts, and manual checks of user pages; precision validation was also manual and based on samples of 50 records per field.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41597-022-01190-z

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

11 of 30 cited sources checked · 4 supported · 1 flagged · accuracy not yet measured

  • Premisereasoning

    Little is known of the editors and their contributions to the addition of scholarly references to Wikipedia.

    “Hence, little is known of the editors and their contributions to the addition of scholarly references to Wikipedia.”
    Quote in the paperSubstantial claim3 of 5 sources checked
    • cites [2] Nielsen, F. Å. Scientific citations in Wikipedia. First Monday 12, https://doi.org/10.5210/fm.v12i8.1997 (2007). source

      The paper’s Abstract (page unavailable) examines the number and distribution of scientific citations in Wikipedia and notes that dedicated enthusiasts can influence citation statistics. It does not say how much is known about the editors who add scholarly references or their individual contributions; searches for those topics found no such passage. (LLM, conf=0.74)

      Experimental — accuracy not yet measured
    • cites [3] Halfaker, A. & Taraborelli, D. Research:Scholarly article citations in Wikipedia - Meta. https://meta.wikimedia.org/wiki/Research:Scholarly_article_citations_in_Wikipedia (2019).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [4] Kikkawa, J., Takaku, M. & Yoshikane, F. DOI Links on Wikipedia: Analyses of English, Japanese, and Chinese Wikipedias. In Proceedings of the 18th International Conference on Asia-Pacific Digital Libraries (ICADL 2016), 369–380, https://doi.org/10.1007/978-3-319-49304-6_40 (2016). source
    • cites [5] Kousha, K. & Thelwall, M. Are wikipedia citations important evidence of the impact of scholarly articles and books? Journal of the Association for Information Science and Technology 68, 762–779, https://doi.org/10.1002/asi.23694 (2017). source

      The Discussion (page unavailable) speculates that one book was added to many pages by a single editor and that students or instructors may add other citations, indicating uncertainty about who made those additions. It does not establish the broader claim that little is known about Wikipedia editors or their contributions to scholarly references generally. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [6] Lin, J. & Fenner, M. An analysis of Wikipedia references across PLOS publications. figshare https://doi.org/10.6084/m9.figshare.1048991.v3 (2014). source

      The Introduction (page unavailable) says there is insufficient data on how scholarly content is referenced in Wikipedia. The Conclusion and Discussions (page unavailable) calls for more research into editing by different users, but does not specifically say that little is known about which editors add scholarly references or their individual contributions. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
    • Rests on an assumption: Scholarly references on Wikipedia support or improve the quality of Wikipedia content.
  • Findingown data

    49,235 Crossref DOIs fulfilling the conditions of both individual scholarly articles and identifiable research fields are included in our dataset only.

    “On the other hand, 49,235 Crossref DOIs fulfilling these conditions are included in our dataset only.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The highest precision was 98.0% in clinical medicine, environment/ecology, and psychiatry/psychology.

    “The highest precision was 98.0% (in clinical medicine, environment/ecology, and psychiatry/psychology).”
    Quote in the paperNot a substantial claim
  • Findingown data

    The first appearances of 923,894 scholarly references (611,119 unique DOIs) in 180,795 unique pages on English Wikipedia as of March 1, 2017 were identified and stored in the dataset.

    “We identified the first appearances of 923,894 scholarly references (611,119 unique DOIs) in 180,795 unique pages on English Wikipedia as of March 1, 2017 and stored them in the dataset.”
    • Also stated in: § Basic statistics · plain
      “On the whole, we identified the first appearances of 923,894 scholarly references (611,119 unique DOIs) in 180,795 unique pages.”
    Quote in the paperSubstantial claim
  • Findingown data

    The spikes seen at 1, 2, and 3 in Fig. 5 are caused by the activities of a certain editor, as shown in Table 8.

    “The spikes seen at ①, ②, and ③ in Fig. 5 are caused by the activities of a certain editor, as shown in Table 8.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one. 30
  • Probably not a substantial claim. 11
  • The quote only partly states this claim. 1

  • It may state two separate claims as one.

    Premisecitation

    The methodologies in previous studies used to identify the scholarly references on Wikipedia can be classified as: (1) extracting them from either page texts (Wikitexts) or Wikipedia external links; (2) detecting the pages contain them by using web search engines; and (3) analyzing usage log data.

    “The methodologies in previous studies used to identify the scholarly references on Wikipedia can be classified as follows: (1) extracting them from either page texts (Wikitexts) or Wikipedia external links2–4, (2) detecting the pages contain them by using web search engines5, and (3) analyzing usage log data6.”
    Quote in the paperSubstantial claim3 of 5 sources checked
    • cites [2] Nielsen, F. Å. Scientific citations in Wikipedia. First Monday 12, https://doi.org/10.5210/fm.v12i8.1997 (2007). source
    • cites [3] Halfaker, A. & Taraborelli, D. Research:Scholarly article citations in Wikipedia - Meta. https://meta.wikimedia.org/wiki/Research:Scholarly_article_citations_in_Wikipedia (2019).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [4] Kikkawa, J., Takaku, M. & Yoshikane, F. DOI Links on Wikipedia: Analyses of English, Japanese, and Chinese Wikipedias. In Proceedings of the 18th International Conference on Asia-Pacific Digital Libraries (ICADL 2016), 369–380, https://doi.org/10.1007/978-3-319-49304-6_40 (2016). source
    • cites [5] Kousha, K. & Thelwall, M. Are wikipedia citations important evidence of the impact of scholarly articles and books? Journal of the Association for Information Science and Technology 68, 762–779, https://doi.org/10.1002/asi.23694 (2017). source
    • cites [6] Lin, J. & Fenner, M. An analysis of Wikipedia references across PLOS publications. figshare https://doi.org/10.6084/m9.figshare.1048991.v3 (2014). source

      Split into 3 atoms — 1 confirmed, 1 partially supported, 1 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured

  • It may state two separate claims as one.

    Findingcitation

    The authors have proposed methods to identify the first appearances of the scholarly references on Wikipedia by using the paper titles and their identifiers and built a dataset of first appearances of the scholarly bibliographic references on English Wikipedia articles.

    “Considering this background, the authors have proposed methods to identify the first appearances of the scholarly references on Wikipedia by using the paper titles and their identifiers and built a dataset of first appearances of the scholarly bibliographic references on English Wikipedia articles8.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [8] Kikkawa, J., Takaku, M. & Yoshikane, F. A Method to Identify the Edits Adding Bibliographic References to Wikipedia. Journal of Japan Society of Information and Knowledge 30, 370–389, https://doi.org/10.2964/jsik_2020_033 (2020). Text in Japanese. source
    • Rests on an assumption: Scholarly references on Wikipedia support or improve the quality of Wikipedia content.

  • It may state two separate claims as one.

    Premisecitation

    Several studies have been conducted regarding scholarly bibliographic references on Wikipedia; however, most of them have focused on the scholarly document itself.

    “Several studies have been conducted regarding scholarly bibliographic references on Wikipedia; however, most of them have focused on the scholarly document itself2–6.”
    Quote in the paperSubstantial claim3 of 5 sources checked
    • cites [2] Nielsen, F. Å. Scientific citations in Wikipedia. First Monday 12, https://doi.org/10.5210/fm.v12i8.1997 (2007). source

      The paper describes its own study of scholarly references in Wikipedia, so it supports the claim that such research has been conducted. In its methods passage (section labeled Abstract; page unavailable), it says the analysis extracted journal titles and counted citations by journal, rather than focusing on individual scholarly documents. It does not establish that most studies focused on the scholarly document itself. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [3] Halfaker, A. & Taraborelli, D. Research:Scholarly article citations in Wikipedia - Meta. https://meta.wikimedia.org/wiki/Research:Scholarly_article_citations_in_Wikipedia (2019).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [4] Kikkawa, J., Takaku, M. & Yoshikane, F. DOI Links on Wikipedia: Analyses of English, Japanese, and Chinese Wikipedias. In Proceedings of the 18th International Conference on Asia-Pacific Digital Libraries (ICADL 2016), 369–380, https://doi.org/10.1007/978-3-319-49304-6_40 (2016). source
    • cites [5] Kousha, K. & Thelwall, M. Are wikipedia citations important evidence of the impact of scholarly articles and books? Journal of the Association for Information Science and Technology 68, 762–779, https://doi.org/10.1002/asi.23694 (2017). source

      The “Wikipedia-Cited Scientific Publications” section describes several investigations of references in Wikipedia articles, including studies of the types of documents cited. It does not establish that most studies of scholarly bibliographic references focused on the scholarly document itself. (LLM, conf=0.79)

      Experimental — accuracy not yet measured
    • cites [6] Lin, J. & Fenner, M. An analysis of Wikipedia references across PLOS publications. figshare https://doi.org/10.6084/m9.figshare.1048991.v3 (2014). source

  • It may state two separate claims as one.

    Findingown data

    Overall precision was 93.3% and exceeded 90% in 20 out of 22 research fields.

    “We then evaluated the precision for detecting the first appearance, which was 93.3% as a whole and exceeded 90% in 20 out of 22 research fields8.”
    • Also stated in: § Assessment of the precision · plain
      “The precision for overall is 93.3% (=1,026/1,100∗100)”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [8] Kikkawa, J., Takaku, M. & Yoshikane, F. A Method to Identify the Edits Adding Bibliographic References to Wikipedia. Journal of Japan Society of Information and Knowledge 30, 370–389, https://doi.org/10.2964/jsik_2020_033 (2020). Text in Japanese. source

      Table 7 (Section 5.1) reports 93.3% overall precision and precision of at least 90% in 20 of 22 fields. But four of those fields are exactly 90%, so only 16 fields exceeded 90%. (LLM, conf=0.94)

      Experimental — accuracy not yet measured

  • It may state two separate claims as one.

    Findingown data

    In 2008-04, 13,224 references were added, of which ProteinBoxBot added 7,856 (59.4%).

    “2008-04 | 13,224 | ProteinBoxBot added 7,856 (59.4%) references (② in Fig. 5).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    The references in the dataset are added by 74,456 users, 63 bots, and 37,748 IP editors.

    “These references are added by 74,456 users, 63 bots, and 37,748 IP editors.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    With regard to research fields, clinical medicine, molecular biology & genetics, and multidisciplinary are top 3 for the number of total DOIs and exceed 100,000.

    “With regard to research fields, “clinical medicine,” “molecular biology & genetics,” and “multidisciplinary” are top 3 for the number of total DOIs and exceed 100,000.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Findingown data

    The dataset includes not only the dataset of English Wikipedia as of March 1, 2017, but also English Wikipedia as of October 1, 2021, where the same methodology was applied.

    “It includes not only the dataset of English Wikipedia as of March 1, 2017, but also English Wikipedia as of October 1, 2021, where we applied the same methodology.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    The two datasets contain the common DOI links at high rates, 77.84% of the mwcites dataset and 91.90% of our dataset.

    “There are some differences in the setting of the scope of the target, however these two datasets contain the common DOI links at high rates, 77.84% of the mwcites dataset and 91.90% of our dataset.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    In 2008-01, 28,837 references were added, of which ProteinBoxBot added 23,763 (82.4%).

    “2008-01 | 28,837 | ProteinBoxBot added 23,763 (82.4%) references (① in Fig. 5).”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    For the 399,054 cases where timestamps differed, the time lag in days had an average of 723.2, median of 1.5, and standard deviation of 811.0.

    “As for the 399,054 cases that the timestamps between the two datasets were not equal, we calculated the time lag for them in days. The average was 723.2, the median was 1.5, and the standard deviation was 811.0.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    There are 13 distinct editors in 22 research fields; ProteinBoxBot, David Eppstein, Materialscientist, and The Vintage Feminist are the top editors in multiple fields.

    “There are 13 distinct editors (ProteinBoxBot, David Eppstein, Materialscientist, The Vintage Feminist, Daniel-Brown, Meodipt, NotWith, RJHall, Rjwilmsi, Sasata, Smith609, V8rik, and Wilhelmina Will) in 22 research fields. Of these editors, ProteinBoxBot, David Eppstein, Materialscientist, and The Vintage Feminist are the top editors in multiple fields.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    In 2007-11, 21,447 references were added, of which ProteinBoxBot added 10,565 (49.4%).

    “2007-11 | 21,447 | ProteinBoxBot added 10,565 (49.4%) references (① in Fig. 5).”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    In the overlapping analysis by DOI names, 159,952 DOI names are included only in the mwcites dataset; of them, 137,375 were Crossref DOIs and 20,767 were invalid DOI names.

    “Based on the difference set, 159,952 DOI names are included only in the mwcites dataset. Out of them, 137,375 were Crossref DOIs and 20,767 were invalid DOI names.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    ProteinBoxBot added 99,150 scholarly references, which accounted for 10.7% of the total.

    “Furthermore, ProteinBoxBot added 99,150 scholarly references, which accounted for 10.7% of the total.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingnone given

    Our dataset contains the first appearances of scholarly references on English Wikipedia as of March 1, 2017, and DOI names corresponding to them.

    “Our dataset contains the first appearances of scholarly references on English Wikipedia as of March 1, 2017, and DOI names corresponding to them.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one. The quote only partly states this claim.

    Findingreasoning

    10,458 Crossref DOIs fulfill the conditions of both individual scholarly articles and identifiable research fields and are included only in the mwcites dataset.

    “Then, 10,458 Crossref DOIs fulfill the conditions of both individual scholarly articles and identifiable research fields by Step 1-2.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    The timestamps in both datasets were the same in 415,272 (51.0%) cases; our dataset timestamps were older in 399,008 (49.0%) cases; reverse cases were 46 (0.01%).

    “The timestamps in both datasets were the same in 415,272 (51.0%) cases of all. For others, the timestamps in our dataset were older than those in the mwcites dataset in 399,008 (49.0%) cases, and the reverse cases were 46 (0.01%).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingcitation

    The dataset presented in this paper is available at Zenodo.

    “The dataset presented in this paper is available at Zenodo22.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] Kikkawa, J., Takaku, M. & Yoshikane, F. Dataset of first appearances of the scholarly bibliographic references on English Wikipedia articles as of 1 March 2017 and as of 1 October 2021. Zenodo https://doi.org/10.5281/zenodo.5595573 (2021). source

  • It may state two separate claims as one.

    Findingown data

    From the mwcites dataset, using DOI as the type of identifier and timestamps by March 1, 2017, 1,020,508 DOI names in total and 721,836 unique were extracted, referenced on 229,090 pages.

    “DOI names of 1,020,508 in total and 721,836 in unique referenced on 229,090 pages were extracted.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    Our dataset covers bibliographic metadata, research fields along with page ids and page titles of Wikipedia articles, and revision ids, editors' information, and timestamps of each edit.

    “As shown in Table 1, our dataset covers bibliographic metadata, research fields along with page ids and page titles of Wikipedia articles, and revision ids, editors’ information, and timestamps of each edit.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    In 2007-12, 64,833 references were added, of which ProteinBoxBot added 54,991 (84.8%).

    “2007-12 | 64,833 | ProteinBoxBot added 54,991 (84.8%) references (① in Fig. 5).”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    ProteinBoxBot is a bot that adds scholarly references related to molecular and cellular biology automatically at a large scale, and is the 1st editor in 6 out of 22 fields related to these fields.

    “ProteinBoxBot25,26 is a bot that adds scholarly references related to molecular and cellular biology automatically at a large scale, and is the 1st editor in 6 out of 22 fields related to these fields.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [25] User:ProteinBoxBot - Wikipedia. https://en.wikipedia.org/wiki/User:ProteinBoxBot (2021).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [26] WikiProject Report: Molecular and Cellular Biology. Wikipedia Signpost. https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2008-01-28/WikiProject_report (2015).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Findingown data

    In 2009-07, 13,351 references were added, of which Yeast2Hybrid added 8,201 (61.4%).

    “2009-07 | 13,351 | Yeast2Hybrid added 8,201 (61.4%) references (③ in Fig. 5).”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Findingown data

    In the overlapping analysis by pairs of DOI names and page ids, 814,326 pairs are common, accounting for 79.90% and 88.29% of the mwcites dataset and our dataset, respectively.

    “Based on the product set, 814,326 pairs are common, accounting for 79.90% and 88.29% of the mwcites dataset and our dataset, respectively.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    After removing errors, unique DOI names were 675,798 and 99.7% of them were Crossref DOI.

    “After removing errors, unique DOI names were 675,798 and 99.7% of them were Crossref DOI.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Premisenone given

    The mwcites dataset covers a lot of language versions of Wikipedia and multiple identifiers other than DOI names.

    “The mwcites dataset covers a lot of language versions of Wikipedia and multiple identifiers other than DOI names.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Halfaker, A., Mansurov, B., Redi, M. & Taraborelli, D. Citations with identifiers in Wikipedia. figshare https://doi.org/10.6084/m9.figshare.1299540 (2018). source

  • Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

    Premisenone given

    The most difficult part is that the scholarly reference at the time of its first appearance is composed of insufficient or incomplete information, and more detailed information is added in later revisions.

    “The most difficult part is that the scholarly reference at the time of its first appearance is composed of insufficient or incomplete information, and more detailed information is added in later revisions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Wikipedia is one of the largest referrers of Crossref DOIs as of 2015.

    “In particular, according to Crossref, which assigns Digital Object Identifiers (DOIs) to scholarly documents massively, Wikipedia is one of the largest referrers of Crossref DOIs as of 20151.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] Bilder, G. Geoffrey Bilder: Strategic Initiatives Update #crossref15. https://www.slideshare.net/CrossRef/geoffrey-bilder-crossref15 (2015).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    The mwcite dataset extracted the first appearances of identifiers such as DOI, arXiv, PubMed (PMID & PMCID), and ISBN on 298 language versions of Wikipedia as of March 1, 2018.

    “The mwcite dataset extracted the first appearances of identifiers such as DOI, arXiv, PubMed (PMID & PMCID), and ISBN on 298 language versions of Wikipedia as of March 1, 2018.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Halfaker, A., Mansurov, B., Redi, M. & Taraborelli, D. Citations with identifiers in Wikipedia. figshare https://doi.org/10.6084/m9.figshare.1299540 (2018). source

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    The mwcite dataset contains page ids and page titles of Wikipedia articles, revision ids and timestamps of each edit, and types and values of identifiers.

    “This dataset contains page ids and page titles of Wikipedia articles, revision ids and timestamps of each edit, and types and values of identifiers.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Halfaker, A., Mansurov, B., Redi, M. & Taraborelli, D. Citations with identifiers in Wikipedia. figshare https://doi.org/10.6084/m9.figshare.1299540 (2018). source

  • Probably not a substantial claim.

    Study design and conductown data

    In the judgements, changes between two revisions were confirmed based on bibliographic information including author names, journal names, published years, volume and issue numbers, pages, and URI of individual scholarly references retrieved from Crossref metadata.

    “In the judgements, we confirmed changes between two revisions based on bibliographic information including author names, journal names, published years, volume and issue numbers, pages, and URI of individual scholarly references retrieved from Crossref metadata.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. It may state two separate claims as one.

    Premisecitation

    David Eppstein is a computer scientist, Materialscientist has received a Ph.D. in Physics, and The Vintage Feminist has a social science with politics degree.

    “David Eppstein is a computer scientist27, Materialscientist has received a Ph.D. in Physics28, and The Vintage Feminist has a social science with politics degree29.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [27] User:David Eppstein - Wikipedia. https://en.wikipedia.org/wiki/User:David_Eppstein (2021).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [28] User:Materialscientist - Wikipedia. https://en.wikipedia.org/wiki/User:Materialscientist (2021).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [29] User:The Vintage Feminist - Wikipedia. https://en.wikipedia.org/wiki/User:The_Vintage_Feminist (2018).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductown data

    The first appearance dataset was built using the methods described above, and the precision of the proposed methods was evaluated by checking each diff between the candidate revision of the first appearance and the previous revision manually by the first author.

    “The first appearance dataset in this study was built using the methods described above, and we evaluated the precision of the proposed methods by checking each diff24 between the candidate revision of the first appearance and the previous revision manually by the first author.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] Help:Diff - Wikipedia. https://en.wikipedia.org/wiki/Help:Diff (2021).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductown data

    Random samples of 50 records for each research field, 1,100 records in total, were taken from the dataset, and each was judged whether it is the first appearance.

    “In particular, we took random samples of 50 records for each research field, that is, 1,100 records in total from the dataset, and judged whether each of them is the first appearance.”
    Quote in the paperSubstantial claim
35 of 35

  • The paper relies on this without support.

    Scholarly references on Wikipedia support or improve the quality of Wikipedia content.

    “Referencing scholarly documents as information sources on Wikipedia complements these policies, as these cited sources support or improve the quality of Wikipedia content.”
    “Referencing scholarly documents as information sources on Wikipedia is important because it supports or improves the quality of Wikipedia content.”
1 of 1

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. The International DOI Foundation. Factsheet DOI Resolution Documentation - 4. Which RA? https://www.doi.org/factsheets/DOIProxy.html#whichra (2020).

Cited by

Sorted by citations

Related

Metadata

Authors
Kikkawa, J., Takaku, M., & Yoshikane, F.
Affiliations
University of Tsukuba (JP)
Corresponding author
Kikkawa, Jiro
ORCID
Venue
Scientific Data
Publisher
Springer Science and Business Media LLC
Published
2022-03-14
Volume
9
Issue
1
Pages
85-85
ISSN
2052-4463 (ISSN-L)
DOI
10.1038/s41597-022-01190-z
PMC
PMC8921307
OpenAlex
W4220839478
Cited by
4
References
12
Citations by year
2026: 1 · 2025: 1 · 2024: 1 · 2023: 1
Type
data paper
Language
en
Open access
Licence
Locations
  • Scientific Data (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
MEXT | Japan Society for the Promotion of Science (JSPS KAKENHI Grant Number JP21K21303, JSPS KAKENHI Grant Number JP20K12543)
Topics
  • Social Sciences › Communication › Wikis in Education and Collaboration
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
Keywords
Wikipedia, digital object identifiers, Wikipedia editors, citation history, English Wikipedia

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review