CoChat

NLP-driven citation analysis for scientometrics

JHA, RAHUL, JBARA, AMJAD-ABU, QAZVINIAN, VAHED, RADEV, DRAGOMIR R. · 2017 · Natural Language Engineering
Cited by 105

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Abstract

This paper summarizes ongoing research in Natural-Language-Processing-driven citation analysis and describes experiments and motivating examples of how this work can be used to enhance traditional scientometrics analysis that is based on simply treating citations as a ‘vote’ from the citing paper to cited paper. In particular, we describe our dataset for citation polarity and citation purpose, present experimental results on the automatic detection of these indicators, and demonstrate the use of such annotations for studying research dynamics and scientific summarization. We also look at two complementary problems that show up in Natural-Language-Processing-driven citation analysis for a specific target paper. The first problem is extracting citation context, the implicit citation sentences that do not contain explicit anchors to the target paper. The second problem is extracting reference scope, the target relevant segment of a complicated citing sentence that cites multiple papers. We show how these tasks can be helpful in improving sentiment analysis and citation-based summarization.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1017/s1351324915000443

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 12 references

Research works printed but not linked: 85

  1. Swales, 1990, Genre Analysis: English in Academic and Research Settings
  2. Kaplan, 2009, Proceedings of the 2009 Workshop on Text and Citation Analysis for Scholarly Digital Libraries, 88
  3. 10.1007/s11192-013-1027-3 · DOI
  4. 10.1002/asi.23168 · doi:10.1002/asi.23168
  5. 10.1002/asi.23101 · DOI · Paper page
  6. 10.3115/1220575.1220666 · DOI · Paper page
  7. 10.3115/974235.974260 · DOI · Paper page
  8. 10.1002/asi.23149 · DOI
  9. Nanba, 2004, Proceedings of the 11th SIG Classification Research Workshop, 117
  10. 10.1002/asi.22929 · DOI
  11. 10.1002/asi.1181 · DOI · Paper page
  12. 10.1002/asi.23083 · DOI · Paper page
  13. 10.1002/asi.23133 · DOI
  14. Garfield E. 1964. Can citation indexing be automated? Statistical Assoc. Methods for Mechanized Documentation, Symposium Proceedings. Washington, US.
  15. 10.3115/1599081.1599168 · DOI · Paper page
  16. 10.1177/030631277500500106 · DOI · Paper page
  17. 10.1371/journal.pone.0113184 · DOI
  18. 10.1002/asi.20373 · DOI · Paper page
  19. 10.1007/s11192-014-1271-1 · DOI
  20. Zitt M. , and Cointet J.-P. 2013. Citation impacts revisited: how novel impact measures reflect interdisciplinarity and structural change at the local and global level. ArXiv e-prints.
  21. 10.1016/j.ipm.2010.03.010 · doi:10.1016/j.ipm.2010.03.010
  22. 10.1177/030631277700700111 · DOI · Paper page
  23. 10.1007/s10579-012-9211-2 · DOI · Paper page
  24. Qazvinian, 2013, Generating extractive summaries of scientific paradigms, J. Artif. Int. Res., 46, 165
  25. Papineni, 2002, Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, 311
  26. 10.1002/asi.23054 · doi:10.1002/asi.23054
  27. 10.1177/030631284014001006 · DOI · Paper page
  28. Kim, 2009, Proceedings of the 18th ACM Conference on Information and Knowledge Management, 385, 10.1145/1645953.1646004 · DOI
  29. 10.1002/asi.22753 · DOI
  30. 10.2196/jmir.2012 · DOI · Paper page
  31. Qazvinian, 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, 555
  32. 10.1093/ije/dyl189 · DOI · Paper page
  33. 10.1002/asi.23135 · DOI
  34. Lafferty, 2001, Proceedings of the 18th International Conference on Machine Learning, 282
  35. 10.1371/journal.pone.0005429 · DOI · Paper page
  36. 10.3115/1225733.1225751 · DOI · Paper page
  37. 10.1002/asi.22976 · DOI
  38. Nakov P. I. , Schwartz A. S. , and Hearst M. A. 2004. Citances: citation sentences for semantic analysis of bioscience text. In Proceedings of the SIGIR’04 workshop on Search and Discovery in Bioinformatics, Sheffield, UK.
  39. 10.1016/j.joi.2013.04.001 · DOI
  40. 10.1007/s11192-014-1233-7 · DOI
  41. Small, 1982, Citation context analysis, Progress in Communication Sciences, 3, 287
  42. 10.1523/JNEUROSCI.0003-08.2008 · DOI · Paper page
  43. 10.1002/asi.23151 · DOI
  44. Braun, 1987, Literature of Analytical Chemistry: A Scientometric Evaluation
  45. 10.3115/981658.981684 · DOI · Paper page
  46. 10.1007/1-4020-4102-0_13 · DOI
  47. 10.1002/asi.20707 · DOI · Paper page
  48. Athar, 2012, Proceedings of the Workshop on Detecting Structure in Scholarly Discourse, 18
  49. Quirk, 1985, A Comprehensive Grammar of the English Language
  50. 10.1007/s11192-014-1411-7 · DOI
  51. 10.1162/089120103321337421 · DOI · Paper page
  52. Mohammad, 2009, Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, 584
  53. Morante, 2012, Proceedings of the 1st Joint Conference on Lexical and Computational Semantics, 265
  54. Angrosh, 2013, Proceedings of the First Australasian Web Conference - Volume 144, 59
  55. 10.1002/asi.22890 · DOI
  56. Ziman, 1968, Public Knowledge: An Essay Concerning the Social Dimension of Science
  57. 10.1016/j.joi.2013.05.001 · DOI
  58. 10.1002/asi.23120 · doi:10.1002/asi.23120
  59. 10.1002/asi.22785 · DOI
  60. 10.1007/s11192-009-0041-y · DOI · Paper page
  61. 10.1007/s11192-006-0147-4 · DOI · Paper page
  62. 10.5860/crln.68.5.7804 · DOI · Paper page
  63. 10.2307/2529310 · DOI · Paper page
  64. Hodges T. L. 1972. Citation Indexing-its Theory and Application in Science, Technology, and Humanities. Ph.D. Thesis, University of California at Berkeley.
  65. Bornmann L. , and Marx W. 2013. Standards for the application of bibliometrics in the evaluation of individual researchers working in the natural sciences. ArXiv e-prints.
  66. 10.1093/applin/12.4.365 · DOI · Paper page
  67. Erkan, 2004, Lexrank: Graph-based centrality as salience in text summarization, Journal of Artificial Intelligence Research (JAIR), 22, 457, 10.1613/jair.1523 · DOI · Paper page
  68. 10.1002/asi.23100 · DOI · Paper page
  69. 10.1002/asi.22978 · DOI
  70. Weinstock, 1971, Citation Indexes
  71. 10.3115/981658.981695 · DOI · Paper page
  72. Athar, 2011, Proceedings of the ACL 2011 Student Session, 81
  73. Athar, 2012, Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 597
  74. Abu-Jbara, 2013, HLT-NAACL, 596
  75. Abu-Jbara, 2011, Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 500
  76. 10.1002/bies.201100067 · DOI · Paper page
  77. 10.1002/asi.21197 · doi:10.1002/asi.21197
  78. 10.1002/asi.22897 · DOI
  79. 10.1002/asi.23107 · DOI
  80. Qazvinian, 2010, Proceedings of the 23nd International Conference on Computational Linguistics (COLING-10), 895
  81. 10.1533/9781780630250 · DOI · Paper page
  82. Jbara, 2012, Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 80
  83. Nanba, 1999, IJCAI ’99: Proceedings of the 16th International Joint Conference on Artificial Intelligence, 926
  84. 10.1007/BF02017571 · DOI
  85. Bradshaw S. 2003. Reference directed indexing: redeeming relevance for subject search in citation indexes. In Proceedings of the 7th European Conference on Research and Advanced Technology for Digital Libraries, Trondheim, Norway. · DOI

Cited by

Metadata

Authors
JHA, R., JBARA, A.-A., QAZVINIAN, V., & RADEV, D. R.
Venue
Natural Language Engineering
Published
2016-01-25
Volume
23
Issue
1
Pages
93-130
ISSN
  • 1351-3249 (ISSN-L)
  • 1469-8110
OpenAlex
W2565026778
Cited by
105
References
12
FWCI
9.47
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: NLP-driven citation analysis for scientometrics. https://app.cochat.ai/papers/10.1017/s1351324915000443
The paper
JHA, R., JBARA, A.-A., QAZVINIAN, V., & RADEV, D. R. (2017). NLP-driven citation analysis for scientometrics. Natural Language Engineering. https://doi.org/10.1017/s1351324915000443

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.